Mega Code Archive

The Dynamics of Agile Software Processes, Part I

Title: The Dynamics of Agile Software Processes, Part I Question: This article was published at http://bdn.borland.com.. however.. i think everybody should come along this .. so i post it here too... Enjoy... Answer: The Dynamics of Agile Software Processes, Part I: Characteristics - by Randy Miller Abstract:This article describes the characteristics of good agile software development process design. The economic boom of the mid- to late 1990s brought a wave of software development startup companies whose goal was to take advantage of e-commerce and other emerging technologies. Competitive pressures on these new companies forced them to examine and often adopt the radical approaches to software engineering that were necessary to rapidly deliver solutions to market. These companies could not afford to wait six to eight months to deploy applications. Instead, new releases had to be delivered in three to six week time periods, or what is now called internet time1. The need to deliver applications in internet time did not initially foster mature software development processes. Instead, most companies ran without processes because they could not afford the overhead instilled by the leading processes of the day. However, most realized that the right process enables developers to be more productive. This need to deliver applications more rapidly (and you can define rapidly based on your industry and type of application) caused a vacuum to be created in the area of software processes. How can a software development organization deliver to their demanding schedules under a software development process that effectively coordinates all of the different groups involved? Characteristics of Agile Processes The answer came from one of the more unlikely sources, a large company. Large companies had been oscillating between the larger processes necessary to satisfy quality initiatives such as the Capability Maturity Model, and smaller processes such as rapid application development for some time. As competitive pressures dictated the need for them to compete with the smaller companies, they began experimenting with lighter weight processes. Some of the more popular and better documented processes preferred the name "agile software processes". The name "agile software process", first originated in Japan [Aoyama 1998]. The Japanese faced similar competitive pressures, and many of their companies, like their American counterparts, promoted cycle-time reduction as the most important characteristic of software process improvement efforts. To reflect its importance, time would become one of the ten characteristics of an agile process [Aoyama 1998]. Characteristic 1: Modularity To understand the first characteristic of an agile software process, we must first define what is meant by process. Process is central whenever work is performed. A process is defined as: "a collection of activities that takes one or more inputs and creates an output that is of measurable value [Hammer 1993]." Modularity is a key element of any good process. Modularity allows a process to be broken into components called activities. A software development process prescribes a set of activities capable of transforming the vision of the software system into reality. An activity is a unit of work within a process that has a clear purpose. An activity usually has only one purpose, to produce some output of measurable value. An example of an activity is "Create project plan". The name of the activity clearly defines what is supposed to be done. It is also clear that there will be a project plan delivered as a result of completing this activity. A good process clearly defines its activities so that others can easily follow it. Modularity allows activities to be added to the process, if necessary. Modularity also allows activities to be removed from the process if they are found to be extraneous. Characteristic 2: Iterative Agile software processes do not attempt to deliver a big bang solution. Instead, they focus on short cycles. Within each cycle, a certain set of activities is completed. These cycles will be started and completed in a matter of weeks. However, a single cycle (called an iteration) will probably not be enough to complete the entire project. Therefore, the short cycle is repeated many times to refine the deliverables. Iterative development acknowledges that we probably will not build the system exactly as the user wants it initially. Therefore, we plan to rework a part of a system to remove mistakes or to make improvements based on user feedback. The philosophy of iterative development is that we tend to get things wrong before we get them right. Characteristic 3: Time-Bound Iterations become the perfect unit for planning the software development project. We can set time limits (normally between one and six weeks is normal) on each iteration and schedule them accordingly. Chances are, we will not (unless the process contains very few activities) schedule all of the activities of our process in a single iteration. Instead, we will only attempt those activities necessary to achieve the goals set out at the beginning of the iteration. Functionality may be reduced or activities may be rescheduled if they cannot be completed within the allotted time period. Time-boxing is a popular technique for bounding an iteration [Rising 2000]. A time-boxed iteration has a fixed time limit. This technique forces trade-offs to be made in the elements of the project. It also encourages optimization of the agile process. One thing that time boxes are not allowed to be used for is to pressure project members into making poor decisions, working overtime, and cutting corners on quality [Highsmith 2000]. Characteristic 4: Parsimony Agile processes are more than just a traditional software development process with some time constraints. Attempting to create impossible deadlines under a process not suited for rapid delivery puts the onus on the software developers. This leads to burn-out and poor quality. Instead, agile software processes focus on parsimony. That is, they require a minimal number of activities necessary to mitigate risks and achieve their goals. By minimizing the number of activities, they allow developers to deliver systems against an aggressive schedule, while maintaining some semblance of a normal life (sleeping at night ...). All software projects contain certain activities which must be completed to deliver the system. We must write the code to create the system. We must test until the system works as it is intended to. These activities are not negotiable. Until someone radically changes the way that we create software, there will be a certain set of mandatory activities necessary to deliver systems. Software projects also contain risks. And we all know the effect of ignoring risks [Gilb 1988]. The agile software process is not a foolhardy process. The goal of the agile software process is to successfully deliver a system. Therefore, your agile process must contain activities to attack the worthwhile risks that may strike your project. Parsimony dictates, however, that the set of activities chosen be the minimal set necessary to accomplish this goal. In short, each activity in the agile software process that is deployed must either be necessary to deliver the system or be designed (and traceable) to attack a risk found on your project. Characteristic 5: Adaptive During an iteration, new risks may be exposed which require some activities that were not planned. The agile process adapts the process to attack these new found risks. If the goal cannot be achieved using the activities planned during the iteration, new activities can be added to allow the goal to be reached. Similarly, activities may be discarded if the risks turn out to be ungrounded. Characteristic 6: Incremental An agile process does not try to build the entire system at once. Instead, it partitions the nontrivial system into increments which may be developed in parallel, at different times, and at different rates. We unit test each increment independently. When an increment is completed and tested, it is integrated into the system. Each increment may require several iterations to complete. The first iteration of an increment creates an initial cut at system functionality. Subsequent iterations will be dedicated to "finishing" the functionality, removing the rough edges. User feedback helps to determine what finishing touches should be placed on the functionality delivered during the increment. Increments allow us to develop our system in parallel instead of lock step. Developing increments in parallel allows us to complete the system faster. We utilize techniques such as scaffolding to "stub in" the incomplete areas of the system. The goal is to use smaller steps to get earlier customer feedback. We utilize subsequent iterations to integrate this feedback. Additionally, we can test the system more thoroughly in a limited time period if we get pieces of the system early. While more effort will be required of testers due to regression testing, the system is more likely to be thoroughly tested if an earlier start is made. Characteristic 7: Convergent Convergence states that we are actively attacking all of the risks worth attacking. As a result, the system becomes closer to the reality that we seek with each iteration. As risks are being proactively attacked, the system is being delivered in increments. We are doing everything within our power to ensure success in the most rapid fashion. Convergence is taken for granted in most software development processes, and subsequently, is often assumed. However, if risks such as "excessive feature creep" are not addressed early in the project, the project can spiral out of control. Excessive feature creep is not the same as changing requirements. Requirements will change over the course of a project. These changes are handled by the iterative and adaptive characteristics of the agile process. If requirements do change, there should be an activity in the process to capture these changes. However, a user can change his mind more rapidly than software can developed. If the requirements are not converging, things are probably out of control. Design is another important area for convergent processes. Trial and error programming can cause developers to "paint themselves into a corner". This may cause the system to have to be completely reengineered before its first version is even delivered. Many software development processes include design activities to keep projects from meeting such a fate. An analogy to these design activities is to map out the areas painted in an order that allows you to be stepping out the door of your room upon painting your final brush stroke. No software process can guarantee convergence for all projects. A certain amount of innovation is usually necessary for any substantial software development project. However, a good agile process publishes boundary conditions, or risks that a project might face that the process is unable to mitigate [Fowler 2000]. The agile software process converges when kept within this given set of boundary conditions. Characteristic 8: People-Oriented Agile software development processes empower developers by creating small teams around an increment [Aoyama 1998]. Small teams are a central theme of agile processes. Even large projects using agile software processes divide their members into smaller teams. This gives people the sense that they are not isolated elements in a large organization [Aoyama 1998]. Increments provide the ideal deliverable for these teams. The agile process also fosters a horizontal approach to process improvement. Changes to the process may come from new techniques or technology discoveries. However, most changes to the process come from the bottom up. The changes form a pattern called the organic cycle [Demirors 1996]. In an organic cycle, changes surface in the organization, are evaluated by the project team, are added to the existing process if found to be useful, and evolve as part of the on-going work. Agile processes evolve through adaptation in an organic manner. Developers that are empowered raise their productivity, quality, and performance. After all, they are the best individuals in the organization to know how to make these changes. Much of people-orientation is cultural and beyond the scope of any process. However, agile software development processes set up dynamics that encourage people to collaborate and fosters a people-oriented culture. Many of the characteristics of the agile software development process favor the extraordinary software developer. However, like a precision race car, you will not win any races without good drivers. Getting good drivers and retaining them is perhaps the most important element to delivering a quality product. Characteristic 9: Collaborative Agile processes foster communication among team members. Communication is a vital part of any software development project. When a project is developed in pieces, understanding how the pieces fit together is vital to creating the finished product. There is more to integration than simple communication. Quickly integrating a large project while increments are being developed in parallel, requires collaboration. The extent to which agile processes foster collaboration determines their scalability. Processes which contain approaches to activities (such as pair programming) or actual activities to foster communication will tend to scale better than those that do not. However, activities to promote communication must be balanced with the desire to be parsimonious. Scalability, of course, is a potential boundary condition of a given agile process. Characteristic 10: Complimentary Complimenting is an aspect of good process design that utilizes downstream activities to validate and enhance the outputs of earlier activities. Complimentary activities are activities that work together to produce a better result than they would individually. For example, the activities, "write user stories", "create acceptance tests", and "estimate the user story", are complimentary because they produce descriptions of functionality that can be tested and written within the scope on an iteration. These three activities are part of Extreme Programming. Complimentary activities help the stakeholders (customers, developers, managers, testers, ...) correct mistakes by creating a dynamic that ensures success. The idea is that we revisit an artifact, the output of an activity, many times during the process. As we revisit these artifacts, we look at them in a different way. The result is a tendency toward better deliverables that add value to the project. Conclusion Creating a good software development process is an enabler to project success, but agile software development also requires a culture that fosters creativity and accelerated software development. Process is a piece of this puzzle but it is not the only piece. You'll know when your process is rightweight when it doesn't take extraordinary people to do ordinary tasks (lower bound). However, a rightweight process allows extraordinary people to accomplish extraordinary things (upper bound). Once the process is rightweight, your people emerge as the greatest contributor to competitive advantage. Bibliography [Aoyama 1998] Mikio Aoyama, "Agile Software Process and Its Experience", Proceedings of the 1998 International Conference on Software Engineering, p. 3-12. [Demirors 1996] Onur Demirors and Dennis Frailey, "Why Not Let Software Engineers Define Their Processes?", Software Process Improvement Forum, 3(2) March/April 1996, p.12-15. [Fowler 2000] Martin Fowler, "Put Your Process on a Diet", Software Development 8(12) December 2000, p.32-36. [Gilb 1988] Tom Gilb. Principles of Software Engineering Management, Addison Wesley. [Hammer 1993] Michael Hammer and James Champy. Reengineering the Corporation, Harper Business, New York, 1993. [Highsmith 2000] James A. Highsmith III, Adaptive Software Development, Dorset House Publishing. [Rising 2000] Linda Rising and Norman S. Janoff, "The SCRUM Software Development Process for Small Teams", IEEE Software 17(4) July/August 2000, p.26-32. [Thomas 1998] Dave Thomas, "Web Time Software Development", Software Development, 6(10) October 1998, p.78-80.