Monday, August 31, 2015

Agile Project Challenges

At the software project level, numerous challenges emerge in the use of Agile. For example, project managers in charge of software development and maintenance projects are interested in formal requirements documentation, scoping, and estimation of the work required in the projects. Agile methods, based on the Agile Manifesto, give far less importance to these project factors. An upfront and detailed business analysis is also eschewed in an Agile approach although its significance cannot be overestimated. Outsourced projects find it difficult to get a user on board the project as expected in an Agile approach. Component and third-party services developers face similar challenges when it comes to Agile projects.

An important aspect of an Agile approach is the phenomenal importance it gives to an individual. Therefore, people, their skills, motivation, and communication are crucial ingredients of a successful Agile project. Cockburn and Highsmith (2001) stress amicability, talent, skill, and communication as critical factors for Agile success. How to implant these attributes within a Agile project team? This requires greater understanding of the sociocultural aspect of a project than its technical aspects. Facilitation, leadership, and personal contacts assume far greater significance in a pure Agile approach than in a planned approach to a project. In contrast, formal methodologies in software projects (particularly large-scale software projects) separate an individual's subjective elements (e.g., motivation) from the actual process. Formal or planned methods typically create a work breakdown structure and assign resources to the tasks within the project.

The issues specific to the use of Agile methods mentioned at the start of this chapter need to be considered in the context of the already known issues and challenges that are faced in software development. These issues have been discussed by a range of practitioners as they are encountered right from the start of the project. (See Dooley [2009b] and the interesting discussions on information technology [IT]-related issues in Agile Software Development in Large Organizations [Brenner; Lindvall et al., 2004].)

Some of these specific issues and challenges in IT projects are as follows:

- Requirements: One of the most common challenges in a project from the very beginning of software development is the lack of formal and sufficient requirements. Bennatan (2009) highlights the most common requirements problem as insufficient information at the beginning of the project. An Agile project officially kicks off much earlier than a traditional, planned project in terms of its requirements modeling exercise. Lack of sufficient information is quite acceptable at the start of Agile projects. This, however, compounds the original challenge of lack of information and formal requirements in any IT project. The importance of early requirements capture, modeling, and analysis cannot be discounted in a software project. In addition to scoping the needs of the user, formal requirements modeling also identifies dependencies among the requirements. These dependencies may not be evident unless a majority of the project requirements have been understood and modeled at the beginning of the project.


- Changing purpose of software: Software systems are developed for a specific business purpose; however, that purpose keeps changing, thereby requiring changes to the software to make it adapt to the new business requirements. Changing the software to reflect continual business adaption is a major challenge. Continual demand for changes can lead to errors, poor service, and business resentment. As the number of users increases, the difficulty of adapting to change also increases. In a pure Agile approach, the purpose of the software also evolves as the development progresses. The complexity of a system in operation (such as the number of users, performance, and scalability) requires to be understood in development through a formal, planned methodology.


- Rework: Most software products undergo rework after the project is completed. The business needs identified at the start of a project usually change by the time it is completed. Agile projects are quite comfortable with these changing requirements. These requirements, however, are encapsulated in story cards pasted on visible charts and walls. Changing the requirements later in the life of the system, reworking the code written by someone else, and tracing the changes from requirements to code lead to major challenges in software projects. The question of how these story cards can be used when the system moves into maintenance needs to be explored in much greater detail than has been done thus far.


- Budget and resource relationship: While the budgets and costs appear as precise figures, the resources (typically human resources) are very difficult to quantify. Therefore, a loose relationship emerges between costs and resources for a project. Managing budgets without tight correlation to resources can skew the entire project.


- Scope creep: This is a challenge even after formal requirements have been captured and documented. Slowly increasing scope, or changing scope, can easily derail a project—mainly because the effect of scope creep is not linear. A small increase in the required functionality can result in affect the plans for its data conversion, integration, and testing. This, in turn, results in time and budget overruns. While the well-intentioned demand for a new functionality has to be accommodated (and is done so quite well in an Agile approach), extreme care is required in modeling, discussing, and agreeing on the changes and the corresponding effect till the end of the development life cycle.


- Technology creep: This can be an interesting and challenging factor that emanates from the availability of newer technologies than the ones used at the start of the project. While the scope creep appears from the business user's side, the technology creep can appear from the developer's side in a project. As the original technology with which project estimates were made is superseded, this creep factor can throw the entire project estimate off balance. The time, resources, and deliverables based on a new development environment, test tools, and deployment can be dramatically different. The need to incorporate the newer technology or its newer version may be justified; but it is important to model or prototype the change as well as reevaluate the project plans and cost benefits in the context of the new technologies.


- Soft issues: These are the people issues that invariably exist in a software project. Morale and motivation, and leadership and facilitation—these issues provide a major challenge to a software project mainly because they are very difficult to quantify. For example, interactions among team members, personal motivation and goals, and changing personal circumstances—all of these impact a project, but most of them remain hidden underneath the facade of a person. Project managers are not trained in handling these soft issues and yet their impact is substantial. The role of a developer and a project manager is quite blurred in an Agile approach. The two roles are distinct in practice. This leads to a mismatch between the expectations of a product owner and the development team. Developers may not necessarily be managers, and vice versa—leading to further challenges in terms of soft factors.


- Inaccurate estimates: A vital challenge in software projects is the need to make estimates for the project to be sanctioned. And these estimates are to be made at an early stage, with minimal information about the project. Use of previous projects data (metrics) and experience in the domain can help alleviate part of this challenge.


- Tracking slippages: While time slippage is an inevitable part of a software project (due to some of the issues above), a crucial aspect of project management is to incorporate those slippages back into the project plans as soon as possible. This "feedback" of slippages and missed deliverables requires a detailed project plan to start with, and the ability of project managers to incorporate the changes quickly. Similar to the scope and technology creep factors, the slippages in time, budgets, and deliverables are also not linear; every delay in the delivery of a module can have multiple ramifications on other deliverables, people, and user expectations of the rest of the project.


- Version control and release: There is a significant amount of unpredictability in terms of version releases, especially in large and complex systems made up of multiple packages. Lack of sophistication of version control during development, testing, and operation can all add to the challenges in software projects.


In addition to the general challenges in typical software projects, there are some specific issues that appear in Agile approaches in those projects. While Agile projects subscribe to the Agile Manifesto and have some excellent principles and practices to guide them, these pure Agile approaches also have corresponding challenges. Some Agile-specific issues and challenges are as follows:


- The priority is to satisfy the customer. While this can be achieved through early and continuous delivery of valuable software, there are many practical situations that prevent early delivery of a working code. For example, there may be requirement dependencies that cannot be easily elicited through user stories; without due consideration to these dependencies, a working code may not be easily delivered to the customer's satisfaction.


- Capitalize on changes to the customer's advantage. This is achieved by welcoming late, changing requirements. Yet such changes impact the budget and business operations. Third-party software development may only occur when requirements are fixed and any change needs to be subjected to stringent change-control requests. Handling of multiple parties involved in a project requires legal contracts and controlled changes that are not easy to implement in Agile projects.


- The need to demonstrate and deliver working software every couple of weeks to a month has always been the desire of developers from the beginning of computing. Prototypes and mock-ups have been used in the past; and even today, these are standard tools for customer engagement, requirements gathering, and expectation management. Agile approaches thrive on developing and extending prototypes to full-scale products. Yet, at times, a continuously working software may not be feasible because of factors such as technical dependencies or changing requirement


- Developers and business stakeholders must work together in a collaborative style on a daily basis throughout an Agile project. This collaborative working style exposes business experts to the product development cycle (Thomsett, 2009). This may not always be convenient or logistically possible as the key business stakeholders are also focusing their attention on running the business. Agile development scenarios in which the whole team is co-located is inevitably more efficient in its communication than the other extreme, in which team members are distributed all around the globe (Dooley, 2009a). The logistics of getting the team together not only across geographical boundaries but also across those of cultures, language, time zones, and value systems heavily impact Agile project communications. Motivated and self-aligned individuals are the key to a project's success, irrespective of the methodology being used. Should it then be really a privilege of Agile only? The nonavailability of such individuals was one of the root causes for large and complex projects to follow formal, planned, and mature software development life cycles (SDLCs) in the first place.


- Practicality of face-to-face conversation needs to be considered. While a face-to-face conversation is indeed the most effective method of conveying information, its practicality is always question. For example, in some instances, such as while dealing with offshore projects or third-party services development, this conversation is not possible. In other situations, even if such conversation is made possible, it is not binding. For example, a verbal agreement between the user and a developer in terms of a field positioning is quite acceptable in an Agile project; but if that field position has compliance requirements associated with it, then it has to be documented formally.


- A working software may not always be the primary measure of progress. There are numerous and vital activities that need to take place in real-life projects that do not necessarily produce a working software, for example, work on modeling requirements with use cases. Another example is the creation of detailed architectural diagrams that do not directly produce a working software. Yet, these (and similar) activities are vital for the project. Focus on sustainable development is important but indexing it to a working software can be a challenge in itself.


- Team commitment, especially as part of Agile planning, can be highly subjective. Extracting team commitment to developing a certain number of stories can defeat the basis of the Agile Manifesto. Yet, without some indication of the development effort, the rest of the planning within and around the project can suffer.


- Self-organized teams cannot always be "designed." Good self-organized teams have to "happen" on the basis of establishment of working relationships, trust, and common understanding. Such "happening" of a self-organized team is an absolutely crucial element to the success of an Agile project. Large and complex projects do not have the privilege (in terms of time as well as quality of personnel) to enable the "happening" of a self-organized team.


- According to the Agile Manifesto, "Responding to change over following a plan," is important. Therefore, locking a development effort to a plan is against the Agile value. Yet, how many projects in real life can progress without a firm baseline that is indeed lockedin? In practice, even pure Agile projects need a certain amount of commitment and planning before the development begins—albeit it is not fully documented as done in planned projects.


- Setting a goal in a sprint (specific to the Scrum method) is an important aspect of that sprint. Yet, this goal can be based on a subjective understanding of the team in terms of what the user wants. There is usually a need for explicitly stating the goals as is done in planned methods. For example, the stories within a sprint are geared toward a goal. However, during the actual sprint, new ideas and values may be discovered that would change the goal. Does the team have the liberty to change these goals? In practice, the team may not have the privilege to change the goals—especially if the project is an outsourced development where contracts for delivery of software have been signed.


- Planning in Agile projects (especially Scrum) can be based on project velocity. This project velocity is the number of stories a team can actually deliver ("done") per sprint with normal speed. There is no commitment in ascertaining the project velocity and, therefore, there is a constant struggle between the Agile and planned elements within a large project.

Taken from : The Art of Agile Practice: A Composite Approach for Projects and Organizations

No comments: