Do you use Agile methodology? The most common answer, it seems, is: 'Sort of!' Is this a trump card to get away with the nitty gritty of Agile? Or is it the cover up for a failure? The real answer is uncovered when we start analysing the context.
This article examines how a namesake is used and Agile methodology is manipulated to suit a project’s context. It also consolidates the views of IT industry individuals on the modal and if it is in any way useful.
Software product development is a process that translates vague business requirements into quality software products. During the tenure of software development, the vague requirements can become ambiguous or become concrete. The project funder may add new features or remove existing requirements.
Depending on the requirements, a software project can become cumbersome to a point that it gets binned, or things can move smoothly and it gets delivered ahead of schedule. It is important to create robust architecture to cater for changes in requirements. Managing these changes is important and is done by adopting different types of software development practices: Agile, Scrum, Kanban, Waterfall, Prototyping and Spiral, to name a few.
One of the most used frameworks is Agile. Most of the software companies have now picked up the framework, but a lot of them fail to implement it correctly and they term it instead as ‘Hybrid Agile’.
Throughout our study, I have identified the context or the environment in which software is developed. I then tried to go through the contexts and analyse them. The contexts are the factors that affect the decisions. Factors can be inside and outside the control of the development team. The decisions are not only about adopting a particular software development practice, but also about the budgets, resources, timelines, etc.
I tried to identify how the factors impact the chosen methodology and how the methodology is really implemented in the Software Development Life Cycle (SDLC). Then I analysed if the ‘hybrid’ concept of Agile development is a useful methodology to follow, or if actually causes harm. I also looked into why certain aspects of Agile are deliberately not used and what can be done to avoid this.
Research methodology
As part of the research, I contacted over 100 individuals. Out of that, I got 86 relevant responses after ‘filtering out the noise’. To gather data from different perspectives, I tried to include a broad set of candidates. All the respondents come from different backgrounds within the software industry.
As the backgrounds differ, the candidates carry different skill sets. A premise can be set that they have different opinions on project implementation methodologies. Some would be critical of methodologies while some would favour more standards in the Software Development Lifecycle (SDLC). Table 1 lists the set of profiles that the respondents fell into.
| TABLE 1: ROLE TYPE |
Selecting a broad range of respondents is useful, as it gathers views of almost every role in a software project team. However, it comes with a cost. The data set is at times biased when categorising the results per role. There needs to be a separate study to research the different viewpoints of individuals within multiple disciplines in IT. Although I selected ten different roles, I did not categorise these in the final findings. The reason is to encapsulate individual views and roles from their feedback. |
| Software developers |
| Software support engineers |
| Software testers |
| Business analysts |
| Scrum masters |
| Product owners |
| Project leads/team leads |
| Project managers |
| Delivery managers |
| Solution/technical architects |
It is also important to categorise the environments in which respondents work on a project. An individual’s perspective on a project changes as the environment changes. A developer working on a project with a team size of ten would have a different view to another developer who works on the same project with a team size of five. Certain factors that would affect his/her thinking would include:
- Delivery date;
- Co-ordination needed;
- Expertise of teammates;
- Leadership;
- Delivery model;
- Testing efforts needed.
The above example shows that just by changing one factor (team size), it can impact the viewpoint of a developer within a team. I would like to emphasise that the context in which a project is carried out is critical to analyse a project’s agility and how agile a project is managed. I have identified the below environmental contexts that might have an impact on a respondent’s view of their project.
|
TABLE 2
|
|
Environmental Context
|
| Project type |
Consulting project |
In-house product development |
Offshore development |
| Organisation size |
New (<5 years) |
Medium (10-15 years) |
Settled (>15 years) |
| Domains |
Banking and financial |
Core IT |
Telecom |
Although there could be more possible categories, I have concentrated on project type, organisation size and domain. I did not put enough emphasis on the team dynamics or team size as that would be too detailed an observation and out of the scope of this article. Type of project would categorically confirm how involved a client is in the project development lifecycle. Since Agile projects need a lot of client interaction, it would be necessary to identify if the client is part of the development process or if they are only giving views on the final product (either at the end of the project, e.g. Waterfall approach, or during the project e.g. Phased approach).
I have selected organisational size as one of the factors since the more mature an organisation, the more reluctant it is to change their processes and practices. A young organisation is ‘agile’ in itself, as it is learning and is willing to adapt. There are a number of factors attributed for this reluctance to agility, but it is out of scope of this article.
Finally, I categorised the industries with their respective domains. I could only reach out to banking, core IT and telecom professionals. There are, however, other domains that would have been useful but would have added a lot of complexity, so were excluded.
There are certain entities that can be measured to quantify the scope of Agile development in a certain environment. As the hypothesis says that ‘Hybrid Agile’ is not Agile, I had to confirm that Hybrid Agile exists and how can we define it and measure it.
Hybrid Agile is a development process that does not cover all aspects of Agile software product methodologies. It is important to emphasise that software project delivery methods can be measured and can be analysed to find out if a certain methodology is enforced properly or not. I have used the below measurable entities to analyse if Agile methodology is properly used, or what are the conditions that prohibit companies to use Agile methodologies and if a company is at all willing to use Agile frameworks. It is a hypothesis that companies claim to use Agile but are only using Agile for the sake of using it. In practice, we believe Agile is used only as a namesake.
|
TABLE 3
|
|
Measurable Entities
|
| Resource allocation |
Extended deadlines |
Team structure |
| Normal changes allowed |
Atomic measurable releases |
Team dynamics |
| Architectural changes |
Duration of releases |
Budget overrun |
Agile development
We consolidated Agile principles and definitions as defined by different academic scholars and industrial analysts. We also interviewed candidates for their personal views on definition of Agile development. Ken Schwaber and Jeff Sutherland, in their 'Scrum Guide', define Scrum as: "A framework within which people can address complex adaptive problems, while productively and creatively delivering products of the highest possible value." Alistair Cockburn, in Agile Manifesto, writes that the priority is to satisfy customers through early and continuous delivery of valuable software. He emphasised return on investment for the customer and also mentions that uncertainty is expected and should be managed within the project.
As part of an Agile manifesto, as defined by Scrum institutes, individuals and interactions should be favoured over processes and tools; working software over comprehensive documentation; customer collaboration over contract negotiation; and responding to change over following a plan. There are other definitions that article looks into and brings interpretation of the principle of Agile:
- Change is inevitable;
- The customer is the king and always right;
- Adapt and improve;
- Be self-sufficient. Be productive;
- Follow processes and best practices but do not let them hinder productivity. (Be less formal when needed);
- Clear and concise communication.
As we look into Agile development, it appears there are too many processes and a reluctance to change leads to a failed project or an expensive project. This is not good for either customer or company. Return on investment should not only satisfy the customer, but also the company itself. Finally, the most important part in Agile development is how the communication is managed. A clear and concise communication is key to all issues. As long as the client is aware of the progress and can see tangible response to queries, they will not make unexpected change requests. Also, for good communication, an informal approach should be used that can have a good impact on both the sides.
Agile concepts are implemented through different frameworks such as Scrum, Kanban, Extreme programming and Agile Unified process. Out of these, Scrum is one of the most popular methods. Our research has been mostly based around Scrum and its associated practices. Scrum defines a set of best practices through which it implements the principles of Agile. In theory, the framework looks promising and something that can help a project team develop a project, even after including changes during the development. However, when it comes to practice, it is altogether a different story. This is covered in the next section.
Analysis
[caption id="attachment_30556" align="alignright" width="300"]
Moving a task on the Sprint plan during the daily scrum[/caption]
The results have been very supportive of the hypothesis. A significant portion of the participants use Scrum framework to implement Agile principles. From the results, it is evident that despite using the Scrum framework, Agile is not implemented as it should be. Even the concepts of the Scrum framework are not implemented properly. The results clearly suggest that if the foundations we use are not strong enough, we cannot guarantee a fruitful result.
To start with, the questionnaire asked whether changes were allowed 'during the Sprint'. Sprint, in a Scrum framework, is a time-boxed event where no changes are allowed to the shippable product. It is considered as an 'atomic entity', where the development team is allowed to work only on the assigned tasks and is not supposed to take requirements from the outside world. Changing requirements between the Sprint means changing the atomic nature of Scrum Sprint.
However, from the study, it seems that the over 70 per cent of the time, changes were allowed in the Sprint. (Some 22 per cent said changes were not allowed, while another 8 per cent said that they were not part of the development team.) Once the requirements change, the corresponding definition of the 'expected product' at the end of the Sprint would also change. This would mean that the developers were either burdened to deliver more, or relaxed as the definition of the feature has changed.
A development team needs to be self-sufficient and any impediment should be escalated to the Scrum Master (in Scrum framework), so that it is resolved as soon as possible. There should not be any dependency outside the team for development purposes. When we see the results, they are not at all synched to the definition of what an Agile team should be. Some 83 per cent of the respondents confirmed that their team was not self-dependent and had dependencies on the outside world (i.e. team not cross-functional). Having dependency outside the team would lead to delays in the project. This is because securing a resource is an overhead and comes with time and cost. Even if the external resource is abundant, it is not always used effectively. Agile advocates not only responding to changes, but also to responding effectively.
I also looked at documentation of the projects. Delivering documentation is part of the project delivery. However, in practice, most of the developers believe that it is not something they consider a part of the project. They do it only when forced to do it. Whatever the development team think, only 42 per cent deliver it outside the development phase (i.e. before the release). Over 30 per cent of the respondents claim that they do documentation during the Sprint: something that is contrary to their beliefs on documentation. A face-to-face interaction with a few developers reveal that one of the reasons for this is that they need to keep the project managers satisfied.
I also categorised the results based on the context in which the projects were developed. New companies tend to adapt to new processes. They happen to follow most of the principles of Agile. Mature companies (>15 years) tend to be the worst offenders. Some of them claim to implement Agile but, in real practice, they fail to implement a mix of Waterfall and Agile. In particular, they tend to use their own flexible software development methodology that suits their needs.
Overall, I could only find very few companies that implement all Agile principles. A lot of respondents who claim they do use it, do not seem to be satisfied with the responses they get by adopting Agile. They tend to blame the management for enforcing the process because they do not see any value in it.
Hybrid Agile - conclusion
The principles of Agile development have a lot of meat on them, in theory. When it comes to practice, however, we tend to leave out certain aspects of the principles. In due course, we move from core Agile development to something called a Hybrid Agile development. A Hybrid Agile development, since it is not Agile, does not result in what we expect from an Agile project and we tend to blame Agile practice instead of blaming the Hybrid Agile practice. Instead of changing our best practices, we pass the buck back and blame the methodology, even when we never implemented it properly.
The survey conducted clearly shows that when implementing Agile methodology, developers tend to not implement all aspects of Agile development. This could possibly be linked to the age of the firm. The more mature the firm is, the more reluctant it is to adopt Agile practices and instead follow its own practices. This is possibly because the current practices have been working reasonably well and the firm is more reluctant to change.
Another reason for not implementing all aspects of Agile is that the project team is not close to the business, and the business is not able to commit itself to the project throughout the full project life-cycle. Budget resources can be one of the factors affecting this. Finally, one of the most important reasons is that team does not look beyond task completion. Myopia restricts them to view the project as a set of tasks that need to be completed using a set of standard processes and practices. If a project team visualises the benefits of the Agile development, perhaps they would use it properly instead of blaming a broken Hybrid Agile approach for issues in the project.
As we see that best practices are not followed, the 'blame game' begins and Agile methodology becomes the scapegoat. Should this practice persist, we will soon blame the methodology and potentially revert to our traditional development process (Waterfall or others). Change is the only constant; we need to adapt and continuously improve for the betterment of not only the project, but also the company. This would foster better relationships with clients, and result in higher returns on investment.
Author: Shilabhadra P. Parida, Solutions Architect – Business Online, IT Centre, Cabinteely, Dublin 18