Agile has become a fraud
From its inception, the Agile methodology has taken over software development within enterprises, but has it really?
Agile has been promoted as the saviour of software development, which has become over-encumbered with process, rigidity, and almost universal cost-overruns and failures to meet business goals. It promised that developers would be free to let their creativity run wild while businesses would have much more say in the process. However, Agile has become loaded with rigid processes and there is far more direction from business than cooperation.
The principlesβ³
To tackle why Agile is not working as expected, and what might be able to be done about it, we will look at its principles and see what they really mean.
Customer-centredβ³
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
The important question here is: Who is the customer? In software, this is nominally the business, but who is the business? For this, we need to see who is ultimately going to be the best judge of whether the software is working or not. This is not managers, nor higher-level stakeholder who might be signing off on the project, but those who will have to live with the software on a daily basis.
That is the workers whose work satisfaction will heavily depend upon the software working well, so they must be the ones who not only test it, but are intimately tied into its development. This is because they will be the true ones who can know whether whatever is being developed will make their work easier. They are the ongoing reality-check that ensures that the software actually converges upon a workable solution. They need to be iterating and tweaking the design with help from the developers, who are the ones making it function properly behind the scenes.
This is true customer-centredness, and all managers must be facilitating those workers to be in charge of directing the implementation process. This way, all those doing the legwork will be empowered to do their best, because they actually want it to work in the best possible way. For the developers involved, they will feel part of creating something that makes the lives of who they are working with, and building it for, better.
Welcome changeβ³
Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
Let's get this out of the way at the start. The only people who actually welcome change are those neurologically driven to. Everyone else is naturally change-adverse because it disrupts the routines they have developed to cope with their lives. Change for change's sake is stupid.
What makes a lot more sense is to not expect that no changes will be needed, so that our own mental inertia doesn't get in the way of changes that may benefit us by making our lives better. For business workers, that is about making changes that improve productivity and reduce stress. That is facilitated by having processes that capture suggestions for improvements from those workers so that when it comes time to make the changes they suggest, they are ready and motivated to participate in making the solution.
Deliver frequentlyβ³
Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
Each part of a solution successfully implemented enables building the parts that are dependent upon it. That is the real benefit. While it may signify progress to higher-ups, and make everyone involved feel they are accomplishing something, it still comes down to creating the bedrock upon which the whole solution relies. That is the right perspective to have as a working mental framework.
How long each part takes will depend upon its complexity and how much can be broken down into testable modules so that they can be relied upon for what comes next. It is not about enforcing a delivery regime, but building working blocks and making them available when ready. Time is not the goal here, but working software.
While frequent delivery may help with exposing the software with the development team, it is counter-productive to deliver it to end-users frequently. As an example, unless using Firefox as one's main browser, every time it is opened will trigger an update, which is just a plain nuisance. Larger businesses will have to do much testing to ensure it is compatible with their other software stack, but at least have the ability to roll out updates automatically to their users. Everybody else would have to be presented with an update prompt, which they don't really want to have to deal with.
Daily cooperationβ³
Business people and developers must work together daily throughout the project.
This is about working together so that all are on the same page all throughout the project. All participants must be able to interact with each other at any time that suits those doing the work. That is about facilitating the free interplay of ideas so that there are no unnecessary delays and frustrations. That is not about forcing people, but letting them find their time and space to interact in the best way.
Trustβ³
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
No one likes being micromanaged, and such managers or bullies must not be involved. If people cannot actually be trusted, they should not be on the team. Feeling trusted allows participants to concentrate on building the software rather than how others are considering whether they are performing properly. Software development requires thinking clearly about a lot of moving parts that must work together, and being distracted by personal stresses impedes that essential holistic mode of thinking.
Face-to-faceβ³
The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
For half the population that is introverted or perhaps has neurological conditions that makes them adverse to working in too much company, face-to-face is not the ideal. However, being able to have a form of interaction when necessary is. So, face-to-face may only be the most efficient for those that like to work that way. It comes down to how much interaction does there actually need to be? A large part of the time for developers will be about coding what has been discussed.
For the business workers, it may be about reflecting upon what would be good interfaces for them as they may not have really clarified their thoughts in that way. While it may seem to some that sitting together and nutting out what to do is what it is all about, that process depends upon the background thinking and preparation being done beforehand. That preliminary stuff is not trivial and cannot just all be made up in the moment of a creative orgy. Experienced people are needed, but that does not mean they have all thought about the situation in the way needed. That takes time.
Working softwareβ³
Working software is the primary measure of progress.
Measures of progress must be grounded in reality, and that reality is best served by what works. That is far better than guestimates of the percentages that are complete, which are usually underestimates because the final effort to complete a task can take an inordinate amount of time to do properly. While basic blocks may be able to be built and demonstrated easily, the interactions between them may be complex, and so take more time to make them work properly, and get to a point where they can be demonstrated. Thus, the timing of the release of each working software drop must be flexible.
Sustainableβ³
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
No one can work at peak performance for very long, especially if the activity is stressful. Expecting people to be able to do that is unrealistic. People need times of relaxation in between blocks of measured effort. Balance is essential here, being far more useful than manic spurts followed by crashing, which only undermine sustainable and steady progress. Balance avoids unnecessary stresses which only wear out the body and debilitate emotional steadiness and clear thinking.
Software development is a long-term process, so it must be planned as such, with allowance for long-term stability of the people and activities involved. The team must be monitored so that what they need can be made available when they are ready, but not in a way that forces them to rush through their work, or leaves them to get bored because something is held up. The team building the software is critical. Everyone else is their support network, including managers.
Good designβ³
Continuous attention to technical excellence and good design enhances agility.
Design is critical, but it does not, and cannot, all be done up front, just because everyone does not know all they need to know beforehand. That means that everyone is experimenting to find what works best in the context of the solution. Refactoring needs to be a part of the process so that opportunities to make the product more efficient, coherent or understandable are implemented as soon as possible, because such changes will make later development easier and avoid rework to back-fit it at an inconvenient time later on, which may break working functionality.
Technical excellence is not necessarily about sophisticated or novel solutions, but what is going to help make the product more stable and easier to maintain. That means that more straightforward and understandable solutions are favoured over intricate and obtuse ones that few understand and are more likely to be broken because of that. Maintainable software is always better than what developers and users swear at.
Simplicityβ³
Simplicity β the art of maximizing the amount of work not done β is essential.
Only put in what is needed to make the product fit for purpose. That does not necessarily mean basic or obsessively utilitarian. Users must enjoy using the software, so some flourishes that make repetitive tasks even a small bit easier or less distracting from their overall tasks will pay dividends in user-friendliness. This is why having business users as essential drivers will ensure such small but satisfying additions are included, whereas their managers may think they are superfluous, if they think about them at all.
Software is meant to be used, and whether it is fit for purpose is dependent upon whether it is worthwhile using. Business users are the best judge of that over any preferences from their managers or developers. Little things done often can be the difference between finding the software delightful or a pain to use. These can be as simple as spatial arrangements of screens to what is shown. Something may seem useful theoretically, but may be distracting in practice. Let users decide, and that may take some time to find out. Do not be afraid to ditch what does not actually work for them.
Self-organisationβ³
The best architectures, requirements, and designs emerge from self-organizing teams.
Most people will tend to make a lot of little choices to make their working life better, and those will change over time as the work evolves or they discover that some things do not work for them as expected. We are not all totally so self-aware that we can predict what we need in the future of an unknown multi-month process. Teams are no different, and when they need to maximise their productivity, they need to be able to make their own decisions about what works for them or does not as they go along. In this, managers are their support team, and not their directors.
Reflectionβ³
At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
Reflection is an integral part of self-organisation, so how formal and often the reflection process is dependent upon the team. Such times will be better after a period of intense activity which may have precluded time to make micro-adjustments to working rhythms. Technically, a team working in balance without excess pressure will manage their working arrangements and optimise themselves as they go along. If not, then something must be done to relieve the stress that is precluding them from doing that. It is the manager's job to make that stress go away.
Methodologiesβ³
Methodologies are a collection of methods, and so will tend to regiment software production.
Methodologies, like any ideology, are an anathema to creative thinking, purely because they rely upon regimenting and constraining thought processes to stop deviations from what the methodology is designed to do. For creativity to function well, methodologies need to be guidelines, not the definer of what is done. That allows creativity to break out of what is expected and come up with a better solution. That is the essence of agility.
Many processes have become regular parts of Agile implementations, regardless of whether the teams using them find them useful. Creativity needs some focus, but those expected to be creative need to be able to define how they can best do that. Modern Agile is locking down too much of what should be their own process, so that people feel too constrained to work creatively.
What is managing?β³
The general idea of managers is that they will control those under them because they are responsible for them performing as required.
The idea of managers as being controllers has thoroughly distorted what they should be doing, which is to facilitate their workers doing their best sustainably. Unfortunately, this has been conflated with their own ideas of how important they are and thus how they see their workers, and ultimately how much they trust them. That is prison guard mentality compared to what is required to actually required to get people to perform at their best.
Essentially, managers are part of the support team for their workers, because it is their workers who are creating the worth of the business. Getting in the way of workers getting on with their work is failing as a manager. Getting workers what they need to be as sustainably productive as they can is being successful as a manager. To manage workers who need to be creative requires being fairly hands-off, while being attentive to what may be affecting their performance and stepping in to help them find the way that gets them back on track.
As such, management is required to walk a fine line of supporting their workers, but being able to take charge and restore situations to productivity when they go off the rails. That requires patience and trust. It is not about controlling their workers, but about being in control of themselves so that they are not creating problems for their workers. That may mean pushing back on excess pressure from their own managers who may be suffering from their own delusions as to what managing means.
Managers are critical to how well a business performs, but they need to see themselves as the inverse of what they may have been thinking they are. They are part of the overhead of the business that support those who actually produce what the business makes their money from. Software development teams are part of that overhead, but they are building what will make the productive workers more so. Their managers need to let them create that in the best way they can. Agile may help that, but it is not the goal, and nor should it place any limits on those using it to make them less creative.