2 Agile in a Nutshell
2.1 The Origins of Agile
Agile software development is not a specific methodology, a process, or a prescription on how to do your job. Rather, it is a set of beliefs, a philosophy, that should guide a software development team in making optimal decisions. Agile was not created out of thin air of course, it was very much a reaction to the then ubiquitous approach called Waterfall. In Waterfall large software projects are divided into specific stages, each stage should be completed before the next stage can start. Subsequently these stages are problem analysis, designing the project, writing the software, testing it and finally implementation and maintenance. Here you can find a clear and neutral introduction into the Waterfall approach. The goal of Waterfall is delivering complete and faultless software. The gravity of a Waterfall project is the documentation, in which every requirement and aspect of the software is written down meticulously. The underlying conviction is that the software is written faster and is of higher quality when all aspects of it are decided upon upfront.
Projects done with the Waterfall method have a major pitfall, however. They can take a very long time to be completed. The combination of sequentiality and completeness might cause projects to last many years before software is delivered. Each time an error or incompleteness is found in one of the later stages, the project moves back to the previous stage to fix it. Because of the long duration of the entire process, it often occurred that the end result was no longer a good fit for the ever changing market. The problem analysis might be done years ago and the problem has changed in the meantime.
2.2 The Manifesto
During the nineties, several reactions to the cumbersome Waterfall came to be. A number of influential thinkers in the world of software development started to explore different ways of writing software. Alternative processes were suggested, such as Scrum and Xtreme programming. Eventually, in 2001 a group of seventeen people came together in Utah and drew up the Manifesto for Agile Software Development. Their short 68 word statement is:
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
If you think “well, that is rather vague”, that is on purpose. It is not a process you should follow or a methodology that prescribes how you should approach software development. Rather, they are core values that guide the development team with the many choices it has to make along the way. At every crossroads the option that is most in line with these values should be selected.
Agile has a radically different approach to software design, by recognising a number of flaws in Waterfall thinking. First, it is impossible to think through all the aspects of a complex design and architecture before starting to write code. Software has to grow organically instead. Secondly, customers typically don’t really know what they want from the product, until they start interacting with it. Legally it might be a good idea to have all the aspects checked off before getting to work, but it will not keep the customer satisfied. Finally, customers and stakeholders will lose interest and the faith a project will be completed if it takes a long time before a working product is delivered.
2.3 The Twelve Principles
The Manifesto was accompanied by a set of twelve principles that flow from these values. They are more applicable than the four values and are thereby the principle guidelines when making choices. They are (numbering added by me):
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
Business people and developers must work together daily throughout the project.
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
Working software is the primary measure of progress.
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
Continuous attention to technical excellence and good design enhances agility.
Simplicity–the art of maximizing the amount of work not done–is essential.
The best architectures, requirements, and designs emerge from self-organizing teams.
At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
In Chapter 4 we will interpret them in the data science context.