The One-Page Guide to Agility

Agile development can look a bit crazy, at first.

I'm writing this guide to explain the foundations of agility, outline its benefits, and show why it's not so crazy after all.

A Brief Definition

Agile development = "pragmatic iterative development". If you're agile, you produce software in small stages (i.e. you break each project into a series of mini-projects) and you use practical techniques pioneered by other successful teams. That's a very simple definition of agile development.

Many processes fit that definition. Extreme Programming (XP) is one of them; the others include Feature Driven Development, Crystal Clear and Scrum. Each process is different. For instance, up-front design is encouraged by Feature Driven Development but it's discouraged by XP; pair programming is required by XP but other processes say it's optional or don't mention it.

The most important common thread, uniting all agile process, is their commitment to pragmatic iterative development. Other common elements include flexibility, concurrent workflows and rapid feedback.

History of Agility

In the mid 90's, researchers documented a major company's development process. The process bore the hallmarks of what we now call "agility": it was iterative, encouraged individual creativity, allowed lightweight to non-existent design documents, and included frequent automated tests of the evolving software. Can you guess the company?

It was Microsoft. [1]

The Microsoft document, which you'll find amongst the references listed below, illustrates two important points:

Agile techniques are older than the "agile" buzzword. For instance, iterative development dates back to the 1950s, was used on the space shuttle, and is now endorsed by the US Department of Defense. [2]

Agile techniques have proven their worth in successful teams. Microsoft is only one example. When Dr Alistair Cockburn set out to study successful teams in the mid 1990's, he discovered similar techniques all around the world. Ironically, many teams apologized for not following traditional processes. Only when team after team made the same apology - all of them working on successful projects - did he realise that no apology was necessary. [3,7]

The techniques we now call "agile" evolved over several decades. They are popular because they work.

Benefits of Agility

When Dr Cockburn studied successful teams, he found more than just the widespread use of agile techniques. He also discovered that agility saves time. Dr Cockburn estimates that agile techniques reduce costs by at least 30%. [4]

When considering such big savings, it's only natural to expect a corresponding drop in quality. Interestingly, that's not what happens; agile processes enhance speed and quality.

Exactly the same thing happened when Japanese car makers adopted "lean manufacturing" in the 1980s - their cars became cheaper and better. Authors Tom and Mary Poppendieck report many parallels between the Japanese auto industry and agile development. For instance, both rely on rapid feedback, continuous quality control, and concurrent workflows (instead of the serial workflow we call "waterfall). [5]

I'd like to draw your attention to two points in particular:

Agile processes "debug" requirements. There's more to quality than merely making the software conform to the requirements; you must also make the requirements conform to the users. Agile processes create software in small steps, getting rapid feedback from users at every step. Seeking regular feedback amounts to "debugging" the requirements - testing to make sure they really meet the users' needs and correcting any deficiencies found. Traditional "waterfall" processes gather requirements carefully, but offer few tools to verify their correctness.

Agile processes use continuous quality control. They test throughout the project and fix bugs promptly. (They never defer testing and fixing to the end of the project.) Imagine taking the agile approach to its logical conclusion: we'd test so regularly that our software would always be fully tested; we'd fix all bugs immediately. With such a regime, the bug count would hover near zero for the entire project. This is a rather extreme approach - in fact, it's the exact approach used in Extreme Programming.

Other agile processes are similar but not so strict. Typically you're allowed a few unfixed bugs but you still have to produce a high-quality release at the end of each month.

By debugging requirements and continuously monitoring quality, agile processes detect problems early and fix them before they grow. The net effect is lower risk and higher certainty of project outcome.

Visible Progress

The agile approach to quality control has an important side effect: it demonstrates the project's status. Because tested software is released at regular intervals, the team's progress (or lack thereof!) is clearly displayed. You never have that awkward situation we see on traditional projects: the developers claim to be "70% complete" but no-one's sure how many bugs there are or how long it will take to fix them. In fact, no-one's sure whether the developers really are 70% complete; perhaps there's another 70% which they haven't even started! :-)

On an agile project, "70% complete" is a much more precise statement. It means 70% of the planned features are debugged, tested and could go live tomorrow (if we wanted). We can even fire up the latest release to see the features with our own eyes. We can't kid ourselves (or our customers) with overly-optimistic progress reports. [6]

Fixed Price Contracts

A fixed price project is a great time for agile development, since under a fixed budget you need to work quickly without sacrificing quality, and that's exactly what agile processes let you do.

Unfortunately, there's been some confusion about agility under fixed price contracts. (I'm talking about contracts with fixed price and scope). Advocates of Extreme Programming (XP) say you can't use agile processes on fixed price projects. While that may be true for XP, it's not true for agile processes in general. Take Crystal Clear for example, it's an agile process that was developed on fixed price projects. Crystal Clear supports changing requirements, just like XP, but it also supports the other alternative: fix the requirements at the start and don't change them later.[7]

Conclusion

Agile development is good software engineering - different, but good.

Agile techniques have proven themselves in successful teams around the world. They save money, enhance quality, and demonstrate progress with the ultimate status report - working software.

References

[ 1 ] How Microsoft Builds Software, M.A. Cusumano and R.W. Selby. (Costs US$10 unless you're a member of the ACM) Cusumano and Selby studied Microsoft from 1993 to mid-1995, 6 years before the official "birth" of the agile movement.

[ 2 ] Iterative and Incremental Development: A Brief History, Craig Larman and Victor R. Basili. (Costs US$19 unless you're a member of the IEEE)

[ 3 ] Crystal Light Methods, Dr. Alistair Cockburn.

[ 4 ] Process: The Fourth Dimension, Dr. Alistair Cockburn.

[ 5 ] Website and book (Lean Software Development), Tom and Mary Poppendieck

[ 6 ] I first saw this described in Debugging the Development Process, the 1994 book by Microsoft's Steve Maguire. The same idea crops up throughout the agile community, but it's seldom described so clearly.

[ 7 ] Crystal Clear, Dr. Alistair Cockburn.

Page History: Created 14 Feb 05,

last updated 11Jan 06