10 Years Agile – a perspective from outside the room

Recently Alistair Cockburn invited 32 agile practitioners to join him at the Snowbird ski resort, for a retrospective on the agile movement as a whole.  I followed the event via Twitter, sitting under a sun umbrella here in the southern hemisphere, reading the tweets from the snow.

The event was both a celebration of the first 10 years of agile, and an attempt to answer three questions which Alistair circulated before the meeting:

  1. What problems in software or product development have we solved (and therefore should not simply keep re-solving)?
  2. What problems are fundamentally unsolvable (so therefore we should not keep trying to “solve” them)?
  3. What problems can we sensibly address – problems that we can mitigate either with money, effort or innovation? (and therefore, these are the problems we should set our attention to, next.)

There was clearly much to celebrate from the past 10 years of agile, but when it came to the future, there seemed to be concerns in several areas: the slowness with which good practices are adopted in some quarters; the risk of losing momentum now that there is no longer a common “enemy” (as heavy process was, when agile began); and a perceived lack of strength, clarity and consensus in the group’s answer to Alistair’s third question – “what next?”

Reflecting on those concerns, here are my “ah-has” from my virtual participation:

Gradual Change is Normal (Part 1)

Mike Cohn wrote a great post likening agile to Object Orientation.  Eventually, OO was taken for granted, and people stopped talking about it.  He suggests that agile will go the same way, and I agree.

However, these things can take a really, really long time. Here’s an example, that extends Mike’s OO one: By the late 90’s, OO had blossomed so much that the premier OO conference, OOPSLA, moved away from its original goal of making OO practical and understandable.  OOPSLA’s work was done.  But, at the turn of the century, there were still a lot of people who weren’t using OO.  Legions of VB6 developers making business apps on the Microsoft platform were not using OO at all.  Slowly, over the decade that followed, OO seeped into the Microsoft development community.  Languages were replaced and improved, and starting in about 2007 we saw wide adoption of Object-Relational Mapping and a variant of Smalltalk’s decades-old Model-View-Presenter pattern.  (Yes, Microsoft played a role in the timing, but so did changing sentiment within the developer community.)

There are two key things to note from this example:

  • We are currently seeing (real) OO go mainstream in the Microsoft community –  about a decade after it went mainstream in most other parts of the development community.
  • This happened without any industry association driving it.   There was no equivalent to the Agile Alliance, cajoling Microsoft developers into using Objects.  There was no need for such a body, because OO had become adequately embedded in the industry’s  consciousness back in the 90s.  It was like a virus that had infected enough  of the population that it could no longer be eliminated.  Slowly, over the decade that followed, it reached the rest.

So don’t sweat it. Agile isn’t going away.  It will continue to influence people, and to be embraced by new users, long after the early adopters have taken it for granted and stopped talking about it.

Gradual Change is Normal (Part 2)

In a post on his delightfully-named blog, Jonathan House wrote the following after the Snowbird event:

Does Agile really make a difference? – Not so much a question that showed up at the conference, but one that kept running around in my head, kicking over garbage cans and spray-painting the cat. … It’s clear we’ve made excellent progress over the past 10 years, but it’s still not so clear to me why businesses aren’t beating down the door to really adopt Agile throughout the enterprise …

Jonathan, I’m fairly sure this is not a reflection on agile.  It is reflection on business.  Business leaders often fail to follow advice, even when they actually agree that the advice is good.  I don’t say this out of personal dissatisfaction; it is a well-researched fact.  There’s even a book on the subject, called The Knowing-Doing Gap by Jeff Pfeffer and Bob Sutton.  (I mentioned Pfeffer and Sutton in my Agile Roots talk.  As an agilist and geek, I find their work extremely credible and relevant).

I am convinced that you are observing not a failure in agile, but a significant problem in business in general.

(Having said that, it wouldn’t hurt if we in the agile community could familiarize ourselves with how the more self-aware sections of business community do already understand agile, just with different words and under different names.  It will be much more polite, and therefore more likely to succeed, if we listen before we lecture).

Diversity is Good

I think some people who were following the Snowbird event may have been disappointed at an apparent lack of consensus.  I know I was, at first.  But then I remembered that diversity is good.  Think of the importance of genetic diversity in populations of animals; remember how the messy diversity of capitalism out-performed Soviet central planning; and think back to the diversity of the original 17 signatories to the Agile Manifesto – a diversity which was alive and well both before and after they signed the Manifesto.

By having a range of people, with differing interests and priorities, the agile community is much more likely to successfully address the many and varied challenges of the next decade.

I think the point with diversity is not just to accept it, but to actually encourage it.  Good agile, particularly by experienced practitioners, will be situationally specific.  I suspect this point has not fully seeped into the overall consciousness of the wider agile community.  Perhaps one of the tasks for the next 10 years is to promote the acceptance of “situationally specific agile”, and to help people learn (i.e. practice) how to do it.

There is Plenty to Do

Several people pointed out that agile lacks a clear “enemy” now.  When agile started, heavyweight process was the enemy. But today there is no clearly-identifiable external enemy.  So what is the point of the agile movement now?

David Anderson seemed to reflect the mood of the group when he wrote:

The mission now is incremental improvement. It’s evolution, education and improving levels of maturity, rather than a revolution. The enemy is now within. The enemy is as Joshua Kerievsky put it “all the crap I see out there” despite 10 years of Agile methods.

David Anderson went on to write

I don’t believe the Agile movement knows how to operate without something to revolt against. Agile came, it served its purpose, it had a positive effect. What next? Perhaps it is time to move on?

Is the really nothing more to do?  Nothing more but combating the enemies within, of complacency and poor execution?  I can see the logic of what Joshua and David (and others) are saying, but please, please don’t let it be true!  There is so much more to do.  Here are just a few topics that come to mind:

  • The spread of agile beyond software (as per Jonathan House’s comment above, and others on the day)
  • Jeff Patton’s story mapping and feature thinning – two wonderful techniques which deserve to become much more prominent in agile’s second decade
  • Advancement of knowledge/skill in Ri-level/situationally-specific agile
  • Philippe Kruchten’s herd of elephants (elaborated/commented on here by Scott Ambler)
  • Really paying attention to “Individuals and Interactions”.  For 10 years we’ve been getting this first point of the Agile Manifesto round the wrong way!!  So you can’t tell me there’s nothing left to do ;-)  James Coplien phrased it well in his outstanding reflection on the first 10 years: “We have test scripts and jUnit trumping individuals and interactions”.
  • Sharing our failures as well as our successes, as several people mentioned on the day
  • Contracts for agile projects – currently, only a partially-solved problem at best.

We’ve only been doing knowledge work in teams for a short part of human history.  The previous millennia did not prepare us well for the last few decades of complex, abstract, co-operation in non-trivial teams.  So of course this is still more to learn.

Summary

We need to keep thinking, keep talking and keep learning.

We need to keep agile’s sense of humanity – of valuing, respecting and caring for people in the workplace.

We need to retain and treasure the global and local communities of practitioners, which the agile movement has created.

We need to keep all these things, even though there is no longer a common enemy to unite us.  We need to learn to operate in an agile community that doesn’t just accept, but actively promotes, its own diversity.

I recently stumbled across a quote from the French poet Stéphane Mallarmé:

to define is to kill, to suggest is to create

Let us not define our future.  Let’s suggest it.

Earned Value Starter Kit

At the end of my talk on Earned Value, I’ve been giving away the “Earned Value Starter Kit” that we put together at Optimation.  It is a fairly simple spreadsheet, which makes it easier to get started with ‘lite’ Earned Value. In case you miss out on a copy, you can download it here.

Thanks to the folks at work for making this freely available.

Rules of the Green Line

imageI draw usually draw Earned Value charts with the cost (AC) line in red, and the progress (EV) line in green. In this post, I’m going to outline some basic rules for getting a good “green line”.  The recommendations in this post are for people doing “lite” Earned Value as described in my posts and live demo.

The basic principle

When you’re part way through the project, you want the green line to give an accurate trend.  That way, you can used it for planning and making decisions.  If it was significantly inaccurate, then it wouldn’t be much use to you.

Actions

So, how do you make sure that it is as accurate as (practically) possible?  Here are suggestions that I’ve come up with, after about 5 years with this style of “lite” Earned Value.  These suggestions are specific to software development projects using agile(ish) processes.

  1. Aim for an approximately linear project structure.  In other words, try to have each month of the project containing approximately the same mix of design, build and test.  Don’t load all the design into the front of the project (waterfall style) because that undermines the predictive power of EV.  EV predictions work best when the nature of the work is roughly the same in all stages of the project – something that is (approximately) achievable with agile processes.
  2. Know your test strategy.  From an EV perspective, the nicest approach is to achieve the agile ideal of spreading the testing work uniformly throughout the project. But I recognise that some projects, to a lesser or greater degree, do need a User Acceptance Testing phase at the end.  That can be dangerous from an EV perspective, because it offers a temptation to consider things “done” even when their quality is not known.  In the worst case, this can get ugly: you think you’ve finished everything, and both your red and green lines are at 90-something percent.  But then you start UAT – which unearths lots of problems. As you fix the problems your cost keeps on rising, up and up way past 100%.  Ouch!  That’s why it’s much better to test as you go, if you can.  If you can’t, put aside a bucket of time and money to accommodate defect result ion during UAT.
  3. A task only counts towards the green line when it’s complete.  Partially complete tasks don’t count at all.  This is the most conservative approach and therefore, I strongly believe, the most realistic.  If, as discussed above, you must have a UAT phase at the end, you should still do at least some testing at the time that you develop the feature, so you know whether it’s “done”.
  4. The green line is always based on estimated task sizes.  You must NEVER tweak the Earned Value numbers for completed tasks to match how long they actually took.  That completely screws things up, because now your past is measured in “actuals” but your future is measured in “estimates” – so you lose all ability to predict the future from the past. To re-iterate, the green line must be based on estimated task sizes.
  5. If you add new scope during the project, make the estimated sizes for new tasks consistent with tasks already in the project.  E.g. say you are measuring task sizes in “points”, and you are thinking of adding a 10 point task. When you add it, do a little sanity check to see, “Is it really the same size as the 10 point tasks that we already have?”.  You want all “10 point tasks” to be roughly the same size, regardless of when you added them to the project.
  6. If you split a task, reallocate its original points to the sub-tasks. For instance if you divide an epic user story into several smaller ones, make sure that the total point value of the small stories equals the value of the original epic.  I.e. chopping up a task mustn’t change the total number of points in the project.
  7. Don’t worry about “derived requirements”, don’t even track them.  Derived requirements are those (hopefully) little things that are not stated in the planned scope, but which turn out to be essential to implementing it.  I often visualise them as little gaps in the stated requirements – implicitly and unavoidably part of the project, but not foreseen in our planning. For Earned Value purposes, the easiest way to handle these is to not track them at all.  Just implement them as necessary, without entering them in your Earned Value system/tool/spreadsheet. For a discussion of why this makes sense, see my STN and Encylopedia articles.

I hope you find these suggestions useful.  “Lite” approaches to Earned Value are still evolving, and are not yet well documented.  So, suggestions for improving this list are most welcome.

Programming Languages for Kids – time to go mobile

At the BarCamp following the NZCS 50th Anniversary Conference, there was an interesting discussion on programming languages for kids.  It was suggested that the key players ( Phrogram, Small Basic, Scratch etc)  don’t get it quite right.  That rings true to me.  Reflecting on this after the session, I realised the reason: When I was a kid, part of the attraction to programming was in the sense of mastering a technology that was new (i.e. new to the whole world, even adults, rather than merely “new to me”).  Phrogram, Small Basic and Scratch are languages for a technology that is no longer new – desktop computers.

Today, how can we offer kids the excitement of mastering a technology that even adults don’t understand, a sense of being part of something that is leading edge on a global scale?  By letting them program mobile devices, I suspect.  It’s not just about the mobility, but its also about the sensors and devices that are on those devices.  Imagine this:

A maths class is learning about trigonometry.  As a practical exercise, they are tasked with measuring the height of a tree using only their mobile phone.  With a little guidance (if necessary) they realise that the simply need to use use the device’s sensors to measure 3 things: (a) the position of the base of the tree, (b) the location of another point some distance away and (c) the angle from the second point to the top of the tree.  Items (a) and (b) can be measured with the GPS, and (c) with the accelerometer.  Tie them all together with a simple program, get it to do some simple trig, and hey presto, the cell phone is a mobile tree-measuring device.

To me, this seems like a far more motivating scenario than writing for desktop computers.  And, as several people implied in this morning’s discussion, perhaps motivation is the key to getting enough young people into IT.

This evening, I searched the net to find such a tool.  It took a while, but eventually I found App Inventor, which is a new product from Google.  It’s a bit like “Scratch for Android”.  It is still in restricted Beta, but I believe it has the potential to motivate today’s kids in the way 8-bit micros motivated my generation in the 80’s.

 

[Update 19 Sept: I’ve just heard that there have been some rather negative reviews/criticisms of App Inventor, although I get the impression that at least some of them relate to its immaturity/beta status.  Will be interesting to see how it matures.]

Encyclopedia Article

This page provides shortcuts to my Earned Value article in the Encyclopedia of Software Engineering.

  • Link to online copy. As of 27 June 2011, you can purchase just the EV article (without having to buy the whole encyclopedia.) 
  • Hard copy of the whole encyclopedia (about 120 articles on other topics, in addition to my one on Earned Value.  Each article is about 30 pages and all are peer reviewed by experts in the relevant field.)

I’m sorry but I cannot provide copies of the article here, since the copyright is held by the publisher.  By the way, I make no money from the sale of the article, so I hope you’ll consider me reasonably unbiased when I say that it’s well worth reading.  (Well, OK, about as unbiased as an author can be about his own work!).

To those who’ve seen my Earned Value talk with the animated interactive charts, this article does include some important details which I could not fit into the talk – in particular how to obtain objective "progress" numbers, and a discussion of how Earned Value relates to the critical path on software projects.

The article also forms a “bridge” between the purely graphical approach, which I use throughout my talk, and the numerical/mathematical approach which is the norm in the EV community.  As such, I believe it is one of the few pieces of writing on Earned Value which brings together all four of the following viewpoints:

  • Agile project management
  • "Traditional” (non-agile) project management
  • An intuitive graphical “feel” for the subject
  • The standard mathematical formulae of Earned value

I hope you find it to be balanced, reasonably comprehensive and, most  importantly of all, useful.

Recent Earned Value Posts of Interest

Glen Alleman posted on the difficulty (or otherwise) of the maths in EVM, and several of us commented.  He  followed that with an example in which the maths is very simple, which I quite liked as an introduction to Earned Value.

Marcin Niebudek wrote an article, in which he adds a budget line to an agile burn chart, measuring both in percentages.  Nice to see I’m not the only one doing that. (Even if he does draw the chart up the "wrong" way ;-)

Updates

June 2011: another interesting one from Glen Alleman, on the relationship between the ANSI standard criteria and simple/agile EV.

July 2011: A nice burn chart example from the folks at Atomic Object.  It doesn’t include the cost line, and I’d prefer if it did, but it does offer a clean solution to some of the challenges of charting change.

Speaking on Earned Value at NZ Computer Society Conference

I’ll be speaking at the New Zealand Computer Society’s 50th Anniversary Conference, on the topic of Earned Value Management.

I’m looking forward to being part of an interesting conference, and hopefully helping to lift the profile of EVM in New Zealand.

Link to presentation abstract

( The abstract’s reference to “Number 8 wire” may be unclear to overseas readers :-)  The phrase simply means “New Zealand ingenuity”, in this part of the world.)

Earned Value in a Nutshell

I’ve been looking for a way to describe the “essence” of Earned Value Management (EVM).  How can I describe the core of what EVM is about – without resorting to an impenetrable jungle of acronyms?

This is particularly important when describing it to people outside EVM’s traditional strongholds of defense and aerospace.  Outside those areas, EVM is under-utilised, and I suspect much of the reason is due to its apparent complexity.  I’ve been an EVM fan for about 5 years now, and I still come across unfamiliar acronyms.  If EVM is to be more widely used, it has to be presented in a way that is accessible to a wide audience.

Here’s what I came up with:
Continue reading “Earned Value in a Nutshell”

Added transcript to People Skills video

This week’s “People Skills for Nerds” post now includes a written transcript of the talk. (For those who, like me, prefer to read than to watch videos ;-)

People Skills for Nerds – a Summary Video

This video summarises they key points I aim to make in this blog, that people skills are:

  1. Important, and
  2. Learnable (with time and practice)

The video is an edited extract of my talk at the AgileRoots conference, at Salt Lake City in 2009.  For the benefit of people who, like me, prefer skim reading to watching videos, this post also includes a written transcript (below).

Continue reading “People Skills for Nerds – a Summary Video”