Imagine two novice chess players, working as a team. (We’ll assume their opponent is a computer, so it can’t overhear them talk.)  Our two novices will benefit greatly from their collaboration.  They’ll discuss all their thinking – everything from possible moves, to correcting each others mistakes, to “can you remind me how a knight moves?”.  Working together like this they will make fewer mistakes, and generate better moves than each would have done alone.

But what if we paired two chess masters?  Scientific research proves that the thinking of experts is different. Much of it happens as automatic pattern recognition rather than conscious reasoning.  When a chess master looks at a chess position, (good) possible moves spring to mind immediately.  As described in my previous post, these “candidate moves” are generated directly by the brain’s underlying neural network.  Neural networks can’t explain their own reasoning, so the master doesn’t consciously know why those moves came to mind.  Of course, the moves are indeed based on the master’s long experience, but the mapping from prior experience to current ideas is not open to inspection by the conscious mind.

So how will our two chess masters collaborate?  Neither can explain to the other why they thought of particular moves – because the conscious, effortful part of the mind doesn’t actually know. So what will their conversation consist of?

• They can’t discuss the justification for their proposed moves, because as we just saw, they don’t know the justification.  (Several writers, including Nobel-winner Daniel Kahneman, point out that if you ask an expert, “Why did you suggest that?”, their effortful mind will indeed come up with a justification of the idea, but the justification is produced after the fact. It doesn’t necessarily match the invisible reasoning that originally created the idea.  In fact, I’d suggest that since the invisible reasoning consisted of pattern matching in a neural network, any attempt at a logical sequential description would be incomplete at best.)
• Instead, imagine that they pool their sets of candidate moves, and then discuss which of those moves to actually play.  This approach makes sense because, as we saw in the previous post, selecting one of the candidate moves requires effortful thought.  And effortful thoughts can be perceived and described by the thinker.  So let’s assume our two chess masters try to share these thoughts.  If they do, I believe they’ll run into another problem: verbalizing the thought process slows it down – a lot.  A master thinks through many possibilities: What would happen if I play this move?  How would my opponent probably respond?  How would I respond to that?  A master can process these “what ifs” relatively quickly.  Not nearly as quickly as the automatic thinking that produced the initial set of candidate moves, but much, much faster than the rate of human speech.   To share thoughts verbally, the master must slow down to speaking speed.  In summary, although it’s possible to verbalize these thoughts, there’s a high (performance) cost in doing so.

Pair Programming

Now let’s consider the analogy to pair programming. Novice programmers don’t yet make significant used of automatic pattern recognition, instead they rely more heavily on effortful thought.   Therefore their thoughts are open to introspection and can be verbally shared.  Experts, on the other hand, make significant use of the “automatic” part of their mind.  So most of their thought processes are not open to conscious introspection, and the remainder are so fast that verbalization carries a huge performance cost.

I think this may explain the confusion and frustration that some experienced programmers, including myself, feel when we’re asked to pair.  We can’t see how to map our automatic thought processes into the driver/navigator model of pair programming.  Furthermore we probably can’t explain what the problem is – at least, I know I couldn’t, until I read Kahneman’s effortful/automatic description of the brain.  His book points out that because automatic thought is, well, automatic, we lack awareness of its role in our thinking.  Since we lack awareness, we struggle to explain the difficulty we have with pairing. I hope that greater awareness of Kahneman’s work will give us a suitable vocabulary to describe the problem.

I also hope to stimulate discussion.  The thought processes of an expert, which elegantly combine the automatic with the logical, are extremely efficient. I believe pairing undermines this efficiency – by leading coders to create after-the-fact justifications of their automatic intuitions, and forcing them to unpack their conscious thoughts into spoken language.  I suspect paired experts may even find themselves forced back into novice-like patterns of thought.

What do you think, does pairing prevent experts from performing at their best?

Update, 31 Jan: I should clarify what I mean by “expert”.  I use the term in the sense used by Daniel Kanheman, Anders Ericsson and other researches into expertise.  But you might have seem some pair programming studies use the term “expert” in a different way, particularly those studies where all test subjects were students.  In those studies the word “expert” simply means “good performer” – a student who gets unusually good grades.  These cannot be experts in the Kahneman/Ericsson sense of term.  Developing that kind of expertise takes many years, far longer than any university degree.  Furthermore, Kahneman/Ericsson expertise is not simply about possessing knowledge, in the manner of an A student; instead it is about a mode of thought in which much of the knowledge is possessed and processed unconsciously, through automatic pattern-recognition honed by long experience.

…conversation with the situation.

What a lovely phrase.  It comes from the classic book The Reflective Practitioner: How Professionals Think In Action by Donald Schön. Schön was an influential professor at MIT.  His book was published in 1983, so none of his case studies come from IT, and yet he finds something that agile software developers will recognise.  In all the professions he studied, professionals relied heavily on feedback and reflection (i.e. being observant and thoughtful, and changing your mind as necessary).  He found that this approach is not just the normal way for professionals to work – it’s also the best way.

Here are some favourite quotes from the book:

At the same time that the enquirer [i.e. professional] tries to shape the situation…, he must hold himself open to the situation’s back-talk. He must be willing to enter into new confusions and uncertainties.  Hence, he must adopt a kind of double vision.  He must act in accordance with the view he has adopted, but he must realize that he an always break it open [change it] later, indeed must break it open later in order to make new sense of his transaction with the situation.

… he recognizes that the situation, having a life of its own distinct from his intentions, may foil his [plans] and reveal new meanings.

Implications

Schön’s work sheds light on some key issues in software development:

Issue 1 – BDUF

Firstly, consider the agile aversion to Big Design Up Front (BDUF).  BDUF is not an on-going conversation with the situation.  It’s just a period of thinking, followed by an attempt to give the situation a lecture!  That doesn’t work in any other professions.

Issue 2 – Handing over “designs”

Secondly, there are implications for passing work from one person to another.  Some years ago, as a software architect, I had difficulty working with a more junior developer.  Our interactions just didn’t seem to be successful.  I now realise what was going on.  He thought I was handing over a description of what he should build; but I thought I was handing over a conversation with the situation.

It’s difficult to hand over an in-flight conversation, for someone else to continue in your absence.  And yet, when a senior hands over a “design” to a junior, that is exactly what’s happening.  The senior is stepping back from the conversation with the situation, and expecting the junior to continue it.

Here’s this conversation I’ve been having with the problem. Here’s a rough outline of what I said and how the situation replied.  So I think we might proceed as follows…<outline of suggested solution goes here>.  But, don’t just take my word for it.  Continue the conversation, thinking and altering course when necessary, as I would have done if I had continued the conversation myself. Finally, if you need to change course, but don’t know how, ask me and we’ll figure it out together.”

That’s the verbose way to say it.  Unfortunately, we often cut corners and say, “Here, build this”.  Junior programmers don’t know that, “Here, build this”, really means , “Please continue my conversation.”  In fact lots of senior programmers don’t know that either – I know I didn’t.  It was only when I read Shön’s book that I finally had the mental tools to really understand the situation.

Issue 3 – “BA” or “Designer”

Finally, the conversation metaphor tells us about the role of business analysts, on both traditional and agile projects.  At the recent  #10yrsagile event in Utah, Jeff Patton and others commented on the difference between “BAs” and “designers”. The “designer” model directly supports “conversations with the situation”, since the person in the designer role clearly undertakes such dialog with the situation.  But in the BA model, if interaction between the BA and developer is either (i) infrequent, (ii) asynchronous or (iii) one-way, then the conversation with the situation breaks down.

Example

Here is a detailed real example, of a conversation with the situation while coding.

Support from Other Authors

Several other authors have shared similar findings.  For instance, in his book “The Design of Design” Fred Brooks quotes the following diagram (originally from Maher, Poon and Boulanger):

In this diagram, the problem and solution both shape each other, as we move left-to-right through time.  (I remember it as the “crocodile diagram”, since the zig-zags look like the teeth of a crudely-drawn crocodile.)

And, of course, no discussion of these topics would be complete without referring to the classic paper from Jack Reeves. Schön’s work adds weight to his arguments.

Conclusion

When we put all this together, we are led to a surprising conclusion regarding specifications (and their agile equivalents, such as acceptance tests or criteria that are prepared in advance of coding).

Regrettably, we are all encouraged to make this mistake.  On traditional projects, there may be implicit or explicit pressure to minimise the number of change requests.  On agile projects, there may be pressure to keep the velocity up, and some developers may respond by doing just enough to meet the pre-written acceptance tests, and quite deliberately “not asking too many questions”.  In so doing, they rob the project of their natural talent for “looking around and taking initiative”.  Finally, on projects of all kinds, there is often insufficient time budgeted for the Validation (fitness to purpose) part of “Verification and Validation”.

I hope that, with increasing recognition of Schön’s work, will come the realisation that true professionals do not comply with a spec, they surpass it.

The event was both a celebration of the first 10 years of agile, and an attempt to answer three questions which Alistair circulated before the meeting:

1. What problems in software or product development have we solved (and therefore should not simply keep re-solving)?
2. What problems are fundamentally unsolvable (so therefore we should not keep trying to “solve” them)?
3. What problems can we sensibly address – problems that we can mitigate either with money, effort or innovation? (and therefore, these are the problems we should set our attention to, next.)

There was clearly much to celebrate from the past 10 years of agile, but when it came to the future, there seemed to be concerns in several areas: the slowness with which good practices are adopted in some quarters; the risk of losing momentum now that there is no longer a common “enemy” (as heavy process was, when agile began); and a perceived lack of strength, clarity and consensus in the group’s answer to Alistair’s third question – “what next?”

Reflecting on those concerns, here are my “ah-has” from my virtual participation:

Gradual Change is Normal (Part 1)

Mike Cohn wrote a great post likening agile to Object Orientation.  Eventually, OO was taken for granted, and people stopped talking about it.  He suggests that agile will go the same way, and I agree.

However, these things can take a really, really long time. Here’s an example, that extends Mike’s OO one: By the late 90’s, OO had blossomed so much that the premier OO conference, OOPSLA, moved away from its original goal of making OO practical and understandable.  OOPSLA’s work was done.  But, at the turn of the century, there were still a lot of people who weren’t using OO.  Legions of VB6 developers making business apps on the Microsoft platform were not using OO at all.  Slowly, over the decade that followed, OO seeped into the Microsoft development community.  Languages were replaced and improved, and starting in about 2007 we saw wide adoption of Object-Relational Mapping and a variant of Smalltalk’s decades-old Model-View-Presenter pattern.  (Yes, Microsoft played a role in the timing, but so did changing sentiment within the developer community.)

There are two key things to note from this example:

• We are currently seeing (real) OO go mainstream in the Microsoft community –  about a decade after it went mainstream in most other parts of the development community.
• This happened without any industry association driving it.   There was no equivalent to the Agile Alliance, cajoling Microsoft developers into using Objects.  There was no need for such a body, because OO had become adequately embedded in the industry’s  consciousness back in the 90s.  It was like a virus that had infected enough  of the population that it could no longer be eliminated.  Slowly, over the decade that followed, it reached the rest.

So don’t sweat it. Agile isn’t going away.  It will continue to influence people, and to be embraced by new users, long after the early adopters have taken it for granted and stopped talking about it.

Gradual Change is Normal (Part 2)

In a post on his delightfully-named blog, Jonathan House wrote the following after the Snowbird event:

Does Agile really make a difference? – Not so much a question that showed up at the conference, but one that kept running around in my head, kicking over garbage cans and spray-painting the cat. … It’s clear we’ve made excellent progress over the past 10 years, but it’s still not so clear to me why businesses aren’t beating down the door to really adopt Agile throughout the enterprise …

Jonathan, I’m fairly sure this is not a reflection on agile.  It is reflection on business.  Business leaders often fail to follow advice, even when they actually agree that the advice is good.  I don’t say this out of personal dissatisfaction; it is a well-researched fact.  There’s even a book on the subject, called The Knowing-Doing Gap by Jeff Pfeffer and Bob Sutton.  (I mentioned Pfeffer and Sutton in my Agile Roots talk.  As an agilist and geek, I find their work extremely credible and relevant).

I am convinced that you are observing not a failure in agile, but a significant problem in business in general.

(Having said that, it wouldn’t hurt if we in the agile community could familiarize ourselves with how the more self-aware sections of business community do already understand agile, just with different words and under different names.  It will be much more polite, and therefore more likely to succeed, if we listen before we lecture).

Diversity is Good

I think some people who were following the Snowbird event may have been disappointed at an apparent lack of consensus.  I know I was, at first.  But then I remembered that diversity is good.  Think of the importance of genetic diversity in populations of animals; remember how the messy diversity of capitalism out-performed Soviet central planning; and think back to the diversity of the original 17 signatories to the Agile Manifesto – a diversity which was alive and well both before and after they signed the Manifesto.

By having a range of people, with differing interests and priorities, the agile community is much more likely to successfully address the many and varied challenges of the next decade.

I think the point with diversity is not just to accept it, but to actually encourage it.  Good agile, particularly by experienced practitioners, will be situationally specific.  I suspect this point has not fully seeped into the overall consciousness of the wider agile community.  Perhaps one of the tasks for the next 10 years is to promote the acceptance of “situationally specific agile”, and to help people learn (i.e. practice) how to do it.

There is Plenty to Do

Several people pointed out that agile lacks a clear “enemy” now.  When agile started, heavyweight process was the enemy. But today there is no clearly-identifiable external enemy.  So what is the point of the agile movement now?

David Anderson seemed to reflect the mood of the group when he wrote:

The mission now is incremental improvement. It’s evolution, education and improving levels of maturity, rather than a revolution. The enemy is now within. The enemy is as Joshua Kerievsky put it “all the crap I see out there” despite 10 years of Agile methods.

David Anderson went on to write

I don’t believe the Agile movement knows how to operate without something to revolt against. Agile came, it served its purpose, it had a positive effect. What next? Perhaps it is time to move on?

Is the really nothing more to do?  Nothing more but combating the enemies within, of complacency and poor execution?  I can see the logic of what Joshua and David (and others) are saying, but please, please don’t let it be true!  There is so much more to do.  Here are just a few topics that come to mind:

• The spread of agile beyond software (as per Jonathan House’s comment above, and others on the day)
• Jeff Patton’s story mapping and feature thinning – two wonderful techniques which deserve to become much more prominent in agile’s second decade
• Advancement of knowledge/skill in Ri-level/situationally-specific agile
• Philippe Kruchten’s herd of elephants (elaborated/commented on here by Scott Ambler)
• Really paying attention to “Individuals and Interactions”.  For 10 years we’ve been getting this first point of the Agile Manifesto round the wrong way!!  So you can’t tell me there’s nothing left to do ;-)  James Coplien phrased it well in his outstanding reflection on the first 10 years: “We have test scripts and jUnit trumping individuals and interactions”.
• Sharing our failures as well as our successes, as several people mentioned on the day
• Contracts for agile projects – currently, only a partially-solved problem at best.

We’ve only been doing knowledge work in teams for a short part of human history.  The previous millennia did not prepare us well for the last few decades of complex, abstract, co-operation in non-trivial teams.  So of course this is still more to learn.

Summary

We need to keep thinking, keep talking and keep learning.

We need to keep agile’s sense of humanity – of valuing, respecting and caring for people in the workplace.

We need to retain and treasure the global and local communities of practitioners, which the agile movement has created.

We need to keep all these things, even though there is no longer a common enemy to unite us.  We need to learn to operate in an agile community that doesn’t just accept, but actively promotes, its own diversity.

I recently stumbled across a quote from the French poet Stéphane Mallarmé:

to define is to kill, to suggest is to create

Let us not define our future.  Let’s suggest it.

Here are some brief notes on the presentation.

Introduction

General background on Crystal Clear

Big Lessons that I’ve Learned from Crystal Clear

• Methodology-per-process.  How you can (and should) tune your process to the project at hand.
• Defusing debates about documentation, by recognising the two purposes of documents: supporting the current project; and supporting future maintenance programmers.  In my projects, I have found it best not to kill two birds with one stone.
• Shu-Ha-Ri, the stages of learning skills and how that applies to agile.  People at different stages have different needs for prescriptive process versus flexibility; and different comfort levels with finalizing process elements “later”.
• Simple = good.  The simplicity of Crystal Clear is backed by both theory and practice.  Its simplicity supports the design goals of Crystal, which are safety (delivering a successful project), efficiency (cost-effectiveness) and habitability (people are comfortable using Crystal).

Links to other resources

I mentioned various other resources during the presentation.  They included:

• Material on iteration lengths: why there is no real value in saying “my iteration is shorter than yours”.
• Collaboration: the Dance of Contribution.  A lengthy piece, with a lot to take in.  I skim read it the first time I saw it, but I got more out of it when I re-read it some months later.  My favourite bit: “No idea is rejected out of hand and no idea gets a free pass.”  I’m still learning how to apply it ;-)
• Are requirements a document or a database?  I found Eric Sink’s article explains this dilemma very well.  (As for his comments about agile in the same article, I think he’s touching on an issue which James Bach described well here. Although they may not phrase it this way themselves, I’d suggest that Eric and James are both “Ri-style” practitioners, objecting to the over-emphasis of “Shu-style” agile.  As we discussed today, both styles have their place.)
• I also mentioned Eric’s “Career Calculus” page, and suggested that the same approach applies to process.  The most important thing is not where you start; but how fast you improve.
• Ron Jeffries on documentation.  His recommendations are broadly similar to the Crystal ones except (a) he has a firm view that the tests are the requirements documentation (Crystal is more flexible about how requirements are noted down) (b) he implies that you might not need future facing documentation (I would assume that if your software was worth writing, then it’s probably worth maintaining, which means it usually is worthwhile to write a future-facing document)

That all looks very brief, but on the day it took 50 minutes – honest.

By the way, I did it as an “incremental presentation”.  After each of the four main points, I paused for questions.  The idea was that each main point was like an “iteration”, and pausing for questions was like “releasing” that iteration.  It seemed to work.  We had good discussion after most points, and interleaving questions with presentation made it easier to fit the timeslot. (If all questions are left to the end, how much time should you leave?  Will people ask lots of questions?  Will they ask none?)

If you were there (or you weren’t) please let me know if you have any questions that are not covered in these notes.  Either leave a comment below, or email me.

Update 6 March 08:

I gave the presentation again today.  Here are some of the BA-related resources that I mentioned:

Scott Ambler on:

• Agile analysis
• Thoughts on the role of BA’s in agile projects
• Artifacts on agile projects

A couple of interesting points came out in discussion afterwards.  They included the idea of “emphasising facillitation over artifact creation”, the concept of ubiquitous language, and the notion that in some cases asking the customer to describe what they need is like a bunch of blind men describing an elephant.  Everyone you ask says something different, and what the business really needs might be different again.  In cases like these (where the customer organisation is, for whatever reason, unable to speak with a single authoritative voice) my thoughts are: don’t just say “that’s the customer’s problem”, or “we can’t do agile in these circumstances”, but instead apply BA skills to help the customer to articulate their needs.

>Finally, here is the page on negotiation, which I mentioned at the end.

I’ve just read a fascinating article on where talent comes from, over on Freakonomics.com.  It outlines research into the key factors that develop talent.  The key factors are:

"setting specific goals, obtaining immediate feedback and concentrating as much on technique as on outcome". [emphasis added]

For instance, life-long learning comes naturally to surgeons because they get instant feedback on the effectiveness of their decisions.  But similar learning doesn’t come naturally to doctors in fields where there is a  long delay between decision and outcome.

The research finds that development of talent is heavily dependent on the practitioner receiving prompt feedback on their decisions.  The timeliness of the feedback matters. 

Waterfall processes lack timeliness of feedback.  A classic example would be an analyst or architect making decisions at the start of a waterfall project.  Many months may pass before they learn the result of their decisions.  (Sometimes they never learn the result because they move to another project before the first one is finished.  Such so-called "experience" is of little value.)

In agile development, decision makers get feedback quickly.  Timeboxed iterations give feedback in a few weeks; unit tests give feedback in minutes; and paired programmers give feedback in seconds.  Research suggests this feedback doesn’t just improve the program; it also improves the programmers.

“You can’t have the same team do the same project twice; a bunch of stuff changes the second time around. You can’t have 2 teams do the same project; it’s too hard to control all the variables, and it’s prohibitively expensive to try it in any case. The same team doing 2 different projects in a row isn’t an experiment either.

About the best you can do is gather statistical data across a lot of teams doing a lot of projects, and try to identify similarities, and perform some regressions, and hope you find some meaningful correlations.”

That’s exactly how Crystal Clear came about.  Alistair Cockburn spent a lot of time studying successful teams, looking for the similarities, then he documented them.  That’s not the kind of half-baked fad which Steve criticises, it’s about as scientific as it gets (in our field).  In fact, it was scientific enough to earn Alistair his Ph.D.  His doctoral dissertation is on-line and makes suprisingly easy reading.  (Or you can find a shorter outline here.) I encourage you to read it, to see how Alistair grappled with the very problems Steve describes. (Eventually, he solved them much more successfully than Steve would expect. He documented his results as Crystal Clear).

Steve makes some very valid criticisms of the Agile movement.   However, his assertion of a lack of scientific rigour is incorrect (in Alistair’s case, at least).  Alistair conducted exactly the kind of research which Steve calls for, without any pre-conceived notions of what the outcome should be (he started years before the agile movement was founded).  He came up with a simple, effective process – which happens to be agile.

Interestingly, Alistair’s research-based process is somewhat different from XP.  As I’ve said before, XP is a subset of agile, not a synonym.

For instance: if the customer says they want a detailed Gantt chart, that’s a position. (It’s a position which is not very compatible with most agile processes).  But their interest is in understanding the status of the project. A chart like this might meet their interest equally well, while simultaneously meeting your interest in agility.

Customers also have a legitimate interest in protecting themselves from large cost increases and from large scope cuts.  Many agile teams unnecessarily adopt an incompatible position.

This is why its so important to understand that Agile is not just XP.  You have a much wider toolbox to draw on than just the practices in Extreme Programming.  Be prepared to draw on the whole toolbox, and your own creativity, to find the process that best suits your customer.  And be prepared to not use certain tools on certain projects.  For instance, this XP team (pdf) did not use pair programming; they replaced it with an alternative.

As agile developers, we know that good software is about customer-valued features. So is good process.

By the way, if XP is the only form of agile that you know, this page probably won’t make any sense. Am I really saying that TDD is not part of the core definition of agile? Yes I am. Look for TDD in the Agile Manifesto. It’s not there. While TDD is a good thing, and I like it, it’s not the core essence of agility. You’re welcome to define XP in prescriptive terms, mandating pairing and TDD, but those same prescriptive terms do not describe the agile movement as a whole. It is, after all, about “people and their interactions; over processes and tools“.

Defining agility by describing XP is like defining democracy by describing America. Such a “definition” obscures the underlying concept with details of the chosen example.

Agility is not defined by TDD, just as democracy is not defined by having a President. Democracy is defined as “choosing leaders in competitive elections”. It makes no difference whether you have a federal system or not; whether the executive is or isn’t separate from the legislature; or which method you use to count the votes – it’s still democracy.

Do we have a similar definition of agile development? One which is uncorrupted by specific examples and which defines the core underlying concept? In many respects, we don’t. The agile manifesto is the founding document of the agile movement (and the source of the term “agile development”). It’s a great definition of what agile developers think; but it doesn’t concisely define what they do. In other words, it talks about values and principles but not about practices.

Here’s my soundbite-size definition of agile:

The agile movement asserts that software development works best under:
Simple, iterative processes, which empahsise creativity and collaboration.

Other definitions include this one from Scott Ambler and two really nice ones from Alistair Cockburn, here.

Notes:

I use the word “creativity” to mean “inventiveness” – i.e. “producing solutions to difficult problems” (not just artistic ones). The most difficult problems are those involving (i) lack of clarity of the situation [e.g. requirements], (ii) multiple goals, (iii) complexity and (iv) time considerations – sounds like software!

The Crystal Clear book, by Alistair Cockburn, is now available.  The preface gives an excellent overview and this sample chapter is also available on line.  (You’ll have to buy the book to read the rest :-)

Briefly

Crystal Clear is an agile methodology for projects with small teams, less than about 10 people in size.

Team size is important, because if you have a small team it’s easiest to adopt a methodology that already fits.  It is more difficult to start with a larger one and prune it back.

Why Crystal Clear?

Several aspects of Crystal Clear impress me:

• It’s based on observations of many successful teams.  You can read the surprisingly entertaining story of Alistair’s observations here. This background gives Crystal a sounder base than most competing methodologies, which typically trace their roots back to a smaller number of projects. For example, XP was “born” on the C3 project.
• It has a different slant on things.  We’ve come to expect a certain “mechanical” feel to processes, “Do A, B and C, and the result will be D, where D is great software!”.  We’ve come to expect methodologies to contain details such as diagramming notations and workflows.  And yet, somehow, that’s not what matters most.  Crystal Clear describes the things that really do matter, the things that make the most difference to the success or failure of a project.
• It supports fixed price contracts.  Whether you love them or hate them, you probably use them.  Crystal Clear is flexible enough to suit.
• It’s not “all or nothing”.  Crystal Clear will be useful to the people who adopt it, but it will also be useful to people who don’t adopt it.  At least, not all of it. Teams using RUP, XP or other methodologies can “borrow” useful techniques from Crystal Clear.  Likewise, if you are doing Crystal Clear, you can borrow good bits from other methodologies.
• It gives you useful guidance on how to adopt the methodology, and how to tailor it to your organisation.  I find this a stark contrast with RUP, which says “tailor me”, but doesn’t seem to give enough guidance on exactly how to do so.
• It “feels right”, because it has a lot in common with the successful projects I’ve worked on in the past.  This is not surprising, considering it’s based on observations of successful projects!
• Crystal Clear is just a book (and a growing community of users).  If you want to adopt Crystal Clear, it will only cost you the price of a book, plus a little of your time (days, rather than months).

More Information

While the book is the best resource, you may also like to see:

• The author’s site (See this page in particular.  It describes how Crystal Clear relates to other “Crystal” methodologies, which are for larger teams)
• Crystal Clear discussion on Wiki Wiki web
• My “Methodology Map”
• Martin Fowler’s comments about Crystal

Summary

While no methodology is a magic formula for success, if you’re involved in small-team software development, you should read this book.  Crystal Clear is not mutually exclusive with other methodologies, so you’ll find its insights valuable no matter how your team is organised.

Amazon’s page for the book is here.

Iterative vs Waterfall

T: Serial (“waterfall”) development is best (or at least normal/acceptable)
A: Always use iterative development

Workflow

T: Design, Build and Test follow each other, in that order
A: Design, Build and Test influence each other, so they should be concurrent and iterative

Requirements Quality

To ensure requirements are well-defined and well-understood:

T: document them thoroughly; users see the system when finished
A: document them concisely; users see the system early and often

Knowledge Transfer

To ensure everyone understands the system:

T: Place importance on written documentation
A: Place importance on frequent verbal communication; regular working releases; and clear, readable source code

Team Composition

T: Team members should be specialists (BA, Designer, Developer, Tester etc)
A: Team members should be generalists (each with their own particular strengths)

Team Size

A successful process is one that:

T: Organises a large team
A: Gets the same output from a small team

Planning

T: Make a Gantt Chart at the start; maintain it during the project
A: Make a Task/Feature List at the start; maintain it during the project

Monitoring

Measure progress:

T: against the Plan
A: against the Task/Feature List (tracking which ones are completed). But don’t just monitor your progress; also monitor the quality of that progress by testing and releasing regularly.

Managing Change

It’s hard to change software that’s already written, so:

T: Control or minimise change
A: Make change easier

Predicting Completion

Predict final completion date and cost using:

T: the updated Gantt Chart
A: the team’s progress to date

Process Definition

To help people successfully follow it, the development process should be:

T: thoroughly documented
A: concise and memorable

Process Improvement

Process improvement is best driven by:

T: metrics
A: the people involved

Philosophy

T: Software development is a construction process, so making detailed plans is advisable.
A: Software development is a creative process, so making detailed plans is unrealistic.

Instead, agile processes make less-detailed plans and manage risk through iteration, continuous quality control, and feedback.

Success is

T: Meeting the initial predictions of cost and schedule
A: Delivering value for money