Compliance to Spec Considered Unprofessional

In keeping with the theme of this blog, which is “the neglected essentials of software development”, I’d like to share something I’ve learned recently.  It’s about how Professional people in other fields think – people like architects, town planners and doctors.  As they work, they engage in an on-going

…conversation with the situation.

What a lovely phrase.  It comes from the classic book The Reflective Practitioner: How Professionals Think In Action by Donald Schön. Schön was an influential professor at MIT.  His book was published in 1983, so none of his case studies come from IT, and yet he finds something that agile software developers will recognise.  In all the professions he studied, professionals relied heavily on feedback and reflection (i.e. being observant and thoughtful, and changing your mind as necessary).  He found that this approach is not just the normal way for professionals to work – it’s also the best way.

Here are some favourite quotes from the book:

At the same time that the enquirer [i.e. professional] tries to shape the situation…, he must hold himself open to the situation’s back-talk. He must be willing to enter into new confusions and uncertainties.  Hence, he must adopt a kind of double vision.  He must act in accordance with the view he has adopted, but he must realize that he an always break it open [change it] later, indeed must break it open later in order to make new sense of his transaction with the situation.

… he recognizes that the situation, having a life of its own distinct from his intentions, may foil his [plans] and reveal new meanings.

Implications

Schön’s work sheds light on some key issues in software development:

Issue 1 – BDUF

Firstly, consider the agile aversion to Big Design Up Front (BDUF).  BDUF is not an on-going conversation with the situation.  It’s just a period of thinking, followed by an attempt to give the situation a lecture!  That doesn’t work in any other professions.

Issue 2 – Handing over “designs”

Secondly, there are implications for passing work from one person to another.  Some years ago, as a software architect, I had difficulty working with a more junior developer.  Our interactions just didn’t seem to be successful.  I now realise what was going on.  He thought I was handing over a description of what he should build; but I thought I was handing over a conversation with the situation.

It’s difficult to hand over an in-flight conversation, for someone else to continue in your absence.  And yet, when a senior hands over a “design” to a junior, that is exactly what’s happening.  The senior is stepping back from the conversation with the situation, and expecting the junior to continue it.

Here’s this conversation I’ve been having with the problem. Here’s a rough outline of what I said and how the situation replied.  So I think we might proceed as follows…<outline of suggested solution goes here>.  But, don’t just take my word for it.  Continue the conversation, thinking and altering course when necessary, as I would have done if I had continued the conversation myself. Finally, if you need to change course, but don’t know how, ask me and we’ll figure it out together.”

That’s the verbose way to say it.  Unfortunately, we often cut corners and say, “Here, build this”.  Junior programmers don’t know that, “Here, build this”, really means , “Please continue my conversation.”  In fact lots of senior programmers don’t know that either – I know I didn’t.  It was only when I read Shön’s book that I finally had the mental tools to really understand the situation.

Issue 3 – “BA” or “Designer”

Finally, the conversation metaphor tells us about the role of business analysts, on both traditional and agile projects.  At the recent  #10yrsagile event in Utah, Jeff Patton and others commented on the difference between “BAs” and “designers”. The “designer” model directly supports “conversations with the situation”, since the person in the designer role clearly undertakes such dialog with the situation.  But in the BA model, if interaction between the BA and developer is either (i) infrequent, (ii) asynchronous or (iii) one-way, then the conversation with the situation breaks down.

Example

Here is a detailed real example, of a conversation with the situation while coding.

Support from Other Authors

Several other authors have shared similar findings.  For instance, in his book “The Design of Design” Fred Brooks quotes the following diagram (originally from Maher, Poon and Boulanger):

In this diagram, the problem and solution both shape each other, as we move left-to-right through time.  (I remember it as the “crocodile diagram”, since the zig-zags look like the teeth of a crudely-drawn crocodile.)

And, of course, no discussion of these topics would be complete without referring to the classic paper from Jack Reeves. Schön’s work adds weight to his arguments.

Conclusion

When we put all this together, we are led to a surprising conclusion regarding specifications (and their agile equivalents, such as acceptance tests or criteria that are prepared in advance of coding).

If you produce a system that exactly matches its specification, you have acted unprofessionally. Either, you didn’t maintain a conversation with the situation after you got the spec; or you kept on talking with the situation, but you ignored it’s advice whenever it disagreed with the spec.

Regrettably, we are all encouraged to make this mistake.  On traditional projects, there may be implicit or explicit pressure to minimise the number of change requests.  On agile projects, there may be pressure to keep the velocity up, and some developers may respond by doing just enough to meet the pre-written acceptance tests, and quite deliberately “not asking too many questions”.  In so doing, they rob the project of their natural talent for “looking around and taking initiative”.  Finally, on projects of all kinds, there is often insufficient time budgeted for the Validation (fitness to purpose) part of “Verification and Validation”.

I hope that, with increasing recognition of Schön’s work, will come the realisation that true professionals do not comply with a spec, they surpass it.