Let’s start this post with a thought experiment.  Not in software development, but in playing chess.

Imagine two novice chess players, working as a team. (We’ll assume their opponent is a computer, so it can’t overhear them talk.)  Our two novices will benefit greatly from their collaboration.  They’ll discuss all their thinking – everything from possible moves, to correcting each others mistakes, to “can you remind me how a knight moves?”.  Working together like this they will make fewer mistakes, and generate better moves than each would have done alone.

But what if we paired two chess masters?  Scientific research proves that the thinking of experts is different. Much of it happens as automatic pattern recognition rather than conscious reasoning.  When a chess master looks at a chess position, (good) possible moves spring to mind immediately.  As described in my previous post, these “candidate moves” are generated directly by the brain’s underlying neural network.  Neural networks can’t explain their own reasoning, so the master doesn’t consciously know why those moves came to mind.  Of course, the moves are indeed based on the master’s long experience, but the mapping from prior experience to current ideas is not open to inspection by the conscious mind.

So how will our two chess masters collaborate?  Neither can explain to the other why they thought of particular moves – because the conscious, effortful part of the mind doesn’t actually know. So what will their conversation consist of?

• They can’t discuss the justification for their proposed moves, because as we just saw, they don’t know the justification.  (Several writers, including Nobel-winner Daniel Kahneman, point out that if you ask an expert, “Why did you suggest that?”, their effortful mind will indeed come up with a justification of the idea, but the justification is produced after the fact. It doesn’t necessarily match the invisible reasoning that originally created the idea.  In fact, I’d suggest that since the invisible reasoning consisted of pattern matching in a neural network, any attempt at a logical sequential description would be incomplete at best.)
• Instead, imagine that they pool their sets of candidate moves, and then discuss which of those moves to actually play.  This approach makes sense because, as we saw in the previous post, selecting one of the candidate moves requires effortful thought.  And effortful thoughts can be perceived and described by the thinker.  So let’s assume our two chess masters try to share these thoughts.  If they do, I believe they’ll run into another problem: verbalizing the thought process slows it down – a lot.  A master thinks through many possibilities: What would happen if I play this move?  How would my opponent probably respond?  How would I respond to that?  A master can process these “what ifs” relatively quickly.  Not nearly as quickly as the automatic thinking that produced the initial set of candidate moves, but much, much faster than the rate of human speech.   To share thoughts verbally, the master must slow down to speaking speed.  In summary, although it’s possible to verbalize these thoughts, there’s a high (performance) cost in doing so.

Pair Programming

Now let’s consider the analogy to pair programming. Novice programmers don’t yet make significant used of automatic pattern recognition, instead they rely more heavily on effortful thought.   Therefore their thoughts are open to introspection and can be verbally shared.  Experts, on the other hand, make significant use of the “automatic” part of their mind.  So most of their thought processes are not open to conscious introspection, and the remainder are so fast that verbalization carries a huge performance cost.

I think this may explain the confusion and frustration that some experienced programmers, including myself, feel when we’re asked to pair.  We can’t see how to map our automatic thought processes into the driver/navigator model of pair programming.  Furthermore we probably can’t explain what the problem is – at least, I know I couldn’t, until I read Kahneman’s effortful/automatic description of the brain.  His book points out that because automatic thought is, well, automatic, we lack awareness of its role in our thinking.  Since we lack awareness, we struggle to explain the difficulty we have with pairing. I hope that greater awareness of Kahneman’s work will give us a suitable vocabulary to describe the problem.

I also hope to stimulate discussion.  The thought processes of an expert, which elegantly combine the automatic with the logical, are extremely efficient. I believe pairing undermines this efficiency – by leading coders to create after-the-fact justifications of their automatic intuitions, and forcing them to unpack their conscious thoughts into spoken language.  I suspect paired experts may even find themselves forced back into novice-like patterns of thought.

What do you think, does pairing prevent experts from performing at their best?

Update, 31 Jan: I should clarify what I mean by “expert”.  I use the term in the sense used by Daniel Kanheman, Anders Ericsson and other researches into expertise.  But you might have seem some pair programming studies use the term “expert” in a different way, particularly those studies where all test subjects were students.  In those studies the word “expert” simply means “good performer” – a student who gets unusually good grades.  These cannot be experts in the Kahneman/Ericsson sense of term.  Developing that kind of expertise takes many years, far longer than any university degree.  Furthermore, Kahneman/Ericsson expertise is not simply about possessing knowledge, in the manner of an A student; instead it is about a mode of thought in which much of the knowledge is possessed and processed unconsciously, through automatic pattern-recognition honed by long experience.

Update, 15 July: Arlo Belshee replied with a very informative comment, below.  He addresses the concerns of this post as follows:

Pairings of experts can work very well. But they have to stop explaining why to each other, and just state partial what. Assume the other expert gets the why, or can create his own. Don’t slow down the thinking to talk; shorten the talking to fit within the fast cycles and focus on the part that will challenge the other guy’s thinking.  [emphasis added]