Samuli Heljo | Driving a Scrum team at the edge of chaos

The Blog

Driving a Scrum team at the edge of chaos

published December 12, 2010

Recently I have tried to understand chaos theory and complex adaptive systems and how those relate to Scrum teams. Especially, I am interested in the “edge of chaos” concept and how that applies to my work. There are sources like this and this, claiming that team of people are very creative at the edge of chaos. When I tried to understand what this really means, I had to start from the Agreement and Certainty Matrix.

The Agreement and Certainty Matrix
Also known as the “Stacey matrix”, is a management tool by Ralph Stacey for leaders to get guidance on how to manage people’s work depending on a problem’s complexity and team’s agreement levels.

Close to certainty

Issues or decisions are close to certainty when cause and effect linkages can be determined. This is usually the case when a very similar issue or decision has been made in the past. One can then extrapolate from past experience to predict the outcome of an action with a good degree of certainty.

Far from certainty

At the other end of the certainty continuum are decisions that are far from certainty. These situations are often unique or at least new to the decision makers. The cause and effect linkages are not clear. Extrapolating from past experience is not a good method to predict outcomes in the far from certainty range.

Agreement

The vertical axis measures the level of agreement about an issue or decision within the group, team or organization. As you would expect, the management or leadership function varies depending on the level of agreement surrounding an issue.

(source: http://www.gp-training.net/training/communication_skills/consultation/equipoise/complexity/stacey.htm)

In IT world, If we are building a new solution with a technology that is new to developers we are moving from left to right in the matrix. When team does not agree on how technology should be applied, movement is upwards in the matrix. So far this makes sense. If we have challenges on only one area, project tends to be complicated but when both areas are challenging, we are working with a complex problem. Scrum was especially designed to deal with complex problems, where plan-driven approach will not produce best results. In the real world there are no clear boundaries like: “Aha, we just moved into the complex zone, better get us some Scrum”. Even though plan-driven works on simple projects, I think that also they benefit from agile execution.

People are most creative at the edge of chaos
Is this really true? According to excellent book “Group genius“, author claims that a lone genius is a myth and many (if not all) great inventions are a result from collective creativity and cannot be traced back to only one individual. I bet you have seen this personally as well. One person says something, a second one adds a thought, and a third person rephrases the idea. Suddenly a new concept emerges, which is a result from teamwork thus making the team more than a sum of it’s parts. But must all this happen at the edge of chaos and what does that even mean?

At the edge of chaos
Group genius states that collective creativity is the source of great inventions and thus we will need a team to come up with most creative solutions. On the other hand, Stacey matrix states that technology must be challenging and people must disagree if we are working near chaos. This makes sense, as team that always agrees might suffer from groupthink. The scale of Stacey matrix must also be team and situation dependent, as “technologically challenging” depends on the skill level of a team.

Joseph Campbell defines edge of chaos like this:

the edge of chaos is a place where there is enough innovation to keep a living system vibrant, and enough stability to keep it from collapsing into anarchy.

Michael Crighton (In the lost world) like this:

It [the edge of chaos] is a zone of conflict and upheaval, where the old and the new are constantly at war. Finding the balance point must be a delicate matter – if a living system drifts too close, it risks falling over into incoherence and dissolution; but if the system moves too far away from the edge, it becomes rigid, frozen, totalitarian. Both conditions lead to extinction. Too much change is as destructive as too little. Only at the edge of chaos can complex systems flourish. . . And, by implication, extinction is the inevitable result of one or the other strategy – too much change, or too little

If we define a state in which we will not make any progress because “everyone is doing their own thing” as “chaos” then we might have us a setting that can describe “the edge of chaos” in practice. To find out the actual border between chaos and the edge, we must face a situation where we feel that the work is not progressing and stagnation happens. When the team begins to stall because of the disorder, we know that chaos state is reached and some order must be restored to regain momentum. Are you with me?

How to increase chaos in a team
Why would you want to do that? To increase team’s creativity and to energize it to move away from it’s comfort zone.

Come up with a new point of view and encourage team to discuss why current solution will (or will not) work.
Facilitate discussions so that also introverts are heard. More insights more chaos.
Ask team how their solution will work in future and are they taking product vision from all angles into discussion.
Ask if their solution could be more generalized.
Focus team on quality and encourage them to avoid one-off solutions.

I do not mean that you should try to guess future requirements and build features that won’t be needed. Purpose is to gain new viewpoints and to twist the problem.

How to decrease chaos in a team
When stagnation happens and chaos is reached, disorder should be decreased so that work can continue. Some ways for a team to decrease disorder are:

Decrease democracy and select one person to come up with a solution that will work as a starting point.
Take a small vertical slice and implement a proof of concept.
Stop discussing and get to work. Once you have something to work with, start discussions again.
Concentrate on one subsystem in order to reduce complexity on technology axis.

Communication at the edge of chaos
It may take a while for a team to really understand how others perceive a problem and to come up with a potential solution. Without some way to communicate progress team will easily slip into the chaos. I like to visualize things, so I draw. I mean draw like crazy. Class diagrams, domain models, architecture diagrams, story boards, you name it. The purpose of these visualizations is to help team to understand what it has accomplished, give it a shared canvas for discussing about solutions, and to communicate social representations. Biggest value for diagrams is just after drawing and they are not final project documentation as coded solution will most likely be somewhat different from the models.

Some concept models from our current work.

Productivity, creativity, and stress levels
If I have understood the very edge of chaos correctly, it is not very productive state in terms of end user value. Creative yes, productive no. When team is working to gain understanding of a complex issue, it inevitably uses more time to communication and change than when working on straightforward tasks. I feel that actual productivity increase is reached afterwards when we have understood the problem, made a consensus, and pulled out to less complex zone to do actual work. To enter “hyper productive” state, team will have to work like a pendulum, balancing around the edge of chaos. Working near chaos can be quite stressful for a team and it is important to watch how it copes with the stress. It is not a state where I would like to keep team too long at one time, because it will start to wear members down. However, amount of stress at the edge of chaos will likely depend on people’s thinking styles and others are more susceptible to it.

I will have to give more thoughts to this. If you know any good links or have real life experiences to share, I am very interested.

Samuli @ 12:32 (No Comments)

tags: Scrum, Social psychology