Wednesday, March 24, 2021

The only constant is change

A classic response towards change failures in traditional organizations, most notably, Software Release failures, is "Then make changes less often." This gives a good feeling that for a prolonged period of time, things will be predictable and stable. Unfortunately, it's a local optimization issue that actually makes things worse in the long term. Why do we think like that, and why is that thought a problem?



The gut reaction

Let's say that a change just failed. This is an uncomfortable event. Naturally, we don't like discomfort. The easiest way to deal with discomfort is to postpone the re-occurrence of the uncomfortable event into the future. In software development, we do this by postponing the next release as far as possible.

This provides an immediate short-term effect on our psyche: We have just achieved certainty that until the next release happens, there will be no further discomfort, and thus we have reduced immediate stress by postponing it.

Unfortunately, this has done nothing to reduce the probability of that next change failing.

Now, let's see what happens as a consequence of this decision:


The immediate after-effect

Because there is now a prolonged period of time until our next release happens, and the amount of development work is not decreasing, the batch size for the next release increases by exactly the amount of delay in the release. So, if, for example, we used to release once per month and reduce this to once per three months, the batch size just went up 200% with a single decision.

Of course, this means that the scope of the next change is also going to increase, it will become more complex to deliver the next release.

The knock-on effect

A bigger, more complex change is more difficult to conduct and has a higher potential for failure. In consequence, we're going to be failing stronger and harder. If we have 3 small changes, and 1 of them fails, that's a failure rate of 33%. If we now combine all 3 changes into 1 big change, that means we end up with a 100% failure rate!

You see - reducing change frequency without reducing change scope automatically increases likelihood and impact of failure.

If now we decide to follow our gut instinct again, postponing the next failure event, we end up in a vicious circle where change becomes a rare, unwanted, highly painful event: We have set the foundation for a static organization that is no longer able to adapt and meet customer demands.

The outcome

The long-term consequence of reducing change frequency is that we can poorly correlate effort and outcome, it becomes indistinguishable what works and what doesn't - and thereby, the quality of our work, our product, our processes and our metrics deteriorate. We lose our reason to exist on the market: providing high quality and value to our customers on demand.


"If it hurts, do it more often."

Let's just follow the previous computation:
If 1 big change fails 100% of the time, maybe you can slice it into 3 smaller changes, of which 2 will succeed, reducing your failure rate by 66%?

So, instead of deciding to reduce change frequency, you decide to increase it?

The immediate after-effect

Because there is now a shorter period of time until the next release, there will be a reduced time between when something is developed and until you see the consequences. We close the feedback loop faster, we learn quicker what works and what doesn't. And since we tend to be wired to not do things that become painful, we do more of the things that work, and less of the things that don't.

The knock-on effect

Managing small changes is easier than managing complex change. Thereby, it becomes less risky, less work and less painful to make multiple small changes. Likewise, since we get faster (and more frequent) feedback on what worked, we can optimize faster for doing more things that provide actual value.

The outcome

By making rapid, small changes, we can quickly correlate whether we improved or worsened something, and we can respond much more flexibly towards changing circumstances. This allows us to deliver better quality and feel more confident about what we do.


Summary

The same vicious circle created by the attitude, "If we change less often, we will have fewer (but significantly more) painful events" can become a virtuous cycle if we change our attitude towards, "If we do it more often, it'll become easier and less painful each time."

Your call.

4 comments:

  1. I'm curious about sprint duration. I've only ever used 2 week sprints but I do also like the idea of fewer releases. If we did releases less often, say 4 weeks, then a release could be contain a better increment. We could devote more time to evangelizing the contents. Then what about 6 week sprints? Have you experimented with different sprint durations and what have you discovered?

    ReplyDelete
    Replies
    1. Hey Tim, thanks for commenting.
      First things first: Sprints and releases are unrelated.
      The SG2020 is even explicit about this, "The Sprint Review should never be considered a gate to releasing value."

      Many great Scrum teams deliver working software multiple, some even dozens, of times, per day.


      Let me quote Einstein: “If I had an hour to solve a problem, I'd spend 55 minutes thinking about the problem and 5 minutes thinking about solutions.”
      This approach is entirely valid, IF - and only if - your problem statement doesn't change over time.
      Shorter sprints are better suited to manage the volatility of the problem.
      Longer sprints tend to result in more obsolete plans and missed opportunities.

      When you can reliably predict the future for more than a month down the line, then a classic Waterfall is probably more suitable than Scrum.

      And by the way, if you can reliably predict your market a month down the line, I suggest moving from software development to stock trading. There's a fortune waiting to be made by people who can reliably forecast disruptive changes. ;-)

      Delete
  2. Not to mention the amount of work that you do not warehouse longer than necessary. This are sometimes huge batches of work waiting to give value to users in return, just sitting there.

    ReplyDelete
    Replies
    1. And that gets worse once we realize that the value of software diminishes rapidly.

      Delete