Dice Estimation

Although I am a proponent of "#noestimates," that is - the concept that estimation is often the consequence of unsound mental models, and in itself somewhat wasteful, because we often don't know - I still propose an estimation technique that is as useful as it is simple. BUT - it requires context. Let's epxlore this context:

Problems with estimation

Regardless of how we put it, one major challenge is that many organizations use estimates as if there was certainty that they can become "correct" - which is a common dysfunction, because it results in undesirable, wasteful behaviours. These could be, for example, as upfront design, padding, "challenging estimates" - and even post-hoc adjustment of the figures to match reality. None of these behaviours help us deliver a better product, and they may even impede quality or time-to-market.

All these aside: we don't know until we tried - that is, there's this thing called "the high probability of low-probability events," so regardless of how much effort we waste on trying to get those activities "right", we could still be wrong due to something we didn't factor.

These are the common reasons for the #noestimates movement which simplys reject the discipline of upfront estimation in favor of monitoring the flow of work via execution signals to gain both predictability and control. You may ask: "If estimation is so obviously flawed, then why would you propose yet another estimation technique?"

Category Errors

A common challenge teams face is that not all work is equal: Giving a trivial non-tech example, it doesn't take much expertise to realize that building a tool shack isn't the same category as building a house - and that's not the same category as building a Burj Khalifa.

By simply letting uncategorized work flow into teams, we risk that extremely large packages block the flow of urgent high-value items for a prolonged time. Hence, a first categorization into "Huge, Large, Small" does indeed make sense - The key realization is that these aren't equal.

Hence, dice estimation first categorizes work into the right bin: everything that's workable can be taken up by the team, while everything that isn't workable should be managed appropriately before being worked upon.

Six-sided dice

Story Points worked out terribly, and they led to many dysfunctions - so let's avoid that side track. Instead, let's say that we estimate items in dice values. For small items, our estimation unit would be team-days, for large items, it would be weeks, and for large items, it would be months.

That also leads us to the first logical connection:
One Small Dice at 5 or 6 could already classify for a "1" on a medium dice. We could think about splitting it in two if we want to take it on. On items bigger than 5, we should exercise caution: do we understand it properly, is the cost of implementation warranted - is it really the best thing to do next?

This also applies when an item exceeds 5 on the week count - at that point, it becomes a huge dice: we might want to check a lighweight business case and pivot before proceeding.

How Dice Estimation works

Dice estimation is very easy - you simply ask, "If each day of work was a dice roll, what would be the amount of sides on our dice so that we have enough time to hit a '1' - on average - by the time we are Done?"

That is: if the team expects to be done on 2 days, they would choose 2. If they would expect 3 days, they would choose 3 - if they expect 6, it would be 6 days.

Measuring Confidence

Since dice rolls aren't linear, this form of estimation focuses on levels of confidence in completion within a predefined period of time. Hence, despite the fact that we estimate in days, dice estimation values confidence - and thereby clarity and understanding - above quantity of work.

Estimating with uncertainty

Just like a D6 doesn't mean that "we will roll 6 times, then we will have rolled a 1" - what we estimate is the amount of faces on the dice and thereby the expected amount of days that the item will be done.

Think of it like this: If you have a coin, you have a 50% chance to get heads on the first try - and while statistically, you'd expect to get at least one heads in 2 attempts, it's theoretically possible that you need to flip the coin an infinite amount of times and never get heads.

The estimate in team-days helps us manage our work on the Dailies in an efficient manner - as long as the estimated amount of days hasn't elapsed, it's safe to assume we are on track. Any item reaching and/or exceeding the estimate should be scrutinized: are we stuck, did we misunderstand something - what is going on? The "estimate," thus, isn't even estimating effort - it's forecasting a pivot-or-persevere checkpoint so that we minimize sunk cost.

Lightweight estimation

We can estimate work items either as they arrive, during refinement, or when we get busy - that doesn't even matter. The key reasons for estimation are first, avoiding category errors and second: setting up an intuitive checkpoint so that we don't run out of bounds.

With Dice Estimation, we avoid category errors and institute checkpoints in execution

Of course, you could use dice estimation during Sprint Planning or in a PI-Planning as well, but you don't even need cadences to take advantage of Dice Estimation.

Predictability and Forecasting

As we know, a dice averages at 3.5, so even if we didn't do anything yet except categorizing by dice size, we know that our backlog of workable, small items would take about 2 weeks for 3 items. Once we've done a little bit of work and know how much the completed items took, we can either run a simple Monte Carlo Simulation or apply the Law of Large Numbers to figure out how much known work we have ahead of us. The same applies to the Large and Huge dices, which gives us a fairly decent understanding of when an item might be delivered, based on backlog position.

Not a Performance Metric

The dice metaphor should make it easy to explain how neither can one measure the perfomance of a dice-roller based on how often they roll a certain number, nor does it make sense to "squeeze" estimates - the probability of guessing the right number won't increase by restricting the "allowed" numbers that can be picked. If anything, squeezing estimates would lead the team to spend more time on examining exceptions - hence, reducing performance.

Why you should switch from Story Points to Dices

Dice Estimation is a minimum effort method of right-sizing work that helps with:

1. Sufficient predictability
2. Easy plannability
3. Execution checkpoints
4. Avoiding large batches that block flow
5. "Split, Pivot or Defer" decisions

Time to discard the "T-Shaping" model!

 A common concept in the "Agile" community is that of T-Shaping. Regardless in which form or fashion it is promoted, the narrative is basically this: People have certain skills, and if they are one-trick ponies or are too generalist, they need to form a so-called "T-Shape." Nonsense!

The concept of the T-Shape

A specialist is a person with a very narrow skillset (making that person an "I"-Shape) while a generalist has a broad skillset (making that person a "dash"-Shape). 

A broad set of skills plus one deep expertise is a "T", multiple deep expertises are "Pi." And then, no model couldn't be "improved" by adding complexity, so we get variations like "M" Shapes, i.e. non-generalists with multiple expertises, and "E" shapes who aren't only experts, who also have "experience, exploration and execution." - although that's clearly buzzword bingo terrain, because what kind of an expert doesn't also have these?

Even I have blogged about T-Shaping, for example, here.

With the model, we were moving the burden onto people: "You must form a T/Pi/E-Shape." Thats nonsense, because it's not like people wouldn't do that on their own volition. In fact, most people who find that they can't use their skills at work will spend their time acquiring off-work skills, such as, for example, learning an instrument: everyone is Pi-Shaped, the only question is if it helps the company.

And now, I announce the time to ditch this model in favor of another model that already predates the Agile Manifesto - "Psychological Flow", by Mihály Csíkszentmihályi.

Psychological Flow

Let's take a look at the model Csíkszentmihályi proposes: He uses the dimensions of "Challenge Level" and "Skill". The very assumption that people have "one" or even a very limited number of skills is already absurd. Consider, for example: breathing. Do you know anyone who doesn't have this skill? Depending on what would qualify as a "skill," people are experts in thousands of them.

Instead of thinking about skills as discrete variables with a discrete set of values (e.g., "none", "basic", "advanced", "expert" ...) - we could think of "skill" as a continuum of different, integrated topics on a continuum spectrum.

Most importantly, the psychological flow proposes a second dimension: the level of challenge, in relation to the skill. For example, asking a Chess Grandmaster to state how a pawn moves is like a relaxation exercise. On the other hand, someone who never cared about Chess won't even take an interest in this question, "why should I care?"

If you'd ask a senior project manager to take care of large, critical programme, that brings out excitement and a sense of, "I am the right person for this job, and I can do it." Their brain would immediately sort through potential approaches and how to make it a success.

A college fresher, on the other hand, might be anxious about where to start and what to do.

That's the basic concept.

Notice that neither the "challenge level" nor the "skill level" are measured on an absolute scale - they can be considered from the perspective of the person who is doing the work: Is this work "low challenge" or "high challenge," does it require "low skills" or "high skills?"

How is Flow related to T-Shapes?

Well, it is - and it isn't. Instead of asserting that a person has a certain skillset required to do the job, we reframe the skilling challenge:

1. Is a task adequate to a person's skill level?
1. We should not under-challenge people, lest they bore out.
2. We should not over-challenge people, lest they become become worried or anxious.
2. How do we ensure that people receive challenges which allow them to use high skill?
1. High challenge, medium-skill tasks get people to become interested in what they do and that's how they grow.
2. High challenge, high skill tasks bring the best out of people.

The challenge, hence, is to identify meaningful, highly challenging work which is at least within grasp of people: medium skills get people excited, high skills get people to perform at their best.

Failure in "T-Shaping" isn't on the individual - it's team management not ensuring people have sufficient challenge to grow and show their performance.

(note based on feedback: in a self-managed team, "team management" isn't top-down - it's what teams decide to do)

T-Shaping ignores Flow

The T-Shaping model makes an implicit assumption: that all the work in the expertise domain is of interest to a person, and that it's a good use of their skills. This isn't always the case. Part of the work may make a very advanced person fall asleep, while being quite challenging for someone new to the domain. Hence, the best way to distribute work in a team isn't based on a certain "T-Shape," but to ensure that people reach a flow state - that is:

A team as a whole performs best not when people exercise a certain "T", but when the work provides everyone with sufficient, yet not overwhelming, challenge.

We shouldn't upskill with the goal of acquiring an abstract "T-Shape" - that's a weird framing already. 

Instead, we should distribute work in a team so that everyone is excited to do what they do, while ensuring nobody is overchallenged and anxious or underchallenged and bored.

Whether the result is something like a "T-Shape," doesn't even matter - because the primary result is a highly qualified team whose members are comfortable taking on slightly bigger challenges every day. 

Let's forget the T-Shape.

Let's refocus on the question, 

"How can we distribute work within our team, so that everyone gets to spend the highest amount of time possible in a flow state?"

Delivered, Deployed, Done?

While an agile organization should avoid over-engineering a formal status model, it's necessary to provide standards of what "Done" means so that people communicate at an even level. The highest level of confusion arises in large organizations where teams provide piecemeal components into a larger architecture, because teams might define a "Done" that implies both future work and business risk until the delivery is actually in use.

In such a case, discrimating between "Deployed" and "Done" may be useful.

What's Done?

At the risk of sounding like a broken record, "Done means Done," "It's not Done when it's not done" and "You're not Done when you're not done."

That is, when you're sitting on a pile of future work, regardless of whether that pile is big or small, you're not done. This is actually quite important: While softening your DoD gives you a cozy feeling of accomplishment, it reduces your transparency and will eventually result in negative feelings when the undone work comes back to bite you.

As such, your enterprise DoD should encompass all the work that's waiting. Unfortunately, in an Enterprise setting, especially when integrating with Waterfall projects or external Vendors, you may have work waiting for you a year or more down the line. The compromise is that teams put work items into an intermediate, "Deployed" status when the feature is live in Production, and set it to "Done" at the appropriate time in the future.

What's Deployed?

In situations where a team has to move on before taking care of all the necessary work to move to "Done", because there are factors outside their own control, it may be appropriate to introduce an intermediate status, "Deployed." This allows teams to move on rather than idly waiting to do nothing or wasting their energy getting nowhere.

In large enterprise situations, teams often deliver their increments and the following haven't been taken care of yet:

• Some related open tickets
  
• User training
  
• Incidents
• Feature Toggles
• Business Configuration
  
• E2E Integration
• Tracking of Business Metrics
• Evidence of Business Value

This status is not an excuse - it creates transparency on where the throughput in the value stream actually is blocked, so that appropriate management action can be taken.

Interpreting "Done" vs. "Deployed."

Let's take a look at this simple illustration:

Team 1

If they would soften up their DoD to equate "Deployed" and "Done", then the business risk is hidden, and it becomes impossible to identify why the team isn't generating value, even though they're delivering. They lose transparency with such an equivocation.

A strict discrimination between "Deployed" and "Done" surfaces the organizational impediment in this team and makes the problem easy to pinpoint.

Team 2

It wouldn't make sense to discriminate between "Done" and "Deployed", because the process is under control and there is no growing business risk. This team could just leave items "in Progress" until "Done" is reached and doesn't benefit from "micromanaging" the status.

Planning with Capacity Buffers

I get asked quite often some questions along the line, "How do we deal with work that's not related to the Sprint Goal?" The typical agile advice is that all work is part of the Product Backlog and treated as such and the work planned for the Sprint is part of the Sprint Goal.

In general, I would not recommend this as a default approach. I often advise the use of Planning Buffers.

Where does the time go?

Teams working in established organizations on legacy systems often find that the amount of work which doesn't advance the product makes up a significant portion of their time. Consequently, when they show up in a Sprint Review, the results tend to go into one of two directions: 

Either, the team will have focused on new development, angering existing users why nobody tackled known problems - or, the team will have focused on improving legacy quality - angering sponsor why the team is making so little progress. Well, there's a middle ground: angering everyone equally. 

In any case, this is not a winning proposition, and it's also bad for decision making.

Create transparency

A core tenet of knowledge work is transparency. That which isn't made explicit, is invisible.

This isn't much of an issue when we're talking about 2-5% of the team member's capacity. Nobody notices, because that's just standard deviation.

It becomes a major issue when it affects major portions of the work, from like a quarter upwards of a team's capacity. 

Eventually, someone will start asking questions about team performance, and the team, despite doing their best, will end up in the defense. That is evitable by being transparent early on.

Avoid: Backlog clutter

Many teams resort to putting placeholders into their backlog, like "Bugfix", "Retest", "Maintenance" and assigning a more or less haphazard number of Story Points to these items.

As the Sprint progresses, they will then either replace these placeholders with real items which represent the actual work being done - or worse: they'll just put everything under that item's umbrella.

Neither of these is a good idea, because arguably, one can ask how the team would trust in a plan containing items they know nothing about. And once the team can't trust it ... why would anyone else?

Avoid: Estimation madness

Another common, yet dangerous, practice, is to estimate these placeholder items, then re-estimate them at the end of the Sprint and use that as a baseline for the next Sprit.  

Not only is such a practice a waste of time - it creates an extremely dangerous illusion of control. Just imagine that you've been estimating your bugfixing effort for the last 5 Sprints after each Sprint, and each estimate looks, in the books, as if it was 100% accurate. 

And then, all of a sudden you encounter a major oomph: you're not meeting up to your Sprint Forecast, and management asks what's going on. Now try to explain why your current Sprint was completely mis-planned. 

So then, if you're neither supposed to add clutter tickets, nor to estimate the Unknowable - then what's the alternative?

Introduce Capacity Buffers

Once you've been working on a product for a while, you know which kinds of activities make up your day. I will just take these as an example: New feature development, Maintenance & Support, fixing bugs reported from UAT - and working on other projects.

I'm not saying that I advocate these are good ways to plan your day, just saying if this is your reality - accept it!

We can then allocate a rough budget of time (and therefore, of our develoment expenses) to each activity.

An example buffer allocation

Thus, we can use these buffers as a baseline for planning:

Buffer Planning

Product Owners can easily allocate capacity limits based on each buffer. 

For example, 10% working on other projects, 25% UAT bugfixing and 25% maintenance work, which leaves 40% for development of new features. 

This activity is extremely simple, and it's a business decision which requires absolutely no knowledge at all about how much work is really required or what that work is.

In our example, this would leave the team to plan their Sprint forecast on new feature development with 40% of their existing capacity.

As a side remark: every single buffer will drain your team's capacity severely, and each additional buffer makes it worse. A team operating on 3 or more buffers is almost incapacitated already.

These things are called "buffer" for a reason: we prefer to not use them, but we plan on having to use them. 

Sprint & PI Planning with Buffers

During the planning session, we entirely ignore the buffers, and all buffered work, because there is nothing we can do about it. We don't estimate our buffers, and we don't put anything into the Sprint Backlog in its place. We only consider the buffer as a "black box" that drains team capacity. So, if under perfect circumstances, we would be able to do 5 Backlog items in a week, our 60% allocated buffer would indicate that we can only manage 2 items.

Since we do, however, know that we have buffer, we can plan further top value, prioritized backlog items that do contribute to our team's goal, but we would plan them in a way that their planned completion would work out even when we need to consume our entire buffer.

So, for example: if our Team Goal would be met after 5 backlog items, we could announce a completion date in 3 Sprints, since our buffers indicate that we're most likely not going to make it in 1 Sprint.

Enabling management decisions

At the same time, management learns that this team isn't going at full speed, and intervention may be required to increase the team's velocity. It also creates transparency how much "bad money" we have to spend, without placing blame on anyone. It's just work that needs to be done, due to the processes and systems we have in place.

If management would like "more bang for the buck", they have some levers to pull: invest into a new technology system that's easier to maintain, drive sustainability, or get rid of parallel work. None of these are team decisions, and all of them require people outside the team to make a call.

Buffer Management

The prime directive of activity buffers is to eliminate them.

First things first, these kinds of buffer allocations make a problem transparent - they're not a solution! As such, the prime directive of activity buffers is to eliminate them. and the first step to that is shrinking them. Unfortunately, this typically requires additional, predictable, work done by the team, which should then find its way into the Product Backlog to be appropriately prioritized.

Buffers and the Constraint

If you're a proponent of the Theory of Constraints, you will realize that the Capacity buffers proposed in this article have little relationship to the Constraint. Technically, we only need to think about capacity buffers in terms of the Constraint. This means that if for example, testing is our Constraint, Application Maintenance doesn't even require a buffer - because the efforts thereof will not affect testing!

This, however, reuires you to understand and actively manage your Constraint, so it's an advanced exercise - not recommended for beginners.

Consuming buffers

As soon as any activity related to the buffer becomes known, we add it to the Sprint Backlog. We do not estimate it. We just work it off, and keep track of how much time we're spending on it. Until we break the buffer limit, there is no problem. We're fine. 

We don't "re-allocate" buffers to other work. For example, we don't shift maintenance into bugfixing or feature delivery into maintenance. Instead, we leave buffer un-consumed and always do the highest priority work, aiming to not consume a buffer at all.

Buffer breach

If a single buffer is breached, we need to have a discussion whether our team's goal is still realistic. While this would usually be the case in case of multiple buffers, there are also cases where buffers are already tight and the first breach is a sufficiently clear warning sign.

Buffer breaches need to be discussed with the entire team, that is, including the Product Owner. If the team's goal is shot, that should be communicated early.

Buffer sizing

As a first measure, we should try to find buffer sizes that are adequate, both from a business and technical perspective. Our buffers should not be so big that we have no capacity left for development, and they shouldn't be so small that we can't live up to our own commitment to quality.

Our first choice of buffers will be guesswork, and we can quickly adjust the sizing based on historic data. A simple question in the Retrospective, "Were buffers too small or big?" would suffice.

Buffer causes

Like mentioned above, buffers make a problem visible, they aren't a solution! And buffers themselves are a problem, because they steal the team's performance!

Both teams and management should align on the total impact of a buffer and discuss whether these buffers are acceptable, sensible or desirable. These discussions could go any direction.

DevOps teams operating highly experimental technology have good reasons to plan large maintenance buffers. 

Large buffers allocated to "other work" indicate an institutional problem, and need to be dealt with on a management level.

Rework buffers, and bugfixing is a kind of rework, indicate technical debt. I have seen teams spend upwards of 70% of their capacity on rework - and that indicates a technology which is probably better to decommission than to pursue.

Buffer elimination

The primary objective of buffer management is to eliminate the buffers, Since buffers tend to be imposed upon the team by their environment, it's imperative to provide transparent feedback to the environment about the root cause and impact of these buffers.

Some buffers can be eliminated with something as simple as a decision, whereas others will take significant investments of time and money to eliminate. For such buffers, it tends to be a good idea to set reduction goals.

For example, reducing "bugfixing" in our case above from 25% to 10% by improving the system's quality would increase the team's delivery capacity from 40% to 55% - we nearly double the team's performance by cutting down on the need for bugfixing - which creates an easy-to-understand, measurable business case!

Now, let me talk some numbers to conclude this article.

The case against buffers

Imagine you have a team whose salary and other expenses are $20000 per Sprint.

A 10% buffer (the minimum at which I'd advise using them) would mean not only that you're spending $2000 on buffers, but also that you're only getting $18000 worth of new product for every $20k spent!

Now, let's take a look at the case of a typical team progressing from a Legacy Project to Agile Development:

Twice the work ...

Your team has 50% buffers. That means, you're spending $10k per Sprint on things that don't increase your company's value - plus it means your team is delivering value at half the rate they could! 

Developers working without buffers, would be spending $20k to build (at least) $20k in equity, while your team would be spending $20k to build $10k in equity. That means, you would have to work twice as hard to deliver as positive business case!

Every percent of buffer you can eliminate reduces the stress on development teams, while increasing shareholder equity proprtionally!

And now let's make that extreme. 

Fatal buffers

Once your buffer is in the area of 75% or higher, you're killing yourself!

Such a team is only able to deliver a quarter of the value they would need to deliver in order to build equity!

 

In such a scenario, tasking one team with 100% buffer work, and setting up another team to de-commission the entire technical garbage you're dealing with is probably better for the business than writing a single additional line of code in the current system.

Please note again: the problem isn't the capacity buffer. The problem is your process and technology! 

High Performance: No Buffers

High Performance teams do not tolerate any capacity buffers to drain their productivity, and they eliminate all routine activity that stops them from pursuing their higher-ordered goal of maximizing business value creation. As such, the optimal Capacity buffer size is Zero.

Use buffers on your journey to high performance, to start the right discussion about "Why" you're seeing the need for buffers, and the be ruthless in bulldozing your way to get rid of them.

The value of idle time

The Product Backlog is a mandatory part of Scrum. Together with the Sprint Backlog, they define both the planned and upcoming work of the team.

There's a common assumption that it's considered good to have a decently sized product backlog, and as many items in the Sprint Backlog as the team has capacity to deliver. Let's examine this assumption by looking at a specific event.

The "no backlog" event

It was Tuesday evening. I had just a busy day behind me.
I was chilling, browsing the Web, when I received a message on LinkedIn. The following conversation ensued:

Mind Lukasz' last statement: "The most impressive customer service ever".
Why was this possible?

Had Lukasz contacted me half an hour earlier, this dialogue would never have happened. Why? Because I would have been busy, doing some other work. Lukasz would have had to wait. His request would have become part of my backlog.

Service Classes

There's a lot of work I am pushing ahead of me on a day-to-day basis.
But I classify my work into three categories:

1. Client related work - I try to cap the amount of client related work, to maintain a sustainable pace.
   It's a pretty stuffed backlog where things fall off the corners every day.
2. Spontaneous stuff - I do this stuff as fast as I see it, because I feel like doing it.
   The hidden constraint is that "as I see it" depends on me not being engaged in the other two types, because these take 100% of my attention.
3. Learning and Improvement - That's what I do most of the time when not doing Project work.
   I consider web content creation an intrinsic part of my own learning journey.

These categories would be called "service classes" in Kanban.
I am quite strict in separating these three classes, and prioritize class 1 work over everything else.

Without knowing, Lukasz hit my service class 2 - and during a time when I was indeed idle.
Since class 1 has no managed backlog, I got around to Lukasz' request right as it popped up, and hence, the epic dialogue ensued.

Service Classes in Scrum

If you think of the average Enterprise Scrum team, class 1 is planned during Sprint Planning, and class 2 activities are undesirable: all the work must be transparent in the Sprint Backlog, and the SBL should not be modified without consent of the team, especially not if this might impact the Sprint Goal.

Many Scrum teams spend 100% of their workload on class 1, going at an unsustainable pace, because the constantly descoped class 3 work warrants future-proofing the work.
Even if they plan for a certain amount of class 3 work, that is usually the first thing thrown overboard when there's pressure to deliver.

The importance of Spontaneity

Few Scrum teams take care of Class 2 work, and Scrum theory would dictate that it should be placed in the Product Backlog. This just so happens to be the reason why Scrum often feels like drudgery and developers are getting uncomfortable with practices like Pair Programming.

"Spontaneous stuff" is a way to relax the mind, it helps sustain motivation and being totally uncommitted on outcomes allows creativity to flourish.

Load versus Idle Time

As mentioned, class 1 is bulk work. As workload increases, the percent amount of class 1 activity quickly approaches 100%. Taking care of class 3 activity means that increasing load quickly diminishes idle time activity to go to Zero.

Since I already mentioned that idle time activity creates magic moments, both for team members and customers, high load with zero idle time destroys the "magic" of a team.

Wait Time Idleness

One source of Idle Time is Process Wait Time.
In a Lean culture, wait is seen as detrimental waste. This is both true and false. It is true when the organization doesn't create value during wait, while incurring costs. It is false when this wait is used to generate "magic moments".

Buffer Time Idleness

Both Scrum and Lean-Kanban approaches encourage eliminating idle time, as would the common "agile Scaling" frameworks. Team members are constantly encouraged to pull the next item. or help others get work in progress done faster.
This efficiency-minded paradigm only makes sense if the team controls the end-to-end performance of the process, otherwise they might just accumulate additional waste. Theory of Constraints comes to mind.

On the other hand, buffer removal in combination with a full backlog disenchants the team - there will be no more "magic moments": Everything is just plan, do, check, act.

Idle Time and Throughput

The flawed assumption that I want to address is that buffer elimination, cross-functionality and responsibility sharing would improve throughput. Maybe these will increase output, but this output will be subject to the full lead time of any other activity.

Backlogs vs. Idle Time

Genuine idle time means that the input backlog currently has a size of Zero and a parallel WIP of Zero as well. There is no queue: neither work-in-progress nor work-to-do.
An idle system doesn't require queue management. When idling, throughput for the next request is exactly equal to work time - the maximum throughput speed we could hope to achieve. This kind of throughput speed can look absolutely mind-boggling in comparison to normal activity cycle times.

The impact on organizational design

A perfect organization takes advantage of idle time points that maximize throughput speed - not efficient utilization avoiding idle time.

Summary

The conversation with Lukasz is an example of the benefits of having idle time in your work.
This kind of idle time allows for "magic moments" from a customer perspective.

Just imagine an organization where "magic moments" are the norm, and not the exception.
This requires you to actively shape demand: when demand is roughly equal to capacity, we can eliminate backlogs.
Demand queues destroy the magic.

Eliminate the queues. Make magic happen.

Why "Agile Development" will not solve your problem

Do we even need "Agile" to improve time-to-market, reducing cost and increasing success rates? After reading this article, you may have doubts.

Do these statements sound familiar to you?

• Software Development is too expensive
• We are lagging years behind the demand.
• Too many initiatives (projects etc.) fail.

The solution? Scrum. No, just kidding. Scrum doesn't have a solution there. Neither does LeSS or SAFe. Or XP, or any other "Agile Framework" that focuses on the development part of an organization. The solution rests in how we think about organizing our products long before developers actually get involved.

Too many "Agile Transformation" initiatives are focused on the output of software development and forget the big picture of the system in which they are operating. 

The Product Development Funnel

Take a look at this abstract development funnel. Every organization works somehow like this, while the slope of the funnel and the processes in each stage may vary widely. 

Let us examine the core concepts of this funnel.
To keep this article at least somewhat simple, let us equivocate "product" and "product portfolio" - because from a certain level of abstraction, it's irrelevant whether we have one product or an entire portfolio thereof: both capacity and capability are still finite.
Likewise, let us ignore the entire dispute around Lean/Agile vs traditional program and/or portfolio management, because irrespective of how you do it, you are still dealing with demand, choosing opportunities and prioritizing effort.

Customer Demand

Whether our customers are people out there on the market who purchase our product, or internal stakeholders of our product, there are always unmet needs. The amount of unmet needs that our product could meet is the amount of potential work that is still required, and that's potentially infinite - unlike our funding and capacity.

At the expense of taking a grossly oversimplified definition here, let us define "Marketing" as the effort invested into making others aware of potential needs and wants - and in return, becoming aware of what these are.

Oddly enough, even delivery is a marketing activity in some sense: as people use the product, they become more aware of additional needs. Hence, the old economic rule, "supply creates its own demand" ensures that the demand part of the funnel will never dry up in a well-run product. 

Development portfolio

Every demand that's deemed sufficiently important will eventually end up in some kind of portfolio, the known list of undone upcoming work. This is where the first stage of filtering occurs. Filter mechanisms include product strategy, economic viability or even internal politics. The specific reasons for filtering are irrelevant in this context.

Every item that lands in the portfolio will subsequently put stress on the development organization. In an odd twist of fate, the worse an organization's ability to manage the portfolio, the lower the satisfaction with the developed product will eventually be, irrespective of how good or bad development was.

The reason is simply because once the capacity of the development organization is exceeded, accepting further demand will not yield better outcomes.

Running Initiatives

Once a portfolio item has been scoped for development, it will become a development initiative. Organizations do have this habit of finding it easier to start initiatives than to complete them. Every initiative started will eventually result in one of three states: Live (successfully delivered), Dead (descoped without delivery) - or Undead (lingering in a limbo state where people are waiting for results that may never come).

Let's briefly examine Live and Dead: An initiative that goes "Live" is the only type of initiative that results in something called "Business Value", or even potentially positive Return on Invest. It's therefore highly desirable to get initiatives live.

A Dead initiative is an initiative that has been terminated before it has generated business value. Every cent spent on dead initiatives was waste. Completing an initiative with a negative ROI is just throwing the good money after the bad, so killing them is still the better alternative.

The biggest problem though, are neither Live or Dead initiatives - it's the Undead: An organizational culture that doesn't have the willingness to kill initiatives that have exceeded their life expectation will generate and accumulate undead initiatives. 

The problem with these undead initiatives is that they drain your organization's energy. People spend time and effort to work on and track the Undead, and this time isn't available to get other initiatives Live, so they turn otherwise healthy initiatives into Undead as well. Talk about Zombie Apocalypse!

The health of a product development organization can easily be measured by how many Undead Inititiatives they permit - every initiative ever started that was not clearly terminated or completed. The easiest, yet oftentimes most uncomfortable, fix to increase this health is by taking inventory and killing all the zombie initiatives.

Delivery Capability

The delivery capability is the one place where a common "Agile Transformation" focuses. The expectation is that somehow, magically, the adoption of daily standups, cadenced plannings, reviews and retrospectives will improve performance. It doesn't. (At least, not significantly - unless your developers are really stupid, which I claim they aren't.) And, on a more abstract level, the expectation is that this approach will somehow improve the ratio of met versus unmet demand. It doesn't.

It might look like a good idea thus to increase the capacity of the development process, i.e. add more developers or provide better tools and extra resources to make developers more efficient. It isn't.

What sounds good in theory often results in proportionately increased expectations: When development capacity is increased through additional funding, the organization will expect to be able to launch more initiatives - and thus, the problem persists. It might even get worse, but that's another story, to be told at another time.

Retrospectives and change initiatives focused on "how can we do better in delivering" might completely miss the point if the development organization is in a state of continuous overburden: the overburden will never go away as long as more demand enters the system than what can be delivered.

Everything these frequent events accomplish is that the constant pressure to deliver creates psychological stress and eventually developers burn out: Development performance may even decline!

Let's talk about waste

In this section, let us move backwards through the funnel, as the waste accumulates throughout the funnel, like a clogged pipe. We have to unclog it where the blockage is caused, rather than where it occurs.

Wasted development effort

What should developers spend time on? Developing working solutions, obviously. Yet, developers often spend significant portions of their time doing things not related to this. Let's take a crude tally:

• Status meetings for all those running initiatives.
• Especially the zombies.
• Task switching effort between those initiatives. 
• Waiting for input on an initiative.
• Catching up with changing requirements in an initiative.

The sum of all the effort invested into these items is the "low hanging fruit" in any optimization strategy. Reducing the amount of running initiatives reduces the amount of wasted development effort, thus increasing available capacity to get meaningful work done.

Kanban addresses this by limiting Work-in-Progress (WIP), although this only helps when we limit WIP at a higher level, that is: we must control the influx of initiatives, not the amount of development tasks, in order to gain any benefits.

Scrum addresses this by limiting the amount of items a team takes on during a Sprint - however, this is not a fix when more portfolio initiatives get started than completed: the consequence is an ever-growing product backlog. The problem is hidden, not solved.

In the ideal Scrum, the Scrum Team has full control over the entire product development funnel. Since this would require the organization to have moved from cost accounting to throughput accounting, from funding work to funding teams and towards an integrated cross-functionality that puts business acumen into the Scrum Team - this is a type of Scrum most organizations don't have. Again, this implies that the improvement potential then doesn't relate to delivery.

Coordination Waste

The second stage of organizational waste occuring in product development is the coordination of initiatives.  That's where we get project managers and matrix organizations from, and it's the reason why developer's calendars are cluttered with appointments. As every initiative has someone in charge, for simplicity's sake, let's call them "Initiative Owner". This initiative owner wants to be informed about what is going on and will take care that their initiative gets due attention.

If there is only one initiative, then all of an organization's efforts can focus on completing this one initiative.

Add another initiative, and the organization has to solve the problem of coordinating both the efforts between those initiatives - and to solve blockages in either initiative without blocking any other work. The effort required for these activities increases exponentially in complexity with each ongoing initiative. 

This coordination overhead would not even exist if there was only one initiative. The time and money sunk into coordinating parallel initiatives is pure waste.

At this point, we already need to ask the question why we see value in having multiple parallel initiatives - and why we believe that the value of starting another initiative outweighs the waste caused thereby. Likewise, we can ask the question whether the reduction of initiatives reduces the waste by a level that warrants deferring, descoping or even discarding one or more initiatives.

As coordination waste is multiplicative on outcome throughput, the lever of optimizing coordination waste outweights the lever of optimizing team-level productivity.

Budget Waste

Every organization determines one way or another what will be developed. Regardless if this happens via a Product Owner prioritizing an item in a product backlog or a SteerCo starting a traditional project, at some point a decision will be made to start a new initiative. In one form or another, at this point, the organization makes an investment decision: budget is allocated to the initiative. The biggest waste of budget is to start an initiative that doesn't get completed. Any initiative started without sufficient delivery capacity to complete both the currently running and the new initiative, will predictably induce waste into the organization. 

The simplest form of reducing budget waste is by deferring the decision about starting an initiative until there is sufficient free capacity within the organization to complete this initiative without impacting any other initiative.

As budget waste yields coordination waste, and coordination waste yields capability waste, the lever of reducing budget waste is even stronger than that of reducing coordination overhead.

Marketing waste

Probably the most well-hidden form of waste is demand waste, or "Marketing waste". Any form of demand that isn't eventually met by the product generates waste. The discovery, itemization and exploration of this type of demand generate cost without value. Having more demand than one can meet is only good because it creates options, yet one needs to be careful lest one loses focus in all of these options. There's even the economic dilemma of demand pull inflation where the mere prospect of additional demand will increase cost, but that's another story, to be told another day.

The simplest way of decreasing marketing waste is by limiting demand to a level where supply is at least still sensibly correlated to demand, and the organizational processes of managing demand are low effort. 

Marketing waste propagates into the product organization at full force: Poorly defined or invalid value hypotheses block scarce downstream capacity, while an oversupply of demand leads to pressure on the system. 

Limiting the waste

As heretical as this sounds, the most effective way of improving a development organization has nothing to do with improving development - and that's where all the team level agile approaches go wrong.

The most effective strategy for increasing effectiveness is by pulling the longest levers first:

1. Limit the amount of initiatives within the development organization. To achieve this:
2. Limit the influx of approved initiatives. To achieve this:
3. Limit the rate at which demand is translated into initiatives. To achieve this:
4. Limit the influx of demand into your organization.

"Do not try to do everything. Do one thing well."

- Steve Jobs

Here is an overview of what an organization's development funnel would look like if all settings were optimal:

The optimization in this context focuses on reducing the Organizational Waste not related to delivering Useful Products, and on the decrease of Opportunity Cost by eliminating the waste associated with pursuing low-value demand.

When we take into account Little's law, we realize that this optimization approach achieves:

• Reduced cost of meeting demands ("feature cost savings")
• Reduced Time-To-Market for features
• Increased Success rates of initiatives

Depending on how significant the associated waste and opportunity cost within an organization currently is, the leverage may be massive, while the organizational change (staffing, training, skill distribution etc.) is insignificant. All that's required is - thinking differently!

And the funniest part - we haven't even touched development!

Conclusions

1. Think about what you really need to optimize before looking at "Agile Frameworks" as a solution. 

2. Change your thinking before optimizing the work.

3. Optimize what you work on before optimizing how you work.

Do you really want high utilization?

Let's end the discussion about whether we should optimize for maximum utilization right here, right now - with a metaphor. Ponder your own answers for the questions.

Your features are the cars. Your teams are the lanes.

Lane 1 is optimized for maximum utilization (80%+).
Lane 2 tries high utilization (50%).
Lane 3 actively minimizes utilization (as close to 0% as possible).

Question: If your goal is to get from A to B as fast as possible - on which lane would you travel?

Question: What happens when a car suddenly needs to brake? (i.e. an impediment occurs)

Question: What happens when a car needs to enter your lane? (i.e. new information becomes available)

Transfer-Question: What is the fastest way to obtain business value in product development?

Concluding Question: Since minimal time-to-market maximizes ROI - which utilization strategy should you pursue?