Telemetry Canvas - figuring out the right metrics

To create transparency for the key information about your company, your system, your product - you need to align your metrics. Here is a simple canvas that can help you sort your thoughts and start your journey to data driven decision making:

The Telemetry Canvas

The canvas is simple to understand. There are two main dimensions:

Events

In an IT platform, there are mainly two types of events: Those created, conducted, orchestrated, managed or performed by automation - and those performed by humans (platform users).
Anyone whose work is affected by events should have a say in defining the most relevant items for their work.

Technical events

Everything the system and/or platform does by itself, or in support of user activity, is a technical event. We can measure technical factors such as incoming or completed transactions, inventory levels et cetera. Of course, we can categorize these by transaction type, technical component or business scenario, depending on what we are looking for.

Quality checks, build failures or network alerts are also technical events that occur frequently, and require attention.

User actions

Whatever users do may also be relevant to the performance of our organizations. If our goal is to grow our userbase, new registrations are a great metric to look at. On a marketing campaign, the next logical extension would be lead conversions. Or trending products. We might also look at revenue generation - and whatever has an impact on things we care for.
Even abandoning our product is relevant for our business performance, and can be classified as "action by inaction", potentially being a relevant user action.

Business outcomes

Events by themselves are meaningless. They derive their meaning by their impact on our business.

Good for business

We are looking for certain events, such as the successful start or completion of a transaction or the generation of revenue. Many of these events fall into the category of "The more, the merrier". The best events are those that cause no work, yet generate profits.

Bad for business

Some events are always bad news, for example complaints, technical errors or system outages. Even if nobody likes to have these, they are part of working reality, and we need to pay attention to the effort we sink into them.
In many organizations, the invisibility of the "bad news" metrics on the radar causes the organization to accumulate technical debt that may eventually kill the product or even the entire company!
The best businesses aren't those who successfully ignore the bad news - it's those who know that they have less bad news to handle than they can stomach!

Deriving metrics

Once we know which events we're looking at, we can determine how we measure them.

For example: When a transaction arrives in the system - we also want to know when it is completed: we measure not just our transaction rate and inventory, we need to know the throughput rate as well. This gives us visibility into whether we're accumulating or reducing backlog, whether we're sustainable or unsustainable!

Optimization

Once we have defined our metrics, we can set optimization goals. Some events are good for our business, others are bad. The general optimization direction is either "lower is good" or "higher is good". In rare cases, we have range thresholds, where neither too high nor too low is desirable.

The easiest way is to start by capturing data on the current state of a metric, then answering the question: "Is this a problem? If so, how big is it?" - determining whether the current value is good, acceptable or inacceptable.

Using the Telemetry Canvas

The canvas is a discussion facilitation tool, so don't use it on your own.

Step 1: Invite stakeholders

Bring all stakeholders in your product together, preferably not all, but representatives from each group. This is a non-exhaustive list of people you might want to involve:

• Salespeople, who generate income from the product
• Marketeers, who drive the product's growth
• Finance, who validate the product's revenue
• Developers, who build the solution
• Operations, who have to deal with the live system
• Customer Service, who have to deal with those who bought it
• UX, who design the next step
• Legal, who definitely don't like to have trouble with the product

The more of these functions rest within your team, the easier this exercise will be - although typically, most will be located somewhere else in the organization.

Step 2: Brainstorm events

Give everyone the opportunity to draft up events that are important to their work. There is no "right" or "wrong" at this stage, and there are no priorities, either.

It's important to remember that not all events occur within the platform, some occur around the platform, and that some events can also be caused by inaction.

Get people to write each event on sticky notes.

Step 3: Locate events on the matrix

People tend to have a pretty good understanding whether an event is good or bad, so where to place the event on the vertical should be easy. In some cases, it's unclear whether an event is good or bad - then default to "Bad", because every event means data processing and work, and work that's not good is probably a bad thing.

Likewise, define the horizontal category. In some complex systems, it's unclear whether it's a user action or a technical event. Try defaulting to "user action" - you haven't discussed where to get the data from, anyway.

Step 4: Define measurement systems

As events themselves are of no value, we need to define the measurements that we want to derive from events. These can also be combination metrics, such as "Lead Time" or "Inventory Growth". What matters is that everyone in the room can agree on what would be measured.

Write each of the measurements onto post-its and put them into the field corresponding to one of the event(s) they rely on.

Step 5: Prioritize

Not all metrics are sufficiently important. Let each stakeholder name up to three metrics that matter to them - you still need to put work into setting up data collection, and it doesn't help you to have five hundred things on your "toDo" list. 

This is not a point-based system, so it's not about dot-voting, so you end up with a bunch of individual priorities. 

Although it's good if multiple stakeholders value the same metrics, since that reduces complexity, it's not necessary that stakeholders agree on the value and importance of metrics.

Step 6: Validate

You should have a number of metrics in each quadrant now. If you're missing one of the quadrants, your measurement system is probably biased. Should that be the case, ask, "What are we missing?" Try reprioritizing metrics until you have at least two in each segment.

Step 7: Agree and align

Get everyone to agree that they have their most important metrics on the canvas. Address potential concerns. If necessary, re-iterate that this is not intended to replace current measurement systems nor a final version - it's just the beginning of a journey to align on data transparency.

Step 8: Invite for follow-ups

Once the metrics are agreed, let everyone know that there will be different sessions to define the metrics in more details, that is: how the data will be collected, how it will be interpreted and how it will be represented. This consumes more time and is not in full detail interesting for everyone.

Step 9: Agree on Next Steps

The Canvas is ready, but it's just a canvas. Make sure you have an action plan of what will happen next. Here's what is suggested:
1. followup sesssions for defining the metrics,
2. do some implementation work to measure them,
3. present the metrics in a Review,
4. start using the available metrics in decision making,
5. Inspect, Adapt and Improve.

Joint Metrics - aligning business and development

"How do we bring business and development closer together?" - the key is to create transparency into what others see.

Start the discussion about what you should make transparent, so that everyone can draw the same conclusions. Making both technological and business information visible to everyone in real time will help you cut down a lot of pointless discussions about the best course of action.

Everybody is right!

Every person has their perspective of what is the most important thing, and often, each perspective is valid. For example, a technical person will consider that technological stability and high quality code are important. Salespeople care for neither - they want to close as many good deals as fast as possible. People in service support feel stuck with tons of trouble tickets, social media marketers want campaigns to go viral. And the CEO just wants a smooth, expansive operation.

A developer can only use their time once. So - what should they focus on? How can a Product Owner know whether it's more important to boost sales or to fix defects?

Classic HIPPO Prioritization

Most organizations prioritize activities like this: Either we do the thing demanded by the person who shouts loudest, or the Highest Paid Person will give their opinion on what should be done ("HIPPO Priority").

Unfortunately, neither the people with the loudest voice, nor those with the highest paycheck, tend to have a full understanding on the implications of their demand. Follow the HIPPO, and everyone else will be unhappy. Disregard the HIPPO and risk being laid off.
Either way, the organization gets stuck under the tyranny of the Urgent - shifting attention between a series of disasters and escalations to fix.
There is no freedom to think of the Big Picture, maximize business value or consider what will happen a few years down the line.

A systemic view is needed

In a healthy organization, developers in their right mind wouldn't want sales to fail - and marketing wouldn't want the technology to fail. They are often simply unaware why their personal goals have such drastic consequences elsewhere!

The solution requires overarching transparency, laying all the cards on the table.
 In most modern organizations, there is some kind of data that people utilize, yet everyone gathers their data from a different source and interprets it in their local context. This is not to advocate a central Data Warehouse, Master Data Management or a specific data representation tool here - the problem can't be fixed technologically: A sysop would look at logfiles, developers at source code, sales at transaction records and marketing at campaign information.

None of the data is related on the surface. Yet, in a closed system, all of these are sides of the same coin (probably, an "infinity dice" would be a more applicable metaphor).

Breaking the local optimization

Every stakeholder can define metrics for their specific area of expertise. Sales is very adept in defining what is great, what's okay and what is intolerable when it comes to closing deals. Hence, it's very easy for them to define a metrical system that creates overall visibility on how healthy sales are. Developers can do the same for their systems, finance for revenue - and so on.

And then we lay all of this information on the table. When everyone has a say in what we're looking at, we have objectivity in whether we're doing great, alright or meh in the big picture. And make it visible to everyone.

Bringing the puzzle together

Imagine you log into your company account - and the first thing you see is where your company is doing great - and where it just plain sucks. From the customer service rep, all the way to the CEO, from technology to business, everyone will have at their fingertip the information where your biggest strength and where your biggest weakness is.

It will become very intuitive and easy to make key decisions, and even when diplomatic compromize is needed, people will at least understand the impact of their choices.

Leaving the hamster wheel

Many organizations are challenged to break free from the hamster wheel of tasks and activities. Product evolution is often nothing more than putting band-aids on cracked pavement. Systems are fundamentally broken, because it's all about meeting short-term goals, and rarely about larger long-term beneficial change. The future gets sacrificed for today's needs.

Planning strategically

We need to figure out where we're constantly fire-fighting, where we're in calm waters, and which problems correlate. We can use the transparency of the data to introduce measurable strategic objectives, such as, "reduce technical debt from 50000 years to 100 years", or "increase conversion rate from 1.2% to 2%". It's totally valid to have multiple strategies with multiple objectives and multiple targets in place at the same time.

Building empathy

Another great advantage of building a common metric portfolio for everyone in the organization is that we start to get empathy for one another. Developers see when sales is struggling, marketing realizes when development is drowning. The discussion moves away from "What do I need next?" towards "Who has the biggest problem and how can we contribute to improvement?"

The nonsense called "Enterprise Agile Development"

The "Manifesto for Agile Software Development" is the basis for many "agile approaches". Enterprises worldwide have been sold to the idea that they need to become "Agile" in order to remain competitive. And anecdotal evidence of the success of "Agile" is abundant.

There's a dirty little secret that most organizations, coaches and consultants are either unaware of, don't understand - or they just don't realize the impact thereof: "Agile", as originally proposed, is a local optimization, intended to improve the work of software developers! This begs the question: "What if Software Development isn't even the problem?"

Synopsis

If we look at organizational processes from end to end, we realize that even if Product Development were a flawless, instantaneous activity, not much would be different in the big picture. Why do we spend so much time and energy to make irrelevant changes?

95% of changes made by management today make no improvement. -Peter Scholtes

"Agile Transformation" is often one of these ineffective changes.
To find a better solution, we need to frame the problem differently.

The scope of "Agile"

"Agile" is intended to bring the entire IT development organization together. It's irrelevant whether we're talking Scrum, Crystal, XP or whatever, this idea is fundamental to "Agile".

From the time a "requirement" (or: "user story" - or whatever) is scoped for delivery until it's delivered, people from IT collaborate, involving business stakeholders, to minimize cost, overhead, quality risks and delay in the process.

Why "Agile" looks so appealing!

It sounds very appealing to any IT manager worldwide to reduce defect rates, cost and cycle times by margins of fifty percent plus each, while making IT employees happier, especially since those benefits can actually be achieved compared to siloed approaches!

But what if I told you that none of this matters at enterprise level?

The real picture

In a 100.000+ people enterprise, there are hierarchies, reporting lines, budgets and highly complex dynamics going on. Not everyone talks to everyone else.

In an enterprise context, it's fairly normal that from idea until approval, different rounds of experts, stakeholders and boards are involved - long before an item lands in IT's actual toDo list.

The idea goes through some kind of process, where people who are not involved in actual development clarify what's actually going on, whether it's worth doing and have to make sure it will get done.

While Scrum specifically has a "Product Owner" and Scrum proponents might argue that this is "refinement" for which the Product Owner has this responsibility - in an enterprise, that's too much work for one person, so it will be delegated: We end up with handovers in the process, multiple queues and waiting lines. 

This picture depicts what's actually going on in most larger organizations:

Careful - "Agile" doesn't even talk about the big picture!

The lead time of the yellow chevrons, in a Waterfall world, tends to be 3-6 months, and in an Agile universe, it's much shorter. 

Let's do a thought experiment: What if the yellow part of the process were conducted by a little fairy with a magic wand that could complete all these activities at zero cost, in zero time and without any defects? 

How agile, how fast, how cheap would this process now be?

In large enterprises, each of the gray chevrons takes weeks to months, is quite expensive and error-prone. Considering the end to end process chain, even if we disregarded everything happening in the "Agile" parts of the process - we'd still have a slow, inflexible and expensive process! 

The overall effect of introducing "Agile Software Development" into Enterprise processes is neglegible.

End-to-end Agility

Now, we get to the revolutionary idea of simplifying the entire process by means of moving to "Business Agility": bringing developers straight to the users!

A paradigm shift: entirely removing the men-in-the-middle!

Once an item is prioritized, we jump straight into development. There's just a small fly in the ointment: how do we prioritize it? In order to determine whether it's the most valuable thing, it still needs to be refined - so the same process still applies!

The problem, unfortunately, in the Enterprise world, is not even whether we have a handover in the process between a "Refinement Team" and a "Development Team". The real problem is overall lead time!

As long as it takes weeks to do the clarification rounds, the umpteen mandatory boards only meet once in a full moon, every new feature requires comprehensive user trainings and there are manual compliance audits, there is no way for a change in development to significantly affect overall outcomes!

Unless you're part of the problem, you're not part of the solution.

The illusion of improvement

When overall processes retain their massive overhead, and communication structures remain untouched, Agile Development alone is irrelevant to the overall business outcomes. Even if we get it right - which is extremely hard in the governance straight jacket so nicely provided by many organizations - the enterprise as a whole will not feel much of an impact by the introduction of "Agile Development" into a cluttered, overcomplicated jungle of processes, rules and regulations.

If we are looking for significant improvements to the enterprise, Software Development is often not the right place to look - yet agilists have spent nearly a decade developing increasingly intricate ideas for optimizing exclusively this part!

Moving Enterprise Software Development from "Waterfall" to "Agile"  is like using white sand instead of yellow sand to mix concrete - we feel as if we made a difference, while an outside observer wouldn't recognize any change!

And that's why Enterprises who have invested heavily into "Agile Software Development" often feel underwhelmed with the outcome and become entirely disillusioned.

A change in mindset

"Agile" brought a massive change in mindset, it gave developers a voice - and it's people from Technology who are now educating the business world how to change their ways of working, in order to take advantage of Agile processes.
Unfortunately, we see that many agilists are stuck in their little Technology world, not realizing that optimizations made exclusively from a Technology perspective alone are exactly as ineffective - and potentially equally harmful - as those made by Finance (namely, Cost Accounting - but that's another story, to be told another time).

Much more important than doing an "Agile Transformation" and converting legions of decently functioning teams with decently performing individuals to "Agile ways of working" would be to look at the real problem the organization is facing - it's not a development problem. It's a flow problem, in end to end value.

I believe that not just agilists, but struggling organizations as a whole, will find significant value in moving beyond the new structure and processes suggested by Agile Frameworks and tackle the real issues their organization is facing, which are hidden in plain sight when adopting Agile Frameworks.

In the past years, I have come to realize that we need to move beyond IT, beyond Technology, beyond Product Development, to see the need for a Unity of Purpose, optimization at the bottleneck and relentless improvement. Try not making any change to anything other than the bottleneck: Development can work however they consider fit, unless and until that's where the bottleneck is.

Closing remarks

Let's just not make changes that don't matter. Too much harm and grievance has been inflicted upon managers, developers and entire organizations in the name of this "better way of working". So many "Agile Transformations" caused great people to lose their job, their sanity or the respect they deserve, while giving the organization nothing to show in return. It's time to stop this madness.

Let's put Agile Frameworks where they perform best: into Product Development. Let's not try to stretch their purpose beyond their applicability, and let's not overzealously convert everyone to one specific way of working. Give people the freedom to work in whatever way they consider is best for them, Offer people Scrum, Kanban, SAFe or LeSS as an option if it makes them happier - but don't force anyone into any of this, especially not when development isn't even the problem.

The value of idle time

The Product Backlog is a mandatory part of Scrum. Together with the Sprint Backlog, they define both the planned and upcoming work of the team.

There's a common assumption that it's considered good to have a decently sized product backlog, and as many items in the Sprint Backlog as the team has capacity to deliver. Let's examine this assumption by looking at a specific event.

The "no backlog" event

It was Tuesday evening. I had just a busy day behind me.
I was chilling, browsing the Web, when I received a message on LinkedIn. The following conversation ensued:

Mind Lukasz' last statement: "The most impressive customer service ever".
Why was this possible?

Had Lukasz contacted me half an hour earlier, this dialogue would never have happened. Why? Because I would have been busy, doing some other work. Lukasz would have had to wait. His request would have become part of my backlog.

Service Classes

There's a lot of work I am pushing ahead of me on a day-to-day basis.
But I classify my work into three categories:

1. Client related work - I try to cap the amount of client related work, to maintain a sustainable pace.
   It's a pretty stuffed backlog where things fall off the corners every day.
2. Spontaneous stuff - I do this stuff as fast as I see it, because I feel like doing it.
   The hidden constraint is that "as I see it" depends on me not being engaged in the other two types, because these take 100% of my attention.
3. Learning and Improvement - That's what I do most of the time when not doing Project work.
   I consider web content creation an intrinsic part of my own learning journey.

These categories would be called "service classes" in Kanban.
I am quite strict in separating these three classes, and prioritize class 1 work over everything else.

Without knowing, Lukasz hit my service class 2 - and during a time when I was indeed idle.
Since class 1 has no managed backlog, I got around to Lukasz' request right as it popped up, and hence, the epic dialogue ensued.

Service Classes in Scrum

If you think of the average Enterprise Scrum team, class 1 is planned during Sprint Planning, and class 2 activities are undesirable: all the work must be transparent in the Sprint Backlog, and the SBL should not be modified without consent of the team, especially not if this might impact the Sprint Goal.

Many Scrum teams spend 100% of their workload on class 1, going at an unsustainable pace, because the constantly descoped class 3 work warrants future-proofing the work.
Even if they plan for a certain amount of class 3 work, that is usually the first thing thrown overboard when there's pressure to deliver.

The importance of Spontaneity

Few Scrum teams take care of Class 2 work, and Scrum theory would dictate that it should be placed in the Product Backlog. This just so happens to be the reason why Scrum often feels like drudgery and developers are getting uncomfortable with practices like Pair Programming.

"Spontaneous stuff" is a way to relax the mind, it helps sustain motivation and being totally uncommitted on outcomes allows creativity to flourish.

Load versus Idle Time

As mentioned, class 1 is bulk work. As workload increases, the percent amount of class 1 activity quickly approaches 100%. Taking care of class 3 activity means that increasing load quickly diminishes idle time activity to go to Zero.

Since I already mentioned that idle time activity creates magic moments, both for team members and customers, high load with zero idle time destroys the "magic" of a team.

Wait Time Idleness

One source of Idle Time is Process Wait Time.
In a Lean culture, wait is seen as detrimental waste. This is both true and false. It is true when the organization doesn't create value during wait, while incurring costs. It is false when this wait is used to generate "magic moments".

Buffer Time Idleness

Both Scrum and Lean-Kanban approaches encourage eliminating idle time, as would the common "agile Scaling" frameworks. Team members are constantly encouraged to pull the next item. or help others get work in progress done faster.
This efficiency-minded paradigm only makes sense if the team controls the end-to-end performance of the process, otherwise they might just accumulate additional waste. Theory of Constraints comes to mind.

On the other hand, buffer removal in combination with a full backlog disenchants the team - there will be no more "magic moments": Everything is just plan, do, check, act.

Idle Time and Throughput

The flawed assumption that I want to address is that buffer elimination, cross-functionality and responsibility sharing would improve throughput. Maybe these will increase output, but this output will be subject to the full lead time of any other activity.

Backlogs vs. Idle Time

Genuine idle time means that the input backlog currently has a size of Zero and a parallel WIP of Zero as well. There is no queue: neither work-in-progress nor work-to-do.
An idle system doesn't require queue management. When idling, throughput for the next request is exactly equal to work time - the maximum throughput speed we could hope to achieve. This kind of throughput speed can look absolutely mind-boggling in comparison to normal activity cycle times.

The impact on organizational design

A perfect organization takes advantage of idle time points that maximize throughput speed - not efficient utilization avoiding idle time.

Summary

The conversation with Lukasz is an example of the benefits of having idle time in your work.
This kind of idle time allows for "magic moments" from a customer perspective.

Just imagine an organization where "magic moments" are the norm, and not the exception.
This requires you to actively shape demand: when demand is roughly equal to capacity, we can eliminate backlogs.
Demand queues destroy the magic.

Eliminate the queues. Make magic happen.

Scrum is setting you up to fail!

The amount of debates where agilists claim, "But Scrum addresses <this topic> already!" - then proceed to quote a sentence, or even a single term from their framework's rules are staggering. The phrase, "we need to be pragmatic, and Scrum is idealistic" heats up the debate.

My take: 

In some cases, frameworks like Scrum are helpful. By themselves, however, they aren't. They provide no helpful guidance and rely on the strong assumption that the solutions to an organization's core problems already exist within the team' sphere of control. 

This assumption is borderline insane, because people wouldn't need a rule or framework for something they know how to do.

Even in regards to my article about demand, I got the reply, "Scrum does address the issue. That's what you got a Product Owner for." and "SAFe uses the term 'Demand Management' at Portfolio level, therefore SAFe has a solution." - I say that this is about as helpful in practice as stating, "We have the cure for cancer already. That's what scientists are for: They even use the term cancer research."
Yes. And: What exactly is the solution to the problem beyond assigning responsibility or attaching a label somewhere?

Let's focus on Scrum, just to be talking about something specific.
In all fairness, many Scrum practitioners state, "Scrum doesn't solve your problems, it only highlights them" - which is my answer to everyone who would claim that "Scrum does address this already." Maybe you get a label. You don't get a solution. Scrum itself has no helpful answers, not even the hint of a direction.

Scrum's dangerous assumptions

Scrum makes a lot of strong assumptions. Most of the time, these assumptions are just not valid and will cause a Scrum adoption to shipwreck.
These are all examples of conditions that Scrum is simply assumed to have:

No blocking organizational issues

Scrum can only work when the surrounding organization is at least basically compatible with Scrum. Scrum's assumption is that you are well aware of how to ensure that:

• Organizational processes are fundamentally compatible with agile development
• A meaningful portfolio strategy exists
• Demand funneling "somehow works"
• Individual incentive schemes don't get in the way of team or organizational goals
• The organization improves where it matters
• You have stable teams

And what if not?

Unproblematic contracts

Scrum teams must operate in an environment where developers and customers share common goals, and developers are contractually enabled to maximize organizational value. Scrum assumes that you have a contract situation where:

• There is no functional split between different organizations (e.g. outsourced manual test - or worse, outsourced users)
• Financial incentives encourage optimizing around value rather than activities
• The team meets all legal requirements to deliver all required components
• The development organization benefits from producing better / more / faster outcomes

And what if not?

People get along

Scrum assumes people can and will communicate with a goal to create value.
You have to know by yourself how to achieve the following states:

• No communication gaps where significant information gets lost
• Stakeholders care and show up to provide essential feedback
• Managers understand and avoid what demotivates the team
• People have a sufficient level of trust to raise issues and concerns
• When all things fail, people focus on learning and improvement, avoiding blame.

And what if not?

Development issues

Since its inception, Scrum has removed all aspects of technical guidance. As such, there's now the hard assumption that:

• Teams have the necessary skills to produce a "Done" Increment
• Teams know about quality engineering practices
• The team's software isn't a steaming pile of ... legacy
• Teams are able to produce a meaningful business forecast
• Teams can cope with technology shifts

And what if not?

The danger of these assumptions

To assume that none of these problems exist is idealism. If you make these assumptions, you will shipwreck.
To assume you can safely operate Scrum when multiple of those problems exist, you're also going to shipwreck.
To assume that attending a Scrum training course equips you to take on this gorilla is also going to shipwreck.

To assume that Scrum has a solution to any these problems is false hope or snake oil, depending on perspective. Scrum assumes that they have already been solved - or at least, that you well know how to solve them. Scrum tackles none of them.

What if not

The Scrum Guide has no guidance on any of these topics, as all of these problems are assumed to be manageable and/or solved in a Scrum context.

Where these problems are significant, Scrum isn't the droid you're looking for.

Health Radars are institutional waste!

There's a recent trend that organizations transitioning to agile ways of working get bombarded with so-called "health checks" - long questionnaires asking many questions, that need to be filled in by hundreds or maybe even thousands of people in short cycles. They deprive organizations of thousands of hours of productivity, for little return on this invest. 

Radar tools are considered useful by consultants with little understanding of actual agility. 
My take is that such tools are absolute overkill. What you can do - to save time and effort, and get better outcomes.

The problems of health radar tools

Health radars are deceptive and overcomplicate a rather simple matter. They also focus on the wrong goal.

A radar is only helpful when things are happening outside where you could otherwise see them.

If an organization wants to be agile, the goal should be to improve line of sight, not to institutionalize processes which make you comfortable with poor visibility.

The need for a radar reveals a disconnect between coaches/managers and the organizational reality.

Early transition radars

When an organization doesn't understand much about agile culture and engineering practice, you don't need a health radar to realize that this isn't where you want to be: time-to-market sucks, quality sucks, customer satisfaction sucks, morale sucks. No tool required.

Initial health radar surveys usually suffer from multiple issues:

• Culture: Many traditional enterprises are set up in a way that talking about problems isn't encouraged. The health radar results often look better than reality.
• Dunning-Kruger effect: people overestimate their current understanding and ability, as such, overrate it.
• Anchoring bias: the presented information is considered far more reliable for decision making than it is.

I don't think it needs much further explanation why taking a health radar under these conditions can actually be a threat, rather than a help.

Repeat surveys

The next problem with health radars is that they are usually taken in cyclical intervals, usually ranging from monthly to quarterly. Aside from people starting to get bored having to answer the same fifty questions every month (oddly enough, agile development would encourage automating or entirely eliminating recurrent activity!).

Frequently repeating the surveys thus suffers from:

1. Disconnect between change and data: Especially in slow-moving environments, the amount of systemic change that warrants re-examination of the state tends to be low, so the amount of difference over time that can actually be attributed to actual change in the system is low. 
2. Insignificant deltas: Most change actions are point-based optimizations. Re-collecting full sets of data will yield changes that are statistically insignificant in the big picture.
3. Fatalism: When people see that there are dozens of important topics to be changed, and that progress is really slow, they might lose hope and be less inclined to make changes.
4. Check-the-box errors: With increasing frequency of surveys, more and more people will just check some boxes to be done with it. The obtained data is statistically worthless. It might even require additional effort to filter out. Likewise, the consequently reduced sample size reduces the accuracy of the remaining data.

Those are the key points why I believe that constantly bombarding an entire organization with health radars can actually be counterproductive.

A much simpler alternative

With these four rather simple questions, you can get a clear and strong understanding about how well a team or organization is actually doing:

Sometimes, those questions don't even need to be asked. They can be observed, and you can enter the conversation by addressing the point right away.

The four questions

To the observant coach, the four questions touch four different domains. If all four of these domains are fine, this begs the question: "What do you even want to change - and why?" - and taking a Health Radar survey under these conditions would not yield much insight, either.

Usually, however, the four questions are not okay, and you can enter into a conversation right away.

1 - Product Evolution

The first question is focused on how fast the product evolves.

If the answer is "Quarterly" or slower - you are not agile. Period.

Even "daily" may be too slow, depending on the domain. If you see inadequate evolution rates, that's what you have to improve. 

And don't get misled - it may not be the tool or process: it may be the organizational structure that slows down evolution!

2 - User attitude

The second question is focused on users.

If the answer is, "We don't even know who they are" - you are not agile. Period.

Some teams invite selected users to Reviews, although even this can be deceptive - having an informal chat with a real user outside a meeting can be revealing indeed.

3 - Developer attitude

The third question is focused on members of the development organization.

If the answer is anywhere along the lines of "I'm looking for job offers" - you are not agile. Period.

Sustainable development can only be achieved when developers care about what they do, are happy about what they do and willing to take the feedback they receive.

4 - Continuous Improvement

The fourth question is focused on how improvement takes place.

If the answer is along the lines of "We can't do anything about it" - you are not agile. Period.

People need to see both the big picture and how they affect it. The system wouldn't be what it is without the people in it. The bigger people's drive to make a positive impact, the more likely the most important problems will get solved.

The core of the matter is what people do when nobody tells them what to do. Until people have an intrinsic drive to do the right thing, you're not going anywhere.

The conversation

Depending where you see the biggest problem, have a conversation about "Why": "Why are things the way they are?" - "Why are we content with our current situation?"- "Why aren't we doing better?" - "why do we even want to be agile if we're not doing our best to make progress here?"

People can have an infinite amount of reasons, so this is the perfect time to get NEAR the team and their stakeholders.

Following up

The followup set of questions after a prolonged period can be a series of "What" questions: "What's different now?" - "What have we learned?" - "What now?"

Summary

Drop the long questionnaires. They waste time, capacity and money. 

Learn to observe, start to ask questions. Reduce distance in the organization.

You don't need many questions to figure out what the biggest problem is - and most of all, you don't need to "carpet bomb" the organization with survey forms.  Keep it simple.

Often, people know very well what the problems are and why they have them. They just never took the time to get things sorted. All you need to do is help them in understanding where they are and discovering ways forward. 

Let's talk about Demand

While "Agile Development" focuses exclusively on supply mechanics, an understanding of demand mechanics is essential to managing sustainable product development - agile or non-agile, hence I want to lay a foundation of the effect demand has on a development organization.


Synopsis
A basic understanding of the economic dynamics of Supply and Demand is essential to succeed with product development. As development capacity is a limited resource, free market dynamics do not apply to product development. Without understanding and managing demand, agile product development sets itself up for failure, yet few (if any) agilists understand "demand". This article provides a basic explanation of the impact demand has on your development organization - and how you can use this knowledge to increase your odds of succeeding.

Demand in economics

There are huge misunderstandings in the agile community as to what is "demand" - down to the point where people claim that demand management isn't anything other than working the Product Backlog. Since we have ITIL, a lot of IT people seem to confuse "Demand Management" with "Requirements Management" - and conflate "demand" with "requirement", or, using agile lingo, "user stories".
Let's get back to basics for a bit.

The most fundamental concept of Keynesian economics is Supply and Demand. In a free, unrestricted market, this is the gist of Supply and Demand:

Where supply and demand meet, there's an equilibrium

"Demand" is simply how much of a given good or service people request - and "Supply" is how much of that will be available. There is also "aggregate supply" and "aggregate demand" for the whole range of good or services within an economy.

This model leads to some very intuitive statements:

• As price decreases, demand increases. People demand more at a low price.
• As price increases, supply increases. People are more willing to offer things at a high price.
• At some point, there's a market equilibrium: supply meets demand.

To make this article simpler, from now on, I will infer "good" to mean "any kind of service provided by IT development", "quantity" to mean to "the amount of this good [created or requested]" and "price" as "cost of that good". I will ignore the definition of "price" as in whether that cost is opportunity cost, total cost of ownership or whatever - that too, might be a topic in and of itself. For now, just think of the most comprehensive definition of "price" you can imagine.

While macroeconomic concepts can not be applied to a division of an organization without further examination of the existing conditions, this would go significantly beyond a short blog article. To keep matters simple: The development organization (typically IT) offers certain services ("supplier") to the rest of the business and to customers ("consumers"). From there, we get into the nitty-gritty that may be worth another article.

In this context, "demand" is simply everything that consumers want, which the supplier could provide. There is no constraint that "demand" needs to be known or voiced. It can only be itemized once it's known and articulated. Therefore, demand starts long before a specific item appears in a Product Backlog.

Scarcity and Supply Limits

When a resource (in this case: time) in the production of a good is limited, this generates "Scarcity": the quantity of the good is capped due to the supply limt of the resource.

Beyond the supply limit, price is irrelevant. The demand will not be met.

On the right side of the supply limit, price and demand are irrelevant: There is no way to meet the demand - it's economically impossible. Just imagine that you wanted two suns on the sky - there is only one, so it's simply irrelevant how much you would pay to have a second - there won't be another.

We see the same thing happening when a development team (or: the development organization) has a fairly fixed size. You can slice it however you want, a day still has only 24 hours and 10 people are still only 10 people, so there is some kind of limit in place.

Let us simplify our model once again and ignore the possibility of hiring additional people, because we would still need to observe two effects here described quite appropriately in "The Mythical Man-Month" and "The Phoenix Project", that there's no guarantee that additional headcount will increase the supply limit. Even if we could increase headcount and developers were fungible, there would be still diminishing returns and scarcity of available developers, another topic that we can wonderfully descope for another article.

Increasing demand

As our business grows more successful over time. the demand for goods from the development organization increases. Whereas in the initial phase of our business, developers were experimenting a lot and had a fair amount of time to figure out the best solution, increasing demand brings development closer to its supply limit.

While a low demand can be met at a low cost, there is no way to meet the high demand beyond the supply limit that many development organizations experience.

As we get closer to this supply limit. businesses get the impression that their development becomes slower, less flexible and unproportionally costly. Close to the supply limit, the marginal price increases dramatically with ever diminishing returns on invest.

As demand increases, cost increases significantly faster than quantity

The average unit price of getting a new feature also goes up - not because development cost went up, but because of increasing competition over a limited good. Notice that this is demand competition, not supply competition. Demand competition means that people will use their influence and available means to get what they want, so the people with less influence or money will not be supplied.
Or, to translate the impact of demand competition into the world of product development: At some point, the development organization will no longer be able to serve everyone.

If now we translate this into the world of software again: as businesses become more successful, their demand for IT continuously increases. When a specific request arrives in the development organization, it's either on a point in the demand curve where supply can meet demand - or it isn't. If it is not, then it will not be in the future, because demand is moving up ...

A very simple rule of thumb in a limited supply system is: Unless demand decreases at some point, if you can't get it now, you never will.

... unless...
... demand decreases!

Demand Inflation

Which it won't, because everyone intuitively figures out this rule of thumb. As a consequence, we see a followup phenomenon: When stakeholders start to realize that their requests aren't being met, they try to get the development organization to commit to delivering as many goods as eary as possible, because later, it will be even harder to get this commitment!

This is the point where we enter the vicious circle of starting as many initiatives as possible, irrespective of whether there is enough capacity to complete these initiatives. The capacity problem is delegated to the product organization once there's commitment to deliver upon a certain request.

This vicious circle can't be broken on the supply side

The mechanics we see at work here are exactly the same mechanics that you saw during the World Banking Crisis: As soon as a bank declares that they can't pay back everyone's deposits, everyone will try to be the first to secure as much of their deposit as possible.
The announcement that "we can't serve everyone" is enough to start a spiral so powerful that it can destroy corporations and topple governments - what makes you think that your Product Owner will fare better?

Demand Management

A missing piece in the typical agile approach is that typical IT people understand little to nothing about supply and demand - and even less about demand managment.

Demand management isn't the exercise of itemizing and keeping an inventory "user stories" or other backlog items, and determining which items get implemented next (and which won't get implemented at all).

At the very minute a stakeholder receives the idea that maybe the development organization can't do it all, you start a "bank run", inviting all stakeholders to flood you with as many requests as possible, worsening your problem rather than fixing it!

Demand Shaping

A terrible misunderstanding propagated by many agile coaches and trainers is the idea that it's enough to manage the supply side of product development and have a mechanism to determine the priority and content of the product backlog, then being vocal about saying, "No" to requests for goods that have no chance of being implemented.
Simply managing the product backlog is too late in the process and a local optimization.

Let me give you an example of what "demand shaping" means:

The Diner Metaphor

You go into a Mexican diner to have lunch. The diner has already shaped your demand long before you looked at the menu.  You would be irrational to request breakfast for lunch, and you would be even more irrational to request a sushi platter there.
The restaurant probably won't have to tell anyone that both of these requests won't be served, because nobody expects to get served these dishes to begin with. Nobody would feel offended when the waiter tells a person with a sushi craving that there's an excellent sushi bar down the road.

If the diner is high class, they will not serve you a "product backlog", or a list of 150 different dishes that you can choose from. Their menu will contain maybe four to ten items, all of which the restaurant manager knows to be fast and profitable to produce at a high quality.

The next step of demand shaping happens when you approach the door and see this giant sign "Lunch Menu $6.99"  with some really tasty looking food imagery. At this point, most customers will already have lost their interest in the menu and place their order. The placement, the imagery and the price tag are enough to reduce the demand for other items by a good 80% - speaking from a Pareto Principle, the sign plus their brand is enough to give the diner what can be called "demand control".

At the same time, the manager will constantly track which dishes get ordered and if there are new trends on the market, without any direct intention to provide any of these. Even if the customers are aware of this, they have no way to influence the manager's decisions as to what the future menu will look like beyond what they choose to order. Next week's special offer and menu will be optimized to maximize sales, while minimizing effort, and if the manager makes the right predictions, customers will be happy even though they were never asked for their opinion.
This is called "demand monitoring" and "demand prediction".

The failure of IT Development

Whereas the Mexican Diner in our example made it clear that they're not a Sushi Bar, a typical company's IT department wants to have full and exclusive control of everything pertaining to Information Technology.
All of this is operating under the assumption that the the demand on a development organization can be met by the available supply.
Once demand is beyond supply, which is a natural consequence of having a successful product with unshaped demand, the organization must choose which demand it will serve and which it doesn't.

There are multiple strategies for dealing with over-demand:

1. Start the "Demand Inflation" spiral and get into a neverending battle about priorities.
2. Encourage a "Bank Run", which will destroy the credibility of your organization.
3. Start "Demand Shaping" to reduce over-demand.
4. Specialize. Stop claiming to do everything. Let others do what they can.

Smart organizations, irrespective of their development approach, will pick option 3 to reduce demand as far as possible, then pick option 4 once that is no longer an option.

Agile Frameworks like Scrum and SAFe will lead unsuspecting Product Owners/Managers into the trap of picking options 1 and 2, which may eventually lead to a failure of the entire organization.

The meta-failure of "Agile"

Managers who look towards "Agile" as ways of improving Time-to-Market and Quality easily get swayed by the claims that "Agile" will improve these metrics, while also increasing employee engagement and customer satisfaction. Hence, they transition their organizations towards agile ways of working. Product Managers get re-eductated to follow "Agile Product Management Practice", which focuses exclusively on short-term supply management and entirely ignores long-term demand management.

As a sweeping statement, agilists with an IT background are terrible at understanding market dynamics and the nature of demand. They only understand the supply side, as that's what they're typically working on. Hence, they can't help on the demand side. Letting a myopic supply-centric approach become your product strategy will be disastrous.

If your "Agile Transformation" has no demand management strategy, or if you think that "let's put everything into a backlog, then pull the most valuable items first" is a demand management strategy, you are going to shipwreck! The bank run is inevitable.

Conclusion

"Agile frameworks have little to nothing worthwhile to offer in terms of demand management. They focus exclusively on managing the supply."

Understanding and managing demand is an important part of managing a sustainable development organization. Agile frameworks have little to nothing worthwhile to offer in terms of demand management. They focus exclusively on managing the supply. Claims that demand control is not required in an "Agile" environment are myopic, made from ignorance - and can lead to catastrophic outcomes in the long term.

Organizational managers and product people alike need to understand demand management principles and practices to steer a product towards sustainable success.  An understanding of the "market" created and engaged by the development organization is essential to determine what the best next steps are.
In some cases, backtracking and demand reduction may be required before one can even begin working with a Product Backlog. Failure to understand this may result in the entire "Agile Tranformation" becoming a no-win scenario for all people involved.

To Explore

This article relies on a lot of shortcuts and makes a number of strong assumptions.
A number of additional items still need to be explored, and I invite the reader to do some of this exploration on their on until I have time to provide additional material.
These items are:

• The applicability of Keynesian Economics to [IT] product development
• The definition of a "good" in the context of [IT] product development
• The definition of "price" / "cost" in the context of [IT] product development
• The Supplier / Consumer Relationship in the context of [IT] product development
• The interaction between product development and non-developmental IT
• Why "Development Manpower" is not a fungible good
• Why a "Supply Limit" exists
• The causes of demand increase in product development
• The causes for starting a bank run on product development
• The effects of a bank run on product development
• Demand shaping in software development
• The effect of demand shaping on development performance
• The effect of having "cross-functional teams" on the complexity of the work
• The trade-offs, benefits and disadvantages of specialization

Why "Agile Development" will not solve your problem

Do we even need "Agile" to improve time-to-market, reducing cost and increasing success rates? After reading this article, you may have doubts.

Do these statements sound familiar to you?

• Software Development is too expensive
• We are lagging years behind the demand.
• Too many initiatives (projects etc.) fail.

The solution? Scrum. No, just kidding. Scrum doesn't have a solution there. Neither does LeSS or SAFe. Or XP, or any other "Agile Framework" that focuses on the development part of an organization. The solution rests in how we think about organizing our products long before developers actually get involved.

Too many "Agile Transformation" initiatives are focused on the output of software development and forget the big picture of the system in which they are operating. 

The Product Development Funnel

Take a look at this abstract development funnel. Every organization works somehow like this, while the slope of the funnel and the processes in each stage may vary widely. 

Let us examine the core concepts of this funnel.
To keep this article at least somewhat simple, let us equivocate "product" and "product portfolio" - because from a certain level of abstraction, it's irrelevant whether we have one product or an entire portfolio thereof: both capacity and capability are still finite.
Likewise, let us ignore the entire dispute around Lean/Agile vs traditional program and/or portfolio management, because irrespective of how you do it, you are still dealing with demand, choosing opportunities and prioritizing effort.

Customer Demand

Whether our customers are people out there on the market who purchase our product, or internal stakeholders of our product, there are always unmet needs. The amount of unmet needs that our product could meet is the amount of potential work that is still required, and that's potentially infinite - unlike our funding and capacity.

At the expense of taking a grossly oversimplified definition here, let us define "Marketing" as the effort invested into making others aware of potential needs and wants - and in return, becoming aware of what these are.

Oddly enough, even delivery is a marketing activity in some sense: as people use the product, they become more aware of additional needs. Hence, the old economic rule, "supply creates its own demand" ensures that the demand part of the funnel will never dry up in a well-run product. 

Development portfolio

Every demand that's deemed sufficiently important will eventually end up in some kind of portfolio, the known list of undone upcoming work. This is where the first stage of filtering occurs. Filter mechanisms include product strategy, economic viability or even internal politics. The specific reasons for filtering are irrelevant in this context.

Every item that lands in the portfolio will subsequently put stress on the development organization. In an odd twist of fate, the worse an organization's ability to manage the portfolio, the lower the satisfaction with the developed product will eventually be, irrespective of how good or bad development was.

The reason is simply because once the capacity of the development organization is exceeded, accepting further demand will not yield better outcomes.

Running Initiatives

Once a portfolio item has been scoped for development, it will become a development initiative. Organizations do have this habit of finding it easier to start initiatives than to complete them. Every initiative started will eventually result in one of three states: Live (successfully delivered), Dead (descoped without delivery) - or Undead (lingering in a limbo state where people are waiting for results that may never come).

Let's briefly examine Live and Dead: An initiative that goes "Live" is the only type of initiative that results in something called "Business Value", or even potentially positive Return on Invest. It's therefore highly desirable to get initiatives live.

A Dead initiative is an initiative that has been terminated before it has generated business value. Every cent spent on dead initiatives was waste. Completing an initiative with a negative ROI is just throwing the good money after the bad, so killing them is still the better alternative.

The biggest problem though, are neither Live or Dead initiatives - it's the Undead: An organizational culture that doesn't have the willingness to kill initiatives that have exceeded their life expectation will generate and accumulate undead initiatives. 

The problem with these undead initiatives is that they drain your organization's energy. People spend time and effort to work on and track the Undead, and this time isn't available to get other initiatives Live, so they turn otherwise healthy initiatives into Undead as well. Talk about Zombie Apocalypse!

The health of a product development organization can easily be measured by how many Undead Inititiatives they permit - every initiative ever started that was not clearly terminated or completed. The easiest, yet oftentimes most uncomfortable, fix to increase this health is by taking inventory and killing all the zombie initiatives.

Delivery Capability

The delivery capability is the one place where a common "Agile Transformation" focuses. The expectation is that somehow, magically, the adoption of daily standups, cadenced plannings, reviews and retrospectives will improve performance. It doesn't. (At least, not significantly - unless your developers are really stupid, which I claim they aren't.) And, on a more abstract level, the expectation is that this approach will somehow improve the ratio of met versus unmet demand. It doesn't.

It might look like a good idea thus to increase the capacity of the development process, i.e. add more developers or provide better tools and extra resources to make developers more efficient. It isn't.

What sounds good in theory often results in proportionately increased expectations: When development capacity is increased through additional funding, the organization will expect to be able to launch more initiatives - and thus, the problem persists. It might even get worse, but that's another story, to be told at another time.

Retrospectives and change initiatives focused on "how can we do better in delivering" might completely miss the point if the development organization is in a state of continuous overburden: the overburden will never go away as long as more demand enters the system than what can be delivered.

Everything these frequent events accomplish is that the constant pressure to deliver creates psychological stress and eventually developers burn out: Development performance may even decline!

Let's talk about waste

In this section, let us move backwards through the funnel, as the waste accumulates throughout the funnel, like a clogged pipe. We have to unclog it where the blockage is caused, rather than where it occurs.

Wasted development effort

What should developers spend time on? Developing working solutions, obviously. Yet, developers often spend significant portions of their time doing things not related to this. Let's take a crude tally:

• Status meetings for all those running initiatives.
• Especially the zombies.
• Task switching effort between those initiatives. 
• Waiting for input on an initiative.
• Catching up with changing requirements in an initiative.

The sum of all the effort invested into these items is the "low hanging fruit" in any optimization strategy. Reducing the amount of running initiatives reduces the amount of wasted development effort, thus increasing available capacity to get meaningful work done.

Kanban addresses this by limiting Work-in-Progress (WIP), although this only helps when we limit WIP at a higher level, that is: we must control the influx of initiatives, not the amount of development tasks, in order to gain any benefits.

Scrum addresses this by limiting the amount of items a team takes on during a Sprint - however, this is not a fix when more portfolio initiatives get started than completed: the consequence is an ever-growing product backlog. The problem is hidden, not solved.

In the ideal Scrum, the Scrum Team has full control over the entire product development funnel. Since this would require the organization to have moved from cost accounting to throughput accounting, from funding work to funding teams and towards an integrated cross-functionality that puts business acumen into the Scrum Team - this is a type of Scrum most organizations don't have. Again, this implies that the improvement potential then doesn't relate to delivery.

Coordination Waste

The second stage of organizational waste occuring in product development is the coordination of initiatives.  That's where we get project managers and matrix organizations from, and it's the reason why developer's calendars are cluttered with appointments. As every initiative has someone in charge, for simplicity's sake, let's call them "Initiative Owner". This initiative owner wants to be informed about what is going on and will take care that their initiative gets due attention.

If there is only one initiative, then all of an organization's efforts can focus on completing this one initiative.

Add another initiative, and the organization has to solve the problem of coordinating both the efforts between those initiatives - and to solve blockages in either initiative without blocking any other work. The effort required for these activities increases exponentially in complexity with each ongoing initiative. 

This coordination overhead would not even exist if there was only one initiative. The time and money sunk into coordinating parallel initiatives is pure waste.

At this point, we already need to ask the question why we see value in having multiple parallel initiatives - and why we believe that the value of starting another initiative outweighs the waste caused thereby. Likewise, we can ask the question whether the reduction of initiatives reduces the waste by a level that warrants deferring, descoping or even discarding one or more initiatives.

As coordination waste is multiplicative on outcome throughput, the lever of optimizing coordination waste outweights the lever of optimizing team-level productivity.

Budget Waste

Every organization determines one way or another what will be developed. Regardless if this happens via a Product Owner prioritizing an item in a product backlog or a SteerCo starting a traditional project, at some point a decision will be made to start a new initiative. In one form or another, at this point, the organization makes an investment decision: budget is allocated to the initiative. The biggest waste of budget is to start an initiative that doesn't get completed. Any initiative started without sufficient delivery capacity to complete both the currently running and the new initiative, will predictably induce waste into the organization. 

The simplest form of reducing budget waste is by deferring the decision about starting an initiative until there is sufficient free capacity within the organization to complete this initiative without impacting any other initiative.

As budget waste yields coordination waste, and coordination waste yields capability waste, the lever of reducing budget waste is even stronger than that of reducing coordination overhead.

Marketing waste

Probably the most well-hidden form of waste is demand waste, or "Marketing waste". Any form of demand that isn't eventually met by the product generates waste. The discovery, itemization and exploration of this type of demand generate cost without value. Having more demand than one can meet is only good because it creates options, yet one needs to be careful lest one loses focus in all of these options. There's even the economic dilemma of demand pull inflation where the mere prospect of additional demand will increase cost, but that's another story, to be told another day.

The simplest way of decreasing marketing waste is by limiting demand to a level where supply is at least still sensibly correlated to demand, and the organizational processes of managing demand are low effort. 

Marketing waste propagates into the product organization at full force: Poorly defined or invalid value hypotheses block scarce downstream capacity, while an oversupply of demand leads to pressure on the system. 

Limiting the waste

As heretical as this sounds, the most effective way of improving a development organization has nothing to do with improving development - and that's where all the team level agile approaches go wrong.

The most effective strategy for increasing effectiveness is by pulling the longest levers first:

1. Limit the amount of initiatives within the development organization. To achieve this:
2. Limit the influx of approved initiatives. To achieve this:
3. Limit the rate at which demand is translated into initiatives. To achieve this:
4. Limit the influx of demand into your organization.

"Do not try to do everything. Do one thing well."

- Steve Jobs

Here is an overview of what an organization's development funnel would look like if all settings were optimal:

The optimization in this context focuses on reducing the Organizational Waste not related to delivering Useful Products, and on the decrease of Opportunity Cost by eliminating the waste associated with pursuing low-value demand.

When we take into account Little's law, we realize that this optimization approach achieves:

• Reduced cost of meeting demands ("feature cost savings")
• Reduced Time-To-Market for features
• Increased Success rates of initiatives

Depending on how significant the associated waste and opportunity cost within an organization currently is, the leverage may be massive, while the organizational change (staffing, training, skill distribution etc.) is insignificant. All that's required is - thinking differently!

And the funniest part - we haven't even touched development!

Conclusions

1. Think about what you really need to optimize before looking at "Agile Frameworks" as a solution. 

2. Change your thinking before optimizing the work.

3. Optimize what you work on before optimizing how you work.

A few thoughts on SAFe

I often get asked what my general stance on this framework is, hence I would like to share my personal opinion with you.

The short answer is: “We don’t live in Cockaigne. Making the world slightly better sure beats dreaming up how it would be perfect. Still, that’s not an excuse to get up on the wrong foot.”

The long answer is this post.

Why use SAFe?

Being an enthusiast agilist, people sometimes ask me how I can support this framework. To me, that’s quite easy to answer – although the answer decomposes into a large number of facets.

Premade decisions

Some organizations have decided to implement SAFe before I get involved with them. I see my responsibility to help organizations find better ways of working, and regardless where they stand, I can do this. SAFe can be a vehicle for simplifying portfolio processes, adopting more reliable development practices, remove pointless status meeting and many other things. Is that local optimization? Could be. But it makes peoples’ lives better.

Where SAFe helps

I know that many agilists will cringe at the idea of actively suggesting SAFe. I don’t see SAFe nearly as much as the problem as the wrong expectations associated with it. Depending on where you currently stand, SAFe can be a stepping stone to a simpler, more effective organization with faster, more flexible decision processes. Which portion of SAFe is needed for that? That totally depends on which problem you want to solve. Don’t use a 40t truck to bring a fork from the kitchen to the dinner table – but when you’re stuck with a gigaton of organizational process mud, Scrum just won’t cut through the problems faster than they accumulate.

Getting out of discussion loops

SAFe’s high market penetration and widespread acceptance cuts short a lot of discussions whether certain concepts are “esoterical” or “applicable in Enterprises”. Pointing to SAFe as a resource collection can break stalemates caused by people who didn’t yet spend time familiarizing themselves with lean and agile mindset. Probably the most common concept that’s incredibly difficult to crack without having a chance to demonstrate is that teams need to be told and controlled in regards to “what to do when” in order to get results.

The SAFe Big Picture

I think that SAFe’s Big Picture is a stroke of genius, because it’s a handy diagram that can be used as a discussion starter to point out what is currently amiss, overcomplicated or ill implemented. Having a deep understanding to explain these concepts to the client is – in my opinion – essential to ensure that we don’t just go about implementing something that meets the form and actually solves the problem at hand. Implement SAFe as per picture is an entirely different issue.

SAFe concepts

True to one of Dean Leffingwell’s favorite Bruce Lee quotes, “Take whatever works, and take it from wherever you find it”, there are a lot of great ideas and concepts included in SAFe. While steering clear of mindlessly applying changes to an organization in places where it doesn’t help, SAFe’s comprehensive practice catalog is a great way to focus discussions with people who have never heard about these concepts before.

Common practice

Arguably, there’s nothing new to SAFe. Everything is “common practice” that has been used time and again in many organizations. Applying SAFe principles and practices will neither make you an industry leader nor a thought leader, but it may get companies unstuck and modernized. This, I think is the main value of SAFe.

What to expect from SAFe?

Expectation management is an important aspect of any change initiative. Agilists who expect a simple, flexible organization where teams have full technical and procedural autonomy will find themselves sorely disappointed by SAFe – because environments where that makes sense aren’t the target audience for SAFe.

SAFe addresses complex organizations where teams are bound by higher-order dependencies, either due to the complexity of the product itself or due to the sheer size of the value stream. A single team can easily build a web platform used by millions – and still, that same team would find themselves overchallenged if said platform was just a part of a much larger ecosystem: There’s a reason why companies like Amazon have more than ten developers.

Staying in the Amazon example, that’s an example of a company with Information Technology in their DNA – they know how to create software, build digital value streams and decouple subsystems. Many enterprises have a DNA where IT is just a fulfilment agency of non-digital business models. It would be a category error to treat these enterprises exactly like a Digital Unicorn. SAFe won’t get you there, either – what it can do, though, is set the change process in motion.

SAFe and the meta endgame

Let’s talk about the digital endgame for a bit. Many organizations struggle with survival. Former market leaders fade to insignificance or disappear into bankruptcy because they don’t understand digital product development. Talk about Kodak, Sears, Blockbuster or recently Thomas Cook. Non-digital giants are in their own endgame already. Unless they change massively, they will disappear.

For such organizations, a good implementation of SAFe can bring them closer to finding their place in a constantly disrupted marketplace. Then, they must move on lest they get stuck in a new status quo.

SAFe and success

A SAFe organization would approach enterprise initiatives differently than traditional Project / Program / Portfolio management. Cutting beyond labels, an “Epic” or “Initiative” (whatever agilists prefer) is still a kind of project. An ill-defined Epic won’t fare better in the face of change than a traditional project. There’s a lot to be said about “How” and “Why” we would even want to use an Epic Portfolio. Organizations that fail to address how they go about scoping, budgeting or implementing software will find very limited benefit in SAFe.

To be more successful with SAFe, it has to go far beyond IT. Portfolio management happens at senior management level, and it must be aligned with non-IT business units. Finance must be on board - accounting must change. We must move away from defining scope and content upfront and become rigorous at examining incremental value and axing initiatives when a value hypothesis turns out to be invalid. This requires a level of courage that many organizations lack. Culture must change as well.

SAFe and developers

A common gripe that many developers have with SAFe is that it leads to higher pressure and doesn’t address the fundamental problems. They therefore claim that either nothing has improved or things got even worse. From a developer’s perspective, and having seen many poor SAFe implementations myself, I could even agree.

I need to put this into perspective, though: Some companies I worked with will state that SAFe has cut their time-to-market in half and significantly reduced the failure rate of critical initiatives. What the developers don’t see – these successes that are totally outside their field of vision have secured their paychecks for many months.

To make SAFe a positive for experience for developers, the organization must work on many topics that may elude many managers: putting developers into control of their own work, providing clear, transparent objectives and meaningful work, improving the working environment and becoming an attractive employer across the board. This must be scoped in the transformation as well.

SAFe and massive change

Put bluntly, if you don’t require massive change, SAFe probably isn’t for you. And with this, I don’t mean a giant (maybe intercontinental) shuffling of the Org Chart. SAFe requires you to make drastic changes at every level, in every way. The organizational change is the easy, simple – and shockingly – insignificant part.

You have to rethink what value is, how you create it, how your organization supports it. You have to rethink which structures work, which don’t, what is local optimization and what is global. You have to rethink the importance explicit, implicit and tacit knowledge have for you. You have to rethink what “knowledge work” is and how you treat your workforce. You have to rethink how management works. How leadership works. How accounting works. How controlling works. How customer satisfaction in a digital environment works. How small things affect the Big Picture. And you have to put all of these pieces of the puzzle together to end up with a sustainable organization.

Key Challenges

SAFe has a number of challenges to overcome that aren’t automatically addressed by the implementation – they are inherent to how traditional organizations tick. I believe that these challenges can be overcome by the right people, given time and patience.

Break the Glass Ceiling

SAFe’s big picture is palatable for people who have zero agile experience, and this picture can be used to provide a satisfactory answer for every potential question one could ask. This gives decision-makers the confidence that this approach will work. Unfortunately, this gives them so much confidence that they will gladly delegate the transformation to members of their organization or consultants who are engaged. This delegation means that “the glass ceiling” is often maintained, i.e. that senior management observes, rather than changes themselves. 

The solution? As a manager, get involved. Be part of the transformation: “be the change that you want to see in the world”.

Mind the Context

To quote H.L. Mencken, “For every complex problem there is an answer that is clear, simple, and wrong”. SAFe has many such answers, and why the answer is wrong isn’t because the answer itself is wrong, but because SAFe doesn’t address your local context. Whether an  answer makes sense, and the action is useful in context or a different approach would be more appropriate – isn’t answered by SAFe. The more confident an organization is that SAFe has the right answers, i.e., the more blindly they trust in SAFe, the less Lean and Agile they will become. 

You require highly educated systems thinkers to solve problems at enterprise level.

Learning Culture

SAFe has a highly complex learning portfolio. Many organizations feel overwhelmed by this and simply skip most of it. Only the managers that lead Release Trains go to Leadership training, SAFe for teams costs too much, and SAFe DevOps is “for when we have extra money”.

This creates a catch-22 situation for SAFe: It’s so complex that you need to invest a lot of time and money to understanding it. And because organizations who want SAFe are usually in search of ways to save time and money, this learning doesn’t happen.

As the proverb goes, “you pay the price for learning one way or another”. Most managers opt for the invisible way of lost productivity, because they don’t understand how massive this cost actually is and because of the way organizations are set up: When developers are struggling with productivity for months, that can be argued easier than getting an extra training and coaching budget.

You need to be serious on learning, not only SAFe, but also Lean Management and Agile Development Practices, to get any sensible result out of any kind of “Agile Transformation”.

Question your setup

When I ask managers which elements from SAFe they need, they take seconds to reply: “Everything”. Asking for specific aspects, like the System or Shared Service Team, the answer is “Of course”. Without looking into all of the details of the framework, these elements, as well as the concept of team-level Product Owners, the functional separation of RTE and Scrum Masters, having multiple Business Owners and many other things create counterproductive dynamics that can reduce an organization’s flexibility, their ability to deliver value and speed of decision making.

These mechanisms address challenges that enterprises may have, but they should be applied with utter caution and avoided if possible. As organizations move further in their agile journey, they will find that these mechanics eventually become impediments to organizational agility and the delivery of value.

The introduction of new concepts should be taken with caution – which it often isn’t. New mechanisms should be implemented only to address a significant, well-understood problem.

Engage middle managers

Managers, traditionally, keep themselves out of operative details lest they be called “micromanagers”. Under the umbrella of “team autonomy”, they disengage even further from the work of the teams. What sounds good is actually SAFe’s biggest problem, because managers don’t learn how the work really works. Even after a prolonged period of time, managers thus often face two key problems: First, they lack the understanding and insight to spot and resolve local optimization. Second, as team autonomy increases and interaction with managers is actively reduced, teams lose trust in management decisions: proximity creates trust!

At a minimum, managers need to start “walking gemba” and experiencing how people work. Even better, they need to get into the trenches and engage deeper and more often with teams, without having their “manager hat” on, so that they can get unfiltered firsthand information of what the new way of working actually is like.

Start thinking agile

Just like there is no pill that turns a couch potato into an athlete, maintaining agility requires constant change and attention. With its undoubtedly huge suggestion catalog and massive training portfolio, SAFe may create an illusion that by following down this road, one becomes agile. In my perspective, by following all the suggestions of SAFe, one does become a SAFe organization – but this organization will not be agile unless people ask tough questions and challenge the assumptions of the framework.

An organization must learn to scrutinize everything it does, regardless of where the idea came from, and rigorously cut down on complexity wherever and whenever possible. This is the only way to avoid accumulating the same kind of “technical debt” in their structure and processes that a piece of un-refactored code would have. Constant, small changes of the structure must be as integral to the work as doing this to the product.

Maybe that topic would be called “Organizational refactoring”, … something to discuss in the future.