When is Data Driven Decision Making applicable?

Agile companies usually declare "Data Driven Decision Making" as a fundamental principle, because data forms the basis for empiricism.
The standard Retrospective format lists "Gather data" as one of the key stages. But when is it actually a good idea to start gathering data - and when do we stop gathering data?

To understand why we would even gather data, let us start with the problem. Basically, we want to gather data in order to help define the problem we observe and how we are going to solve it. Gathering data without a problem that needs to be solved is waste.

Here are a few types of problems you might be facing:

We don't (really) have a problem with that

Well, then ... work on something that is a problem. Easy.

We already know the root cause and solution

Well, then ... Go Fix it! Easy.

We know the root cause, but no solution

We need to devise an experiment that may bring us one step closer to the solution. After weeding out alternatives, we definitely need to gather data of some sort to help us decide if this experiment can be successful and - after trying it out - whether it actually helped.

We know the root cause, but it's outside our sphere of control

If you can't control it, but you can influence it, the best strategy may be to "make it the problem of someone who can solve it". Find a problem owner who can actually help.
There is not much value in gathering more data, unless that data is required to win a problem owner.

We know the root cause, but it's outside our sphere of influence

This happens, for example, when the problem is caused by a new law passed by the government. You have two choices here: Either, adapt to the new situation or work around. Gathering data will not help you solve the problem, because no solution you devise will be realistic.
However, you might want to collect data which helps you re-define the problem "given the existing external constraints".

We don't know the root cause, or: We don't understand our problem

This is the obvious use case for "Gather data" - go find the root cause, and learn which data can help you do that.

Summary

Data Driven Decision Making has it's uses. However, it is also limited in use. Gathering the wrong or unnecessary data is waste and should be avoided. Which data is "wrong or unnecessary" depends on the problem you are facing.
Therefore, a clear understanding of your problem is essential before actually spending time with data acquisition.

Know your customer!

The Agile Manifesto states "Customer Collaboration over Contract Negotiation". But that's not easy to practice without understanding who the customer actually is.
As organizations grow, the question "Who is the customer?" becomes increasingly difficult to answer, and the answer is often hidden in plain sight. 
In this article, we will discuss how to identify and classify customers, using the practical example of an e-Commerce Platform.

Product Customers

The most obvious category of customers are those who actually use the product. However, even they come in two categories: Those who actively go out to use the product - and those who get to use what others got for them.

Primary Customers

This is the kind of customer you can actively reach out towards. When they like the product, they will be generating revenue for you. It's a good idea to build a direct feedback cycle with them.
These could be web users who like to do online shopping.

Secondary Customers

This is the kind of customer you may not even know to exist. Whether they like the product or not is out of reach. If anything, they would give feedback to the Primary Customers - but you don't have much control over that.
These could be the workers in a company who has decided to get their supplies from our online shop.

Value Stream Customers

How does the value stream "Order to cash" map for your company? Typically, it's not the same people who have an idea, deliver it - and have to live with the consequences. In big companies, ideas are collected, then brought into a type of solution design, then implemented, launched - and then become "business as usual".
Depending on how your organization looks like, that will usually involve different departments, teams and people. Each person involved in product creation has a place in the value stream, and to know "who will receive the result of my work" is a good idea. These people are the value stream customers.
Yes, they are also customers! Life can be quite miserable if you ignore this fact.

(Other) Internal Customers

Sometimes also called "Stakeholder", but let's be explicit here. These are people in the company who actually want something from the product and who will usually assign a portion of (their) company budget towards the realization of a request.
For example, that could be Marketing or Sales desiring access to business performance metrics - or Customer Service asking for a functionality reducing the amount of service calls.
These people usually do not actively use the product, but they do have an important impact on how it is being used.
Since they are in the "large" revenue chain of the Product and receive services from Software Development, they are clearly also customers and it's a good idea to keep them happy for mutual benefit. One thing you should never forget: They do not actively create value for the product, they merely assist in value creation. No matter how many features you build for Marketing, there is no benefit unless you actually see an increase in Product Customers!

Indirect Customers

Living in a society, there is a whole bunch of people who neither want your product, buy your product - or even care about your product, but still place demands on your product.
For example, that could be the Better Business Bureau, the legislature around Data Security - or even the IRS. There is no way to "Please" these customers, but they will kill you if you don't fulfil their requests. The best thing to do is invest minimal effort to keep them off your back.

Summary

In large organizations, the relationships between supplier (you) and customer are often either unknown or agreed in contractual form. If they are unknown, there is a huge risk of misfiring. Hand-Over Contracts (responsibility lists) are good to create transparency, but do not help to solve problems or grow the business. To be successful, you have to go beyond the contractual level with your customer - and collaborate with them for mutual benefit.
The simplest Customer relationship every Scrum team should be aware of is that with the PO. The interface contracts are Definition of Ready (PO => Team) and Definition of Done (Team => PO). There are more customers - how do you handle your relationship with them?

"Bug Team" for Legacy Systems

Legacy systems often have an equally large "defect backlog" - a list of well-known, relevant issues that need to be resolved at some point. The work of digging in old dirt is not fun. No developer enjoys doing this. Consequently, these defects tend to linger. But they do impede customer value, develpoment speed and sustainability. So - what to do?
A common solution is: "Let's make a bugfixing team."

Organizational role of the bugfixing team

The bugfixing team may have directly or indirectly measurable targets, typically a reduction.Targetted metrics may include, for instance, defect count, defect cycle time, technical debt, customer complaints, waste incurred by sighting [customer] problems, unplanned system outages, release delay etc.
Their value lies in the reduction of ongoing loss. At the same time, theyprevent of further loss.

From a technical perspective, the bugfixing team is a development team like any other. They work with backlog items and bring them towards meeting a Definition of Done. To achieve this, they elaborate Acceptance Criteria, work on the source code, verify solutions, produce and deploy builds.

Why a bugfixing team is good

The elephant in the room finally disappears: Something is happening.
The introduction of a bugfixing team is a clear signal from management that Quality is being taken serious.
A dedicated bugfixing team has much better levers than regular development teams of not only "fixing the symptom", but digging all the way down to the root cause.
At best, the bugfixing teams gain intricate knowledge about what makes the system click and tick - where the risks, threats and opportunities are. This knowledge is very valuable not only in solving bugs, but in sustaining the current (legacy) platform and potentially in the introduction of a new platform in the future.

A bugfixing team has no choice other than working as a "feature team", because bugs are hardly limited to individual components - although they typically cluster. In this way, bugfixing teams are a way of successfully introducing the idea of feature teams in an organization.

Why a bugfixing team is bad

The bugfixing team may create an impression within the organization that "regular development teams" are absolved from delivering quality. 

Management and Scrum Masters/coaches must clearly communicate and emphasize at every opportunity that the bugfix team is dealing with existing impediments, but is not an excuse to shove "undone work" onto Production: There is no excuse for introducing "new bugs".

Developers on dedicated bugfixing team easily get frustrated: While other developers get to create fancy new stuff, they merely fix things which should have been working a long time ago already. Innovation potential on a bugfixing team is much lower.

Long term perspective

A bugfixing team should strive to "run out of work" as quickly as possible. The best state is reached when there are no important legacy bugs left to fix. The knowledge a bugfix team has obtained will be very valuable for the organization. After working in bugfixing for about half a year, developers possess a deep, intricate understanding of the system which other developers may not possess. 

This knowledge is very valuable to the organization: The bugfix team should be encouraged and motivated by the idea that the role is temporary and the entrance towards "something greater".

Summary

A bugfix team is not necessary in every organization. Management should state quickly that the main idea is getting out of the quagmire of Poor Quality and the institution is temporary. Developers must understand their responsibility is not "working off defects", but closing the manholes in the system, so they can move on as a team.

The Product Owner in charge of the bugfixing team should prefer "defect areas" over small-scoped, local problems, although these provide quick-wins.
The Scrum Master working with the bugfixing team must harp constantly on the "Continuous Improvment" of the system, the team and the organization. Only by focussing on Continuous Improvement will the team overcome the unsatisfactory treadmill of fixing an endless flow of reported defects.

A potential solution to prevent developers from feeling stuck in "bugfix mode" is to rotate the "bugfixing team" in the organization every few months.

Management and PO must keenly observe when the point is reached where the loss associated with existing defects turns the team into a negative business case. At that point, the team should stop being a "bugfixing team" and take on more valuable feature development activity.

Conclusion

If there is a clear business case for introducing a bugfixing team - by all means: at least consider the option. When moving forward - keep focused and provide perspective to bugfixing developers.

MoSCoW - yes or no?

I was recently confronted with the question: "What is wrong with MoSCoW prioritization method?" - as a rhetorical question. The answer: "It is based on wishful thinking, the idea that when we turn something into 'Must', we somehow will have it." - effectively suggesting not to use MoSCoW. Let me take a different stand here.

As a Product Owner, I am forced to make tough calls. And I admit that I use MoSCoW as a tool.

The criticism of MoSCoW

It is correct to state that "just because someone claims something must be done, it does not mean that it will be done.". Just because my customer says we must deliver that new teleporter by the end of the month, doesn't mean he will get it.
Likewise, simply defining all work as "mandatory" doesn't mean that more will be delivered.
To suggest that people from business don't understand this is merely beating up a strawman.

Maybe they are unaware of the consequence of their action when defining a "must", and the organization is in such a mess that people don't understand the complex intricacies of what "defining as must" means.

Not a prioritization tool

MoSCoW can be very easily misused when used with the wrong question, such as "Is this a 'Must' requirement?". But that's actually just a stupid question, because everyone knows the answer will be "Yes", regardless of how important something really is. 

By using MoSCoW as prioritization tool, you will end up with a backlog consisting pretty much of 100% "Must do" stuff. Next thing you know is that everyone is blaming the team and/or PO for not doing stuff that "must be done". This approach completely destroys trust. 

Take a closer look at the four terms: They are not priorities, they are categories.  

Then, learn to ask the right question:

MoSCoW as a filter

By asking the requester/stakeholder: "What happens if we don't do it?" - the answer pretty much falls into one of the following categories: 

a) We are all dead. ("must do")

b) We will suffer a lot. ("should do")

c) Someone's uncomfortable. ("could do")

d) It doesn't matter. ("won't do")

Like this, MoSCoW can be used as a very easy, quick, reliable and comprehensible way to brush off stakeholder requests that are definitely not going to be delivered: In practice, that's everything from "C" and "W". 

Not a delivery requirement

The first thing to realize is that "Must/should" is not a priority. It also does not mean "We must/should deliver this". It should be understood as: "At this time, we understand that this must/should be added into the backlog." In due order. Weighted against all of the other stuff that is already there. The backlog only consists of "Must" or "Should" items. So, these terms are effectively worthless fill words in the context of a Product Backlog.

Just like all Java programmers know Java, the term "Java" means absolutely nothing to a group of "Java programmers" - so the words "must" or "should" means nothing in a group consisting solely of " 'must or should' items".

Backlog content

"Must", in this context, does not refer to a specific date or scope of delivery. 

It refers to being appropriate for addition to the Product Backlog.

If the PBL is too big already, the PO can suggest: "If this must go into the backlog, you need to take out one thing, because I'm not going to juggle more chainsaws than we can sensibly handle."
Like that, stakeholders can be forced to make tough decisions.

The terms "must/should" are not absolute. They becomes context-sensitive. For example, "Must do" might turn into "Must do instead of [doing something else]" or: "Must do before [doing certain other things]".

You can use a very simple sentence to explain why some "Must" item just got kicked out of the backlog; "When this issue was brought up, we decided we must add this item to the backlog. Given this new information, we decided that now we won't do any work on it."

This is a highly powerful mechanism for limiting both Scope and Work in Progress.

Used appropriately, it's a real-time adaption mechanism for keeping the Product Strategy both focused and flexible.

Summary

MoSCoW is actually a principle and not a tool: We always separate work into stuff we do - and stuff we don't do - and we always call it a day at some point. Assigning a category to "not done work" only makes this explicit. The labelling process is just turning the principle behind separation of concerns into a tool.
Like every other tool, it can be abused when understood as a tool. When used to define inclusion, you will encounter a myriad of problems.  But that doesn't mean we should throw the baby out with the bath wather.

Use MoSCoW to define exclusion. It keeps the backlog short and creates transparency on why certain things are in focus - and others are not.