TOP Structure - the Organizational Domain

 It sounds tautological that every organization needs organization - and yet, most companies are really bad at keeping themselves organized, and it hasn't gotten better with the advent of Remote Work.

Although it's technically correct that organization is non-value adding, it is essential to get organization right:

The Organizational Domain is the second core pillar in the TOP Structure

People

Do we have the right people in the right places, are they equipped and do they have the necessary support to succeed? People aren't just chess pieces we can freely move around on an org chart - they're individuals with needs and desires, and if we don't take care of our people, performance will decline.

Collaboration

Can our people collaborate efficiently and effectively? Are the right people in touch with each other? How much "telephone game" are we playing? Do we have policies that cause us to block one another? Do we optimize for utilization of individuals, or getting stuff done?

Learning

Do we get genuine learning from events, or are we continuously repeating the same mistakes? Do we have functioning feedback loops? Are we figuring out the levers for meaningful change, and do we turn all of this into action? And do we only focus on how we execute, or also what we work on, and how we think?

Why Organization often doesn't work

Especially project organizations and large "Programs" commonly neglect investing into working with people, improving collaboration or creating a learning environment.

Even "Agile" environments often delegate the responsibility for organization to the Scrum Master, although none of the items mentioned above can be done by a single person on a team - they're everybody's job: team members, support roles and management alike.

When the Organizational pillar isn't adequately represented, we quickly accumulate "organizational debt" - an unsustainable organization that becomes more and more complex, costly, slow, cumbersome and unable to deliver satisfactory outcomes.

Check your own team - on a scale from 1 to 10, how well are the above mentioned organizational aspects tended to?

TOP Structure - the domain of Architecture

In software, there's a critical intersect between technology, that is - how we turn ideas into working software - and our organization - that is, who is part of development and how they interact.

Architecture is at the crossover point of Technology and Organization

This domain is Architecture, and it exists one way or another - if we don't manage it wisely, the outcome is haphazard architecture, most likely resulting in an inefficient organization delivering a complex, low value solutions in a long time and at a high cost.

Am I trying to advocate for a separate architecture team? No. Take a moment and think about Conway's Law: If we have the wrong organization, the consequence is the wrong architecture, the consequence is the wrong technology - and the consequence of that is a failing business.

Architecture is bi-directional. The right organization depends as much on technical choices as vice versa. We need a closed feedback loop between how we develop, and how we organize ourselves.
In many companies, the architectural feedback loop is utterly broken, hence they're doing with 50 people what could be done with 10.

One of the key organizational failures which lead to the need for "Scaling Agile" is that architecture is either disconnected from workplace reality, or not even considered to be important. By architecting both our organizational system and our technology to minimize handovers, communication chains and process complexity, many of the questions which cause managers to ponder the need for "Scaling Frameworks" are answered - without adding more roles, events or cadences.

This form of architecture doesn't happen in ivory towers, and it doesn't require fancy tools - it happens every day, in every team, and it either brings the organisation in a better direction, or a worse direction.

When was the last time you actively pondered how technical and organizational choices affect one another, and used that to make better choices in the other domain?

Make - or Buy?

Determining which systems, components or modules we should "Make" and which we should "Buy" (in extension: use from Open Source) is a challenging aspect for every IT organization. Even when there's a clear votum by management of developers in favor of one option, that vote is often formed with a myopic perspective: managers prefer to "Buy" whatever they can, whereas hardcore developers prefer to "Make" everything. Neither is wise.
But how do we discern?

There are a few key factors at play here:

Factor	Details
Availability	When there's an affordable, ready-made solution, then "Buy" to avoid reinventing the Wheel. Be sure that ready means ready and "affordable" has no strings attached.
Uniqueness	you need to "Make" anything that's unique to your business model.
Adaptability	When there's only a small need for change and customization, "Buy" is preferable. Never underestimate "a small change."
Sustainability	"Buy" only when both initial cost plus lifecycle cost are lower. Include migration and decommissioning costs.
Skill	If you need specialists that you don't and won't have, "Buy" from someone who has.
Dependency	If your business would have to shut down when the solution becomes unavailable, "Buy" puts you at your vendor's whim.
Write-off	You can "Buy" to gain speed even when all indicators favor "Make," if - and only if - you're willing to write off everything invested into the "Buy" solution.

Choose wisely - the answers are often not as obvious as they seem.

U-Curve Optimization doesn't apply to deployments!

Maybe you have seen this model as a suggestion how we should determine optimum batch size for deployments in software development? It's being propagated, among other places, on the official SAFe website - unfortunately, it sets people off on the wrong foot and suggests them to do the wrong thing. Hence, I'd like to correct this model -

In essence, it states is that "if you have high transaction costs for your deployments, you shouldn't deploy too often - wait for the point where the cost of delay is higher than the cost of a deployment." That makes sense, doesn't it?

The cause of Big Batches

Well - what's wrong with the model is the curve. Let's take a look at what it really looks like:

The difference

It's true that holding costs increase over time, but so do transaction costs. And they increase non-linearly. Anyone who has ever worked in IT will confirm that making a huge, massive change isn't faster, easier or cheaper than making a small change. 

The amount of effort in making a deployment is usually unrelated to the amount of new features part of the deployment - the effort is determined by the amount of quality control, governance and operational activity required to put a package into production. Again, experience tells us that bigger batches don't cause less effort for QC, documentation or operations. If anything, this effort is required less often, but bigger batches typically require more tests, more documentation and more operational activity each time - and the probability of Incidents rises astronomically, which we can't exclude from the cost of change if we're halfway honest.

Metaphorically, the U-Curve graph could be interpreted as, "If exercise is tiresome, exercise less often - then you won't get tired so often. The optimum amount of exercise is going to door to receive the pizza order, but rather order half a dozen pizzas at once if the trip to the door is too exhausting, and then just eat cold pizza for a few days." 

Turning back from metaphors to the world of software deployment: It's true that for some organizations, the cost of transaction exceeds the cost of holding. This means that the value produced but unavailable to users is lower than the cost of making that value available. And that means that the company is losing money while IT sits on undeployed, "finished" software. The solution, of course, can't be to wait even longer with not deploying, and losing even more money - even if that's what many IT departments do.

As shown in the model, the optimum batch size isn't achieved when the company is stuck between a rock and a hard place - finding the point where the amount of money lost by not deploying is so big that it's worth to spend a ton of money on making a deployment.

The mess

Let's look at some real world numbers from clients I have worked with. 

As I hinted, some companies have complex, cumbersome deployment processes that require dozens of people weeks of work, easily costing $50000+ for a single new version. It's obvious that due to the sheer amount of time and money involved, this process happens as rarely as possible. Usually, these companies celebrate it as a success when they're able to go from quarterly releases to semiannual releases. But what happens to the value of the software in the meantime? 

Just assuming that the software produced is worth the cost of production (because if it wasn't, why build it to begin with) - if the monthly cost of development is $100k, then a quarterly frequency means that the holding cost is already at $300k, and it goes up to over half a million for semiannual releases. 

Given that calculation, we should assume that the optimal deployment frequency is when the holding cost reaches $50k, which would be two deployments per month. That doesn't make sense, however: when 2 deployments costs $50k each per month, then 100% of the budget would flow into deployment - of nothing. 

Thus, the downward spiral begins: fewer deployments, more value lost, declining business case, pressure to deliver more, more defects, higher cost of failure, more governance, higher cost of deployments, fewer deployments ... race to the bottom!

The solution

So, how do we break free from this death spiral?

Simple: when you're playing a losing game, change the rules.

The mental model that deployments are costly and we should optimize our batch size to only deploy when the cost of deployment outweighs the holding cost is flawed. We are in that situation because we have the wrong processes to begin with. We can't keep these processes. We need to find processes that significantly reduce our deployment costs:

The cost of Continuous Deployment

Again, using real world data from a different client of mine: 

This development organization had a KPI on deployment costs, and they were constantly working on making deployments more reliable, easier and faster. 

Can you guess what their figures were? Given that I have anchored you at $50k before, you might think that they have optimized the process maybe to $5000 or $3000.
No! If you think so, you're off by so many orders of magnitudes that it's already funny. 

I attended one of their feedback events, where they reported that they had brought down the average deployment cost from $0.09 to $0.073. Yes - less than a nickel!

This company made over 1000 deployments per day, so they were spending $73 a day, or $1460 a month, on deployments. If we calculated the accumulated cost of deployments for the whole period, they were still spending over $5000 for three months' worth of software development. But the transaction cost for each single deployment is ridiculously low.

Tell me of anything in software where the holding cost is lower than 7 Cents - and then tell me why we are building that thing? Literally: 7 Cents is mere seconds of developer time!

With a Continuous Deployment process like this, anything that's worth enough for a developer to reach for their keyboard is worth deploying without delay!

And that's the key message why the U-Curve optimization model is flawed:

Anything worth developing is worth deploying immediately.

When the cost of a single deployment is so high that anything developed isn't worth deploying immediately, you need to improve your CI/CD processes, not figure out how big you should make that batch.

If your processes, architecture, infrastructure or practices don't permit for Continuous Deployment, the correct solution is to figure out which changes you need to make so that you can continuously deploy.

How Equivocation destroys Agile

"Scrum engages groups of people who collectively have all the skills and expertise to do the work and share or acquire such skills as needed" - What does that even mean?

This is just a door-opener for one of many equivocation fallacies which are destroying the entire "Agile" space. Let's dig into it.

Equivocation

We're talking about "equivocation" when a term is used in multiple inconsistent ways, which thereby invokes false images in the minds of the audience. A simple example of how quickly this can get out of hand: "White Supremacy is racism. Since the White House is also White, it's also a sign of insitutional racism." - it's clear that the term "White" means two different things (in one case, a mindset - in the other, a description), but it's already hard to get the thought out of our heads.

Equivocations in the "Agile" space are much more subtle, thus harder to spot, but potentially no less damaging. So, let's get into some of the equivocations that are permeating "Agile." (and I purposefully don't dig into how "Agile" itself doesn't have a single definition people would agree on)

Customer

Based on Investopedia, "A customer is an individual or business that purchases another company's goods or services. Customers are important because they drive revenues; without them, businesses cannot continue to exist."

And yet, we have this concept of "internal customers" popping up left and right.

The first Agile Principle reads, "Our highest priority is to satisfy the customer through early and continuous delivery of valuable software." So, every Agile team needs a customer - and who's that customer for most agile teams? Well, they deliver Business Support Systems, so they say that "Business is our customer!" - circular logic based on the Investopedia definition: the business is important, because without it, the business can't continue to exist. Does that make sense?

It doesn't even end there: Modern software is complex. In larger companies, we often see platform teams building platforms that other teams work on. These say, "The other development teams are our customer!" - how much more profit would we make when we get more and more development teams asking for services from our platform team while nothing in our company's relationship with the open market changes?

To quote Wikipedia, "Leading authors in management and marketing, like Peter Drucker, Philip Kotler, W. Edwards Deming, etc., have not used the term "internal customer" in their works. They consider the "customer" as a very specific role in society which represents a crucial part in the relationship between the demand and the supply. Some of the most important characteristics of any customer are that: any customer is never in a subordination line with any supplier; any customer has equal positions with the supplier within negotiations, and any customer can accept or reject any offer for a service or a product."

The equivocation fallacy is a dual use of the word "customer," in one sense meaning "entity on whom a business depends as a source of revenue" - and in another sense meaning "entity consuming the outcomes of the work done by a group of people." 

Why is that a problem? Demand.

When interacting with the open market, we would like to maximize demand - that is, we would like to have more customers constantly requesting more of our product, and more features from our product - to get them addicted for more, more and more. That enables growth, and makes our company sustainable.

On the other hand, internal demand is undesirable, because it diverts resources away from serving the real customer: internal entity A doing work for internal entity B means that B must spend time with A, which B doesn't have for the customer - and A must spend time to do something for B. That costs money. This money isn't profit any more, and the time B spends with A is profit not generated: internal demand costs twice. We therefore would like to minimize internal demand.

Equivocating "customers" to include both internal and external people opens the doors for a destructive game: maximizing internal demand. It makes internal service teams feel good about themselves and their work, while the inner proceedings bleed the company dry of critical resources.

By clearly differentiating between "customers" as "those who will determine whether they will pay for what we do" and "consumers" as "those who need what we do," we avoid this pitfall.

We won't go as deeply into other equivocation fallacies - just teasing what they are, and the damage that they're doing.

Experts

I chose that little quote from the Scrum Guide as an opener on purpose, because we hear very often that "developers are the experts, trust them."

The equivocation fallacy: Scrum requires developers to be experts in order to function properly. Many Scrum folks argue that being a developer means they're an expert.

This harms the developers when they don't get the support they would sorely need, and it harms the company when the team doesn't perform properly.

Owner

Actual Owners, according to Investopedia, are "a person or entity that receives the benefits of ownership." (emphasis added)

However, when you look at Scrum's Product Owner - how often are these individuals the real beneficiaries of the stuff they allegedly "own?" They have a bunch of accountabilities and responsibilities, but rarely own anything.

The equivocation? "Being a beneficiary of" and "Being accountable for."

The consequence?

While on the one hand, an owner can do whatever they please with their property, they tend to have a self-interest in maximizing their benefits. Without this self-interest, an Owner can merely minimize the effort they have to put into whatever they are supposed to "own."

 

Similar things apply to other forms of "Ownership" - process ownership, code ownership, system ownership or anything else: let people reap the benefits of whatever they're supposed to own, or it's going to backfire.

Value

Investopedia defines "Value" as "the monetary, material, or assessed worth of an asset, good, or service." In the "Agile" space, however, there's a notion that a Product Increment is the value produced by the team. 

That's a confusion of cause and effect: If the Product Increment has value, then the team has produced value. But merely because the team has produced an increment, it doesn't mean it is value - it could be that it actually has a negative value.

Value-Add

Not specifically an issue of "Agile," more a confusion in the entire IT industry - "value added activity" is an activity that increases the value of a product or service. Analysis, design, documentation, testing and support do not create any assets. They are therefore "non-value-added activity."

The equivocation in software development is that all these activities are also value-adding, since they're contributing to the creation of value.

Careful: Just because an activity is non-value-adding, that doesn't mean it's not required. It just means it's not increasing the value. 

Think of it like this: If you do more of activity X, does the company's total value go up? If not, then X is a non-value adding activity.

The issue?

We would like to maximize our share of value-added activity, and minimize our share of non-value-added activity. By equivocating, for example, testing, to be a value-added activity, we paint a picture that testing is something we should do more of, because hey, it creates value. No. It doesn't: It's an activity necessary to create and maintain value, but all other things equal, more testing doesn't mean we're delivering more value.

Value Stream

One of the latest fads that came up with SAFe is the "value stream." It's been around since the early days of Lean (although the concept was already used by Henry Ford around a century ago.)

Yet, if you'd ask how SAFe's "Development Value Stream" is different from a Software Development Lifecycle Process, people might not know how to answer - "Isn't a value stream just a high level process?"

That's the equivocation fallacy of value streams: Whereas a value stream is defined as "the set of value-added activities between customer need and realization of value by the customer"- when we modify the meaning of "customer," "value" and "value-add," then it is nothing other than a process.

The problem?
Clearly optimized value streams are make-or-break for the success of any business. Everything is subject to an organization's core value streams. Processes need to optimize around them. When we can't discriminate the two any more and locally optimize a support process at the expense of our actual value stream, we endanger the whole company.

Closing remarks

Equivocations make improvement difficult, because the thing we're talking about may not be the thing we're talking about. Let's get clear first what we really mean when we're talking about something, remove the equivocation, find a proper label that means what it says - and improve on the things that are hidden in plain sight by using the wrong words.

Collaboration Patterns we know from science

Team structures - should be a straightforward enough topic, although in many organizations it isn't. Here are six phenomena you may remember from science class - and how they relate to your organizational structure.

To keep matters simple, this post refers to "entities" - which could either be individuals, teams or entire departments. While the nature of the entity changes, we are concerned with the relationship of the entities with each other. Since some terms have different definitions in different domains, let us refer to the point of origin.

Cohesion

(Origin: Chemistry)

Cohesion is the connection between entities of one substance. Organizational cohesion, thus is the bonding strength between entities of the same category.

Examples

We have organizational cohesion when there is collaboration occurring within one team.

We have poor organizational when team members act as a group of individuals, picking their own work items.

---

Adhesion

(Origin: Chemistry)

Adhesion is the strength with which two different entities stick together. Organizational adhesion, for example, is the amount of effort it would require to separate two different entities.

Examples

We have high organizational adhesion when a process has a complex critical path.

Two business units serve different customer segments independently have low adhesion.

---

Covalence

(Origin: Chemistry)

Covalence happens when two atoms share an electron to form a bond, which molds these two distinct entities into one "complete" entity. Within an organization, covalence occurs when two or more entities share resources.

Examples

Component ownership causes covalence - let's say team A owns the Customer entity, and team B owns the Contract entity. When B needs to access the Customer, they rely on whatever A provides - whereas when A references the Contract, they rely on whatever B provides.

In situations where the Contract relies on a new or modified attribute of the Customer - such as consumer credit score, team B must coordinate with team A on how and when a change can be made, and team A might want to store a "previously rejected" attribute that must be provided by Team B. Covalence thus means that while the inner dealings of A and B become intertwined, an outside change from covalent entities requires that the change works for all covalent entities.

---

Bridge

(Origin: Chemistry)

A bridge connects two entities to turn these into one common, stable structure. Bridges require covalence bonding between two entities plus the presence of a third entity. The bridge is an entity that connects two entities by being the missing part in both.

Organizational bridges are entities equally bound to two or more other organizational entities to form one entity of higher complexity.

Examples

The analyst role is often an organizational bridge - they are close to business from development perspective and close to development from a business perspective: they are neither, but connect the two entities to turn demand into solutions.

---

Coupling

(Origin: Physics)

Capacitative coupling occurs when an energy transfer occurs between two separated conductors. Organizational coupling thus occurs when two structurally separated entities affect the outcomes of one another. We would refer to "tight coupling" when either entity could cause blocking interference to the other, and "loose coupling" if there's a generally uncritical impact. If there is no interference, we would consider the entities uncoupled.

Examples

We see tight organizational coupling when the Maintenance Team decides to shut down the Deployment process, thereby incapacitating the development pipeline.

Loose organizational coupling could be the relationship between Marketing and Sales - while they can technically work with or without the other, they do have a performance effect on each other.

---

Coherence

(Origin: Physics)

Coherence is the ability of a signal to withstand interference. We discriminate between spatial and temporal coherence: Spatial coherence is the ability of a signal to withstand interference over distance, whereas temporal coherence is the ability of the signal to withstand interference over time. In an organization, it's the ability of the information to cross entity boundaries without getting distorted by interfering signals (e.g., from other work items, other projects or line management.) Note that coherence is only relevant in the context of cohesion - incohesive entities who don't work towards a common goal require no coherent signal transmission.

Examples

Low spatial coherence would be a process with a lot of "telephone game," where information is modified in each step.

High spatial coherence would be provided by a synchronization event which ensures that all stakeholders have the same understanding on a subject.

Low temporal coherence are the deviations from a plan over time, usually caused by unanticipated events.

Using metrics properly

Getting metrics right is pretty difficult - many try, and usually mess up. The problem?
Metrics require a context, and they also create a context. Without a proper definition of context, metrics are useless - or worse: guide you in the wrong direction.

A Metrics system

Let's say you have a hunch, or a need, that something could - or should - be improved. To make sure that you know that you're actually improving, create a metrics system covering the topic. To build this system, it should cover the organizational system in an adequate - that is, both simple and sufficient, model consisting of:

• Primary metrics (things we want to budge)
• Secondary metrics (things we expect to be related to our primary metric)
• Indirect metrics (things we expect NOT budge)

An example

We start with a problem statement, "Our TTM sucks." Hence, our metrics system would start with the primary metric "time-to-market" as a primary metric. A common sense assumption might be that an improvement to TTM will make people do overtime, or that people become sloppy. Thus, we add the secondary metric "quality" - we would like to observe how a change to TTM affects quality, and we set an indirect metric "overtime" - we set a constraint that people shall not do extra hours.

Systematic improvement

In order to work with your metrics system adequately, there's a common five-step process which is at the core of Six Sigma:

Define 

• Define our problem statement: what problem do we currently face?
• Define our primary metric.
• Become clear on our Secondary and Indirect metrics.

Measure

• Get data to determine where these metrics currently are.
• Set an improvement target on our primary metric.
• Predict the effects on secondary metrics 
• Set boundaries on indirect metrics.

Analyze

• Understand what's currently going on.
• Understand why we currently see the unwanted state in the primary metric.
• Determine what we'd like to do to budge the primary metric.

Improve

• Make a change.
• Observe changes to all the metrics.

Control

• If our Primary metric budged significantly and all other metrics are where we'd expect them to be, our change was successful.
• If that wasn't the case - we messed up. Backtrack.
• Determine which metrics we'd like to retain in the future to make sure we're not lapsing back.

Metrics are thus always bounded to a specific problem you would like to address.

Pitfalls to avoid

Getting metrics systems completely right is challenging, and many organisations struggle with getting metrics right.

Incomplete metric systems

The most common problem is that we often only define primary metrics, which paves the way for building Cobra Farms, that is: we improve one thing at the expense of another thing, which might create an even bigger problem that we just didn't realize.

Red Herring metrics

Another issue is confusion between outcomes and indicators. This is also often associated with a Cobra Farm, but from another angle - we fail to address the actual problem and pursue the problem of the metric.

For example, if management wants to reduce the amount of reported defects, the easiest change is to deactivate the defect reporting tool. That reduces the amount of defect reports, but doesn't improve quality.

This is also called "Goodhart's Law:" A metric that becomes a target stops being useful.

Vanity metrics

It's a human tendency to want to feel good about something, and metrics can serve that basic need. For example, we might track the amount of hours worked per week. That metric constantly goes up, and it always hits the target. But it's not valuable: It tells us nothing about the quality or value of the work done.

Uncontrolled metrics ("waste")

We often collect data, just in case. And we don't connect any action trigger with it. Let's take a look at, for example, deployment duration: It's a standard metric provided by CI/CD tools, but in many teams, nothing happens when the numbers rocket skyward. There are no boundaries, no controls, and no actions related to the metric. If we don't use the data available to act upon it, the data might as well not exist.

Bad data

Sometimes, we have the right metric, but we're collecting the wrong data, or we collect it in the wrong way. That could range anywhere from having the wrong scale (e.g. measuring transaction duration in minutes, when we should measure in milliseconds - or, vice versa)  to having the wrong unit (e.g. measuring customer satisfaction in amount of likes instead of NPS) to having the wrong measurement point (e.g.measuring lead time from "start work" instead of from "request incoming.) 

This data will then lead us to draw wrong conclusions - and any of our metrics could suffer from this.

Category errors

Metrics serve a purpose, and they are defined in a context. To use the same metrics in a different context leads to absurd conclusions. For example, if team A is doing maintenance work and team B is doing new product development: team A will find it much easier to predict scope and workload, but to say that B should learn from A would be a category error.

Outdated metrics

Since we're talking about metric systems rather than individual metrics, when the organizational system on which we measure has changed, our metrics may no longer make sense. Frequently revisiting our measurement system and discarding control metrics which no longer make sense and either discarding or adjusting them is essential to keep our metric system relevant.

Why we need clearly defined goals

 A common issue I observe in many organizations - there's no visible goal beyond, "Get this work done," and people don't even see the point in "wasting time to set a goal." The problem: many organizations are just tactical engines generating work and handling exceptions in the generated work. Yet, most of this work has no goal. Goal-setting is not an esoterical exercise, and here's why.

The Hidden Factory

One concept of Lean Management is the "Hidden Factory" - and this concept deserves some kind of explanation. A factory is an institution that applies work to input in order to generate some outputs. So far, so good. A "hidden factory," on the other hand, is a factory within a factory, doing work without generating viable outputs: either nothing - or waste.

To understand the problem of hidden factories, think like a customer.

You get the option to buy your product from one of two providers.

Company A has a straightforward process, turning raw input into consumable output, and Company B has a process that turns the same input into something, then into something else, then into something else, then into the same consumable output.

This extra work makes Provider B's product is more expensive to produce, without any discernable difference to the product sold by Company A.

Company A and Company B thus sell products which are identical in all aspects - except price. Company B has to charge a premium to cover the extra work they do. Which product would you purchase?

As customers, we do not care how much work is required to do something. Given all other things equal, we choose to opt for the cheapest, fastest way of meeting our needs.

And companies are no different here. But how does that relate to goal-setting?

Induced and intermediate work

Many companies are great at inducing work - and once that work has been induced, it becomes necessary, and the people doing it must do it, otherwise the outcome can no longer be achieved.

Let's pick a practical example.

We have chosen to use a Design Process that requires any changes to be made as a sequential series of steps:

1. Request the element to be changed
2. Describe the change to be made
3. Prototype the change
4. Validate the change
5. Implement the change
6. Verify the change

While you may argue that all of these steps are common sense and necessary, our specific process choice has just locked us into a process that turns replacing an image into a full day's work - a change that could be done by a competent developer in a couple of minutes.

How is that relevant to goal-setting?

Confusing Task and Outcome

Referring to our fictional process, an individual may have the specific responsibility of describing changes. Their input is a change request, and their output is a change description. As a customer, I address this organization to say, "I want a new backdrop on my website." Our execution agent of step 2 will say, "I am overburdened. I have too many change requests on my desk. I need someone to help me describe the changes." If we would ask them "Why do you need to describe the changes?" - they might say, "So that they can be prototyped." If we'd press and ask, "And why do they need to be prototyped?" - the answer could be, "So that we can validate the change." - which, of course begs the question, "And then - I get what?" - "An implementable change."

You see where this is going: Everyone has reasons why they do the things they do, and from the way this organization is set up, their reasons are indeed valid. And still, nobody really understands why they do the things they do. 

We should assume that everyone whom we ask should answer, "So that you can get your new backdrop." In many companies, however, that is not the case.

And that's where goals come into play.

Goals

Every company has a few - usually very few - first-order goals, and a specific context that provides constraints within which these goals can be realized. Surviving and thriving on the market is a baseline almost all have, and most of the time, the primary goal is to achieve this by means of advancing the products which define the company. That would be an example of a first-order goal.

From that, we get into second-order goals, that is - into derived goals which help us achieve this primary goal. Build a better product. Build the product better. Sell more. Sell faster. Sell cheaper. You name it.

These, of course, would be realized via strategies - which in themselves have multiple subordinate goals. For example: Increase product quality, add features to the product, reduce discomfort with the current product, improve perception of the product in its current state - again, the possibilities are endless.

At some point in the rabbit hole, somewhere deep down in the loop of operational delivery, we may then see a business analyst stating, "I am overburdened. I have too many change requests on my desk. I need someone to help me describe the changes." - but why are they doing it? Are we describing change in order to increase product quality, in order to sell more, or to sell cheaper?

It's easy to realize that "adding people to do the work" may indeed make sense when our goal is to add features to improve our product. And yet, it seems entirely backwards when our goal is to "sell cheaper."

That's why we are setting goals. It allows everyone, on all layers of an enterprise, regardless of their specific responsibility, to quickly and easily determine, "Do the things which I am doing help us achieve our goals?"

If the answer to that simple and straightforward question is, "No" - then this begs two followup questions:

• Why am I doing things which are not helping this company achieve its goals?
• What do we need to change, so that I am contributing to our goals?

The impact of goals

Well-defined goals immediately expose the Hidden Factories and induced work, and they set the stage for reducing waste in our processes as well as leading employees to do more meaningful, more important work.

Poorly defined goals - such as "to do X amount of work" - encourage establishing and inflating Hidden Factories, and they set the stage for wasteful processes and unhappy employees who may be doing totally worthless work, without ever realizing.

Undefined goals - or the absence of goals - remove the yardstick by which we can measure whether we are contributing to anything of relevance or merely adding waste. Without a goal, work is meaningless and improvement impossible.

The importance of goals

Goal-setting is important for organizations both large and small in:

1. Guiding decision-making
2. Enabling Improvement
3. Eliminating Waste

While a goal itself doesn't do any of these, a goal sets the stage for these. Once you know your goal, you can take it from there.

Why test coverage targets backfire

 Most organizations adopt a "Percent-Test-Coverage" metric as part of their Definition of Done. Very often, it's set by managers, and becomes a target which developers have to meet. This approach is often harmful. And here's why:

The good, bad and ugly of Coverage Targets

Setting coverage targets is often intended as both a means of improving the product, as well as improving the process - and getting developers to think about testing.

The good

It's good that developers begin to pay closer attention to the questions, "How do I test this?" as well as, "Where are tests missing?" Unfortunately, that's about it.

The bad

The first problem is that large legacy code bases starting with zero test coverage will put developers into a pickle: If I work on a component with 0% test coverage, then any amount of tests I write will still keep that number close to Zero. Hence, the necessary compromise becomes not setting a baseline number, just ask for the number to increase. The usually envisioned 80+% targets are visions for a distant future rather than something useful today.

Looking into the practice - as long as the organization is set up to reward minimizing the amount of time invested into unit testing, the outcome will be that developers try to meet the test coverage targets with minimum effort. 

Also, when developers have no experience in writing good tests, their tests may not do what they're supposed to do.

The ugly

There are many ways in which we can meet coverage targets that fulfill Goodhart's Law:

Any metric that becomes a target stops being useful.

Often, developers will feel unhappy producing tests that they create only to meet the target, considering the activity a wasteful addition to the process, which provides no benefit.

At worst, developers will spend a lot of time creating tests that provide a false sense of confidence, which is even worse than knowing that you have no tests to rely on.

But how can this happen? Let's see ...

The first antipattern is, of course, to write tests that check nothing.

Assertion-free testing

Let's take a look at this example:

Production code

int divide(int x, int y) { return x/y; }

Test

print divide(6,2);

When looking at our code coverage metrics, we will see a 100% coverage. But: the test will always pass - even when we break the production code. Only if we were to inspect the actual test output (which is manual effort that we probably won't do) - we will what the test does: There is no automated failure detection, and the tests aren't even written in a way that we would detect a failure: Who would detect the problem if suddenly, we got a "2" instead of a "3" - we don't even know which result would have been correct!

Testing in the wrong place

Look at this example:

Class Number {
int x;
void setX(int val) {x=val;}
void getX() {return x;}
void compute() { x=x?x/x^x:x-x*(x+x); }
}

n = new Number();
n.set(5);
assert (N.x == 5);
assert (N.get() == 5);

In this case, we have a code coverage of 75% - we're testing x, we're testing the setter, and we're testing the getter.

The "only" thing we're not testing is the compute function, which is actually the one place where we would expect problems, where other developers might have questions as to "What does that do, and why are we doing it like that?" - or where various inputs could lead to undesirable outcomes.

Testing wrongly

Take a peek at this simple piece of code:

int inverse (int x) { return 1/x; }
void printInverse(int x) { print "inverse(x);" }

assert (inverse(1) == 1);
assert (inverse(5) == 0);
spyOn(print);
printInverse(5);
assert (print.toHaveBeenCalled());

The issues

There are numerous major problems here, all of which in combination make that test more dangeous than helpful:

Data Quality

We may get a null pointer exception if the method is called without initializing x first, but we neither catch nor test this case.

Unpexpected results

If we feed 0 into the function, we get a Divide by Zero. We're not testing for this, and it will lead to undesired outcomes.

Missing the intent

The inverse function returns 0 for every number other than 1 and -1. It probably doesn't do what it's expected to do. How do we know? Is it poorly named, or poorly implemented?

Testing the wrong things

The print function's output is most likely not what we expect, but our tests still pass.

Conclusion

If we rely on coverage metrics, we might assume that we have 100% test coverage, but in practice, we may have very unreliable software that doesn't even work as intended.

In short: this way of testing tests that the code does what it does, not that it does what it should.

Then what?

The situation can be remedied, but not with numerical quotas. Instead, developers need education on what to test, how to test, and how to test well.

While this is a long topic, this article already shows some extremely common pitfalls that developers can - and need to - steer clear from. Coverage metrics leave management none the wiser: the real issue is hidden behind a smoke screen of existing tests.

Dice Estimation

Although I am a proponent of "#noestimates," that is - the concept that estimation is often the consequence of unsound mental models, and in itself somewhat wasteful, because we often don't know - I still propose an estimation technique that is as useful as it is simple. BUT - it requires context. Let's epxlore this context:

Problems with estimation

Regardless of how we put it, one major challenge is that many organizations use estimates as if there was certainty that they can become "correct" - which is a common dysfunction, because it results in undesirable, wasteful behaviours. These could be, for example, as upfront design, padding, "challenging estimates" - and even post-hoc adjustment of the figures to match reality. None of these behaviours help us deliver a better product, and they may even impede quality or time-to-market.

All these aside: we don't know until we tried - that is, there's this thing called "the high probability of low-probability events," so regardless of how much effort we waste on trying to get those activities "right", we could still be wrong due to something we didn't factor.

These are the common reasons for the #noestimates movement which simplys reject the discipline of upfront estimation in favor of monitoring the flow of work via execution signals to gain both predictability and control. You may ask: "If estimation is so obviously flawed, then why would you propose yet another estimation technique?"

Category Errors

A common challenge teams face is that not all work is equal: Giving a trivial non-tech example, it doesn't take much expertise to realize that building a tool shack isn't the same category as building a house - and that's not the same category as building a Burj Khalifa.

By simply letting uncategorized work flow into teams, we risk that extremely large packages block the flow of urgent high-value items for a prolonged time. Hence, a first categorization into "Huge, Large, Small" does indeed make sense - The key realization is that these aren't equal.

Hence, dice estimation first categorizes work into the right bin: everything that's workable can be taken up by the team, while everything that isn't workable should be managed appropriately before being worked upon.

Six-sided dice

Story Points worked out terribly, and they led to many dysfunctions - so let's avoid that side track. Instead, let's say that we estimate items in dice values. For small items, our estimation unit would be team-days, for large items, it would be weeks, and for large items, it would be months.

That also leads us to the first logical connection:
One Small Dice at 5 or 6 could already classify for a "1" on a medium dice. We could think about splitting it in two if we want to take it on. On items bigger than 5, we should exercise caution: do we understand it properly, is the cost of implementation warranted - is it really the best thing to do next?

This also applies when an item exceeds 5 on the week count - at that point, it becomes a huge dice: we might want to check a lighweight business case and pivot before proceeding.

How Dice Estimation works

Dice estimation is very easy - you simply ask, "If each day of work was a dice roll, what would be the amount of sides on our dice so that we have enough time to hit a '1' - on average - by the time we are Done?"

That is: if the team expects to be done on 2 days, they would choose 2. If they would expect 3 days, they would choose 3 - if they expect 6, it would be 6 days.

Measuring Confidence

Since dice rolls aren't linear, this form of estimation focuses on levels of confidence in completion within a predefined period of time. Hence, despite the fact that we estimate in days, dice estimation values confidence - and thereby clarity and understanding - above quantity of work.

Estimating with uncertainty

Just like a D6 doesn't mean that "we will roll 6 times, then we will have rolled a 1" - what we estimate is the amount of faces on the dice and thereby the expected amount of days that the item will be done.

Think of it like this: If you have a coin, you have a 50% chance to get heads on the first try - and while statistically, you'd expect to get at least one heads in 2 attempts, it's theoretically possible that you need to flip the coin an infinite amount of times and never get heads.

The estimate in team-days helps us manage our work on the Dailies in an efficient manner - as long as the estimated amount of days hasn't elapsed, it's safe to assume we are on track. Any item reaching and/or exceeding the estimate should be scrutinized: are we stuck, did we misunderstand something - what is going on? The "estimate," thus, isn't even estimating effort - it's forecasting a pivot-or-persevere checkpoint so that we minimize sunk cost.

Lightweight estimation

We can estimate work items either as they arrive, during refinement, or when we get busy - that doesn't even matter. The key reasons for estimation are first, avoiding category errors and second: setting up an intuitive checkpoint so that we don't run out of bounds.

With Dice Estimation, we avoid category errors and institute checkpoints in execution

Of course, you could use dice estimation during Sprint Planning or in a PI-Planning as well, but you don't even need cadences to take advantage of Dice Estimation.

Predictability and Forecasting

As we know, a dice averages at 3.5, so even if we didn't do anything yet except categorizing by dice size, we know that our backlog of workable, small items would take about 2 weeks for 3 items. Once we've done a little bit of work and know how much the completed items took, we can either run a simple Monte Carlo Simulation or apply the Law of Large Numbers to figure out how much known work we have ahead of us. The same applies to the Large and Huge dices, which gives us a fairly decent understanding of when an item might be delivered, based on backlog position.

Not a Performance Metric

The dice metaphor should make it easy to explain how neither can one measure the perfomance of a dice-roller based on how often they roll a certain number, nor does it make sense to "squeeze" estimates - the probability of guessing the right number won't increase by restricting the "allowed" numbers that can be picked. If anything, squeezing estimates would lead the team to spend more time on examining exceptions - hence, reducing performance.

Why you should switch from Story Points to Dices

Dice Estimation is a minimum effort method of right-sizing work that helps with:

1. Sufficient predictability
2. Easy plannability
3. Execution checkpoints
4. Avoiding large batches that block flow
5. "Split, Pivot or Defer" decisions

Ten things eerily close to Scrum that you may misunderstand

 There are some ideas around Scrum that sound a whole lot like they're based on the Scrum Guide - when, indeed, they aren't. Even worse: you might be doing them in a way that could cause problems. Let's dig in.

1 - Scrum Roles

If I ask you what the Scrum roles based on the Scrum Guide are, you're probably very quick to answer: "Scrum Master, Product Owner, Developer."

Wrong.

What? How can that be wrong?

Well, because the Scrum Guide doesn't mention any roles. Indeed, it doesn't even use the term, "role" any more. Scrum Master, Product Owner and Developer are merely accountabilities. That means, someone has to be accountable.

2 - Dedicated Scrum Master

The concept of a "dedicated Scrum Master" isn't mandatory based on the Scrum Guide. Neither is the concept that "a Scrum Master shouldn't be technical, so they don't dump their own bias on the team."

These ideas are often marketed in an attempt to provide job safety for the myriads of people who can do nothing else. You can do Scrum very effectively even if Scrum Mastery is an accountability that rotates among developers, for example, on a per-Sprint basis.

Caveat - they need to know what they need to do, and have enough time for doing it.

3 - Product Owners aren't developers

Neither is the concept of a "Product Owner who isn't a developer, so they don't interfere in the work" prescribed by Scrum. It's entirely possible that, for example, a senior developer assumes the PO accountability. As long as the Product Goal is clear, the Product Backlog well-maintained, Sprints deliver high value and stakeholders know what to expect, there's not going to be much of an issue.

4 - Team autonomy

Are Scrum teams really autonomous? Doesn't Scrum rely on team autonomy?

Try searching for the term "autonom" in the current Scrum Guide - you'll be surprised! Team autonomy isn't mandatory for Scrum. In fact, in larger organizations, it can't be - because if you have larger products, multiple teams need to collaborate.

Before proceeding, let that sink in:
Scrum teams are not free radicals.

5 - Each team has their own Product Owner and Product Backlog

Certain "scaling" approaches suggest that each team has their own Product Owner and Backlog. Well - for a Sprint backlog, that's true. But having a separate Product Backlog for each team adds problems rather than solving them. The Scrum Guide states that multiple teams working on the same product, "should share the same Product Goal, Product Backlog, and Product Owner."

Note that this isn't a Scrum rule, only a suggestion. I will leave it as an exercise to the reader to figure out why that is a good suggestion though. And if the answer is too difficult, try the book "Scaling Lean and Agile Development" by Larman and Vodde.

6 - Teams have their own Definition of Done

One of the first exercises Scrum masters typically do with a new team is to draft up their own Definition of Done. That's not necessary in larger organizations, because the organization could already provide a DoD. "But that's not Agile..." - no! On the contrary, it ensures that the term "Done" has a consistent meaning anywhere in the organization. It reduces complexity and misunderstanding. It's quite irritating when a stakeholder has to ask in every Sprint Review, "What ... exactly ... does 'Done' mean, when you say that?" 

Teams are encouraged to add details to the organizational Definition of Done. If they deviate from the organization-wide DoD, though, they invite trouble. Caveat - an organizational DoD should be absolutely minimal, lest it slows down teams with pointless busywork.

7 - Refinement meetings

"In order to prepare for the upcoming Sprint, the Product Owner invites the team for at least one Refinement meeting during the ongoing Sprint." That's a really common practice - but it's not what Refinement is!

Refinement is no Scrum event, for a good reason. Taking a peek at the top couple backlog items, maybe doing a small spike, or creating a wireframe, are all activities that can't really be done very effectively in a centralized meeting. They're better done by brooding over the items from your desk. And sometimes, reaching out to a user has a delay, and we really need the answer before moving forward. The asynchronity of refinement also ensures we don't disrupt the flow of work by having yet another meeting on our calendar.

8 - PO acceptance

During the Review, the team demos their work to the Product Owner, who then accepts it.

Search for anything even remotely resembling this concept in the Scrum Guide. It just isn't there. This sentence contains so many flawed concepts that if you practice this, I wholeheartedly advise a Scrum refresher training. 

The Review is about stakeholder alignment, and it's a working session, not an approval meeting.

The Product Owner also doesn't supervise any work done by developers - together with all the stakeholders, they inspect the outcomes of the Sprint, regardless of how much work was done or not done. 

Nobody "accepts" individual work items of the team. Developers meet their DoD, and that's it. If the DoD isn't adequate or the backlog items not sufficiently valuable, that's not a problem we fix by adding an approval step to the Review. We fix it by improving our DoD, refinement and planning approaches.

9 - One Increment per Sprint

This is probably one of the oldest misunderstandings of Scrum - that at the end of each Sprint, the team delivers one Product Increment which contains all the changes made to the product for the duration of the Sprint.

An increment is produced whenever a modification has been made to the product, and it's in a usable state. Scrum and MinimumCD are not at odds. Scrum teams can - and actually should - produce as many increments as possible during a Sprint, and also deploy and/or release them as early as possible.

During the Sprint Review, the sum of all increments produced since the last Sprint Review is inspected, as these are the outcomes of the Sprint. This sum could be anywhere from zero to infinity. A team which only works towards a single, integrated Increment at the end of a Sprint, will be much more likely to find themselves empty-handed, so that's a bad strategy to begin with.

10 - Backwards-facing Retrospectives

"The Scrum Team discusses what went well during the Sprint, what problems it encountered, and how those problems were (or were not) solved."

That's a literal quote from the Scrum Guide, so isn't isn't the Retrospective pattern of "What went well, what didn't go well, what we could improve?" - the best way to conduct a Retrospective? No.

"The Scrum Team identifies the most helpful changes to improve its effectiveness." - that's also a quote. Of course, we need to have consider what happened in the last Sprint. The purpose of our Retrospective, however, is looking forward, identifying the most helpful changes. If all you do in your Retrospectives is dwell on the past and make miniscule adjustments so that the same old problems don't constantly haunt you, you're not future-proofing your team.

The most helpful change you can make is that which will make you most successful in the future - which may not necessarily be fixing a problem you had in the past.

Bonus - Ceremonies

It sounds like an innocent mistake, but there's a huge issue hidden behind this label. Scrum events are called "event" instead of "ceremony" for a reason.

An event is, literally, "when something notable happens."
A ceremony is, literally, "doing what we always do in the way we always do it." 

Organizations implementing Scrum "ceremonies" usually find themselves getting very low value from these, not understanding why they're important - and not thinking about better ways to achieve better outcomes. 

As a consequence, we see purely mechanical Scrum which helps nobody, gets on developers' nerves, and the one thing that makes agility tick - Double Loop Learning - is overboarded before it ever started.

Why test coverage tagets backfire

Most organizations adopt a "Percent-Test-Coverage" metric as part of their Definition of Done. Very often, it's set by managers, and becomes a target which developers have to meet. This approach is often harmful. And here's why:

The good, bad and ugly of Coverage Targets

Setting coverage targets is often intended as both a means of improving the product, as well as improving the process - and getting developers to think about testing.

The good

It's good that developers begin to pay closer attention to the questions, "How do I test this?" as well as, "Where are tests missing?" Unfortunately, that's about it.

The bad

The first problem is that large legacy code bases starting with zero test coverage will put developers into a pickle: If I work on a component with 0% test coverage, then any amount of tests I write will still keep that number close to Zero. Hence, the necessary compromise becomes not setting a baseline number, just ask for the number to increase. The usually envisioned 80+% targets are visions for a distant future rather than something useful today.

Looking into the practice - as long as the organization is set up to reward minimizing the amount of time invested into unit testing, the outcome will be that developers try to meet the test coverage targets with minimum effort. 

Also, when developers have no experience in writing good tests, their tests may not do what they're supposed to do.

The ugly

There are many ways in which we can meet coverage targets that fulfill Goodhart's Law:

Any metric that becomes a target stops being useful.

Often, developers will feel unhappy producing tests that they create only to meet the target, considering the activity a wasteful addition to the process, which provides no benefit.

At worst, developers will spend a lot of time creating tests that provide a false sense of confidence, which is even worse than knowing that you have no tests to rely on.

But how can this happen? Let's see ...

The first antipattern is, of course, to write tests that check nothing.

Assertion-free testing

Let's take a look at this example:

Production code

int divide(int x, int y) { return x/y; }

Test

print divide(6,2);

When looking at our code coverage metrics, we will see a 100% coverage. But: the test will always pass - even when we break the production code. Only if we were to inspect the actual test output (which is manual effort that we probably won't do) - we will what the test does: There is no automated failure detection, and the tests aren't even written in a way that we would detect a failure: Who would detect the problem if suddenly, we got a "2" instead of a "3" - we don't even know which result would have been correct!

Testing in the wrong place

Look at this example:

Class Number {
int x;
void setX(int val) {x=val;}
void getX() {return x;}
void compute() { x=x?x/x^x:x-x*(x+x); }
}

n = new Number();
n.set(5);
assert (N.x == 5);
assert (N.get() == 5);

In this case, we have a code coverage of 75% - we're testing x, we're testing the setter, and we're testing the getter.

The "only" thing we're not testing is the compute function, which is actually the one place where we would expect problems, where other developers might have questions as to "What does that do, and why are we doing it like that?" - or where various inputs could lead to undesirable outcomes.

Testing wrongly

Take a peek at this simple piece of code:

int inverse (int x) { return 1/x; }
void printInverse(int x) { print "inverse(x);" }

assert (inverse(1) == 1);
assert (inverse(5) == 0);
spyOn(print);
printInverse(5);
assert (print.toHaveBeenCalled());

The issues

There are numerous major problems here, all of which in combination make that test more dangeous than helpful:

Data Quality

We may get a null pointer exception if the method is called without initializing x first, but we neither catch nor test this case.

Unpexpected results

If we feed 0 into the function, we get a Divide by Zero. We're not testing for this, and it will lead to undesired outcomes.

Missing the intent

The inverse function returns 0 for every number other than 1 and -1. It probably doesn't do what it's expected to do. How do we know? Is it poorly named, or poorly implemented?

Testing the wrong things

The print function's output is most likely not what we expect, but our tests still pass.

Conclusion

If we rely on coverage metrics, we might assume that we have 100% test coverage, but in practice, we may have very unreliable software that doesn't even work as intended.

In short: this way of testing tests that the code does what it does, not that it does what it should.

Then what?

The situation can be remedied, but not with numerical quotas. Instead, developers need education on what to test, how to test, and how to test well.

While this is a long topic, this article already shows some extremely common pitfalls that developers can - and need to - steer clear from. Coverage metrics leave management none the wiser: the real issue is hidden behind a smoke screen of existing tests.

Bugfixing team - good or bad idea?

In many organizations struggling with quality, a gut reaction is setting up a "bug fixing team," which has only one purpose: Cleaning up the myriads of known bugs in the software so that other teams can focus on new features.

There's a concept that "developers can go faster if they don't have to worry about the bugs in their code." This concept is a delusion, puts unrealistic expectations on the bugfix team, and will ultimately lead to disappointment.

Why bugfix teams are bad

Bugfix teams are a counterproductive workaround stopping the organization from solving their actual problems:

• Developers don't own their work. They can fire-and-forget, because it's no longer their problem after delivery.
• Developers don't fully understand how their code actually works, hence there's nobody who has deep knowledge of the inner workings of the product.
• Developers don't learn how to write better code, because they don't have to work through bad code.
• Technical debt continues to grow, because the Bugfix team will only work on known problems while others introduce more problems.
• Building a new feature on a buggy baseline doesn't add to low quality - it multiplies.
• The perceived progress of the delivery teams is an illusion, because it doesn't mean that the delivery doesn't contain a time bomb.
• The Continuous Improvement process is subjected to quantity of output, which is the very reason why the problem occurred to begin with.

I have been part of Bugfix teams. It's a frustrating, ungrateful job and an uphill battle unless all other developers also take responsibility for the quality of their own work.

How bugfix teams can succeed

There is a way to set up a successful Bugfix team, however:

•  The "Bugfix" team is a team like any other, but they can fully commit to - and focus on - improving quality and have no commitment on new features. And they get a spotlight in stakeholder reviews to demonstrate their improvements.
•  All other teams practice a rigorous Stop-the-Line process and do everything in their power to not let low quality pass downstream. The policy is, "Bugfix team improves fundamentals, others stop deterioration and improve within the context of delivery."
•  The Bugfix team responsibility rotates.

Technically, that will make the Bugfix team a "legacy rewrite" team rather than a "locate, isolate and fix the bug" team. Their mission shaping the future of the product, not merely dealing with its past.

Conclusion

My stance on the topic is, "no Bugfix team unless you can clearly explain how you're actually going to solve the problem that leads you to needing it."

You can't solve the problem of low quality simply by throwing money at it.

Test Automation Specialists - what a waste!

"Test Automation Engineers require someone who will write the test cases for them, and they can start automating right after the software is deployed onto the test environment."

It's 2022, and I just had this conversation yesterday. Seriously?

Overspecialization leads to Waste

If we work like that, we needlessly create bottlenecks, we test too late, our tests will be of low quality, and we will spend a lot of time on identifying, tracking and reworking defective deliveries. We produce a lot of work, but not quality.

In a well-functioning software organization, there can be no such role as a "test automation engineer" who is fed work by "test designers." 

Although it sounds "resource efficient," that approach generates massive amounts of waste in all categories of Lean Muda: 

Defects

When automated tests aren't written by developers, and only available once a software package is released, we end up with three kinds of defects:

1. Product defects, i.e. things the product should (or shouldn't) do, but that weren't correctly implemented, which require another software update.
2. Automation defects, i.e. things the test automation should (or shouldn't) do, but that weren't correctly implemented. Until clarified, these lead to significant communication overhead.
3. False negatives, i.e. inadequate automation not catching existing defects. This is significantly more likely when automation isn't written close to the code, and might omit dangerous cliffs in the product implementation.

Handover

Whenever actors in a process change, there's a handover. At the point of handover, we induce delay, lose information and risk the introduction of defects. This makes our process slow and error-prone. When test itself becomes a source of error, we should stop and think.

Motion

Handovers also mean that we're moving artifacts: test descriptions, build versions and defects - without need. We also need to coordinate this motion, which means we induce additional meetings into our process, which also slows us down and adds non-value added overhead. Motion also leads to inventory waste.

Inventory

As previously mentioned, un-automated tests will pile up as inventory, as will un-tested deliveries. When tests fail, the defect notifications are made available at a point in time when developers are working on other things. Hence, we are adding three queues to our process. Each queue contains work waiting to be done, and the combined length of all these queues is value delay for our customers.

Waiting

As mentioned in my article on CI/CD industry benchmarks, the time required between when a developer makes a change, and when a developer receives feedback on whether everything is correct shouldn't exceed a few minutes, preferably just a few seconds. If automation begins upon delivery, this period of time is impossible to warrant. The delivered product has to wait a significant amount of time until it can produce customer value, and developers will turn their attention elsewhere.

Overprocessing

Having different people automate tests and write productive code leads to "overprocessing," often called "gold-plating:" As mentioned in my article about the test pyramid, we should try to automate as close to the code as possible. When we're doing high-level tests which could instead be low-level tests, we make our test suite more complex, slower and less reliable than possible.Unit tests are extremely quick both to write and execute. Test automation engineers who automate black box/gray box tests will spend a significantly higher amount of work to write tests with poorer outcomes.

Overproduction

Since the amount of labor to create a product increment in server code and test automation are extremely hard to balance, the strategy of automating after the fact usually leads to developed product features piling up, waiting to be automated. As a consequence, the decision is often to release these without proper test automation coverage to flush the waiting queue. This builds up "technical debt" which deteriorates product sustainability.

Human potential

When developing software with multiple people, we act as a team. We can learn from each other, and we can align: Someone who can automate tests, can also write features. Someone who can write features, can also automate. As a team, we can align during planning "who does what," and fill our information gaps so that those doing the work have full information and don't need anyone else to fill in for them. Techniques like pair programming or code reviews ensure that even if one person makes a mistake, someone else will catch it before a defect even enters the system. It's a limiting belief to think that somehow people can't learn - we all can, if the conditions are favorable. Maybe not everyone will be the greatest developer, but we don't need to: we work within our ability, and grow our capacity a little every day.

To confine a test automation engineer to automating defined test specifications, and to confine coders to write code without understanding how to do that without defects - is an insult to our human intellect, and a very expensive one, at that:  All the above seven wastes cost a lot of money that we could save if we would just improve our collaboration, communication and skillset.

Lean Software Test Automation

Please, let me clarify. I don't believe that test automation expertise is a waste - the waste is actually hidden in the process, and how we use our talent.

A much leaner approach which avoids all of the above eight wastes is to educate and empower - use QA to help developers learn how to define the right tests, how to apply processes like Test-Driven Development and Behaviour-Driven Development, and guide them on their journey to deliver Zero Defect Quality. And in reverse, having developers upskill our test automation experts to help them deliver integrated pieces of value including working software.

The purpose of test automation in such an organization isn't to find defects - it's to maintain the high frequency of delivery of high quality products, and to serve as a living documentation for all behaviours of the product which is maintained as part of the development process without overhead.

Decision-making in an agile organization

A challenging question in Agile organizations is: "Which decisions should be made where?" - we quickly end up in a heated debate of "team autonomy versus command and control." A red herring - and a false dichotomy that misses the big picture. There is a better way.

Making decisions

Let us first discuss briefly about the three ways that decisions can be made in companies:

Top-Down

Traditional organizations may be most familiar with this type of decision. When something needs to be decided, a manager needs to be informed about what is to be decided, preferably also with some information on the consequences of various decision types. At a convenient moment in time, the manager will decide, and the affected parties will act based on the decision.

Depending on how information is presented to the manager, the manager may decide based on an unknown bias. Managers are often unable to fully understand the consequences of their choices. In turn, teams have to deal with consequences that may not even make sense to them, but they lack the power to change things. The system encourages currying favor over full transparency, and a "making things look good" is worth more than "doing what is right." 

A lot of time is lost in presenting information to management in order to prepare the decision, even when the decision is seemingly obivous. Revising the decision requires significant evidence as well, even after the negative repercissions are already visible. Hence, top-down decision on the work are rarely efficient, effective or even helpful. And that's why they are avoided in leaner organizations wherever possible.

Autonomy

Agile organizations, especially those centered around Scrum, emphasize team autonomy - and thereby, autonomous decision making. Many developers don't like managers interfering with their work, and prefer this way of making decisions. Some get very religious on the "autonomy" part, and will insist on teams getting their way.

Autonomous decisions are both feasible and efficient when the team operates in isolation: As long as it works for the team, we're good. When the team isn't happy with a choice, they can quickly revise it and improve.

Team autonomy sounds great, but isn't a feasible approach when multiple teams interact.

Let's say that team A would like to define a customer as firstName and lastName referenced by customerID, and team B would like to define the customer as fullName and title referenced by customerRefID: Who is right?

If A and B decide autonomously, the system won't integrate, so everyone is "wrong."

Should management decide?

That brings us to the third option, which requires willingness to compromise:

Federation

Let us redefine team boundaries to objective boundaries. People contributing to the same goal, working in the same context, or on the same products, need to collaborate, lest they get stuck dealing with friction rather than making progress.

In larger organizations, we have multiple teams. They might be planning, working and delivering independently - yet, at some level of abstraction, they share the same goal.

As an example, if we're working on an Online Shop, we benefit little if our checkout process is optimized at the expense of user registration: our total revenue might go down if we force people to provide payment data upon registration already. Neither the "Checkout," the "Payment," nor the "Registration" teams can autonomously decide which approach is the best - they need to collaborate!

Thus, a "federation" is born: People whose work overlaps for one reason or another can federate. A federation compromizes on local autonomy to optimize globally. We have some centralization, and some decentralization, and we apply a higher-ordered decision framework to determine both how, and what, we need to centralize.

A federation offers us fast and efficient team-level decisions, as well as sustainable and effective overarching decisions.

Guardrails of federated decision-making

A number of common pitfalls can render a federation ineffective:

1. Not involving people who are affected by outcomes.
2. Involving people in the discussion who aren't involved in the outcome.
3. Forming decision queues.

This happens when we violate the general rule of thumb that any federation should decide as little as possible, and as much as necessary. 

The following decision-making flow can assist us in determining which decisions we would like to keep autonomous, and which are better to federate.

In sequence, we can ask three questions to determine whether we require a central federated decision session:

1. Will the decision affect others?
• We only need to involve those who are affected.
• If we affect nobody outside the team, we don't need others to agree.
2. Will the decision have long-term impact?
• Easily reversible choices can be subjected to experiment, and inspecting outcomes.
  
• Decisions that have a high cost of change require deeper discussions.
  
3. Is the decision highly time-critical?
• If there is a high cost of delay, "it's easier to ask for forgiveness than get permission."
• If cost of delay is lower than cost of change, the decision should be made centrally.

As we know from governments, federation has a cost attached and easily leads to bureaucracy. Hence, with every centralized decision, we should inspect and adapt on, "What are the costs and benefits of centralizing this?" - and, "Are there things we could change so that we could do this autonomously?"