Tuesday, January 19, 2021

The problem of Black Box Tests

One of the most fundamental enablers of agile ways of working is the ability to swiftly and reliably detect problems in your product - that is, to test it efficiently and effectively. Unfortunately, the "traditional" approach of black-box testing a running application is hardly useful for this purpose. 

I have created an executable use case to illustrate the problem. 


Let's take a look at this little service, and assume that it would be your responsibility to test it in a way that you can reliably tell whether it works correctly, or which problems it has:

You can call this service in the mini applet included on this page if you have JavaScript enabled..

Alea iacta est

You need Javascript to run this demo.

Yes, just try it out - roll the dice!
That's about as simple as an application can get.
Can you imagine how much effort it takes to properly test this simple application?

It's not enough to roll the dice once and get a number between 1 and 6 - how do you know that there isn't a possibility that the application might generate results outside that range?

And how would you know that you have fair dice? Call the service a thousand times and assume that you would get an approximately even distribution of values? What would be your thresholds for assuming that the dice are "fair"? What if 5% or fewer, or 25% or more results go to one number, which is statistically still possible with a decent probability?
You see the difficulty already.

But let's make this more difficult:

Hello

You need Javascript to run this demo.

What if I told you that this is a call to the same service?
Yes, exactly - you didn't know everything the service does when you created your test concept before.
There's a different feature hidden in the service: if you pass a user name to the request, it will greet you!

This adds a whole new dimension to the test complexity: you have to test with - and without - a user name. And would you want to try different user names?

  But that's not everything:

You lose!

You need Javascript to run this demo.

Did you even catch that this one behaves different?
What if I told you that this is another call to the same service?
Yes, exactly - you still didn't know everything the service does when you created your test concept.
There's another different feature hidden in the service: you can load the dice and cheat!

If you tell the service to cheat you, you will get unfair dice.

So, now you need to run your entire test set from above twice again - with and without cheating.

And we haven't even looked into whether there are multiple ways of cheating, or whether the cheating function always triggers correctly when the variable is set (hint: it doesn't). Good luck without knowing the application where the malfunction is.

But we're not done yet:

I win  

You need Javascript to run this demo.

Did you catch the difference here?
What if I told you that this is yet another call to yet again same service?

There's yet another different feature hidden in the service: if I use my name to cheat, I will get loaded dice in my advantage!

By now, you're probably doubting whether you understood the application at all when you started testing it.

The code

Now - let me blow your mind and tell you how little source code was required to totally blow your test complexity and effort out of proportion:


That's it. This little snippet of code is entirely sufficient to keep a Black Box tester busy for hours, potentially days, and still remain unable to make a reliable statement on whether they missed anything, and which problems the product may or may not have.

Depending on how your application is designed, a few minutes of development effort can generate a humongous mountain of effort in testing.
 
And that's why you can't possibly hope to ever achieve a decent test coverage on an application without knowing the code.

Testability

There's another problem: this code wasn't written with testing in mind (or, much rather: purposely written with poor testability in mind -- hee hee) so you have no way of ever coming up with an easier way to test this service, until it's rewritten.

And that's why you can't maintain sustainable high quality unless developers and testers actively collaborate to build highly testable software that is easy to work with, both for software changes and testing. 

Think of minimizing sustainable lead time - consider the total effort from request to release, and consider it both for initial creation and future modification. There's no point in optimizing for development speed if you slow down testing more than that, and likewise, there's no point in delivering minimal code if the consequence is totally bloated test scope.

Otherwise, you'll not be very agile.

Friday, January 1, 2021

Low-Code, No-Code, Full-Code - The Testing Challenge

In the Enterprise world, there's a huge market for so-called "Low Code" and "No Code" Solutions, and they do have a certain appeal - you need to do less coding, and as such, need less developers, to achieve your business objectives, because they bring a lot of "out-of-the-box" functionality. 

So why is it even something to talk about - and how does that relate to "Agile" ways of working?


Let's explore this one from a quality perspective.


The Paradigms

No-Code: Configure and Go

"No Code" solutions are especially appealing to organizations that have no IT department and are looking for something that someone without IT knowledge can configure in a way that's immediately useful.

An early implementation of no-code platforms will typically not even include a staging environment where people try out things. Many times, changes are immediately live, on a productive system. That's great for small organizations who know exactly what they're doing because it absolutely minimizes effort and maximizes speed.
It turns into a nightmare when someone, somehow, by pure accident, managed to delete the "Order" object and now you're happy-hunting for a couple thousand unprocessed orders that your angry customers are complaining about - with no way to remedy the system.

And it turns into an even worse nightmare when the system doesn't do what it's supposed to do, and you've got a chance smaller than hell freezing over of figuring out why the black box does what it actually does instead of what it's supposed to do.

When introducing Quality Assurance on a No-Code platform, organizations are often stuck using third-party testing software that uses slow, flaky, difficult-to-maintain, expensive UI-based tests which will eventually get in the way of high speed adaptability. Clean Code practices applied to testing are usually a rare find in such an environment.


Low-Code: Configure and Customize

"Low Code" solutions are especially appealing to managers who are out to deliver standardized software to their organization fast. Many of these systems bring huge chunks of standard capability out-of-the box and "only need customization where your organization doesn't do what everyone else does."

That sounds appealing and is a common route in many organization, who often find out only years after the initial introduction that "you can't sustain your market position by doing what everyone else does" - your business does require a lot of customization to stand out in the market, and the platform often doesn't accommodate for that. 

Most vendor solutions don't provide a suite of functional tests for your organization to validate the standard behaviour, which means you often end up creating duplicate or highly similar code in your customization efforts - or use standard functions that don't do what you think they would. Worse yet, many use proprietary languages that make it very difficult to test close to the code. In combination, that makes it extremely hard to test the customization you're building, and even harder to sustainably keep the platform flexible.



Full-Code: Design, Build, Optimize

"Full Code" solutions sound like the most effort and the slowest way of achieving things. But looks can be deceptive, especially to a non-expert, because a modern stack of standard frameworks like Spring, Vue and Bootstrap, can literally make it a matter of minutes for a developer to produce the same kind of results that a low-code or no-code platform configuration would provide, without any of the quality drawbacks of Low-Code or No-Code.

Your organization has full control over the quality and sustainability of a full-code solution. It depends entirely upon what kind of engineering practices you apply, which technologies you use and which standards for quality you set for yourself.


Quality Control

To sustainably high quality at a rapid pace, you need full quality control:
  • You must be able to quickly validate that a component does what it's supposed to do.
  • You must be able to quickly figure out when something breaks, what it was, why and how.
  • When something breaks, you must be able to control blast radius.
  • You need a systematic way of isolating causes, effects and impact.
The most common approach to maintain these is a CI/CD pipeline that runs a robust test automation in the delivery process. To make it feasible that this control is exercised upon every single change that anyone makes at any point in time, it should not take longer than a few minutes, lest people are tempted to skip it when in a hurry.

The problem with both No-Code and Low-Code solutions is: In many cases, such platforms aren't even built for testability, and that becomes a nightmare for agile development. Instead of running a test where and how it is most efficient to run, you invest a lot of brainpower and time into figuring out how to run the test in a way that fits your technology: You have subjected quality to the technology, instead of the other way around!

In a low-code environment, this can become even more problematic, when custom components start to interfere with standard components in a way that is unknown and uncontrollable in a huge black box.


Non-functional Quality

Although I would not outright suggest to opt for a full-code solution (which potentially is not in the best interests of your organization, and it's entirely implausible without skilled developers), I would like to share a list of non-functional quality attributes that may not be considered when selecting a new system, platform or service.

In order to remain agile - that is, to be able to quickly, effectively and easily implement changes in a sustainable manner - your platform should also accommodate for the following non-functional quality requirements:

Factor Decisions
Testability
How much effort is it to test your business logic?
This must go far beyond having a human check briefly whether something works as intended. It needs to include ongoing execution, maintenance and control of any important tests whenever any change is made. And remember: any function you can't test may cause problems - even when you're not intentionally using it!
Traceability
How closely are cause and effect related?
You don't want a change to X also to affect Y and Z if that wasn't your intent! Are you able to isolate changes you're making - and are you able to isolate the impact of these changes?
This should go for the initial setup as well as for the entire lifecylce of the product.
Extensibility
How much effort does it take to add, change or remove business logic?
Adding a form field to a user interface is a start, not the end. Most of your data has a business purpose, and it may need to be sent to business partners, reported in finance, analyzed in marketing etc. How much effort does it take to verify everything turns out as intended?
Flexibility
How often will you be making changes?
If you're expecting a change a year, you can permit higher test efforts per change, but when you're looking at new stuff in a weekly manner, you could be overwhelmed by high test or change efforts, and cutting corners will become almost inevitable.
Security
Can you really trust your system?
Although every system could have vulnerabilities, and standard softwares tend to have fewer, but how can you test for Zero-Days unless you can fully test the intricate inner workings?
Also, some legislation like GDPR forces you to expose certain data processings, and you may need to provide evidence what your system does in order to do that. This is extremely difficult when some behavioural description of certain aspects are a black box.
Mutability
How much effort would it take to migrate to a new platform and decommission the current platform?
When you introduce a system without having an understanding of how much time, effort and risk is involved in a migration or decommissioning initiative, it might be easier to kill your current company and start another business than to get rid of the current technology. That means you could find yourself in a hostage situation when the day comes that your platform is no longer the best choice for your business, and you have no choice except continuously throwing good money after the bad.

As a general rule of thumb, low-code and no-code platforms tend not to emphasize these, so the value your organization places on these non-functional requirements correlates with the plausibility of selecting this approach.

Conclusion

With a lot of these to be said, if you're in the comfortable situation of introducing a new technology, ensure that you check the non-functional requirements and don't get blinded by the cute bucket of functionality a low-code or no-code solution may offer. If your platform does poorly especially on traceability, testability or mutability, you're going to trade off your agility for some extremely painful workaround that could increase the Cost of Ownership of your solution beyond feasible limits.

It wouldn't be the first time that I'd advise a client to "trash everything and start with a blank folder. Within a few years, you'll be faster, have saved money and made better business."

Culture Conversion

Many times, I hear that "SAFe doesn't work" both from Agile Coaches and companies who've tried it, and the reasons behind the complaint tend to boil down to a single pattern that is missing in the SAFe implementation - culture conversion. Let's explore why this pattern is so important, what it is, and how to establish it.



The Culture Clash

Many enterprises are often built upon classical management principles: Workers are seen as lazy, selfish and disposable "resources". Decisions are made at the top, execution is delegated. We have a constant tug-of-war between "The Business" and "Development". All problems are caused by "Them" (irrespective of whom you ask) - and the key objective is always to pass the next milestone lest heads roll.  There is little space for face-level exchange of ideas, mutual problem solving, growth and learning.

If you try to use an agile approach, which is built upon an entirely different set of principles, practices and beliefs, you'll get a clash. Either workers care, or they don't. Either people are valuable, or they aren't. Either they can think, or they can't. You get the idea. Behind that is a thing called "Theory X/Y." 

Self-fulfilling prophesy

When you treat people like trash, they'll stop caring about their work. When you don't listen to your developers, they fall silent. When you punish mistakes, workers become passive. And so on. This lose-lose proposition turns into a death spiral and becomes a self-fulfilling prophesy.

Likewise, when you create an environment built upon mutuality, trust and respect, people will behave differently. Except - you can't just declare it to be so, and continue sending signals that the new values are "just theoretical buzzwords that don't match our reality." Because, if you do that, this will again be a self-fulfilling prophesy.


Breaking the vicious circle

You can't change everything overnight, especially not an entire organization. Some people "get it" immediately, others take longer. Some may never get it. Even when you desire and announce a new culture, it can't be taken for granted. You have to work towards it, which can be a lot of effort when dealing with people who have built their entire careers on the ideas of the old culture.  

Resilience over robustness

A lot of this doesn't happen in the realm of processes, org charts and facts - what's truly going on happens mostly in the realm of beliefs, hopes, fears. As such, problems are often difficult to identify or pinpoint until a dangerous symptom becomes manifest. Hence, you can't simply re-design an organization to "implement" this new culture. The best you can do is institute checks and balances, early warning mechanisms, buffer zones and intentional breaking points.

Buffer Zone

Often, you may need time to collect striking evidence that would convince others to let go of certain un-helpful practices. These might include, for example, HR policies, project management or accounting practices. When you can't quite yet eliminate these things, it's quite important for the culture conversion to also include a conversion of such activities, so that they don't affect the teams. At the same time, you need a strategy laid out with clear targets for abolishing these things, lest they become "the new normal" and culture converters start believing them to be right or even essential.


The Culture Conversion Pattern

When you operate in an environment where cultural elements that conflict with the intended future culture exist and will likely interfere with the sustainability of the change, you need mechanisms that let you:

  • Establish the desirable culture
  • Minimize undesirable culture infringement
  • Mitigate damage from culture infringement
  • Breaking points when undesirable culture gets too strong
  • Identify culture clash

Specific people must take on this responsibility, it's not sufficient to say "We should do this." Someone must be in control of these activities and the entire organization must rigorously apply the above mechanisms, inspecting and adapting relentlessly upon failure.

Failure on any of these will provide a backdoor for the existing, undesirable culture to quickly usurp the new culture, and the culture change will fail.

The SAFe Zone

A healthy SAFe organization would institute the "Program Level" to provide exactly this resilience for culture conversion. The Product Management function would protect the agile organization against low value work and overburden, the RTE function would safeguard against Command and Control, and the architect would be the bulwark against unsustainable engineering. Product Owners and Scrum Masters would provide an additional safety cushion to protect the teams.

These roles must unite to drive the need for transparent, un-political value optimization, mutual collaboration and quality-focused development practice both towards the teams and the non-agile surrounding organization.


Failing Culture Conversion

Let's say your Program Level is being pressured to introduce cultural dysfunctions from the previously existing surrounding organization into the Agile Release Train, and they can't push back. In their function as a culture converter, they are now converting the new culture back into the old culture, and as such, working against the Agile Transformation. If you do not identify and deal with this issue swiftly and strongly, you're setting the fox to keep the geese: The fledgling new culture will be steamrolled by the existing culture in no time.




Summary

When you are using SAFe, ensure that the ART Roles are both willing and able to act as culture converters, and give them the support they need to function properly as such, mostly by relieving them of any and all responsibilities that relate to the "old" culture you want to abolish.

By overriding, short-ciruiting or ignoring the culture conversion function, you're dooming the culture transformation, and since the new ways of working rely on the new culture, you're going to train wreck. 

SAFe sucks when you mess up the culture conversion.



Thursday, December 10, 2020

Test Cocooning

 "How do you deal with Legacy code that lacks test coverage?" - even miniscule small changes are hazardous, and often, a necessary rewrite is postponed forever because it's such a nightmare to work with. Even if you have invested time into your test coverage after taking over the system, chances are there are still parts of the system you need to deal with that aren't covered at all. So this is what I propose in this situation:


Test Cocooning is a reversed TDD cycle, and it should be common sense.


The Cocooning process

Test Cocooning is a pretty straightforward exercise: 
  1. Based on what you think the code does, you create a cocooning test.
    1. If the test fails, you didn't understand the code correctly and you have to improve your test.
    2. If the test passes, you have covered a section of the code with a test that ensures you don't accidentally break the tested aspect of the code.
  2. Based on what you think the code does, you make a breaking change.
    1. If the test fails in the way you thought it would, you have a correct understanding of that piece of code.
    2. If the test passes, you didn't understand the code correctly and you have to improve your test (back to step 1)
    3. Intermediate activity: Of course, you revert the change to restore the behaviour that you have covered with test.
  3. Within the scope of your passing test, you begin to improve:
    1. Create lower-levelled tests that deal with more specifics of the tested code (e.g. unit tests.)
    2. Refactor based on the continuous and recurrent execution of all the relevant tests.
    3. Refactor your tests as well.
  4. Re-run the original cocooning test to ensure you didn't mess up anywhere!

Once a cocooning cycle is completed, you should have reworked a small section of your Legacy code to be Clean(er) Code that is more workable for change.


Iterating

You may need to complete multiple cocooning cycles until you have a sufficient amount of certainty that you have workable code.


Backtracking

The important secret of successful Test Cocooning is that you need to backtrack both on the code and your tests - after completing all relevant cocooning cycles, you'll need to re-run:

  • your cocooning tests against the original legacy code. 
  • your unrefactored original cocooning tests against the new code.
  • your unrefactored original cocooning tests against the original legacy code.
Yes, that's painful and a lot of overhead, but it's your best bet in the face of dangerous, unworkable code, and believe me - it's a lot less painful than what you'll experience when some nasty bugs slip through because you skipped any of these.


Working Cocooned code

Once you have your test cocoon, you can work the cocooned code - only within the scope of your cocoon - to fix bugs and to build new features.

Bugfixes

Fixing bugs relies on making a controlled breach to your cocoon.
Metaphorically speaking, you need to be like a spider that caught a bug and sucks it dry before discarding the woven husk.
  1. Create a test cocoon for the current behaviour which passes under the current faulty(!) conditions of the code segment that exactly reproduces the bug as though it were desired behaviour.
  2. Create a test which fails due to the bug, i.e. add a second test that exactly reverses the cocooned behaviour.
  3. Write the code that meets the requirement of the failing test.
    1. As a consequence, the cocooned passing test for the bug should now fail.
    2. Ensure that no other tests have failed.
    3. If another test has failed, ensure that this is intentional.
  4. Eliminate the broken cocoon test that reproduces the bug's behaviour.
    1. If there were other tests that failed, now is the time to modify these tests one by one.
  5. Backtrack like described above to ensure that nothing slipped. 

Modifying features

Modifying existing behaviour should be treated exactly like a bugfix.

New functionality

If you plan to add new functionality to Legacy Code, your best bet is to develop this code in isolation  from the cocooned legacy and only communicate via interfaces, ensuring that the cocoon doesn't break. 
When you really need to invoke new code from the Legacy, treat the modification like a bugfix.

Rewrite

A rewrite should keep the cocoon intact. Don't cling to any of the Legacy code and consider your cocooning efforts "sunken cost" - otherwise, you risk reproducing the same mess with new statements. 



Closing remarks

  1. I believe that test cocooning requires both strong test and development expertise, so if you have different specialists on your team, I would highly recommend to build the cocoon in pairing.
  2. Cocoon tests are often inefficient and have poor performance. You do not need to add these tests to your CI/CD pipeline. What you must add to your pipeline is the lower-level tests that replicate the unit behaviour of the cocoon. It's totally sufficient to rerun the cocoon tests when you work on the cocooned Legacy segment.
  3. Cocooning is a workaround for low-quality code. When time permits, rewrite it with Clean Code and you can discard the cocoon along with the deleted code.
  4. Do not work on Legacy Code without a solid Cocoon. The risk outweighs the effort.

Friday, December 4, 2020

Test Coverage Matrix

Whether you're transitioning towards agile ways of working on a Legacy platform or intend to step up your testing game for a larger system developed in an agile fashion, at some point, it pays to set up a Coverage Matrix to see where it pays to invest effort - and where it doesn't.



Before we start

First things first: the purpose of an agile Coverage Matrix isn't the same as a traditional project-style coverage matrix that's mostly concerned with getting the next release shipped. I don't intend to introduce a mechanism that adds significant overhead with little value, but to give you a means of starting the right discussions at the right time and to help you think in a specific direction. Caveat emptor: It's up to you to figure out how far you want to go down each of the rabbit holes. "Start really simple and incrementally improve" is good advice here!

What I'm proposing in this article will sound familiar to the apt Six Sigma practitioner as a simplified modification of the method, "Quality Function Deployment." And that's no coincidence.


Coverage Characteristics

Based on the ISO/IEC 9126, quality characteristics can be grouped into Functionality, Reliability, Usability, Efficiency, Maintainability and Portability. These are definitely good guidance. 

To simplify matters, I like to start the initial discussion by labelling the columns of the matrix:

  • Functionality ("Happy Cases")
  • Reliability ("Unhappy Cases")
  • Integration
  • Performance
  • Compliance
  • UX
Of course, we can clarify a lot more on what each of these areas means, but let's provide some leeway for the first round of discussion here. The most important thing is that everyone in the room has an aligned understanding on what these are supposed mean. 
If you are in the mood for some over-engineering, add subcategories for each coverage characteristic, such as splitting Performance into "efficiency", "speed", "scalability", "stress resilience" etc. That will bloat up the matrix and may make it more appropriate to flip rows and columns on the matrix.

Test Areas

Defining test areas falls into multiple categories, which correlate to the "Automation Test Pyramid". 

  • User journeys
  • Data flow
  • Architectural structure
  • Code behaviour
There are other kinds of test areas, such as validation of learning hypotheses around value and human behaviour, but let's ignore these here. Let's make a strong assumption that we know what "the right thing" is, and we just want to test that "we have things right." Otherwise, we'd open a can of worms here. You're free to also cover these, adding the respective complexity.


Functional areas

In each test area, you will find different functional areas, which strongly depend on what your product looks like.

User journeys

There are different user journeys with different touchpoints how your user interacts with your product. 

For example, a simple video player app might have one user flow for free-to-play, another for registration, another for premium top-up, and another for GDPR compliant deregistration as well as various flows such as "continue to watch my last video" or "download for offline viewing". These flows don't care what's happen technically.


Data flow

Take a look at how the data flows through your system as certain processes get executed. Every technical flow should be consistent end-to-end.

For example, when you buy a product online, the user just presses "Purchase", and a few milliseconds later, they get a message like "Thank you for your order." The magic that happens inbetween is make or break for your product, but entirely irrelevant for the user. In our example, that might mean that the system needs to make a purchase reservation, validate the user's identity and their payment information, must conduct a payment transaction, turn the reservation into an order, ensure the order gets fulfilled etc. If a single step in this flow breaks, the outcome could be an economic disaster. Such tests can become a nightmare in microservice environments where they were never mapped out.


Architectural structure

Similar to technical flow, there are multiple ways in which a transaction can occur: it can happen inside one component (e.g. frontend rendering), it can span a group of components (e.g. frontend / backend / database) or even a cluster (e.g. billing service, payment service, fulfilment service) and in the worst case, multiple ecosystems consisting of multiple services spanning multiple enterprises (e.g. Google Account, Amazon Fulfilment, Salesforce CRM, Tableau Analytics).

In architectural flow, you could list the components and their key partner interfaces. For example:

  • User Management
    • CRM API
    • DWH API
  • Payment 
    • Order API
    • Billing API

Architectural flow is important in the sense that you need to ensure that all relevant product components and their interactions are covered.

You can simplify this by first listing the relevant architectural components, and only drilling down further if you have identified a relevant hotspot.


Code behaviour

At the lowest level is always the unit test, and different components tend to have different levels of coverage - are you testing class coverage, line coverage, statement coverage, branch coverage - and what else? Clean Code? Suit yourself.

Since you can't list every single behaviour of the code that you'd want to test for without turning a Coverage Matrix into a copy of your source code, you'll want to focus on stuff that really matters: do we think there's a need to do something?


Bringing the areas together

There are dependencies between the areas - you can't have a user flow without technical flow, you won't have technical flow without architectural flow, and you won't have architectural flow without code behaviour. Preferably, you don't need to test for certain user flows at all, because the technical and architectural flows already cover everything. 

If you can relate the different areas with each other, you may learn that you're duplicating or missing on key factors.


Section Weight

For each row, for each column, assign a value on how important this topic is. 

For example, you have the user journey "Register new account." How important do you think it's to have the happy path automated? Do you think the negative case is also important? Does this have impact on other components, i.e. would the user get SSO capability across multiple products? Can you deal with 1000 simultaneous registrations? Is the process secure and GDPR compliant? Are users happy with their experience?

You will quickly discover that certain rows and columns are "mission critical", so mark them in red. Others will turn out to be "entirely out-of-scope", such as testing UX on a backend service, so mark them gray. Others will be "basically relevant" (green) or "Important" (yellow).

As a result, you end up with a color-coded matrix.

The key discussion that should happen here is whether the colors are appropriate. An entirely red matrix is as unfeasible as an entirely gray matrix.


A sample row: Mission critical, important, relevant and irrelevant



Reliability Level

As the fourth activity, focus on the red and yellow cells and take a look at a sliding scale on how well  you're doing in each area and assign a number from 0 to 10 with this rough guidance:

  • 0 - We're doing nothing, but know we must.
  • 3 - We know that we should to more here.
  • 5 - We've got this covered, but with gaps.
  • 7 - We're doing okay here.
  • 10 - We're using an optimized, aligned, standardized, sustainable approach here.

As a consequence, the red and yellow cells should look like this:

A sample matrix with four weighted fields.

As you would probably guess by now, the next step for discussion would be to look at the big picture and ask, "What do we do with that now?"


The Matrix

Row Aggregate

For each row, figure out what the majority of colors in that row is, and use that as the color of the row. Next, add up all the numbers. This will give you a total number for the row. 

This will give you an indicator which row is most important to address - the ones in red, with the lowest number.

The Row Aggregate


Column Aggregate

You can use the same approach for the columns, and you will discover which test type is covered best.  I would be amazed if Unhappy Path or Compliance turn out to have poor coverage when you first do this exercise, but the real question is again: Which of the red columns has the lowest number?


The Column aggregate



After conducting all the above activities, you should end up with a matrix that looks similar to this one:


A coverage matrix

Working with the Matrix

There is no "The right approach" to whether to work on improving coverage for test objects or test types - the intent is to start a discussion about "the next sensible thing to do," which totally depends on your specific context.  

As per our example, the question of "Should we discuss the badly covered topic Performance which isn't the most important thing, or should we cover the topic of architectural flow?" has no correct answer - you could end up with different groups of people working hand in hand to improve both of these, or you could focus on either one.



How-To Use

You can facilitate discussions with this matrix by inviting different groups of interest - business people, product people, architects, developers, testers - and start a discussion on "Are we testing the right things right, and where or how could we improve most effectively?"

Modifications 

You can modify this matrix in whatever way you think: Different categories for rows or columns, drill-in, drill-across - all are valid.

For example, you could have a look at only functional tests on user journeys and start listing the different journeys, or you could explicitly look at different types of approaching happy path tests (e.g., focusing on covering various suppliers, inputs, processing, outputs or consumers)

KISS 

This method looks super complicated if you list out all potential scenarios and all potential test types - you'd take months to set up the board, without even having a coversation. Don't. First, identify the 3-4 most critical rows and columns, and take the conversation from there. Drill in only when necessary and only where it makes sense.




Tuesday, December 1, 2020

Refactor, Rewrite or Redesign?

Have you ever heard the term "Refactoring Sprint?" There is a lot of confusion around what Refactoring, Rewrite und Redesign are, what they entail, as well as how and when to use them - so here's a small infographic:



Refactoring

Refactoring is all about small changes to the way code has been written to reduce technical debt as the code and our understanding thereof grows. It's a continuous exercise and should always be an integral part of the work. Safe refactoring is a low-risk exercise that happens within the scope of existing unit tests, which should verify that indeed we haven't made any unwanted changes.

Examples of Refactoring are: Extracting a variable, renaming a function, consolidating statements or moving code from one place to another.

It should go without saying that in professional development, Refactoring should never be a separate task and should never be done in big batches. Your Definition of Done should include that no essential refactoring is left to do.


Rewrite

We rewrite code when we discover that a current code segment is no longer the best way to achieve a specific purpose. Depending on your test coverage, rewrites are a high-risk exercise that are the basis for future risk reduction.

Examples of Rewrites are new and better means (e.g. a new technology) to do the same thing, or stumbling upon a legacy that is difficult to work with (e.g. complex methods without tests).

Smaller rewrites (e.g. method rewrites) should be done autonomously as regular part of the work when the need is discovered and when developers have sufficient confidence that they can do this safely.
Larger rewrites should be planned with the team, as this exercise could consume significant portions of time and may need additional attention to failure potential.


Redesign

A redesign must happen when the purpose of an existing component has changed in ways that the current solution is no longer the best possible thing. Redesign is a high-risk and highly time consuming exercise that should be done to get out of a corner. If you have highly malleable code that was well refactored on a continuous basis, redesign should hardly ever be a case unless you have major changes to the business context in which a system is operated.

Examples of Redesign might include moving from batch processing to data streams, changing the database paradigm, or acquiring a new line of business that requires processing different data.

Redesign should always be a planned whole-team exercise that might even break down into a hierarchy of mid- and short term goals. Redesign is a mix of technical and business decision, so it should be aligned with Product.




Wednesday, November 18, 2020

16 misconceptions about Waterfall

Ok, Agilists. It's 2021, and people are still using Waterfall in corporate environments. With this article, I would like to dismantle the baloney strawman "Waterfall" that's always proclaimed as the archenemy of all that is good and would encourage you to think about how exactly your suggested "Agile" is going to do better than the examples I have taken from real-world, professional Waterfall projects.

Here are some things that many agilists may have never experienced in about Waterfall projects. I did.


What you think Waterfall is, but isn't

There are numerous standard claims about what's wrong with Waterfall, which I would generously call "statement made from ignorance," although there could be more nefarious reasons why people make these claims. Point is: many of the common claims are not generally true.


Big Bang vs. Incremental

Waterfall doesn't mean that until the determined end date of the project, there will be nothing to show. I remember when I stated that I worked in a 5-year Waterfall project, people from the Agile community called that insane. It's not. We had a release every 3 months. That means that the project had a total of 20(!) Increments, each with its own scope and objectives: Yes - Waterfall can be used to build products incrementally! In corporations, that's actually normal.


Upfront Design vs. Iterative Design

With each delivery, project managers, analysts and business people sit together and discuss the roadmap: which requirements to add or remove, and which priorities to shift. I have once worked in a product that was created in pure Waterfall for almost 20 years, and nobody could have anticipated the use cases delivered in 2010 when the product's first version hit the market back in 1992. Even Waterfall projects can iterate. Especially for enterprise systems.


Death March vs. Adaptivity

When you think that someone sits in a closet and produces the Master Plan, which must be slavishly adhered to by the delivery teams, you're not thinking of a properly managed Waterfall project. While yes, of course, there is a general plan, but a Waterfall plan gets adapted on the fly as new information arises. Timelines, staffing, scope, requirements, objectives - are all subject to change, potentially even on a weekly basis if your project manager is worth their salt.


Fixed Scope vs. Backlog

If you've ever done Project Management, you know pretty well that scope is very malleable in a project. When an organization determines that meeting a fixed timeline is paramount, Waterfall fixed time projects can be pretty similar to Sprints in managing scope. While of course, you get problems if you don't manage the Critical Path properly, that's not a Waterfall problem - it's carelessness. 


Fixed Time vs. Quality

Probably one of the main complaints about Waterfall is that a team delivering on a fixed schedule will push garbage downstream to meet the timeline. Again, that's not a Waterfall issue - it's a "fixed time" issue. If you flex the time, and fix the work package, there's nothing inherent to Waterfall that implies a willful sacrifice of quality.

(And, as a witty side note - if you believe that fixed time is the root cause for low quality: how exactly would Scrum's Sprint timebox solve that problem?)


Assumptions vs. Feedback Learning

Complex systems serving a multitude of stakeholders are incredibly hard to optimize, especially when these stakeholders have conflicting interests. The complexity in Waterfall requirement analysis is usually less in trying to get a requirement right, as it is in identifying and resolving conflicting or wrong demands. The time spent upfront to clarify the non-developmental interferences pays off in "doing the right thing." Good analysts won't be making wild assumptions about things that could potentially happen years down the line. When a release is launched, good Waterfall projects use real user feedback to validate and update the current assumptions


Handovers vs. Collaboration

Yes. There are stage-gates in a Waterfall. I myself have helped organizations implement Quality Gates long before Scrum was a thing. Don't misunderstand Q-Gates, though. They do not mean that an Unknown Stranger hands you a Work Package which you will hand over to another Unknown Stranger at the next Gate. What typically happens: As soon as analysts have a workable design document, they'll share it with developers and testers, who take a look, make comments and then meet together to discuss intent and changes. Good Waterfall organizations have collaboration between the different specialists whenever they need to.


Documentation vs. Value Creation

A huge misconception is that "Waterfall relies on heavy documentation" - it doesn't, depending on how you operate. Heavy documents are oftentimes the result of misfired governance rather than caused by the Waterfall approach itself. It's entirely feasible to operate Waterfall with lightweight documentation that clarifies purpose and intent rather than implementation details, if that's what your organization is comfortable with. Problems start when development is done by people who are separated from those who use, need, specify or test the product - especially when there's money and reputation at stake. 


Process vs. Relationships

As organizations grow large, you may no longer have the right people to talk with, so you rely on proxies who do a kind of Telephone Game. This has nothing to do with Waterfall. A good Waterfall Business Analyst would always try to reach out to actual users, preferably power users, who really know what's going on and build personal relationships. As mutual understanding grows, process and formality becomes less and less important, both towards requesters and within the development organization - even in a Waterfall environment.


Resource Efficiency vs. Stable Teams

There's a wild claim that allegedly, Waterfall doesn't operate with stable teams. Many Waterfall organizations have teams that are stable for many years, in some cases, even decades. Some of the better ones will even "bring work to the team" rather than assigning work to individuals or re-allocating people when something else is urgent. The "Resource efficiency mindset" is a separate issue, unrelated to Waterfall.


Big Batch vs. Flow

Kanban and Waterfall can quite well coexist. Indeed, I have used Kanban in a Waterfall setting long before I first heard of Scrum where requirements flowed through three specialist functions, and we had an average cycle time of less than one week from demand intake to delivery. Waterfall with Small Batches is possible, and can perform exceptionally well.


Top-Down vs. Self-Organized

I've worked with corporations and medium-sized companies using Waterfall, and have met a lot of Project Managers and Team Leads who have worked in a fashion very similar to a Product Owner: taking a request, discussing it with the team, letting the team figure out what to do how and when, only then feeding back the outcome of this discussion into the Project Plan. Waterfall can have properly self-organized teams.


Push vs. Pull

Whereas in theory, Waterfall is a pure "Push"-based process, the field reality is different. If you have a decent Waterfall team lead, it will basically go like this: We see what work is coming in, we take what we can, and we escalate the rest as "not realistic in time", and get it (de-)prioritized or the timeline adjusted. De facto, many Waterfalls teams are working pull-based.


Overburden vs. Sustainable Pace

Yes, we've had busy weekends and All-Nighters in Waterfall, but they were never a surprise. We could anticipate them weeks in advance. And after these always came a relaxation phase. Many people working in a well built, long-term Waterfall project call the approach quite sustainable. They feel significantly more comfortable than they would be under the pressure to produce measurable outcomes on a fortnightly basis! Well-managed Waterfall is significantly more sustainable for a developer than ill-managed Scrum, so: Caveat emptor!


Resources vs. Respect

Treating developers as interchangeable and disposable "resources" is an endemic disease in many large organisations, but it has nothing to do with Waterfall. It's a management mindset, very often combined with the cost accounting paradigm. The "human workplace" doesn't coincide well with such a mindset. And still, the more human Waterfall organizations treat people as people. It entirely depends on leadership.


Last Minute Boom vs. Transparency

Imagine, for a second, that you would do proper Behaviour Driven Development and Test Driven Development in a Waterfall setting. If you do this and deliver an update like twice a month and properly respond to feedback, Waterfall doesn't need to produce any nasty surprise effects. The Last Minute Boom happens when your development methodology is inapproprate and your work packages are too big, not because of Waterfall.


All said - what then is, "Waterfall?"

"Waterfall" is nothing more and nothing less than an organized, sequential product development workflow where each activity depends on the output of the previous activity.

There are really good uses for Waterfall development, and cases where it brilliantly succeeds. It's incorrect to paint a black-white image where "Waterfall is bad and Agile is good", especially not when equivocating "Agile" to a certain framework.

Proper Waterfall

A proper Waterfall would operate under the following conditions:
  1. A clear, compelling and relateable purpose.
  2. A human workplace.
  3. A united team of teams.
  4. People who know their ropes.
  5. A "facts are friendly" attitude.
  6. Focus on Outcomes.
  7. Continuous learning and adaptation.
  8. Reasonable boundaries for work packages.
  9. Managing the system instead of the people.

All these given, a Waterfall project can have a pretty decent chance to generate useful, valuable results.

And when all the above points are given, I would like to see how or why your certain flavor of "Agile" is doing better.


My claim


I challenge you to disprove my claim: "Fixing the deeper mindset and organizational issues while keeping the Waterfall is significantly more likely to yield a positive outcome than adopting an Agile Framework which inherits the underlying issues."





Tuesday, November 17, 2020

Is all development work innovation? No.

In the Enteprise world, a huge portion of development work isn't all that innovative. A lot of it is merely putting existing knowledge into code. So what does that mean for our approach?

In my Six Sigma days, we used a method called "ICRA" to design high quality solutions.


Technically, this process was a funnel, reducing degrees of freedom as time progressed. While we can formidably argue about whether such a funnel is (always) appropriate in software development, I would like to salvage the acronym to discriminate between four different types of development activity:

Activity Content Example
Innovate Fundamental changes or the creation of new knowledge that allows us to determine which problem to solve in what way, potentially generating a range of feasible possibilities. Creating a new capability, such as "automated user profiling" to learn about target audiences.
Configure Choosing solutions to a well-defined problems from a range of known options.
Could include cherry-picking and combining known solutions.
Choosing one of many cookie cutter templates for the new company website.
Realize Both problem and solution are known, the rest is "just work", potentially lots of it. Including a 3rd party payment API into an online shop.
Attenuate Minor tweaks and adjustments to optimize a known solution or process. Adding a highlight, validation rule or event trigger to a form field.

Why this is important

Think about how you're developing: depending on each of the four activities, the probability of failure, hence, the predictable amount of scrap and rework, decreases. And as such, the way that you manage the activity is different. A predictable, strict, regulated, failsafe procedure would be problematic during innovation, and highly useful on attenuation - you don't want everything to explode when you add a single line of code into an otherwise stable system, which might actually be a desirable outcome of innovation: destabilizing status quo to create a new, better future.

I am not writing this to tell you "This is how you must work in this or that activity." Instead, I would invite you to ponder which patterns are helpful and efficient - and which are misleading or wasteful in these activities. 

By reflecting on which of the four activities and the most appropriate patterns for each of them, you may find significant change potential both for your team and for your organization, to "discover better ways of working by doing it and helping others do it."


Thursday, November 12, 2020

PI-Planning: Factors of the Confidence Vote

 The "Confidence Vote" is a SAFe mechanism that is intended to ensure both that the created PI Plan is feasible, and also to ensure that people understand the intent behind creating the common plan - what it means, and what it doesn't. Implied in SAFe are two different kinds of confidence vote with slightly different focus.







Train Confidence Vote

The "Train Confidence Vote" is taken on the Program Board - i.e. the aligned, integrated PI plan across all teams. All participants of the PI-Planning are asked to simultaneously vote on the entire plan. Here are the key considerations, all of which should be taken into account:

Objectives: Feasibility and Viability

First, we should look at the ART's PI objectives realistic, and does it make sense to pursue them? Do we have our priorities straight, and are we focused on delivering maximum value to our customer?

High Confidence on PI objectives would imply that these objectives are SMART (Specific, Measurable, Ambitious, Realistic, Timebound) within the duration of the PI.

Features: Content and Scope

Do we have the right features, do all of them provide significant progress towards our objectives, did we pick a feasible amount, and did we arrange them in a plausible order and are the right people working on them? Is the critical path clearly laid out, and is the load on the bottleneck manageable?

High Confidence on Features would imply that everyone is behind the planned feature arrangement.

Dependencies: Amount and Complexity

If we have too many dependencies, the amount of alignment effort throughout the PI will be staggering, and productivity is going to be abysmal. You also need to manage external dependencies, where the Train needs something from people who aren't part of the Train, and you need to pay extra attention when these people didn't even attend the PI-Planning.

High Confidence of Dependencies would imply that active efforts were made to eliminate as many dependencies as possible, and teams have aligned already how they deal with the inevitable ones. When people either mark a high amount of dependencies without talking about them, or you feel that some weren't mentioned, that should reduce your confidence drastically.


Risks: Quantity, Probability and Impact

Risks are normal part of life, but knowingly running into disaster isn't smart. Were all the relevant risks brought up? Have they been ROAM'ed properly? How likely will you be thrown off-track, and how far?

When you consider risks well under control, that can give you high confidence in this area - when you feel like you're facing an army of gremlins, vote low.


Big Picture: Outcomes and approach

After looking at all the detailed aspects, take one step back: Are we doing lots of piecemeal work, or do we produce an integrated, valuable product increment? Do we have many solitary teams running in individual directions, or do we move in the same direction? Do you have the impression that others know what they're doing?

When you see everyone pulling on the same string and in the same direction, in a feasible way, that could give you high confidence. When you see even one team moving in a different direction, that should raise concerns.


Team Confidence Vote



During your breakout sessions, the Scrum Master should frequently check pulse on team confidence. The key guiding question should be: "What do you need so that you can vote at least a 4, preferrably a 5, on our team's plan?"

Your team plan is only successful when every single member of the team votes at least a 3 on it, so do what it takes to get there. It's entirely inacceptable for a team member to lean back comfortably and wait for the team confidence vote and then vote 2, they should speak up immediately when they have concerns. Likewise, it's essential that teams have clarified all the issues that would lead them to vote low on their team's plan before going into the PI confidence vote.

When your team can not reach confidence, do not hesitate - involve Product Management and the RTE immediately to find a solution!

Here are the factors you should consider in your team confidence vote:

Objectives

Does your team have meaningful objectives, are you generating significant value?

Understanding

Do you really understand what's expected from you, how you're contributing to the whole, what makes or breaks success for you - what the features mean, what your stories are, what they mean, and what's required to achieve them?

Capacity and Load

Do you understand, including predictable and probable absences, how much capacity your team has?  How likely can you manage the workload? Have you accommodated for Scrum and SAFe events? Would unplanned work break your plan?

Dependency Schedule

Can you manage all inbound dependencies appropriately, do you trust the outbound dependencies to be managed in a robust way? What's your contingency plan on fragile dependencies?

Risks

Are you comfortable with the known risks? Do you know your Bus Count, and have you planned accordingly? Do you trust that larger-scaled risks will be resolved or mitigated in time?

Readiness

Right after the PI-Planning, you will jump into execution. Do you have everything to get on the road?



Closing remarks

This list isn't intended to check each factor individually, and it isn't intended to be comprehensive, either. It is merely intended to give you some guidance on what to ponder. If you have considered all these, you probably haven't overlooked anything significant. If you still feel, for any reason, that you can't be confident in your plan, by all means, cast the vote you feel appropriate, and start the conversation that you feel is required.
It's better to spend a few minutes extra and clarify the concerns than to find out too late that the entire PI plan is garbage.

Monday, November 2, 2020

Delivered, Deployed, Done?

While an agile organization should avoid over-engineering a formal status model, it's necessary to provide standards of what "Done" means so that people communicate at an even level. The highest level of confusion arises in large organizations where teams provide piecemeal components into a larger architecture, because teams might define a "Done" that implies both future work and business risk until the delivery is actually in use.

In such a case, discrimating between "Deployed" and "Done" may be useful.


What's Done?

At the risk of sounding like a broken record, "Done means Done," "It's not Done when it's not done" and "You're not Done when you're not done."

That is, when you're sitting on a pile of future work, regardless of whether that pile is big or small, you're not done. This is actually quite important: While softening your DoD gives you a cozy feeling of accomplishment, it reduces your transparency and will eventually result in negative feelings when the undone work comes back to bite you.

As such, your enterprise DoD should encompass all the work that's waiting. Unfortunately, in an Enterprise setting, especially when integrating with Waterfall projects or external Vendors, you may have work waiting for you a year or more down the line. The compromise is that teams put work items into an intermediate, "Deployed" status when the feature is live in Production, and set it to "Done" at the appropriate time in the future.

What's Deployed?

In situations where a team has to move on before taking care of all the necessary work to move to "Done", because there are factors outside their own control, it may be appropriate to introduce an intermediate status, "Deployed." This allows teams to move on rather than idly waiting to do nothing or wasting their energy getting nowhere.

In large enterprise situations, teams often deliver their increments and the following haven't been taken care of yet:
  • Some related open tickets
  • User training
  • Incidents
  • Feature Toggles
  • Business Configuration
  • E2E Integration
  • Tracking of Business Metrics
  • Evidence of Business Value
This status is not an excuse - it creates transparency on where the throughput in the value stream actually is blocked, so that appropriate management action can be taken.


Interpreting "Done" vs. "Deployed."

Let's take a look at this simple illustration:


Team 1

If they would soften up their DoD to equate "Deployed" and "Done", then the business risk is hidden, and it becomes impossible to identify why the team isn't generating value, even though they're delivering. They lose transparency with such an equivocation.
A strict discrimination between "Deployed" and "Done" surfaces the organizational impediment in this team and makes the problem easy to pinpoint.

Team 2

It wouldn't make sense to discriminate between "Done" and "Deployed", because the process is under control and there is no growing business risk. This team could just leave items "in Progress" until "Done" is reached and doesn't benefit from "micromanaging" the status.

Wednesday, October 21, 2020

Where to put the Business Analysts?

A common question that needs to be answered during agile transitions is, "Where do we put the Business Analysts?"

In a traditional project organization, it's quite common that they receive orders from Product / Project Management, create solution designs and hand these over to development teams, this is a poor approach for agile organizations.



Avoid: BA working for PM

We often see that the BA is a go-between for users and Agile Teams, or even for Product Management and Agile Team, both of which are done in the name of efficiency at the expense of quality.

There are numerous highly dysfunctional antipatterns associated with this approach, i.e. things that cause more problems than they solve, including, without limitation:

Antipattern Problem
Works as Requested

When users ask for something suboptimal, that's what they'll get, because developers are unaware of the user's real need, and the Product Owner also lacks the necessary information to acknowledge alternate solution proposals.

Works as DesignedWhen Business Analysts make invalid assumptions about the technical solution, developers will strugge to implement based on their design, since developers are not in a position to challenge the underlying assumptions.
Dysfunctional PO

When a PO gets prioritized, "must-do", fully analyzed designs that need to be implemented, their role becomes dysfunctional. All they can do is "push tickets" and fill in templates of work. The PO's main function is invalidated. 
Product Owners struggle to find purpose and meaning in their work, and in such a setup, it's no loss to eliminate them entirely.

Telephone GameThe amount of information lost when users talk to analysts who talk to product owners who talk to developers is staggering. The amount of communication overhead and productivity loss caused by this setup potentially outweighs the benefits of doing business analysis outside the team.
BottleneckSeparating the BA out as a special function typically makes them a bottleneck. When push comes to shove, incomplete designs are handed to development in a hurry, which often causes more trouble later than the amount of work the BA wasn't able to complete.

Try: BA is part of the Agile Team

An alternative, significantly more agile, approach is to make the BA part of the agile team they're doing analysis for. 
In this setup, the BA is a dedicated member of the Agile Team they're working with - figuring out both the customer needs in the solution, and the developer needs in the design. Their accountability is being a member of the Development Team, contributing towards the Team Goals

In this setup, their job isn't "Done" when a design document is written, but when the user's need is successfully met.

From this position, the Business Analyst supports the refinement and elaboration of business value, interacting with users, not as a go-between, but as a facilitator for developers and users.

Business Analysts also support the decisions of the Product Owner, ensuring that backlog items are "Ready" both quantitatively and qualitatively when there is development capacity to pull these items.

This approach to Business Analysis in an agile setup makes BA expertise equally, and potentially even more, important to the success of development as in a traditional setup, without creating any of the above-mentioned antipatterns.


The HR issue

The main challenge that has to be addressed when talking about the new role of the BA is an HR issue:
From a practical perspective, the BA gets "degraded" from being pretty high in the hierarchy of the organization all the way "down to" team member. This often causes resistance, making it look like the way of least resistance is to opt for the prior choice, which creates irreconcilable conflict within the development organization.

As such, there are multiple items to clarify before we can make the BA as valuable as possible in an agile setting:

Focus area Problem
HR

Address potential HR impediments that make it within a BA's own best interests to not be part of an agile team, but rather outside. Such impediments include salary, career progession and other incentives set by the organization.

Line Organization

In organizations where BA itself is a separate silo, work with the BA's manager to ascertain them that making the BA's part of the Agile Team does not diminish their importance or influence. The main thing that needs to change is that BA's now receive their work from the team.

BA Individuals

Work with the BA's themselves to ascertain them that being part of an Agile Team is, in fact, not a degradation and to discover and resolve the personal issues they have with the new, different ways of working.


Wednesday, October 14, 2020

How to resolve the Planning Conflict

There's a seeming conflict that might become apparent: On the one hand, "delivering early and often" is an Agile principle - and on the other hand, "deferred commitment"  is a Lean principle. This might create a planning conflict. How do you resolve it?



Planning purpose

First, we must realize that there are different reasons for planning. 

Within the team / development organization, the purpose of planning is to make sure that we have a realistic goal and we all understand what we need to do.

Towards our customers, the purpose of planning is different. They don't care who does what, and when. They care when they'll get what.

Towards other stakeholders in our organization, the purpose of planning is again different. They need to know when they're expected to contribute, and when they can get a contribution from us.


Defer commitment?

First thing to realize here is: "Who are we committing towards?" Are we committing inside the teams to maximize value - or are we committing a certain date or scope to our customers or stakeholders?

Customers and stakeholders plan their actions based on our commitment, so in this regard, we shouldn't commit anything that we can't keep, because otherwise, we may be creating significant, non-value-adding re-planning and organizational overhead. Broken customer commitments will damage our trust, If you can deliver without having to give a commitment, that's better, and even when you need to commit so that others can plan, try to commit as late as possible.

The trust issue

"Deferred commitment" requires trust. 
  • Trust in the team, that they do the best they possibly can. 
  • Trust in the organization, that they enable the team to succeed.
  • Trust in the customers and stakeholders, that they want the team to succeed.
Asking for early commitment hints at a lack of trust. The solution is not to enforce strict commitment, but to build trust. In a trustful relationship, deferred commitment shouldn't be an issue for anyone.


Deliver early?

Inside our team, we plan to deliver as much value as early as possible, because "you got what you got". To minimize risk and to avoid falling for Parkinson's Law, we should avoid keeping activity buffers that allow us to "do extra work", and we should remember that early delivery is our feedback and learning trigger.


Resolving the conflict

There is no conflict.
We work towards two separate events: 
The team's first point of feedback, and the point of business completion.
  • The first date is the earliest point in time when we can get feedback. It allows us to validate our assumptions and to verify our product. There is no guarantee of completion or finality. For internal planning, we look for earliest possible dates, so that we can reduce risk and deliver value quickly.
  • The second date is the latest point in time when we can complete a topic. We communicate this date as late as possible and try to avoid having to lock it in if we can. This minimizes the danger of expectation mismatch. For external communication, we look for latest feasible dates, so that other people's decisions don't rely on our unvalidated assumptions.

Addendum

Based on the feedback that "deferred commitment" in a Lean context is referring to decisions:
The statement "Scope X will be completed at date Y" consists of two decisions made today: a decision about what, as well as a decision about when. If there is no need to decide this today, we should not.
We try to avoid locking in a decision that has a significant risk of being wrong.
That is not the same as "we won't deliver any value until some undefined date in the future." It means, "we can't guarantee you the value until we know more."

Thursday, October 8, 2020

Why you shouldn't set Predictability targets

While predictability is pretty important for management, and having an agile team/organization deliver in a predictable manner is certainly aspirable, setting targets for predictability is a terrible idea. And here's why:


Blue lines are reinforcing, red lines negative reinforcement, orange items are under the teams' control.


As soon as we set Predictability as a target, we create a reinforcement loop that rewards teams for spending more time planning and less time actually developing. The same reinforcement loop also destroys the very thing called "agility", i.e. the flexibility of "responding to change over following a plan."

As a consequence of both reinforcement loops initiated by setting a predictability target, we reduce the ability to actually deliver business value. As such:

Developers who work towards a Predictability objective do so at the expense of Business Objectives.

If that's not yet clear enough, let me put it bluntly:

Predictability targets hurt your company's bottom line.

 

Hence, I will strongly advise to resist the urge of using predictability as a KPI and setting predictability targets on software development.