The Discovery Questions in agile development

For teams just transitioning to agility, the question arises: "How do we test in an agile team? Which test types do we need?"
There is stuff like the Agile Test Quadrants which are often suggested as guidance. On a different level of abstraction, we have the Test Pyramid. You might look for gaps in your existing test approach and try to cover them as well as possible. You will soon discover that this will not really help, as you merely introduce a structure, but don't solve the underlying "Why" questions.
In another article, we discussed what agile testing is all about.

Let us look at a small visualization of what we are actually looking for when we are discussing quality:

The quality discovery questions.
The matrix can be used to guide a Retrospective on quality.

When we have a new idea (or: Product Backlog item), we first need to discover "What do we want to build in order to make this happen?". This is the specification. The quality of the specification determines the quality of the solution. A basic idea of (A)TDD is to consider functional tests and specification to be the same thing.

In the next step, we need to discover "How do we want to build it, so that it works properly?" This is the source code. Code quality affects both the solution quality and the way how to control quality in the process.

After the code has been built and deployed (into a staging environment) there may still be things going on that we never expected. These are side effects. We need to answer the question: "What happens that we didn't think of?" We don't want that to be many things, but it can always happen.

When we deliver Working Software to the customer, we learn: "What do real users do with the product and how do they use it?" This is user feedback. We use this feedback toadapt our current product in order to maximize value, As we learn more about our users and markets, we will need to adjust our priorities, both strategically for the product and in testing.

Uncertainty

During planning, we try to remove as much uncertainty from implementation as possible. The better our tests are in this stage, the lower our risk of building the product wrong. We do not want to guess whether we built it right based on our idea. We want to be certain. Until we can do this, our first priority is to eliminate the risk of building the product wrong.

After building, we need to validate whether our idea made sense. The more accurate our tests are in this stage, the less likely we will deliver an unsatisfactory product. With the final product in hand, there should be no need to check whether we did do things right based on our idea - only whether our assumptions seem solid.

Early feedback

Each of the four quadrants correlates to a phase in development: Planning, building, deployment and operations. A good way to check whether your current testing approach is effective is by asking the question: "Which activities we do are we doing in which phase?"

If, for example, you realize that you are discovering or verifying acceptance criteria after deployment, you have potential for improvement. A good way to approach this improvement is by asking the question: "Which activities can we do earlier in order to remove the need to do them (so) late?"

Conclusion

A solid agile test strategy removes uncertainty as early as possible. 

The best way to start implementing agile testing is not by focusing on a prescribed set, but by asking: "How can we remove as much uncertainty as early as possible?"