TWEAK: Knowledge

Creating great products requires the right environment, the right people - but also the right knowledge. In an ever evolving world, knowledge is not something you can "have", but something to continually strive for. On the quest for mastery, knowledge needs to be taken with special consideration. For each domain, you should see if your team is "struggling to keep up", "well in the loop" or "forerunner". While not everyone can win the race, it is a good idea to invest effort into getting ahead of the game.

As a Scrum Master, you should encourage everyone to broaden their knowledge regarding:

• Technical ExcellenceRegardless of whether it is Java, CSS, JavaScript, Databases or virtualization technologies - the team must be keen on their technical skills.
• Engineering Practices
  Testing, Refactoring, Emergent Design, Automation and many others are all essential to build the product right - all of these are easy to learn, but extremely difficult to master. 
• Product Domain
  Developers who are only technical experts will build the wrong product, unless they actively strive to understand the domain of the product they develop. Low hanging fruit may be out of your reach when you can't make the right connections.
• Agility
  The ideal of infinite Return on Investment requires responding to demand without any delay. Every step you take to be more agile will pay off - but what steps are available to you?
• Organizations
  The team must take steps to actively shape the organization in a positive fashion. To do this, they must understand well how their organization and the world around them "ticks".

Since "knowledge" itself is so broad, there is no definite way to approach broadening knowledge. You may read books, share experiences with your team or others in the organization - build up Communities of Practice, join Special Interest Groups or whatever you see fit. You need to make sure is that people are thirsty for more knowledge. You can't "push" knowledge. People must want it.

TWEAK: Ambition

Ambition is the desire not to just fulfil a duty, but to excel - do something better, achieve something better, be something better. There are many ways to increase ambition in a team, but Trust, Willingness and Energy are preconditions. Ambition is usually present in every person, but unfortunately dormant in poorly managed organizations.
As a Scrum Master, you should look out for these factors to raise ambition:

• Feedback Loops: Acknowledge and credit:
• Positive attitude
• Good ideas
• Good results
• Opportunity: Provide the means for developers to:
• try out new things
• be themselves
• Accomplish something
• Safety: Give people an environment where they can safely:
• Try things that might fail
• Say the wrong thing
• Be silly when they feel the need to

There are pretty much two styles of ambition. Negative ambition cares only for personal advantage and is destructive towards the organization. Positive ambition cares for mutually building things up and usually benefits the individual as much as the organization. Raising up ambition is as important as turning negative ambition into positivity.

TWEAK: Energy

Energy is how things are in motion. Low Energy usually implies that something, somewhere is impedited. Leadership activates energy, on which others either thrive. Similar to physics, in an agile environment, there is the law of "conservation of energy" , i.e. energy exists - the question is: In which state?
As a Scrum Master, here are some forms of energy to look for:

• Motion: Do you see things happening regarding:
• Improving the product?
• Improving the process?
• Teamworking?
• Technical excellence?
• Becoming a better organization?
• Heat: Do you see people caring about these?
• Light: Do people communicate about these, make them transparent?
• Magnetic: Where is conflict or tension regarding these?
• Nuclear: Where is unharnessed, raw potential to create motion, heat or light?

You must understand where the energy is - then, how it can be harnessed - and with whom to work. Where is the heat, where is magnetism - Do you want to shed some light, or set things into motion? There are many ways and there is no "final state". Harnessing energy is a continuous process, and the less directed it is, the more agile the organization becomes. 

The only important aspect of working with energy is: Always make sure the energy is positive. 

TWEAK: Trust!

Your team requires a high level of trust within, from the outside in, and towards the outside. Otherwise it will be difficult to perform properly: Performance will be wasted on "looking good" rather than "being good". Schemes and politics will reduce results accordingly, so trust is your primary factor to work with.

As a Scrum Master, here are the dimensions you should consider:

• Does the team trust 
• each other?
• the Scrum Master?
• their management?
• Product Owner?
• Customers?
• Is this trust mutual?
• What is the basis of the trust relationship?
• Is the trust relationship also transitive, for example: Does the Product Owner also trust the Management dealing properly with the team?
• Is the trust relationship also indirect, for example: Do Customers and Management trust each other?

Building trust is the first and most important pillar for team success. Only when trust is present, technical improvement activities will be effective.

Planning Poker: Why Points, not hours

Many Scrum teams struggle with Planning Poker and it's proper use. A recurring question is: "Can't we just use hours instead?". A mistake some teams make is converting the Points back into Man-Days.

The purpose of Planning Poker

Sprint Planning should contain a C-C-C process (Card, Conversation, Confirmation). In brief: 

1 - the PO displays what they want

2 - the team can probe whether they understand the idea in the same way

3 - the team discusses how they want to approach the topic,

4 - Everyone agrees that this is what will be done.

5 - Rinse, repeat until the team decides their capacity is reached.

Estimation in Story Points is used to quantify complexity of the task. It is actually expected that estimates vary until the team has reached a certain agreement on the complexity. 

Disregarding the accuracy of an estimate, a precise discussion of the factors contributing to complexity will bring the team's estimates closer to each other.

An example dialogue during Planning Poker

The Product Owner introduces the new Widget that will shoot our sales through the roof. Tim estimated a 20, Jane estimated a 3. Jane reasons: "All we need is to build some plastic device, put a chip on it and contact our service URL."

Tim then turns to the PO, "So, you expect the Widget to be usable from anywhere?" - "Yup" - "Even when no Wi-Fi is available?" - "Yup." - "Can we build a first version assuming Wi-Fi is present?" - "Yup." - the precision increased, but the Story has already been split into two pieces. Also, everyone on the team now understands why Tim had serious reservations about the complexity of the issue.

While Tim may now agree with Jane regarding the estimate for the first version, Jane may now also have serious reservations about the second stage, because nobody knows how complex this issue gets if "everywhere" also includes underground, in the absence of cellular and satellite signals.

Estimation is for Innovation

One of the first things which classic project managers must get used to is the idea that software development, as the name indicates, is development. Development is not just working according to plan, but the creation of something which was not in existence before. 

Software is always new: If we use something existing, we simply buy and/or include it. But we don't build it. If we build it, it's new. This means, nobody has done it before - or, at least, not exactly in the same way that we need it. 

How can you know how long something takes that nobody has done before?

You can't - and this is why at best, we estimate, not a detailed forecast.

Let's estimate

To give you some understanding of the difference between an estimate and the quantification of an effort forecast, let's play a game.

As a Product Owner, I have great visions and would ask you to do some estimation for me.

Here are 5 items which will bring great value to mankind:

• Cure for Cancer
• Immortalism
• Cold Fusion
• Warp Engine
• Terraforming

Now, I would like you to estimate these items for me. You can probably come up with a numbering between 1 and 20 for these items, where "1" is the least complex and "20" the most complex issue.

Almost intuitively, your numbering approach will probably revolve mostly about your understanding of the domain as well as the uncertainty you have. The actual "work to do" will probably play a miniscule role.

Estimates are not Man-Days

Can you now assign hours/months/years to each of these items?

In principle, the problem is identical to Software Development: We may have a certain (hopefully, more intricate) understanding of the domain, but we have never done that exactly same thing before. 

We will not know which solution is correct until the solution is correct and in place.  

Since the course of action is prone to change due to any new information we gain while we are working, it makes little sense to plan a specific course of action. Every step depends on the last step taken, and every result either encourages or discourages the next step we had in mind. Sometimes, next steps only become obvious once a specific result has been obtained.

Summary

The purpose of Planning Poker is the facilitation of discussion about complexity and uncertainty. We care for having a common understanding and dealing with the "known unknown". Removing any last bit of uncertainty about the issue might require more effort than actually coming up with a working solution, so we simply accept this uncertainty.

If the work is totally clear, the item is most likely quite small. When things interact, start to depend or become unclear - numbers will increase. A numerical estimate therefore indicates complexity and uncertainty, not work to do.

We purposefully de-scope capacity planning, because we understand that as soon as numbers increase, there will always be a rather high residual error. We value working software over guesswork numbers, so we only estimate enough to know what we intend to do - not what we will be doing for certain. 

We may have hindsight 20-20, but regarding the future, we will always be guessing.

Effective facilitation

The problem of teams new to agility is that they need to self-organize and may not feel ready for that: Self-organization should never be confused with anarchy or absence of order! But how do you conduct an orderly meeting if it is not organized by the team lead?

The answer is: Facilitation replaces direction.

Who needs to facilitate? The obvious answer would be: "The Scrum Master" - for a Scrum team. Or the team coach for another agile team. But that's not the whole story.
For example, Product Owner might want to facilitate Grooming, Refinement, Planning and Reviews without relying on assistance.
For beginning teams, the facilitator should definitely be an experienced coach or Scrum Master. New Scrum Masters need to learn facilitation as quickly as possible. Without an experienced facilitator, you will be doomed to a boatload of frustrating, boring meetings - and there is no guarantee that it will get better!

More advanced teams do not rely on coaches or Scrum Masters for facilitation: The team can somehow facilitate, based on the team's working agreements and social structure and may just require coach intervention for special occasions.


But how do you effectively facilitate?

Purpose

Borrowing from Dan Pink's "Drive", Purpose is an important element of work.  Whatever meeting you are facilitating, ensure that the purpose is clear and meaningful. This will create engagement.
An important aspect of facilitation is ensuring that the purpose does not get diluted or lost. Intervene when the purpose is obviously lost, but permit potentially valuable sidetracks.

Do nothing

A facilitator's job is not to tell others what to do, say or think. Their job is to get others to do, say or think things. Here are three ways how you can actively do nothing to facilitate the meeting:

• Do not feel compelled to break the silence after a question was asked. Be comfortable with silence. Wait until someone else speaks.
• Accept that some of the things which are said may be off topic or misleading: Do not contradict. If you feel the need to intervene, ask a question without making a statement.
• When a discussion is going on, you may want to take notes, support with visualization or just silently reflect. Do not interfere in a well-going discussion unless the time box is up.

Ask questions

Questions are your most powerful tool for facilitation.
As facilitator, your responsibility is not to tell others anything about the domain of the discussion - your responsibility is to get them to speak. The best way to do this is by asking the right questions.

Games / Exercises

You may want to add some games or other exercises for lightening up the mood or directing the discussion. This may be good to break the ice or to prevent boredom, but be wary that people do not focus on the exercise. This can become a distraction from the original purpose! Do not load a firework of effects without a clearly planned purpose.

Visualize

A good way to facilitate during moderation is by visualizing. This does not typically mean drawing fancy charts, but it means putting the spoken words into something visible for the eye. This is valuable because misunderstandings or uncertainties become really obvious very fast. For example, drawing an arrow to the wrong bubble will immediately be corrected - without you having to say a single word!

Summary

Don't try to change everything at once. Whether you are new to facilitation or "still learning", pick one item that you wish to improve upon and change it for the next meeting. Learn from the feedback, inspect and adapt.
Over the years, your facilitation will continually become more effective through incremental change.

Ask the right questions

Questions are your most powerful tool for facilitation.
Many people are afraid to ask questions, because they feel it exposes uncertainty or weakness. But that is not true if you ask the right questions.
The right questions can reveal problems, uncover implicit knowledge, foster mutual understanding and create learning - just to name a few benefits.

Asking the right questions is an art. With one question, you can start, end, derail or destroy a discussion. It's completely up to you to pick the right question. But you can ask many questions:

• Open questions: For example, "What do you think we should do?" - you spark discussions with these. Use them to get others to talk.
• Closed questions: For example, "Do we all agree to do it like this?" - you end discussions with these. Use them to end topics or meetings.
• Probing questions: For example, when a developer proposed a specific approach to a problem, you may ask "What will happen if step 3 doesn't work out?" Use these to create deeper thoughts on specific matters.
• Counter questions: For example, when someone asks you: "Can you arrange an XXL TV for our CI build monitor?" - you may want to ask "What will you be seeing on this monitor?" to get developers to reflect on their expectations or gain further insight.
• Power questions: For example, you may ask "Why are we getting so many customer complaints?" - to provoke deeper thoughts on one specific topic without limiting options.
• Weak questions: To give a negative example, by asking, "Can't we just add more tests?" - you will do the following: Put your opinion above others' expertise, pre-empt a solution that may not even help and end the stream of thought. At worst, you expose a level of ignorance that will cause others to not take you seriously any more.

What you can do

You, too, can learn to ask the right questions.
For your next meeting, prepare one question which fits into that meeting. It may be that you already know the answer for yourself, but you want to check with the team.
Make sure it's a power question and make sure it's formulated as an open question.
Be prepared to add some probing follow-up questions.

Find the right setting to ask your question and wait for the discussion to ensue.

How fancy does a Retrospective need to be?

Especially new Scrum Masters might feel compelled to conduct a firework of Special Effects in order to conduct a Retrospective.
Fancy charts, stand-up exercises and highly facilitated techniques abound: An entire website dedicated to methods for facilitating Retrospective invites experimentation.

But how much glitter and glamour does a Retrospective need?

The Pattern: Funnel results

The general approach, as you can also find in the book "Agile Retrospectives", is a five-step approach to create clear results.
1) Set the stage
2) Gather data
3) Generate insight
4) Decide what to do
5) Close the Retro

That's good. But how formal does it need to be?

The Antipattern: Focus on effect, not content

Maybe you feel tempted or compelled to have a clear method for each of these steps, and want to use memorable patterns to guide the team. Here on this link is what the worst nightmare of an antipattern might look like:

Start out with an exercise with a prepared flip chart, continue with another flip chart, go on with another flip chart, proceed with yet more flip charts and conclude with - guess what - another flip chart!
You can now make this completely awkward for everyone by setting up flips with fancy pictures ("visualization") and asking the participants to write post-it's and put them on the appropriate sections of the chart.

Everyone will remember this Retro for certain - but not positively.

Who needs special effects?

Ask yourself one question: "Why would the team want a Special Effect in the Retrospective?" - if you can not answer the question adequately, any effect you are using is most likely waste.

However, if you find many compelling reasons for why the team can not have a good Retrospective without fancy effects, you should lay them out to the team.  Gather feedback to understand if your reasoning is actually just your opinion, or agreed fact within the team.

Hold a Meta-Loop Retrospective

If the team considers retrospectives stale, consider a Meta-Loop Retrospective: "What do you think about our Retros? How can we improve them?"
Maybe the points revealed would go in the direction of "Make more change happen", or "Reduce time investment", or "Better follow-up". Rarely do the points go in the direction of "Fancier effects, please."
Act on the Meta-Loop Retro before planning new effects. Only plan effects that match improvement potential suggested by the team.

Mind your audience

Regardless of what method you use for your Retrospective, please remember whom the Retrospective is for: It is the team.

Usually, the team consists of a wide variety of people. Some are result-oriented, others data-driven, some are silent observers, others active forerunners - just to name a few conflicting constellations.
Any method which specifically entices one character may be a put-down for another. Chances are that the things which sound fancy to an outgoing Scrum Master are not fun at all for an introvert Developer.

When you are conducting a Retrospective, do not think about "Which effects will make this Retro fancy?" - but ask the question "What would the team actually enjoy?".

Simplicity is essential

When you are conducting the first Retrospective with a team, you may be well advised to understand your team and their expectations first. The only thing you need to ensure is that the Retrospective results in viable change action. And then follow up and make it happen.

The best Retrospective may happen by simply having an open, focused discussion. But that depends on your team.

Use the minimum overhead and the minimum amount of facilitation which will give the team a Retrospective meeting their expectation.
You may occasionally use fancy effects to lighten up the mood, but do not rely on them: Any added effect could become a distraction rather than furthering the discussion.

What type of Product Owner are you?

The Scrum Primer has a very clear definition for a Product Owner. "[The PO] is responsible for maximizing return on investment (ROI) by identifying product
features, translating these into a prioritized list, deciding which should be at the top of the list for the next Sprint, and continually re-prioritizing and refining the list ..."

In corporate practice, however, PO is not PO.  Here are two considerations. These are not intended to value persons or their skill, but simply the PO's place in the organization.

Strong vs. Weak Product Owner

You can determine your PO strength with a single question: If they have a great new idea for their product, what do they do? There are many possible answers, but let's take this power scale for orientation:
The strongest Product Owners simply proceed when they are convinced.
Moderately strong PO's rally stakeholder support before proceeding.
Weak Product Owners ask others for permission.
Really weak Product Owners will not even proceed with their own ideas.

Real vs. Fake Product Owner

What does "real" vs. "fake" even mean?
Real Product Owners build their product to make it successful.
They hold the cash pile and can freely decide whether to spend money to build that new fancy feature or to throw a pool party for the team, should they consider the latter to be more valuable.  They thrive on unspoken customer need, because usually, the customer does not even know what they need.

Fake Product Owners, on the other hand, assist someone else in building their product. They receive requirements on a silver platter, grooming, refining and prioritizing them - and do whatever is necessary that everyone is happy with the result. They usually get assigned teams and/or budgets, and their responsibility is delivering a positive business case lest management decides to discontinue the product.

The Matrix

PO != PO

Strong & real Product Owners are the ideal of Scrum and agile Software development. With the right product vision and the right team, they will be wildly successful.


Weak & real Product Owners will continually run a tremendous risk of building the wrong product. In a highly political environment, they are most likely doomed to fail.

Strong & Fake Product Owners are quite comparable to Product Managers - the product is theirs to create, given outside input. To succeed, they must either depend on a Real Product Owner or hope that their direction pleases the customer.

Weak & Fake Product Owners are not even all that rare - we typically call them "Business Analysts". They are not involved in the financial success of the product. Their role is limited to preventing predictable failure.

Conclusion

It is neither good or bad to be anywhere on the matrix. It only becomes "bad" when you are that Product Owner and you feel that you should not be in this specific quadrant - then you must ask "How can we change that?"
If you have just concluded that your product is lacking a strong & real Product Owner, now would be a good time to look for one ...

You, too, can increase agility!

Individuals stuck in a classic waterfall organizations often ask "How can I be agile here?" - and quickly conclude, "I can't". This, however, is untrue. You can be agile. You will never be as agile as someone working in an agile organisation, but you still can be agile.

There are different levels at which agility can be realized in an organisation:

Agility affects all of these levels.

You can always influence some of these things, but you can not change all of them. If you are working at organizational level, agile practices are beyond your scope. Likewise, if you work as an individual developer, the structure of the organisation is beyond your scope.

Here are just a few pointers on how you can increase agility on your level:

Practice Level

Every developer is in charge of their own work. Agile engineering practices, such as for example Clean Code, Refactoring, Test Driven Development and Emergent Design are completely up to you.
While developers can not expect a traditional Project Manager to give them freedom for completely refactoring your 12m LOC legacy code or add unit tests to everything, there is nothing stopping you from writing better code today.
As a developer, you can increase practice level agility and convince others by results. Even if others are not convinced, you will be more agile than if you do not heed these practices.

Your primary constraint will be the scope of your work. You must accept that you alone can not save the Whales and that unless your organization supports wider practices like Scrum or Continuous Integration, you can only improve your own work.

Team level

Depending on how agile your organization already is, you may have "agile teams" already - or you may be led by a team lead. Assuming you are in either of these positions, the entire structure and practice of the team is up to you.
Even if your Product/Project/Line manager dictates the work items and their deadlines, you still can practice additional agility with your team.
Continuous Integration, Pair Programming, Standups, Retrospectives and Reviews are just examples of good agile practices that will all increase the quality of your work and the credibility of your team.
As a team, you can do all of this - as a Team Lead, you can direct your team in this direction to increase team level agility.

Your primary constraint will be the Product/Project context. You must accept that agile teams will not make unrealistic expectations more realistic and and that wrong plans do not automatically correct themselves.

Product Level

Gaining the freedom to develop an entire agile product is a large step in many organisations. To be agile at product level requires a lot of discipline and dedication.
Product level agile practices include working with a backlog, prioritizing and delivering by value, building increments, permitting room for learning and feedback in the product delivery cycle etc. Additionally, you must build your teams and processes around your product rather than vice versa.
As a Product Manager / Product Owner, you can institute all of this with the backing of your developers.

Your primary constraint will be that you need practice and team level agility, to succeed with product level agility. You must also accept that if the outside organization is still arranged in silos, you will still not build the optimal product and be fighting many political battles.

Organisation Level

Working with an entire organisation to increase agility is the best way to remove global impediments. However, you can not simply declare, "From tomorrow, we will be agile". Command and Control mindset will most likely still be ingrained within your organisation and silo structures will be rooted much deeper than org charts.

Organisation level agility can be supported by top management and can be stepwise instituted by middle management, but this level is very slow and requires much patience. 

As upper/middle manager, you can encourage higher agility and remove impediments during the Agile transformation. You can provide training, get coaching for yourselves and agile practitioners - but then that's about it. As an agile manager, your responsibility is not telling others what to do - but to provide freedom to make decisions and learn from mistakes. However, early on in the transformation, your responsibility is also enablement to make informed decisions and deal with ignorance.

Your primary constraint working on an organisation level is the current state of the organisation and the mindset of the workforce. You must accept that by doing things and making decisions for the teams, you are quite likely being counter-productive - so your biggest impediment is actually the entire non-agile organisation and everyone who is stuck in a traditional mindset. 

Summary

Regardless of where you are, you can do something to be more agile. Taking small steps is actually easier and faster at practice level - but the "big wins" are made slowly on organisation level. For maximum effectivity, all four must go hand in hand.

The "engine" of agile transformation

Everyone can do something to be a little bit more agile tomorrow. You can take small, thoughful steps: Grind the gears, but consider the whole picture. Moving too fast in one place might break the entire approach. Have patience.

Act on your Retrospective!

Some teams sit together nicely during the Retrospective, but they will dismiss the Retro as soon as it ends. Thats an antipattern: Improvements elaborated during the Retro are forgotten, nothing changes. This makes the Retro a complete waste and discourages the team.

The Retro should result in an Actionable Item and follow up on this action in the subsequent Retro. Consequently, a failure to actually make the proposed change should spark a Retro topic all by itself.

So, here are a few steps to turn your Retro into change happening:

Link cause and effect

During the Retrospective, there should be a visible line between the input which sparked the proposed change and the change intention. As the Scrum Master, unless the line is blatantly obvious, do not hesitate to iterate the journey verbally: "We discovered that ... which leads us to ... and we hope to address this by ..." . Not only will this increase clarity in the team - if you get it wrong, now is a good time to clarify this.

Be clear

State precisely what you want to change, why you want to change - and what the desired outcome of the change is. Likewise, define who should take the action item and how you will know if you got your intention. As Scrum Master, do not let the team "get away" without having an agreement on these teams.

Follow up

There is nothing worse than having volatile change initiatives without impact. Somewhere during the Sprint, you should have taken the discussed action. And hopefully, there are already results in the next Retrospective. As Scrum Master,  encourage the team to share the effect of the change during the Retrospective before digging into new items.

Learn from action

Provide a forum to discuss the impact, celebrate success and acknowledge learnings from failure. This encourages team members to make the next adjustment, because they will feel that the Retrospective is valuable.
If the change failed, be open about it and discuss whether you want to reverse to the former state or try an alternative route.
If the change was impeded, you can already use that as an entry point for discussion: What can we do about that, how can we be more successful next time?

Summary

As Scrum Master, you must lead the team towards higher productivity. This can only succeed if you take action on your Retrospectives. This requires planning, dedication and followup. Invest time both during the Sprint and the Retrospective to make sure that improvement becomes tangible to the team.

Prepare your Retrospectives!

Why do we have Retrospectives? It may seem obvious, but the idea is not to burn a couple hours in order to reminiscence the past. The idea is to do something better in the future. But how can you get that? You should know what you actually want - and that concerns the entire team!

The best way to get a meaningful result out of the Retrospective is by being prepared. Here are a few pitfalls which indicate your Retro should be prepared better:

Lack of purpose

Many teams focus on the ceremonial part of the event - doing things "because the Scrum guide says so" without comprehending the basic purpose. Much more important than how, where or when a Retro is conducted is the "Why". As mentioned before, the purpose of a Retro is driving the Continuous Improvement. Even before the team enters the room, they should be energized with the idea to drive change.
The Scrum Master can help build this energy level during the entire Sprint, but also by facilitation.

Whimsical input

Some people will only start to consider the Retro when it begins. Improvement potential unearthed during the Sprint is forgotten, they make up something in the heat of the moment just to have said something. This adds complexity to the Retro and may distract from valuable topics. A good way for the Scrum Master to counter this is by setting up an "Impediment Board" and encouraging people to add their impediment items throughout the Sprint. We can then bring this board to the Retro and simply start asking "Now - what?"

No pulse

Especially Scrum Masters sharing multiple teams often fall into the trap of not having taken a pulse for the Retro. Not knowing what is the perspective of the team, they need to waste precious Retrospective time fishing in troubled waters, trying to uncover an issue. Effectively, this is the Scrum Master's equivalent to "Whimsical input": Whimsical facilitation. The best way for the Scrum Master to get a pulse is by silently observing throughout the sprint and giving individual team members to speak up over a cup of coffee in a 1:1 setting.

Conclusion

Retrospectives have a clear, unique and very important purpose of introducing process, structural and organizational change to improve the team.
To get the most out of your Retrospectives, everyone should be well prepared.

Keep your Retrospectives focused!

The retrospective should be oriented towards a meaningful result, as you want something to get better. However, many teams struggle with the notion of Continuous Improvement being the team's responsibility, and so their Retrospectives are derailed.
Unfortunately, it is not only the teams who derail a Retro, but also well-meaning Scrum Masters.

Here are some problems that indicate your Retro may not be meeting its purpose:

Whine-fest

Yes, especially in transitioning organizations, there is often a lot to complain about. Even in well-running environments, we can complain about many things. However, the purpose of the Retro is not to provide a ranting forum, but to elaborate one thing that should be modified. 

Blinders

Often, teams zoom in on one thing that concerns them and their work, ignoring the big picture. Maybe the first thing a team should work on is something that only remotely concerns them, but affects the organization at large? It's just a maybe. But give room to the thought, in order to prevent local optimization. Think Lean: "Optimize the Whole!"

Predetermined agenda

Over-zealous Scrum Masters may be determined to set not only the topic for the Retrospective, but also steer the team towards accepting their solution to the problem. A Scrum Master has the full right to add an observation to the Retrospective, but they must realize they are facilitators, not actors!

Method Circus

There is a tool called "Retromat" out there on the Internet which might encourage over-ambitious Scrum Masters to fire off a wild pandemonium of effects in an attempt to make the Retro memorable. Unfortunately, the Retromat simply randomizes a set of methods without providing a continuum. This distracts, rather than focuses the team. A Scrum Master should be aware that Retrospective tools are intended to focus the team on the issue at hand, not on the method itself. While the Retromat itself is not bad, care must be taken to choose appropriate methods suitable for the session context.

Conclusion

A good Retrospective must be sufficiently open so that everyone can name their biggest concern and gives everyone a voice in the solution process, but quickly provides a funnel towards a single workable change. The facilitation of the Retro should support this process as naturally as possible, while still keeping focus on a meaningful overall outcome.

Why a good meeting room matters

Scrum has a set of ceremonies that are conducted in a meeting atmosphere.
They are Planning, Refinement, Review and Retrospective. All of them are essential to a well functioning Scrum team.
Let us de-scope the Daily Standup here, because it should be conducted in the team room anyways.

But you can't just have the meeting anywhere.
Here are a few factors of inadequate rooms:

• A noisy room causes distraction and breaks the focus. Keep the interfering outside noise level at a minimum.
• A dimly-lit room might make people sleepy and add difficulty towards both noting down information and reading notes. You need sufficient lighting. At the same time, avoid scorching lights.
• A crammed room discourages moving around, which both decreases the energy level and the potential for interactive problem solving methods. Everyone needs the freedom to move around unhindered.
• If the team is exposed to external scrutiny, this may reduce trust and willingness to discuss critical problems.
  Especially for the Retro, you will require a secluded environment where the team has privacy to discuss even highly political or personal issues.

It is completely valid to ask "Is our meeting room adequate?" - and treat "No" as an impediment to resolve.

Try to get the best possible space for the meetings to make them sufficiently productive.
If your team does not have access to adequate meeting space in the office, consider renting external facilities. It is better to pay a bit of money for space than to reduce team effectiveness.

Are problems your problem?

Many organizations, and likewise the individuals working there, struggle with the notion of "Problems".
A "shoot-the-messenger" culture makes it impossible to name, much less claim ownership for, problems. To be able to still deal with reality, the very term "problem", inherently considered bad and a career-killer, gets sugar-coated or diluted. People start talking about "impediments", "issues" instead - and avoid doing that as long and much as possible.

Problems are not bad

Innately, a "problem" is just a "problem". It means that something does not work out as originally planned. This is not bad, it just means we have to change the plan - we might still reach the desired outcome.
The first step is to accept that "problem" is a neutral term, indicating "need for change" - neither good, nor bad.
 

Apples and ... Problems

Let's compare a problem to an apple. You can either get the value from the apple (eating it) or let it rot. Once you have the apple, you either take advantage of it, or it spoils.  That's totally up to you.
Treat your problems the same way: When you see one, try to get as much value out of it as possible. By ignoring it, your value will be Zero, but you will still be stuck with the cost of disposal.

Where do Problems come from? 

Can't we just make a better plan to avoid problems? Don't problems just reveal the incompetence of the planner? These are typical questions from a problem-averse company.
But we must accept that the world isn't simple. Problems arise not only because we aren't omniscient, but also because nobody's perfect and the world is changing. While some organizations expect their managers to be omniscient and everyone on their staff to be perfect, they still don't live in a bubble: The world around them is still full of imperfection and is changing at a rapid pace.
Because of this, we can't plan for everything - and even if we did, the plan would still fail. And that's a problem.

A problem is no problem

Many people consider it a flaw to admit having a problem. Nothing could be further from the truth: Being keen in observing discrepancies between plans and how they work out indicates a good analytic capability. The biggest concern is that in a complex system, a problem in one area may require adjustments in another area. Only by communicating that there is need for adaption and why this need arises will the entire system remain stable.

No problem is a problem

Denial of problems or being completely oblivious about problems in the surroundings is a typical response by those who have been conditioned by their environment. Sentences like, "In our organization, there are no problems" are common in companies where the assumption is that "sufficiently intelligent people can plan and anticipate everything". Not only are both denial and obliviousness problems - an environment where openness and integrity are not valued is another problem.
Usually, an environment where nobody sees a problem is the most unhealthy, because needed change is often procrastinated until too late.

Talk about problems

You can improve the effectiveness of yourself, your work - and your organization with a few simple steps:

1. Admit that problems exist. Call them what they are.
2. Don't shoot the Messenger. People who name your problems are not your enemy, they want to help you improve.
3. Live Continuous Improvement. Deal with problems before they grow out of proportion.

3 Ways to increase your Retro effectiveness

Inspect and Adapt is at the heart of agility: "Responding to change over following a plan". The Retrospective is Scrum's standard ceremony to discuss potential improvements. However, especially new teams struggle at making Retrospectives effective. Consequently, the ineffective Retro becomes an impediment towards agility. Here are a few tips for new Scrum teams to increase the effectiveness of their Retrospectives.

Get a good room

Since in a Retro, you expect to make a significant change towards a meaningfully positive change, do not let the room hinder the team from coming up with good ideas for change.

Focus

Every minute spent on sidetracks and gimmicks destroys organizational value. Everyone has the responsibility to keep focused during the Retrospective and the Scrum Master's facilitation of the Retro should support this process as naturally as possible.

Preparation

The best way to get a meaningful result out of the Retrospective is by being prepared. On the other hand, being completely unprepared is the best way to make any meeting for any reason whatever - completely ineffective.
Both good teams and Scrum Masters prepare the Retro in advance. Then, they follow through and turn this well-prepared Retro into meaningful and highly effective action.

Summary

Continuous Improvement, is the only way to reach high performance. The Retrospective is the standard mechanism of establishing Continuous Improvement.
On the downside, poor Retrospectives will result in inherently low agility.

A Scrum Master must frequently reflect, "How can we make our Retrospective more effective?" - and adjust their own behaviour accordingly.

Measuring Agile Transition Progress

Many organisations ask, "How do we know if we're making progress?" - and, the answer is usually "To measure is to know". So, we need to have a measurement system which will help us determine success.
It does not matter whether we are looking to understand whether we are asking, for example, if our Scrum transition is making meaningful progress or whether our organization overall is improving. The underlying question "How do we know" and the need for a measurement system remains.

Ditch the Bullshit metrics

Before we dig into "How can we properly measure?", let us look at a few improper measurement systems. Many traditionally minded companies are trying to measure progress by looking at stuff like "Are we becoming better at following our Plans?" , "Do we produce more Lines of Code?"  - or even completely pointless metrics like "Do we complete more Story Points?".
Some still struggle with a Capacity Planning mindset and try to use "hourly estimates" to track progress.

Let us ignore for this post the widely varying specific reasons why these metrics are bullshit. There is a common underlying theme here: "All of these metrics are performance metrics." The problem you are trying to solve is not a performance problem, so why would you use performance metrics to solve a problem outside that domain? That's like measuring room temparature to figure out when the next bus arrives - bullshit.

Change your mindset for appropriate measurement

Regardless of whether you are looking at product development, agile transition or organization transformation - all of these also have a common underlying theme: You're not doing it for fun. You are trying to solve a problem. 

Therefore, the first step towards an appropriate progress metric: Admit that you actually have a problem - and acknowledge that you are trying to solve it. Without the honesty, "We have a problem and we want to know if we are solving it", you will never end up with an appropriate measurement system - because your measurement system should also acknowledge that you are dealing with the root cause of the problem.

Accept an undefined state

Traditional planning expects plans to be made up in advance, and then a report can be produced to state "Topic X = 20% progress". This, however, requires that you can predetermine the end point.

Unfortunately, when you are trying to solve a problem, that may not be so easy. Why do you still have the problem if you know exactly how to solve it? 

The first step is to acknowledge that there may be a total of 3 things you are not sufficiently aware of:

1 - Where do you stand (A)?

2 - Where do you want to go (B)?

3 - What is actually the best way from A to B?

A plan assumes that both A and B, as well as the path inbetween, is understood sufficiently well to define the path for transition. 

So, you must accept the idea that status measurement will not work out as well.

Define the Primary Issue

It's not as simple as stating "We did Waterfall, now we want to do Scrum" - that is not the problem you are trying to solve. Your problem is usually more deep-rooted, such as "Our customers are dissatisfied with the quality or delivery speed of our product" - otherwise, why would you change your current way of working?

Once you understand that you are not trying to go from Traditional Project Management to Agile Development, you will understand that you need to ask and answer completely different questions in order to even understand "Where do we stand?" - likewise, the question "Where do we want to go?" looks different.

For example, "We stand at ..." becomes "We are losing customers faster than we can gain new customers" - and automatically, the real question to answer is no longer "How can we increase self-organization?" - but: "What can developers do that would reduce our loss of customers?"

Consequently, the metric we'll be talking about is no longer driven by the agile framework of our choice, but by the primary problem we are trying to solve. 

Define transient success

Our agile transition is not defined "successful" when we are, for example, 97% compatible to Scrum based on the Nokia (Scrumbut) test - but when we have reached an organizational state where the Primary issue (and hopefully many others) is resolved.

We also need to understand that never does an organization only have one problem. Organizations are complex systems which are in themselves subjected to a chaotic system - the Market. By the time we have resolved one issue, it may already have changed - or, our understanding thereof may have changed.

Much rather than trying to make a better plan, we should simply accept that at any given time, we are not aiming to complete the plan, but trying to reach a more positive future state. When our understanding of the problem we have - or actually the problem itself - changes, then we should simply accept this. 

We can acknowledge that the future is shifting, the steps made in the past were useful - but that the next action in the present must be different from what we had assumed. 

Accept adaptive measurement

Every change that solved a problem yesterday can be considered yesterday's success. It was a real success, but today we need to solve a different problem, so we need a new way of discerning this success. In today's measurement system, yesterday's success may be insignificant or even counterproductive - so, we should no longer pursue yesterday's metrics.

Track meaningful problems

The most meaningful metric you can utilize is actually quite simple, and it's rather binary: "Are we actually solving our problems?"

We can elaborate this metric to a finer granularity by breaking down our Epic Problem.

For instance, "Poor quality" would refine into more intricate problems, such as: "Problems reported by Customer", measuring "How often and how long is the same problem being reported by customers before it gets resolved?" - "How many times did we miss an obvious software defect?" - "How long does it take us to catch a defect?"etc. etc. All of these metrics can then be easily drilled down and resolved, one by one.

Unfortunately, with these metrics in place, we most likely won't reach perfection, so our "target state" (e.g. Zero Defects@Customer, Zero Effort Testing, 100% Test Coverage) is more of a utopia than achievable. Therefore, tracking progress our "Epic Problem" becomes pointless.

Define how big you want the problem to be

The fact that our Epic Problem may never be 100% resolved should not stop us from defining precise targets for the definitely meaningful sub-problems.

For instance, we can determine: "Reduce defects in delivered product by 50% by end of year" - "Reduce acceptance test duration by 25% by next month without decreasing quality" - etc. These are specific, measurable, achievable, realistic and timebound: SMART measurement.

As seen from the two examples above, these metrics have no direct correlation with agility - likewise, they can become quite complicated to track. We can still use agile principles, practices and frameworks to improve these metrics. Afterwards, we can determine if the application of agility did help us reach these objectives.

Keep it simple and Lean

While some people feel the need for precision and metrical process control, we don't really need a complex, precise measuring system. Usually, we have a clear understanding of what is "good enough". We don't want to over-engineer. If someone says "We don't test enough", it may well suffice to say "Well - improve testing until we test enough". The person who said "It's not enough" may have a decent understanding of what "enough" means. 

You should only negotiate clear numerical goals if you presume disputing opinions.

Own your problems

The steps described above indicate that dealing with problems is very, very similar to the responsibility of a Product Owner - and that's no coincidence. You want somebody to take ownership. Someone who drives the resolution of the Epic Problems you are trying to solve - and also someone who drives the resolution of each workable sub-problem you are dealing with: You naturally end up with a Chief PO (Problem Owner) and Area PO's (Problem Owners).

Get a Problem Board

Probably the easiest way of dealing with your problems is to track them just like your team tracks the Stories on their Storyboard. You can see what's "toDo", what's "in Progress" - and what's "Done". This storyboard becomes the natural information radiator of your progress on your Agile journey.

Organize Problem Solving

If you are moving towards Scrum, it also makes sense to arrange an entire Scrum-like organization around the Problem Board. You can work with Plannings ("What problem will we tackle in this sprint?") - dailies ("What problem have we solved today - etc.") - hold Retros ("How did our problem solving go?") etc. etc.

Track Problem Solving

You are becoming "more agile" when you are getting faster and better at solving problems. Keeping a high level overview of how long it takes, on average, from the time you become aware of a problem until you no longer have this problem, is the best measure of how agile your organization is.

Summary

To measure the success of your agile transition, you must be willing to name and face your problems. The best indicator of success is how you deal with your problems. By institutionalizing problem solving, you both establish a transition measurement system and take specific action.

Do not look for "feel good" indicators. Accept problems as your primary metric. 

You will feel good when you have less problems.

3 Pitfalls to avoid as a Product Owner

The Product Owner is the "single wringable neck" in Scrum. This implies that a huge burden is resting on their shoulders. However, we learned from Lean Management that "single points of failure" are a bad idea. So - how can the PO prevent being the SPoF?

1 - Micromanagement

Some product owners - and even Scrum teams - feel that all decisions about the Product, including features, UI and UX must be made by the Product Owner.
( Let us ignore for a second the fact that this is turning developers into mindless drones, because Software development is exclusively about decision making - the "implementation" is the easy part, finding out how to do it best is the hard - and fun - part. )

This implies that the Product Owner must be available whenever a decision is being made.
Also, that the Product Owner would need to understand the intricacies of UI, UX, SEO, SMO etc. (which would make the PO quite a jack-of-all-trades) - it means that the PO is considering the Product on a very base level.
The Product Owner should be more concerned with strategic propositions of the Product - such as which customer segment to target, what value proposition to offer - and how to turn a more or less clearly understood market need into a feasible backlog item.

2 - Being a designer

Some go as far as having the PO explicitly define all Acceptance Criteria and spoonfeed them to the team. There are two problems in this:

• This forces the Product Owner to spend a significant amount of time with intricate details of the "How" in implementation. 
• The product design is biased around the Product Owner's understanding of technology.

Let us consider a Site Landing, for example: "As a customer, I want to register so that I can use subscription services".
The Product Owner proceeds to design the Wireframe including all form fields and buttons, defines the events and hands that to the team. But that is bad for the product. Why?

Because the solution was pre-empted: Point in case - customers DON'T want to fill in registration forms. (just consider yourself: When was the last time you enjoyed that process?) Customers want to use services. Registration is the standard way of making subscription services available. But there may be different ways of identifying subscribers, such as - referring to an existing Google / Facebook accounts, etc. - you might even want to use facial or fingerprint recognition. That's easier for the user and serves the same business purpose. And users don't end up with the umpteenth password to remember.

3 - Being a Scrum Master

While the Product Owner is responsible for the success of the product, they are not responsible for enabling the team to work un-impedited. Scrum clearly discriminates the Product Owner and Scrum Master roles, for multiple reasons.

One of these reasons is, of course, that the Product Owner wants maximum value in the shortest possible time - while the Scrum Master might have to shoo back the Product Owner on decisions affecting technology that might compromize sustainability of the Product. The most common decision (un-enlightened) Product Owners make is to cut short the testing in order to speed up delivery. In these cases, a Scrum Master must intervene.

If the Product Owner perceives the urge to go about educating others about Scrum and resolving team impediments, this means that the Scrum Master isn't doing their job. No accusation to the Scrum Master - maybe that's an organizational issue. But if the Product Owner does the job of the Scrum Master, then the team remains in dysfunction: Not only does the burden on the PO increase, the key impediment does not get resolved.
It is better for the Product Owner to step back, sit with the team and Scrum Master - and find a solution for this issue than for the PO to sprint into action.

Summary

Being a good Product Owner may - occasionally - imply doing something that is definitely outside the sphere of your core responsibility, i.e. envisioning and packaging a great product. However, if you see that you are spending a significant amount of time with stuff that should be done by others, ask yourself "What am I doing here?" - give it to the team, and refocus.

You will only succeed as a Product Owner if you do what a PO should do best - not when you do what others should be able to do better.

3 characteristics of outstanding Product Owners

The success of your product depends to a large extent on your Product Owner. If the PO directs the product in the right direction, your success chance will be maximized. If not ... your product may well fail.

How do you decide the future of your product?

Here are three characteristics of outstanding Product Owners:

1 - Obsessed with customers

Customers are not merely those who pay for what you produced. They will use the product - they know what annoys and what pleases them. They are not just consumers. They have opinions, ideas.

Every Product Owner must understand their customers, how they act, what they think, what they want.

Good Product Owners have a good understanding of who their customers are and how they use the Product - and what they will pay for.

Great Product Owners drive this to obsession, constantly looking for new and better ways to understand their customers better, to get closer to their customers.
They become the personification of "the customer" when discussing the next Product Increment - and they become the avatar of "the Product" in their interactions with the Customer.

Outstanding Product Owners are emotionally attached to the Customer and the Product. They feel joy when something is done well, but they also suffer when customers suffer, they feel pain when the product is inadequate - and they can't "not take it personal".

2 - Caring for numbers

Numbers are not everything, but numbers are important. You can't be a good Product Owner if you only groom and discuss user stories. You must correlate them to the needs of the customer and the business.

Every Product Owner must understand the Product Vision, communicate it clearly and base Releases, Epics, Stories and Features on this vision. They must always be able to draw lines forward and backward between these items and be ready to restrict everything that is not in line with the product vision.

Good Product Owners are not only able to map stories. They can relate them to business figures. They understand which Epic, story or feature delivers how much business value. They can clearly discriminate between "gold plating" and Minimum Viable Product. They will deliver the highest value first and cut discard items which do not have a good bottom line.

Great Product Owners will go one level beyond that: They do not only attach a business figure to features, they relate Performance Metrics, such as Clicks, Conversion Rate and hard cash earnings. They do not only predict which feature is most valuable - they build a product that lets them measure, so that they can inspect and adapt their product to maximize value and customer satisfaction.

Outstanding Product Owners transcend all of that - they have the data and are ready to produce accurate analysis at a fingertip. Additionally, they understand the large scale implications of a change: They do not focus only on the product, but on the ecosystem in which the product exists. This includes being aware of potential on short-term gains that might decrease future sustainability - for example the phenomenon called "Mudflation" in MMO's.

3 - Disregarding agendas

Of course, the Product Owner works in your organization. But the loyalty of the Product Owner lies with the Product, not with a specific portion of the organization. They own the product and the product directs them - not management!

Every Product Owner must be in charge of their product. If you're looking for someone to whom you can spoonfeed what to do when, you're not looking for a Product Owner - you're looking for a Project Manager. That's OK, but don't call it Scrum, don't claim to be Agile - and don't wonder why you're wasting lots of money on a failed product: It was a management decision.

Good Product Owners take ownership, just like the title of the role proclaims. This means that they will do what, based on their understanding, is the right thing to do. If management wants the Product Owner to do a certain thing with the product, they will provide clear information on why they want that - and must be willing to accept a "No". A good Product Owner will tell management if an idea is stupid.

Great Product Owners care more for the product than for themselves. They will do the right thing, even if it is a personal disadvantage. They can not be threatened or scared by backroom politics or titles. Neither will they be swayed by personal gain - you can not bribe them with pocket money or promotion. If a suggestion is bad for the product, it will be rejected. Likewise, just because a suggestion is good, it will find it's place in the Product Backlog based on product value, not on someone's title or rank.

Outstanding Product Owners are beyond your company. They choose to dedicate a portion of their life to the product - if you want something from them for a political reason, be prepared for rejection. If you have a suggestion that would destroy their product, be prepared for fire. If you want to gut the product for short term gain, you have to accept that they will make the product successful without you.

Summary

The decision making process of an outstanding Product Owner is driven by customers, numbers and value. You can't own an outstanding Product Owner - and they own the Product.

Before you hire a Product Owner, decide if you really want a Product Owner. Your choices are still to hire a Project Manager, an Analyst or a Requirements Engineer. None of these are Product Owners.

An outstanding Product Owner is not a person to whom you simply delegate work in the development process.

3 things you won't get from me as a Product Owner

The Product Owner is the "single wringable neck" in Scrum. Their understanding of the product and the vision combined with their ability to decide determines success or failure of a product development team.

So, it's only fair that the Product Owner should be able to provide every information about what's going on? No.

Here are 3 things which you will not get from me when I'm a Product Owner.

Assignments

If you are a team member, do not ask me to give you an assignment.

I don't assign stuff because I couldn't. I could. Easily. But I don't. Here is why:

Even before the Sprint, I determine what matters most to the Product. During Sprint Planning, we mutually agree on the team's priority. Your top priority is visible on the Scrum board. It's the topmost item that is not in the "DONE" column.

If you are uncertain which task to pick, you have team members to help you out. Again, during Sprint Planning, the team has agreed on the necessary work items required to get the Stories or Features properly done. I let you, as the team, break it down because you know your trade. And I expect that when you do that, you know what's most important and what is the next step from where you currently stand.

If you don't understand the context of something you work on, I'll gladly discuss with you. If you don't know if something is necessary in order to deliver the value, I'm also open. But I won't tell you what to do - or how to do it.

If you, as a team member, don't know what to do next, I see a serious impediment in team communication. That's something to discuss with the Scrum Master or cover in the next Retro. But the Product Owner is really the wrong person here.

Gantt Charts and Reports

If you are a manager, don't ask me for a Gantt Chart or a % progress report on your backlog items.

I don't produce these because I couldn't. I could. Easily. But I don't. Here is why:

I decide what gets done and when the team engages. I can't tell you when it's guaranteed to be done. You wouldn't want a trash product - you want quality.
Percent Progress Reports are a distraction. You already know from Windows Installer, "The last 1% may take longer than the first 99%". It takes as long as it takes.

I can tell you how big your chunk is and where it sits in my backlog. I can give you an estimate if we'll tackle it in the next sprint, within this quarter - or not.

If you're not satisfied with my answer, we can re-negotiate priority and scope, thereby getting your stuff through the door faster, but we can't negotiate quality or delivery date. Sorry.

Performance Appraisals

If you are Human Resources, don't ask me for individual performance appraisals of developers.

I don't give you an assessment because I couldn't. I could give you my opinion. Easily. But I don't. Here is why:

First, because it's just my opinion. It's tainted by how much I have interacted with that person. It does not reflect total contribution.

Second, is the team producing significant value or not?
If not, that's my problem. If they do, they are doing a good job and then that's your assessment.

Third, let me ask: Why do you even need individual appraisals? Is it "rationalization"?
Let me be witty here: You don't ask your dentist which of your teeth works best, so that you can pull those out that contribute least. You'll want to keep all of them, unless you want to reduce your diet to pulp and soup.


A team is a team. Teams have very complex mechanics. Teams work as teams. If they don't, ask the Scrum Master what they are doing. If they do, then each person has their part.

If you want any information about the team, ask the team. They know best.

Summary

As Product Owner, I will give you the most valuable product in the shortest possible time. I am not the Team Leader or a Project Manager, despite the fact that I have significant experience in those things.
We will never get anywhere as long as I act as if I were the brightest bulb in the lamp.
I trust the self-organization of highly competent individuals to do what is right.
If that is not happening, that's something to improve upon - not to undermine.

No changes to the Sprint?

A very common question that is echoed by many Scrum teams: "What if we discover during the Sprint that a story will not help the user?"

The problem manifests in different facets, such as: A story was improperly groomed, new information became available which invalidated old information, a hypothesis about the product was disproved, etc.

Let's ignore for a minute the fact that "if we had known better, we would not have accepted this story into the sprint". We didn't and so we did. It's there. Now what?

The easy (and wrong) answer

Following the strict Scrum rules, the question is easy to answer: Continue the Sprint, deliver, use the Retro to elaborate what went wrong and do it better next time.

That's obviously wrong
We're not in business to "do Scrum". We're in business to deliver working, useful software. If we're not doing that, we're losing our reputation, credibility and - consequently, our customers - in turn: our jobs.

The correct answer

Common sense dictates the correct answer: Do what is right. Now, what is right? We deliver working, useful software.

Rather than referring to Scrum, we need to refer to the Agile Manifesto in answering this question:

Let's start with the often-ignored first sentence: "We are uncovering better ways of developing software by doing it and helping others do it." Is it "better" to build something useless? Obviously not. Being agile indicates that we're doing the best we can and from there, we look for even better ways. By simply doing what we're told, we neither do the first, nor the second.

Now, let's screen the Agile Values:

People and Interactions over Processes and Tools - you don't want to follow "the Scrum process" when a face-to-face communication can solve a problem that only exists because of a process.

Customer Collaboration over Contract Negotiation - when your "sprint contract" says to build and deliver X, yet X doesn't make sense to the customer/user, collaborate to find the Y which does.

Responding to Change over Following a Plan - when new information reveals that the Sprint Plan is no longer valid, respond to this change.

Scrum or Agile?

Scrum is an agile framework. The Agile Manifesto is the basis of Scrum. Without it, Scrum is a hollow, lifeless shell that will fail to deliver value. It would merely be a cargo cult, a "scrum-but" of the worst kind.
Scrum thrives by living the Agile Manifesto.

Refer back to the Agile Manifesto before clobbering someone with the Scrum Guide or Scrum Primer.

Summary

When you discover mid-sprint that your initial sprint plan (or, even a part thereof) does not make sense - grab your team, including the Product Owner and find a solution.
Never follow an invalid sprint plan: Change the plan, but do it in an agile fashion!

SysOps and Development: An Antipattern

Developers develop, Sysops operate. Obvious.
Well, that's classical thinking.

The consequence is that problems do not get resolved on time: Users suffer, value is lost and the reputation of the IT department is tarnished.

Obvious problems often don't get resolved in a timely manner - the loser is: everyone!

What you see above is real data taken from an organization where some Scrum teams are responsible for creating features and another team is responsible for operating the platform.

Just taking this example, the biggest current issue is an encoding problem - something that happens in the "real world", but not in a controlled development environment: Continuous Integration was successful, a feature hit the market - and boom!

Separate Responsibilities

In the example above, developers are busy churning out new features while SysOps are busy with fire-fighting.
Taken from the chart, it's been 3 days since the problem started - 3 days where SysOps are constantly stressed while developers are busy with stacking more features on top of existing problems.

XP proclaims "A Red Master is developers' Priority 1" --- well, whose priority 1 should a Red Production be? SysOps or everyone?
In a classic Kanban system, when the production line goes down, it should stop the entire machinery until the issue is resolved. This also holds true for agile teams: If anywhere in the organization, there is a problem, stop. Fix.
Don't go on producing more problems.

Perverse incentives

Developers are appreciated for delivering new features, while SysOps are appreciated for keeping the platform stable. Consequently, Devs prefer to work on new stuff rather than fixing existing stuff. Unfortunately, the organization as a whole suffers. Not rowing in the same direction can't yield any better results.

The very fact that people have different incentives here implies is a problem with autonomy and self-organization!
You must align the incentives of everyone working on the product, not only "development team".

Definition of "Done"

Regarding developers higher for delivering a new feature than making an old feature stable yields the problem above: "Just add a new story into the Product Backlog and we'll deliver it with the next sprint." - while this is a classic approach to Agile Software development "There are no bugs, missing Acceptance Criteria are new stories" and Scrum's "The Sprint is Closed towards modification".
This is a severe misunderstanding: Can you actually consider a story "Done" when it causes problems to the customer?

When you see that there's an operative problem with a new feature, it means that in your retrospective, you should put your DoD under scrutiny.

DevOps: A solution

As the agile proverb goes,  "You build it, you run it."
As long as developers do not feel the operational pain, they are not interested in removing it. Likewise, as long as SysOps are unable to fix the root cause of a problem in source code, they become apathic towards code-related problems.
Moving both the operative problem as well as the solution space into the team's sphere of responsibility, developers will take an interest not only in the code they write and how it meets the acceptance criteria, but also how the code actually performs in the real world.

Organizationally, this can be done as simple as moving a SysOp into the development team and then holding the team accountable for events on the productive platform.
The Developers become Dev-Ops who will come up with innovative solutions to detect, analyse and prevent problems. A DevOps can solve issues in a fashion that would be completely unthinkable for a SysOp who is confronted with a "build ready" black box software.




Side note
The above screenshot was taken from Medullar, towards which I, personally, am completely biased. I am one of the core developers of this easy to use, open source, cloud-based, universal monitoring and problem solving solution which was designed to require minimum levels of intrusion and resource capacity while offering maximum flexibility.
Medullar allows you not only to detect and analyse problems in any kind of environment. It provides an API and UI to remedy them in real time. It even comes bundled with it's own test framework, so you don't need anything else to improve your operative performance.

What's better: Scrum, Kanban or XP?

The question "Which agile framework is the best?" is often raised by people who are fairly new to the world of agile frameworks. In short: I can't answer this question without asking a counter-question: "Best for what?"

 

 

When your only tool is a hammer, every problem becomes a nail!

Let me analyze the problem in regards to the image above.

Scrum: keep things in place

Scrum is primarily known for the sprint cycle process and the ceremonies: Grooming, Refinement, Planning, Daily Standup, Review and Retrospective.
Scrum discerns only the roles Product Owner, Scrum Master and Developer. You need a Product and Sprint Backlog consisting of Stories, in the latter broken down into Tasks.
You'll need to define suitable Working Agreements, such as a Definition of Ready and Definition of Done with your team, then use Inspect+Adapt on these rules to optimize your process continuously.

Scrum is quite rigorous. If you do not have any of these things, you end up with a disdained "Scrum-But". If you do something beyond, you usually also end up breaking your Scrum.

Scrum is a wonderful framework for creating structure, clarity and order. Additionally, Scrum reduces volatility, enhancing predictability and confidence.

Kanban: Create flow

Kanban is often confused with the Kanban board, the most notable artifact. Making work visible, creating a smooth flow of work as well as identifying and removing bottlenecks are all key factors of successful Kanban.

Kanban is very flexible. You can start with Kanban wherever your organization is, without interference to structure or process. From there, you start optimizing.

Kanban is a wonderful framework for making impediments visible and optimizing delivery.

XP: Professional Craftsmanship

Extreme Programming is mostly concerned with Engineering Practices. Techniques such as Pairing, Test Driven Development, Continuous Integration and Refactoring will lift your developers to new levels of excellence.
Design concepts such as Whole Team approach, Collective Code Ownership, Clean Code, Emergent Design will broaden your team's horizon and capacity.
Lean concepts such as sustainable pace, Metaphors, Small batches, Standards , Automation and Continuous Improvement reduce waste and risk.

XP is process-agnostic. You can pick individual practices from XP that solve specific needs in your team, although taking the batch is advised for best results.

Summary

None of these agile frameworks is "better". Each of them has a specific area of use. As such, the frameworks become "better for solving a specific problem". Therefore, you need to define your problem first before answering the question "Which framework is best?" - a hammer is better for fixing a nail, but not for a screw. Of course, you can cut a board with a screwdriver, but a saw would suit your need better.

Take some time, define your specific problem, then work from there.
Here are some rules of thumb:

• Solve organizational or process issues with Scrum.
• When you can't seem to get anything "done", try Kanban.
• When your perceive quality or architectural issues, apply XP.

These frameworks are not exclusive. You can mix them - and high performance teams combine them. Scrumban is famous, for instance.

Disclaimer: None of these frameworks will actually make you successful in the long term without an Agile Mindset. If you have issues with Agile Values or Agile Principles, your only fix is returning to the Agile Manifesto - regardless of which framework you're using!

Why Test Driven Development?

TDD is sometimes proclaimed as a means for early discovery of all defects. Sometimes, people who are new to Software Craftsmanship practices wonder what tools they should use for TDD.

What TDD is not about

TDD is not so much about what tool you are using. TDD is tool-agnostic. The best way to do TDD is to use the same programming language. If you're looking for tools, you look in the wrong direction.
It's not about having a high test coverage. It's not about minimizing quality risks. In fact, it's not a Quality Assurance technique at all. If you use TDD for finding bugs, you're definitely on the wrong track.

What TDD is about

The key to successful TDD is to produce code in a fashion that each component is fully inspectable down to the most granular level, so that:

* You have a good software design

* You fully understand what each component, method and API does

* You minimize overhead of future reverse-engineering

* You lay a basis for easy changes, so that now you don't need to waste time on thinking about possible future changes

* If you need to change something fundamentally, you know where to best do it

* If something doesn't work as intended, you can isolate the root cause quickly

Summary

The main reason to employ TDD is not if you have "lots of defects". Chances are that you will not only not reduce the defect count with TDD on a poor quality legacy application - but that you will invest a lot of effort into increasing coverage without getting any measurable benefit to the user (customer) at all.

The best reasons to employ TDD are: when developers complain that they can not make changes to the software easily or they complain that finding the root cause of defects takes forever.

TDD is architecture+design topic.
The first minute you start doing TDD, each new line of code written, you will be working towards a better software design. It definitely pays off in the long term. 
Just don't look at the release defect metric, because you will not see the benefit there.

The five key ingredients for successful software development

All you need is good code that meets the requirement and you're set? That is short-sighted!

You need to keep a keen eye not only on the following key processes and artefacts to be successful in the long term.

Working Software (Code)

That's obvious. If you don't have a product, you have nothing.
This not only includes what the customer sees, but also what is going on behind the scenes. The statement, "A kitchen where someone cooks is always messy" might apply to the developer's office, but it must not apply to the code.

Smelly code costs real money and binds capacity that would be better focused towards innovation and progress.

Test Suite

To understand whether your product works correctly and does the right thing, you need a proper test suite. I radically proclaim "If your test isn't at least as clean as your productive code, don't bother".
A common (anti-)pattern I observe is that organizations de-prioritize testing in order to meet certain goals. Or, they actually put an emphasis on testing, but approach testing wrongly. The consequence? Flaky or unreliable tests that cause more trouble than they solve.

The money invested into testing is misspent if your tests are not well-managed, clean and comprehensible. However, abolishing your test suite equals abolishing your investment into sustainability. Don't ever reach this point.

From day 1 of a new product, invest into a proper test suite. Treat your test suite as the Holy Grail of long-term sustainability.

If you have existing software with that, go seek that Holy Grail - today!

Working Build

To actually reach Continuous Integration - or beyond, you need a smooth, swift build process. You should be able to modify your build as easily as you can modify your product's features.

I see many teams and companies who under-emphasize their build process. A sad fact is that they all run into build problems sooner or later. Some can't manage their dependencies any more, some don't understand why their build is unstable - and the build becomes a nightmare for every developer!

Like you manage your code and the test suite, you must manage your build.This includes the build management system, the build artifacts as well as the build infrastructure.

Environment

A good product is not impeded by the environment, which includes technical infrastructure as well as corresponding tools and integrated components.

An overly complex or under-performing environment are equally detrimental to the success of your product.

For example, the wild growth of turnkey service solutions is the best way to lose control of what is actually going on. It is quite easy, for instance, to include a tracking or ad service. It is, however, incredibly hard to debug your software if such an external service is causing a malfunction in your product.

Another example, if you are using a myriad of virtual machines without having a proper hardware management will cause tremendous trouble when trying to find out what went wrong where, and why. The time lost to a poorly managed environment can quickly exceed the time you are actually spending on the product you're building.

 

Runtime

"Works on my machine" - isn't it a running gag?

How do you control that the product also works in production? Of course, you need monitoring and feedback.

But technical monitoring doesn't cut it.  Have you ever taken a detailed look into your log files? Whenever I engage with a new client, I take the liberty of scanning the productive logs for stack traces and the other obvious stuff ("Exception", "ERROR", "CRITICAL", "FATAL"). Doing that only takes a minute with grep and I usually find a myriad of stuff for the next retro.

Sometimes, the software doesn't display any obvious sign of malfunction for your (Dev-)Ops while thousands of users around the world have no clue why they can't reach their intended goal.

To understand and control your runtime properly is a tremendously important - yet oftentimes underestimated - success factor.

Conclusion

You must take care of five critical ingredients:

Code, Test, Build, Environment and Runtime.

Give sufficient attention to all of these will drastically reduce your risk of failure.
Ignore one of them and you will find yourself struggling to keep customer satisfaction and progress up.