Thursday, September 24, 2015

Done, Done-Done, Done Done-Done?

The "Definition of Done" is a fundamental artifact in Scrum.
It allows the team to decide when they can safely consider their work "Done".

However, many times, team fall into the trap of "Done-Done".
The root cause is usually external dependencies, such as being unable to test software ("Story Done" vs. "Test Done") or a separate Operations team ("Go-Live Done").

At scale, matters quickly worsen: Sprint-Done doesn't mean that there is a Working Release, much less that it has been deployed to Production.

As a first step, the solution is usually to come up with separate definitions of Done.
When describing these, please note that I mean: "First Step" of improvement, not something to be proud of. Having half a dozen of Definitions of Done is a smell, an indicator that something is very, very wrong!

Here are some definitions I have already encountered in the past:

Story Done

The basic activities required to say "We're done with development". Basically, this typically contains fulfilling all Acceptance Criteria, Code Review, Pull Request approved. Ideally, the Acceptance Tests are fully automated as well, although that one is already a compromise in many large organisations.

Sprint Done

The basic activities required to say "We're done with development". Basically, this includes that the Product Owner has accepted all delivered Stories and having merged all Stories into a Green Master.

Test Done

After developers have built "Working Software", it must be deployed and running on an integrated test environment where you can test the interaction with other components and services as well as with real people.
Some parts of the tests that may need to be done include Integration Testing, Performance Testing, Exploratory Testing etc. Just refer to the Test Quadrants to get some ideas of what may be needed.

Deployment Done

After tests are passed, the software must be Up and Running on a Live System. Alternatively, if you are shipping Vendor Software, it means you have a Shippable Product that has gone out to the customer. 

Release Done

 A Release is "Done" when no more activities related to the release are oustanding. Beyond deploying Working Software, it may include automated monitoring and reporting, looking at first figures on the Live Server etc.


The illusion of "Done"

At worst, you end up with a state of Done-Done-Done-Done-Done-"Not Done-Back To Step 1", where faulty stories need to be fixed weeks after they were marked "Done".
As a consequence, old work starts to flow back into current sprints, reducing predictability, velocity and morale.

Oftentimes, companies take this venue because either:
  • There is too much pressure to get something "Done" 
If this is the case, management must be educated that they are sacrificing a high amount of sustainability for an insignificant increase of speed.
Listing the "hidden factory" work that must be done by the team outside of "Story Done" and figuring that into the total cost of delivery may help.
  • They don't know how to do it differently
 From a strategic level, this is easy to fix. Operatively, it requires a lot of organizational change.

 

Minimize "Definitions of Done"

Team Definition of Done

A "Story" should not be "Done", followed up with additional work to get the "Sprint Done". The first step is to eliminate this gap.
"Story Done" should be attained when there is no more work that the team can do in the sense that from the team's perspective, there is Working Software by the time Story is set to "Done". A story is never "done" unless it is already integrated into the Master and ready to deploy to Production.
Provide a single Definition of Done for the team that includes everything the team can do. Accept no compromise, no intermediary steps.
Other teams can do this, so your team can do it, too!

Organizational Definition of Done

Never take the route of splitting into stuff like "Testing Done", "Release Done", "Deployment Done".
When setting out, catalog all "undone work" which the team does not do that happen before actually bringing the software into production into a single "Organizational DoD" (ODoD).
After creating a catalog, start out by eliminating the obvious waste.
Then comes the tricky part.
Every item on the ODoD should be considered an impediment for story delivery: Anything "urgent" is always blocked by the ODoD.
Depending on your organization, it may take years to move all items from the ODoD towards the team. This should not discourage you from undertaking the journey: Every reduction to the ODoD increases the organization's flexibility and risk of failure in case of emergency.


The Vision of "Done"

There is a single Definition of Done for each team.
The teams act upon this definition and not anything else.

The team can complete all activities necessary to reach Done.
No other party must become active for any item to become Done.

"Done" means "Done".
There is no work left to do when the team moves the card to "Done".

We know it's over.
The risk of failure is already eliminated, so there will be no residual risk of having to undo.


Tuesday, September 22, 2015

Weeding out the backlog

When companies transition to Scrum, or any other Agile methodology, they usually start with the problem of arranging all the identified "undone work". Where do we start?
In the worst case, huge amounts of "overdue" items are all considered "Priority 1". In this situation, the Product Owner must clearly indicate where the team should start.

Other organizations seem to suffer from a kind of collection addiction, keeping a backlog of hundreds of items that won't ever get done.

Here are some tips for maintaining a nice, tidy, workable backlog.


Intially setting up a backlog

Before even bothering to produce detailed user stories or doing task breakdown, the Product Owner should simply draft out the basic areas of the Product. 

Post-It notes with maybe one or two words will do the trick. Then, draft the matrix below on a board and place the sticky notes:

The backlog item placement matrix
Importance and urgency: Arrange from High to Low
Put your backlog items on this matrix!

Impact

When weeding out based on impact, you must identify your customers and stakeholders. For each item you identified, you must ask "Who will suffer if this is not implemented?
If the impact of not implementing the item is low or not even measurable, the item is certainly not important.

Urgency

Next, you weed out based on urgency. A good initial question to ask when chaffing out for initial urgency is: "If we do not deliver within 1 month, will there be significant damage to our business case?" - if the answer is "no", the item has low urgency.

Unclear cases

If you're unsure, feel free to put the item on the dividing line between low/high.
When that region clutters, you can benchmark by using the top item on the dividing line. Ask yourself for each item: "Does this have higher impact / urgency?" to make a decision quickly.

Scrap

Ideally, you will end up with a good number of items in the "Scrap" section. Congratulations, they are off the table! You do not need to waste any further minute with these topics.

Defer

For anything in the section "Defer": put those items on the bottom of your backlog and leave them there. Do not waste time now grooming them, because you will not work with them in the near future.

Why?

For anything in the section "Why?":

If you would invest a lot of effort for little impact, move the item to the "Scrap" section.

When you're just starting a new product, it's probably a bad idea to have anything from this in your backlog, because you want to get your product going - NOW!

When you're working on an existing product, you may want to look for "quick wins" in this section. Anything that will give you a credibility boost at low cost should be considered.

As long as the item still has a positive Return on Investment, you should do it, but only if you can squeeze it into the schedule.

Only groom items which will bring your product ahead with minimal effort. 
Move the others into the backlog until further notice.

Do Now

Should you have significantly more than five items here or dispute over priority, you need to weed out further.

The items remaining in this section are your high level backlog ("Epic items").
These items need to be made workable, such as by using INVEST. Break them down to feature level and start with the Story Breakdown.

Ordered weeding

If you feel uncertain about being too lax with weeding and up with a cluttered "Do Now" section", you should use "Magic Numbering" on the cards twice.
During the first round, simply arrange the cards by impact. After finishing, arrange them into buckets from 1 to 5 with ascending impact and note the impact on the cards. Unless you have less than 6 items, each number should have at least 1 item.

During the second round, arrange the cards by urgency from "low" to "high" in ascending order from left to right.
Then, use the noted numbers of impact to move the cards up (high impact) or down (low impact). Make sure their horizontal position remains unchanged. when arranging them on the vertical axis.

Afterwards, draw the lines - not in the middle, but between the "3" and "4" rows and columns, so purposefully create a bias towards "High".

Wrapping it up

You may flinch when you see that "lots of valuable stuff has gone off the list", but if you did the arrangement correctly,  only "even more valuable stuff will get done".

Your initial backlog should be very small and tidy at this point, easily workable for further breakdown.
Likewise, you can immediately and effectively quench discussion about any other topic with the simple argument "It won't get done now, anyway".

Recursion

You can apply the same process with the story / feature breakdown of each item. When moving into the first sprint of the team, it's actually a good idea to not overload the team with information. Weed the Ready ("groomed, broken down, workable") backlog to no more than five workable items for the first sprint.

Sprint planning

After the sprint, you will learn about the team's capacity. The amount of items you need to have Ready for the next Sprint should be roughly equal to twice the team's velocity. 
Once an Epic ("high level") item is fully groomed and there are not enough Ready items in the backlog, you take the next Epic and break it down. Start with the "Do Now" items and pull from "Defer" or the "Quick Wins", depending on what currently makes more sense.

Iterating

Impact and Urgency are subject to change. For instance, in May, the annual fiscal report is not urgent - in December, that's an entirely different story! Or, you have a customer threatening with a lawsuit - that may make an otherwise insignificant topic very urgent.

Whenever an item is added to / removed from the Epic Backlog, priorities may change. Between each sprint, you should bother taking a couple of minutes to do an iteration of weeding to make sure you are properly set up for the next sprint. 
Consequently, stories that were groomed on a detailed level may move to the "Scrap" section. That is okay. If the current product priorities indicate that this makes sense - so be it!
Don't bother tracking backlog that nobody will work on in the near future.

A good indicator of when you should weed:
You have items in the backlog that were not touched for more than 3 Sprints, and nobody has bothered asking about them.


Use frequent weeding to keep a significant, concise backlog.

Saturday, August 8, 2015

What "Agile testing" is all about

The job of the tester is to find defects. Better testers find more hidden defects.
Developers get the specs wrong and don't work properly. Hence, the QA department to expose their faulty craftsmanship before it hits the customer. 
Obvious.

Or - so I though. As do many testers coming from a traditional background.

That's not how it has to be like.

Before we dig into the "What" and "How" of agile testing, let's discuss some aspects of agility first briefly.

In principle, we don't need testers to look for defects

The Agile Manifesto isn't just four sentences about values to fit onto a Powerpoint slide. There are some principles behind the values as well. And those principles, combined, form a foundation for agile craftsmanship.

Trust
Let's take a peek at principle #5: "Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done."

This is not about blind trust. It is about a well-informed decision to be able to trust. Good developers don't need to me micromanaged and have someone scrutinously check whether they did their job well. Yes, they, too, are human - but they should be enabled to get things done properly, not checked on. If your developers make mistakes all the time and simply can't get anything right despite of how much support you give them, you just hired the wrong people!

As such, it can not be the responsibility of testers to check on developers!

Excellence
Principle #8 claims: "Continuous attention to technical excellence and good design enhances agility."

Technical excellence implies not repeating mistakes and proactively looking for ways to prevent them. Developers who exercise a high level of diligence will not let any mistake slip that will be easy to catch. Retroactively testing functionality is not going to reveal anything new, because it should be taken for granted.

As such, the classic functional test to report defects should not add any value!
(If it does, that would be a topic for Principle #12, Retrospectives.)


Why we still need agile testers

So, the tester job is not about finding defects - then what is it about?
Let us observe software from a high level.
There are four key activities in software development, regardless of which implementation model we follow:
  1. Determining why to build it
  2. Determining what to build
  3. Determining how to build it
  4. Delivering the built product to customers

We can turn this into a cycle, because once customers use the software, they will most likely have wishes on what else they want. In case they do not utter this wish, we need to figure out where to go next.

With each of these activities, the agile tester can add value. But let us start with the traditional activity, assuming a feature to build has already been determined.

Testing what to build
With Behaviour Driven Development (BDD) or Acceptance Test Driven Development (ATDD), developers can make sure that the product they build does what customers want or need.
Someone needs to define these behaviours and turn them into executable tests. The creation of proper tests which minimize the risk of failure is a routine activity for professional testers.
The Behavioural test suite provides the framework for developers in which they develop.
Using software like Gherkin, Fitnesse, Geb and Spock - or their likes will ensure that no software with any functional defect will ever be released to the customer ... asserting the tests are properly written!

(As a witty side note: If the tests are not properly written, then a classic "post hoc" test will not fare any better.)


Testing how to build

That's the bread and butter of TDD - in general, developers need tests to ensure that their methods are robust and do what they are supposed to do. Oftentimes, trivial problems like Null Pointers are overlooked and cause serious malfunction. Testers pairing with developers can increase the robustness of unit tests tremendously, ensuring we have a better product that is easier to understand, refactor, debug etc.

Testing the delivery

In the domain of exploratory testing, we try to understand what the product actually does when it's built. From this can come "D'oh" effects as well as ideas for other uses of the product. Testers are usually skilled at systematically and effectively exposing the little side effects nobody thought of. 

Testing why to build something
The team  might want to look into data on whether users are actually using the product as expected - and adapt their test strategy accordingly. The fundamental question "What do our users actually do with the product?" can not be answered on a code level. Also, it exposes what is actually happening out there in the real world: A-B tests, sampling plans etc. are all test domain - a great value a tester can add to the product!



Summary

Agile testing doesn't have the same purpose as traditional testing. Ignoring testing will result in a failed product, but the test is not about exposing defects or making developers look like dunces. It is about maximizing the value and sustainability of the product build.



Thursday, July 2, 2015

Reasons for not scaling Scrum

Ever since Scrum has become a staple "best practice" in software development, people have asked this same question: "How can we scale Scrum in our organization?"

Yes, Scrum can be successfully scaled. Yes, there are frameworks out there like Scrum-of-Scrums, SaFE,  LeSS. And there are companies which built up their very own framework, like Spotify.
Some work better, some not so well. But which one works for you?

Let me be a party pooper: None will - if you implement them as a prescriptive set of rules to deal with your organization.

Without being preposterous, I dare say that many organizations looking for a "Scaled Scrum solution" are asking for a solution without understanding what their problem actually is. And they become susceptible to the old adage, "A fool with a tool is still a fool".

 Here are some reasons why you should not scale Scrum in the first place:

Scrum Ceremonies break
  • The Backlog grooming is an opportunity for the team to clarify what they need to know in order to implement a story. But how does the team clarify things where they are partially responsible - do they clarify together with those who are responsible for other parts? It is better to suffer the waste of receiving unnecessary information or to suffer potential lack of disinformation. So, you induce mandatory waste into the process.
  • The Sprint Planning is the point where the team decides what they will work on during the next iteration. But since everything on the top of the backlog is high value high priority, scaling organizations also need to answer the "Which team does what - and why that team and not another?". It is an entirely new level of Sprint planning which is non-existant in a small scale implementation - and it is also inherent waste.
  • After the Daily Scrum, you still have no idea what is going on and where the real problems are ... the main purposes: synchronizing and focussing, are not met - because you need to synchronize with people outside your team and others may focus on things you don't even care about! But how do you know - without yet another meeting where no working software is produced?
  • The Sprint Review may not be between teams and customers - it often becomes an internal festival where developers show results to each other: Customer centricity becomes a tech talk. Maybe you need an "IT review" and a "Customer Review"?
  • The Sprint Retrospective is specifically intended a place for the team to be amongst themselves. But what if the problem is bigger than one team? Who solves problems that are inherent to the organization, but not visible to any team? Who will address the elephant in the room?
Scrum Roles break
  • The Product Owner should be steering the team to deliver maximum value. Once this turns into multiple teams, the availability of the PO to the team becomes an issue.  Antipatterns like the "absent PO" or "disempowered PO" become commonplace.  Manytimes, PO Consortiums or a PO line hierarchy arise. The problems between separating accountability and responsibility are obvious - they lead to bureaucracy and CYA - both dangerous to the product and the organization itself!
  • The Scrum Master can't work like before.  All of a sudden, it's not about a team's impediments, but about the teams' impediments, so the conflict of interest "Our team vs. Greater Good", which the role was intended to resolve, actually becomes their main problem.
  •  The Developers also can't work like before. In the past, they were empowered to do whatever they needed to do in order to make a better product, as long as the team approves. Well - that ends when other teams are working on the same product. They need to align not only within the team, but also across team boundaries, losing the essential freedom to maximize value while minimizing cost and risk.
Scrum artefacts break

  •  The Product Backlog contains enough stories for the team to work on. However, it should always stay at a manageable size. Is 50 a manageable size? How about 100 or 200? Well, probably not. However, if you want properly sliced stories for 5 teams and each team can do 10 stories per sprint, you'll be ending up with the need to have roughly 150 properly sliced high priority stories in the backlog. Tell me what is important when you have 100 "high priority" items on the radar!
  • The Scrum Board breaks, because one team's doesn't really tell us what is going on. You need to look at all teams' Scrum boards to get a grip. Unfortunately, the boards only tell a rudimentary tale. The real story is told in the Daily - and since you're not attending all the dailies, you can't understand what the boards are meant to convey.
  •  The Velocity breaks, because each team has their own velocity. Either, teams will align estimates across teams or teams will have individual estimates on their own stories. In the prior case, teams have to abandon their former way of estimating. Previous estimates become an invalid basis for comparison.
  • Release Planning breaks if velocity is broken because teams use individual estimates, the  PO will get completely confused because all of a sudden, Story Points  become team-dependent and not a valid capacity metric.

Communication breaks
Yeah, I know for many organizations, the Agile Manifesto is just a boring mandatory Power-Point slide at the beginning of the Scrum Training. But remember the first value? People and interactions over processes and tools.
  • Bigger organizations sooner or later become so big that you don't know who knows or does what.
  • When developers don't know who can help them on a specific issue, they are likely to implement a sub-optimal solution.   
  • More people always means more communication. More communication means less time for implementing solutions.

  • Chances are when communication time exceeds coding time, you would be more effective with less people.
  • Reducing communication is not a solution: Keeping people ignorant or introducing Chinese Whispers will not make your product better.

Product Vision breaks
  • As single Product Owner, you must be a superior, exceptional leader to align a scaling amount of stakeholders who all have their own objectives on your vision. If you are up to the task, maybe Presidency of a government would be more suitable than scaling a Scrum organization?
  • If you work with a Product Owner Consortium, please take a minute and have every Product Owner explain their main objective to a neutral observer, then ask the neutral observer to describe the Product. Be prepared for a surprise!
Continuous Improvement breaks
  • It's already not easy to fix something broken you have full control over.
  • Now, try fixing something which is broken for you, but works for another team!
  • Next, try fixing something which is broken for someone else, that affects what you do, but has been established by yet another team - without making anything worse for anyone!
Team autonomy breaks
  • When teams lose the flexibility to fix their own problems, they slow down, morale and output drops.
  • When teams lose control over their own work, being dragged into processes they do not want or need but that others want or need, they slow down, morale and output drops.
  • When new people get onboarded, how will you deal with the team structure? You can't simply add new people to teams all the time, because a team of 20 doesn't work. 9 is already an overweight. You also can't rip teams apart, that will devastate productivity and morale. You also can't just put new people into a new team, because they don't get familiar with the existing Working Agreements unless they work in an existing team. So, any new person you add results in un-answered organizational questions.
     

Summary

I am not saying you can't scale Scrum. However, you should resist the temptation of scaling just because you think you should - chances are, you can't and don't know it yet. The consequences of scaling are severe and the cost of scaling may far outweigh the benefits.

It may be necessary to scale, but it is significantly better to avoid and postpone scaling as long as you can. Once it becomes inevitable, you need to be aware of the detrimental impacts - and deal with them before they destroy your scaled organization!

A LeSS course is a good way to get input how to solve the most common problems of scaling. But you can't take LeSS blindly and hope it will solve your problems for you. You still need to find out what works for you!


Thursday, May 7, 2015

Agile defect management strategy

"We don't have any defects, so we don't need a defect management strategy".
If you are in that situation, you're fooling yourself. Good teams are not those who don't make any mistakes (that's an illusion!) - they catch their mistakes before any harm is done.
And for that, they have a defect management strategy in place. And that strategy must be continuously inspected and adapted.

The following is a high level outline for an initial agile defect management strategy:

Define: What is a defect?

There's nothing more cumbersome than the discussion whether something is a defect, "Works as designed" or a Change Request. And that discussion does not solve the problem at hand. It's a red herring. You do not want to decide this on a case-by-case basis.
  Align the team and all stakeholders on what you call a "defect" and how to define it. You are free to inspect+adapt the identification criteria frequently. But don't make ad-hoc decisions after getting enraged about individual reports.

How will you track defects?

It doesn't really matter whether you decide to use Post-It's, your Backlog tracking tool or a dedicated application lifecycle management suite. What matters is that you use appropriate means and everyone agrees on the approach.
Issues to consider are: Who needs the information, what will they do with it?

Separate: What is not a defect?

At least as important as the definition of defects is a clear definition of what will not be handled as a defect. This doesn't need to be very formal, specification by example does the job quite nicely.  Typically, issues that are not defects become stories in the Product Backlog and are at the whim of the Product Owner.    
 Some product owners will claim that "as long as it doesn't affect revenue, it's not a defect". We will not discuss here whether that is a good approach. It is merely one of many possible decisions in a project.
 Again, align the team and all stakeholders on what is not a defect and where the dividing line is.

Handle: How will you deal with defects?

It may be obvious to some, but it's not bad to have an open discussion. Depending on the formality and scope of the project, you may want to discuss topics such as severity, priority, SLA's etc. 
Furthermore, you should discuss how you will ensure that the issue is gone once and for all.

Improve: How will you deal with non-defects?

Just because it's a bug doesn't mean the software is doing great. Some "non-defects" are actually improvement suggestions. Others are distractions from the product vision. Yet others are simply a waste of time. While you will need to decide the next steps on a case-by-case basis, you should have a clear strategy on the next steps. After all, your purpose is not to clear the defect list, but to provide a great product that excites customers.

Flakiness: What if you don't know?

Bugs may also be flaky. You may need specific approaches for dealing with the "Bugfoot" (someone is adamant to have seen it, but there is no proof) or the "Heisenbug" (the nature of the bug changes by observing it). The worst thing you can do is sink near-infinite effort into analysis, but you also don't want to disappoint your customers. Claim the common ground.


Conclusion

Invest a bit of time to create a clear defect management strategy. Inspect and adapt your approach whenever necessary. Stay lean and pragmatic. Align.
This will make your agile project much more successful.


True Collaboration over Daily Standups

Daily Standups go a great length in synchronizing and building teams on a day-to-day basis. Often, they make the difference between a loosely coupled group of developers working on related topics and actually working towards common goals.
Yet, there are numerous problems habitually associated with them.

The nature of the Daily Standup

Following textbook definition, we would have all team members huddle in front of the Scrum Board, then each team member shares "What have I done yesterday, what will I do today, what impedits me?". The main purpose is to synchronize one another, but also to build mutual accountability.
Any stakeholder who is interested may participate in Daily Standups as a silent bystander to get a feeling of what is going on: "Status Meetings" and "Status Reports" are a thing of the past.
Of course, this is also part of their value. However, their biggest value is barely hinted: Finding opportunities for improving the collaboration.

What happens when you don't have Daily Standups

Without daily standups, there is a huge risk that team members run into different directions, providing code that does not turn into a maximum value solution at the end of the sprint. One worst-case scenario: people work in parallel on incompatible solutions to the same problem - and find out when the customer complains.
You can't be a team unless you share troubles, solutions, failures and success.


Dealing with Problems

It's natural to face challenges in the implementation of software. This-or-that framework doesn't behave as expected, here or there a configuration item is causing trouble, an infrastructure component is required - and whatever. Technical experts are keen on solving their own problems, because after all, that's what makes them experts.
Oftentimes, there is someone else on the team who has experienced the same type of problem before. Becoming aware of someone else's problem, they would gladly extend a helping hand. But how do they become aware unless someone is sharing their troubles?

Tangents

The Sprint defines a clear scope and time box. Sometimes, it is very interesting - or academically challenging - to not only work on the task at hand, but to derive a related task. For example - we need to do a site index. Because it's both related and my personal favourite, I also implement a fuzzy search. Cool, but it will cost me 5 days of the Sprint - and nobody asked for that.
The shared accountability in the Daily Standup encourages the team to focus on the common objectives and provides an opportunity for team members to say "That's good, but we need you to do something else."

Status Reports

Transparency makes agile implementations strong. Lack of transparency causes problems. Managament's first reaction when they don't know what is going on: Forced status reporting. The first stage is boring, hour-long status meetings that become a mixture of sleeping pill and justification rally. In the second stage, they become written, formal and tracked - merely to satisfy the internal need of being informed. Usually, both methods are implemented simultaneously, in cumulation burning a good 20 hours of productivity per week.
Not many developers like writing formal status reports, so daily standups actually increase morale.

Why having Daily Standups is bad

After the previously mentioned topics, who could argue that Daily standups are actually bad?
Let's look in detail what Daily Standups actually are. They are a predefined standard process executed in a rhythm. As such, they induce waste.

No meaningful update

"Yesterday, I was working on the search engine. Today, I will continue working on the search engine. Nothing impedits me." - great. Now, how much did that help my team? The good news: Like this, the standup only takes 5 minutes. The bad news: We're actually wasting 5 minutes of each team member each day.
Numerous approaches, such as "Improving the 3 questions", have been implemented in different organizations. None of these solve the fundamental problem, they merely try to expose the symptom.
If it's something that takes more than 1 day, why has it not been sliced further - why is nobody else involved, what has actually changed since yesterday - and is there a way for other team members to join me so that we can deliver faster, a better solution etc.?
I might bring up even more questions: If I've been working on it for a full day already and plan to work more - who has done the testing? Is anyone doing code reviews? I should be rallying support for my topic rather than just informing what I do. Or is my task not important enough for the team to contribute?

Hiding the Elephant in the Room

Have you ever seen how everyone focuses on their stories, their responsibilities in a Daily? Yes, everyone can align that on the Sprint Goal. But is that sufficient? There is probably a prioritization issue when 3 or more stories are Work in Progress, because nobody seems to ask the intrinsic question "Which one is really the most important of these?" - probably the team is more focused on delivering quantity than getting the top priority done!
Many teams justify this behaviour by stating: "But Tim is the only Database expert, so he's the only one who can do that story!". A measly excuse, really - because they have a bottleneck and aren't even working to resolve that. In fact, the team subconsciously knows they have a sustainability and quality issue: If Tim gets sick or goes on vacation, the team may be completely blocked. All the stress is on one shoulder!
So, the real question we should be asking is not: "Which is the next story for Tim?", but "How can we collaborate on this one, so that the next Database story doesn't rely solely on Tim?"

Value Delay

Why should I wait for the Daily Standup to rally collaborators? I should do it immediately. If I need to get some exploratory testing done - why should I wait until the next stand-up to find a team member who has some time? 
Again, this may come down to priorities. If I am doing is the most valuable thing the team can do at this time - why am I the only one? If I am not doing that - why am I not working on the most valuable thing? Why should I wait until the next Daily to deliver most value?

Restating the obvious

The worst stand-up I have ever participated in was actually in the most efficient team. We did something called "Mob Programming". The team was great, we were flying through the backlog at a rapid pace. Everyone was constantly and intensely involved. But our daily standups were terrible: "Yesterday, we completed the stories A,B,C. Today, we will work on the top priorities in the following order..." Only one person spoke, because - everyone already knew what we were doing, why we were doing it and how they would contribute! Everyone was highly focused. Nobody was doing anything outside the agenda.
The stand-up wasn't adding any value because everyone knew everything before we huddled!

Is this inherent to the Stand-Up?

All of the above can be solved by actually improving the nature of the stand-up. But many times, standups are not like that. Why? Because people are not looking for collaboration, they are merely fulfilling their responsibility. Let's be frank: If you aren't looking to collaborate, your Daily Standup is a farce!

Summary

Daily Standups are great. If people are working on different tasks, the Standup provides an opportunity to resynchronize, harness synergy and refocus.
You should improve your Daily Standup to the point where they become a magnet for collaboration and a catalyst for value creation. Once they fit that purpose, you should look for different places to improve.
Daily Standups are an artifact in themselves and provide no bottom line value. They assist in obtaining value - no more, no less. As the level of collaboration among teams improves, the amount of actually new and relevant information would decrease.  In an ideal team, collaboration would be an ongoing, permanent condition and the Daily Standup would only interrupt the ongoing collaboration to fulfil a non-value adding ceremony.
At that time, Daily Standups turn into waste. Few teams ever reach this state.

Improve your standup to be valuable, but remember that the vision is great collaboration, not great Standups!


Friday, April 10, 2015

Continuous Flow over Sprint Planning

The biggest gripe agilists have with "traditional Waterfall" projects is "Big Upfront Design", or, in short: BUFD. Many detailed design documents are full of feature requests that were just put in "in case we might need it". When the specification goes into development, there's too many trees to see the forest and people lose perspective of what is really important because a contract has been agreed to deliver everything. As a consequence, when time runs short, the customer oftentimes ends up with a half-baked software that has lots of fluff but lacks essential features. As the Standish Group Chaos Report states, the consequence is often colossal failure.

The value of Sprint Planning

Sprint Planning is an important discipline of Scrum. The main outcome of Sprint Planning is setting a short-term objective for the next sprint. The team and stakeholders agree on what will be covered, why - and in which order. On the positive side, the information gap between development and business is minimized. On the negative side, unrealistic expectations and bad ideas are weeded out before any effort is wasted.
Many teams use "Planning Poker" and "Story Points" to determine how much they are capable of delivering. 


What happens when you don't do Sprint Planning

Maybe 100 years ago it was possible to go to Klondike with nothing but a pickaxe and make a fortune. However, that was unreliable and many found their demise rather than wealth and glory. Those who want to succeed lay out a plan, stake the best claims - and replan early when their assumptions turn out wrong. And this is what we should be doing in agile as well.

We use the Sprint Planning to determine where the value lies and where it's worth to lay a claim and mine the gold. Continuing in this figure, we determine how rough the terrain will be, how deep we will dig and what tools we'll use to bring forth the wealth.


Sprint plannings are sometimes considered a waste by both developers and stakeholders. Yes, they do consume time. However, as the old adage goes, "Failing to plan is planning to fail". We can make this short: If you don't have a plan, you'll just deliver some garbage. Customer unhappy, developers unhappy, project ends.

Everyone wants to deliver high quality working software with maximum value. It makes developers proud for a job well done, customers happy to work with the product - and business happy because money is received.

Why having Sprint Planning is bad

Disclaimer: This is about "Sprint Planning", not about planning.


Commitment versus Intention

Many immature organizations consider Sprint Planning as a way of retaining some illusion of control. Developers are asked to commit to completing the Sprint's stories and held accountable to meeting their quota. Note that this is a misinterpretation of the intention: The team commits to what they will be working on in terms of direction and focus, not in terms of quantity!

Committing to a plan

The Agile Manifesto is quite clear on valuing "Responding to change over following a plan". CSM courses teach that the Sprint Plan is iron-clad: Within the time box, we do not tolerate change. To weasel out of this corset, we create back doors. The most prominent are buffers and a sprint termination process.
It's easy for the team - or Scrum Master - to stand up and say "You want a change? We can make a change. But we'll terminate the sprint, discard the efforts up until today and restart." This creates an atmosphere of fear. Stakeholders become afraid and developers unwilling to make "just in time" changes.

A sample perhaps? At the beginning of the Sprint, we said we need a button "More details" on the product display. While implementing it, we realize it won't deliver value: We need a "Buy Me" button instead!
Who will muster up the courage to terminate the entire Sprint for a useless button? Nobody. And so, we let the developers waste time to build a worthless feature and put another story into the product backlog so that in maybe 3 week's time we get a valuable feature that could be available in a day's work.

And what happens next? In the Review, developers proudly demo the worthless function to a disgruntled marketing department who aren't even interested any more and end users who ask "Why would I want this?"
 
Wouldn't it be significantly better to have a mechanism "I realize we don't need X, let's just discard it and do Y instead"?

Yelling "Scope Creep!"

Sprint planning has the intention of determining the scope of development in a sense that developers are aware what defines "success" for this sprint. Many teams will - unfortunately - go as far as to deliver exactly the committed Acceptance Criteria and nothing else.
They may not account that users, just like they themselves, are often uncertain on what the final product should look like. Yes, it's good to have a clear picture of the outcome before starting.

Let's be picturesque again.

When you go to an interior architect, they often create a fully explorable 3d model of your future living room before ordering a single tile and let you review and adjust the placement of every single object in real time.
What you may just be doing is asking the customer to come up with a fully explorable 3d model of their future living room - not giving them the opportunity to review and adjust the placement of any single object for a full sprint. And much less would you give them the opportunity to ask "While we're putting up this deco fountain, would an ivy plant would look nice here?". And even less would you suggest it by yourself, because hey, that's Scope Creep!

Again, this is not a flaw of Scrum or Sprint Planning. It is a flaw of how Planning is oftentimes practiced.

Even if planning not used to keep the customer out of the development process, it is better to make just-in-time adjustments than to defer the delivery of visible value.

Boredom

Sprint plannings are bulk sessions. All topics relevant in the sprint are bundled up together and in a meeting marathon that may well take a couple hours, one topic after the other is covered.
The worst part of this is that as the meeting progresses, interest wanes. Obviously, a well groomed backlog would put the lower-valued topics later in the plan. However, even the last item of the backlog is still the most valuable item the team can work on from all topics to follow.
Team members and stakeholders will not be discussing and estimating the 7th user story in a row with the same level of attention and fervour as they discussed the 1st story. As such, the risk of lapsing ideas increases as planning progresses.

It is good to have a clear plan for the sprint, but it is better to not overburden anyone and keep the discussion fresh and vital.

Hiding behind a curtain

We planned 4 stories, we delivered 4 stories. Everything is good.
Is it? One story was 2 man-days, another was 15. It should have been an Epic, sliced into multiple bite-sized portions. The timebox may become a curtain veiling the true complexity of individual stories. Stakeholders may lose the valuable opportunity to refine the "MVP" of a story as it starts to become unexpectedly complex during a sprint. 

Planning provides a direction. It should not replace the opportunity to re-negotiate every single story during the implementation phase. The value of software development can be multiplied by a magnitude when during the development of each line of code, Inspect+Adapt is applied. This may well include scrapping and reworking committed Acceptance Criteria mid-implementation.

It is good to make a plan with the stakeholders before getting to work. It is better to refine this plan every single day even while completing the work.

Overproduction

Two of the Seven Deadly Wastes identified in Lean is "Overproduction" and "Storage". This is what planning often does.
The team does not know for certain how much they can deliver, but they don't want to be idle, either.
So, sprint planning typically covers 1-2 stories more than the team will actually scope. This may be "pull forward". Given the nature of agile projects, not only the pull-forward, but also the planned scope may not be reached. By the time the sprint ends, objectives may have changed and none of the unfinished stories become part of the next sprint's plan.

Congratulations, you've created a storehouse for undone work and have produced plans that may never come to fruition. Not very Lean.

It is good to have enough items in the backlog to optimize value delivery. It is better to have a mindset in the team that not everything we plan is a commitment.
But it is even yet better to not plan anything that you won't be doing.

By producing a continuous flow of stories "just in time", we prevent the terrible waste of planning and storing story details we will never use.


Summary

There is a reason why "Responding to Change" is considered more valuable than "Following a plan".
Not making a plan is a terrible idea and will devastate the product. But it is better to defer any detailed planning to the latest point in time where it makes sense. The best point in time is "When we are doing it". This will not only be within the sprint - it may be even after we started realizing the story!

Do not use Sprint Planning to lock out the golden opportunity to deliver maximum value.
Use a continuous flow of stories to inspect and adapt whenever it makes sense.





Tuesday, March 31, 2015

Continuous Deployment over Sprint Reviews

Waterfall required a big leap of faith (and often, a suicide leap) for business stakeholders: Open your wallet, wait a couple years and pray that you find the results useful. In Scrum, we make things more tangible: Every couple weeks,  stakeholders and developers gather and look at working software, discussing the good, bad and the ugly. The best part? Everything that is "Done" is already an asset, even long before the project's delivery date.

The value of Reviews

Truth be told, stakeholders often don't know what they really need unless they have something they can look at and fiddle with. Let's humour for one second the thought that a comprehensive specification and UML diagrams exist. 
Once the first couple lines of code are written, maybe just a GUI without any functionality - maybe just a few algorithms without a database connection ... the relevant users can take a look and decide whether that's workable, where the weak spots are and what could be done to make the construct more useful.
None of this is possible by looking at complex specification documents, because everything a document review does is add more levels of detail, not innovating new, efficient ways of doing things better.

What happens when you don't have Reviews

Or, alternatively, "What happens when stakeholders don't attend reviews, don't give feedback during reviews - or developers show funny powerpoint slides instead of working software".

In the Review, developers and business stakeholders re-sync mutual expectations and understanding. The next sprint can be planned more efficiently with the knowledge where the most value lies - from here.

Not having frequent, good reviews means that developers continue working with assumptions rather than with users. It also means that the most powerful Inspect+Adapt mechanism is taken from the team. Obvious faults in concepts will not be discovered until someone fiddles with the software or validates the system against their understanding of business.


Why having Reviews is bad

There is nothing intrinsically bad about Reviews. But there is something bad about having a set date, maybe once every 2 weeks, where Reviews are conducted.

Slow Feedback

Reviews are definitely an improvement towards simply "throwing stuff over the fence" because they enable feedback. Unfortunately, they encourage developers to move stories into "Done" without checking whether users actually like the result. Just asking you personally: Are you proud to call something "done" without knowing whether the person you're doing it for will like it? Ideally, we receive feedback even while developing the feature, ideally so fast that we can actually respond to change. Maybe there's no need to continue developing after a certain point, maybe we need to do things completely differently than we initially assumed.

Batch processing

The biggest problem of Reviews is the batch processing. If you got 5 Stories are on the sprint, 4 will be done long before the Review, and stakeholders can't see them until the timebox is over. This turns sour when these stories build upon one another and the first story already had a fundamental flaw: The entire sprint may need to be discarded.

Induced waste

This means we have an interval of a couple days where developers already start forgetting the "Why" and "How" before they have to recoup everything to produce a demo. Sometimes, they counter this by writing presentation notes upfront, which do not provide any value to the customer.
Worse than this - if different stakeholders attend in order to review different stories, their time is wasted. Marketing doesn't care about this fancy new form for sales - and sales doesn't care about the new campaign algorithm. But both patiently sit through the non-value adding time.

Lost value

The first item in the sprint backlog is the most valuable item of the sprint. So, it gets done first. That's common sense. However, many teams don't deploy to Production until the PO accepted the sprint's "Done" items in the Review. 
This means that from the time the most valuable item is done until the Sprint Review is completed, we have the most valuable deliverable rotting in the repo, producing zero value!

Summary

It is definitely good to have Reviews and a serious improvement over not having them. Reserving (half) an hour every 2 weeks to ensure you deliver high value, remain on the right track and have happy stakeholders is definitely a good idea.
But it is better to receive instant feedback on every work item completed. 
Continuous Deployment is a rigorous, highly disciplined approach to move completed features directly to where you receive this feedback - to the real users. You won't need to wait for a specific future date, you will instantly know how users respond. And from a business perspective: you don't keep the dollars on the shelf, you reap them!





Monday, March 23, 2015

Continuous Improvement over Retrospectives

The Retrospective is Scrum's most powerful tool for sustaining and improving productivity. After each sprint, the team gathers the most critical pain points and agrees on a few specific ones that should be dealt with during the next sprint. Success is tracked from sprint to sprint.

The value of Retrospectives

Harnessing the power of retrospectives, the organization gains transparency on why developers are impedited and through the improvement activity, productivity is brought to whole new levels. 
In organizations strongly embracing change, it is not rare for newly formed Scrum teams to see a productivity boost of well over 200% within a couple of months.
Yes, you read right: Retrospectives are the key to hyperproductivity.

What happens when you don't have Retrospectives?

Well, that one is easy to answer. The team gets into a treadmill of delivering feature after feature, sprint after sprint, release after release, year after year. The same old problems linger, get persistently ignored and nothing changes until someone and disrupts the process.
The most common kind of disruption here is that developers simply get fed up and leave. The next type of disruption is that the next "powerful methodology" comes up and because it promises so much better results, Scrum gets abandoned. Those are still okay. But what if the disruption is that a competitor throws out a much better product, delivering much faster and cheaper - and takes over your customers?

In Waterfall projects, people conducted either "Lessons Learned" after each major Release - or "Post Mortem" after each failed project (to the point that Post Mortem became the Standard project closure process). But hardly ever was anything done about it.
Retrospectives try to change this behaviour by frequently inspecting and adapting the process. You fix what's broken rather than cleaning up the shambles.

So - why should ne not have can anyone say Retrospectives are bad? Well, they are not "bad". But they are a suboptimal state. Consider the following a stimulation of thought rather than bashing.

Why having Retrospectives is bad

Deferral of thought

Oftentimes, team members purposefully do not think about improvement until the Retro, because the Retro is the time to bring up issues. This is not an inherent problem of Retros, but of mindset. It is absolutely false to consider that we should not think about Continuous Improvement when we are outside the Retro meeting.

Deferral of solution

In each Retrospective, the team will agree on the most pressing 1-3 issues, so that productivity is not replaced with internal improvement. Unfortunately, this kind of thinking is absurd: If the team has too many issues, it may simply be better to solve them all at once before loading up yet more technical debt.
If there are pressing issues that damage productivity more than their resolution costs, why should you defer the solution for a couple sprints?

Absolution for suboptimal work

One of the major outcomes of Retrospectives are adjustments to the Definition of Done. On the flip side, the DoD is immutable for the duration of a sprint. However, if team members are already aware that their DoD is suboptimal, they should not adhere to inefficient standards but rather do what common sense dictates: Huddle, resolve, continue efficiently.

Thinking within the box

Retrospectives are team-centered. Purposefully, external stakeholders are NOT invited, as the Scrum Primer states. Is it right to consider all problems to be local when everyone knows they are not? Maybe, the team can't even see a problem until an external stakeholder escalates and then there's an emergency: Sprint termination, crisis meetings and general panic. Is this necessary?
Why would you give the team a forum to speak their entire mind - but not to the customer? Is their opinion worth less than that of the developers? Especially if they could help with the solution! So - how will the team harness the improvement potential that others could offer?

Problem thinking

A key element of Continuous Improvement is problem solving. But it isn't the only element. This is not intrinsic to Retros, but to the practice implemented in many teams. When do they have time to think about mastering their expertise, turning Good to Great? As long as there are pressing issues, solving a problem may defer the strife for excellence. However, it may be better at becoming exceptional in one area than becoming okay in all areas.

Summary

It is good to have Retrospectives. If your reason for not conducting Retrospectives is that "We don't find anything to improve" or "We can't accomplish anything anyways" - that's your #1 pressing problem and reason to call an emergency Retrospective.
On the other hand, Retrospectives are on the same line as any other Scrum event: It is good to have them, but it is better to work in flow: As soon as something comes up, harness it. Do not batch up, do not defer. Involve stakeholders, engage customers. 
Everyone should be on a continuous watch for improvement potential and should jump at every opportunity to produce a positive bottom line for the team, the company, the product - and the customer.

Friday, March 20, 2015

Taking Initiative over dedicated Scrum Masters

The second value of the Post-Scrum Manifesto states "Taking Initiative over dedicated Scrum Masters".
To understand what this means, we must briefly glance at why we even have Scrum Masters in Scrum.
 

The role of the Scrum Master

Who is the Scrum Master anyways and what do they do? A small (yet in Scrum trainings, most emphasized) portion of their responsibility is keeping the rigour of Scrum, reminding stakeholders and team members of their role, facilitating the events and getting impediments solved.
Probably the biggest responsibility is moulding the developers into a team, creating a powerful workforce able to cooperatively turn user stories into valuable, working software - by any means necessary.

And with that, the Scrum Master's main responsibility is: to become superfluous.

What happens when you don't have a Scrum Master

For example, teams who are not aware of the Tuckman model may never reach a superior performance: Being busy fighting fires day-in, day-out may prevent progression. A Scrum Master should be aware of this situation and free the team to pass through the Storm.

Initial teams often abandon "pure Scrum" very quickly: Daily Standups turn into detail discussions or get skipped altogether, Review Meetings become Tech Talks, Retrospectives become venting sessions. The Scrum Master guides teams to be more effective, but also coaches members on why these events are important and what their intended outcome is - and when they are missing the mark.

For many organizations transitioning to Scrum this statement is (should be) applicable: The Scrum Master is empowered to resolve impediments outside the team's sphere of control. They may freely move beyond team boundaries to make the team more effective. Most of all, not being a developer, they have time to do this.

And, let's not forget the most important aspect of becoming agile: It does not suffice - or, to put it more bluntly: It does not work - to have "a bunch of nerds in IT doing Scrum". It may help a bit when things are in utter chaos. To be effective, the organization must embrace agility. The Scrum Master should evangelize management, stakeholders on the benefits and responsibilities of working in an agile context and win other developers who are currently working in less effective structures for the new, improved approach.


Why having a Scrum Master on the team is bad

Let us start with the most important responsibility of the Scrum Master:

Orchestrated Agile transformation

Disclaimer: There is nothing, absolutely nothing wrong with bringing experience on board.  When everybody looks to one person "So, how should we do Scrum?" - that's a terminal disease: Failure and success of Scrum would hinge on one person. However, Scrum is a team discipline, so that's counterproductive. Deming realized a good 30 years ago in his 14 Principles that "the transformation is everybody's job!".  As such, everybody should be leading the Agile transformation.

Lack of impediment resolution

It is good if there is someone on the team who can assist developers in removing their impediments. However, there is already an organizational issue when people are impedited and unable to resolve this by themselves. Having a mindset of "I can not resolve this by myself" is a killer to motivation and productivity. As such, every team member should feel both responsible and empowered to remove their own impediments.

Ineffective Events

Events, such as the standup are not for Scrum. Much less are they for the Scrum Master. As long as team members rely on the Scrum Master to facilitate events - at worst: reporting to the SM -  they do not reap the full benefit. Each event serves a team purpose, so it should not even be necessary for the SM to organize or attend these: The team should drive every events that is valuable to them! 
For this, every team member should understand the value they obtain from a event - the SM can't do that for them!

Fixing symptoms, not causes

Each time the SM intervenes against outside interference to keep the team uninterrupted, that is a fix to the symptom. The root cause is that someone doesn't understand what they are doing. It is good to have a pitbull biting anyone who unknowingly damages productivity. It is better if there is nobody who does that.

Wrong source of improvement

The team members may justified have trust in the SM's ability to improve, but every activity introduced by the SM does not originate from the team, which means that they did not have the mindset to identify the problem or come up with the solution. The very fact that the SM adds value to the team is a shortcoming of the team. In a truly self-sufficient team, an SM can not add value.

Driven evangelization

It is good when the SM can evangelize others in the agile agenda. It would be better if the team's results would inspire others to participate. As the old adage goes "Preach by any means necessary. If inevitable, use words." - the SM can assist as an information radiator, ideally the team is already a shining beacon. The SM can (should) not be busy establishing disciplines like software craftsmanship, refining backlogs etc. in stead of the team. The more they involve in these activities, the less mature the team is.

Summary

Anything that the SM does for the team indicates a dysfunction in the team. There is no excuse for anyone on the team to not be doing what the SM does. A good SM is the key ingredient of a highly successful team. A great SM will make themselves superfluous.


Tuesday, February 24, 2015

Becoming great at answering the wrong questions

"The right answer depends on the question"

This proverb has long guided me and I always strive to ask better questions.

This week, I evaluated some devops tools and felt oddly reminded on how similar the effectiveness of DevOps is to a product owner's story writing.

The wrong user story doesn't deliver the business value you're looking for.
Here are some of the highlights for poorly written user stories that I encountered while evaluating tools. Marketing makes these stories look like they are what you'd really want in your DevOps stack. But let's take a closer look.


As a DevOps, I want to see way back in time when certain error messages occurred so that I know how long it has been going on.

Yikes! If there are unknown error messages in the logs which nobody has been taking care of for months on end and nobody realizes - or cares - then either it's not important or your organization is in a mess. You don't need a tool that permits you to discover errors a year back in time, you need a strategy to effectively deal with problems as soon as they occur!
Let me rephrase that story for you: "As a DevOps, I want to be notified immediately when something abnormal happens so I can analyze and resolve the problem before it hits the customer!"

As a DevOps, I want to have a one-click solution which connects stack traces in log files with the corresponding source code segment so that I can analyze the effect on the user easily.

Good luck on that. I personally prefer to have robust, well-tested software, If your software is throwing significant amounts of stacktraces, then the effect on the user isn't your primary problem.
Let me slice this one for you: "As Sysop, I don't want any stack traces in production logs, because I don't want to operate a system that doesn't do what it's supposed to do." - and - "As a Developer, I want a software test that has sufficient path coverage so that I won't run afoul of undefined behaviours during refactoring." - and - "As application user, I expect the software to behave as intended in each and every circumstance so that my business outcome is predictable."


As a Security expert, I want to be notified in real time when user data is compromized.

Sounds great. But what are we really talking about here?  Why do you even care to know that in real time? There are predictable, controllable ways in which user data can be compromized. In this case, your strategy shouldn't be to introduce realtime notification, but prevention. Any minute you're investing in data theft detection would be significantly better invested in hardening your systems.
Let me rephrase that one: "As a data security officer, I want all known security loopholes closed so that we don't even have data security incidents!"


As a DevOps, I need data for every possible failure scenario so that in case of incidents, I have enough data.

Tools will give you a false sense of control when you ask the wrong question. Your question is not to have data for everything, but to eliminate root causes - so that you won't need data.

Your job as a DevOps is not to hoard a boatload of data that can't be humanly understood, your job is to eliminate operative risks within the software so that the essential monitoring data can be reduced to a manageable and comprehensible level.

Conclusion

Tools are not solutions unless you first define the problem.
Your DevOps Strategy will fail unless you first learn to ask the right questions.

Monday, February 2, 2015

Shared Vision over dedicated Product Owners


The first value of the Post-Scrum Manifesto states "Shared Vision over dedicated Product Owners".
To understand what this means, we must briefly glance at why we even have Product Owners in Scrum.

The role of the Product Owner

The primary role of the PO is to communicate the product vision with all stakeholders and maximize product value. When there is a misunderstanding or dispute, the PO facilitates the dialog. Moreover, the PO owns the Product Backlog. They decide what will be part of the product - and what won't. They set the priority and are accountable for making the right calls. Customer facing, the PO is responsible for what the team delivers. Team facing, the PO is responsible for what the customer wants.

What happens when you don't have a Product Owner

Let's just assume we have a textbook Scrum team, but the PO is missing. The first thing that will practically always happen is that stakeholders start bombarding the team with feature requests, everyone considering their own requirements top priority. The team now has the job of juggling all those requests while delivering working software. 
If the team is lacking a person who is capable of calling shots on either-or options, they run a great danger of implementing inferior solutions, dissatisfying the customer. 
On a more positive note, the responsibilities of the PO will typically be picked up somewhere, either through the customer side or the team. Unfortunately, there is no guarantee this will happen before the product failed.

Why having a dedicated Product Owner is bad

Having a dedicated PO communicate stakeholders on the product vision and making tough calls on value choices is good, but it's still a bottleneck. Since being agile is all about never ever maneuvering yourself into a corner, it simply doesn't make sense to enforce a bottleneck.
The need for a dedicated PO actually indicates symptoms for a number of problems:

Different levels of understanding

It's good if someone can explain the product vision to others, it's better if everyone shares it and product vision communication should never be a unidirectional process. 

Single Point of failure 

When someone needs to call shots, it means that this person is can singlehandedly cause an otherwise successful product to go awry. A good PO will never be a dictator, but since the PO will always filter information through their personal viewpoint, unsound decisions may be made. The more decisions PO doesn't need to make because the team has the knowledge and ability to make them, the lower the risk of having a SPOF actually failing.

Also, developers may become paralyzed or make wrong decisions if the PO is unavailable for clarification.

Stakeholders management over consensus

When someone needs to call shots, it means that different stakeholders set different priorities. This may always happen, but is a situation where one person has the final say really better than being able to come to consensus? 

Translate: Business-Developer, Developer-Business

The PO facilitates the dialog between team and customer, especially in situations where business expectations are technically unsound. In theory, this sounds good. In practice, we have an agile value "Customer collaboration over contract negotiation". Nowhere does it say customer collaboration is limited to the PO. If every developer collaborates with customers on a daily basis, the mutual understanding will be there without having a facilitator in the middle.

Assumption of incompetence or apathy

We get taught in Scrum classes that PO's understand the business value and know which increment is best to advance the product's value. Superficially, and in an immature organization, that is true: Oftentimes, developers have no understanding of business value. But that shouldn't be considered a given law of the universe. In Programmer Anarchy, there is a radical shift: Developers not only understand what is the best value for the customer, they are so closely interlinked with customers that they drive: They deliver working software and let the customer decide whether they're getting their money's worth. Sounds odd? Making developers business savvy, instilling them with a keen sense of where the money lies is sure better than simply assuming they're either idiots who know squat or who frankly don't care how their living is earned.
Just consider yourself as a developer: Which would you prefer? Having someone tell you that you have no sense of business - or having someone bring you to the point where you can do everything they can?
It is good if the team has a product navigator - it is better if everyone can read the compass to the pot of gold.


Summary

It is not wrong to have a PO. But don't focus on establishing and strengthening the role of the PO- Raise up the entire team to fulfill the responsibilities of the PO, and the dedicated PO will be superfluous.

Thursday, January 22, 2015

The Post-Scrum Manifesto

Is Scrum too heavy for you? Only if you're truly agile!


We are still uncovering better ways of developing software by doing it and helping others do it.
Through this work we have come to value:

Shared Vision over dedicated Product Owners
Taking Initiative over dedicated Scrum Masters
Continuous Improvement over Retrospectives
Continuous Deployment over Sprint Reviews
Continuous Flow over Sprint Planning
True collaboration over Daily Standups


That is, as long as you don't have the items on the left, 
you probably need to work on the items on the right.

Now, please go ahead and flame me :-)

Wednesday, January 21, 2015

Agile process models, Six Sigma and Cynefin



Back in my days as Six Sigma Master Black Belt, I learned and taught that every process can be modelled as a function. We write, on an abstract level, "y=f(x)". We use tools like SIPOC and Ishikawa Diagrams ("Fishbone") to model the relationship between inputs and outputs.

Since most processes have some variation, we accept that in practice, the function really implies "y=f(x+e)", where e is an error term within certain tolerance limits, typically impacting the accuracy of the model by no more than 5%. Because e is so small and we can't control it anyways, we ignore it in the model.

But what do you do when y=f(x) is simply an invalid assumption?

That's where agility comes into play.

Simple

The first difference between product manufacturing and software development is that product manufacturing is reproducible and repeatable. You know the exact process to apply to certain inputs to get the predetermined output. This is the "simple" world based on the Cynefin Framework.

But nobody in their right mind would build the same software the same way twice. Rather, if you have it, you'd use it. Or, you make some changes. Regardless. The output is not identical.

Complicated

What typically happens is that once a customer (user) sees a piece of software, they get ideas of what else they need. In this case, it's more like "y=f(x,y)": Given a result, the customer defines the new result. The process isn't reproducible anymore, because the output of each stage is input again. As a Six Sigma practitioner, I already had some issues with outputs that were process inputs: SIPOC becomes confusing, but it was workable.

At this point, it makes sense to work incrementally, because you can't know the final output until you have seen intermediate output. This is why agile frameworks typically apply backlogs rather than finalized requirement specification documents. We accept that the user may need to gain some understanding of the system to decide what has value and what doesn't. We also accept that the user may change their opinion on anything that has only been a vague idea yet.
Failure to implement checks and balances will move the team into the Complex domain within a couple weeks. Validation is not a onetime initial activity but must actually become stricter throughout the project lest errors accumulate and disorder leads straight into a Chaotic environment.
There is no way to reach the Simple domain, lest you acquire a good crystal ball that lets you accurately predict the future. (If you got that, you probably shouldn't develop software but play the stock market.)

Complex

You get into a mess when you only assume that the customer likes the result, but can't know for certain whether they do. The definition of quality may be unavailable until after the result is available to users. This scenario is fairly typical in web development, so as developers we have to guess what customers will accept. When users don't like something, we made an error. And there's no way of knowing that it was an error until we deployed the feature. We have to live with the fact that "y=f(x,y,e)" We can't eliminate the error term from our model, much rather, we have to accomodate towards the ever-present risk. We can only try to minimize risk by getting frequent feedback. The more time between two deployments and the more content within a single deployment, the more likely we made some critical error in the user experience.
Processes like A-B testing and continuous delivery become critical success factors. 
While you cannot completely eliminate the randomness, creating a high speed feedback mechanism, such as actively involving users in every minor detail produced, minimizes the effect of errors and effectively permits working similar to a Complicated environment.
The absence of control processes which deal with randomness may cause disorder to quickly shift the team into the Chaotic domain.

Chaotic

The worst thing that can happen is that yesterday's truth may be today's error. Customers may change their preferences on a whim. Imagine you work in a project where yesterday, you were requested to build a web portal for e-commerce, today the customer wants a document management system instead. Any plan is invalidated when you can't rely on your requirements, user stories or feature requests - or whatever you call them. Your model becomes a form of "y=f(e)", where neither x, you input, nor y, your previous output, are relevant indicators of success or failure.
This is where Waterfall projects with long planning phases may drift: By the time the plan is realized, demand shifted due to factors outside the project's sphere of control. An example would be a Waterfall team building a well-defined, perfectly fine php platform over the past 2 years meeting all business requirements, only to find out the newly hired CTO has just announced that all newly launched systems must be implemented in pure Java.
The only good news about the Chaotic domain is: You don't have to be afraid that it gets worse. Everything the team does may be waste.
The best way to deal with the Chaotic domain is moving away from it by delivering iteratively to minimize the effect of uncontrollable random on results. Frequent releases, no more than a couple years, provide the opportunity to move into the Complex domain.



Conclusion

The nicest statistical process model doesn't help when there is no feasible way to keep error terms under control or any model you come up with has relevant variables which can not be controlled. Six Sigma techniques for statistically modelling processes is useful when the process is repeatable and reproducible (good AR+R), but it won't get you far in a domain where R+R is neither given nor desired.