Becoming great at answering the wrong questions

"The right answer depends on the question"

This proverb has long guided me and I always strive to ask better questions.

This week, I evaluated some devops tools and felt oddly reminded on how similar the effectiveness of DevOps is to a product owner's story writing.

The wrong user story doesn't deliver the business value you're looking for.
Here are some of the highlights for poorly written user stories that I encountered while evaluating tools. Marketing makes these stories look like they are what you'd really want in your DevOps stack. But let's take a closer look.

As a DevOps, I want to see way back in time when certain error messages occurred so that I know how long it has been going on.

Yikes! If there are unknown error messages in the logs which nobody has been taking care of for months on end and nobody realizes - or cares - then either it's not important or your organization is in a mess. You don't need a tool that permits you to discover errors a year back in time, you need a strategy to effectively deal with problems as soon as they occur!
Let me rephrase that story for you: "As a DevOps, I want to be notified immediately when something abnormal happens so I can analyze and resolve the problem before it hits the customer!"

As a DevOps, I want to have a one-click solution which connects stack traces in log files with the corresponding source code segment so that I can analyze the effect on the user easily.

Good luck on that. I personally prefer to have robust, well-tested software, If your software is throwing significant amounts of stacktraces, then the effect on the user isn't your primary problem.
Let me slice this one for you: "As Sysop, I don't want any stack traces in production logs, because I don't want to operate a system that doesn't do what it's supposed to do." - and - "As a Developer, I want a software test that has sufficient path coverage so that I won't run afoul of undefined behaviours during refactoring." - and - "As application user, I expect the software to behave as intended in each and every circumstance so that my business outcome is predictable."

As a Security expert, I want to be notified in real time when user data is compromized.

Sounds great. But what are we really talking about here?  Why do you even care to know that in real time? There are predictable, controllable ways in which user data can be compromized. In this case, your strategy shouldn't be to introduce realtime notification, but prevention. Any minute you're investing in data theft detection would be significantly better invested in hardening your systems.
Let me rephrase that one: "As a data security officer, I want all known security loopholes closed so that we don't even have data security incidents!"

As a DevOps, I need data for every possible failure scenario so that in case of incidents, I have enough data.

Tools will give you a false sense of control when you ask the wrong question. Your question is not to have data for everything, but to eliminate root causes - so that you won't need data.

Your job as a DevOps is not to hoard a boatload of data that can't be humanly understood, your job is to eliminate operative risks within the software so that the essential monitoring data can be reduced to a manageable and comprehensible level.

Conclusion

Tools are not solutions unless you first define the problem.
Your DevOps Strategy will fail unless you first learn to ask the right questions.

Shared Vision over dedicated Product Owners

The first value of the Post-Scrum Manifesto states "Shared Vision over dedicated Product Owners".

To understand what this means, we must briefly glance at why we even have Product Owners in Scrum.

The role of the Product Owner

The primary role of the PO is to communicate the product vision with all stakeholders and maximize product value. When there is a misunderstanding or dispute, the PO facilitates the dialog. Moreover, the PO owns the Product Backlog. They decide what will be part of the product - and what won't. They set the priority and are accountable for making the right calls. Customer facing, the PO is responsible for what the team delivers. Team facing, the PO is responsible for what the customer wants.

What happens when you don't have a Product Owner

Let's just assume we have a textbook Scrum team, but the PO is missing. The first thing that will practically always happen is that stakeholders start bombarding the team with feature requests, everyone considering their own requirements top priority. The team now has the job of juggling all those requests while delivering working software. 

If the team is lacking a person who is capable of calling shots on either-or options, they run a great danger of implementing inferior solutions, dissatisfying the customer. 

On a more positive note, the responsibilities of the PO will typically be picked up somewhere, either through the customer side or the team. Unfortunately, there is no guarantee this will happen before the product failed.

Why having a dedicated Product Owner is bad

Having a dedicated PO communicate stakeholders on the product vision and making tough calls on value choices is good, but it's still a bottleneck. Since being agile is all about never ever maneuvering yourself into a corner, it simply doesn't make sense to enforce a bottleneck.

The need for a dedicated PO actually indicates symptoms for a number of problems:

Different levels of understanding

It's good if someone can explain the product vision to others, it's better if everyone shares it and product vision communication should never be a unidirectional process. 

Single Point of failure 

When someone needs to call shots, it means that this person is can singlehandedly cause an otherwise successful product to go awry. A good PO will never be a dictator, but since the PO will always filter information through their personal viewpoint, unsound decisions may be made. The more decisions PO doesn't need to make because the team has the knowledge and ability to make them, the lower the risk of having a SPOF actually failing.

Also, developers may become paralyzed or make wrong decisions if the PO is unavailable for clarification.

Stakeholders management over consensus

When someone needs to call shots, it means that different stakeholders set different priorities. This may always happen, but is a situation where one person has the final say really better than being able to come to consensus? 

Translate: Business-Developer, Developer-Business

The PO facilitates the dialog between team and customer, especially in situations where business expectations are technically unsound. In theory, this sounds good. In practice, we have an agile value "Customer collaboration over contract negotiation". Nowhere does it say customer collaboration is limited to the PO. If every developer collaborates with customers on a daily basis, the mutual understanding will be there without having a facilitator in the middle.

Assumption of incompetence or apathy

We get taught in Scrum classes that PO's understand the business value and know which increment is best to advance the product's value. Superficially, and in an immature organization, that is true: Oftentimes, developers have no understanding of business value. But that shouldn't be considered a given law of the universe. In Programmer Anarchy, there is a radical shift: Developers not only understand what is the best value for the customer, they are so closely interlinked with customers that they drive: They deliver working software and let the customer decide whether they're getting their money's worth. Sounds odd? Making developers business savvy, instilling them with a keen sense of where the money lies is sure better than simply assuming they're either idiots who know squat or who frankly don't care how their living is earned.

Just consider yourself as a developer: Which would you prefer? Having someone tell you that you have no sense of business - or having someone bring you to the point where you can do everything they can?

It is good if the team has a product navigator - it is better if everyone can read the compass to the pot of gold.

Summary

It is not wrong to have a PO. But don't focus on establishing and strengthening the role of the PO- Raise up the entire team to fulfill the responsibilities of the PO, and the dedicated PO will be superfluous.

The Post-Scrum Manifesto

Is Scrum too heavy for you? Only if you're truly agile!

We are still uncovering better ways of developing software by doing it and helping others do it.
Through this work we have come to value:

Shared Vision over dedicated Product Owners

Taking Initiative over dedicated Scrum Masters

Continuous Improvement over Retrospectives

Continuous Deployment over Sprint Reviews

Continuous Flow over Sprint Planning

True collaboration over Daily Standups

That is, as long as you don't have the items on the left, 

you probably need to work on the items on the right.

Now, please go ahead and flame me :-)

Agile process models, Six Sigma and Cynefin

Back in my days as Six Sigma Master Black Belt, I learned and taught that every process can be modelled as a function. We write, on an abstract level, "y=f(x)". We use tools like SIPOC and Ishikawa Diagrams ("Fishbone") to model the relationship between inputs and outputs.

Since most processes have some variation, we accept that in practice, the function really implies "y=f(x+e)", where e is an error term within certain tolerance limits, typically impacting the accuracy of the model by no more than 5%. Because e is so small and we can't control it anyways, we ignore it in the model.

But what do you do when y=f(x) is simply an invalid assumption?

That's where agility comes into play.

Simple

The first difference between product manufacturing and software development is that product manufacturing is reproducible and repeatable. You know the exact process to apply to certain inputs to get the predetermined output. This is the "simple" world based on the Cynefin Framework.

But nobody in their right mind would build the same software the same way twice. Rather, if you have it, you'd use it. Or, you make some changes. Regardless. The output is not identical.

Complicated

What typically happens is that once a customer (user) sees a piece of software, they get ideas of what else they need. In this case, it's more like "y=f(x,y)": Given a result, the customer defines the new result. The process isn't reproducible anymore, because the output of each stage is input again. As a Six Sigma practitioner, I already had some issues with outputs that were process inputs: SIPOC becomes confusing, but it was workable.

At this point, it makes sense to work incrementally, because you can't know the final output until you have seen intermediate output. This is why agile frameworks typically apply backlogs rather than finalized requirement specification documents. We accept that the user may need to gain some understanding of the system to decide what has value and what doesn't. We also accept that the user may change their opinion on anything that has only been a vague idea yet.

Failure to implement checks and balances will move the team into the Complex domain within a couple weeks. Validation is not a onetime initial activity but must actually become stricter throughout the project lest errors accumulate and disorder leads straight into a Chaotic environment.
There is no way to reach the Simple domain, lest you acquire a good crystal ball that lets you accurately predict the future. (If you got that, you probably shouldn't develop software but play the stock market.)

Complex

You get into a mess when you only assume that the customer likes the result, but can't know for certain whether they do. The definition of quality may be unavailable until after the result is available to users. This scenario is fairly typical in web development, so as developers we have to guess what customers will accept. When users don't like something, we made an error. And there's no way of knowing that it was an error until we deployed the feature. We have to live with the fact that "y=f(x,y,e)" We can't eliminate the error term from our model, much rather, we have to accomodate towards the ever-present risk. We can only try to minimize risk by getting frequent feedback. The more time between two deployments and the more content within a single deployment, the more likely we made some critical error in the user experience.

Processes like A-B testing and continuous delivery become critical success factors. 

While you cannot completely eliminate the randomness, creating a high speed feedback mechanism, such as actively involving users in every minor detail produced, minimizes the effect of errors and effectively permits working similar to a Complicated environment.
The absence of control processes which deal with randomness may cause disorder to quickly shift the team into the Chaotic domain.

Chaotic

The worst thing that can happen is that yesterday's truth may be today's error. Customers may change their preferences on a whim. Imagine you work in a project where yesterday, you were requested to build a web portal for e-commerce, today the customer wants a document management system instead. Any plan is invalidated when you can't rely on your requirements, user stories or feature requests - or whatever you call them. Your model becomes a form of "y=f(e)", where neither x, you input, nor y, your previous output, are relevant indicators of success or failure.

This is where Waterfall projects with long planning phases may drift: By the time the plan is realized, demand shifted due to factors outside the project's sphere of control. An example would be a Waterfall team building a well-defined, perfectly fine php platform over the past 2 years meeting all business requirements, only to find out the newly hired CTO has just announced that all newly launched systems must be implemented in pure Java.

The only good news about the Chaotic domain is: You don't have to be afraid that it gets worse. Everything the team does may be waste.

The best way to deal with the Chaotic domain is moving away from it by delivering iteratively to minimize the effect of uncontrollable random on results. Frequent releases, no more than a couple years, provide the opportunity to move into the Complex domain.

Conclusion

The nicest statistical process model doesn't help when there is no feasible way to keep error terms under control or any model you come up with has relevant variables which can not be controlled. Six Sigma techniques for statistically modelling processes is useful when the process is repeatable and reproducible (good AR+R), but it won't get you far in a domain where R+R is neither given nor desired.

Let's talk about DevOps!

Imagine you're a sysop. Your systems are up and running, so you sit back and chill. 

Yeah, I said, "Imagine". Now, I mean, seriously. Not like you'd have time for that. Every platform requires frequent maintenance, you wouldn't want any service to deteriorate. While you're juggling security updates, intrusion alerts, overflowing storage, network load and whatnotever could possibly go wrong on the average working day, those nasty Scrum teams come around and ask for a configuration change, giving you only a couple days to implement.

They can't even formulate a proper Request for Change including a detailed risk assessment and simply want you to "install that one module" and because you think that the new storage module is more important, they request "just give us the root password and we'll do it ourselves".

Anyways. That's beyond reasonable. Someone will compromise the server in one way or another and everyone will point fingers at you. Worst case, you'll get the sack. No chance in hell freezing over!

Ok, that was "Why are admins always so slow" for all of you developers out there. 

Now, let's imagine you're the developer. You've got your story backlog, you've got the code lined up and are ready to deploy. Inside your Scrum team, you can simply prioritize the issue, and everyone will swarm to solve it within the business day. Unfortunately, this time you need a change to the system configuration, a certain module needs to be installed. And that means you have to go into the lion's den of sysadmin. And probably they'll ask to have everything written up formally, only to shove it into some tracking tool where it will rot until a date far beyond sprint review. Chances are, your story won't make it, because they'll be too slow.

That is the flip side of the coin. Now, who is wrong?

Trick question!

This dilemma exists in many companies and the result is neverending fingerpointing and blame-games. But - that's not how our story needs to end.

How can you synchronize Development and Operations without infringing on either, while satisfying both? What can Development offer to Operations to make their life easier, how can the mutual pain be eased? 

In small startups, you can simply move the admin role inside the sphere of responsibility for the Scrum team, but as soon as the organization grows, difficult questions will arise: How do we keep servers secure across teams, how do we maintain dozens of individual systems which can go berserk every minute, etc?

Some people go as far as claiming "#NoOps: Developers can do everything!", but let's be honest. You're not very likely to find someone who can do the highly specialized fulltime job of a developer and still be a highly specialized fulltime admin, while at the same time having time to keep up with new business requirements and user activity on the platform - and still working on an affordable payscale. On the other hand, distributing administration within the team may simply not be feasible, because too many half-competent (no offense) developers run a risk of making a dangerous modification towards the servers.

The main result I have seen so far coming out of #NoOps claims is servers that are hardly maintained, with configurations that are insecure, incomprehensible and run a tremendous risk of falling apart any minute. Many developers don't even understand why they shouldn't simply log in as root. And that's just the tip of the iceberg. It goes further over using telnet login, creating tablespaces with "autoextend" and more. (I apologize to all the admins out there for the head bruises incurred by facepalming too hard and to all the developers out there who don't understand where the problem is).

Development must understand the pain of Operations rather than pointing fingers - and Operations must understand the pain of Development. 

DevOps are the solution: Let's just admit that administration is necessary and that developers are developers. And then work out solutions from there. Rather than usurping admin activity into the dev team, let's facilitate a positive information exchange, foster mutual understanding and simplify the interaction.

For instance, you can automate the configuration management with CI, or you can go as far fully automate deployment and eliminate the risks that always come with major releases. Development can relieve Operations of any activity that really no admin wants to do (i.e. working through tedious, poorly written instruction manuals) , and in turn, Operations can provide platforms where change is smooth to implement and easy to track. 

Refactoring running rampant

The purpose of refactoring is: Increase the quality of code without changing the functionality of the code. Unfortunately, we too often forget that "quality" is in the eye of the beholder.

Let me give you one example. This is pseudo Java code snippets, but you can see where it leads.


Originally, there were 2 classes which contained code like this:

class A
int X;
int Y;
public int compute ()
{
   return this.X + this.Y;
}
class B
int E;
int F;
public int compute()
{
  return this.E - this.F;
}

Well, those are highly similar methods, so "obviously", this calls for refactoring.

Let's start refactoring to a common level:

class A
int X;
int Y;
private int _compute ()
{
   return this.X + this.Y;
}
public int compute ()
{
   return this.compute();
}

class B
int E;
int F;
private int _compute()
{
  return this.E - this.F;
}
public int compute()
{
   return this.compute();
}

We have some duplicate code now, so we want to eliminate it by moving the method into a new class:

abstract class operatorClass
public int compute ()
{
   return _compute;
}
class A extends operatorClass
private int _compute ()
{
   return this.X + this.Y;
}
class B extends operatorClass
private int _compute ()
{
   return this.E - this.F;
}

At least, now there is a common level, but there is still highly similar code. Let's do something about it.

class A extends operatorClass
A()
{
  setOperator ("+");
}
class B extends operatorClass
B()
{
  setOperator ("-");
}

abstract class operatorClass
int X;
int Y;
String operator;
public int compute()
{
   switch(operator)
   {
      case "-" : return X-Y; break;
      case "+" : return X+Y; break;
      default: throw new InvalidOperationException (operator); break;
   }
}
protected void setOperator(String operator)
{
   this.operator=operator;
}

Yay! We eliminated the "nearly duplicate" method in 2 different classes and reduced the amount of code in both of them.

Unfortunately, there is a small fly in the ointment here:

• Case-Statements are poor code. Reason? They are more difficult to unit-test. Path coverage comes to mind. It's also a violation of the Single Purpose Principle.
• The "operatorClass" is now doing stuff it shouldn't be doing, that is: making a decision that should be made on a more abstract level, i.e. on the level when the object is created - a violation of the Dependency Inversion Principle!
• We actually introduced the possibility for error into the operatorClass' "compute" method. Trying to call "compute" with invalid operators was not possible before!
• Oh, and that, of course means, we need additional Exception Handling. We didn't even go into the new "InvalidOperatorException" class that we must create.
• Each time we implement a new class that implements a new "compute" method, we must modify the "operatorClass", so we just violated the Open/Closed Principle!
• Not to mention the application's performance has just deteriorated. It will now be slower, because of the "case" statement that must be evaluated. It will also consume more memory, because an additional variable must be initialized

Summary:
While the clode looks cleaner when you only look at the level of A and B, we merely "shoved dirt under the rug", but we didn't help at all!

Lesson learned

Refactoring is not a purpose in itself.
Not every refactoring is actually a positive change.
When refactoring, you must set a clear purpose of what you want to accomplish - and why. Even when you do not break (unit) tests and the code becomes shorter, you may be doing something tremendously harmful.
I strongly advise doing Code Katas occasionally to get a grip on how to refactor beneficially.

Software Development Lifecycle - Testing

The Software Development Lifecycle: Testing

What you see above is the "Test Cycle" as I learned and practiced in Waterfall environment, for years.
Now, I don't even want to go into how in theory, you can add significantly more test phases in here.Neither how in practice, smoke, integration and regression tests are usually neglected.

The simple fact that developers hand over software they consider "works as designed" to test is ingrained into the mind of waterfall software project specialists.

As I mentioned in another post about test coverage, defects occur even when developers consider that their software is defect free.

Let us consider for a minute that each test costs time.
While a piece of code is in test, developers continue to produce more working software. Yeah, I know that the Waterfall theory says that once development is finished, the product is handed off to test. But seriously - has this ever been reality? Do developers really sit there twiddling thumbs until testers report defects? Do companies really pay developers to sit idle while testers are busy?
If you are seriously working in such an environment, I would have a great optimization suggestion for your management.


So, developers build on code they consider to be working while test time passes. If a defect is then found in a component they are building on - yet, given the defect, the new component did "work as designed", the defect fix may cause rework not only in the defective component, but also in the current work-in-progress: Fix efforts may already be twice as high -or even higher- as if the defect was discovered before the developer started a new topic.

The problem is intensified when developers don't induce defects into new components, but into components that have already been accepted in the past. Ignoring the fact that oftentimes, when schedules are tight, regression testing is the first activity to be descoped, it's always the last thing testers do. This approach actually designed to maximize the amount of time that a defect can stay in the software - and therefore, maximizes the amount of damage a defect can do!

Is this smart? No!

You will never deliver cost effective high quality products unless you un-learn this model!
Forget everything you learned about Design-Develop-Test. It's the wrong philosophy. You can't improve it. It doesn't even get better when you increase the amount of time for regression tests or put regression testing in front of functional testing.

The Solution

A paradigm shift is needed.
Here is a non-exhaustive list of changes you must make, preferably in this order:

1. Introduce mechanisms that let your developers know whether they introduced defects before they pick up a new task.
2. Don't even let developers start on a new topic until there is confidence that their last piece of work didn't introduce defects.
3. Automate testing. Enable developers to run any test they need or want to run at any given point in time, as often as they need to. Don't make them wait days - or weeks - for test results!
4. Eliminate the "tester role" (but not the testers). In Scrum, we speak of a "Developer" even when we mean "the test expert" because everyone is accountable for high quality. Make programmers cowork with test experts before actually starting to write code.
5. Create test awareness. Make sure developers know exactly which tests must pass before they create code.
6. Introduce test driven development (TDD). Give developers access to the tests before they actually start coding.
7. Change your process: Create quality awareness and accountability. We utilize "pre-commit hooks". Developers cannot even commit defective code unless they specifically override, but even then, the defect will be tracked on every single commit until resolved.
8. Implement Continuous Integration. Let developers know immediately if their component damaged the product build. A wait time of 10 minutes is already tough, days simply aren't acceptible!
9. Implement Continuous Delivery: Developers should never be working on their own environment or an individual branch for many days without merging back to the master. They should be working in small increments that can be delivered fast. This minimizes the risk that days of work need to be scrapped because of a wrong premise.

Your future process should be fully integrated, eliminating time gaps between design, development and testing. Testing should be an activity that starts before development, should go on in parallel to development and should be completed by the time the programmer moves a story, feature or task to "Done".

If you still need a "test phase", always think that any single day that a defect is within the software, you increase the cost of poor quality. Think different!

Test Coverage

What you see above is a classical test report as you would expect it within a Waterfall project. Typically, test managers produce such statistics at frequent intervals to report the progress of testing.

As with any diagram, this one isn't worth much without explanation.

The blue bar is the amount of test steps conducted per day, the red bar is the amount of defects the software contained on each day.

Now, what you see is an amount of roughly 15000 test steps being run over the course of 2 weeks and roughly 800 defects discovered.

In large-scale Waterfall projects where I worked previously, this would have been a boatload of effort for a team of maybe 20 testers.

A job well done for the test manager, you can be sure QA would be praised.

What is this metric - really? 

It's the results of automated integration and acceptance testing for one of our products. What you see here is only my personal tests: Don't even get me started that I'm not a fulltime developer. All of the tests you see displayed here - accumulated - ran within less than 1 hour including defect reporting.
None of the defects discovered made it past the evening. All were fixed within the same business day!

The consequence of such activity?

We use Continuous Delivery, so we are able to deliver working software on a daily basis - and we haven't had a single critical fault in the system on a live installation.

Years ago, when I was working exclusively Waterfall, I couldn't have believed by myself that not only a single person could execute as many as 4000 test steps per day. I wouldn't have believed - and that is probably the more critical learning here:

A single change to a single component could wreak havoc in use cases that merely rely on interfaces and have no direct connection to the implementation of the changed component!
When a programmer might consider their work "Done" - without proper automated test coverage, these beasts are still lurking deep in the dark!
I wouldn't have run the tests if I knew that my code was bugged! I typically stay in development until I am confident that my code is defect-free! I mean, hey - I got unit tests to make sure I didn't do anything bad! (and that's already more than we often had in Waterfall projects)
In a Waterfall, I would have handed the topic off to test and waited for any potential bug reports.
And chances are, testers wouldn't have done a Regression test and wouldn't have discovered the issue before the change went live.

It's not that I became a crappy developer by doing Agile. On the contrary. All of my Waterfall experience is still there. I make fewer mistakes than ever. But mistakes still can happen. The difference is that they don't make it into production any more.

Lesson learned

If you're a Developer, you are well advised to automate tests for your components so that you know if you accidentally broke something. However, don't stop there. Unless you have integration test suites, you may not know when your work has detrimental impact on the system's overall functionality. Chances are if you are only automating functionality and not use cases, your software still works, but your users can't.

Hence, agilists say "Automate everything". (well, everything that makes sense) - it does pay off!

We're Agile now, we don't do testing any more!

Scrum only has Product Owner, Scrum Master and Developer.
Testing is not part of the Scrum process, so we can eliminate it - and save money by sacking the testers.

I've had the joy of working with a company that seriously thought about taking this road.
It was a shock for me not because I come from a testing background, but because it's wrong on so many levels:

Agile means: Deliver valuable software

Buggy software isn't valuable. We don't expect to get praised for a "job well done" if the result isn't both usable and useful. How do you know that it's usable without testing? How do you know it's useful without testing?`

Agile means: Deliver value often

Working on bugfixes drains resources. The only way to prevent bugfixes is by not introducing bugs to the customer in the first place. How do you know that you don't have bugs without testing?

Agile means: Deliver value fast

You need a profound understanding of what your components do - and why. Without testing, you will be spending more and more time in understanding the impact of change as the project progresses. Specifically, without systematic testing, you may completely lose control!

Agile means: Eliminate waste

A standard trumpet: Testing is waste, because if there are no defects, tests don't add any value.
Well. That may sound true if you have a superficial understanding of (software) engineering. 

Ask yourself: Would you hire an architect who didn't run calculations to verify that your house won't crash over your head? No. Would you ride a car where designers didn't validate traffic safety regulations? No. 

So why do you want to build a piece of software without those checks? 

Because bankruptcy isn't as bad as death? Great! Tell me your company name and I will make a fortune by short-selling!

Agile means: Working with feedback

Each time a piece of code is finished, it should be exposed to customer feedback. Unless you want to look like a fool in front of your paying customers, you better have a strategy in place to make sure that they like what they see. You must verify the correctness and validate the applicability of your solution to their problem before confronting them with results. And the formal process for doing them is called "testing".

Conclusion

Not testing is planning for commercial suicide.

It holds true for Agile even more than for traditional projects, because in a Waterfall, you might get the customer to sign off some crappy product just for political reasons. In Agile, your customer will know that you don't know your trade within a couple weeks.

The most important role in Scrum

It's a philosophical question: Which role is most important in Scrum?

When I went into Jeff Sutherland's Scrum Master class, he stated that "The Scrum Master is the most important person in the team", on the Product Owner Training, I heard "The Product Owner is ..." - and during Scrum Developer Training, I heard the same about Developers.
Now, what is the real deal?

Let's look at this slowly.

The Product Owner

Imagine you have no Product Owner.
Who takes care of the backlog, who grooms it? Who makes the calls when stakeholders quarrel about the direction of the product? Who communicates the product vision to developers and stakeholders?
Ok, let's make it short. If there is no product owner, there is no product, there is no project. There is no need for a team - so obviously, the PO is the most important person.

Finished.
Well, that was quick.
Oh, wait.

The Scrum Master

Imagine you have no Scrum Master.

Who arranges your ceremonies, who takes care of impediments? Who makes sure that management or other stakeholders don't violate the team's self-organization? Who takes care that the Working Agreements are adhered to?

Ok, let's make it short. If there is no Scrum Master, teams will most likely fall into disarray. Maybe not because of themselves, but because of the world around them. A team in chaos will not deliver.

So, obviously, the SM is the most important person.

Now, we've got a conflict already. 

But we're not finished yet.

The Developers

Ok, in traditional organizations, we know that managers think that the worker drones are easily replacible but everything depends on their genius.

But let's get real: You got a product vision, you got a development process. But who does the thinking, where does the code come from? Who makes the vision real?

Guess what - if there's no developers, the best PO in the world is useless!

So, developers are the most important people!

Everyone is important!

Many agilists use the Pig-Chicken metaphor, stating that an Agile team should only consist of "pigs". I don't like this analogy, because first, I don't consider my coworkers to be pigs for obvious reasons.

Second, if you're in Saudi Arabia, chances are that nobody wants bacon, rendering the pig completely worthless.

Simply said, if you're on an Agile team, everything depends on your contribution. You are essential to success. If you're not essential, you're in the wrong team!

The Bottleneck issue

Are you terrified of going on vacation? Can you already guess all the disasters that will happen when you're out of office for more than 2 days?
If yes, you're not alone, but there is something you should do about it.

Oftentimes, new Agile teams have the "Problem" that the intensity of work is so high that people are somehow stuck in their role.

For instance, the Scrum Master may not even get to coach the team because impediments and ceremonies already fill their schedule: The installation of the Continuous Improvment Process may become secondary.
Or, the Product Owner is spending fulltime answering questions of team members to ensure the product is terrific: Good, but the grooming of future backlog items and stakeholder management may suffer.
The worst case is developers taking so many stories that they lose time to hone their agility: The delivered product may be great, but we didn't implement any ways to do the same thing faster and easier.

The Solution

Agile methodologies try to eliminate "Single Points of Failure", i.e. having success or failure hinge on one single person.

• Developers should ideally pair up to make sure that there is no capability required in the project which only one person has. Why? Isn't it more easy to replace you if some else possesses your skills? No! On the contrary: if you can chip in wherever something needs to be done, your value to the company increases!
• The Product Owner, "owns the product" and needs to call shots. However, they should not feel obliged to create a structure where every minute decision entails them.On the contrary, the PO should communicate the vision so clearly that the team can independently decide what is the best way to advance the product. 
• The Scrum master, "owns the process" and is responsible for ceremonies and managing the impediment backlog. A good Scrum Master will not spend their days running after individual impediments and arranging meetings. Ideally, they will coach and empower the team to do this by themselves. 

For both the Scrum Master and the Product Owner, I would refer to an important proverb about aid: "The most important job of a helper is to make themselves superfluous", with the intention "If you enable and empower others to fill your role, you did a good job, otherwise you missed the mark".

Sustainability is one of the Agile Principles, and you will not have sustainable development unless everyone actually strives to enable others to do what they are doing.

How agile are we?

"Hey, we're doing Agile now!" - "We too, but it doesn't work".
How often do we hear this? Too often.

Not everyone who has put the term "Agile" on their IT strategy is doing the same things. More precisely: Everyone understands something else under the term "Agile".

As others have pointed out, "Agile" requires more education than a 2-day training course for prospective Product owners and Scrum masters can provide.


Here is my suggestion for an Agile Maturity Model of Implementation (AMMI).

Agile performers

You can't really predict what their practices are, and frankly, they don't even care so much about specific practices or processes. What they care about is: Valuable software and happy customers. Everything else is subject to that and negotiable.
The main thing they have in common: They thoroughly understand the reasons why their processes and organizations look the way they do.
These performers are the reason why the Agile Manifesto is so short.
They follow, what could be called a "Scientific truth" (credit: Paul Oldfield). They experimented, adjusted and have gained confidence that their current approach is the best way at the current time. Gaining evidence that other approaches will yield better results, they will quickly move on.

Agile transitioners

When getting into a transitioning company, you can most likely predict their processes and organization based on subject literature. You could update the "Scrum Primer" accurately by taking pictures of their ceremonies. They're seriously trying, and probably seeing some improvements over their old methods of working.

Most likely, the way they are doing things is highly imperfect, but hey - empiricism means learning by doing!

What transitioners often have in common is that they lack deeper understanding of how to optimize their agile practices. 

They follow, what could be called a "Political truth" (credit: Paul Oldfield). They were convinced by someone that their current approach is the best choice.

Agile cargoists

"Man, we've been Agile for a year now and nothing has improved! We're doing everything exactly how Spotify is doing it, but ROI hasn't improved!" - Agile cargoists religiously follow prescribed patterns in hope of results. Unfortunately, these never turn up. 

Cargiosts, at first glance, look like transitioners. The big difference is that their Continuous Improvement Process is either political in nature (i.e. the team is not empowered to change) or fad-driven, i.e. researching what others do. 

They follow, what could be called a "Religious truth" (credit: Paul Oldfield). A reputable source claims that this approach is the best, hence it must be so.

Non-Agile

The entire spectrum between "Don't know about it", "Read about it, but it's not for us" and "Tried it, didn't work, we're back to classic project management". 

Agile by Name Only

Get into an ABNO and you will find tons of Project Managers dressed up as Scrum Masters and Business Analysts / Architects vested as Product Owner. 

Most likely, projects start with a Feasibility Study, Requirement Documentation, Budget Planning and Detailed Design before ever involving an "Agile developer". 

The "Agile transition" in these companies was fast and painless: Managers were sent to a 2-day training course, the titles on business cards were adjusted and voilá: A new agile all-star is born!
Agile culture are at best random elements, but not the norm.
ABNO's usually blame Agile for being ineffective and give the entire approach a bad name.
Transitioning an ABNO into an agile organization is probably harder than teaching a granite block how to swim.
The best thing about ABNO's is that they will return to the Non-Agile department sooner or later: Most likely after firing most of their "ineffective developers". At least, they'll have plenty of managers left.

Slicing and dicing user stories

To really understand the effect that the Product Owner has on the success of Agile development, it is vital to understand the art of creating good user stories to deliver results.

As was mentioned in a previous post, a poorly chosen user story will make it tough for the team to deliver, wasting time and potentially effort with poor control over the investment. On the other hand, a well chosen user story will support the early delivery of valuable results.

The creation of good user stories depends on a technique called "Slicing", i.e. preparing user stories in a fashion that they become immediately useful once completed.

Horizontal slicing is bad

A poor approach to slicing is cutting a huge story into horizontal slices. Let us use our "World Hunger" example to explain this approach:

We may want to resolve World Hunger in the following manner:

1. Obtain enough food to feed the entire world
2. Gather enough volunteers to distribute this food in the entire world
3. Establish the logistics to accomplish the distribution process
4. Obtain political support (permission, visa, military/police protection) for the distribution

Let us just assume that we had successfully completed steps 1,2 and 3 - but we don't get permission!

We have everything in place, yet there's a bunch of nuts sitting in well-furnished offices who block results for whatever reason or maybe for no reason at all.

While we gind our teeth in frustration and our supplies spoil, people continue to starve. 

A whole lot of resources were wasted, a lot of time was spent - yet there is no visible result!

What happened? Dependent layers were built, and all activities on one layer depend on another layer for success. We can't even know whether our concept works until all layers have been completed, well - at least to some extent. But until all layers are achieved, there is always this looming sense of failure.

(When this happens in the corporate world, you would find some CxO closing divisions, cutting funding and whatnotever that is bad for your career.)

Delivering with vertical slices

As previously discussed, a vertical slice is intended to deliver fast, with limited scope.
"Feed one person for one day" is a limited task, it produces visible, measurable results and permits scaling.
The result from "feed 1 person for one day" is visible and can be subjected to Continous Improvement. Each day, each person contributes to success and regardless of how far we get, the feeling of accomplishment stays with us.

Once 1 person has been supplied, we can enlarge the process to feed 2, then 5, then 10.
All of a sudden, we will see that we are more efficient by centralizing the supply process, so we set up a user story like "As food distributor, I would like a central place where I can always pick up the food baskets so that I save time in getting food to the Needy."
Once we reach 100 people fed, we may get into stories like "As charity, we need media coverage, so that we can obtain funding to feed another 1000 people"
These stories are again, limited in scope, provide a good measure of success and can be completed in a limited amount of time.

Maybe we need to rework our former approach (by Refactoring) but maybe we can just continue doing things as before.

Summary

Lean identifies multiple types of "Waste". One of these types is producing stuff which we can not use yet ("Inventory"). Another type is producing stuff which may never be used ("Overproduction").

Well-sliced stories which resolve (or even better: eliminate) dependencies on other activities serve the team in reducing the risk of delivering features which are not useful for a long time in the future, or - potentially never.

Good Story slicing positively contributes to the visibility and value of results and negatively, eliminates waste.

Therefore, the Product Owner is a key player in enabling team success.

Making stories manageable

As a Human Being, I want to end World Hunger so that there is no more hunger.
Wow, I've written my first user story and it's even correct based on the template. Now my team can get going and because they are Agile, we will have a solution in no time!

What? You have some feedback proposal? ... mmh, okay. :(

Well, poor stories are often "World Hunger Projects" and this one is no different: It's just being a bit more straightforward to realize.

Too many times, in projects I see stories like this one: As a user, I would like to see my data so that I can see what I have to do next.
And the poor developers are stuck with a mess and can't deliver at sprint's end!

Let's take the example of the World Hunger Story to practice a little bit of slicing:

User Roles

Human Being - hmm, that looks pretty big scope. What is your role in all of this?
Are you a human being who is in the fortunate situation of having excess wealth and want to share it? If so, you are a "potential donor".
You maybe the founder of a charitable organization dedicated to actively do something about hunger.
Are you "afflicted by hunger" and need food for yourself? In that case, your perspective is different.
Or maybe you're a "homicidal maniac" and propose dropping a few nuclear missiles on hunger hotspots. (Hmm, maybe as a PO I would pass on that story.)

By identifying different user roles, it becomes significantly easier to focus on what actually needs to be implemented.

Specific goal

Before we get into the What, we must be clear on Why we want something to be realized.
Well, "that there is no more hunger" sounds cool, but it's not all too clear. How can we tackle this?
Let's put ourselves into the seats of a charity organization.
We have a whole bunch of problems, to name a few:

• We need to have something to give to the hungering people of the world:
• Do we want them to have food for the moment?
• Do we want to help them obtain their own food?
• Do we want to resolve a specific crisis?
• We need to obtain funding
• How do we raise our funds?
• How do we manage our funds?
• How do we spend the money wisely?
• We need people with whom we collaborate:
• Do we need speakers?
• Do we need celebrity sponsors?
• Do we need skilled workers who are willing to go into the region?

Each of these is a specific objective to tackle - and even these may still be too big to tackle with just one team and within a limited amount of time, so we may want to get even more specific!
But let's just stick to one goal now.

Specific approach

So, now we are the charity who want go give food to the Needy.
A whole bunch of ideas may come to mind:

• Food baskets for the neighbour in need
• Soup kitchen in the metropoles of the world
• Shipments of wheat/flour/rice to central Africa

Putting it all together to deliver

Now we may create a much more focused story card:
"As a charity, we want to give food baskets to the neighbour in need so that nobody who lives in our community would have to go to bed with an empty stomach".

It won't really solve the entire problem of World Hunger, but it's a step, and you can have visible results in no time.
The team can brainstorm on the contents of the food basket, identify a needy neighbour - and maybe it can be delivered on this very day, working with real user feedback ("Thanks so much for the milk, but I'm lactose intolerant") and improve their strategy from there.

Summary

A good user story helps the team deliver visible results in a very short time without falling into the trap of missing the mark, running into Analysis-Paralysis and many other problems.
Good stories help the team deliver, motivate the team, enable Continous Improvement, allow the PO to check the direction of the implementation and adjust.
And most of all, they provide real value at a limited investment.

Six types of Agile Products

We're Agile now. We don't need a Project Plan any more, we're freed from deadlines and we don't have to work off predefined requirement lists any more. Yay, this is gonna rock!

Let's go for it!

Not so fast, young Padawan!

Agilists don't abolish processes or planning. When we say that we value adapting to change over following a plan, it doesn't mean we don't plan!

Agile projects might be even more rigorous than any traditional project you have ever worked on!

They are set up differently. Nobody went out to obtain a $100m funding up front, "be seeing you in 5 years with the results". (And nobody in their right mind should fund a project like that!)

Agile's "Inspect and Adapt" follows the decades-old acronym "WYSIWYG": you get what you see. 

This goes both ways. The project will not continue to receive funding when there are no more results!

So, before you go out on your next hackathon, be prepared to deliver as much as possible at any point in time.

Agile products grow over time. There are four primary growth stages of agile products:

Seed - "We know what we want to produce, but haven't really started"

Startup - "Here is what we're working on"

Initial - "Go ahead and use it - we know there's still lots of things we must work on"

Mature - "If you want more, you gotta pay for it!"

In the seed phase, the stakeholders come together and define the product vision. At this time, we may have a project homepage and a project team, but nothing to show for. As a newly formed venture, we might go to Kickstarter, but we need a compelling reason why anyone should fund us.

Actually, unless we have some proof that we can do what we've set out to, chances are we won't get this funding. So, we need to produce something to get an initial taste.

There are three things we can do, we don't need to do all but we should pick our options:

The Mock Prototype will give the stakeholders and future customers something to connect our vision with something more tangible. It will give them an impression of what may one day be - and it will give us an impression of whether we're aiming in the right direction!

In the mock prototype, we'll usually resort to mockups and we might want to spend as little resources on generating it as we can. If we build some code from the mock prototype, we might even salvage some of the code for the final project - although I wouldn't count on it and in many cases even advise against it.
We may borrow apply techniques such as Rapid Application Development to put out one or more prototypes and determine what the product could look like.
As prototypes are built to be disposed after they achieved their validation purpose, if you spend weeks on a single prototype, your project is probably already dead, so we need to do this in a cost-effective and highly disciplined manner.

We leave the prototyping phase once we have a validated understanding of what we actually want to build.

Once we know what we'll be building, we should produce a Walking Skeleton that features most of the end-to-end components of the final product. For instance, we might produce an application consisting of a webserver, a database and a web frontend. It's not going to do anything fancy, but that's not the point of the skeleton. We want to know if we have the right means to get the job done. Filling in the functionality is for later.
Let us use an example on this: If we are trying to build a data mining application, but already fail at the "store data" part because we can't maintain our data, we don't need to worry about building a nice user interface or tuning performance.

We leave the Walking Skeleton phase once we have a validated understanding of what resources and technologies help us build our product.

As soon as we have a reasonable amount of certainty that we can make our vision real, we should work on the Minimum Viable Product.
The MVP is a usable piece of software that does something in line with the product vision, so we are talking about working software produced using good engineering practices.
Because of this, the Product Owner has to make some seriously tough calls on what should be part of the MVP and what not. A good PO should never fall for the temptation to load the MVP with features - in this stage, less is more!

While A Walking Skeleton is a great basis for the MVP, there are often good reasons for building an MVP which doesn't even use any Walking Skeleton.

While we can "alchemize" on the Prototype and to some degree even the Walking Skeleton, we need to have some form or engineering framework in place to produce the MVP. As the product is just in the startup phase, Scrum is a perfect methodology to build up both the team and the product simultaneously.

At this point, take caution that effort was already invested, but you still have nothing to show. Even the value of the MVP is close to Zero, its value is in the learning rather than in the bottom line.
Because of this, the investment into an MVP must be minimized both in cost and time. Chances are if you take more than a few weeks on an MVP, you risk a lot in terms of "building the wrong thing".
For your own sake, if you can't see yourself reaching an MVP anytime soon, just let the project die!

You want to get into the "Initial Market" phase as fast as possible.

This phase determines everything for investors. A good product will start generating revenue in this phase, because you have a Minimum Marketable Product, something that you roll out "ready for use". You want people to actively use the MMP for more than some fiddling. If you can't get customers hooked to the MMP, you might want to cancel the project, because it will most likely be wasted effort!
The transition from MVP to MMP requires delivering the most critical items in your backlog as well as continuously staying in touch with the customer base in order to refine the backlog.
Techniques such as A-B testing will give you feedback on how customers like different approaches to new features.

If your engineering practices are still lacking in the MMP phase, you will most likely kill your product due to technical debt sooner or later.


Let's hope your project has survived the MMP phase and is sufficiently mature to enter a maintenance routine.

A mature product has an established customer base and should have generated sufficient ROI to make the investment meaningful. At this stage, the backlog no longer consists of any release-critical features and most things to be done are merely extensions of existing features. 

When there is nothing which must be added to the product, it is no longer essential to follow a rigorous short-term delivery routine such as the Scrum timeboxes. Also, when the criticality of further extensions is low, funding may be reduced.

Agile practice recommends shrinking the team(s) at this stage and moving on to more critical projects. The reduced team is mostly in "maintenance mode" and occasionally produces new features, so Kanban may become the team's preferred delivery framework.