Why project organizations are doomed

We talk a lot about why we should shift from organizing around projects to stable teams. In Scrum trainings, we use the Stacey Diagram and explain the need for agility with the inevitable Unknown that we encounter. From that we conclude that Waterfall projects with pre-planned structure and approach are doomed to fail. The basic assumption can remain intact that if we had more certainty, projects would work out. And so, we basically still assume that project work is fine - as long as it's done in an agile way. In this article, we will discuss that a project organization is doomed - agile or not. You have no choice except to change how you organize.

Prelude

The Want-Need-Matrix

Let's start out with a simple matrix I borrowed from Marshall Goldsmith. Well, I took the liberty of simplifying it even forther than described in his book, "Triggers", because it's enough to serve our purpose for this article:

The Need-Want Matrix

On the vertical axis are Wants, on the horizontal the Needs.
In the top right we see: We need and want success. To not open up a barrel, I am not defining "success" in detail. It can mean different things to different people. Whether that is recognition, money, gratitude or whatever - we long to have those things, and we need them for our own well-being.
On the upper left, we have what we want, but don't need: distraction. In this case, anything that distracts us from success. The worst kind of distraction is instant gratification, ignoring long-term success to have a temporary High right now. Arguments are another distraction: Fault-finding without problem solving distracts us from proceeding.
On the lower left, we have what we neither want nor need: trouble. Trouble is the opposite of success - distress, losing money, being shamed, ostracized or whatever.
Nobody really wants rules. We just need them to prevent trouble. And they help us keep on track, so that we can achieve success.

The Vicious Circle 

The vicious circle: Distraction, trouble, distraction ... it never ends!

Distraction wouldn't matter nearly as much if it had no impact. This is rarely the case. Typically, the result of distraction is that something slips. And when something slips, that means some kind of trouble. At the minimum, we need to catch up. Or we start making excuses - which only end up causing further trouble. Once we're in trouble, we typically try to get out of the troubling situation. The easiest route is displacement activity - yet another distraction. And so, we end up "fixing" the problems caused by distraction with distraction. Now, how do we get out of that?

The virtuous circle

Moving to success requires rules

Once we're in trouble, we can deflect - or we can improve. We typically set rules to avoid the distraction that caused the trouble.
It hardly makes sense to create rules for every possible distraction, so rules are typically reactive. We see the virtuous circle: We got distracted, we took the hit, set a rule and move forward. Like this, we constantly improve.

Don't take success for granted

What got you here won't get you there!

Be alert: Just because you're successful, doesn't mean you remain so. At any point in time, even currently successful people can run afoul of falling into distraction. There are many reasons why this may happen. One of them being that your environment changes constantly - and that means that today's success may be tomorrow's failure. By continuing to do what you did previously, you might actually be distracting yourself from succeeding in the future!

Keep your rules updated

Sticking to helpful rules keeps you on track

By keenly observing our rules and updating them as necessary, we remain in the success dimension and avoid distractions, consequently - trouble.

Adverse Rules are devastating!

Adverse rules that prevent success!

Sometimes, we set up rules that not only don't prevent trouble - they prevent success and cause the trouble we're trying to prevent!
Once we have adverse rules in place that we can not simply break, our only way of avoiding trouble is to remain in the "Distraction" quadrant.

The problem of projects

A project is usually defined as a specific endeavour to be accomplished within a specific time at a specific cost. Companies do their best to align project objectives with business goals, and therefore, overall success. Then, they task a Project Manager to meet the project objectives - deliver the defined result in TQB.

Same goal, different goals

While the project's stated goal is part of the corporate goals, the goals of the project are not the same. The goal of the project is not to succeed as a company - it is to meet the predefined objective in TQB. The so-called "Iron Triangle" is usually a bit softer in one or two corners, and a Project Manager will optimize to meet the Primary Objective (i.e. deliver on a specific date - or prevent budget overrun) at all costs.
Suddenly, the presumed goal (the Business Objective) is replaced with a different goal - meeting the Project Objectives.
We call this a "distraction", and now our project organization is directed towards a distraction.

Favorable rules, adverse effects

A Project Manager is always expected to do their job properly, i.e. deliver the stated objective in TQB. As their success is based on their Primary Objective, they will create a set of rules to ensure their Primary Objective.
When a developer within the project has the option of working towards the business goal, in conflict with a Project Rule, the developer will be constrained by the project rule. The rule becomes an adverse rule from a systemic perspective.

Perverse incentives

Project rules typically include things like status reports, status meetings, timesheets and many other things. Breaking any project rule will trigger a project investigation, putting the developer's behaviour under scrutiny. This means that the developer will have even less time to work towards the business objective - which means they are in trouble. To avoid breaking adverse project rules and ending up in trouble, developers can only follow project regulations which are a distraction from business success. 

Staying out of trouble by causing trouble

As we discussed in the Virtuous circle, when a distraction is encountered, the only way forward is to accept the trouble and improve the rules. When adverse rules are cast in stone, moving into trouble is no progress. A developer has no option of getting out of the trouble domain, except by staying out of the trouble domain in the first place. Being in the distraction domain, the only way of not getting into trouble - is by staying distracted!

The consequence of distraction is trouble, and so people are shoving a whole bunch of trouble under the rug, hoping it will never surface. Business success is far out of reach.

Summary

Project organizations suffer the malady that a project has it's own goals, which distract from business goals. The rules that create a project structure result in a condition where distraction, i.e. avoiding business success, is the only desirable state that can be attained - even when the project runs perfectly!

A company can not succeed until they break this vicious circle. This means they need to abolish the rules required to run a project organization. This means that the project organization will no longer function.

Failing with a project organization is not the fault of project managers or those initiating the projects. It is an involuntary side effect of running projects.

You need to move away from project oriented structures towards product oriented structures as soon as you can.