An Organizational Measurement System

"How can we create a measurement system for an agile organization?"

There are more and less helpful approaches, and there's also a lot of situational context to consider. And yet, in the big picture, there are numerous metrics that (almost) universally make sense, at least to consider.

"Metrics" by craiyon.com

Metrics Context

Before exploring the metrics, we need to consider that each of these metrics applies in a specific context. For example, Strategy metrics are of secondary concern to team members, whereas Team metrics might be of secondary concern to strategic management. When talking about "secondary concern," that means: "We understand what's being measured, and we should be informed about relevant information coming out of these metrics, but our focus is elsewhere."

While in traditional enterprise contexts, these levels are often staffed with separate individuals, the contextual structure of the metric levels doesn't necessarily coincide with a hierarchy: In a startup, a single individual in a 1-person company might have to make decisions at any of the respective levels, and separation of concerns helps to clarify the context.

The measurement system

Different metrics are applicable at the different organizational levels, and they are only of limited value at others. We discern Leading Indicators and Lagging Indicators. The key difference between these two is that leading indicators enable predictions, whereas lagging indicators provide information that could trigger further inspection and potentially adaptation. A balanced mix of leading and lagging indicators in all core domains (Technology, Organization and Product) provide a solid basis for data driven decisions.

To keep this list somewhat manageable, some metrics, such as for example, "Flow," are comprised of an entire set of standard metrics encompassing this performance indicator.

A potential KPI system for an Agile organization

Layer	Technology Metrics	Organization Metrics	Product Metrics
Strategy	Technical Capability Development Flow	Strategic Risks and Impediments Strategic Information Quality Initiative Alignment to Strategy Budget Allocation Employee Empowerment and Engagement Execution Capability	Market Share Market Analysis Innovation Index
	Development Value Stream Performance Technology Attrition	Value Chain Performance Plan Execution Rate Investment Efficiency Strategic Risk and Impediment Resolution Flow Communication Effectiveness Employee Retention	Customer Retention and Growth Revenue Growth from Initiatives Objective Achievement
Portfolio	Technical Investment Plan Strategic Execution Alignment	Initiative Pipeline Capacity Forecast Portfolio Health Portfolio Risks Key Person Risks Competence Index	In-process Inventory Lifecycle Horizons Product Health Value Differentiation Customer Satisfaction Investment Decision Criteria Revenue Forecasts
	Product Releases Risk Variability	Ability to Execute Portfolio Delivery Flow Portfolio Risk Flow Strategic Initiative Budget Variance Portfolio Resource Utilization Efficiency	Initiative Health Hit-Miss Ratio Benefits Realization Customer Satisfaction Long-term Financial Performance
Product	Delivery Frequency Delivery Cycle Time Deployment Frequency Deployment Success Rate	Dependency Map Risk and Impediments Flow Risk Escalation Frequency	Product Backlog Size Feature Lead Time
	Release Frequency Release Quality Service Quality	Delivery Flow Efficiency In-Process Inventory Constraint Performance Product Risk Flow	Product-Market-Fit Product Usage Net Promoter Score (NPS) Feature Time-to-Market Customer Growth and Churn
Team	Continuous Delivery Performance Mean Time To Detect (MTTD) Mean Time To Restore (MTTR) Code Quality Automated Test Coverage	Skilling and Competencies Innovation and Ideation Rate Process Throughput Rate Known Risks and Impediments Impediment Resolution Throughput Impediment Escalation Rate Process Flow Efficiency	Customer Proximity Feedback Quality User Research
	Continuous Integration Performance Technical Debt Mean Time Between Failure (MTBF)	Ability to Execute Meeting Effectiveness Team Morale and Satisfaction Employee Retention Rate Process Throughput Yield	Inventory Dead Features Customer Engagement Customer Feedback

The organizational levels

This section contains information about the various organizational levels referenced above.

Strategy level

The Enterprise Strategy level defines the long-term direction, vision, and objectives of the entire enterprise. It plays a crucial role in setting the overall course for the organization and aligning it with its core values and mission. This level focuses on making strategic decisions that guide the allocation of resources, determine key initiatives, and ultimately drive the success and growth of the business.

In addition to defining the strategy, this level is also accountable for communicating the strategic direction throughout the organization. Effective communication ensures that all employees understand the vision and are aligned with the organization's goals. It also enables teams at lower levels to make decisions that are consistent with the overall strategic direction.

The Enterprise Strategy level contributes to the organization's success by providing a sense of direction and focus. It helps prioritize initiatives, allocate resources effectively, and avoid wasteful activities that do not align with the long-term goals. This level fosters a cohesive and unified organization that works towards a common purpose. It provides a roadmap for other levels of the organization, allowing them to plan and execute their work in alignment with the broader strategic objectives.

Portfolio Level

The Portfolio level in an organization is responsible for managing and optimizing the collection of projects, programs, and initiatives that collectively contribute to achieving the enterprise's strategic objectives. This level exists to ensure that resources, including financial, human, and technological, are allocated efficiently and effectively across various value streams and projects, aligning them with the overall business strategy.

Portfolio management plays a vital role in evaluating initiative proposals, assessing potential risks and benefits, and determining their strategic fit. This includes analysing factors such as expected outcomes, investments, resource requirements, and potential impacts on other ongoing initiatives.

Additionally, a portfolio fosters alignment and collaboration between different business units and departments by coordinating initiatives and ensuring they complement each other. The Portfolio level avoids duplication of efforts and helps maintain a clear focus on the enterprise's overall strategic direction. This enables the organization to invest in the right initiatives at the right time, and make informed decisions about staffing, resource allocation and project prioritization. This level also provides visibility and transparency into the status and performance of the entire project portfolio, helping stakeholders track progress and make data-driven decisions.

Portfolio agility facilitates strategic agility, allowing the organization to adapt its portfolio in response to changing market conditions and emerging opportunities. By regularly evaluating and adjusting the portfolio based on business needs and performance data, the Portfolio level ensures that the organization stays on course to achieve its strategic goals while remaining responsive to dynamic market forces.

Product Level

The Product level in an organization is focused on delivering value to customers through the creation, enhancement, and management of products and services. This level is dedicated to understanding customer needs and preferences, translating them into actionable product features, and ensuring that the delivered products meet high-quality standards and align with the overall business strategy.

At the Product level, product managers, product owners, and cross-functional teams work collaboratively to design and develop products that address customer pain points and fulfill market demands. They define the product vision, strategy, and roadmap, taking into account market research, customer feedback, data analytics, and business objectives.

The Product level is responsible for understanding market trends, customer preferences, and competitor analysis. They use this information to identify market opportunities and define a clear product vision that aligns with the enterprise's strategic goals.

At the product level, people work to deliver customer-centric products that create value and drive customer satisfaction. Through effective product management, this level ensures that products meet customer needs, are user-friendly, and stay competitive in the market. By continuously iterating and enhancing products based on customer feedback, the Product level helps retain existing customers and attract new ones.

The Product level plays a pivotal role in organizational success by bridging the gap between customer needs and business strategy. It aligns product development efforts with the overall enterprise goals, enabling the organization to build and deliver products that provide value, gain a competitive advantage, and sustain long-term growth and profitability.

Team Level

The Team level in an organization is the operational level where work is executed and tangible results are produced. Each team comprises collectively works towards achieving specific product goals.

Successful teams focus on continuous delivery of working, high quality, shippable product increments. Constant and early feedback from stakeholders and customers enables them to iterate and improve the product rapidly.

The Team level contributes to the organization's success by being the driving force behind the actual creation and delivery of value to customers. By embracing Agile principles and practices, teams become more adaptable and responsive to changing requirements and customer needs. An iterative approach allows them to identify and address issues early, reducing the likelihood of costly late-stage defects.

Ultimately, the success of every organization depends on its teams. Teams are those who have to deliver products on time, operate within given budgets, and meet customer expectations. Self-organised, highly motivated, capable individuals are the keys to excellence in product development.

Closing remarks

While on the one hand, additional metrics mean additional effort - on the other hand, inattention to crucial metrics may lopside and bias decisions and lead to a "Cobra Effect" - that is, people do what makes sense based on the metrics to the detriment of an organization's success. Hence, choosing the right metrics and adequately balancing them is a delicate process.

Setting up a measurement system suitable to your specific organization can't follow a universal recipe - considering context, such as market influences, industry specific constraints and organizational culture is highly relevant to successful measurement. Likewise, abstraction levels should only be institutionalized where necessary.
Are you looking for further guidance? Don't hesistate to reach out to me!

Dealing with Organizational Debt

"Organizational debt" is a metaphorical term used to describe the accumulation of inefficiencies, shortcomings, and suboptimal practices within an organization over time. Similar to technical debt, it refers to the consequences of choosing expedient solutions that will require revisiting and improving the situation later on.

Organizational debt is often created by a desire to "just make it work:" when people opt for quick and convenient solutions to address immediate needs or challenges, they often ignore the long-term consequences. Effectiveness, scalability, and sustainability are often not considered in the heat of the moment. While shortcuts and compromises keep things going, a failure to address the systemic impact of these decisions will eventually take a massive toll on the organization's ability to operate efficiently and adapt to changing circumstances in the future.

"Organizational Debt" image created with craiyon.com

How can we deal with
Organizational Debt?

The following table is a guide which you can use to determine whether you have organizational debt, and how you can address it.

How to identify and address Organizational Debt

Organizational Element	Organizational Debt Indicators	Potential Remedial Actions
Purpose and Mission	Lack of clear mission statement Vague objectives or conflicting goals Disconnect between company mission and daily work	Clarify the organization's mission and objectives Communicate the mission effectively to all members Align goals across departments Identify and explore gaps between vision, mission and execution
Structure	Complex and rigid hierarchy Unclear reporting lines Overlapping roles	Make the hierarchy less felt Streamline the organizational structure Clarify roles and responsibilities
Leadership and Governance	Lack of transparency Poor decision-making processes Leadership conflicts	Foster transparent communication Implement clear decision-making protocols Address leadership issues promptly and proactively
People	High turnover rates Low employee morale Skill gaps within the workforce	Create open feedback channels to proactively resolve dissatisfaction Improve employee engagement and recognition programs Invest in training and development opportunities
Culture and Values	Unhealthy work environment Lack of shared values Lack of identification with values	Cultivate a positive and inclusive culture Reinforce core values through internal communication Lead values by example
Processes and Methods	Inefficient workflows Lack of process clarity Outdated procedures	Identify and resolve bottlenecks Frequently revisit and improve processes Document and communicate standards
Resources	Limited budget Inadequate technology Inefficient resource allocation	Analyze resource allocation and prioritize essential investments Seek cost-saving opportunities without compromising quality Embrace innovation to optimize resource utilization
Stakeholders	Poor customer satisfaction Strained vendor relationships Disengagement	Collect feedback from stakeholders and act on it Enhance customer support and engagement strategies Involve communities in decision-making
Communication	Ineffective communication channels Information silos Miscommunication	Optimize communication channels and protocols Encourage open and transparent communication across all levels Use collaboration tools to facilitate information sharing
Adaptability and Innovation	Resistance to change Reluctance to embrace new technologies Outdated practices	Foster an intrapreneurial culture of innovation and risk-taking Institutionalize grass roots innovation Invest in ongoing training and education
Measurement and Evaluation	Inadequate metrics Unsuitable performance tracking Not relying on facts	Identify relevant success metrics Conduct regular reviews to inspect and adapt Adopt data-driven decisions-making and improvements
Ethical Responsibility	Embellishing outcomes Passing the Buck Failure to address concerns	Provide "psychological safety" Encourage opennenss and non-judgmentalism Develop and promote a code of ethics

Don't hesitate to reach out if you need coaching on how to do this in practice.

Why Agile Coaches need to care about Delivery Speed

I often come across the sentiment that "Agile is not about speed of delivery." I believe that claim is a severe misunderstanding on what makes a team agile - and worse: it's already an early warning sign that at some point in the future, the Agile Coach will struggle to explain what exactly they're doing. Posing a false dichotomy between Agile and Cycle Time sets up their teams to underperform in comparison to teams coached by someone who understands both the impact of Cycle Time, and how to actively reduce it.

An Agile Coach setting themselves up for failure

Cycle Time Matters

Many traditional organizations I encounter still apply outdated delivery models, often stage-gated with various in-process queues and handovers. When a company takes half a year from demand to release - I'd say they're already well above average. In fact, a one-plus year delay from idea to value isn't uncommon. In such scenarios, managers and stakeholders often request their coaches to actively address time to market as a key success metric. For good reasons.

When we talk about agility, speed of delivery plays a crucial role. Cycle Time, which measures the time from development to production, directly impacts an organization's ability to respond quickly to customer needs, market changes, and emerging opportunities. An organization's agility is closely tied to the ability to rapidly deliver value to customers. The quicker they can do this, the more responsive and competitive they become. It's essential for Agile Coaches and Scrum Masters to recognize that optimizing Cycle Time is not a deviation from Agile principles but rather an integral part of fostering agility.

Reducing Cycle Time has several benefits to agility: First, it allows us to obtain customer feedback faster and more often, enabling them to validate assumptions, make adjustments, and iterate more rapidly. This feedback loop facilitates continuous learning and ensures that the delivered software meets customer expectations and quality standards. Shorter Cycle Times also reduce the delay between occurrence and resolution of risks and issues. Additionally, short cycle times reveal potential bottlenecks and delays. All these factors reduce rework and improve our ability to create value.

Furthermore, actively minimizing Cycle Time enhances adaptability and responsiveness to changes in the market. Rapid delivery enables organizations to iteratively experiment, learn, and adapt their product or service offerings based on real-time feedback. This fosters innovation and competitiveness. Agile Coaches who understand these aspects will guide their teams to optimize Cycle Time as a means to promote agility.

Don't make it a false dichotomy!

Assuming that "Agile Mindset" and "Delivery Time Optimization" are at odds would be a false dichotomy, but let's entertain the thought and see what the long-term consequences would be for a team whose coach would make an either-or decision, and focus on either one, or the other. In our scenario, we will assume that the Delivery Time is currently so slow that management is correct in being concerned - i.e., that the team is struggling to deliver one releaseable increment per month, and that Delivery Time Optimization would orient itself on Minimum Continuous Delivery requirements. These given, let's take a look at business relevant outcomes and how they'd develop over months and years:

Metric	Explanation	"Delivery Time" Focused Coaching	"Agile Mindset" Focused Coaching
Time to Market	Timespan between idea and value.	Strong - More deliveries, reduced batch sizes and shorter wait times. Faster commercialization.	Variable - "Results not guaranteed."
Product Quality	Software meeting standards and expectations.	Strong - High delivery frequency requires effective quality practices like automated verification.	Possibly - No direct measurement or systematic approach to address quality.
Customer Satisfaction	Maximizing feedback points while minimizing low-quality experiences.	Strong - Rapid delivery with definitive quality verdicts enables better control of dissatisfaction factors.	Limited - Limited opportunities to improve customer satisfaction through controlled delivery points.
Feedback Integration	Collecting input on ideas and Working Software.	Strong - Abundant and timely feedback, potentially multiple times per day, depending on stakeholder availability.	Uncontrolled - Feedback effectiveness unquantifiably tied to delivery process effectiveness.
Adaptability and Agility	Speed and accuracy in acting upon learnings.	Strong - High delivery frequency fosters adaptability and enables effective change implementation.	Poor - Limited means to determine the level of adaptability.
Collaboration and Alignment	Staying in sync and acting in unison.	Strong - Continuous Delivery exposes lack of alignment or collaboration through pipeline failures.	Difficult to quantify - "Soft" collaboration with unmeasurable outcomes.
Risk Reduction	Minimizing impact of issues and delays.	Strong - Mitigation of risks through proactive risk analysis and automated testing.	Poor - No inherent risk control mechanisms.

Verdict

Agile Coaches who discount speed of delivery will struggle

Agile Coaches who primarily focus on promoting the Agile Mindset without actively driving speed of delivery and its enabling technical practices and processes, such as Continuous Delivery, will likely struggle when asked to show clear, measurable outcomes. Here's why:

1. Lack of Transparency: Stakeholders and decision-makers often require evidence of improvements in key metrics, such as time to market, product quality, customer satisfaction, or change failure rate. Without indicative metrics that demonstrate the effectiveness of technical practices like Continuous Delivery, it becomes challenging to achieve significant improvements in these areas, making it harder for the coach to showcase the impact of their work.
2. Inability to Improve: Sustainable speed of delivery is a lagging indicator for the adequacy and quality of a team's technical practices. The Agile Coach ignoring speed will limit their ability to help teams meet stakeholder expectations. This leads to missed opportunities, increased lead time, and decreased competitiveness, rendering their coaching ineffective.
3. Limited Control over Quality and Risk: Teams that don't implement the practices supporting a rapid succession of deliveries will struggle to address quality issues and effectively manage risks. Systematic quality control mechanisms and risk mitigation strategies are essential enablers to consistently delivering high-quality products, hence scrutinizing and optimizing sustainable speed of delivery creates a strong focus on implementing the necessary supporting practices.
4. Inadequate Feedback and Collaboration: Rapid feedback and effective collaboration are essential drivers of agility. The Agile Coach who solely focuses on the Agile Mindset may overlook the importance of technical practices that enable fast feedback cycles and promote collaboration.
5. Limited Adaptability and Agility: Agility relies on an ability to adequately respond to changing market dynamics. Sustained delivery speed, supported by its enabling technical practices, is a powerful leading indicator for the level of adaptability and implementing effective change. Without leveraging practices that support rapid delivery and iterative improvement, the coach may hinder the team's ability to learn, adjust, and continuously improve.

Conclusion

The Agile Coach who neglects Sustained Delivery Speed and its incorporating technical practices, such as Continuous Delivery, is much more likely to struggle in demonstrating significant long-term outcomes. They may miss critical gaps in quality practices, feedback mechanisms, collaboration, and adaptability, thus leading to predictable challenges in meeting stakeholder expectations, driving improvements, and effective agility.

Important: This article consciously emphasizes "Sustained Delivery Speed." Our concern isn't the speed of typing, but the inherent capability of minimizing the time required for turning an idea into a high quality release candidate. Cutting corners to get one shipment through the door will quickly undermine a team's ability to maintain a high pace of deliveries - this tactic always results in incidents and failures which devastate a team's ability to maintain their pace. Hence, sustained delivery speed is an aggregate measurement that requires observing many technical metrics "under the hood," such as build time, build failures, incident frequency, recovery rates, and many others.

Leading by Absence

You may be familiar with classical leadership approaches - such as leading by doing (being in the trenches, doing the same as you expect others to do), or leading by example (which is slightly different, as you show patterns for others should follow) - but have you ever thought about "leading by absence?" This technique is very important for Scrum Masters and Managers alike to grow teams and personalities. If you're a parent, you may be familiar with how necessary, yet difficult this approach is.

Let's explore how this approach can positively impact leadership dynamics and enhance team dynamics.

How to Lead by Absence

The concept of "leading by absence" refers to a leadership approach where a leader intentionally creates space and provides leeway for their team members to take ownership and make decisions in their absence. Leading by absence means you steps back and allows others to take the lead and responsibility, while still providing guidance and support as necessary.

Leadership by Absence is a delicate act of balance

"Leading by absence" recognizes that effective leadership isn't about being at the forefront or making decisions. Instead, it acknowledges the importance of empowering others and fostering a sense of autonomy, trust, and accountability within the team. By giving team members the opportunity to lead and make their own decisions, leaders focus on nurturing their growth, developing others' skills, and building a more self-reliant and resilient team.

These are some key aspects of leading by absence:

Delegation

To lead by absence, team members need to be clear which tasks and responsibilities they are expected to take over. The leader trusts their team's capabilities and gives them the autonomy to make decisions within their assigned roles.

Support and guidance

Those who lead by absence deliberately step back and create space, while still providing support and guidance when needed. They offer assistance, clarify expectations, and provide resources or feedback to help their team members succeed.

Empowerment

Leaders who choose to be absent aim to empower team members by fostering a culture of ownership and accountability. It encourages individuals to take initiative, make decisions, and contribute their unique perspectives.

Trust-building

Leaders who practice leading by absence build trust with their team members. They demonstrate confidence in their abilities and create an environment where individuals feel valued and supported, which increases motivation and engagement.

Continuous learning

This leadership approach relies on opportunities for learning and growth. Leaders encourage their team members to learn from their experiences, both successes and failures, and promote a culture of ongoing improvement and development.

Conclusion

Leaders who practice "leading by absence" promote collaboration, innovation, and individual growth within their teams. They leverage the diverse skills and talents of their team members while fostering a sense of ownership and shared responsibility for achieving goals.

Note of caution:

Despite the similar name, "Leading by Absence" is the opposite of "Absence of Leadership:" It's a deliberate choice that requires careful consideration and patience. Wheras it empowers and enables teams, an absence of leadership implies a lack of guidance, direction, and support.

To put these into contrast: Leaders who lead by absence create a space for others to step up and become more successful, whereas an absence of leadership puts others into turmoil and endangers their success. Effective leaders strike a balance between providing autonomy and support, ensuring that team members have the necessary resources, guidance, and clarity to thrive in their roles.

Flush the System for Agility!

A key activity that needs to stand at the beginning of every Agile Journey: "Flush the System." What that does? Well - it lays the groundwork so that working in an agile way is even possible.

Areas of investigation

The following areas can be considered "the usual suspects" where flushing activities may be necessary. Please be aware that terminating something is often the easy part - the challenging, and time-consuming part, is obtaining stakeholder support (and potentially: permission) to do so.

Work

Often, at the beginning of an agile journey, people are highly overburdened and it's not even clear who is working on what for how long. That makes it difficult to get anything done - so we need to clean up!

	Itemize ongoing work and complete or cancel tasks accordingly.
	Hand over tasks and responsibilities outside the scope of the Agile team.
	Terminate processes that aren't aligned with Agile ways of working.

Calendar

In many organizations, meetings are so rampant that there's no time left to do any work. And the more knowledge a person has, the more likely they're in this dilemma. Without sufficient focus time for getting any meaningful, valuable work - there's not going to be much "Agile" delivery.

	Cancel unnecessary meetings, such as status or reporting meetings, Jour Fixes, and others.
	Cancel meetings with intended outcomes that should be achieved during Agile events (e.g., Planning, Refinement, Review).
	Cancel meetings with unclear or irrelevant outcomes.

Roles and Responsibilities

Roles and responsibilities need to be clarified and aligned with Agile principles and be made consistent with the new way of working. Unnecessary or counterproductive responsibilities lead to confusion, overhead and conflict.

	Hand over responsibilities that aren't in the scope of the newly coming Agile team.
	Remove unnecessary or counterproductive responsibilities.
	Eliminate "telephone games" in roles and responsibilities that impede information and decision flow.

Incentives and Measurement

Incentives and KPI's often surface as an impediment for collaboration and a focus on value. Historically, these are often founded on beliefs that don't coincide well with Agile Values and Principles, hence cleaning these out will become critical to shaping a better system of work.

	Eliminate KPI's and management metrics distracting from value creation.
	Cancel irrelevant or counterproductive goals.
	Minimize the impact of (or preferably eliminate) incentive schemes and bonuses.

Take action!

As you embark on your Agile journey, "Flushing the System" creates a supportive environment that helps Agile teams to thrive. The above checklist is a powerful guide to streamline work, optimize their calendars, clarify roles, and align outcomes.
This list isn't exhaustive - merely a starting point, and you may find a lot more items you can flush as you look around in your organization.
Remember: "Flushing the System" isn't a one-time event, but part of the greater process of continuous improvement. Regularly assess and refine your system of work to ensure you're not missing "flushing" potential that will lead to better flow.