An Organizational Measurement System

"How can we create a measurement system for an agile organization?"

There are more and less helpful approaches, and there's also a lot of situational context to consider. And yet, in the big picture, there are numerous metrics that (almost) universally make sense, at least to consider.

"Metrics" by craiyon.com

Metrics Context

Before exploring the metrics, we need to consider that each of these metrics applies in a specific context. For example, Strategy metrics are of secondary concern to team members, whereas Team metrics might be of secondary concern to strategic management. When talking about "secondary concern," that means: "We understand what's being measured, and we should be informed about relevant information coming out of these metrics, but our focus is elsewhere."

While in traditional enterprise contexts, these levels are often staffed with separate individuals, the contextual structure of the metric levels doesn't necessarily coincide with a hierarchy: In a startup, a single individual in a 1-person company might have to make decisions at any of the respective levels, and separation of concerns helps to clarify the context.

The measurement system

Different metrics are applicable at the different organizational levels, and they are only of limited value at others. We discern Leading Indicators and Lagging Indicators. The key difference between these two is that leading indicators enable predictions, whereas lagging indicators provide information that could trigger further inspection and potentially adaptation. A balanced mix of leading and lagging indicators in all core domains (Technology, Organization and Product) provide a solid basis for data driven decisions.

To keep this list somewhat manageable, some metrics, such as for example, "Flow," are comprised of an entire set of standard metrics encompassing this performance indicator.

A potential KPI system for an Agile organization

Layer	Technology Metrics	Organization Metrics	Product Metrics
Strategy	Technical Capability Development Flow	Strategic Risks and Impediments Strategic Information Quality Initiative Alignment to Strategy Budget Allocation Employee Empowerment and Engagement Execution Capability	Market Share Market Analysis Innovation Index
	Development Value Stream Performance Technology Attrition	Value Chain Performance Plan Execution Rate Investment Efficiency Strategic Risk and Impediment Resolution Flow Communication Effectiveness Employee Retention	Customer Retention and Growth Revenue Growth from Initiatives Objective Achievement
Portfolio	Technical Investment Plan Strategic Execution Alignment	Initiative Pipeline Capacity Forecast Portfolio Health Portfolio Risks Key Person Risks Competence Index	In-process Inventory Lifecycle Horizons Product Health Value Differentiation Customer Satisfaction Investment Decision Criteria Revenue Forecasts
	Product Releases Risk Variability	Ability to Execute Portfolio Delivery Flow Portfolio Risk Flow Strategic Initiative Budget Variance Portfolio Resource Utilization Efficiency	Initiative Health Hit-Miss Ratio Benefits Realization Customer Satisfaction Long-term Financial Performance
Product	Delivery Frequency Delivery Cycle Time Deployment Frequency Deployment Success Rate	Dependency Map Risk and Impediments Flow Risk Escalation Frequency	Product Backlog Size Feature Lead Time
	Release Frequency Release Quality Service Quality	Delivery Flow Efficiency In-Process Inventory Constraint Performance Product Risk Flow	Product-Market-Fit Product Usage Net Promoter Score (NPS) Feature Time-to-Market Customer Growth and Churn
Team	Continuous Delivery Performance Mean Time To Detect (MTTD) Mean Time To Restore (MTTR) Code Quality Automated Test Coverage	Skilling and Competencies Innovation and Ideation Rate Process Throughput Rate Known Risks and Impediments Impediment Resolution Throughput Impediment Escalation Rate Process Flow Efficiency	Customer Proximity Feedback Quality User Research
	Continuous Integration Performance Technical Debt Mean Time Between Failure (MTBF)	Ability to Execute Meeting Effectiveness Team Morale and Satisfaction Employee Retention Rate Process Throughput Yield	Inventory Dead Features Customer Engagement Customer Feedback

The organizational levels

This section contains information about the various organizational levels referenced above.

Strategy level

The Enterprise Strategy level defines the long-term direction, vision, and objectives of the entire enterprise. It plays a crucial role in setting the overall course for the organization and aligning it with its core values and mission. This level focuses on making strategic decisions that guide the allocation of resources, determine key initiatives, and ultimately drive the success and growth of the business.

In addition to defining the strategy, this level is also accountable for communicating the strategic direction throughout the organization. Effective communication ensures that all employees understand the vision and are aligned with the organization's goals. It also enables teams at lower levels to make decisions that are consistent with the overall strategic direction.

The Enterprise Strategy level contributes to the organization's success by providing a sense of direction and focus. It helps prioritize initiatives, allocate resources effectively, and avoid wasteful activities that do not align with the long-term goals. This level fosters a cohesive and unified organization that works towards a common purpose. It provides a roadmap for other levels of the organization, allowing them to plan and execute their work in alignment with the broader strategic objectives.

Portfolio Level

The Portfolio level in an organization is responsible for managing and optimizing the collection of projects, programs, and initiatives that collectively contribute to achieving the enterprise's strategic objectives. This level exists to ensure that resources, including financial, human, and technological, are allocated efficiently and effectively across various value streams and projects, aligning them with the overall business strategy.

Portfolio management plays a vital role in evaluating initiative proposals, assessing potential risks and benefits, and determining their strategic fit. This includes analysing factors such as expected outcomes, investments, resource requirements, and potential impacts on other ongoing initiatives.

Additionally, a portfolio fosters alignment and collaboration between different business units and departments by coordinating initiatives and ensuring they complement each other. The Portfolio level avoids duplication of efforts and helps maintain a clear focus on the enterprise's overall strategic direction. This enables the organization to invest in the right initiatives at the right time, and make informed decisions about staffing, resource allocation and project prioritization. This level also provides visibility and transparency into the status and performance of the entire project portfolio, helping stakeholders track progress and make data-driven decisions.

Portfolio agility facilitates strategic agility, allowing the organization to adapt its portfolio in response to changing market conditions and emerging opportunities. By regularly evaluating and adjusting the portfolio based on business needs and performance data, the Portfolio level ensures that the organization stays on course to achieve its strategic goals while remaining responsive to dynamic market forces.

Product Level

The Product level in an organization is focused on delivering value to customers through the creation, enhancement, and management of products and services. This level is dedicated to understanding customer needs and preferences, translating them into actionable product features, and ensuring that the delivered products meet high-quality standards and align with the overall business strategy.

At the Product level, product managers, product owners, and cross-functional teams work collaboratively to design and develop products that address customer pain points and fulfill market demands. They define the product vision, strategy, and roadmap, taking into account market research, customer feedback, data analytics, and business objectives.

The Product level is responsible for understanding market trends, customer preferences, and competitor analysis. They use this information to identify market opportunities and define a clear product vision that aligns with the enterprise's strategic goals.

At the product level, people work to deliver customer-centric products that create value and drive customer satisfaction. Through effective product management, this level ensures that products meet customer needs, are user-friendly, and stay competitive in the market. By continuously iterating and enhancing products based on customer feedback, the Product level helps retain existing customers and attract new ones.

The Product level plays a pivotal role in organizational success by bridging the gap between customer needs and business strategy. It aligns product development efforts with the overall enterprise goals, enabling the organization to build and deliver products that provide value, gain a competitive advantage, and sustain long-term growth and profitability.

Team Level

The Team level in an organization is the operational level where work is executed and tangible results are produced. Each team comprises collectively works towards achieving specific product goals.

Successful teams focus on continuous delivery of working, high quality, shippable product increments. Constant and early feedback from stakeholders and customers enables them to iterate and improve the product rapidly.

The Team level contributes to the organization's success by being the driving force behind the actual creation and delivery of value to customers. By embracing Agile principles and practices, teams become more adaptable and responsive to changing requirements and customer needs. An iterative approach allows them to identify and address issues early, reducing the likelihood of costly late-stage defects.

Ultimately, the success of every organization depends on its teams. Teams are those who have to deliver products on time, operate within given budgets, and meet customer expectations. Self-organised, highly motivated, capable individuals are the keys to excellence in product development.

Closing remarks

While on the one hand, additional metrics mean additional effort - on the other hand, inattention to crucial metrics may lopside and bias decisions and lead to a "Cobra Effect" - that is, people do what makes sense based on the metrics to the detriment of an organization's success. Hence, choosing the right metrics and adequately balancing them is a delicate process.

Setting up a measurement system suitable to your specific organization can't follow a universal recipe - considering context, such as market influences, industry specific constraints and organizational culture is highly relevant to successful measurement. Likewise, abstraction levels should only be institutionalized where necessary.
Are you looking for further guidance? Don't hesistate to reach out to me!