Pages

Sunday, July 30, 2023

An Organizational Measurement System

"How can we create a measurement system for an agile organization?"

There are more and less helpful approaches, and there's also a lot of situational context to consider. And yet, in the big picture, there are numerous metrics that (almost) universally make sense, at least to consider.

"Metrics" by craiyon.com

Metrics Context

Before exploring the metrics, we need to consider that each of these metrics applies in a specific context. For example, Strategy metrics are of secondary concern to team members, whereas Team metrics might be of secondary concern to strategic management. When talking about "secondary concern," that means: "We understand what's being measured, and we should be informed about relevant information coming out of these metrics, but our focus is elsewhere."

While in traditional enterprise contexts, these levels are often staffed with separate individuals, the contextual structure of the metric levels doesn't necessarily coincide with a hierarchy: In a startup, a single individual in a 1-person company might have to make decisions at any of the respective levels, and separation of concerns helps to clarify the context.

The measurement system

Different metrics are applicable at the different organizational levels, and they are only of limited value at others. We discern Leading Indicators and Lagging Indicators. The key difference between these two is that leading indicators enable predictions, whereas lagging indicators provide information that could trigger further inspection and potentially adaptation. A balanced mix of leading and lagging indicators in all core domains (Technology, Organization and Product) provide a solid basis for data driven decisions.

To keep this list somewhat manageable, some metrics, such as for example, "Flow," are comprised of an entire set of standard metrics encompassing this performance indicator.

A potential KPI system for an Agile organization
Layer Technology Metrics Organization Metrics Product Metrics
Strategy
Technical Capability
Development Flow
Strategic Risks and Impediments
Strategic Information Quality
Initiative Alignment to Strategy
Budget Allocation
Employee Empowerment and Engagement
Execution Capability
Market Share
Market Analysis
Innovation Index
Development Value Stream Performance
Technology Attrition
Value Chain Performance
Plan Execution Rate
Investment Efficiency
Strategic Risk and Impediment Resolution Flow
Communication Effectiveness
Employee Retention
Customer Retention and Growth
Revenue Growth from Initiatives
Objective Achievement
Portfolio
Technical Investment Plan
Strategic Execution Alignment
Initiative Pipeline
Capacity Forecast
Portfolio Health
Portfolio Risks
Key Person Risks
Competence Index
In-process Inventory
Lifecycle Horizons
Product Health
Value Differentiation
Customer Satisfaction
Investment Decision Criteria
Revenue Forecasts
Product Releases
Risk Variability
Ability to Execute
Portfolio Delivery Flow
Portfolio Risk Flow
Strategic Initiative Budget Variance
Portfolio Resource Utilization Efficiency
Initiative Health
Hit-Miss Ratio
Benefits Realization
Customer Satisfaction
Long-term Financial Performance
Product
Delivery Frequency
Delivery Cycle Time
Deployment Frequency
Deployment Success Rate
Dependency Map
Risk and Impediments Flow
Risk Escalation Frequency
Product Backlog Size
Feature Lead Time
Release Frequency
Release Quality
Service Quality
Delivery Flow Efficiency
In-Process Inventory
Constraint Performance
Product Risk Flow
Product-Market-Fit
Product Usage
Net Promoter Score (NPS)
Feature Time-to-Market
Customer Growth and Churn
Team
Continuous Delivery Performance
Mean Time To Detect (MTTD)
Mean Time To Restore (MTTR)
Code Quality
Automated Test Coverage
Skilling and Competencies
Innovation and Ideation Rate
Process Throughput Rate
Known Risks and Impediments
Impediment Resolution Throughput
Impediment Escalation Rate
Process Flow Efficiency
Customer Proximity
Feedback Quality
User Research
Continuous Integration Performance
Technical Debt
Mean Time Between Failure (MTBF)
Ability to Execute
Meeting Effectiveness
Team Morale and Satisfaction
Employee Retention Rate
Process Throughput Yield
Inventory
Dead Features
Customer Engagement
Customer Feedback

The organizational levels

This section contains information about the various organizational levels referenced above.

Strategy level

The Enterprise Strategy level defines the long-term direction, vision, and objectives of the entire enterprise. It plays a crucial role in setting the overall course for the organization and aligning it with its core values and mission. This level focuses on making strategic decisions that guide the allocation of resources, determine key initiatives, and ultimately drive the success and growth of the business.

In addition to defining the strategy, this level is also accountable for communicating the strategic direction throughout the organization. Effective communication ensures that all employees understand the vision and are aligned with the organization's goals. It also enables teams at lower levels to make decisions that are consistent with the overall strategic direction.

The Enterprise Strategy level contributes to the organization's success by providing a sense of direction and focus. It helps prioritize initiatives, allocate resources effectively, and avoid wasteful activities that do not align with the long-term goals. This level fosters a cohesive and unified organization that works towards a common purpose. It provides a roadmap for other levels of the organization, allowing them to plan and execute their work in alignment with the broader strategic objectives.

Portfolio Level

The Portfolio level in an organization is responsible for managing and optimizing the collection of projects, programs, and initiatives that collectively contribute to achieving the enterprise's strategic objectives. This level exists to ensure that resources, including financial, human, and technological, are allocated efficiently and effectively across various value streams and projects, aligning them with the overall business strategy.

Portfolio management plays a vital role in evaluating initiative proposals, assessing potential risks and benefits, and determining their strategic fit. This includes analysing factors such as expected outcomes, investments, resource requirements, and potential impacts on other ongoing initiatives.

Additionally, a portfolio fosters alignment and collaboration between different business units and departments by coordinating initiatives and ensuring they complement each other. The Portfolio level avoids duplication of efforts and helps maintain a clear focus on the enterprise's overall strategic direction. This enables the organization to invest in the right initiatives at the right time, and make informed decisions about staffing, resource allocation and project prioritization. This level also provides visibility and transparency into the status and performance of the entire project portfolio, helping stakeholders track progress and make data-driven decisions.

Portfolio agility facilitates strategic agility, allowing the organization to adapt its portfolio in response to changing market conditions and emerging opportunities. By regularly evaluating and adjusting the portfolio based on business needs and performance data, the Portfolio level ensures that the organization stays on course to achieve its strategic goals while remaining responsive to dynamic market forces.

Product Level

The Product level in an organization is focused on delivering value to customers through the creation, enhancement, and management of products and services. This level is dedicated to understanding customer needs and preferences, translating them into actionable product features, and ensuring that the delivered products meet high-quality standards and align with the overall business strategy.

At the Product level, product managers, product owners, and cross-functional teams work collaboratively to design and develop products that address customer pain points and fulfill market demands. They define the product vision, strategy, and roadmap, taking into account market research, customer feedback, data analytics, and business objectives.

The Product level is responsible for understanding market trends, customer preferences, and competitor analysis. They use this information to identify market opportunities and define a clear product vision that aligns with the enterprise's strategic goals.

At the product level, people work to deliver customer-centric products that create value and drive customer satisfaction. Through effective product management, this level ensures that products meet customer needs, are user-friendly, and stay competitive in the market. By continuously iterating and enhancing products based on customer feedback, the Product level helps retain existing customers and attract new ones.

The Product level plays a pivotal role in organizational success by bridging the gap between customer needs and business strategy. It aligns product development efforts with the overall enterprise goals, enabling the organization to build and deliver products that provide value, gain a competitive advantage, and sustain long-term growth and profitability.

Team Level

The Team level in an organization is the operational level where work is executed and tangible results are produced. Each team comprises collectively works towards achieving specific product goals.

Successful teams focus on continuous delivery of working, high quality, shippable product increments. Constant and early feedback from stakeholders and customers enables them to iterate and improve the product rapidly.

The Team level contributes to the organization's success by being the driving force behind the actual creation and delivery of value to customers. By embracing Agile principles and practices, teams become more adaptable and responsive to changing requirements and customer needs. An iterative approach allows them to identify and address issues early, reducing the likelihood of costly late-stage defects.

Ultimately, the success of every organization depends on its teams. Teams are those who have to deliver products on time, operate within given budgets, and meet customer expectations. Self-organised, highly motivated, capable individuals are the keys to excellence in product development.

Closing remarks

While on the one hand, additional metrics mean additional effort - on the other hand, inattention to crucial metrics may lopside and bias decisions and lead to a "Cobra Effect" - that is, people do what makes sense based on the metrics to the detriment of an organization's success. Hence, choosing the right metrics and adequately balancing them is a delicate process.

Setting up a measurement system suitable to your specific organization can't follow a universal recipe - considering context, such as market influences, industry specific constraints and organizational culture is highly relevant to successful measurement. Likewise, abstraction levels should only be institutionalized where necessary.
Are you looking for further guidance? Don't hesistate to reach out to me!

1 comment:

  1. michael,

    thanks for this writeup!

    some questions:
    - ist this oriented towards the coaching competency framework w tech, business and org mastery?
    - what's the difference between the product layer and product metrics?
    - can the product layer be somehow related to a value stream/ art/ tribe?
    - what's behind the tech metric 'product releases' in the portfolio layer?
    - what's behind the 'dev vs performance' metric in the strategy layer?
    and probably some others :D

    anyways, lots of food for thought,
    much appreciated

    ReplyDelete