Wednesday, February 25, 2026

Agentic Risk Class Postures

When dealing with Agentic AI, many companies struggle with one basic question: what governance posture should we take? Some say, “Agents are dangerous,” and get labeled naysayers. Others want to “go find out,” and get labeled reckless.

Both impulses have merit. But most agentic implementations require a position in between: contain the danger - and avoid analysis paralysis.

The key differentiator is simple: Can you deal with the consequences?

You don’t want to find out

So here’s the thing: If we don’t experiment, how can we learn?
We may never realize the full benefits of this Agentic AI technology if we refuse to take any risk.

But risks are not created equal. Some risks we can live with. Others, we can't.

And depending on our authority and accountability, some moves are within our control - and others are not even ours to decide. That's why not all risk postures are equal.

It depends on who's taking the risk, why - and whether it is worth it.

Sometimes the predictable consequences are not worth the upside. Other times, the effort required to eliminate possible consequences is prohibitive. That leaves only two honest options: accept risk consciously - or redesign the solution.

To make that decision less emotional and more structural, I've created a simple table. It shows how different agent risk tiers change the way we govern the agent across the Software Lifecycle.

The Agentic Governance Posture

Below is a concise summary of how governance mode, testing depth, nonfunctional requirements, and service operations expectations shift depending on the agent’s risk class.

This model helps you avoid two common failure modes: over-engineering containment for harmless agents, and under-engineering control for agents that can hurt you.

It applies at enterprise scale — and it applies to the agentic tool you just downloaded to make your own life easier.

Ultimately, the choice is yours: risk-positive, risk-conservative, or risk-averse. Just make the choice consciously - before you enroll in the School of Hard Knocks.

Risk Tier Governance Mode Testing Requirements Nonfunctional Requirements Service Operations Requirements
1 - Contained Single accountable owner.
Authority is strictly local.
Worst case: time lost, no external consequence.
Boundary enforcement tests (no unintended writes).
Functional correctness tests.
Basic sanity validation.
Transparent execution.
Predictable behavior.
No hidden side effects.
Operations tied to active use.
Owner directly observes and controls execution.
2 - Operational Delegated authority within explicitly defined scope.
Named accountable authority for outcomes.
Policy-boundary tests.
Reversibility verification.
Negative-case testing.
Threshold and rate-limit tests.
Observability (metrics, logs, traces).
Defined performance expectations.
Traceable execution history.
Known escalation contact.
Kill switch available.
Incident and rollback procedures defined.
3 - Cross-Cutting Authority spans multiple systems or stakeholders.
Formal risk review required.
Active ownership across affected domains.
End-to-end process testing (including negative cases).
Cross-system integration tests.
Drift detection tests.
Exposure and impact simulations.
Full audit trail of decisions and actions.
Decision records retained.
Guardrail monitoring with violation alerts.
Managed change process.
Regular control reviews.
Formal escalation pathways.
4 - Strategic Oversight by the highest accountable authority in the system.
Explicit delegation charter.
Documented risk acceptance.
Scenario simulation before deployment.
Pre-release oversight review.
Segregation-of-authority verification.
Engineered resilience and reliability.
Governance attestations.
Enterprise-grade auditability.
24/7 readiness for high-impact incidents.
Crisis response protocol.
Clear communication pathways.
5 - Sovereign Authority reaches or exceeds the system’s sovereignty boundary.
Multi-step or multi-party authorization required.
Default posture: "air-gapped" control.
Proof that unilateral execution is technically impossible.
Bypass-resistance verification.
Enforced separation of authority.
Non-bypassable controls.
Immutable audit trail.
Highest assurance architecture.
Immediate containment capability.
Escalation to consequence-bearing authority.
Hard shutdown mechanisms.

How to use this table

Start with one uncomfortable question: Whose phone rings when the worst case happened?

Think of Summer Yue's case: She didn't intend for her company correspondence to be deleted. She even explicitly prompted the agent to not do it. But it was still possible. Because for agents, “possible” is enough. They won't hesitate and ask if they got it right: They just do. Rapidly. At scale.

Now work bottom-up through the tiers.

First: is existential impact technically impossible by external control? If you're not certain, be prepared to face it.

Next: could even a short period of unsupervised execution land you in a hole that's hard to get out of? If you can't say how that's prevented - better bring some climbing tools. And no: "I told the agent's prompt ..." - is not prevention. It's hope.

Continue upward through the tiers. Only when you have evidence, not "hope" that authority is constrained, actions are reversible and contained - then you have a predictable risk classification.

And be careful with the word “certainty.” When dealing with Agents, certainty does not mean “it won’t happen.” It means: when the worst case happens, we understand what happened and how to deal with it.

No comments:

Post a Comment