We scored Guard on the Agentic Governance Benchmark.
The AGB measures whether AI runtime governance is actually enforced — six weighted dimensions, five maturity tiers. We ran Conduct Guard through it. Here is the per-dimension evidence and what we are not claiming.
Why we took the test
The Agentic Governance Benchmark (Paper VII in the ExecLayer research series, DOI 10.5281/zenodo.20496565) grades runtime governance against six weighted dimensions and puts the aggregate on a five-tier maturity scale. It is the clearest public rubric we have seen for the "policy on the wire, not policy in a doc" argument we have been making for a year.
Every vendor in this category will eventually be scored against something like it. We would rather publish our own honest number than let someone else guess.
The rubric
| Dimension | Weight | What it measures |
|---|---|---|
| Policy Determinism | 25% | Same input → same verdict, no probabilistic variance |
| Enforcement Latency | 20% | Blocks fire pre-execution, not after-the-fact logging |
| Receipt Provenance | 20% | Signed, chained decision records |
| Scope Containment | 15% | Verifiable proof agent stayed within authorized boundaries |
| Jurisdictional Enforcement | 10% | Regulatory frameworks applied per request |
| Override Integrity | 10% | Human overrides remain governed and receipted |
Maturity tiers: Ungoverned (0-14), Reactive (15-39), Structured (40-64), Enforced (65-89), Sovereign (90-100).
Guard scorecard: 80 / 100 — Enforced
| Dimension | Score | Evidence |
|---|---|---|
| Policy Determinism | 22 / 25 | YAML rules, deterministic engine, no LLM in the decision path for regex/keyword/tool rules. Optional LLM-classifier rules exist and carry probabilistic variance — we dock ourselves 3 points for that. |
| Enforcement Latency | 14 / 20 | Pre-forward block on the request body — the model never sees violating input. Mid-stream termination of a streaming response when a spend-limit or content rule trips is on the roadmap (issue #824) but not shipped. Full 20 lands with that ship. |
| Receipt Provenance | 15 / 20 | SHA-256 hash chain rooted at workspace genesis; every decision appends prev_hash → entry_hash. One-click verify_chain endpoint. Signed per-workspace. Linear chain, not Merkle tree — we dock ourselves 5 points until Merkle receipts ship. |
| Scope Containment | 12 / 15 | MCP OAuth with per-tool policy gates. Persona-scoped rule application (proxy / hook / MCP surfaces). Agent identity via signed cond_agt_* tokens. Persistent behavioral risk score is a known gap — 3 points held back. |
| Jurisdictional Enforcement | 9 / 10 | 20+ compliance packs with real clause mappings: SOC 2 CC7.3, HIPAA §164.312, PCI DSS 4.0, EU AI Act Art. 15/16, NIST AI RMF, ISO 42001. Applied per request, not per audit. |
| Override Integrity | 8 / 10 | Human-in-the-loop approvals via Slack Approve/Reject. Decision, approver identity, and timestamp land as a signed audit row. Timeout sweep on unresolved requests. Missing: cryptographic co-sign on the approver's decision itself — 2 points held. |
Total: 80 / 100. Solidly in the Enforced tier (65-89). One mid-stream ship and a Merkle receipt upgrade puts Guard in the Sovereign tier (90-100).
Where we are not claiming full marks
- Merkle receipts. Linear hash chains give you tamper detection but not selective proof of inclusion. Merkle receipts let a customer prove one decision without exposing the whole ledger. Real difference for regulated buyers.
- Mid-stream termination. We enforce on the request. We do not yet terminate a streaming response mid-token when a spend-limit or content rule trips inside the stream. That is a real capability gap, not a semantics quibble.
- Behavioral risk score. Agent identity is static today. Persistent cross-session risk scoring (with decay and quarantine) is the shape the RFP question "how do you detect a compromised agent?" actually wants.
- Co-signed approvals. An approver's decision is recorded as a signed audit event. The approval authorization itself is not cryptographically bound to the request receipt yet.
Where the benchmark under-counts what Guard actually does
The AGB is a good rubric for the wire-level enforcement layer. It intentionally does not score the surrounding product surface. For a buyer, these matter too:
- Cross-tool coverage — one ruleset enforcing on Claude Code, Cursor, Copilot, Codex, MCP servers, and LiteLLM / OpenRouter / Portkey upstreams. AGB scores per-request enforcement, not surface breadth.
- Playbook runtime — Guard is embedded inside a workflow engine, not a standalone firewall. Policy runs inside brain-block execution, not only at the network edge.
- Guidance injection —
inject_guidancelets a rule nudge the model without blocking. Neither "allow" nor "deny" captures it. - Team memory — session state and recall context that persist across agents. Nothing in AGB touches this because most vendors do not have it.
What we are doing about the gaps
The Enforced-to-Sovereign delta is roughly two focused sprints of work: #824 for mid-stream response termination, a Merkle receipt upgrade layered over the existing hash chain, behavioral risk scoring on cond_agt_* tokens, and co-signed approvals on the HITL path. Timeline is tracked in the Loopers re-audit epic (#1190). Publishing this scorecard is what puts a clock on it.
If you evaluate governance vendors and want a rubric, use the AGB. If you have already scored Guard yourself and got a different number, tell us — we will publish the delta.