This page gives your security team an AI governance framework that works at the moment a system acts. It defines AI governance and shows why the standard frameworks break the instant a system can act. Then it gives you a nine-control framework that you can require of any vendor, including us.
Most AI governance today is written to be read. Nothing enforces it at runtime. It lives in a policy document, a risk register, and a quarterly committee review. That was adequate when AI meant a model that produced a prediction, and a human decided what to do with it. It is not adequate now. In April 2026, a coding agent deleted a production database and its volume-level backups after it hit a credential mismatch. The incident was widely reported as a plain case of excessive agency. No policy PDF stopped it, because a PDF has no enforcement surface at the moment a system acts.
There is a lazy debate about this, and both sides are wrong. "Move fast and break things" is unserious about power grids, supply chains, and money. "Slow down and regulate" is managed decline dressed as caution. The adult position is a third one: adopt fast, and make the method temperate. That is only possible if your governance runs at the moment of action. A slide cannot do that.
What is AI governance?
AI governance is the set of policies, controls, and accountability structures for an AI system. They keep the system safe, compliant, and aligned with an organization's legal and operational obligations across its lifecycle. In practice, it answers three questions. What is the system allowed to do? Who is accountable when it acts? How do you prove both after the fact?
Most established programs rest on the same five pillars: transparency, accountability, fairness, human oversight, and data protection. The reference frameworks map to those pillars. The NIST AI Risk Management Framework organizes the work into Govern, Map, Measure, and Manage. The EU AI Act sorts uses into risk tiers and puts most obligations on high-risk systems. ISO/IEC 42001 defines a management system for the whole program.
These are the correct baseline. Adopt them. But notice what they have in common. They assume a model that makes a prediction, offline, for a human to review. They govern outputs. They say almost nothing about how to bound a system that plans, calls tools, writes files, and takes actions for you. The moment your AI stops predicting and starts acting, the paper stops working.
Why traditional AI governance frameworks break on agentic systems
An agentic system does much more than answer questions well. It plans a multi-step task, selects and calls tools, reads and writes data, and executes. It often does this under a set of credentials that it borrowed from a human. So the unit you must govern is a decision chain with real blast radius. A single prediction is no longer the unit. Committee governance and model cards have no runtime hook into that chain.
The 2026 data shows how wide the gap is.
- 48% of production AI agents run unsecured, with no monitoring or logging of their actions, according to industry security reporting. You cannot govern what you cannot see.
- 88% of organizations reported a confirmed or suspected AI agent security incident in the past year, per the same body of research.
- Prompt injection is the top agentic vulnerability: OWASP and 2026 production data show that a single instruction hidden in a document or webpage can redirect an agent. No malware and no stolen credential is involved.
Two failure modes matter most to a security owner. The first is excessive agency. A system that can act can also act wrongly at machine speed, as the deleted production database showed. The second is credential misattribution. When an agent runs under a human's login, your audit log records that the human deleted the file. Your provenance is false by construction, and separation of duties is gone.
This is also why pilots die. Roughly 88% of agentic AI pilots never reach production, and the model is rarely the reason. The blocker is governance and integration. If a system cannot pass your own security review, its capability does not matter. The same problem sinks do-it-yourself builds on open-source agent frameworks, which ship without governance. Those frameworks leave the RBAC, audit, and sandboxing for you to assemble. For a fuller definition of the category, see our briefing on agentic AI.
What an agentic AI governance framework actually requires
Governance for a system that acts is a runtime architecture. A policy document cannot do the job. That distinction is the whole thing. A policy describes intended behavior and hopes for it. An architecture makes unintended behavior structurally impossible.
That requires an inversion of the default trust model. The standard approach trusts the model's output and repairs the damage afterward. The correct approach is the reverse. The administrator explicitly enables each capability, and everything that is not enabled is inaccessible. Deny by default. The system does not get the benefit of the doubt.
Use robotics safety as your mental picture. The controls you need look like kill switches wired into the system's brain. They let you blind it, mute it, take its hands, and pull its power pack. A flowchart edit such as "delete workflow step two" is far too weak. This is governed AI as a runtime architecture, and it rests on three things that the paper frameworks skip.
- Enforcement at the moment of action: The control fires when the system tries to do the thing. A review three weeks later is too late.
- Inside the perimeter: In regulated environments, data cannot leave the boundary. So agent execution in a vendor's cloud tenant is a non-starter. The perimeter is also where the control has teeth. Your buyer already carries scars from cloud vendor lock-in. This is the same lesson, applied to AI.
- A distinct identity per worker: If the system acts under its own identity, its actions are attributable, and separation of duties survives.
Vendor-agnostic inference belongs here too, as a governance control. You can run on Anthropic, OpenAI, or a self-hosted model, in any combination. Then no single-vendor dependency can become a supply-chain risk. The model providers are the engine. The governance is the vehicle around it. You do not govern against the model providers.
The framework: nine real-time governance firewalls
Here is the framework as a numbered set of controls. Each one is a real-time firewall: it enforces a boundary while the system runs. You do not wait for a later audit to apply it. This is the exact architecture that Mission Control deploys. Require every layer of it from any agentic vendor that you evaluate.
- Identity: Every synthetic worker gets a unique, verifiable identity. There are no shared service accounts and no borrowed human credentials. This identity gives meaning to every downstream control and every audit entry.
- Human RBAC: An administrator defines access with the role-based access controls that your team already runs. Humans grant access. The system never assumes it.
- Synthetic RBAC: The worker carries its own credentials and permission scopes. You provision it like an employee in your existing identity provider. If you can reason about an Okta login, you can reason about this. There is nothing new to learn.
- Execution sandbox: Package and code execution is whitelist-based. The runtime blocks the os, subprocess, and sys modules. There is no arbitrary code execution. That closes the most direct path from a compromised prompt to a compromised host.
- Audit logging: The system logs every model call and file access with complete provenance, attributable to the worker's own identity. This is the direct fix for credential misattribution. The log records what the worker did, under the worker's own name, with no human identity in its place.
- Delegation controls: A worker cannot escalate its own permissions or pass them to another worker. Privilege comes from above. A worker can never manufacture it from within.
- Scheduling: A worker acts only inside defined operating windows. Outside them, it is inert. That shrinks the time surface that an attacker can use.
- Communications: The system logs all messaging between workers. Multi-worker systems get no private channel that your security team cannot see.
- Capability whitelist: Any capability that an administrator did not explicitly define is inaccessible. The design itself bounds the blast radius. The guarantee does not depend on good behavior.
Where these controls end. Be clear-eyed about this, because an oversold governance story is how you lose a security review. These nine firewalls bound what a worker can do. They make every action attributable. They keep the blast radius small and reversible. They do not make the worker's judgment perfect, and they do not replace human review of high-stakes actions. Governance guarantees provenance and containment. It does not guarantee that every decision inside the bounds is the one you would make. A vendor who claims otherwise is selling you a slide.
How to apply this AI governance framework
The value of a framework is that it travels. Turn the nine firewalls into a checklist. Take it into every agentic vendor evaluation, ours included. If a vendor cannot answer these questions plainly, that is your answer.
| Requirement | What to ask | Why it matters |
|---|---|---|
| Distinct identity | Does each worker have its own verifiable identity, or does it run under a human's credential? | No identity means no real audit trail and no separation of duties. |
| RBAC to your IdP | Can we provision and scope the worker in our existing SSO/OIDC directory? | Reuses controls that you already trust. There is nothing new to secure. |
| In-perimeter deployment | Can it run fully on-premises with no external callbacks? | Required for NERC CIP, ITAR, DFARS, CMMC, and HIPAA data that cannot egress. |
| Complete audit trail | Is every action logged, attributable to the worker, and exportable for compliance? | This is your evidence in an audit and your provenance after an incident. |
| Deny by default | Are capabilities whitelisted, with everything undefined blocked? | Bounds the blast radius by design. Hope plays no part. |
| Inference choice | Can we choose or swap the inference provider without a rebuild? | Removes single-vendor supply-chain risk from the deployment. |
| Kill switch | Can we blind, pause, or fully stop a worker in real time? | Excessive agency is the core threat. You need an immediate off switch. |
Map each row to your own regime. This checklist is also the backbone of a broader vendor assessment. Pair it with our guide on how to evaluate AI agents. When you compare named platforms, read enterprise AI agents ranked on governance. Different categories carry different governance profiles. That is why the difference between RPA and AI agents changes what you need to control. Definitions matter too, so we keep a plain one for what an AI agent actually is. For where these controls land in practice, see the industrial AI briefing on regulated operations and the named workers in AI agent examples.
Governing synthetic workers with Swarm
Swarm implements all nine firewalls inside your perimeter, with vendor-agnostic inference. It runs in your own environment: Azure, AWS, Google Cloud, Docker, or Kubernetes. This matters for a regulated buyer. A copilot that lives in one SaaS tenant and calls an external model API cannot meet these controls. The data leaves your boundary, and the actions run under a borrowed login. A synthetic worker on your own systems, provisioned in your identity provider, behind your firewall, can meet them.
The architecture makes unauthorized actions structurally impossible. It does more than discourage them. An administrator enables each capability. Undefined capabilities are inaccessible. The runtime blocks arbitrary code execution outright. Every action lands in an audit trail tied to the worker's own identity. Customer data never leaves the customer environment. That is what lets the same architecture serve NERC CIP, ITAR, DFARS, and HIPAA settings without a separate premium tier. Deployment in your own environment is the default. It is no upsell.
A synthetic worker is governable in places where an agent is not. That matters more than raw capability. Capability without governance is what fills the pilot graveyard. Governance is what gets a system into production.
Conclusion
There is one idea under all of this. Paper governs predictions. Architecture governs actions. The frameworks you already run, NIST, the EU AI Act, and ISO 42001, remain the right baseline. They also become insufficient the moment your AI can act on your systems.
A short decision framework:
- If your AI system can take actions as well as make predictions, you need runtime firewalls. A policy document is not enough.
- If it touches data that cannot leave your perimeter, it must run inside the perimeter.
- If it runs under shared or human credentials, fix identity first. Nothing else is trustworthy until you do.
If all three are true, a governance PDF no longer protects you. See how Swarm deploys inside your perimeter, with all nine firewalls running in your environment.