MachineCraft LogoMachineCraft
Blog

Three questions every compliance team asks before AI agents reach production

Most enterprises stall AI agents on oversight, auditability, and where the thing runs — not on capability. Here are the three questions a risk team asks before production, the bar each one sets, and how to clear it.

MachineCraft4 min readtrustcomplianceagents

Most teams evaluating AI agents already believe the agent can do the work. The demo was convincing. The blocker shows up later, in a different room — when the proposal reaches the people who sign off on what runs in production. Risk, compliance, security. Their job is not to be impressed; it's to be unsurprised.

They tend to ask three questions. None of them are about model quality, latency, or benchmark scores. All three are about control. If you can't answer all three, the agent stays in the demo environment indefinitely — not because it doesn't work, but because nobody will own the consequences of it acting unsupervised.

Here are the three questions, what a real answer looks like, and where we're honest about the line between what ships today and what's shipping in beta.

1. "How do we guarantee it doesn't act without a human signing off?"

The tempting answer is to bolt approval onto a chat window: the agent proposes, a person types "yes," and it proceeds. That is not a control — it's a suggestion the agent is free to ignore. The model still holds the execution pointer, so a confused or prompt-injected agent can talk its way past a conversational checkpoint, or simply not ask. A reviewer who only sees a tidy summary is approving the agent's description of what it's about to do, not the action itself.

A real gate is structural, not conversational. In MachineCraft you mark any node in a flow as an approval gate. When execution reaches that node the agent pauses — it does not call the tool, write the record, or move the money. A reviewer is notified, opens the paused run, inspects the exact state the agent is sitting on, and then approves, rejects, or modifies the proposed action. Only that decision resumes the run. Because the gate lives in the execution graph, not in the prompt, there is no path that quietly skips it.

The thing worth internalizing here: an approval prompt asks; an approval gate blocks. For anything irreversible — a payment, a filing, a record deletion, an outbound email to a regulator — only the second one is defensible.

2. "How do we prove to an auditor exactly what it decided, and why?"

This is where the common answer fails hardest. "The model was 95% confident" is not an audit trail. A confidence score is a number the model emitted about its own output — it isn't evidence of what happened, it doesn't survive a model swap, and no auditor can reconstruct a decision from it. Neither can a chat transcript: prose explains intent, but it doesn't prove which record was read or which key was used.

An audit trail is a record of facts, not narration: which inputs the run saw, which tool it called, which credential it accessed, what it wrote, and which human approved which step, when. MachineCraft logs every action, state transition, credential access, and human approval to a dedicated, exportable log — today, not in beta. That's the artifact an auditor actually wants: a replayable account of what the system did, independent of how the model explained itself.

Credentials are part of that picture. Secrets are stored with Fernet (AES-128-CBC) encryption across 18 providers, with per-workflow usage tracking — so a key is never pasted into a plain-text variable, and every use is attributable to a specific run and step. "Who could have touched this system, and did they?" becomes a query, not an investigation.

3. "Can all of this run inside our security perimeter?"

For a bank, a hospital, or a government team, the first two answers are worthless if the data has to leave the building to earn them. Plenty of agent platforms can only run as someone else's hosted service, which makes the compliance conversation short: the data crosses a boundary you don't control, so the project stops.

MachineCraft separates design from runtime. You build a solution once, then deploy the same artifact to managed cloud, your own private cloud, your data center, or a fully air-gapped network with no outbound internet at all. The differences between environments are environment variables — endpoints, model providers, storage — not a rewrite or a second codebase. The agent, the gates, and the audit log come with it, so the controls you signed off on in evaluation are the controls that run in production. This part ships today; we go deeper on it in Build once, deploy anywhere.

That last point is the one compliance teams underweight until it bites them: a control that only exists in the vendor's cloud is not your control. Running the whole stack — orchestration, approval gates, and the audit log — inside your perimeter is what makes "we reviewed it" and "we run it" the same sentence.

Where this is today — honestly

It's worth being precise about the stage, because naming the line is part of earning the trust:

  • Ships today: audit logging, encrypted credential management, and the deploy-anywhere model. These are live and documented in the changelog.
  • Shipping in beta: the Agent Engine and HITL approval gates — the autonomous orchestration and the structural gate that sits on top of it.
  • On the roadmap, not claimed as shipped: durable checkpointing for long-lived pauses. Today the agent runtime holds run state in memory; if a gate waits hours for a human, surviving an infrastructure restart is roadmap work, and we won't pretend otherwise. Track it on the roadmap.

The reason the two-engine split matters here is that not every task needs an autonomous agent. A document-extraction or RAG pipeline runs on the structured Workflow Engine, which ships today, while the Agent Engine handles the multi-step reasoning that genuinely needs it — and both inherit the same audit trail and the same deploy model. If you're deciding which is which, Workflows vs. agents walks through it; if you're mapping this to a regulated environment specifically, see solutions for regulated enterprises.

None of this makes the compliance conversation automatic. But it changes its shape: instead of "trust the model," you're handing a risk team a gate they can point to, a log they can export, and a runtime that never leaves their perimeter. That's a conversation that can actually end in a yes.

Bring the questions your compliance team will ask

Walk through HITL approval gates, the audit trail, and deploy-anywhere against your own controls — and we'll be straight about what's beta.