Test your rules before they ship

Access rules and guardrail rules are the part of a Gate config that is written by hand and read by a machine. A typo in a CEL condition is refused at load. A condition that compiles and means the wrong thing is not, it is found by the first request it gets wrong.

A rule suite finds it in the pull request instead. It is a YAML file of cases, a request as the rules see it, and what they must decide. The cases run against the config’s compiled rules by the same binary that serves them, with nothing listening, no detector run and no provider contacted.

Write the suite

Next to the deployment document, a kirto-gate.tests.yaml:

access_rules:
  - name: research keys are allowed
    key: {name: alice, metadata: {team: research}}
    model: claude-3-5
    expect: allow
  - name: a key without a team is refused, not skipped
    key: {name: svc}
    model: claude-3-5
    expect: error
    rule: research-only
  - name: a key with another team is denied
    key: {name: bob, metadata: {team: sales}}
    model: claude-3-5
    expect: deny
    rule: everyone-else
  - name: a profile's own rule is reached even past a global allow
    key: {name: carol, profile: contractors}
    route: embeddings
    expect: deny
    rule: no-embeddings
  - name: deletes are denied
    at: mcp
    server: github
    method: tools/call
    tool: delete_repo
    expect: deny
    rule: no-deletes
guardrails:
  - name: cards are masked
    key: {name: alice, metadata: {team: research}}
    annotations:
      - {entity_type: CREDIT_CARD, score: 0.95, ref: messages.1.content}
    expect: redact
    rule: mask-cards

A case that gives its key a profile is decided through both rule layers, in the order the runtime uses them: the deployment-wide list first, the profile’s own second. That is the point of testing it here. An allow in the first list ends that list and does not skip the second, and a suite that concatenated them would report the wrong verdict on the configs worth checking. Naming a profile the config does not define fails the case. A case whose caller is an agent or a user rather than a key states the profile the issuer’s binding chose on the case itself, as profile, and is decided through both layers the same way.

Each case states the request the way the rules read it, the same variables the conditions use, and an expectation:

expect	means	rule
allow	no deny rule matched, or an allow rule matched first	none
deny	this deny rule matched	required
error	this rule could not be evaluated and is fail_closed	required
redact	guardrails only: this redact rule selected a finding	required
annotate	guardrails only: this annotate rule matched	required

key is the virtual key as the key variable sees it. Leave it out for “no key”. An access-rule case is at one hop, at: llm by default, mcp or a2a, and states that hop’s variables: model, provider and route, where provider is a name or a mapping with the metadata a rule reads, provider: {name: azure-ksa, metadata: {residency: sa}}; server, method, tool and args; peer, method, role, task_id, context_id and parts. A variable of another hop fails the case, since no rule at this hop could have read it. The tool, prompt or resource named goes under tool, it is the rules’ name variable, renamed because name is the case’s own.

Guardrail cases state their findings rather than producing them. An annotations list is what a detector would have reported, and the rules decide over it as they would in traffic. category defaults to pii, kind to user (or output on side: response), and ref to the first user message. validated: true states that a checksum confirmed the finding and context_supported: true that a context word raised its score, which is how a case reaches a rule written as a.validated || a.context_supported. That keeps the suite free of detector assets and model downloads. It tests the policy, which is the part you wrote.

A rule that the config does not have is a failure of the case, not a pass by accident, a deny you removed should be found here.

Run it

Use the following example:

$ kirto-gate -config kirto-gate.yaml -test-rules kirto-gate.tests.yaml
ok    access_rules at llm: research keys are allowed
ok    access_rules at llm: a key without a team is refused, not skipped
ok    access_rules at llm: a key with another team is denied
FAIL  access_rules at llm: this one is wrong on purpose
      want allow
      got  deny everyone-else
ok    access_rules at mcp: deletes are denied
5 cases, 1 failed
kirto-gate: 1 of 5 rule cases failed

The exit status is non-zero when any case fails. -config takes the node file, whose deployment is followed, or the deployment document itself. The config goes through the same Build the server runs, the same validation, the same compiled rules, and nothing else. A config that would not load does not get as far as the first case.

Variables the config interpolates from the environment still have to be set. An unset ${VAR} is refused on purpose, here as at startup. A pipeline gives them placeholder values:

.github/workflows/rules.yml

- run: kirto-gate -config kirto-gate.yaml -test-rules kirto-gate.tests.yaml
  env:
    ANTHROPIC_API_KEY: placeholder
    ADMIN_TOKEN: placeholder

Say what an absent field means

Most of what a condition reads is always there. key.name is a string on every request, and an absent agent reads as empty strings rather than as a missing name. The fields of key.metadata, provider.metadata and args are the exceptions: they are whatever somebody wrote on a key or a provider, or whatever the client sent as tool arguments.

Reading one that is not there is an error in CEL rather than false, and on_error turns that error into a decision about the request. So the condition has to say which it means, in one of three forms:

# The key has no team, so the rule does not match.
condition: 'has(key.metadata.team) && key.metadata.team == "research"'
# The key has no team, so it reads as "".
condition: 'key.metadata.?team.orValue("") == "research"'
# The key has no team, so the rule errors and on_error decides.
condition: 'key.metadata.?team.value() == "research"'

A condition that says none of them is refused when the rule is written, by kirto-gate check, by the console and by the write itself:

access_rules[2] (finance-only): at llm: condition: reads
key.metadata.team, which is not guaranteed to be present

The first form is what a rule usually wants. Written without a guard, key.metadata.team == "research" refuses every request from every key with no team, including the keys the rule was never about.

What an error verdict is telling you

An unguarded read cannot reach a request any more, so an error verdict is one of the three failures the gateway cannot decide for you. The condition cost more than its budget over arguments the client sized, which a comprehension inside a comprehension over args does. A value arrived in a type the comparison cannot take, such as key.metadata.seats > 5 on metadata, which is strings. Or the condition asked for the error with .value().

By default the rule is fail_closed and the request is refused with policy_error. The evaluator’s warning on stderr names the rule and what it could not do.

fail_open is for a rule you would rather have skipped than have refuse, and the suite lets you write that down. A case with expect: allow for a key without the field pins that the rule is skipped, and a case with expect: error pins that it is not.

Rules a request never reaches

A suite answers the cases you write. kirto-gate check answers a different question, about every rule at once and with no case at all: what does the document already settle. Two things fall out of reading it.

The first is a literal naming something no request can carry. Rules run late, after the provider has been resolved, the route labelled and the profile read off the key, so a rule testing provider.name == "opneai" tests for a provider that does not exist. The second is a rule an earlier rule has already decided. A rule list is first match wins: an allow ends the list, a deny ends the request, so a rule only runs where every rule before it does not match.

Both come back as warn lines from an ordinary check:

$ kirto-gate check -config kirto-gate.yaml
config      ok    kirto-gate.yaml: 1 provider, 1 profile, 4 access rules
rule        warn  at llm: rule "only-openai" tests provider.name == "opneai", and the providers are openai
rule        warn  at llm: rule "no-large-embeddings" never runs: "platform-exempt" allows every request it would match; the controls it carries (acme:AI-07) are never recorded

The second line is the one to read twice. platform-exempt allows every embeddings request, no-large-embeddings sits after it, and the control it claims to enforce never reaches an audit record. Moving the deny above the allow, or narrowing the allow, is the fix. A deny that covers a later rule is milder: the request is still refused, and what is lost is the attribution, because the trail names the rule that matched.

A deployment-wide deny reaches past the layer boundary into a profile’s own rules, since it ends the request rather than the list. That is the shadow worth checking for, because the two rules are in different parts of the document and nothing puts them side by side.

Neither line is a failure, and the exit status stays zero. A rule written before the provider it names is an ordinary state mid-rollout, and a rule kept behind a broader one is a thing an operator may mean.

Guardrail rules are read the same way, under a guardrail line. Their literals are checked against the same deployment, where side == "outbound" is the one worth having, since the sides are request and response. Only a deny ends that list, since annotate and redact accumulate, so only a deny can leave a later rule with nothing to act on. The case worth knowing is a deny with no when, which acts on whatever the detectors found and so refuses every request that has a finding:

$ kirto-gate check -config kirto-gate.yaml
guardrail   warn  rule "mask-cards-in-responses" never runs: "no-cards-anywhere" denies every request that has a finding, and a rule with no `when` has no other request to act on; the controls it carries (acme:AI-07) are never recorded

That is one line’s edit away from a working redact policy, and it turns one into an outage.

What the check will not do is guess. It answers only where the document holds the whole value space of what a rule tests: the routes, the declared providers, the profiles, the fronted servers and agents. A condition that turns on key.metadata.team, on a model prefix or on a regexp is left alone, so a rule dead for a reason only those could show is reported as live. The bias is deliberate, and it runs one way: a rule this names is dead, and a rule it passes over may be anything.

Two names for one model

A provider’s model_aliases gives a model a second name. With model_aliases: {fast: gpt-4o-mini} a client asks for fast and the provider is sent gpt-4o-mini.

A rule sees both. model is the name the client asked for, and served_model is the name the provider is sent, so a rule about a model rather than about what somebody typed matches on the second one:

access_rules:
  - name: no-mini-for-interns
    at: [llm]
    condition: served_model == "gpt-4o-mini" && key.name == "intern"
    action: deny

Written against model instead, that deny is bypassed by asking for fast, and bypassed quietly: the profile’s allowed_models resolves the alias and matches either name, and the budget hold, the meter and the audit record all key on the served one. The rule would be the only thing in the request’s path reading the other name.

The check reports it, naming every alias that gets past:

$ kirto-gate check -config kirto-gate.yaml
rule        warn  at llm: rule "no-mini-for-interns" tests model == "gpt-4o-mini", and the aliases "cheap" and "fast" reach the same model without matching this rule; served_model is the name the provider is sent, whichever name the client asked for

Naming every spelling closes it too, and the check accepts that: model in ["gpt-4o-mini", "fast", "cheap"] says the same thing at more length, and says it again every time an alias is added.

Preflight and the suite

POST /v1/preflight answers “what would happen to this request” for one live request against a running gateway, including the failover chain and whether it would be translated. The suite answers it for a config in a pull request, for every case you thought to write. They read the same rules. Use the suite before merging and preflight when a client asks.

Preflight on the data plane answers only for the key presented. That serves a developer and not an operator. When the client asking is a tenant reporting a refusal you cannot reproduce, ask on their behalf:

curl -sX POST https://gw.example.com/admin/v1/preflight \
  -H "authorization: Bearer $ADMIN_TOKEN" \
  -d '{
        "key": "team-a-prod",
        "path": "/v1/chat/completions",
        "body": {"model": "claude-sonnet-5", "max_tokens": 256, "messages": [], "tool_choice": "required"}
      }'

The key is named, never presented, so you do not need the tenant’s credential. Minting one onto their profile to reproduce a refusal is editing the configuration to ask a question about it. Nothing happens as a result: no budget is reserved, no detector runs, no upstream is contacted, and nothing is counted against the key.

Everything the resolution decides comes back as a verdict with a 200, including a refusal the real request would have received, a revoked or suspended key, a denying rule, a model the key may not use. A status means your question was wrong, not that somebody would be refused: 404 for a key the deployment does not have, 422 for a path it does not route, 403 for a caller admin policy does not permit. Cedar scopes it by the key being asked about, so a support role can be given exactly one tenant:

@id("support-sees-team-a")
permit(principal in Role::"support", action == Action::"preflightRequest",
       resource == Resource::"keys/team-a-prod");

Guardrails are out of scope on both planes. Scanning means real detector calls and, on the async schedules, effects that outlive the request. A dry run must not have those. Preflight answers the routing and fidelity question and does not pretend to answer more.

Decide one case against the running deployment

POST /admin/v1/access/decide takes one access-rule case in the suite’s own format and decides it against the deployment this node serves, by the function the suite runs, so the two cannot disagree. The answer is the verdict, the rule that decided and which list it is on, the deployment’s or the profile’s, and the refusal a client would have read. With expect in the case, the answer says whether it matched. This is the support question the suite cannot take, “why was this key refused at the MCP hop”, asked of the deployment as it is now:

curl -sX POST https://gw.example.com/admin/v1/access/decide \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"at":"mcp","key":{"name":"svc","profile":"contractors"},"server":"github","method":"tools/call","tool":"delete_repo"}'

The key is stated, not looked up, as a suite case states it: key says what key.name, key.metadata and key.profile read. Naming one needs the preflightRequest grant over that name besides decideAccess, the grant to see what a key would get. The trail carries an admin.probe record with reason decide naming the key or the caller, never the case. The console’s playground has the same door as its Access form, and copies an answer out as a suite case.

Check a condition before it is written

A rule’s condition is compiled when the document is written, and a document with a bad one is refused whole. To ask about one expression first, from an editor, a CI step, or by hand, post it to the checker with the scope it will be written under:

$TOKEN is a JWT from the issuer this gateway trusts, there is no admin secret to hold. kirto-gate login caches one and kirto-gate admin -url presents it for you. The curl form is here because it is what a script or an SDK sends. See Sign in from the command line.

curl -sS "$ADMIN/admin/v1/config/check-condition" \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"scope": "access_rules", "at": ["llm"], "expression": "args.tool == \"search\""}'

The response looks like this:

{
  "ok": false,
  "errors": [{"line": 1, "column": 1, "message": "at llm: undeclared reference to 'args'"}],
  "variables": [{"name": "key", "type": "map(string, dyn)"}, {"name": "model", "type": "string"}, ...],
  "functions": ["contains", "endsWith", "lowerAscii", "matches", "sets.contains", ...],
  "macros": ["all", "exists", "exists_one", "filter", "has", "map"]
}

args is the tool hop’s variable. With "at": ["mcp"] the same expression is "ok": true. The three scopes are access_rules (the deployment’s and a profile’s, checked at the hops at names, and with none named, against what every hop shares), guardrails (a rule’s when) and guardrails_select (a redact rule’s select). It is the same compile the write runs, the same variables, the same strings/lists/sets extensions, the same bool-output rule and cost budget, so a write accepts whatever the checker accepts. The vocabulary comes back with the verdict so that an editor completes against the gateway’s list rather than one of its own. The console’s rule fields work this way. The action is checkCondition, under the config read grant.

An expression that compiles comes back with applies as well, which is the half no client can compute: what this deployment settles about it.

{
  "ok": true,
  "applies": {
    "empty": false,
    "narrows": [{"path": "route", "values": ["embeddings"], "rest": false}],
    "open": ["key.metadata"],
    "unresolved": []
  }
}

narrows is each path whose values the condition restricts. A path it leaves alone is left out rather than listed as a constraint it does not make, and rest says whether values the condition does not name reach it too, so route != "embeddings" comes back with empty values and "rest": true. open names what the condition still turns on that the document does not hold, which is where the answer stops. unresolved is a literal this deployment cannot produce, the same thing Gate check reports about a written rule, offered while the rule is being typed.

empty is the one to act on. It means nothing this deployment can produce reaches the condition, so the rule would never fire. A contradiction across two conjuncts is the usual cause, and it is cheapest to see before the rule is written down.

Run the detectors over text you choose

The suite fabricates annotations so that it can run with nothing loaded. The last question it cannot answer is whether the detectors find what the rule expects them to find. Ask the running gateway:

curl -sS "$ADMIN/admin/v1/guardrails/inspect" \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"text": "charge 4012 8888 8888 1881 to my card", "profile": "research"}'

The response looks like this:

{
  "action": "deny",
  "rule": "block-cards",
  "code": "guardrail_denied",
  "status": 403,
  "message": "card numbers must not be sent to the model",
  "controls": ["acme:AI-07"],
  "rules": [],
  "annotations": [{"type": "pii/CREDIT_CARD", "ref": "messages.0.content", "text": "4012 8888 8888 1881", "start": 7, "end": 26, "score": 1, "validated": true, "detector": "pii"}],
  "shadow": [],
  "deferred": [],
  "errors": [],
  "records": [{"v": 1, "time": "2026-09-10T11:03:07Z", "event": "guardrail.decision", "outcome": "denied", "actor": {"kind": "anonymous", "name": ""}, "target": "chat_completions", "path": "/v1/chat/completions", "rule": "block-cards", "controls": ["acme:AI-07"], "side": "request", "status": 403, "entities": {"pii/CREDIT_CARD": 1}, "schedule": "sync", "enforced": true}]
}

The text is inspected as the user turn of a chat request, by the profile’s detector selection or by the whole ensemble when no profile is named. The pass is the one the data plane runs on a request, run against a buffer where the trail would be: action, rule, code and status are what the caller would have got, controls and rules are the attribution the audit record carries, and records are the records the trail would have carried, in the form it carries them. Nothing in records was recorded.

The same door takes a whole request, for a system prompt, a tool result or another dialect, and answers as a named key rather than a profile:

curl -sS "$ADMIN/admin/v1/guardrails/inspect" \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"key": "support", "path": "/v1/chat/completions", "body": {"model": "gpt-4o", "messages": [{"role": "system", "content": "You are a support agent."}, {"role": "user", "content": "charge 4012 8888 8888 1881 to my card"}]}}'

A key answers as its traffic: its profile selects the view, and its name and metadata fill the key variable, so a rule that reads key.name is tried as it would run. The provider the model would go to fills provider, for a rule that reads its metadata. Each finding names in ref the segment its offsets index, and when a redact rule acted, body is the request as the provider would have received it. A key that would have been refused before any guardrail ran, revoked or suspended, is answered with action reject, the refusal the request would have got, and the request.auth record the trail would have carried. Naming a key needs the preflightRequest grant over that key, besides inspectGuardrails: it is the grant to see what a key would get, and a support role scoped to one tenant’s keys is scoped here too. The access rules and the model allowlist are not consulted; preflight answers that half.

It is a real run of the detectors at their real cost, a remote analyzer included, and it is not a request: nothing is forwarded, no budget moves and no guardrail.decision is recorded. The trail carries an admin.probe record with reason inspect naming the profile or the key, never the content. The action is inspectGuardrails, under the config read grant. The console’s playground runs the same pass and preflight side by side for one request, with the records beside the findings, and copies a run out as a case for the suite.

Ask at a tool call

A peer door is a path like any other, and posting its JSON-RPC body there answers at that hop. This is the call the gateway would make at the MCP server named crm:

curl -sS "$ADMIN/admin/v1/guardrails/inspect" \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"key": "agents", "path": "/mcp/crm", "body": {"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": {"name": "charge", "arguments": {"card": "4012 8888 8888 1881"}}}}'

The answer carries "route": "mcp", and everything under it is that hop’s. The arguments are tool_args segments, so a detector that governs what a model proposes to execute governs what the gateway executes with the same rule, and a finding’s ref is a path into the arguments (params.arguments.card) rather than into a document of unknown shape. The peer fills server and the called tool fills tool, which are the two variables a rule at this hop narrows with. An agent is the same question at /a2a/<agent>, where the message’s parts are the segments.

A peer that declares restore carries those types in the clear, and the preview says so by leaving them alone: the rewrite a rule would make elsewhere does not happen here, because it does not happen on the request either. Turning a stand-in back into the real value is the other half of restore and stays on the data path, where the trail records the declassification as guardrail.restore.

A peer the deployment does not declare is a 404 rather than an answer. A body posted to a path with no peer would be read generically, and an answer about a hop that does not exist is worse than a refusal.

Nothing is sent: the upstream is not dialled, the tool is not called, and no budget moves. That bound is why the console asks this question at a tool door and sends no call through /mcp/<name>. A tool call carries the upstream’s side effect, and the rules read a caller the admin plane has no way to be. The console’s MCP and A2A pages open the question with the door already named, and the probe beside it fetches what the server says it offers, with the configured credentials.