Six detector types, and any checkpoint on the Hub.

what is measured: the ensemble the gateway compiles, scored at byte offsets
the command: kirto-gate guardrails eval
what a reference pins: a Hugging Face commit, and the Hub's own digests
where the graph runs: in this process, or on a KServe v2 server

Six kinds of detector run in one ensemble: patterns with checksums, a transformer token classifier, a sequence classifier, embeddings scored against your own example sentences, a chat model handed a policy in prose, and an analyzer on another machine. A rule reads whichever of them it names, so an expensive one can run in shadow beside a cheap one that enforces.

You choose the model, from no approved list. Any checkpoint on the Hub is a reference plus a commit, verified against the Hub's own digests on the way in, and it runs in this process or on a KServe server with the GPU. The cost is the subject of this page: a number measured on your traffic, through the path the gateway runs.

Four stages after the checkpoint

A model on a hub carries precision and recall on its card, and those describe the model. What a rule acts on comes out of four stages after it. The tokenizer splits text into vocabulary items. The window scheme cuts a sequence past max_tokens into overlapping windows, 512 tokens with a stride of 14 by default, so an entity at a window edge is seen once with context on both sides. The decoder folds subword scores back into words and lands them on the request's own bytes. The threshold turns a score into a verdict.

Gate writes all four. A token-classification model computes logits and nothing else: the softmax, the BIO grouping, the subword fold and the byte arithmetic are the pipeline code Hugging Face writes in Python, which Presidio inherits by calling that pipeline and a bare graph call inherits none of. Because the decoding is Gate's, the number a rule acts on is the binary's, and it has to be measured through the binary.

The harness compiles what the gateway compiles

kirto-gate guardrails eval runs labelled text through the same inspection a request gets. The config is compiled the way the gateway compiles it, models are fetched and verified, the native libraries load, the thresholds apply, and each case is presented as one chat turn. A case carries spans at byte offsets or a set of labels, and an empty label list is a clean text that counts against a detector which flags it. A case can also say where in the request it sits: an injection arriving inside a retrieved document is a tool_result case, and a config whose detectors read only user scores a miss on it.

flag	what the run is then evidence of
`-sweep`	The average precision, the area under the precision-recall curve with no threshold in it, which is the number two checkpoints are compared on. Then precision, recall and F1 at each 0.05, which is where the threshold of the one you keep is read off.
`-invariance`	Whether the verdict depends on the language. The same meaning in every column of a row has the same right verdict in each, so a verdict that changes between them is a defect with no gold needed.
`-detector`	One member's findings alone. Without it every finding counts, shadow-mode detectors included, because shadow is where a detector is measured.
`-types`	Which entity types or labels are scored. A person-only corpus is not charged for the locations the ensemble also found.

Spans are scored twice. Exact is the same bytes and the same type. Relaxed is any overlap of the same type, one finding per marked span, and it is what a redaction has to achieve. The gap between the two columns is the offset arithmetic's own error: a clitic swallowed, a comma kept, a surname dropped because the tokenizer glued a question mark to it. A tag on a case names a slice, and the report repeats its totals for each, so a change that breaks one hazard shows on that line and nowhere else.

The measurement happens before a rule may read the detector. mode: shadow runs a member and records what it found without letting policy act on it, so a deny threshold is chosen from the traffic the deployment carries. Run a detector in shadow mode, measure a detector.

Why it runs through the binary

Two lines of the Gulf-names report moved without the model changing. A name inside an English sentence went from 0.57 to 1.00 exact F1, and a name after a title from 0.90 to 1.00. The decoder moved both.

XLM-R's tokenizer keeps الشمري? as one word, and the subword it scored most confidently was the question mark, labelled as no entity. Under the default aggregation the punctuation decided the word and the name was lost. Punctuation at a word's edge is now a word of its own before any aggregation runs. A token-level F1 printed beside a checkpoint never shows that, which is why the measurement goes through the gateway.

Any checkpoint, pinned to a commit

kirto-gate artifacts pin OWNER/NAME resolves a Hugging Face repository to a commit and prints the declaration to paste. A gateway accepts a commit and refuses a branch, because a branch lets the artifact behind a detector change under a configuration that did not. On the first build each node lists that commit, takes the Hub's own digests, a sha256 for every LFS file and a git blob id for the rest, fetches the files and verifies every byte. The resolved manifest is cached under the commit, so a restart resolves with no network.

A gated repository or a mirror takes an endpoint following the HF_ENDPOINT convention that Artifactory and Nexus proxies present, and a token that is a credential like every other: a literal is refused wherever the document travels, and it is sent only to that endpoint, never to a host the download redirects to. An air-gapped host takes the cache directory copied across, or a file list written down with a sha256 and a size each. Either way the digests are what the gateway believes, and the host is only where the bytes are. A detector pointed at model: and tokenizer: paths directly still works, and makes the operator the one who vouches for the file. Pin a model from the Hub.

Nothing in the ensemble is a checkpoint you have to accept. The models Gate ships are declared the way yours are, on a commit with digests, and a rule picks between them one rule at a time. The numbers published beside them describe the build that runs them, which is what the reports are.

Where the graph runs

Because the decoding is Gate's, where the graph runs is a deployment question with no bearing on correctness. There are three transports, and the column that matters is the last.

detector	tokenizer	inference	who decides the spans
`type: nlp`	here	here	Gate
`type: nlp, with nlp.remote`	here	on the server	Gate
`type: remote`	on the server	on the server	the analyzer

nlp.remote ships token ids to a box that has a GPU and gets logits back, over the KServe v2 inference protocol that Triton, KServe, Seldon and OpenVINO Model Server all serve. Only the tokenizer and the label map are fetched, the graph is never downloaded, and the repository need not hold one. Every piece of span arithmetic stays where the bytes are. A GPU is worth having on one box and not on every gateway replica, and the offsets deciding which bytes get redacted belong in the process holding them.

type: remote is the opposite trade. It speaks presidio-analyzer's /analyze, so an existing Presidio deployment is a detector with no adapter to write, and what comes back is that analyzer's idea of where an entity begins. Presidio counts offsets in code points and Gate counts bytes, so they are translated at that edge and nowhere else, and offsets: utf16 declares the third convention for an analyzer written in Java, JavaScript or .NET. The annotation record is Presidio's RecognizerResult throughout, so what the ensemble emits is valid input to presidio-anonymizer and the presidio-research evaluators.

The two native libraries, the tokenizer shim and ONNX Runtime, are both reached by dlopen. A deployment without either keeps every other tier and fails at startup with an error naming the missing one. No setting uses a library only when it happens to find one, because that would make one configuration mean two different things on two machines, with nothing in the logs to say which you got.

Six types, one ensemble

type	what it is	what it needs
`pii`	103 recognizers with their checksums and context words, plus your own patterns	nothing
`nlp`	a transformer token classifier, for the entities a pattern cannot describe	a tokenizer, and a graph here or on a server
`classify`	a sequence classifier: Prompt Guard, ProtectAI's injection DeBERTa, a moderation checkpoint	the same head as nlp
`embed`	a handful of your own example sentences per topic, scored by the nearest topic's lead over examples of ordinary traffic	the same head as nlp
`llm`	a chat model handed a policy in prose and a closed list of labels	a provider you already configured
`remote`	an analyzer on another machine	a presidio-analyzer /analyze endpoint

A rule names the members it reads, so an expensive detector can run in shadow beside a cheap one that enforces, and a schedule says whether anybody waits for it. Every detector that can enumerate what it emits does so, and the union is the vocabulary a configuration speaks: an fpe_alphabets entry or an MCP server's restore list naming a type nothing emits is refused at load, where its author is reading. The guardrail pipeline, schedules.

The measurement covers the path behind the checkpoint. kirto-gate guardrails eval compiles the config the way the gateway compiles it, loads the same native libraries and scores at byte offsets, so the number a rule acts on is the binary's.
Decoding is Gate's, which is why it has to be measured. A token classifier computes logits and nothing else. The softmax, the BIO grouping, the subword fold and the byte arithmetic are Gate's code.
Any checkpoint on the Hub, pinned to a commit. Verified against the Hub's own digests on the way in, with no approved-model list in between.
The graph runs where the hardware is. In this process, or on a KServe v2 server with the GPU, and the span arithmetic stays where the bytes are either way.
A refusal carries the command that fixes it. A repository holding PyTorch weights and no ONNX export is refused with the optimum-cli line that produces one.
Six types in one ensemble. A rule names whichever it wants, so an expensive detector runs in shadow beside a cheap one that enforces.

What a number here does not settle

The gateway cannot check what a model server runs. With nlp.remote the pinned commit fixes the tokenizer and the labels. Whether the box returning the logits holds that commit's graph is a pairing you keep.
A remote analyzer decides its own spans. type: remote inherits the analyzer's boundaries along with its findings, and one with no entities filter leaves the vocabulary open. An open vocabulary refuses nothing.
The harness scores text somebody labelled. It says what a detector does on the cases in the file. What it does on live traffic is answered by mode: shadow.
A set nobody else can write for you. The public corpora are news and encyclopaedia text, with no chat turns, no Gulf names and no identifier beside a name. A hundred sentences in the register your users write is an afternoon, and it is the only number about your deployment.

The posts behind this

Why the number comes from the binary. A model card reports precision on the checkpoint, and what enforces is that checkpoint behind a tokenizer, a window scheme, a decoder and a threshold. One rule about where a word ends, in the last of those, took a score from 0.57 to 1.00 with the model byte for byte unchanged.
Calling a detector. A gateway can call any detector, and what comes back is a verdict about the string it handed over. Presidio, Lakera, Prisma AIRS, Guardrails AI and NeMo are all called this way. Deciding on the verdict is still the job of whatever holds the connection.

Measure a detector Pin a model from the Hub