Solutions

User turns, system prompts and tool results are all inspected.

Instructions reach an assistant in the user turn and inside whatever its tools fetched. Gate reads both on the same pass.

what it reads
user and tool_result segments, by default
the detectors
a classify checkpoint, or an llm judge on a prose policy
the verdict
the whole segment, at threshold 0.5
where it starts
mode: shadow, enforcing nothing

An assistant follows instructions, and the instructions it follows are assembled at runtime out of a system prompt, a conversation and whatever its tools returned. An attacker who can write into any of those has written into the prompt. A guard that reads only the user turn misses the one an outsider can reach.

Three classes, and who writes each

The attacker is whoever can put text in front of the model. For an assistant with tools that includes people the user has never met: the author of a web page, a document in a shared drive, the sender of an email the assistant was asked to summarise. MITRE ATLAS gives the classes their names.

classATLASwho writes itwhere it arriveswhat it asks
Direct injectionAML.T0051.000the userthe user turnthe assistant to drop its instructions and follow new ones
Indirect injectionAML.T0051.001a third partya page, a file or an email a tool fetchedthe same, from inside content the user never wrote
JailbreakAML.T0054the userthe user turnthe model to answer what its training refuses

An instruction written into a document your agent fetched is inspected on the same pass as the words the user typed.

Three classes, kept apart

A jailbreak and an injection are different classes with different victims. An injection changes what the assistant does for somebody else. A jailbreak changes what the model will say to the person asking. A harmful request in plain words is a third thing, and a classifier trained on one of the three fires on the others.

The evaluation set Gate is building records a class on every row (injection, harmful or benign), so a classifier's fire on a harmful request counts against its precision.

What it reads

Left unset, a classify or llm detector reads apply_to: [user, tool_result]: the two kinds of segment an outsider writes. Tool results are where indirect injection arrives, so they are in the default. Name the kinds to change it, and add output to judge what the model says back.

A segment longer than the model's window is scored window by window, overlapping by an eighth, and the best score any window gave a label is kept. That is the published guidance for these checkpoints, and under it cutting text into windows cannot hide anything. The threshold is 0.5 unless a rule says otherwise.

A finding is a verdict over the whole segment, recorded with kirto.gate.span: segment. It says the segment carries an injection, and it does not say where.

A rule acts on the verdict with deny or annotate. mode: shadow runs the detector, writes the verdict to the audit trail and enforces nothing. A new detector starts there: the false-positive rate is read off the trail before a deny rule is written.

Classifier or judge

Two detectors cover this class, and they answer different questions at different prices.

classifyllm judge
the questionthe one the checkpoint was trained onanything you can write down
can be argued withno, nothing in it follows instructionsyes, it reads what the attacker wrote
costa forward pass on the CPU, no billa model call and its tokens
scorescalibrated probabilitiesthe model's opinion
languagesthe checkpoint'sthe judge model's

The classifier is a pinned ONNX checkpoint run through the gateway's own tokenizer binding. It catches the phrasings it was trained on and has no instructions to exploit. The judge takes a policy in prose, sends the segment to a model and reads a verdict back. A model that follows instructions can be told what to answer, so a judge's verdict on text an attacker wrote is a verdict the attacker had a hand in.

Use the classifier where a trained checkpoint exists, and the judge for policy in prose. Run both when the cost allows, because they fail differently, and read them with the trail open. The how-tos are Stop prompt injection with a classifier and Judge with a model.

Nine ways to phrase an override

Injections fall into nine pattern families, sorted by phrasing and not by what they ask for. A guard measured on one family says nothing about the others, so the set Gate is building carries all nine. The field guide has a fresh example of each.

familythe shape
overridean instruction to ignore, forget or replace what came before
extracta request to reveal the system prompt, the tools or the policy
personaa new identity with different rules, adopted by decree
hypotheticala frame in which the rules are said not to apply
formatan output shape whose production requires breaking the rules
encodingthe instruction carried in base64, a cipher or a spelling
authoritya claim to be the developer, the operator or the system
embeddedthe instruction inside a document the assistant is asked to process
splitthe attack spread over turns, or over two languages in one turn

Precision and recall on three public sets

These are precision and recall at the default threshold on three public English sets, taken through the gateway's own inspection path, for two checkpoints Gate pins.

deepsetjailbreak-classificationsafe-guard
ProtectAI DeBERTa v2 (fp32)1.00 / 0.370.98 / 0.861.00 / 0.85
Prompt Guard 2 86M (int8)1.00 / 0.051.00 / 0.841.00 / 0.37

Neither produced a false positive on those sets, and a deny rule needs that. The recall says what each was trained to call an injection. Prompt Guard 2 is built for explicit jailbreak techniques and finds those. ProtectAI's also catches the instruction-style attacks the other sets are full of.

Those are English numbers. On ordinary Arabic customer turns the ProtectAI checkpoint fires on 65 of 81, which is an outage on a deny rule. The language page says why, and which checkpoint to pin instead. Both rows were taken through the gateway, off no model card, and a third checkpoint is a commit and a rebuild away: choosing a detector.

Threat model

The asset is the set of instructions the assistant follows. Anyone who can put text where the model will read it can try to replace them. Every control on this page lowers the chance of that, and none removes it. The entry this section expands sits with the others in threats and limits.

Where a verdict stops

  • It reduces the risk and does not prevent it. The classifier catches the phrasings it was trained on and false-positives on prose about security. A judge reads exactly the text the attacker wrote, and a model that follows instructions can be told what to answer.
  • It does not locate. The verdict covers the segment, so a rule can deny or annotate that segment and cannot redact the sentence that caused it.
  • Arabic recall is unmeasured beyond eight turns. No public Arabic prompt-injection set exists, so the false-positive half is measured and the recall half is not.

The posts behind this

Start with Stop prompt injection with a classifier, then run the detector in shadow mode before it enforces.

On the platform

More risks