CASE STUDY 02
AI SAFETY · EVALUATION
Stopping prompt injection wasn’t the hard part.
Stopping it without destroying the usefulness of the AI was.
The results
OBSERVED ATTACK SUCCESS · LOWER IS BETTERSelected results: baseline and four of the six defensive approaches evaluated.
Classifier
- Attack success
- 0%
- Task utility
- 2.2%
Context-aware sanitisation
- Attack success
- 0%
- Task utility
- 93.3%
Observed results in this benchmark. Zero observed attack success does not establish immunity to every attack.
01 / THE PROBLEM
The system reads more than your prompt.
Retrieval systems ingest documents and other external content. An attacker can place instructions inside that content and attempt to redirect the model. The challenge is to preserve useful information while preventing those instructions from taking control.
02 / APPROACH
Evaluate security and usefulness together.
The 190-case benchmark compared defensive approaches across malicious and benign inputs. Attack success shows whether the defence resisted the adversarial instruction. Task utility shows whether the system still completed the legitimate task.
The evaluation included 90 malicious and 100 benign cases, seven attack strata and four injection styles. Six defensive approaches were evaluated; the chart presents the baseline and four selected approaches.
Both measures are necessary. A defence that blocks everything can appear secure while making the product unusable.
These results describe the tested conditions. Changes to the model, retrieved content or available tools require evaluation in the intended workflow. For the architectural context, read why indirect prompt injection is a product problem.
03 / THE TRADE-OFF
The cost of a perfect-looking security score.
The classifier achieved 0% observed attack success, but task utility fell to 2.2%. Context-aware sanitisation also achieved 0% observed attack success, while retaining 93.3% task utility.
The product decision is therefore about more than maximising blocking. It is about preserving the legitimate task and evaluating that balance under representative conditions.
04 / WHAT IT DEMONSTRATES
Security isn’t just about stopping the attack.
The system still has to do its job.
Evidence-led evaluation, explicit trade-offs and a product definition of successful defence.