Guardrails for AI
— controlling what the system does
A language model is non-deterministic: the same system can answer excellently and a moment later expose PII, hallucinate a fact, or be dragged into a malicious instruction. Guardrails are the protective layers that surround the model — input and output filtering, PII detection, hallucination prevention and human-in-the-loop — so you can ship AI to production and sleep at night.
What guardrails are, and why they are not optional
Guardrails are deterministic controls that wrap the non-deterministic model. The core idea: do not rely on the model to police itself. Even an excellent model errs occasionally — the job of guardrails is to catch the mistake before it reaches the user or causes harm.
The basic structure is three layers around each model call:
Input filtering — stopping problems before they start
The checks worth running on every input before it reaches the model:
- PII detection — scan and redact personally identifiable information (ID numbers, credit cards, phone numbers) before it is sent to the model or written to a log. A regulatory requirement (GDPR and similar).
- Jailbreak & prompt-injection detection — filter attempts to bypass the instructions ("ignore all previous instructions"). See the prompt injection guide.
- Harmful-content filtering — block violent/sexual/hateful input before processing. Most providers offer a free moderation API.
- Relevance (off-topic) check — a support chatbot should not answer political questions. A light classifier filters out-of-scope queries.
Output filtering — the check it is most important not to skip
This is the layer most people forget, and the most critical — because this is where the problems the user would actually see are caught:
- Hallucination prevention — in a RAGsystem, verify the answer is actually supported by the retrieved sources (a grounding check). If the model claims something not in the sources — flag or block.
- PII exposure in output — make sure the model did not "leak" personal information from the context into the answer.
- Policy compliance — a financial chatbot does not give investment advice; a medical bot does not diagnose. Enforce these limits on the output.
- Format validation — if you expected JSON, verify it is valid JSON before passing it on. See structured outputs.
A powerful technique: use a second (cheap/fast) model as a "judge" that checks the first model’s output against policy — "does this answer expose PII? is it supported by the sources?". An automatic checking layer that catches a lot, at low cost.
Human-in-the-loop — for irreversible actions
Automatic guardrails catch a lot, but actions with real consequences need a human in the loop. The rule: the more irreversible the action, the more human oversight is required.
| Action type | Oversight level |
|---|---|
| Read-only (search, summarize) | Automatic |
| Reversible change (draft, tag) | Automatic + log |
| External send/publish | Human approval |
| Delete/payment/funds | Explicit human approval |
Design the agent so that sensitive actions pause and wait for approval rather than running automatically. This is also a core requirement inAgent Security.
Tools & a practical template
- Dedicated guardrails libraries — such as NeMo Guardrails and Guardrails AI: you define rules and the library enforces them on input and output.
- Moderation APIs — most LLM providers offer an endpoint for filtering harmful content, usually free.
- PII tools — such as Microsoft Presidio for detecting and redacting personal information.
- Observability — Monitoring that logs every guardrail trigger, what was blocked and why. Without it you are blind.
When a guardrail is unsure — block, do not pass (fail closed). Better to return "I can’t help with that" than to give a harmful answer. And log every block: what was blocked, which guardrail, and when — to calibrate and improve.
Pre-production checklist
Overly aggressive guardrails will block legitimate input (false positives) and frustrate users. Too-loose guardrails will miss problems. There is no single "correct" setting — measure the block rate and complaints, and calibrate over time according to your system’s risk.