AI GatewayReviewed 2026-01-13
Shadow mode for AI policy changes
Run AI gateway policy changes in shadow mode to compare decisions before enforcing them.
Shadow mode is a standard AI gateway feature for safe rollouts.
Policy Gateway evaluates policies in shadow mode while leaving live responses untouched, so you can compare outcomes and rollback safely.
Definition
Shadow mode for AI policy changes
Shadow mode for AI policy changes evaluates new policies against live traffic without enforcing them, producing audit logs and decision metadata for comparison.
Why it matters
- Validate policy changes before enforcement.
- Detect false positives early.
- Keep audit logs for policy approvals and compliance reviews.
How it works
- 01Enable rollout.shadow in policy JSON and set a sample_percent.
- 02Use policy_target to scope shadow evaluation to specific apps or routes.
- 03Compare shadow decisions against enforced output to decide if you can promote the policy.
Runnable cURL snippet
curl https://api.abliteration.ai/policy/chat/completions \
-H "Authorization: Bearer $POLICY_KEY" \
-H "Content-Type: application/json" \
-H "X-Policy-Target: support-bot" \
-d '{
"model": "abliterated-model",
"messages": [{"role":"user","content":"Summarize the refund policy."}],
"policy_id": "shadow-rollout"
}'Example policy JSON
{
"policy_id": "shadow-rollout",
"name": "Shadow rollout",
"owner": "Platform team",
"description": "Evaluate policy changes in shadow mode.",
"rules": {
"allowlist": ["refunds", "support"],
"denylist": ["credential theft"],
"flagged_categories": ["self-harm/intent", "sexual/minors"],
"response_pattern": "rewrite",
"rewrite_instead_of_refuse": true,
"redact": true,
"reason_codes": ["ALLOW", "REWRITE", "REDACT"]
},
"org_controls": {
"project_keys": true,
"user_quotas": false,
"audit_logs": true,
"data_classification": "confidential",
"user_quota": { "requests": 0, "tokens": 0, "window": "daily" },
"project_quota": { "requests": 0, "tokens": 0, "window": "monthly" }
},
"rollout": {
"shadow": { "enabled": true, "sample_percent": 100, "targets": ["support-bot"] },
"canary": { "enabled": false, "sample_percent": 0, "targets": [] },
"rollback_on_spike": true
},
"refusal_replacement": { "mode": "rewrite", "escalation_path": "policy-review@company.com" }
}Before and after
Before (no shadow mode)
Policy changes ship without preview and can break production UX.
After (shadow mode)
decision: rewrite rollout_mode: shadow enforced: false
Run the Policy Gateway simulator
Test shadow outcomes locally before you enable them on live traffic.
Run a simulationFAQ
Frequently asked questions.
Does shadow mode change the user response?
No. Shadow mode evaluates policy decisions without enforcing them in the live response.
Can I target shadow mode to one app?
Yes. Use policy_target targeting to scope shadow evaluation to a project or route.