Shadow mode for AI policy changes

Definition of Shadow mode for AI policy changes

Shadow mode for AI policy changes evaluates new policies against live traffic without enforcing them, producing audit logs and decision metadata for comparison.

Why Shadow mode for AI policy changes matters

• Validate policy changes before enforcement.
• Detect false positives early.
• Keep audit logs for policy approvals and compliance reviews.

How it works

• Enable rollout.shadow in policy JSON and set a sample_percent.
• Use policy_target to scope shadow evaluation to specific apps or routes.
• Compare shadow decisions against enforced output to decide if you can promote the policy.

Runnable cURL snippet

curl https://api.abliteration.ai/policy/chat/completions \
  -H "Authorization: Bearer $POLICY_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Policy-Target: support-bot" \
  -d '{
    "model": "abliterated-model",
    "messages": [{"role":"user","content":"Summarize the refund policy."}],
    "policy_id": "shadow-rollout"
  }'

Example policy JSON

{
  "policy_id": "shadow-rollout",
  "name": "Shadow rollout",
  "owner": "Platform team",
  "description": "Evaluate policy changes in shadow mode.",
  "rules": {
    "allowlist": ["refunds", "support"],
    "denylist": ["credential theft"],
    "flagged_categories": ["self-harm/intent", "sexual/minors"],
    "response_pattern": "rewrite",
    "rewrite_instead_of_refuse": true,
    "redact": true,
    "reason_codes": ["ALLOW", "REWRITE", "REDACT"]
  },
  "org_controls": {
    "project_keys": true,
    "user_quotas": false,
    "audit_logs": true,
    "data_classification": "confidential",
    "user_quota": { "requests": 0, "tokens": 0, "window": "daily" },
    "project_quota": { "requests": 0, "tokens": 0, "window": "monthly" }
  },
  "rollout": {
    "shadow": { "enabled": true, "sample_percent": 100, "targets": ["support-bot"] },
    "canary": { "enabled": false, "sample_percent": 0, "targets": [] },
    "rollback_on_spike": true
  },
  "refusal_replacement": { "mode": "rewrite", "escalation_path": "policy-review@company.com" }
}

Before and after

Policy changes ship without preview and can break production UX.

decision: rewrite
rollout_mode: shadow
enforced: false

Run the Policy Gateway simulator

Test shadow outcomes locally before you enable them on live traffic.