Policy-as-code for LLM behavior

Definition of Policy-as-code for LLM behavior

Policy-as-code for LLM behavior is the practice of defining LLM rules in versioned, reviewable JSON so outcomes are predictable and auditable.

Why Policy-as-code for LLM behavior matters

• Policies are versioned and can be reviewed like code.
• Reason codes make moderation outcomes deterministic.
• Rollouts and simulations reduce risk during policy changes.

How it works

• Define allowlist and denylist topics in policy JSON.
• Add flagged categories and response patterns for safe outputs.
• Apply the policy via the AI gateway endpoint with policy_id.

Runnable cURL snippet

curl https://api.abliteration.ai/policy/chat/completions \
  -H "Authorization: Bearer $POLICY_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Policy-User: user-4481" \
  -H "X-Policy-Project: knowledge-bot" \
  -d '{
    "model": "abliterated-model",
    "messages": [{"role":"user","content":"Share our internal incident response steps."}],
    "policy_id": "policy-as-code"
  }'

Example policy JSON

{
  "policy_id": "policy-as-code",
  "name": "Policy as code",
  "owner": "Security team",
  "description": "Explicit allowlist and denylist behavior.",
  "rules": {
    "allowlist": ["public docs", "status page"],
    "denylist": ["internal incident", "credential sharing", "security bypass"],
    "flagged_categories": ["self-harm/intent", "violence/graphic", "sexual/minors"],
    "response_pattern": "refuse",
    "rewrite_instead_of_refuse": false,
    "redact": true,
    "reason_codes": ["ALLOW", "REFUSE", "REDACT"]
  },
  "org_controls": {
    "project_keys": true,
    "user_quotas": true,
    "audit_logs": true,
    "data_classification": "restricted",
    "user_quota": { "requests": 100, "tokens": 10000, "window": "daily" },
    "project_quota": { "requests": 5000, "tokens": 500000, "window": "monthly" }
  },
  "rollout": {
    "shadow": { "enabled": true, "sample_percent": 25, "targets": ["knowledge-bot"] },
    "canary": { "enabled": true, "sample_percent": 10, "targets": ["knowledge-bot"] },
    "rollback_on_spike": true
  },
  "refusal_replacement": { "mode": "refuse", "escalation_path": "security@company.com" }
}

Before and after

Assistant shares internal steps without a clear rule set.

decision: refuse
reason_code: REFUSE
denylist_hits: ["internal incident"]

Run the Policy Gateway simulator

Validate policy-as-code rules before rolling them into production.