Policy Gateway connectors
Open-source connectors for Next.js, FastAPI, LangChain, and LiteLLM to route traffic through Policy Gateway.
These connectors are small, open-source helpers that route traffic through the Policy Gateway AI endpoint.
Each connector includes a 5-minute quickstart, local dev mode, and sample log output for dashboards.
Quick start
Service notes
- Pricing model: Usage-based pricing (~$5 per 1M tokens) billed on total tokens (input + output). See the API pricing page for current plans.
- Data retention: No prompt/output retention by default. Operational telemetry (token counts, timestamps, error codes) is retained for billing and reliability.
- Compatibility: OpenAI-style /v1/chat/completions request and response format with a base URL switch.
- Latency: Depends on model size, prompt length, and load. Streaming reduces time-to-first-token.
- Throughput: Team plans include priority throughput. Actual throughput varies with demand.
- Rate limits: Limits vary by plan and load. Handle 429s with backoff and respect any Retry-After header.
Connector matrix
#All connector source lives in the repo under the /connectors directory:
connectors/policy-gateway-nextjs- Next.js App Router route handler wrapper.connectors/policy-gateway-fastapi- FastAPI dependency and client helper.connectors/policy-gateway-langchain- LangChain wrapper + audit callback.connectors/policy-gateway-litellm- LiteLLM helper for policy routing.
Next.js (App Router) quickstart
#Add a route handler that forwards requests to Policy Gateway:
import { createPolicyGatewayRoute } from "@/app/lib/policyGatewayRoute";
export const POST = createPolicyGatewayRoute({
apiKey: process.env.POLICY_GATEWAY_KEY,
policyId: "policy-gateway",
policyProjectId: "support-bot",
});FastAPI quickstart
#Use the PolicyGatewayClient dependency inside your FastAPI route:
from fastapi import FastAPI, Request
from policy_gateway import policy_gateway_client_from_env, policy_user_from_request
app = FastAPI()
client = policy_gateway_client_from_env()
@app.post("/chat")
async def chat(request: Request):
body = await request.json()
policy_user = policy_user_from_request(request)
return await client.chat_completions(body, policy_user=policy_user)LangChain quickstart
#Wrap ChatOpenAI with Policy Gateway headers and policy_id:
from policy_gateway_langchain import build_policy_gateway_chat
llm = build_policy_gateway_chat(
api_key="YOUR_POLICY_KEY",
policy_id="policy-gateway",
policy_user="user-12345",
policy_project_id="support-bot",
)
response = llm.invoke("Summarize our refund policy.")
print(response.content)LiteLLM quickstart
#Route LiteLLM traffic through /policy with extra headers:
from policy_gateway_litellm import policy_completion
response = policy_completion(
messages=[{"role": "user", "content": "Summarize our refund policy."}],
policy_id="policy-gateway",
policy_user="user-12345",
policy_project_id="support-bot",
)
print(response["choices"][0]["message"]["content"])Local dev mode
#Point connectors at a local Policy Gateway instance to test before production rollout:
export POLICY_GATEWAY_BASE_URL=http://localhost:5000/policy
Sample log output
#Each connector includes a sample log file you can plug into dashboards:
2026-01-13T10:12:42Z policy_id=policy-gateway decision=rewrite reason_code=REWRITE policy_user=user-12345 policy_project_id=support-bot
Common errors & fixes
- 401 Unauthorized: Check that your API key is set and sent as a Bearer token.
- 404 Not Found: Make sure the base URL ends with /v1 and you call /chat/completions.
- 400 Bad Request: Verify the model id and that messages are an array of { role, content } objects.
- 429 Rate limit: Back off and retry. Use the Retry-After header for pacing.