Cloudflare AI Gateway: Auto Router, Costs, and Setup

👁 114 reads

AI Gateway is Cloudflare’s free proxy layer for routing requests to OpenAI, Anthropic, Google, and open-source models, giving you centralized logs, cost controls, and rate limits. As of September 30, 2026, it includes Auto Router, an intelligent routing feature that selects cheaper models for simpler prompts - dropping the cost per successful task from $0.0210 to $0.0084 in Cloudflare’s internal benchmark. For a broader overview of the ecosystem, check out our guide to Cloudflare.

Why put a gateway in front of your LLMs

Once your application or team calls multiple models, standard operational friction sets in: scattered API keys, blind spots around spend, and full outages when a single provider goes down. Placing AI Gateway in the middle provides:

• Logs and observability: Every prompt, token count, cost estimate, and latency figure logged in one place.
• Response caching: Identical requests serve from cache, cutting redundant spend to zero.
• Rate limits and budgets: Prevent runaway agents or single users from blowing through monthly budgets overnight.
• Automated fallbacks: Automatically route to backup providers if your primary model fails.
• Data loss prevention (DLP): Detect sensitive data in prompts before it leaves your infrastructure (free tier includes two built-in profiles).
• Unified key management: Bring your own keys (BYOK) or pay via Cloudflare credits.

The core concept mirrors OpenRouter. The key distinction: OpenRouter is primarily a model marketplace with consolidated billing, while AI Gateway is an infrastructure control layer that works natively with your direct provider accounts. For a breakdown of intermediary pricing structures, read our analysis on how OpenRouter pricing actually works.

Auto Router: mechanics and cost savings

Instead of hardcoding a specific model, you pass cloudflare/auto as the model identifier. The gateway then runs through a four-step pipeline:

1. Filters models capable of handling the request based on format, configured keys, budget ceilings, and provider uptime.
2. Passes the latest conversation turns through a lightweight classifier running on Cloudflare GPUs. This categorizes the task into one of 14 domains (coding, planning, research, data analysis, etc.) and scores it from 1 to 5 across complexity, ambiguity, stakes, and context dependency.
3. Balances classifier scores against benchmark data and token prices - weighting price higher for straightforward prompts and quality higher for demanding tasks.
4. Factors in prompt caching overhead: switching models mid-thread forces a re-upload of the full context, so the router only pivots if the cost delta justifies breaking cache.

Cloudflare’s benchmark results

Cloudflare evaluated Auto Router against an internal suite of 97 workplace tasks spanning email, calendar management, Slack interactions, file processing, travel logistics, and finance, running three trials per model:

In short: Auto Router cost roughly 35% of Claude Opus 5.5 and 80% of GPT-6 Sol, delivering accuracy on par with Sol while trailing Opus by 10 percentage points. If your workload consists primarily of simple tasks like summaries, routine emails, and classification, real-world savings will skew even higher. If your queries demand deep reasoning, the router will still default to premium frontier models. Keep in mind these are vendor-provided figures without independent validation yet. The router feature itself is free in beta; you only pay for downstream inference.

Cloudflare has also announced cloudflare/auto-best, which routes strictly for maximum output quality from your configured pool without factoring in cost.

Web Search API integration

Starting October 2, AI Gateway provides native web search access via three search providers: Ceramic.ai, Exa, and Linkup. Results inject directly into context windows, billed against standard Cloudflare credits at partner list rates without markup. Notably, partner crawlers must adhere to Cloudflare’s “Verified bots” protocol (declaring identities and honoring robots.txt), and every returned snippet requires a direct source link - a welcome development for publishers handling agent traffic.

How to connect: Python example

Because the gateway exposes an OpenAI-compatible endpoint, migrating existing applications requires changing only your base URL and authentication header. The implementation below is based on Cloudflare’s October 2026 documentation. We have not executed this script, as it requires active account credentials, an account ID, and an AI Gateway API token.

from openai import OpenAI

ACCOUNT_ID = "your_account_id"      # Cloudflare dashboard, right sidebar
GATEWAY_ID = "default"              # Gateway name
CF_TOKEN   = "your_api_token"       # Token with AI Gateway permissions

client = OpenAI(
    api_key=CF_TOKEN,
    base_url=f"https://gateway.ai.cloudflare.com/v1/{ACCOUNT_ID}/{GATEWAY_ID}/compat",
)

response = client.chat.completions.create(
    model="cloudflare/auto",        # Or e.g. "openai/...", "anthropic/...", "workers-ai/@cf/..."
    messages=[{"role": "user", "content": "Summarize in 3 bullet points: what is a CDN?"}],
)
print(response.choices[0].message.content)

Key structural points to notice:

• Model identifiers: Structured as provider/model, such as workers-ai/@cf/meta/llama-3.3-70b-instruct-fp8-fast.
• Authentication: If using unified credits or storing keys inside the gateway, use your Cloudflare token as the bearer key. If supplying your own provider keys dynamically, set your provider key as api_key and pass the Cloudflare token via the cf-aig-authorization header.
• Drop-in integration: Request and response payloads follow the standard OpenAI schema without requiring structural app changes.

If you are working with these interfaces for the first time, take a look at our guide explaining what tokens are and how they are calculated before scaling your deployments.

Pricing breakdown

Who this is for: development teams spending several hundred dollars a month or more on inference who need aggregate visibility, strict audit logs, and unified rate limiting. Solo developers experimenting with occasional prompts can comfortably rely on direct API keys with provider-level spend caps.

Trade-offs and limitations

• Third-party data transit: Your prompt payloads traverse an additional intermediary. Cloudflare has announced plans for zero-data-retention (ZDR) routing, but Auto Router does not yet enforce this across all targets.
• Black-box decision making: Logs show the selected model and task category, but you cannot manually tune internal routing weights or override classifier logic.
• x402 payment protocol: Real-time, pay-per-request model payments bypassing credit top-ups are currently limited to US accounts at rollout.

Sources

• Cloudflare - Auto Router (Sep 30, 2026)
• Cloudflare - Web Search API (Oct 2, 2026)
• Cloudflare - AI Gateway Pricing
• Cloudflare - OpenAI Compatible Endpoints
• Cloudflare - Monetization Gateway and x402 (Sep 30, 2026)

// read moreAI Model API Pricing: Real Costs per 1,000 PromptsBenchmarksCloudflare Pages vs Vercel for AI Websites in 2026AI ToolsCloudflare AI Bot Settings: Block vs Disallow AI TrainingGuides

Kinetyra