Best for: teams that want an open-source gateway they can self-host across a broad provider mix.
Categories › AI Gateway Software
Best AI Gateway Software 2026
LiteLLM is best overall for teams that want to run a gateway themselves: its MIT-licensed core connects one OpenAI-compatible API to 140+ providers and 1,800+ models. Cloudflare AI Gateway is best value for teams already on Cloudflare because its core analytics, caching and rate limiting are free. Portkey is the hosted pick for a published monthly fee: Production costs $49 a month and includes 100,000 recorded logs.
Updated
Compare the deciding features
| Product | Documented buyer fit |
|---|---|
| LiteLLM | Open-source gateway with one API for 140+ providers and 1,800+ models.[1] |
| Portkey | MIT-licensed gateway with one API for 250+ LLMs.[2] [3] [4] |
| Cloudflare AI Gateway | Free core AI Gateway includes analytics, caching and rate limiting.[5] |
| Kong AI Gateway | One endpoint for LLM, MCP and A2A traffic, with provider routing and load balancing.[6] |
| OpenRouter | One API for 600+ models and 80+ providers on Standard.[7] |
| TrueFoundry | Gateway for 1,600+ models with VPC, on-prem and air-gapped deployment.[8] |
| Vercel AI Gateway | One API key for 400+ models, with automatic fallbacks during provider outages.[9] |
| Helicone | Open-source LLM observability with gateway caching, rate limits and automatic fallbacks.[10] [11] |
Ranked shortlist
Best for: teams that want one API across 250+ LLMs and published hosted plans.
Best for: teams using Cloudflare that want free analytics, caching and rate limits for AI requests.
Best for: teams extending Kong API management to LLM, MCP and A2A traffic.
Best for: teams that want one API across 600+ models and 80+ providers on Standard.
Best for: teams that need one gateway across models with VPC, on-prem or air-gapped deployment.
Best for: teams building on Vercel that want one API across 400+ models and automatic fallbacks.
Best for: teams that lead with LLM request observability and need gateway caching, rate limits and fallbacks.
Evidence and sources
- LiteLLM official product pageRetrieved 2026-11-13T17:15:09Z
One OpenAI-compatible API to 140+ providers and 1,800+ models.
- Portkey AI Gateway repositoryRetrieved 2026-11-13T17:15:09Z
Route to 250+ LLMs with 1 fast & friendly API
- Portkey Gateway licenseRetrieved 2026-11-13T17:26:49Z
MIT License
- Portkey pricingRetrieved 2026-11-13T17:26:49Z
Developer Free Forever
- Cloudflare AI Gateway pricingRetrieved 2026-11-13T17:15:09Z
AI Gateway's core features available today are offered for free, and all it takes is a Cloudflare account and one line of code to get started
- Kong AI Gateway documentationRetrieved 2026-11-13T17:15:09Z
Define a single endpoint for any traffic type: LLM, MCP, or A2A.
- OpenRouter pricingRetrieved 2026-11-13T17:15:09Z
Standard 600+ models
- TrueFoundry AI GatewayRetrieved 2026-11-13T17:15:09Z
A unified AI gateway to securely manage and govern AI across 1600+ models
- Vercel AI GatewayRetrieved 2026-11-13T17:15:09Z
One API key, 400+ models.
- Helicone repositoryRetrieved 2026-11-13T17:15:09Z
Open source LLM observability platform. One line of code to monitor, evaluate, and experiment.
- Helicone pricingRetrieved 2026-11-13T17:15:09Z
Gateway Caching Rate limits Automatic fallbacks
1. LiteLLM: best for teams that want to run their own gateway
LiteLLM is an open-source gateway that gives one OpenAI-compatible API to 140+ providers and 1,800+ models. The code outside its enterprise directory is available under the MIT licence. Routing load-balances across multiple deployments (for example Azure and OpenAI) with cooldowns and fallbacks. It lists virtual keys, budgets, RPM/TPM limits, spend by key, user, team and organisation, tag-based spend tracking and audit logs. Logging goes out to Langfuse, OpenTelemetry and GCS, S3 or Azure Blob buckets.
Sources: LiteLLM source · LiteLLM source · LiteLLM source · LiteLLM source
2. Portkey: best for a hosted gateway with a published monthly price
Portkey routes to 250+ LLMs through one API, and its gateway repository is MIT-licensed. The Developer plan is free forever with 10k recorded logs a month. The Production plan costs $49 a month and includes 100k recorded logs, with $9 per additional 100k requests, 30 days of log retention and 90 days of metrics. Portkey also lists guardrails, simple and semantic caching and role-based access control.
Sources: Portkey source · Portkey source · Portkey source
3. Cloudflare AI Gateway: best for teams on Cloudflare that want free core features
Cloudflare lists AI Gateway's core features as free, and they include dashboard analytics, caching and rate limiting. It adds request retries and model fallback, and works from a Cloudflare account with one line of code. Caching serves identical requests from Cloudflare's cache instead of calling the model provider. Rate limits are set as a number of requests in a time frame, for example 100 requests per 60 seconds, with a fixed or sliding window.
Sources: Cloudflare AI Gateway source · Cloudflare AI Gateway source · Cloudflare AI Gateway source · Cloudflare AI Gateway source
4. Kong AI Gateway: best for teams that already run Kong for API management
Kong AI Gateway routes and load-balances across AI providers behind one endpoint for LLM, MCP or A2A traffic. The AI Management Plus plan starts at $25 a month plus usage, billed monthly; the Enterprise plan is billed annually. Model access and token budgets can be scoped by team or department, and usage analytics cover requests, tokens, errors and latency.
Sources: Kong AI Gateway source · Kong AI Gateway source
5. OpenRouter: best for one API across 600+ models
OpenRouter gives one API to 600+ models, with 80+ providers on the Standard plan. Inference is billed at the provider's list price; the fee is 5.5% on Standard and 8% on Business, charged when credits are bought, never on individual requests. A failed attempt is not charged for its model tokens, so with fallback you pay only for the run that answers.
Sources: OpenRouter source
6. TrueFoundry: best for per-seat pricing with VPC and on-prem options
TrueFoundry's AI Gateway manages and governs access to 1600+ models and supports VPC, on-prem and air-gapped deployment. Developer is free; Pro costs $25 per user a month with unlimited seats, SSO and custom roles; the Enterprise plan runs on an annual commitment. Pro includes 20K requests and tool calls per user a month (then $20 per additional 100K requests), built-in guardrails, budget limiting and 30-day log retention.
Sources: TrueFoundry source · TrueFoundry source
7. Vercel AI Gateway: best for teams building on Vercel that want zero token markup
Vercel AI Gateway offers one API key for 400+ models and automatic fallbacks during provider outages. It charges no markup and no platform fee on tokens, and every team gets $5 a month of included credit on the free tier. Budgets cap spend at team, project, API key or user scope, and a request must pass every budget in its scope.
Sources: Vercel AI Gateway source · Vercel AI Gateway source · Vercel AI Gateway source
8. Helicone: best for request logging first, with gateway features
Helicone is an Apache-2.0 open-source LLM observability platform that also offers gateway caching, rate limits and automatic fallbacks. Hobby is free with 10,000 requests and 1 GB of storage; Pro is $79 a month with unlimited seats and alerts; Team is $799 a month with SOC-2 and HIPAA compliance. Data retention runs 7 days on Hobby, 1 month on Pro, 3 months on Team and forever on Enterprise.
Sources: Helicone source · Helicone source
How it works
An AI gateway sits between your applications and model providers such as OpenAI, Anthropic and Google. Your code calls one OpenAI-compatible endpoint, and the gateway routes each request to a provider, retries or fails over when one is down, and records usage. Most gateways add the controls a single provider SDK lacks: virtual keys, budgets and rate limits per team, response caching, request logs and guardrails. Some run as a hosted service and some as software you run in your own cluster.
How to choose
Decide where the gateway must run: a hosted service, your own VPC or Kubernetes cluster, or a local data plane. Check the pricing basis: a platform fee on credits (OpenRouter), a monthly plan with a log allowance (Portkey), a per-seat plan (TrueFoundry), or free self-hosting (LiteLLM). List the controls you need on day one: budgets per team or key, request logs, caching, PII guardrails, SSO. Several of them sit in enterprise tiers.
Frequently asked questions
What is an AI gateway and what do teams use for routing requests across model providers?
An AI gateway is a proxy between your applications and model providers that exposes one OpenAI-compatible API, routes and falls back between providers, and records usage. LiteLLM, Portkey, Cloudflare AI Gateway, Kong AI Gateway and OpenRouter each route requests across providers.
Which LLM gateway can I self-host so prompts stay in my VPC?
LiteLLM is open source with an MIT-licensed core outside its enterprise directory, and Portkey's open-source gateway can be hosted yourself. TrueFoundry lists VPC, on-prem and air-gapped deployment.
How much does an AI gateway cost?
Portkey's Production plan is $49 a month, Helicone's Pro plan is $79 a month, TrueFoundry's Pro plan is $25 per user a month, and Kong's AI Management Plus plan starts at $25 a month plus usage. Cloudflare AI Gateway's core features are free, and OpenRouter charges a 5.5% fee on Standard credits.
Can a gateway fail over when OpenAI or Anthropic is down?
Yes. LiteLLM routing includes cooldowns and fallbacks, Cloudflare AI Gateway lists model fallback, and Vercel AI Gateway applies automatic fallbacks during provider outages.
Is an LLM proxy the same as an AI gateway?
The terms are used for the same layer: an LLM proxy or LLM gateway is a gateway that fronts model providers. LiteLLM titles its own page "Open-Source AI Gateway & LLM Proxy".
Methodology
Every price, limit, licence and feature comes from the vendor's or project's own pages, repository files and documentation, opened on 10 November 2026 (Europe/Helsinki), with the exact URL and a verbatim excerpt recorded for each fact. Prices are shown in the currency the vendor prints, with the billing period named. Vendor families are counted once. Picks follow documented fit for the buying situation and published facts, not paid placement or review-site scores. Vendor statements about compliance are quoted as the vendor states them.
How this comparison is made
Ranked on documented capabilities and verified pricing from official vendor sources. Every fact links to its source and check date, and the criteria are specific to this buying situation.
How we assessed this shortlist
Every price, limit, licence and feature comes from the vendor's or project's own pages, repository files and documentation, opened on 10 November 2026 (Europe/Helsinki), with the exact URL and a verbatim excerpt recorded for each fact. Prices are shown in the currency the vendor prints, with the billing period named. Vendor families are counted once. Picks follow documented fit for the buying situation and published facts, not paid placement or review-site scores. Vendor statements about compliance are quoted as the vendor states them.
Want your software in this comparison? A vendor profile listing costs $9.90 via Stripe and goes live within 24 hours of payment. Comparison-list placements cost a fee, and links are dofollow. Factual corrections are free.
Submit your software →Spot an error or a missing product?
Send a correction with a source link and we review it against the vendor's own documentation.
Suggest a correction