Best for: LiteLLM is an open-source gateway that gives one OpenAI-compatible API to 140+ providers and 1,800+ models.
Categories › AI Gateway Software
Best Self-Hosted and Open-Source AI Gateways 2026
LiteLLM is the pick for a self-hosted gateway that has to reach the most providers, with an MIT-licensed core and 140+ providers behind one OpenAI-compatible API. Agent Router, formerly Envoy AI Gateway, is the pick for Kubernetes teams routing to self-managed models, Bifrost is the pick for a one-command Docker start, and Helicone is the pick when request logging matters most. Portkey is the pick when you want an MIT-licensed gateway with a hosted and enterprise path.
Updated
Compare the deciding features
| Product | Documented buyer fit |
|---|---|
| LiteLLM | LiteLLM is an open-source gateway that gives one OpenAI-compatible API to 140+ providers and 1,800+ models.[1] [2] [3] [4] [5] |
| Agent Router | Agent Router, formerly Envoy AI Gateway, is an Apache 2.0 project of the Agentic AI Foundation that routes to 22 providers out of the box.[6] [7] |
| Bifrost | Bifrost is an Apache 2.0 gateway that unifies 23+ providers behind one OpenAI-compatible API with automatic failover, load balancing and semantic caching.[8] [9] [10] |
| Helicone | Helicone is an Apache-2.0 open-source LLM observability platform that also offers gateway caching, rate limits and automatic fallbacks.[11] [12] [13] |
| Portkey | Portkey routes to 250+ LLMs through one API, and its gateway repository is MIT-licensed.[14] [15] [16] |
| LLM Gateway | LLM Gateway is open source under AGPLv3, so you can self-host it for free, and it also runs as a hosted platform.[17] |
| Kong AI Gateway | Kong AI Gateway routes and load-balances across AI providers behind one endpoint for LLM, MCP or A2A traffic.[18] [19] [20] [21] |
Ranked shortlist
Best for: Agent Router, formerly Envoy AI Gateway, is an Apache 2.0 project of the Agentic AI Foundation that routes to 22 providers out of the box.
Best for: Bifrost is an Apache 2.0 gateway that unifies 23+ providers behind one OpenAI-compatible API with automatic failover, load balancing and semantic caching.
Best for: Helicone is an Apache-2.0 open-source LLM observability platform that also offers gateway caching, rate limits and automatic fallbacks.
Best for: Portkey routes to 250+ LLMs through one API, and its gateway repository is MIT-licensed.
Best for: LLM Gateway is open source under AGPLv3, so you can self-host it for free, and it also runs as a hosted platform.
Best for: Kong AI Gateway routes and load-balances across AI providers behind one endpoint for LLM, MCP or A2A traffic.
Evidence and sources
- LiteLLM official page or repositoryRetrieved 2026-11-10T01:12:10Z
One OpenAI-compatible API to 140+ providers and 1,800+ models.
- LiteLLM official page or repositoryRetrieved 2026-11-10T01:13:00Z
Content outside of the above mentioned directories or restrictions above is available under the MIT license
- LiteLLM official page or repositoryRetrieved 2026-11-10T01:13:02Z
Load-balance across multiple deployments (e.g. Azure/OpenAI)
- LiteLLM official page or repositoryRetrieved 2026-11-10T01:13:02Z
Log Proxy input, output, and exceptions using: Langfuse OpenTelemetry GCS, s3, Azure (Blob) Buckets
- LiteLLM official page or repositoryRetrieved 2026-11-10T01:13:01Z
Prompt Injection Detection, PII Masking on LiteLLM Proxy (AI Gateway)
- Agent Router (formerly Envoy AI Gateway) official page or repositoryRetrieved 2026-11-10T01:13:00Z
Envoy AI Gateway is now **Agent Router**, an Agentic AI Foundation project. Same code, same maintainers, same release cadence and Apache 2.0 license.
- Agent Router (formerly Envoy AI Gateway) official page or repositoryRetrieved 2026-11-10T01:12:11Z
Route to 22 providers out of the box
- Bifrost official page or repositoryRetrieved 2026-11-10T01:13:00Z
Bifrost is a high-performance AI gateway that unifies access to 23+ providers (OpenAI, Anthropic, AWS Bedrock, Google Vertex, and more) through a single OpenAI-compatible API.
- Bifrost official page or repositoryRetrieved 2026-11-10T01:13:00Z
Apache License Version 2.0
- Bifrost official page or repositoryRetrieved 2026-11-10T01:12:11Z
Try Bifrost Enterprise free for 14 days.
- Helicone official page or repositoryRetrieved 2026-11-10T01:12:12Z
Open source LLM observability platform. One line of code to monitor, evaluate, and experiment.
- Helicone official page or repositoryRetrieved 2026-11-10T01:12:10Z
Gateway Caching Rate limits Automatic fallbacks
- Helicone official page or repositoryRetrieved 2026-11-10T01:14:50Z
Self-Hosting Docker
- Portkey official page or repositoryRetrieved 2026-11-10T01:13:01Z
Route to 250+ LLMs with 1 fast & friendly API
- Portkey official page or repositoryRetrieved 2026-11-10T01:13:00Z
MIT License Copyright (c) 2024 Portkey, Inc
- Portkey official page or repositoryRetrieved 2026-11-10T01:12:10Z
Open Source ⓘ Host it Yourself Deploy Locally Universal API Retries & Timeouts Routing Guardrails Automatic Fallbacks Basic Dashboard Load Balancing Community Support
- LLM Gateway official page or repositoryRetrieved 2026-11-10T01:12:11Z
LLM Gateway is open source under AGPLv3, so you can self-host the gateway for free
- Kong AI Gateway official page or repositoryRetrieved 2026-11-10T01:12:10Z
Routing and load balancing across AI providers
- Kong AI Gateway official page or repositoryRetrieved 2026-11-10T01:13:01Z
Apache License Version 2.0
- Kong AI Gateway official page or repositoryRetrieved 2026-11-10T01:13:37Z
AI Gateway Enterprise: This plugin is only available as part of our AI Gateway Enterprise offering.
- Kong AI Gateway official page or repositoryRetrieved 2026-11-10T01:13:37Z
AI Gateway Enterprise: This plugin is only available as part of our AI Gateway Enterprise offering.
1. LiteLLM: best for the broadest provider coverage when you self-host
LiteLLM is an open-source gateway that gives one OpenAI-compatible API to 140+ providers and 1,800+ models. The code outside its enterprise directory is available under the MIT licence. Routing load-balances across multiple deployments (for example Azure and OpenAI) with cooldowns and fallbacks. Logging goes out to Langfuse, OpenTelemetry and GCS, S3 or Azure Blob buckets. Guardrails cover prompt-injection detection and PII masking on the proxy.
Sources: LiteLLM source · LiteLLM source · LiteLLM source · LiteLLM source
2. Agent Router (formerly Envoy AI Gateway): best for Kubernetes teams routing to self-managed models
Agent Router, formerly Envoy AI Gateway, is an Apache 2.0 project of the Agentic AI Foundation that routes to 22 providers out of the box. It handles provider fallback, model name virtualisation and token limits per team, app or model. It supports Kubernetes InferencePool for routing to self-managed models, and a hosted service from Tetrate is available.
Sources: Agent Router (formerly Envoy AI Gateway) source · Agent Router (formerly Envoy AI Gateway) source
3. Bifrost: best for a start with a single Docker command
Bifrost is an Apache 2.0 gateway that unifies 23+ providers behind one OpenAI-compatible API with automatic failover, load balancing and semantic caching. It starts with a single docker run command. Enterprise deployments add adaptive load balancing, clustering, guardrails and an MCP gateway, with a 14-day free trial.
Sources: Bifrost source · Bifrost source · Bifrost source
4. Helicone: best for self-hosted request logging with gateway features
Helicone is an Apache-2.0 open-source LLM observability platform that also offers gateway caching, rate limits and automatic fallbacks. Self-hosting is documented with Docker. Data retention runs 7 days on Hobby, 1 month on Pro, 3 months on Team and forever on Enterprise. Enterprise adds SAML SSO and on-prem deployment.
Sources: Helicone source · Helicone source · Helicone source
5. Portkey: best for an open-source gateway with a hosted and enterprise upgrade path
Portkey routes to 250+ LLMs through one API, and its gateway repository is MIT-licensed. The open-source gateway can be hosted yourself and includes routing, fallbacks, load balancing and retries. The Enterprise plan lists SSO, granular budget and rate limits, private cloud and VPC hosting, and compliance covering SOC 2 Type 2, GDPR and HIPAA.
Sources: Portkey source · Portkey source · Portkey source
6. LLM Gateway: best for teams that accept AGPLv3 and want a hosted fallback
LLM Gateway is open source under AGPLv3, so you can self-host it for free, and it also runs as a hosted platform. The hosted platform charges a flat 5% fee when you buy credits and no platform fee when you bring your own provider keys; models cover 200+ across 40+ providers. Logs hold metadata only by default at no cost.
Sources: LLM Gateway source
7. Kong AI Gateway: best for teams that want a local Docker data plane
Kong AI Gateway routes and load-balances across AI providers behind one endpoint for LLM, MCP or A2A traffic. A local data plane deploys with Docker, and the Kong Gateway repository is Apache 2.0-licensed. Some AI Gateway plugins are available only in the AI Gateway Enterprise offering.
Sources: Kong AI Gateway source · Kong AI Gateway source · Kong AI Gateway source · Kong AI Gateway source
How it works
A self-hosted AI gateway runs inside your own VPC or Kubernetes cluster, so prompts, responses and provider keys pass only through infrastructure you control. The gateway still calls external model providers unless you also route to models you host. Open-source gateways differ in licence (MIT, Apache 2.0, AGPLv3) and in which governance features sit in a paid tier.
How to choose
Read the licence: LiteLLM and Portkey's gateway are MIT, Helicone, Bifrost and Agent Router are Apache 2.0, and LLM Gateway is AGPLv3. Check which governance features (SSO, audit logs, budgets) need an enterprise licence; LiteLLM requires one for SSO beyond 5 users. Bifrost and Kong's data plane start from Docker; Agent Router adds Kubernetes InferencePool support.
Frequently asked questions
What is the best open-source AI gateway I can run in my own VPC?
LiteLLM is the pick for breadth: an MIT-licensed core, 140+ providers, routing with fallbacks and logging to Langfuse, OpenTelemetry or S3. Agent Router is the pick for Kubernetes teams and Bifrost is the pick for a quick Docker start.
Which open-source LLM proxy uses a permissive licence?
LiteLLM and Portkey's gateway are MIT-licensed, while Helicone, Bifrost and Agent Router are Apache 2.0.
Is there a self-hosted gateway that runs on Kubernetes?
Agent Router, formerly Envoy AI Gateway, supports Kubernetes InferencePool for routing to self-managed models.
Do I need to pay for SSO on a self-hosted gateway?
LiteLLM makes SSO free for up to 5 users and requires an enterprise licence beyond that. Helicone lists SAML SSO and on-prem deployment on its Enterprise plan.
Quelles passerelles LLM open source puis-je héberger moi-même ?
LiteLLM, la passerelle de Portkey, Helicone, Bifrost et Agent Router sont open source ; LLM Gateway est sous licence AGPLv3 et peut être auto-hébergée gratuitement.
Methodology
Every price, limit, licence and feature comes from the vendor's or project's own pages, repository files and documentation, opened on 10 November 2026 (Europe/Helsinki), with the exact URL and a verbatim excerpt recorded for each fact. Prices are shown in the currency the vendor prints, with the billing period named. Picks follow documented fit for the buying situation and published facts, not paid placement or review-site scores. Vendor statements about compliance are quoted as the vendor states them. Vendor families are counted once.
How this comparison is made
Ranked on documented capabilities and verified pricing from official vendor sources. Every fact links to its source and check date, and the criteria are specific to this buying situation.
How we assessed this shortlist
Every price, limit, licence and feature comes from the vendor's or project's own pages, repository files and documentation, opened on 10 November 2026 (Europe/Helsinki), with the exact URL and a verbatim excerpt recorded for each fact. Prices are shown in the currency the vendor prints, with the billing period named. Picks follow documented fit for the buying situation and published facts, not paid placement or review-site scores. Vendor statements about compliance are quoted as the vendor states them. Vendor families are counted once.
Want your software in this comparison? A vendor profile listing costs $9.90 via Stripe and goes live within 24 hours of payment. Comparison-list placements cost a fee, and links are dofollow. Factual corrections are free.
Submit your software →Spot an error or a missing product?
Send a correction with a source link and we review it against the vendor's own documentation.
Suggest a correction