B2B Review LabSoftware and business finance research

Categories › AI Gateway Software

Best Self-Hosted and Open-Source AI Gateways 2026

LiteLLM is the pick for a self-hosted gateway that has to reach the most providers, with an MIT-licensed core and 140+ providers behind one OpenAI-compatible API. Agent Router, formerly Envoy AI Gateway, is the pick for Kubernetes teams routing to self-managed models, Bifrost is the pick for a one-command Docker start, and Helicone is the pick when request logging matters most. Portkey is the pick when you want an MIT-licensed gateway with a hosted and enterprise path.

Updated

7 productsbuyer-specific decision factors

Compare the deciding features

Category-specific workflows and plan requirements
Best Self-Hosted and Open-Source AI Gateways 2026: documented features
ProductDocumented buyer fit
LiteLLMLiteLLM is an open-source gateway that gives one OpenAI-compatible API to 140+ providers and 1,800+ models.[1] [2] [3] [4] [5]
Agent RouterAgent Router, formerly Envoy AI Gateway, is an Apache 2.0 project of the Agentic AI Foundation that routes to 22 providers out of the box.[6] [7]
BifrostBifrost is an Apache 2.0 gateway that unifies 23+ providers behind one OpenAI-compatible API with automatic failover, load balancing and semantic caching.[8] [9] [10]
HeliconeHelicone is an Apache-2.0 open-source LLM observability platform that also offers gateway caching, rate limits and automatic fallbacks.[11] [12] [13]
PortkeyPortkey routes to 250+ LLMs through one API, and its gateway repository is MIT-licensed.[14] [15] [16]
LLM GatewayLLM Gateway is open source under AGPLv3, so you can self-host it for free, and it also runs as a hosted platform.[17]
Kong AI GatewayKong AI Gateway routes and load-balances across AI providers behind one endpoint for LLM, MCP or A2A traffic.[18] [19] [20] [21]

Ranked shortlist

  1. Rank1

    LiteLLM

    Best for: LiteLLM is an open-source gateway that gives one OpenAI-compatible API to 140+ providers and 1,800+ models.

  2. Best for: Agent Router, formerly Envoy AI Gateway, is an Apache 2.0 project of the Agentic AI Foundation that routes to 22 providers out of the box.

  3. Rank3

    Bifrost

    Best for: Bifrost is an Apache 2.0 gateway that unifies 23+ providers behind one OpenAI-compatible API with automatic failover, load balancing and semantic caching.

  4. Best for: Helicone is an Apache-2.0 open-source LLM observability platform that also offers gateway caching, rate limits and automatic fallbacks.

  5. Rank5

    Portkey

    Best for: Portkey routes to 250+ LLMs through one API, and its gateway repository is MIT-licensed.

  6. Best for: LLM Gateway is open source under AGPLv3, so you can self-host it for free, and it also runs as a hosted platform.

  7. Best for: Kong AI Gateway routes and load-balances across AI providers behind one endpoint for LLM, MCP or A2A traffic.

Evidence and sources

  1. LiteLLM official page or repositoryRetrieved 2026-11-10T01:12:10Z

    One OpenAI-compatible API to 140+ providers and 1,800+ models.

  2. LiteLLM official page or repositoryRetrieved 2026-11-10T01:13:00Z

    Content outside of the above mentioned directories or restrictions above is available under the MIT license

  3. LiteLLM official page or repositoryRetrieved 2026-11-10T01:13:02Z

    Load-balance across multiple deployments (e.g. Azure/OpenAI)

  4. LiteLLM official page or repositoryRetrieved 2026-11-10T01:13:02Z

    Log Proxy input, output, and exceptions using: Langfuse OpenTelemetry GCS, s3, Azure (Blob) Buckets

  5. LiteLLM official page or repositoryRetrieved 2026-11-10T01:13:01Z

    Prompt Injection Detection, PII Masking on LiteLLM Proxy (AI Gateway)

  6. Agent Router (formerly Envoy AI Gateway) official page or repositoryRetrieved 2026-11-10T01:13:00Z

    Envoy AI Gateway is now **Agent Router**, an Agentic AI Foundation project. Same code, same maintainers, same release cadence and Apache 2.0 license.

  7. Agent Router (formerly Envoy AI Gateway) official page or repositoryRetrieved 2026-11-10T01:12:11Z

    Route to 22 providers out of the box

  8. Bifrost official page or repositoryRetrieved 2026-11-10T01:13:00Z

    Bifrost is a high-performance AI gateway that unifies access to 23+ providers (OpenAI, Anthropic, AWS Bedrock, Google Vertex, and more) through a single OpenAI-compatible API.

  9. Bifrost official page or repositoryRetrieved 2026-11-10T01:13:00Z

    Apache License Version 2.0

  10. Bifrost official page or repositoryRetrieved 2026-11-10T01:12:11Z

    Try Bifrost Enterprise free for 14 days.

  11. Helicone official page or repositoryRetrieved 2026-11-10T01:12:12Z

    Open source LLM observability platform. One line of code to monitor, evaluate, and experiment.

  12. Helicone official page or repositoryRetrieved 2026-11-10T01:12:10Z

    Gateway Caching Rate limits Automatic fallbacks

  13. Helicone official page or repositoryRetrieved 2026-11-10T01:14:50Z

    Self-Hosting Docker

  14. Portkey official page or repositoryRetrieved 2026-11-10T01:13:01Z

    Route to 250+ LLMs with 1 fast & friendly API

  15. Portkey official page or repositoryRetrieved 2026-11-10T01:13:00Z

    MIT License Copyright (c) 2024 Portkey, Inc

  16. Portkey official page or repositoryRetrieved 2026-11-10T01:12:10Z

    Open Source ⓘ Host it Yourself Deploy Locally Universal API Retries & Timeouts Routing Guardrails Automatic Fallbacks Basic Dashboard Load Balancing Community Support

  17. LLM Gateway official page or repositoryRetrieved 2026-11-10T01:12:11Z

    LLM Gateway is open source under AGPLv3, so you can self-host the gateway for free

  18. Kong AI Gateway official page or repositoryRetrieved 2026-11-10T01:12:10Z

    Routing and load balancing across AI providers

  19. Kong AI Gateway official page or repositoryRetrieved 2026-11-10T01:13:01Z

    Apache License Version 2.0

  20. Kong AI Gateway official page or repositoryRetrieved 2026-11-10T01:13:37Z

    AI Gateway Enterprise: This plugin is only available as part of our AI Gateway Enterprise offering.

  21. Kong AI Gateway official page or repositoryRetrieved 2026-11-10T01:13:37Z

    AI Gateway Enterprise: This plugin is only available as part of our AI Gateway Enterprise offering.

1. LiteLLM: best for the broadest provider coverage when you self-host

LiteLLM is an open-source gateway that gives one OpenAI-compatible API to 140+ providers and 1,800+ models. The code outside its enterprise directory is available under the MIT licence. Routing load-balances across multiple deployments (for example Azure and OpenAI) with cooldowns and fallbacks. Logging goes out to Langfuse, OpenTelemetry and GCS, S3 or Azure Blob buckets. Guardrails cover prompt-injection detection and PII masking on the proxy.

Sources: LiteLLM source · LiteLLM source · LiteLLM source · LiteLLM source

2. Agent Router (formerly Envoy AI Gateway): best for Kubernetes teams routing to self-managed models

Agent Router, formerly Envoy AI Gateway, is an Apache 2.0 project of the Agentic AI Foundation that routes to 22 providers out of the box. It handles provider fallback, model name virtualisation and token limits per team, app or model. It supports Kubernetes InferencePool for routing to self-managed models, and a hosted service from Tetrate is available.

Sources: Agent Router (formerly Envoy AI Gateway) source · Agent Router (formerly Envoy AI Gateway) source

3. Bifrost: best for a start with a single Docker command

Bifrost is an Apache 2.0 gateway that unifies 23+ providers behind one OpenAI-compatible API with automatic failover, load balancing and semantic caching. It starts with a single docker run command. Enterprise deployments add adaptive load balancing, clustering, guardrails and an MCP gateway, with a 14-day free trial.

Sources: Bifrost source · Bifrost source · Bifrost source

4. Helicone: best for self-hosted request logging with gateway features

Helicone is an Apache-2.0 open-source LLM observability platform that also offers gateway caching, rate limits and automatic fallbacks. Self-hosting is documented with Docker. Data retention runs 7 days on Hobby, 1 month on Pro, 3 months on Team and forever on Enterprise. Enterprise adds SAML SSO and on-prem deployment.

Sources: Helicone source · Helicone source · Helicone source

5. Portkey: best for an open-source gateway with a hosted and enterprise upgrade path

Portkey routes to 250+ LLMs through one API, and its gateway repository is MIT-licensed. The open-source gateway can be hosted yourself and includes routing, fallbacks, load balancing and retries. The Enterprise plan lists SSO, granular budget and rate limits, private cloud and VPC hosting, and compliance covering SOC 2 Type 2, GDPR and HIPAA.

Sources: Portkey source · Portkey source · Portkey source

6. LLM Gateway: best for teams that accept AGPLv3 and want a hosted fallback

LLM Gateway is open source under AGPLv3, so you can self-host it for free, and it also runs as a hosted platform. The hosted platform charges a flat 5% fee when you buy credits and no platform fee when you bring your own provider keys; models cover 200+ across 40+ providers. Logs hold metadata only by default at no cost.

Sources: LLM Gateway source

7. Kong AI Gateway: best for teams that want a local Docker data plane

Kong AI Gateway routes and load-balances across AI providers behind one endpoint for LLM, MCP or A2A traffic. A local data plane deploys with Docker, and the Kong Gateway repository is Apache 2.0-licensed. Some AI Gateway plugins are available only in the AI Gateway Enterprise offering.

Sources: Kong AI Gateway source · Kong AI Gateway source · Kong AI Gateway source · Kong AI Gateway source

How it works

A self-hosted AI gateway runs inside your own VPC or Kubernetes cluster, so prompts, responses and provider keys pass only through infrastructure you control. The gateway still calls external model providers unless you also route to models you host. Open-source gateways differ in licence (MIT, Apache 2.0, AGPLv3) and in which governance features sit in a paid tier.

How to choose

Read the licence: LiteLLM and Portkey's gateway are MIT, Helicone, Bifrost and Agent Router are Apache 2.0, and LLM Gateway is AGPLv3. Check which governance features (SSO, audit logs, budgets) need an enterprise licence; LiteLLM requires one for SSO beyond 5 users. Bifrost and Kong's data plane start from Docker; Agent Router adds Kubernetes InferencePool support.

Frequently asked questions

What is the best open-source AI gateway I can run in my own VPC?

LiteLLM is the pick for breadth: an MIT-licensed core, 140+ providers, routing with fallbacks and logging to Langfuse, OpenTelemetry or S3. Agent Router is the pick for Kubernetes teams and Bifrost is the pick for a quick Docker start.

Which open-source LLM proxy uses a permissive licence?

LiteLLM and Portkey's gateway are MIT-licensed, while Helicone, Bifrost and Agent Router are Apache 2.0.

Is there a self-hosted gateway that runs on Kubernetes?

Agent Router, formerly Envoy AI Gateway, supports Kubernetes InferencePool for routing to self-managed models.

Do I need to pay for SSO on a self-hosted gateway?

LiteLLM makes SSO free for up to 5 users and requires an enterprise licence beyond that. Helicone lists SAML SSO and on-prem deployment on its Enterprise plan.

Quelles passerelles LLM open source puis-je héberger moi-même ?

LiteLLM, la passerelle de Portkey, Helicone, Bifrost et Agent Router sont open source ; LLM Gateway est sous licence AGPLv3 et peut être auto-hébergée gratuitement.

Methodology

Every price, limit, licence and feature comes from the vendor's or project's own pages, repository files and documentation, opened on 10 November 2026 (Europe/Helsinki), with the exact URL and a verbatim excerpt recorded for each fact. Prices are shown in the currency the vendor prints, with the billing period named. Picks follow documented fit for the buying situation and published facts, not paid placement or review-site scores. Vendor statements about compliance are quoted as the vendor states them. Vendor families are counted once.

How this comparison is made

Ranked on documented capabilities and verified pricing from official vendor sources. Every fact links to its source and check date, and the criteria are specific to this buying situation.

Read the methodology →

How we assessed this shortlist

Every price, limit, licence and feature comes from the vendor's or project's own pages, repository files and documentation, opened on 10 November 2026 (Europe/Helsinki), with the exact URL and a verbatim excerpt recorded for each fact. Prices are shown in the currency the vendor prints, with the billing period named. Picks follow documented fit for the buying situation and published facts, not paid placement or review-site scores. Vendor statements about compliance are quoted as the vendor states them. Vendor families are counted once.

Want your software in this comparison? A vendor profile listing costs $9.90 via Stripe and goes live within 24 hours of payment. Comparison-list placements cost a fee, and links are dofollow. Factual corrections are free.

Submit your software →

Spot an error or a missing product?

Send a correction with a source link and we review it against the vendor's own documentation.

Suggest a correction