LiteLLM Proxy Self-Hosted
Self-hosted AI API gateway providing model routing, access controls and usage tracking.
10 sources checked by an editor on · Product website ↗
Read this page as JSON or Markdown.
Deployment terms: BYOC · control plane · data plane · private offer
· Open requirements shortlist · Estimate operating costs
Compare this listing with another →
LiteLLM LTS proxy security fixes
LiteLLM 1.108.11 LTS fixes Bedrock guarded_text handling and restricts os.environ team settings to proxy administrators.
Scope: LiteLLM 1.108 LTS line; standard-line deployments should use their corresponding maintained release.
Applicability depends on Bedrock guardrails and team-setting permissions.
Action: Upgrade the selected release line and recheck guardrail output and proxy-administrator permissions.
Primary source · Reviewed 2026-11-18
LiteLLM 1.108.12 LTS tool-policy fix
LiteLLM 1.108.12 LTS prevents blocked tool names longer than 64 characters from bypassing tool_policy on /v1/messages.
Scope: LiteLLM 1.108 LTS /v1/messages tool-policy enforcement.
The request must use a blocked tool name longer than 64 characters.
Action: Upgrade the maintained LTS line to 1.108.12 or later and retest tool-policy enforcement.
Primary source · Reviewed 2026-11-19
What lands where
Supplied by
Vendor or project
Official container image, Docker Compose or Kubernetes deployment; production infrastructure examples are documented.
Runs in
Documented customer environment
Customer-operated gateway with optional PostgreSQL persistence and configured model endpoints
- AWS: Customer cloud account
- Azure: Customer cloud account
- Google Cloud: Customer cloud account
- Railway: Customer platform workspace
- Render: Customer platform workspace
- AWS
- Azure
- Google Cloud
- Railway
- Render
Servers / containers
Operated by
Your team
Customer manages gateway availability, credentials, database, access policies, logs and upgrades, including pinning versions and verifying release artifacts with the documented version-appropriate Cosign method; model providers remain separate services.
Deployment boundaries by provider
- AWS: Customer cloud account · Customer-operated
Resources in the customer's cloud account, subscription, project or tenancy. Managed cloud services still have shared responsibilities.
LiteLLM production deployment ↗ · Source checked 2026-11-12
- Azure: Customer cloud account · Customer-operated
Resources in the customer's cloud account, subscription, project or tenancy. Managed cloud services still have shared responsibilities.
LiteLLM production deployment ↗ · Source checked 2026-11-12
- Google Cloud: Customer cloud account · Customer-operated
Resources in the customer's cloud account, subscription, project or tenancy. Managed cloud services still have shared responsibilities.
LiteLLM production deployment ↗ · Source checked 2026-11-12
- Railway: Customer platform workspace · Customer-operated
Customer-controlled application services in a hosting-platform project. The platform operates the underlying infrastructure.
LiteLLM Railway and Render deployment options ↗ · Source checked 2026-11-12
- Render: Customer platform workspace · Customer-operated
Customer-controlled application services in a hosting-platform project. The platform operates the underlying infrastructure.
LiteLLM Railway and Render deployment options ↗ · Source checked 2026-11-12
Application operations and account ownership are separate from infrastructure operations. Confirm external services, telemetry, support access and backup destinations for the chosen path.
| Install method | Official container image, Docker Compose or Kubernetes deployment; production infrastructure examples are documented. |
|---|---|
| Infrastructure | Servers / containers The gateway runs as provisioned application containers or Kubernetes workloads; persistent key/spend storage can use PostgreSQL. Inference compute is a separate dependency rather than included in the gateway. |
| Licensing model | Open Source / Open Core The gateway has an MIT-licensed base and separately licensed enterprise features. The enterprise documentation describes a license key enabling restricted capabilities in the gateway, beyond paid hosting or support alone. |
| License | MIT outside the enterprise directory; enterprise code and features have separate license terms. |
Buyer diligence
Answers apply to the reviewed edition and deployment. Expand a row for qualifications and sources.
How to read these facts
Not researched means no field-specific review is recorded. Not confirmed means sources were checked but did not establish an answer. Availability, eligibility and edition-dependent findings retain the scope given in their evidence. A vendor’s hosted-service certification does not automatically cover your deployment. Evidence coverage is not a feature score.
6 of 12 fields researched (50%) · 6 documented · 0 unresolved. Evidence coverage is not a feature score.
- Single sign-on
Edition-dependentEvidence and scope for Single sign-on
The Admin UI documents OIDC SSO. From 1.76.0, it is free for up to five users; larger deployments require an Enterprise licence. Configure issuer, callbacks, roles and database/master-key prerequisites. This is Admin UI identity, distinct from model API virtual keys.
LiteLLM Admin UI OIDC; user-count/Enterprise entitlement applies · Source ↗ · Reviewed 2026-11-20
Edition-dependent · Editor reviewed.
- Security controls
DocumentedEvidence and scope for Security controls
Release images and Helm charts use keyless Cosign from v1.111.0-dev.3; verify the documented workflow identity and issuer. Earlier releases use historical keys. The October 21 v1.107.0 compromise also affected mutable image tags; compare digests and exposure windows because affected images had valid historical signatures. Follow the incident remediation guidance for executed artifacts. Current advisory review: LiteLLM 1.108.11 LTS fixes Bedrock guarded_text handling and restricts os.environ team settings to proxy administrators. Upgrade the selected release line and recheck guardrail output and proxy-administrator permissions. Current review: v1.108.12 LTS fixes tool_policy allowing a /v1/messages request through when a blocked tool name exceeded 64 characters. Upgrade the maintained LTS line and retest tool-policy enforcement.
LiteLLM 1.108 LTS /v1/messages tool_policy; standard and other maintained lines require their own release review · Source ↗ · Prior security-controls source ↗ · Canonical incident investigation ↗ · Prior security-controls source ↗ · Reviewed 2026-11-19
Documented · Editor reviewed.
- Deployment artifact / IaC
DocumentedEvidence and scope for Deployment artifact / IaC
The official Helm guide documents Kubernetes installation and configurable database/secret settings. Select matching chart/image versions and review the separate scaling and production-readiness guidance.
Upstream self-hosted deployment; not a provider support or production-readiness certification · Source ↗ · Reviewed 2026-11-14
Documented · Editor reviewed.
- Minimum footprint / sizing
DocumentedEvidence and scope for Minimum footprint / sizing
The current production guide starts load testing at 1 vCPU and 4 GiB per worker, preferably one worker per Kubernetes pod. Memory depends on queued requests, payloads, stream duration and logging. PostgreSQL, Redis for multiple workers/instances, HA and model inference are additional.
Gateway per-worker load-test starting point; backing services and inference excluded · Source ↗ · Reviewed 2026-11-20
Documented · Editor reviewed.
- License pricing
DocumentedEvidence and scope for License pricing
Core code outside litellm/enterprise is MIT licensed; the enterprise directory has separate terms. Enterprise features and support need separate entitlement and pricing. Upstream model APIs, infrastructure and operating labour are additional.
Upstream core licence; commercial features, support and hosting priced separately · Source ↗ · Reviewed 2026-11-20
Documented · Editor reviewed.
- Release snapshot / cadence
DocumentedEvidence and scope for Release snapshot / cadence
LiteLLM v1.110.1 was published November 19, 2026. It refreshes the Wolfi base and changes the UI image to slim nginx, removing curl. The maintained v1.108.12 LTS line fixes a tool_policy bypass for blocked tool names longer than 64 characters.
Upstream LiteLLM standard release; select and maintain either the standard or LTS line · Source ↗ · Prior reviewed release context ↗ · LiteLLM v1.108.11 LTS ↗ · v1.108.12 LTS security fix ↗ · Reviewed 2026-11-19
Documented · Editor reviewed.
Show all fields (6 not researched)
- Marketplace delivery
- Not researched Supply a public source.
- Cloud commitment eligibility
- Not researched Supply a public source.
- SCIM provisioning
- Not researched Supply a public source.
- Compliance attestations
- Not researched Supply a public source.
- Air-gapped operation
- Not researched Supply a public source.
- Operating cost estimate
- Not researched Supply a public source.
Scope & considerations
The gateway forwards each request to its configured model provider. Self-hosting the proxy does not ensure prompts stay inside the account: external model APIs, guardrail services and logging integrations can receive data. Use appropriate self-hosted inference endpoints and explicitly review outbound integrations when required. For release integrity, release images and the Helm chart starting with v1.111.0-dev.3 use keyless Cosign signatures bound to LiteLLM's release workflow; earlier releases require the documented historical public keys. The October 21 incident affected PyPI v1.107.0 and Docker Hub/GHCR v1.107.0 plus mutable latest, main-latest and main-stable tags during the documented exposure windows. Check the incident's artifact digests and remediation instructions; the compromised images carried valid historical signatures, so signature verification alone does not establish safety. LiteLLM v1.110.0 changes authentication, guardrail, accounting, and S3 behavior. The 1.108.11 LTS patch also fixes Bedrock guarded_text handling and restricts os.environ team settings to proxy administrators; review both the selected release line and existing incident-response or image-signing guidance. LiteLLM v1.108.12 LTS fixes tool_policy allowing blocked /v1/messages tool names longer than 64 characters. The standard v1.110.1 UI image changes to slim nginx and removes curl while retaining wget and nc, so retest health probes and scripts that assume curl.
Category fit
- AI inference & gateways
- Self-hosted model API gateway for routing, virtual keys, spend tracking and policy enforcement. It routes inference requests; it is not an inference engine for all supported models.
- AI tools & infrastructure
- Self-hosted AI API gateway providing model routing, access controls and usage tracking.
Compare within AI inference & gateways
- LocalAILocal model inference service exposing configurable AI capabilities through compatible API endpoints.
- AWS
- NVIDIA AI EnterpriseCommercial enterprise AI software suite deployed on customer cloud GPU instances or managed Kubernetes clusters.
- AWS
- Azure
- Google Cloud
- Oracle Cloud
- Alibaba Cloud
- Tencent Cloud
- Volcano Engine
- SGLangModel-serving framework for running supported language and multimodal models on customer-controlled compute.
- Google Cloud
- vLLM Self-HostedSelf-hosted model inference and serving engine with OpenAI-compatible APIs.
- AWS
- Azure
- Google Cloud
These products address related workflows; read their category-fit notes before comparing scope. See all of AI inference & gateways →
Installation outline
- Choose the documented deployment option and a dedicated cloud environment. Follow the launch methodology.
- Review the vendor's installation documentation, required permissions and outbound data flows.
- Deploy to a non-production environment; configure identity, networking and monitoring.
- Test backup recovery and upgrades before promoting to production.
Evidence trail
Every claim on this page traces to the sources below. They document the deployment option; they do not constitute a security certification.
- Deployment sourcesChecked
- Production deployment ↗docs.litellm.ai
- Gateway request flow and functionality ↗docs.litellm.ai
- MIT core and enterprise exception ↗github.com
- Licensing and edition terms ↗docs.litellm.ai
- Supply-chain incident and Cosign verification guidance ↗docs.litellm.ai
- Canonical LiteLLM incident investigation ↗docs.litellm.ai
- LiteLLM v1.110.0 release ↗github.com
- LiteLLM v1.108.11 LTS release ↗github.com
- LiteLLM v1.110.1 ↗github.com
- LiteLLM v1.108.12 LTS ↗github.com
- Cloud deployment evidenceChecked
The linked sources describe each deployment path, edition and support scope. Community templates and partner statements are identified in the notes.
- LiteLLM production deployment ↗ · AWS, Azure, Google Cloud
Vendor / upstream documentation · Reviewed 2026-11-12. Vendor support commitment: not assessed; confirm the selected release and contract.
Official production guidance covers EKS, AKS and GKE with Helm, plus Terraform on AWS and Google Cloud. The proxy, database and Redis deployment is separate from the model services it calls.
- LiteLLM Railway and Render deployment options ↗ · Railway, Render
Vendor / upstream documentation · Reviewed 2026-11-12. Vendor support commitment: not assessed; confirm the selected release and contract.
The current upstream deployment guide supplies Railway and Render deployment buttons in addition to its EKS/AKS/GKE instructions, followed by readiness and Admin UI verification. These are customer application deployments on managed platforms; configure persistent data, secrets and external model-provider endpoints separately.
- LiteLLM production deployment ↗ · AWS, Azure, Google Cloud
- Infrastructure evidenceChecked
- Licensing evidenceChecked