vLLM Self-Hosted

Self-hosted model inference and serving engine with OpenAI-compatible APIs.

5 sources checked by an editor on · Product website ↗

Deployment terms: BYOC · control plane · data plane · private offer

· Open requirements shortlist · Estimate operating costs

Compare this listing with another →

What lands where

  1. Supplied by

    Vendor or project

    Official vLLM Docker image or supported Python installation; choose a compatible hardware/backend configuration.

  2. Operated by

    Your team

    Customer manages model acquisition, hardware capacity, endpoint protection, availability, upgrades and model/runtime compatibility.

Deployment boundaries by provider

Application operations and account ownership are separate from infrastructure operations. Confirm external services, telemetry, support access and backup destinations for the chosen path.

Install methodOfficial vLLM Docker image or supported Python installation; choose a compatible hardware/backend configuration.
InfrastructureServers / containers
vLLM is a running inference-server workload. Hardware support and capacity depend on the model and selected backend; the documented container paths include accelerator-specific images, and CPU support is separately documented. In v0.34.0, the KV-cache allocator sizes from free memory after warmup and defaults to gpu_memory_utilization=1.0; revalidate GPU headroom, and set 0.9 or disable extensible cache to retain prior sizing. ROCm 10.0 replaces 7.2 for published images and wheels.
Licensing modelOpen Source / Open Core
The reviewed inference runtime is Apache-2.0 licensed. Its license does not grant rights to the model weights served by that runtime; check model and dependency terms separately.
LicenseApache 2.0 for vLLM; model weights and supporting dependencies have independent licenses and access conditions.

Buyer diligence

Answers apply to the reviewed edition and deployment. Expand a row for qualifications and sources.

How to read these facts

Not researched means no field-specific review is recorded. Not confirmed means sources were checked but did not establish an answer. Availability, eligibility and edition-dependent findings retain the scope given in their evidence. A vendor’s hosted-service certification does not automatically cover your deployment. Evidence coverage is not a feature score.

5 of 12 fields researched (42%) · 3 documented · 2 unresolved. Evidence coverage is not a feature score.

Single sign-on
Not confirmedEvidence and scope for Single sign-on

The security guide documents API-key protection for specified HTTP path prefixes and warns that other sensitive endpoints remain unprotected. It does not establish a native SAML/OIDC user sign-on workflow. Use a reviewed gateway/network design; an API bearer key alone is not corporate SSO or complete endpoint protection.

Upstream inference-server auth surface; federated user integration unresolved · Source ↗ · Reviewed 2026-11-20

Sources checked — unresolved · Editor reviewed.

Deployment artifact / IaC
DocumentedEvidence and scope for Deployment artifact / IaC

The stable deployment guide links the official vllm/vllm-openai image for a CUDA-backed OpenAI-compatible server. Select a release/architecture tag and compatible GPU drivers; model memory, credentials and optional dependencies need separate review.

Stable upstream CUDA image path; not the latest developer-preview guide or a GPU sizing guarantee · Source ↗ · Deployment artifact ↗ (select a stable version and architecture tag; latest is moving) · Reviewed 2026-11-14

Documented · Editor reviewed.

Minimum footprint / sizing
Not confirmedEvidence and scope for Minimum footprint / sizing

The stable memory guide sizes serving around model weights, precision/quantization, tensor parallelism, context length, batch size and cache. A supported accelerator alone does not establish sufficient memory. Select the exact model and target throughput/latency, then benchmark VRAM/RAM and HA replica requirements.

Model-specific inference sizing; no universal numeric footprint established · Source ↗ · Reviewed 2026-11-20

Sources checked — unresolved · Editor reviewed.

License pricing
DocumentedEvidence and scope for License pricing

The reviewed upstream core is Apache-2.0 licensed. Optional services, bundled dependencies, commercial distributions and support can have separate terms. This records the source licence, not an infrastructure or operating-cost estimate. Model weights and external model APIs require their own review.

Upstream core licence; commercial features, support and hosting priced separately · Source ↗ · Reviewed 2026-11-20

Documented · Editor reviewed.

Release snapshot / cadence
DocumentedEvidence and scope for Release snapshot / cadence

The reviewed upstream release is v0.34.0, published 2026-11-16. This is a dated release snapshot, not a support designation, a future cadence promise or approval to replace a provider package.

Upstream vllm-project/vllm release record; commercial editions, LTS branches and provider packages can differ · Source ↗ · Reviewed 2026-11-17

Documented · Editor reviewed.

Show all fields (7 not researched)
Marketplace delivery
Not researched Supply a public source.
Cloud commitment eligibility
Not researched Supply a public source.
SCIM provisioning
Not researched Supply a public source.
Compliance attestations
Not researched Supply a public source.
Security controls
Not researched Supply a public source.
Air-gapped operation
Not researched Supply a public source.
Operating cost estimate
Not researched Supply a public source.

Scope & considerations

This entry covers the inference server, not a hosted model service. It can run supported model weights on customer infrastructure, but those weights may need external downloads or access approval and are not licensed by vLLM. Review default usage telemetry and optional outbound integrations; hosting inference does not automatically create an air-gapped system. vLLM v0.34.0 removes Model Runner V1; configurations setting VLLM_USE_V2_MODEL_RUNNER=0 now fail at startup. It changes the default block size from 16 to 64, enables extensible KV cache by default with gpu_memory_utilization defaulting to 1.0, drops ROCm 7.2 wheels and images in favor of ROCm 10.0, requires huggingface_hub 2.0.0 or later, and removes QuTLASS/fp_quant and Marlin W4A8 support. Review the upstream breaking-change table and test model, quantization, memory, and accelerator compatibility before rollout.

Category fit

AI inference & gateways
Model inference engine and OpenAI-compatible serving API for compatible model weights. Distinct from an API gateway that forwards calls to proprietary hosted models.
AI tools & infrastructure
Self-hosted model inference and serving engine with OpenAI-compatible APIs.

Compare within AI inference & gateways

  • LiteLLM Proxy Self-HostedSelf-hosted AI API gateway providing model routing, access controls and usage tracking.
    • AWS
    • Azure
    • Google Cloud
    • Railway
    • Render
    Customer-operated · Open Source / Open Core
  • LocalAILocal model inference service exposing configurable AI capabilities through compatible API endpoints.
    • AWS
    Customer-operated · Open Source / Open Core
  • NVIDIA AI EnterpriseCommercial enterprise AI software suite deployed on customer cloud GPU instances or managed Kubernetes clusters.
    • AWS
    • Azure
    • Google Cloud
    • Oracle Cloud
    • Alibaba Cloud
    • Tencent Cloud
    • Volcano Engine
    Customer-operated · Commercial Only
  • SGLangModel-serving framework for running supported language and multimodal models on customer-controlled compute.
    • Google Cloud
    Customer-operated · Open Source / Open Core

These products address related workflows; read their category-fit notes before comparing scope. See all of AI inference & gateways →

Installation outline

  1. Choose the documented deployment option and a dedicated cloud environment. Follow the launch methodology.
  2. Review the vendor's installation documentation, required permissions and outbound data flows.
  3. Deploy to a non-production environment; configure identity, networking and monitoring.
  4. Test backup recovery and upgrades before promoting to production.

Watch this listing with RSS

Evidence trail

Every claim on this page traces to the sources below. They document the deployment option; they do not constitute a security certification.

  1. Deployment sourcesChecked
  2. Cloud deployment evidenceChecked

    The linked sources describe each deployment path, edition and support scope. Community templates and partner statements are identified in the notes.

    • vLLM Production Stack deployment on Amazon EKS ↗ · AWS

      Vendor / upstream documentation · Reviewed 2026-11-12. Vendor support commitment: not assessed; confirm the selected release and contract.

      The project publishes Amazon EKS provisioning and vLLM deployment scripts in its versioned Production Stack guide. Verify current chart versions, model licenses, accelerator availability, GPU quotas and storage; this is customer-hosted inference, not a hosted model API.

    • vLLM Production Stack deployment on Azure Kubernetes Service ↗ · Azure

      Vendor / upstream documentation · Reviewed 2026-11-12. Vendor support commitment: not assessed; confirm the selected release and contract.

      The project publishes Azure Kubernetes Service provisioning and vLLM deployment scripts in its versioned Production Stack guide. Verify current chart versions, model licenses, accelerator availability, GPU quotas and storage; this is customer-hosted inference, not a hosted model API.

    • vLLM Production Stack deployment on Google Kubernetes Engine ↗ · Google Cloud

      Vendor / upstream documentation · Reviewed 2026-11-12. Vendor support commitment: not assessed; confirm the selected release and contract.

      The project publishes Google Kubernetes Engine provisioning and vLLM deployment scripts in its versioned Production Stack guide. Verify current chart versions, model licenses, accelerator availability, GPU quotas and storage; this is customer-hosted inference, not a hosted model API.

  3. Infrastructure evidenceChecked
  4. Licensing evidenceChecked

Suggest a correction · Back to directory