
# vLLM Self-Hosted

Self-hosted model inference and serving engine with OpenAI-compatible APIs.

Canonical page: https://launchinyourcloud.com/directory/vllm/
Catalog revision: 170de95bb7d832897d269051ad4500713acf96a62f3d01bc1d4e649679a56fde




Named clouds have provider-specific evidence. Missing clouds do not establish incompatibility. Preserve edition, account boundary, operating responsibilities and publisher disclosures. Source review dates record human checks, not rebuilds.


1 listing.


## vLLM Self-Hosted

Self-hosted model inference and serving engine with OpenAI-compatible APIs.

- Listing: https://launchinyourcloud.com/directory/vllm/
- Publisher listing: No
- Entry kind: product
- Documented cloud IDs: aws, azure, gcp
- Operating model: Customer-operated
- Infrastructure IDs: servers
- License: Apache 2.0 for vLLM; model weights and supporting dependencies have independent licenses and access conditions.
- Licensing model: open-source
- Installation: Official vLLM Docker image or supported Python installation; choose a compatible hardware/backend configuration.
- Architecture: Customer-operated model-serving process, model weights and inference hardware
- Operations: Customer manages model acquisition, hardware capacity, endpoint protection, availability, upgrades and model/runtime compatibility.
- Product website: https://vllm.ai/
- Source review: 2026-11-17; cloud review: 2026-11-12; licensing review: 2026-11-10; infrastructure review: 2026-11-17

### Scope and qualifications

This entry covers the inference server, not a hosted model service. It can run supported model weights on customer infrastructure, but those weights may need external downloads or access approval and are not licensed by vLLM. Review default usage telemetry and optional outbound integrations; hosting inference does not automatically create an air-gapped system. vLLM v0.34.0 removes Model Runner V1; configurations setting VLLM_USE_V2_MODEL_RUNNER=0 now fail at startup. It changes the default block size from 16 to 64, enables extensible KV cache by default with gpu_memory_utilization defaulting to 1.0, drops ROCm 7.2 wheels and images in favor of ROCm 10.0, requires huggingface_hub 2.0.0 or later, and removes QuTLASS/fp_quant and Marlin W4A8 support. Review the upstream breaking-change table and test model, quantization, memory, and accelerator compatibility before rollout.

The reviewed inference runtime is Apache-2.0 licensed. Its license does not grant rights to the model weights served by that runtime; check model and dependency terms separately.

vLLM is a running inference-server workload. Hardware support and capacity depend on the model and selected backend; the documented container paths include accelerator-specific images, and CPU support is separately documented. In v0.34.0, the KV-cache allocator sizes from free memory after warmup and defaults to gpu_memory_utilization=1.0; revalidate GPU headroom, and set 0.9 or disable extensible cache to retain prior sizing. ROCm 10.0 replaces 7.2 for published images and wheels.


### Category fit

- ai-inference-gateways: Model inference engine and OpenAI-compatible serving API for compatible model weights. Distinct from an API gateway that forwards calls to proprietary hosted models.

- ai-tools: Self-hosted model inference and serving engine with OpenAI-compatible APIs.

### Deployment boundaries


- aws: Customer cloud account; Customer-operated; control plane: not-assessed. Source: https://docs.vllm.ai/projects/production-stack/en/vllm-stack-0.1.4/deployment/cloud-deployment/aws.html (checked 2026-11-12). 

- azure: Customer cloud account; Customer-operated; control plane: not-assessed. Source: https://docs.vllm.ai/projects/production-stack/en/vllm-stack-0.1.4/deployment/cloud-deployment/azure.html (checked 2026-11-12). 

- gcp: Customer cloud account; Customer-operated; control plane: not-assessed. Source: https://docs.vllm.ai/projects/production-stack/en/vllm-stack-0.1.4/deployment/cloud-deployment/gcp.html (checked 2026-11-12). 


### Cloud deployment evidence

- vLLM Production Stack deployment on Amazon EKS (aws): https://docs.vllm.ai/projects/production-stack/en/vllm-stack-0.1.4/deployment/cloud-deployment/aws.html
  The project publishes Amazon EKS provisioning and vLLM deployment scripts in its versioned Production Stack guide. Verify current chart versions, model licenses, accelerator availability, GPU quotas and storage; this is customer-hosted inference, not a hosted model API.
  Source type: vendor-documentation; reviewed: 2026-11-12; vendor support: not-assessed.

- vLLM Production Stack deployment on Azure Kubernetes Service (azure): https://docs.vllm.ai/projects/production-stack/en/vllm-stack-0.1.4/deployment/cloud-deployment/azure.html
  The project publishes Azure Kubernetes Service provisioning and vLLM deployment scripts in its versioned Production Stack guide. Verify current chart versions, model licenses, accelerator availability, GPU quotas and storage; this is customer-hosted inference, not a hosted model API.
  Source type: vendor-documentation; reviewed: 2026-11-12; vendor support: not-assessed.

- vLLM Production Stack deployment on Google Kubernetes Engine (gcp): https://docs.vllm.ai/projects/production-stack/en/vllm-stack-0.1.4/deployment/cloud-deployment/gcp.html
  The project publishes Google Kubernetes Engine provisioning and vLLM deployment scripts in its versioned Production Stack guide. Verify current chart versions, model licenses, accelerator availability, GPU quotas and storage; this is customer-hosted inference, not a hosted model API.
  Source type: vendor-documentation; reviewed: 2026-11-12; vendor support: not-assessed.


### Sources

- Official Docker serving images: https://docs.vllm.ai/en/latest/deployment/docker/

- Online serving: https://docs.vllm.ai/en/latest/serving/online_serving/

- Usage statistics and opt-out: https://docs.vllm.ai/en/latest/usage/usage_stats/

- Apache 2.0 license: https://github.com/vllm-project/vllm/blob/main/LICENSE

- vLLM v0.34.0 breaking upgrade notes: https://github.com/vllm-project/vllm/releases/tag/v0.34.0

### Licensing sources

- https://github.com/vllm-project/vllm/blob/main/LICENSE

### Infrastructure sources

- https://docs.vllm.ai/en/latest/deployment/docker/

- https://github.com/vllm-project/vllm/releases/tag/v0.34.0


### Buyer diligence


Research coverage: 5/12 fields (42%); 3 documented, 2 unresolved. Coverage is not a feature score.

Unrecorded fields are not researched, not a statement that the feature is unavailable. Sources checked — unresolved means the cited material did not establish an answer. Hosted-service certifications do not automatically cover customer installations.


- Marketplace delivery: Not researched.

- Cloud commitment eligibility: Not researched.

- Single sign-on: Sources checked — unresolved. The security guide documents API-key protection for specified HTTP path prefixes and warns that other sensitive endpoints remain unprotected. It does not establish a native SAML/OIDC user sign-on workflow. Use a reviewed gateway/network design; an API bearer key alone is not corporate SSO or complete endpoint protection. Scope: Upstream inference-server auth surface; federated user integration unresolved. Source: https://docs.vllm.ai/en/latest/usage/security/ (reviewed 2026-11-20).

- SCIM provisioning: Not researched.

- Compliance attestations: Not researched.

- Security controls: Not researched.

- Air-gapped operation: Not researched.

- Deployment artifact / IaC: Documented. The stable deployment guide links the official vllm/vllm-openai image for a CUDA-backed OpenAI-compatible server. Select a release/architecture tag and compatible GPU drivers; model memory, credentials and optional dependencies need separate review. Scope: Stable upstream CUDA image path; not the latest developer-preview guide or a GPU sizing guarantee. Source: https://docs.vllm.ai/en/stable/deployment/docker/ (reviewed 2026-11-14). Deployment artifact: https://hub.docker.com/r/vllm/vllm-openai. Artifact reference: select a stable version and architecture tag; latest is moving.

- Minimum footprint / sizing: Sources checked — unresolved. The stable memory guide sizes serving around model weights, precision/quantization, tensor parallelism, context length, batch size and cache. A supported accelerator alone does not establish sufficient memory. Select the exact model and target throughput/latency, then benchmark VRAM/RAM and HA replica requirements. Scope: Model-specific inference sizing; no universal numeric footprint established. Source: https://docs.vllm.ai/en/stable/configuration/conserving_memory/ (reviewed 2026-11-20).

- License pricing: Documented. The reviewed upstream core is Apache-2.0 licensed. Optional services, bundled dependencies, commercial distributions and support can have separate terms. This records the source licence, not an infrastructure or operating-cost estimate. Model weights and external model APIs require their own review. Scope: Upstream core licence; commercial features, support and hosting priced separately. Source: https://raw.githubusercontent.com/vllm-project/vllm/main/LICENSE (reviewed 2026-11-20).

- Operating cost estimate: Not researched.

- Release snapshot / cadence: Documented. The reviewed upstream release is v0.34.0, published 2026-11-16. This is a dated release snapshot, not a support designation, a future cadence promise or approval to replace a provider package. Scope: Upstream vllm-project/vllm release record; commercial editions, LTS branches and provider packages can differ. Source: https://github.com/vllm-project/vllm/releases/tag/v0.34.0 (reviewed 2026-11-17).






