{"catalog":"https://launchinyourcloud.com/api/v1/catalog.json","catalogRevision":"170de95bb7d832897d269051ad4500713acf96a62f3d01bc1d4e649679a56fde","kind":"listing","listing":{"architecture":"Customer-operated model-serving process, model weights and inference hardware","categories":["ai-inference-gateways","ai-tools"],"category":"AI inference \u0026 gateways","categoryFit":{"ai-inference-gateways":"Model inference engine and OpenAI-compatible serving API for compatible model weights. Distinct from an API gateway that forwards calls to proprietary hosted models.","ai-tools":"Self-hosted model inference and serving engine with OpenAI-compatible APIs."},"cloudEvidence":[{"clouds":["aws"],"notes":"The project publishes Amazon EKS provisioning and vLLM deployment scripts in its versioned Production Stack guide. Verify current chart versions, model licenses, accelerator availability, GPU quotas and storage; this is customer-hosted inference, not a hosted model API.","reviewed":"2026-11-12","sourceType":"vendor-documentation","title":"vLLM Production Stack deployment on Amazon EKS","url":"https://docs.vllm.ai/projects/production-stack/en/vllm-stack-0.1.4/deployment/cloud-deployment/aws.html","vendorSupport":"not-assessed"},{"clouds":["azure"],"notes":"The project publishes Azure Kubernetes Service provisioning and vLLM deployment scripts in its versioned Production Stack guide. Verify current chart versions, model licenses, accelerator availability, GPU quotas and storage; this is customer-hosted inference, not a hosted model API.","reviewed":"2026-11-12","sourceType":"vendor-documentation","title":"vLLM Production Stack deployment on Azure Kubernetes Service","url":"https://docs.vllm.ai/projects/production-stack/en/vllm-stack-0.1.4/deployment/cloud-deployment/azure.html","vendorSupport":"not-assessed"},{"clouds":["gcp"],"notes":"The project publishes Google Kubernetes Engine provisioning and vLLM deployment scripts in its versioned Production Stack guide. Verify current chart versions, model licenses, accelerator availability, GPU quotas and storage; this is customer-hosted inference, not a hosted model API.","reviewed":"2026-11-12","sourceType":"vendor-documentation","title":"vLLM Production Stack deployment on Google Kubernetes Engine","url":"https://docs.vllm.ai/projects/production-stack/en/vllm-stack-0.1.4/deployment/cloud-deployment/gcp.html","vendorSupport":"not-assessed"}],"cloudReviewed":"2026-11-12","clouds":["aws","azure","gcp"],"deploymentPaths":[{"boundary":"cloud-account","cloud":"aws","controlPlane":"not-assessed","evidence":[0],"operator":"customer"},{"boundary":"cloud-account","cloud":"azure","controlPlane":"not-assessed","evidence":[1],"operator":"customer"},{"boundary":"cloud-account","cloud":"gcp","controlPlane":"not-assessed","evidence":[2],"operator":"customer"}],"description":"Self-hosted model inference and serving engine with OpenAI-compatible APIs.","diligence":{"footprint":{"reviewed":"2026-11-20","scope":"Model-specific inference sizing; no universal numeric footprint established","source":"https://docs.vllm.ai/en/stable/configuration/conserving_memory/","status":"inconclusive","summary":"The stable memory guide sizes serving around model weights, precision/quantization, tensor parallelism, context length, batch size and cache. A supported accelerator alone does not establish sufficient memory. Select the exact model and target throughput/latency, then benchmark VRAM/RAM and HA replica requirements."},"iac":{"artifactRef":"select a stable version and architecture tag; latest is moving","artifactUrl":"https://hub.docker.com/r/vllm/vllm-openai","reviewed":"2026-11-14","scope":"Stable upstream CUDA image path; not the latest developer-preview guide or a GPU sizing guarantee","source":"https://docs.vllm.ai/en/stable/deployment/docker/","status":"documented","summary":"The stable deployment guide links the official vllm/vllm-openai image for a CUDA-backed OpenAI-compatible server. Select a release/architecture tag and compatible GPU drivers; model memory, credentials and optional dependencies need separate review."},"licensePricing":{"reviewed":"2026-11-20","scope":"Upstream core licence; commercial features, support and hosting priced separately","source":"https://raw.githubusercontent.com/vllm-project/vllm/main/LICENSE","status":"documented","summary":"The reviewed upstream core is Apache-2.0 licensed. Optional services, bundled dependencies, commercial distributions and support can have separate terms. This records the source licence, not an infrastructure or operating-cost estimate. Model weights and external model APIs require their own review."},"release":{"releasePublishedAt":"2026-11-16T23:22:00Z","releaseTag":"v0.34.0","reviewed":"2026-11-17","scope":"Upstream vllm-project/vllm release record; commercial editions, LTS branches and provider packages can differ","source":"https://github.com/vllm-project/vllm/releases/tag/v0.34.0","status":"documented","summary":"The reviewed upstream release is v0.34.0, published 2026-11-16. This is a dated release snapshot, not a support designation, a future cadence promise or approval to replace a provider package."},"sso":{"reviewed":"2026-11-20","scope":"Upstream inference-server auth surface; federated user integration unresolved","source":"https://docs.vllm.ai/en/latest/usage/security/","status":"inconclusive","summary":"The security guide documents API-key protection for specified HTTP path prefixes and warns that other sensitive endpoints remain unprotected. It does not establish a native SAML/OIDC user sign-on workflow. Use a reviewed gateway/network design; an API bearer key alone is not corporate SSO or complete endpoint protection."}},"id":"vllm","infrastructure":["servers"],"infrastructureNotes":"vLLM is a running inference-server workload. Hardware support and capacity depend on the model and selected backend; the documented container paths include accelerator-specific images, and CPU support is separately documented. In v0.34.0, the KV-cache allocator sizes from free memory after warmup and defaults to gpu_memory_utilization=1.0; revalidate GPU headroom, and set 0.9 or disable extensible cache to retain prior sizing. ROCm 10.0 replaces 7.2 for published images and wheels.","infrastructureReviewed":"2026-11-17","infrastructureSources":["https://docs.vllm.ai/en/latest/deployment/docker/","https://github.com/vllm-project/vllm/releases/tag/v0.34.0"],"install":"Official vLLM Docker image or supported Python installation; choose a compatible hardware/backend configuration.","installMethods":["docker","source"],"kind":"product","license":"Apache 2.0 for vLLM; model weights and supporting dependencies have independent licenses and access conditions.","licenseModel":"open-source","licenseModelNotes":"The reviewed inference runtime is Apache-2.0 licensed. Its license does not grant rights to the model weights served by that runtime; check model and dependency terms separately.","licenseModelSources":["https://github.com/vllm-project/vllm/blob/main/LICENSE"],"licenseReviewed":"2026-11-10","model":"Fully in your account","modelLabel":"Customer-operated","name":"vLLM Self-Hosted","notes":"This entry covers the inference server, not a hosted model service. It can run supported model weights on customer infrastructure, but those weights may need external downloads or access approval and are not licensed by vLLM. Review default usage telemetry and optional outbound integrations; hosting inference does not automatically create an air-gapped system. vLLM v0.34.0 removes Model Runner V1; configurations setting VLLM_USE_V2_MODEL_RUNNER=0 now fail at startup. It changes the default block size from 16 to 64, enables extensible KV cache by default with gpu_memory_utilization defaulting to 1.0, drops ROCm 7.2 wheels and images in favor of ROCm 10.0, requires huggingface_hub 2.0.0 or later, and removes QuTLASS/fp_quant and Marlin W4A8 support. Review the upstream breaking-change table and test model, quantization, memory, and accelerator compatibility before rollout.","operations":"Customer manages model acquisition, hardware capacity, endpoint protection, availability, upgrades and model/runtime compatibility.","publisher":false,"reviewed":"2026-11-17","sources":[{"title":"Official Docker serving images","url":"https://docs.vllm.ai/en/latest/deployment/docker/"},{"title":"Online serving","url":"https://docs.vllm.ai/en/latest/serving/online_serving/"},{"title":"Usage statistics and opt-out","url":"https://docs.vllm.ai/en/latest/usage/usage_stats/"},{"title":"Apache 2.0 license","url":"https://github.com/vllm-project/vllm/blob/main/LICENSE"},{"title":"vLLM v0.34.0 breaking upgrade notes","url":"https://github.com/vllm-project/vllm/releases/tag/v0.34.0"}],"url":"https://launchinyourcloud.com/directory/vllm/","website":"https://vllm.ai/"},"schemaVersion":1,"url":"https://launchinyourcloud.com/directory/vllm/"}