
# AI inference & gateways

Model-serving runtimes, inference gateways and provider routing.

Canonical page: https://launchinyourcloud.com/categories/ai-inference-gateways/
Catalog revision: 170de95bb7d832897d269051ad4500713acf96a62f3d01bc1d4e649679a56fde




Named clouds have provider-specific evidence. Missing clouds do not establish incompatibility. Preserve edition, account boundary, operating responsibilities and publisher disclosures. Source review dates record human checks, not rebuilds.


Model-serving runtimes, inference gateways and provider routing.

5 listings.


## LiteLLM Proxy Self-Hosted

Self-hosted AI API gateway providing model routing, access controls and usage tracking.

- Listing: https://launchinyourcloud.com/directory/litellm/
- Publisher listing: No
- Entry kind: product
- Documented cloud IDs: aws, azure, gcp, railway, render
- Operating model: Customer-operated
- Infrastructure IDs: servers
- License: MIT outside the enterprise directory; enterprise code and features have separate license terms.
- Licensing model: open-source
- Installation: Official container image, Docker Compose or Kubernetes deployment; production infrastructure examples are documented.
- Architecture: Customer-operated gateway with optional PostgreSQL persistence and configured model endpoints
- Operations: Customer manages gateway availability, credentials, database, access policies, logs and upgrades, including pinning versions and verifying release artifacts with the documented version-appropriate Cosign method; model providers remain separate services.
- Product website: https://www.litellm.ai/
- Source review: 2026-11-19; cloud review: 2026-11-12; licensing review: 2026-11-10; infrastructure review: 2026-11-10

### Scope and qualifications

The gateway forwards each request to its configured model provider. Self-hosting the proxy does not ensure prompts stay inside the account: external model APIs, guardrail services and logging integrations can receive data. Use appropriate self-hosted inference endpoints and explicitly review outbound integrations when required. For release integrity, release images and the Helm chart starting with v1.111.0-dev.3 use keyless Cosign signatures bound to LiteLLM's release workflow; earlier releases require the documented historical public keys. The October 21 incident affected PyPI v1.107.0 and Docker Hub/GHCR v1.107.0 plus mutable latest, main-latest and main-stable tags during the documented exposure windows. Check the incident's artifact digests and remediation instructions; the compromised images carried valid historical signatures, so signature verification alone does not establish safety. LiteLLM v1.110.0 changes authentication, guardrail, accounting, and S3 behavior. The 1.108.11 LTS patch also fixes Bedrock guarded_text handling and restricts os.environ team settings to proxy administrators; review both the selected release line and existing incident-response or image-signing guidance. LiteLLM v1.108.12 LTS fixes tool_policy allowing blocked /v1/messages tool names longer than 64 characters. The standard v1.110.1 UI image changes to slim nginx and removes curl while retaining wget and nc, so retest health probes and scripts that assume curl.

The gateway has an MIT-licensed base and separately licensed enterprise features. The enterprise documentation describes a license key enabling restricted capabilities in the gateway, beyond paid hosting or support alone.

The gateway runs as provisioned application containers or Kubernetes workloads; persistent key/spend storage can use PostgreSQL. Inference compute is a separate dependency rather than included in the gateway.


### Category fit

- ai-inference-gateways: Self-hosted model API gateway for routing, virtual keys, spend tracking and policy enforcement. It routes inference requests; it is not an inference engine for all supported models.

- ai-tools: Self-hosted AI API gateway providing model routing, access controls and usage tracking.

### Deployment boundaries


- aws: Customer cloud account; Customer-operated; control plane: not-assessed. Source: https://docs.litellm.ai/docs/proxy/deploy (checked 2026-11-12). 

- azure: Customer cloud account; Customer-operated; control plane: not-assessed. Source: https://docs.litellm.ai/docs/proxy/deploy (checked 2026-11-12). 

- gcp: Customer cloud account; Customer-operated; control plane: not-assessed. Source: https://docs.litellm.ai/docs/proxy/deploy (checked 2026-11-12). 

- railway: Customer platform workspace; Customer-operated; control plane: not-assessed. Source: https://docs.litellm.ai/docs/proxy/deploy (checked 2026-11-12). 

- render: Customer platform workspace; Customer-operated; control plane: not-assessed. Source: https://docs.litellm.ai/docs/proxy/deploy (checked 2026-11-12). 


### Cloud deployment evidence

- LiteLLM production deployment (aws, azure, gcp): https://docs.litellm.ai/docs/proxy/deploy
  Official production guidance covers EKS, AKS and GKE with Helm, plus Terraform on AWS and Google Cloud. The proxy, database and Redis deployment is separate from the model services it calls.
  Source type: vendor-documentation; reviewed: 2026-11-12; vendor support: not-assessed.

- LiteLLM Railway and Render deployment options (railway, render): https://docs.litellm.ai/docs/proxy/deploy
  The current upstream deployment guide supplies Railway and Render deployment buttons in addition to its EKS/AKS/GKE instructions, followed by readiness and Admin UI verification. These are customer application deployments on managed platforms; configure persistent data, secrets and external model-provider endpoints separately.
  Source type: vendor-documentation; reviewed: 2026-11-12; vendor support: not-assessed.


### Sources

- Production deployment: https://docs.litellm.ai/docs/proxy/deploy

- Gateway request flow and functionality: https://docs.litellm.ai/docs/simple_proxy

- MIT core and enterprise exception: https://github.com/BerriAI/litellm/blob/main/LICENSE

- Licensing and edition terms: https://docs.litellm.ai/docs/enterprise

- Supply-chain incident and Cosign verification guidance: https://docs.litellm.ai/blog/security-incident-oct-21-2026

- Canonical LiteLLM incident investigation: https://docs.litellm.ai/blog/security-incident-oct-21-2026-attack-chain

- LiteLLM v1.110.0 release: https://github.com/BerriAI/litellm/releases/tag/v1.110.0

- LiteLLM v1.108.11 LTS release: https://github.com/BerriAI/litellm/releases/tag/v1.108.11

- LiteLLM v1.110.1: https://github.com/BerriAI/litellm/releases/tag/v1.110.1

- LiteLLM v1.108.12 LTS: https://github.com/BerriAI/litellm/releases/tag/v1.108.12

### Licensing sources

- https://github.com/BerriAI/litellm/blob/main/LICENSE

- https://docs.litellm.ai/docs/enterprise

### Infrastructure sources

- https://docs.litellm.ai/docs/proxy/deploy


### Buyer diligence


Research coverage: 6/12 fields (50%); 6 documented, 0 unresolved. Coverage is not a feature score.

Unrecorded fields are not researched, not a statement that the feature is unavailable. Sources checked — unresolved means the cited material did not establish an answer. Hosted-service certifications do not automatically cover customer installations.


- Marketplace delivery: Not researched.

- Cloud commitment eligibility: Not researched.

- Single sign-on: Edition-dependent. The Admin UI documents OIDC SSO. From 1.76.0, it is free for up to five users; larger deployments require an Enterprise licence. Configure issuer, callbacks, roles and database/master-key prerequisites. This is Admin UI identity, distinct from model API virtual keys. Scope: LiteLLM Admin UI OIDC; user-count/Enterprise entitlement applies. Source: https://docs.litellm.ai/docs/proxy/admin_ui_sso (reviewed 2026-11-20).

- SCIM provisioning: Not researched.

- Compliance attestations: Not researched.

- Security controls: Documented. Release images and Helm charts use keyless Cosign from v1.111.0-dev.3; verify the documented workflow identity and issuer. Earlier releases use historical keys. The October 21 v1.107.0 compromise also affected mutable image tags; compare digests and exposure windows because affected images had valid historical signatures. Follow the incident remediation guidance for executed artifacts. Current advisory review: LiteLLM 1.108.11 LTS fixes Bedrock guarded_text handling and restricts os.environ team settings to proxy administrators. Upgrade the selected release line and recheck guardrail output and proxy-administrator permissions. Current review: v1.108.12 LTS fixes tool_policy allowing a /v1/messages request through when a blocked tool name exceeded 64 characters. Upgrade the maintained LTS line and retest tool-policy enforcement. Scope: LiteLLM 1.108 LTS /v1/messages tool_policy; standard and other maintained lines require their own release review. Source: https://github.com/BerriAI/litellm/releases/tag/v1.108.12 (reviewed 2026-11-19). Supporting source: Prior security-controls source — https://github.com/BerriAI/litellm/releases/tag/v1.108.11. Supporting source: Canonical incident investigation — https://docs.litellm.ai/blog/security-incident-oct-21-2026-attack-chain. Supporting source: Prior security-controls source — https://docs.litellm.ai/blog/security-incident-oct-21-2026.

- Air-gapped operation: Not researched.

- Deployment artifact / IaC: Documented. The official Helm guide documents Kubernetes installation and configurable database/secret settings. Select matching chart/image versions and review the separate scaling and production-readiness guidance. Scope: Upstream self-hosted deployment; not a provider support or production-readiness certification. Source: https://docs.litellm.ai/docs/proxy/deploy/helm (reviewed 2026-11-14).

- Minimum footprint / sizing: Documented. The current production guide starts load testing at 1 vCPU and 4 GiB per worker, preferably one worker per Kubernetes pod. Memory depends on queued requests, payloads, stream duration and logging. PostgreSQL, Redis for multiple workers/instances, HA and model inference are additional. Scope: Gateway per-worker load-test starting point; backing services and inference excluded. Source: https://docs.litellm.ai/docs/proxy/prod (reviewed 2026-11-20).

- License pricing: Documented. Core code outside litellm/enterprise is MIT licensed; the enterprise directory has separate terms. Enterprise features and support need separate entitlement and pricing. Upstream model APIs, infrastructure and operating labour are additional. Scope: Upstream core licence; commercial features, support and hosting priced separately. Source: https://raw.githubusercontent.com/BerriAI/litellm/main/LICENSE (reviewed 2026-11-20).

- Operating cost estimate: Not researched.

- Release snapshot / cadence: Documented. LiteLLM v1.110.1 was published November 19, 2026. It refreshes the Wolfi base and changes the UI image to slim nginx, removing curl. The maintained v1.108.12 LTS line fixes a tool_policy bypass for blocked tool names longer than 64 characters. Scope: Upstream LiteLLM standard release; select and maintain either the standard or LTS line. Source: https://github.com/BerriAI/litellm/releases/tag/v1.110.1 (reviewed 2026-11-19). Supporting source: Prior reviewed release context — https://github.com/BerriAI/litellm/releases/tag/v1.110.0. Supporting source: LiteLLM v1.108.11 LTS — https://github.com/BerriAI/litellm/releases/tag/v1.108.11. Supporting source: v1.108.12 LTS security fix — https://github.com/BerriAI/litellm/releases/tag/v1.108.12.




### Security and lifecycle reviews

Selected editorial reviews only; no flag does not mean safe or supported.

- LiteLLM LTS proxy security fixes (security-advisory): LiteLLM 1.108.11 LTS fixes Bedrock guarded_text handling and restricts os.environ team settings to proxy administrators. Scope: LiteLLM 1.108 LTS line; standard-line deployments should use their corresponding maintained release. Conditions: Applicability depends on Bedrock guardrails and team-setting permissions. Action: Upgrade the selected release line and recheck guardrail output and proxy-administrator permissions. Source: https://github.com/BerriAI/litellm/releases/tag/v1.108.11; reviewed 2026-11-18.

- LiteLLM 1.108.12 LTS tool-policy fix (security-advisory): LiteLLM 1.108.12 LTS prevents blocked tool names longer than 64 characters from bypassing tool_policy on /v1/messages. Scope: LiteLLM 1.108 LTS /v1/messages tool-policy enforcement. Conditions: The request must use a blocked tool name longer than 64 characters. Action: Upgrade the maintained LTS line to 1.108.12 or later and retest tool-policy enforcement. Source: https://github.com/BerriAI/litellm/releases/tag/v1.108.12; reviewed 2026-11-19.




## LocalAI

Local model inference service exposing configurable AI capabilities through compatible API endpoints.

- Listing: https://launchinyourcloud.com/directory/localai/
- Publisher listing: No
- Entry kind: product
- Documented cloud IDs: aws
- Operating model: Customer-operated
- Infrastructure IDs: servers
- License: MIT License for upstream LocalAI. Model artifacts and inference backends can have different license and hardware requirements.
- Licensing model: open-source
- Installation: Run official CPU/GPU Docker images or the documented binaries and configure models, storage and endpoint access.
- Architecture: Customer-operated inference API with selected model backends and local model storage
- Operations: Customer manages models, backends, hardware capacity, API authentication, persistent storage and upgrades.
- Product website: https://localai.io/
- Source review: 2026-11-12; cloud review: 2026-11-12; licensing review: 2026-11-10; infrastructure review: 2026-11-10

### Scope and qualifications

CPU availability does not guarantee acceptable performance for every model. Model galleries and artifact downloads contact external registries; optional integrations and remote backends can change data boundaries.

MIT License for upstream LocalAI. Model artifacts and inference backends can have different license and hardware requirements. This classification covers the reviewed upstream edition; external services and dependencies retain their own terms.

Customer-operated inference API with selected model backends and local model storage. This is a customer-operated server/container deployment; portability does not establish named cloud-provider support.


### Category fit

- ai-inference-gateways: Local inference service with OpenAI-compatible API endpoints and configurable model backends; compatible API support does not require a hosted OpenAI service.

- ai-tools: Customer-operated model-serving infrastructure with CPU and optional accelerator backends.

### Deployment boundaries


- aws: Customer cloud account; Customer-operated; control plane: not-assessed. Source: https://www.pulumi.com/blog/low-code-llm-apps-with-local-ai-flowise-and-pulumi/ (checked 2026-11-12). 


### Cloud deployment evidence

- LocalAI and Flowise on AWS EKS with Pulumi (aws): https://www.pulumi.com/blog/low-code-llm-apps-with-local-ai-flowise-and-pulumi/
  This Pulumi-authored deployment recipe, linked by the LocalAI project, runs LocalAI inference and Flowise on customer EKS infrastructure. The demonstrated configuration uses CPU inference; review model licenses, current images, resource sizing and network exposure rather than assuming GPU support or production hardening.
  Source type: third-party-deployment-guide; reviewed: 2026-11-12; vendor support: not-assessed.


### Community and maintenance

The project documents a human maintainer team and wider contributor community. Release notes recognize multiple named contributors across successive releases and continued runtime/security maintenance.

Reviewed 2026-11-10.

- Maintenance and contributor release notes: https://github.com/mudler/LocalAI/releases

- Maintainer and community description: https://github.com/mudler/LocalAI

### Sources

- Official installation and runtime: https://raw.githubusercontent.com/mudler/LocalAI/master/README.md

- Upstream project and capabilities: https://github.com/mudler/LocalAI

- Current upstream license: https://raw.githubusercontent.com/mudler/LocalAI/master/LICENSE

### Licensing sources

- https://raw.githubusercontent.com/mudler/LocalAI/master/LICENSE

### Infrastructure sources

- https://raw.githubusercontent.com/mudler/LocalAI/master/README.md


### Buyer diligence


Research coverage: 5/12 fields (42%); 4 documented, 1 unresolved. Coverage is not a feature score.

Unrecorded fields are not researched, not a statement that the feature is unavailable. Sources checked — unresolved means the cited material did not establish an answer. Hosted-service certifications do not automatically cover customer installations.


- Marketplace delivery: Not researched.

- Cloud commitment eligibility: Not researched.

- Single sign-on: Documented. The full user-authentication mode supports OIDC single sign-on and GitHub OAuth, with roles and per-user keys. The legacy shared API-key mode is separate. Enable and configure the appropriate auth mode, issuer/client settings and registration policy rather than relying on a bare inference endpoint. Scope: LocalAI full user-authentication mode; legacy shared-key mode differs. Source: https://localai.io/docs/administration/authentication/index.html (reviewed 2026-11-20).

- SCIM provisioning: Not researched.

- Compliance attestations: Not researched.

- Security controls: Not researched.

- Air-gapped operation: Not researched.

- Deployment artifact / IaC: Documented. The upstream repository provides CPU/GPU deployment options and application artifacts. Match the selected image/backend to host accelerators and model requirements; a generic container does not establish model-serving capacity. Scope: Upstream self-hosted deployment; not a provider support or production-readiness certification. Source: https://github.com/mudler/LocalAI (reviewed 2026-11-14). Deployment artifact: https://github.com/mudler/LocalAI. Artifact reference: repository default branch (moving; select matching application release).

- Minimum footprint / sizing: Sources checked — unresolved. LocalAI offers CPU and accelerator-specific installation paths, but the runtime guide does not establish one numeric minimum covering its model gallery. Model/backend, quantization, context, concurrent sessions and enabled modalities determine memory and compute. Size the chosen model and redundant instances separately from the UI/API process. Scope: Model/backend-specific LocalAI sizing; universal production minimum unresolved. Source: https://localai.io/docs/start/install/index.html (reviewed 2026-11-20).

- License pricing: Documented. The reviewed upstream core is MIT licensed. Optional services, bundled dependencies, commercial distributions and support can have separate terms. This records the source licence, not an infrastructure or operating-cost estimate. Model weights and external model APIs require their own review. Scope: Upstream core licence; commercial features, support and hosting priced separately. Source: https://raw.githubusercontent.com/mudler/LocalAI/master/LICENSE (reviewed 2026-11-20).

- Operating cost estimate: Not researched.

- Release snapshot / cadence: Documented. The publisher-designated GitHub release checked was v4.12.1, published 2026-10-23. This is a release snapshot, not a promised cadence, support designation or approval to substitute it into a provider package. Scope: Upstream mudler/LocalAI release record; commercial editions and deployment packages may differ. Source: https://github.com/mudler/LocalAI/releases/tag/v4.12.1 (reviewed 2026-11-14).







## NVIDIA AI Enterprise

Commercial enterprise AI software suite deployed on customer cloud GPU instances or managed Kubernetes clusters.

- Listing: https://launchinyourcloud.com/directory/nvidia-ai-enterprise/
- Publisher listing: No
- Entry kind: product
- Documented cloud IDs: aws, azure, gcp, oci, alibaba, tencent, volcano
- Operating model: Customer-operated
- Infrastructure IDs: servers
- License: Commercial NVIDIA AI Enterprise subscription, consumption or perpetual license; most customer-cloud paths require BYOL.
- Licensing model: commercial-only
- Installation: NVIDIA AI Enterprise or GPU-optimized cloud VM images, standard instances with BYOL, or managed Kubernetes on supported cloud providers
- Architecture: Licensed NVIDIA AI containers and infrastructure software on customer GPU virtual machines, managed Kubernetes clusters or marketplace OpenShift
- Operations: Customer manages cloud GPU instances or Kubernetes, drivers, GPU Operator and container runtime, networking, storage, identities, NGC access, model services, monitoring, upgrades and license activation.
- Product website: https://www.nvidia.com/en-us/data-center/products/ai-enterprise/
- Source review: 2026-11-18; cloud review: 2026-11-16; licensing review: 2026-11-16; infrastructure review: 2026-11-16

### Scope and qualifications

This entry covers customer-operated NVIDIA AI Enterprise software, not a managed AI service. A license is required and is not included with most standard, GPU-optimized VMI or managed-Kubernetes deployments; on-demand NVIDIA AI Enterprise VMIs and private offers use separate included-license processes. Not every component is available on every cloud or deployment model. GPU deployments are generally licensed per GPU, while CPU-only instances require an instance license. Tencent Cloud and ByteDance Volcano Engine have documented standard-instance BYOL paths. These guides require certified software and supported instance types; they do not establish that every GPU, component or provider image is eligible. Their per-instance wording differs from the general per-GPU licensing guidance, so confirm the applicable licensing unit. The November lifecycle matrix lists active 9.0/R620, 8.3/R595, and 7.9 LTSB/R580 branches with different end dates. NeMo Microservices support ended October 1, 2026, after 25.12 became the final GA release; select and patch each component branch independently.

NVIDIA AI Enterprise requires a commercial subscription, consumption or perpetual entitlement. Cloud instances are usually BYOL unless the selected on-demand VMI or private offer explicitly includes licensing. The Tencent and Volcano standard-instance guides state one license per cloud instance, while the general licensing guide describes per-GPU BYOL. Confirm the applicable unit for the selected instance, GPU count and agreement before purchase.

NVIDIA AI Enterprise runs licensed AI containers and infrastructure components on persistent customer GPU or CPU instances and Kubernetes worker nodes.


### Category fit

- ai-tools: Provides an enterprise-supported suite of AI frameworks, infrastructure software, operations components and development tools.

- ai-inference-gateways: Includes supported inference and model-serving components for deploying generative-AI and machine-learning workloads.

### Deployment boundaries


- aws: Customer cloud account; Customer-operated; control plane: not-assessed. Source: https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/overview.html (checked 2026-11-16). 

- azure: Customer cloud account; Customer-operated; control plane: not-assessed. Source: https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/overview.html (checked 2026-11-16). 

- gcp: Customer cloud account; Customer-operated; control plane: not-assessed. Source: https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/overview.html (checked 2026-11-16). 

- oci: Customer cloud account; Customer-operated; control plane: not-assessed. Source: https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/overview.html (checked 2026-11-16). 

- alibaba: Customer cloud account; Customer-operated; control plane: not-assessed. Source: https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/overview.html (checked 2026-11-16). 

- tencent: Customer cloud account; Customer-operated; control plane: not-assessed. Source: https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/tencent-standard-instance.html (checked 2026-11-16). 

- volcano: Customer cloud account; Customer-operated; control plane: not-assessed. Source: https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/bytedance-volcano-engine-standard-instance.html (checked 2026-11-16). 


### Cloud deployment evidence

- NVIDIA AI Enterprise on AWS, Azure and Google Cloud (aws, azure, gcp): https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/overview.html
  NVIDIA documents standard instances, NVIDIA VM images, GPU-optimized VM images, managed Kubernetes and marketplace OpenShift paths across AWS, Azure and Google Cloud. License inclusion varies by image and offer; standard instances and managed Kubernetes generally require BYOL.
  Source type: vendor-documentation; reviewed: 2026-11-16; vendor support: not-assessed.

- NVIDIA AI Enterprise on Oracle Cloud and Alibaba Cloud (oci, alibaba): https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/overview.html
  NVIDIA's current cloud matrix includes standard instances on Oracle Cloud Infrastructure and Alibaba Cloud, NVIDIA AI Enterprise VM images on OCI and GPU-optimized VM images on Alibaba Cloud. Customers must confirm the supported component set, certified instance and license path.
  Source type: vendor-documentation; reviewed: 2026-11-16; vendor support: not-assessed.

- NVIDIA AI Enterprise on Tencent Cloud standard instances (tencent): https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/tencent-standard-instance.html
  NVIDIA documents installing its licensed stack on customer standard instances using a certified Ubuntu operating system, supported NVIDIA drivers, Docker and Container Toolkit, then pulling NGC containers. This is a BYOL installation rather than an included-license marketplace image. Select an instance from the supported-cloud-services matrix; the guide states one license per cloud instance, which must be reconciled with the general licensing guide and the customer agreement.
  Source type: vendor-documentation; reviewed: 2026-11-16; vendor support: not-assessed. Edition: NVIDIA AI Enterprise standard-instance BYOL.

- NVIDIA AI Enterprise on ByteDance Volcano Engine standard instances (volcano): https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/bytedance-volcano-engine-standard-instance.html
  NVIDIA documents installing its licensed stack on customer standard instances using a certified Ubuntu operating system, supported NVIDIA drivers, Docker and Container Toolkit, then pulling NGC containers. This is a BYOL installation rather than an included-license marketplace image. Select an instance from the supported-cloud-services matrix; the guide states one license per cloud instance, which must be reconciled with the general licensing guide and the customer agreement.
  Source type: vendor-documentation; reviewed: 2026-11-16; vendor support: not-assessed. Edition: NVIDIA AI Enterprise standard-instance BYOL.


### Sources

- NVIDIA AI Enterprise cloud deployment guide: https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/index.html

- NVIDIA AI Enterprise cloud deployment overview: https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/overview.html

- NVIDIA AI Enterprise on AWS: https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/aws-overview.html

- NVIDIA AI Enterprise on Azure: https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/azure-overview.html

- NVIDIA AI Enterprise on Google Cloud: https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/google-overview.html

- NVIDIA AI Enterprise licensing: https://docs.nvidia.com/ai-enterprise/planning-resource/licensing-guide/latest/licensing.html

- NVIDIA AI Enterprise on Tencent Cloud standard instances: https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/tencent-standard-instance.html

- NVIDIA AI Enterprise on ByteDance Volcano Engine standard instances: https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/bytedance-volcano-engine-standard-instance.html

- NVIDIA AI Enterprise documentation: https://docs.nvidia.com/ai-enterprise/

- NVIDIA AI Enterprise lifecycle: https://docs.nvidia.com/ai-enterprise/lifecycle/latest/index.html

### Licensing sources

- https://docs.nvidia.com/ai-enterprise/planning-resource/licensing-guide/latest/licensing.html

- https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/tencent-standard-instance.html

- https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/bytedance-volcano-engine-standard-instance.html

### Infrastructure sources

- https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/overview.html

- https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/index.html

- https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/tencent-standard-instance.html

- https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/bytedance-volcano-engine-standard-instance.html


### Buyer diligence


Research coverage: 10/12 fields (83%); 7 documented, 3 unresolved. Coverage is not a feature score.

Unrecorded fields are not researched, not a statement that the feature is unavailable. Sources checked — unresolved means the cited material did not establish an answer. Hosted-service certifications do not automatically cover customer installations.


- Marketplace delivery: Edition-dependent. The deployment matrix distinguishes license-included AI Enterprise VM images/private offers from BYOL paths. A generic GPU-optimized image or Kubernetes installation does not include AI Enterprise licensing merely because it comes from a cloud provider. Scope: Offer-specific licensing; no cloud-commitment eligibility established. Source: https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/overview.html (reviewed 2026-11-16).

- Cloud commitment eligibility: Edition-dependent. NVIDIA explicitly states that AI Enterprise AWS Marketplace private offers are eligible toward an AWS committed-spend agreement. Confirm the specific offer and buyer agreement; this statement does not establish Azure/GCP credit or eligibility for every image or BYOL purchase. Scope: Vendor statement for AI Enterprise AWS Marketplace private offers only. Source: https://www.nvidia.com/en-us/data-center/lp/aws-marketplace-offer/ (reviewed 2026-11-16).

- Single sign-on: Sources checked — unresolved. AI Enterprise is a suite with independently versioned application and infrastructure components. NGC/catalog account authentication does not establish SAML/OIDC for every deployed microservice or workload. Select the actual NIM/framework/management components and confirm their runtime identity integrations and entitlements. Scope: Suite-wide runtime SSO cannot be established from distribution/account access. Source: https://docs.nvidia.com/ai-enterprise/latest/user-guide/index.html (reviewed 2026-11-20).

- SCIM provisioning: Not researched.

- Compliance attestations: Sources checked — unresolved. Government Ready, STIG-hardening and FIPS-library statements describe selected NVIDIA software branches. The reviewed page does not establish a current independent attestation for the complete customer-account deployment, third-party integrations or every suite component. Scope: Selected software-artifact assurances versus a customer-deployment audit. Source: https://docs.nvidia.com/ai-enterprise/planning-resource/ai-enterprise-infrastructure-software/latest/enterprise-software-built-on-open-source.html (reviewed 2026-11-16).

- Security controls: Documented. NVIDIA describes Government Ready infrastructure software and Production Branch application software using STIG-hardened containers, FIPS-compliant libraries and monthly security/maintenance patches. The statement is branch-specific; it does not certify every suite component, cloud image or customer deployment. Scope: AI Enterprise infrastructure software and application Production Branch; not an independent customer-environment attestation. Source: https://docs.nvidia.com/ai-enterprise/planning-resource/ai-enterprise-infrastructure-software/latest/enterprise-software-built-on-open-source.html (reviewed 2026-11-16).

- Air-gapped operation: Documented. NVIDIA documents registering a Delegated License Service appliance on an air-gapped network by manually transferring registration details through its Licensing Portal. This confirms an offline licensing workflow for applicable components, not disconnected compatibility of every AI Enterprise service or model. Scope: NVIDIA License System DLS workflow; component licensing and application dependencies differ. Source: https://docs.nvidia.com/license-system/latest/nvidia-license-system-quick-start-guide/ (reviewed 2026-11-16). Supporting source: When AI Enterprise requires the license system — https://docs.nvidia.com/ai-enterprise/planning-resource/licensing-guide/latest/license-system.html.

- Deployment artifact / IaC: Documented. NVIDIA provides provider-specific image, standard-instance and Kubernetes procedures rather than one universal template. Tencent and Volcano standard-instance guides require a supported instance, certified Ubuntu, NVIDIA driver, Docker, Container Toolkit and NGC tooling, with BYOL. Scope: Supported cloud matrix; component, operating system and instance compatibility must all match. Source: https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/overview.html (reviewed 2026-11-16). Supporting source: Tencent standard instance — https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/tencent-standard-instance.html. Supporting source: Volcano Engine standard instance — https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/bytedance-volcano-engine-standard-instance.html.

- Minimum footprint / sizing: Sources checked — unresolved. The suite includes multiple frameworks, drivers, operators and NIM services, each with its own support matrix. NIM directs buyers to model- and hardware-specific prerequisites. No suite-wide CPU/RAM/VRAM minimum is established; select components, model profiles, supported GPUs, concurrency and redundancy before sizing. Scope: Suite/component/model dependent; complete workload footprint unresolved. Source: https://docs.nvidia.com/ai-enterprise/latest/user-guide/index.html (reviewed 2026-11-20). Supporting source: NIM component-specific prerequisite path — https://docs.nvidia.com/nim/large-language-models/latest/get-started/index.html.

- License pricing: Documented. The current licensing guide lists a one-year self-managed subscription at USD 4,500 per GPU including standard support. Multi-year, perpetual, educational and partner terms differ. Cloud-marketplace and bundled-hardware entitlements have separate rules; this software list price excludes GPU infrastructure and is not a workload quote. For Tencent/Volcano BYOL paths, the previously noted per-instance versus per-GPU unit still needs contract confirmation. Scope: Self-managed NVIDIA Partner Network list price; CSP marketplace terms differ. Source: https://docs.nvidia.com/ai-enterprise/planning-resource/licensing-guide/latest/pricing.html (reviewed 2026-11-20). Supporting source: Cloud BYOL unit requires separate confirmation — https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/tencent-standard-instance.html. Supporting source: Volcano Engine BYOL path — https://docs.nvidia.com/ai-enterprise/deployment/cloud/latest/bytedance-volcano-engine-standard-instance.html.

- Operating cost estimate: Not researched.

- Release snapshot / cadence: Documented. NVIDIA's November 2026 lifecycle material lists active 9.0/R620, 8.3/R595, and 7.9 LTSB/R580 branches with distinct support windows. NeMo Microservices support ended October 1, 2026, after 25.12 was designated the final GA release. Scope: AI Enterprise component and branch lifecycle; no single suite version covers every deployment. Source: https://docs.nvidia.com/ai-enterprise/lifecycle/latest/index.html (reviewed 2026-11-18).







## SGLang

Model-serving framework for running supported language and multimodal models on customer-controlled compute.

- Listing: https://launchinyourcloud.com/directory/sglang/
- Publisher listing: No
- Entry kind: product
- Documented cloud IDs: gcp
- Operating model: Customer-operated
- Infrastructure IDs: servers
- License: Apache License 2.0 for upstream SGLang. Model weights and hardware/runtime dependencies have their own licenses.
- Licensing model: open-source
- Installation: Install the package or use official containers; the reviewed primary installation path targets Linux with supported NVIDIA GPUs and compatible drivers.
- Architecture: Customer-operated inference server and model weights on configured accelerator hosts
- Operations: Customer manages accelerator capacity, model artifacts, runtime compatibility, endpoint access, including isolation of debug and dumper routes, monitoring, and upgrades.
- Product website: https://sglang.io/
- Source review: 2026-11-16; cloud review: 2026-11-12; licensing review: 2026-11-10; infrastructure review: 2026-11-10

### Scope and qualifications

Model downloads may require external registries and gated-model credentials. Pin compatible model/runtime/image versions and review model-specific permissions. Generic container instructions do not establish deployment badges for every hyperscaler. CVE-2026-111391 rates SGLang 0.5.10 through 0.5.22 critical (CVSS v4.0 9.2) for Python code execution through the debug dumper filter. The upstream report demonstrates the shared-API path with DUMPER_SERVER_PORT=reuse: an ordinary API-key holder can reach it when no separate admin API key is configured; access can be anonymous when no API key is configured. With an admin API key configured, that key is required. Exploitation also requires access to the dumper configuration and a subsequent model execution that reaches a dumper hook. Upstream issue #44188 remains open, and the v0.5.23 tag still contains the eval-based filter path even though that tag is outside the CVE record's enumerated affected range. Do not enable or expose the debug dumper endpoint; restrict it at the network layer and verify an upstream fix before treating a later version as remediated.

Apache License 2.0 for upstream SGLang. Model weights and hardware/runtime dependencies have their own licenses. This classification covers the reviewed upstream edition; external services and dependencies retain their own terms.

Customer-operated inference server and model weights on configured accelerator hosts. This is a customer-operated server/container deployment; portability does not establish named cloud-provider support.


### Category fit

- ai-inference-gateways: Model-serving framework for LLM and multimodal inference; an inference runtime rather than a full multi-provider governance gateway.

- ai-tools: Customer-operated inference infrastructure for serving supported model weights.

### Deployment boundaries


- gcp: Customer cloud account; Customer-operated; control plane: not-assessed. Source: https://docs.cloud.google.com/kubernetes-engine/docs/how-to/setup-multicluster-inference-gateway (checked 2026-11-12). 


### Cloud deployment evidence

- SGLang compatibility with GKE Inference Gateway (gcp): https://docs.cloud.google.com/kubernetes-engine/docs/how-to/setup-multicluster-inference-gateway
  Google explicitly supports SGLang as an alternative model server in its customer-operated multi-cluster GKE inference architecture. The worked manifests use vLLM; SGLang requires its own deployment, serving port, metric mapping and compatible Endpoint Picker image. Review GPU/model requirements, licenses and resource availability.
  Source type: provider-guide; reviewed: 2026-11-12; vendor support: not-assessed.


### Community and maintenance

The upstream release history documents hundreds of pull requests from many named contributors and sustained releases; its README describes production use across labs, platforms and enterprises.

Reviewed 2026-11-10.

- Release maintenance and contributor participation: https://github.com/sgl-project/sglang/releases

- Project and production-use context: https://raw.githubusercontent.com/sgl-project/sglang/main/README.md

### Sources

- Official installation and runtime: https://docs.sglang.io/docs/get-started/install

- Upstream project and capabilities: https://github.com/sgl-project/sglang

- Current upstream license: https://raw.githubusercontent.com/sgl-project/sglang/main/LICENSE

- CVE-2026-111391 debug dumper filter code injection: https://nvd.nist.gov/vuln/detail/CVE-2026-111391

- Upstream SGLang dumper-filter RCE issue: https://github.com/sgl-project/sglang/issues/44188

- SGLang v0.5.23 debug dumper implementation: https://github.com/sgl-project/sglang/blob/v0.5.23/python/sglang/srt/debug_utils/dumper.py#L1309-L1320

- CVE-2026-111391 primary record and affected-version range: https://cveawg.mitre.org/api/cve/CVE-2026-111391

### Licensing sources

- https://raw.githubusercontent.com/sgl-project/sglang/main/LICENSE

### Infrastructure sources

- https://docs.sglang.io/docs/get-started/install


### Buyer diligence


Research coverage: 6/12 fields (50%); 4 documented, 2 unresolved. Coverage is not a feature score.

Unrecorded fields are not researched, not a statement that the feature is unavailable. Sources checked — unresolved means the cited material did not establish an answer. Hosted-service certifications do not automatically cover customer installations.


- Marketplace delivery: Not researched.

- Cloud commitment eligibility: Not researched.

- Single sign-on: Sources checked — unresolved. Server arguments document inference and administrative API keys with separate endpoint rules. The reviewed upstream configuration does not establish SAML/OIDC login, user provisioning or a supported IdP integration. Confirm a gateway-based identity design instead of treating a static API key as SSO. Scope: Upstream inference-server/API authentication; federation unresolved. Source: https://docs.sglang.ai/advanced_features/server_arguments.html (reviewed 2026-11-20).

- SCIM provisioning: Not researched.

- Compliance attestations: Not researched.

- Security controls: Documented. The reported shared-API dumper exposure requires DUMPER_SERVER_PORT=reuse and access to configure a filter followed by a model execution. Without a separate admin key, an ordinary API key can reach the route; without any API key, it can be anonymous. A configured admin key is required when present. Keep dumper/debug routes disabled or network-isolated. The CVE enumerates 0.5.10–0.5.22; retained filter code in 0.5.23 is not by itself a complete assessment of every authentication configuration. Scope: Debug dumper trust boundary and deployment precautions; not a blanket claim that all SGLang endpoints or all later releases are unauthenticated. Source: https://github.com/sgl-project/sglang/issues/44188 (reviewed 2026-11-16). Supporting source: Primary CVE record — https://cveawg.mitre.org/api/cve/CVE-2026-111391. Supporting source: SGLang v0.5.23 shared dumper route and authentication level — https://github.com/sgl-project/sglang/blob/v0.5.23/python/sglang/srt/entrypoints/http_server.py. Supporting source: SGLang v0.5.23 filter implementation — https://github.com/sgl-project/sglang/blob/v0.5.23/python/sglang/srt/debug_utils/dumper.py#L1309-L1320.

- Air-gapped operation: Not researched.

- Deployment artifact / IaC: Documented. The upstream installation guide supplies container, Compose and Kubernetes options for model serving. Match model/backend, accelerator drivers and selected image; examples do not establish a universal GPU or memory minimum. Scope: Upstream self-hosted deployment; not a provider support or production-readiness certification. Source: https://docs.sglang.io/docs/get-started/install (reviewed 2026-11-14).

- Minimum footprint / sizing: Sources checked — unresolved. The server configuration sizes model loading, quantization, parallelism, KV cache and admitted requests through memory controls such as mem-fraction-static. It does not establish one safe CPU/RAM/VRAM minimum. Select model, precision, context, concurrency and availability, then validate the chosen accelerator topology. Scope: Model and serving-configuration dependent; complete production footprint unresolved. Source: https://docs.sglang.ai/advanced_features/server_arguments.html (reviewed 2026-11-20).

- License pricing: Documented. The reviewed upstream core is Apache-2.0 licensed. Optional services, bundled dependencies, commercial distributions and support can have separate terms. This records the source licence, not an infrastructure or operating-cost estimate. Model weights and external model APIs require their own review. Scope: Upstream core licence; commercial features, support and hosting priced separately. Source: https://raw.githubusercontent.com/sgl-project/sglang/main/LICENSE (reviewed 2026-11-20).

- Operating cost estimate: Not researched.

- Release snapshot / cadence: Documented. The publisher-designated GitHub release checked was v0.5.23, published 2026-11-02. This is a release snapshot, not a promised cadence, support designation or approval to substitute it into a provider package. Scope: Upstream sgl-project/sglang release record; commercial editions and deployment packages may differ. Source: https://github.com/sgl-project/sglang/releases/tag/v0.5.23 (reviewed 2026-11-14).




### Security and lifecycle reviews

Selected editorial reviews only; no flag does not mean safe or supported.

- SGLang debug dumper code execution (security-advisory): CVE-2026-111391 identifies code execution in the debug dumper filter. Check the debug endpoint exposure and authentication boundary. Scope: CVE record: SGLang 0.5.10–0.5.22. The open upstream report and reviewed v0.5.23 code do not establish a fixed release. Conditions: Requires dumper configuration access and a subsequent model execution reaching a dumper hook. In shared-API mode, an ordinary API key can suffice without a separate admin key; authentication configuration changes exposure. Action: Disable or isolate debug dumper endpoints and verify an upstream fix before treating a newer release as remediated. Source: https://cveawg.mitre.org/api/cve/CVE-2026-111391; reviewed 2026-11-16. Supporting source: Upstream report and authentication conditions — https://github.com/sgl-project/sglang/issues/44188. Supporting source: Reviewed v0.5.23 filter implementation — https://github.com/sgl-project/sglang/blob/v0.5.23/python/sglang/srt/debug_utils/dumper.py#L1309-L1320.




## vLLM Self-Hosted

Self-hosted model inference and serving engine with OpenAI-compatible APIs.

- Listing: https://launchinyourcloud.com/directory/vllm/
- Publisher listing: No
- Entry kind: product
- Documented cloud IDs: aws, azure, gcp
- Operating model: Customer-operated
- Infrastructure IDs: servers
- License: Apache 2.0 for vLLM; model weights and supporting dependencies have independent licenses and access conditions.
- Licensing model: open-source
- Installation: Official vLLM Docker image or supported Python installation; choose a compatible hardware/backend configuration.
- Architecture: Customer-operated model-serving process, model weights and inference hardware
- Operations: Customer manages model acquisition, hardware capacity, endpoint protection, availability, upgrades and model/runtime compatibility.
- Product website: https://vllm.ai/
- Source review: 2026-11-17; cloud review: 2026-11-12; licensing review: 2026-11-10; infrastructure review: 2026-11-17

### Scope and qualifications

This entry covers the inference server, not a hosted model service. It can run supported model weights on customer infrastructure, but those weights may need external downloads or access approval and are not licensed by vLLM. Review default usage telemetry and optional outbound integrations; hosting inference does not automatically create an air-gapped system. vLLM v0.34.0 removes Model Runner V1; configurations setting VLLM_USE_V2_MODEL_RUNNER=0 now fail at startup. It changes the default block size from 16 to 64, enables extensible KV cache by default with gpu_memory_utilization defaulting to 1.0, drops ROCm 7.2 wheels and images in favor of ROCm 10.0, requires huggingface_hub 2.0.0 or later, and removes QuTLASS/fp_quant and Marlin W4A8 support. Review the upstream breaking-change table and test model, quantization, memory, and accelerator compatibility before rollout.

The reviewed inference runtime is Apache-2.0 licensed. Its license does not grant rights to the model weights served by that runtime; check model and dependency terms separately.

vLLM is a running inference-server workload. Hardware support and capacity depend on the model and selected backend; the documented container paths include accelerator-specific images, and CPU support is separately documented. In v0.34.0, the KV-cache allocator sizes from free memory after warmup and defaults to gpu_memory_utilization=1.0; revalidate GPU headroom, and set 0.9 or disable extensible cache to retain prior sizing. ROCm 10.0 replaces 7.2 for published images and wheels.


### Category fit

- ai-inference-gateways: Model inference engine and OpenAI-compatible serving API for compatible model weights. Distinct from an API gateway that forwards calls to proprietary hosted models.

- ai-tools: Self-hosted model inference and serving engine with OpenAI-compatible APIs.

### Deployment boundaries


- aws: Customer cloud account; Customer-operated; control plane: not-assessed. Source: https://docs.vllm.ai/projects/production-stack/en/vllm-stack-0.1.4/deployment/cloud-deployment/aws.html (checked 2026-11-12). 

- azure: Customer cloud account; Customer-operated; control plane: not-assessed. Source: https://docs.vllm.ai/projects/production-stack/en/vllm-stack-0.1.4/deployment/cloud-deployment/azure.html (checked 2026-11-12). 

- gcp: Customer cloud account; Customer-operated; control plane: not-assessed. Source: https://docs.vllm.ai/projects/production-stack/en/vllm-stack-0.1.4/deployment/cloud-deployment/gcp.html (checked 2026-11-12). 


### Cloud deployment evidence

- vLLM Production Stack deployment on Amazon EKS (aws): https://docs.vllm.ai/projects/production-stack/en/vllm-stack-0.1.4/deployment/cloud-deployment/aws.html
  The project publishes Amazon EKS provisioning and vLLM deployment scripts in its versioned Production Stack guide. Verify current chart versions, model licenses, accelerator availability, GPU quotas and storage; this is customer-hosted inference, not a hosted model API.
  Source type: vendor-documentation; reviewed: 2026-11-12; vendor support: not-assessed.

- vLLM Production Stack deployment on Azure Kubernetes Service (azure): https://docs.vllm.ai/projects/production-stack/en/vllm-stack-0.1.4/deployment/cloud-deployment/azure.html
  The project publishes Azure Kubernetes Service provisioning and vLLM deployment scripts in its versioned Production Stack guide. Verify current chart versions, model licenses, accelerator availability, GPU quotas and storage; this is customer-hosted inference, not a hosted model API.
  Source type: vendor-documentation; reviewed: 2026-11-12; vendor support: not-assessed.

- vLLM Production Stack deployment on Google Kubernetes Engine (gcp): https://docs.vllm.ai/projects/production-stack/en/vllm-stack-0.1.4/deployment/cloud-deployment/gcp.html
  The project publishes Google Kubernetes Engine provisioning and vLLM deployment scripts in its versioned Production Stack guide. Verify current chart versions, model licenses, accelerator availability, GPU quotas and storage; this is customer-hosted inference, not a hosted model API.
  Source type: vendor-documentation; reviewed: 2026-11-12; vendor support: not-assessed.


### Sources

- Official Docker serving images: https://docs.vllm.ai/en/latest/deployment/docker/

- Online serving: https://docs.vllm.ai/en/latest/serving/online_serving/

- Usage statistics and opt-out: https://docs.vllm.ai/en/latest/usage/usage_stats/

- Apache 2.0 license: https://github.com/vllm-project/vllm/blob/main/LICENSE

- vLLM v0.34.0 breaking upgrade notes: https://github.com/vllm-project/vllm/releases/tag/v0.34.0

### Licensing sources

- https://github.com/vllm-project/vllm/blob/main/LICENSE

### Infrastructure sources

- https://docs.vllm.ai/en/latest/deployment/docker/

- https://github.com/vllm-project/vllm/releases/tag/v0.34.0


### Buyer diligence


Research coverage: 5/12 fields (42%); 3 documented, 2 unresolved. Coverage is not a feature score.

Unrecorded fields are not researched, not a statement that the feature is unavailable. Sources checked — unresolved means the cited material did not establish an answer. Hosted-service certifications do not automatically cover customer installations.


- Marketplace delivery: Not researched.

- Cloud commitment eligibility: Not researched.

- Single sign-on: Sources checked — unresolved. The security guide documents API-key protection for specified HTTP path prefixes and warns that other sensitive endpoints remain unprotected. It does not establish a native SAML/OIDC user sign-on workflow. Use a reviewed gateway/network design; an API bearer key alone is not corporate SSO or complete endpoint protection. Scope: Upstream inference-server auth surface; federated user integration unresolved. Source: https://docs.vllm.ai/en/latest/usage/security/ (reviewed 2026-11-20).

- SCIM provisioning: Not researched.

- Compliance attestations: Not researched.

- Security controls: Not researched.

- Air-gapped operation: Not researched.

- Deployment artifact / IaC: Documented. The stable deployment guide links the official vllm/vllm-openai image for a CUDA-backed OpenAI-compatible server. Select a release/architecture tag and compatible GPU drivers; model memory, credentials and optional dependencies need separate review. Scope: Stable upstream CUDA image path; not the latest developer-preview guide or a GPU sizing guarantee. Source: https://docs.vllm.ai/en/stable/deployment/docker/ (reviewed 2026-11-14). Deployment artifact: https://hub.docker.com/r/vllm/vllm-openai. Artifact reference: select a stable version and architecture tag; latest is moving.

- Minimum footprint / sizing: Sources checked — unresolved. The stable memory guide sizes serving around model weights, precision/quantization, tensor parallelism, context length, batch size and cache. A supported accelerator alone does not establish sufficient memory. Select the exact model and target throughput/latency, then benchmark VRAM/RAM and HA replica requirements. Scope: Model-specific inference sizing; no universal numeric footprint established. Source: https://docs.vllm.ai/en/stable/configuration/conserving_memory/ (reviewed 2026-11-20).

- License pricing: Documented. The reviewed upstream core is Apache-2.0 licensed. Optional services, bundled dependencies, commercial distributions and support can have separate terms. This records the source licence, not an infrastructure or operating-cost estimate. Model weights and external model APIs require their own review. Scope: Upstream core licence; commercial features, support and hosting priced separately. Source: https://raw.githubusercontent.com/vllm-project/vllm/main/LICENSE (reviewed 2026-11-20).

- Operating cost estimate: Not researched.

- Release snapshot / cadence: Documented. The reviewed upstream release is v0.34.0, published 2026-11-16. This is a dated release snapshot, not a support designation, a future cadence promise or approval to replace a provider package. Scope: Upstream vllm-project/vllm release record; commercial editions, LTS branches and provider packages can differ. Source: https://github.com/vllm-project/vllm/releases/tag/v0.34.0 (reviewed 2026-11-17).






