AI observability & LLMOps
Model and LLM traces, experiments, evaluation and operational visibility.
4 listings, including 4 from other publishers. Entries cover the deployment options documented on their individual pages.
Read this page as JSON or Markdown.
Watch this category with RSS · Build a requirements shortlist
Deployment comparison
| Listing | Documented clouds | Deployment and infrastructure | Licensing | Installation | Operations and limitations |
|---|---|---|---|---|---|
| Arize Phoenix Self-Hosted | AWS, Azure, Google Cloud, Railway, Render | Customer-operated Servers / containers | Source Available Elastic License 2.0; includes restrictions on providing the software as a hosted/managed service and on license-key circumvention. This is source-available. | Use the official self-hosted Docker/Compose or Kubernetes deployment instructions and configure persistent storage and instrumentation endpoints. | Customer manages trace capture, storage, authentication, retention, evaluation providers, backups and upgrades.Scope and evidenceThe self-hosted product is distinct from commercial Arize offerings. Traces may contain prompts and outputs; evaluation jobs can call separately configured external model services even when trace storage is local.
Cloud evidence reviewed 2026-11-12 |
| Langfuse Self-Hosted | AWS, Azure, Google Cloud, Railway, Render | Customer-operated Servers / containers | Open Source / Open Core MIT core; designated enterprise directories and add-on capabilities use separate license terms or keys. | Docker Compose for a single VM; Helm/Kubernetes and official AWS, Azure and Google Cloud Terraform modules. | Customer operates the deployment, persistence, access policies, retention, backups and scaling.Scope and evidenceTraces can contain prompts, outputs and other sensitive application data, so configure capture and retention deliberately. Self-hosted trace storage does not move the underlying application model into the same account. Optional model-based evaluators or the gateway use configured providers, which may be external. The single-VM Compose example does not provide production high availability or backups. Langfuse v4.82.0 fixes an SSRF path in GitHub Enterprise SSO and moves audit logs to ClickHouse. Review outbound SSO connectivity and audit-log storage during upgrade.
Cloud evidence reviewed 2026-11-12 |
| MLflow Tracking Server Self-Hosted | AWS, Azure, Google Cloud | Customer-operated Servers / containers | Open Source / Open Core Apache 2.0 for upstream MLflow; connected model providers and managed distributions have separate terms. | Python MLflow server command, official Docker Compose bundle or Kubernetes Helm deployment. | Customer operates server access, metadata and artifact stores, retention, backups, scaling and version upgrades.Scope and evidenceTraces can capture model inputs, outputs and intermediate metadata. Point clients at the customer-hosted server and review recording/redaction settings. Applications, evaluators and optional model integrations may still call external providers; hosting the tracking server does not host all training or inference runtimes. Production database, storage and security configuration differ from a local quickstart.
Cloud evidence reviewed 2026-11-12 |
| OpenLIT | DigitalOcean | Customer-operated Servers / containers | Open Source / Open Core Apache License 2.0 for upstream OpenLIT. Models and external integrations have separate terms. | Run the official Docker Compose platform or documented Kubernetes deployment with its persistent telemetry services. | Customer manages telemetry collection, analytical storage, capture policies, access controls, retention and upgrades.Scope and evidenceConfigure instrumentation to send telemetry to the self-hosted receiver. Optional LLM-based evaluations and integrations can call external services; hosting the observability platform does not host every model being observed.
Cloud evidence reviewed 2026-11-12 |
Choose 2–4 listings to compare
Browse listing cards
AI observability & LLMOps +1 more
Arize Phoenix Self-Hosted
AI observability and evaluation server for traces, datasets and application experiments.
- AWS
- Azure
- Google Cloud
- Railway
- Render
Buyer facts and open questions →
3 sources checked
AI observability & LLMOps +2 more
Langfuse Self-Hosted
Self-hosted LLM observability, evaluations and prompt-management platform.
- AWS
- Azure
- Google Cloud
- Railway
- Render
Buyer facts and open questions →
10 sources checked
AI observability & LLMOps +2 more
MLflow Tracking Server Self-Hosted
Self-hosted experiment tracking and LLM/agent observability with customer-controlled metadata and artifacts.
- AWS
- Azure
- Google Cloud
Buyer facts and open questions →
3 sources checked
AI observability & LLMOps +1 more
OpenLIT
OpenTelemetry-based AI tracing and evaluation platform for model and agent operations.
- DigitalOcean
Buyer facts and open questions →
3 sources checked
How these listings fit
- Arize Phoenix Self-Hosted
- AI trace collection, evaluation, datasets, experiments and prompt analysis for instrumented model applications.
- Langfuse Self-Hosted
- LLM/agent tracing, evaluation and prompt management deployed with customer-controlled storage. Evaluation and optional gateway providers must be scoped independently.
- MLflow Tracking Server Self-Hosted
- Experiment tracking and LLM/agent tracing server. This entry describes customer-hosted tracking/observability services, not every model runtime, managed MLflow product or inference backend.
- OpenLIT
- OpenTelemetry-native tracing, evaluations and cost/GPU monitoring for AI applications and agents.
Review dates and sources appear on each listing. Functionality, edition, deployment and licensing must be checked together. Filter this category with cloud and deployment controls →