# Arize Phoenix: tracing and evals for LLM apps, inside the client's own infrastructure

> The client's chatbot gives a wrong answer, and the client will not let prompts leave its internal network. Phoenix helps you find the step that failed while the data stays in the client's infrastructure. Read its licence carefully, though.

Original: https://fdetimes.net/en/tools/arize-phoenix-llm-tracing-evals-self-hosted/

A client's internal chatbot has just given a wrong answer about the refund policy. The first thing you need to know is which step failed: retrieval pulled the wrong document, the model made something up, or a tool returned stale data. If you cannot see inside a run, all you can do is guess.

Arize Phoenix lets you see inside that run. It is built by Arize AI and its community, and the source code is on GitHub at Arize-ai/phoenix. FDEs care about it because Phoenix runs on a laptop, in a container or in the client's own cloud.

The reason is practical. Banks and hospitals rarely agree to send prompts and internal documents to an outside SaaS. According to Phoenix's self-hosting documentation, when you host it yourself Arize receives no data at all. Everything stays in the client's infrastructure, even when the system is fully cut off from the internet.

## Traces show where it broke, evals show how often

Phoenix does two things. Tracing breaks a run into its sequence of steps: model calls, retrieval, tools, and any logic the client's team wrote themselves. Evals measure output quality, using evaluators based on an LLM, on code or on human labels.

The two belong together because they answer different questions. A trace explains one specific failure. An eval tells you whether that failure is a one-off or a problem across the whole system, and that is the number the client wants to hear in the review meeting.

**Key point:** A trace tells you which step failed; an eval tells you how often it fails.

## Why does OpenTelemetry matter on a client site?

Phoenix is built on OpenTelemetry and collects data through OpenInference instrumentation, so the trace format does not belong to any one vendor. OpenInference is a set of conventions and plugins that extends OpenTelemetry. Phoenix supports it out of the box, but other backends can use it too.

For an FDE, this detail helps in negotiations. Many clients already run OpenTelemetry infrastructure and are wary of vendor lock-in. You can tell them plainly: the instrumentation written today will still work if they later want to send their traces to another system.

## An afternoon with Docker

For a quick trial, one container is enough:

```bash
docker run --name phoenix -p 6006:6006 -p 4317:4317 arizephoenix/phoenix:latest
```

Port 6006 serves both the UI and the OTLP collector over HTTP. Port 4317 is the OTLP collector over gRPC. All that remains is to point the application's exporter at one of the two ports.

Below is a sketch for a Python application that calls the OpenAI SDK. It uses the OpenInference instrumentor and OpenTelemetry's standard OTLP HTTP exporter:

```python
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from openinference.instrumentation.openai import OpenAIInstrumentor

provider = TracerProvider()
provider.add_span_processor(
    BatchSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:6006/v1/traces"))
)
trace.set_tracer_provider(provider)
OpenAIInstrumentor().instrument(tracer_provider=provider)  # every LLM call now becomes a span
```

If the client's network only allows gRPC, switch to OpenTelemetry's gRPC exporter and point it at port 4317. The instrumentation stays the same.

## Broken retrieval or broken model?

Back to the wrong refund answer. Open the trace and look at the retrieval span first. If the chunks it pulled are last year's policy, the fault lies in the index, not the model.

If the documents are right but the answer still drifts from them, you have a case of hallucination, and the fix lies in the prompt or the model. One case is not enough to draw a conclusion, so the next step is to measure.

Imagine you collect 20 refund questions and write a first evaluator in code, just to check whether each answer cites a source:

```python
import re

def has_citation(answer: str) -> bool:
    # Pass if the answer cites a source in the agreed [source: ...] format
    return bool(re.search(r"\[source:[^\]]+\]", answer))
```

Suppose 14 of the 20 pass and 6 fail. A second evaluator uses an LLM to judge whether each answer sticks to the documents, and you label all 20 yourself. If the LLM agrees with you on only 12, do not hand its numbers to the client yet: fix the evaluator's prompt first.

## The licence is the easiest place to trip

The README calls Phoenix "open-source and self-hosted", but the LICENSE file uses the Elastic License 2.0. That is a source-available licence, not an OSI-approved open-source licence. ELv2 prohibits providing the software to third parties as a hosted or managed service.

The clearest way to see the line is to put two scenarios side by side. The table below is one reading of that clause for FDE work, not a substitute for legal advice:

| Scenario | Reading under the ELv2 terms |
|---|---|
| Install Phoenix in the client's own infrastructure, for the client's team to view traces and evals themselves | This is not providing the software to a third party as a service, so it is the lower-risk scenario |
| A consultancy runs a shared Phoenix instance and sells access to several clients | Runs straight into the ban on providing it as a hosted or managed service |

When drafting the statement of work, state clearly whose infrastructure Phoenix runs in and who uses it. Then have legal confirm the boundary before signing.

## From laptop to production

Arize says the self-hosted version is free and has no feature limits. The company also sells a separate managed product called Arize AX. When a client asks why not use the managed version, the answer usually lies in the requirement that data must not leave the network.

The Docker documentation contains two warnings worth remembering. For Postgres, Phoenix officially supports only version 14 and above, so ask about the client's database version in the first meeting. In production, pin the image version instead of using `latest`, so that the next upgrade does not quietly break a live dashboard.

## Common mistakes when bringing Phoenix into a project

The most common mistake is trusting the LLM judge's numbers straight away. An LLM-based evaluator is only worth something once you have compared it with human labels; if it agrees with you on 12 of 20 questions, as in the example above, the "grounded in documents" rate it reports does not belong on a slide yet.

The second mistake is carrying the `docker run ... :latest` command from a laptop into production. It is fine for a trial, but the documentation recommends pinning the exact version you intend to use. A related slip is forgetting to check Postgres: a database below version 14 is outside official support.

The third mistake is in the contract, not the code: treating the word "open-source" in the README as enough, then planning a shared Phoenix instance for several clients. Reading the LICENSE file before writing the SOW is far cheaper than fixing the SOW after it is signed.

## What to learn first?

Learn the OpenTelemetry concepts before the Phoenix UI: traces, spans, OTLP, the difference between HTTP and gRPC. That knowledge works with any backend. Then practise writing evaluators, because the hardest part is defining what a "correct" answer means for the client's business.

When reading FDE job descriptions, look for words such as observability, evals and tracing. On a CV, a line like "built tracing for an LLM application with OpenTelemetry/OpenInference inside a client's air-gapped VPC, tracking the rate of source-grounded answers" carries far more weight than a list of tool names.

Every LLM application gets answers wrong at some point. The person the client trusts is the one who can point to the step that failed and put a measured number on it, inside the infrastructure the client allows.

**Try this week:**

- Run Phoenix with Docker on your laptop, send traces from a small RAG application to port 6006, then open the UI and read one trace from start to finish.
- Write two evaluators for the same set of 20 questions: one in code (does the answer cite a source?) and one with an LLM (does the answer stick to the documents?), then compare both with labels you assign yourself.
- Read the LICENSE file in the Arize-ai/phoenix repo and write a short paragraph on what your team may and may not do when deploying Phoenix for a client.

## Sources

- [What is Arize Phoenix?](https://arize.com/docs/phoenix)

- [GitHub - Arize-ai/phoenix](https://github.com/Arize-ai/phoenix)

- [Phoenix LICENSE (Elastic License 2.0)](https://github.com/Arize-ai/phoenix/blob/main/LICENSE)

- [Self-Hosting - Phoenix documentation](https://arize.com/docs/phoenix/self-hosting)

- [Docker - Phoenix documentation](https://www.arize.com/docs/phoenix/self-hosting/deployment-options/docker)

- [GitHub - Arize-ai/openinference](https://github.com/Arize-ai/openinference)