Observability Overview
Latitude captures every interaction your AI agent has, from individual LLM calls to multi-turn conversations, and makes them searchable, scoreable, and actionable.Three Levels of Telemetry
Latitude organizes your agent’s activity into three levels:Spans
A span is the smallest unit of work: a single LLM call, a tool invocation, an HTTP request, a retrieval step, or any instrumented operation in your agent’s pipeline. Not every span represents an AI generation. A span might be a database query, an API call, or a custom function, none of which involve tokens or models. Every span captures:- Input and output content
- Latency (start, end, duration)
- Status (success or error)
- Custom metadata and tags
- Token usage and cost
- Model and provider information
Traces
A trace is a complete interaction from start to finish, composed of one or more spans. When a user sends a message and your agent responds (including any intermediate LLM calls, tool uses, and retrieval steps), that entire sequence is one trace. Traces are the primary unit that most reliability features operate on:- Evaluations run against traces
- Scores are attached to traces
- Annotation queues contain traces
- Issues group failures across traces
Sessions
A session is an optional grouping of related traces into a multi-turn conversation. Unlike spans and traces, which are always present, sessions only exist when your application provides asession_id with its telemetry. When the same user has an ongoing conversation with your agent, each turn is a separate trace, but they all belong to the same session.
Sessions let you:
- View the full conversation context across turns
- Evaluate behavior at the session level (not just individual turns)
- Aggregate scores and analytics across a conversation
How Data Flows In
Your application sends telemetry to Latitude using OpenTelemetry-compatible instrumentation. See the Telemetry docs for setup instructions with your specific provider or framework. Once connected:- Your agent processes a request
- Instrumented operations emit spans to Latitude, grouped into traces and optionally sessions as defined by your telemetry client
- If a
session_idis provided, the trace is associated with a session - Once a trace is considered complete (see below), Latitude triggers downstream reliability features
Trace Completion
Latitude does not consider a trace complete the instant a span arrives. Instead, it uses a debounce window. If no new spans arrive for a trace within 5 minutes, the trace is considered done. This matters because:- Complex agent pipelines may emit spans over an extended period
- Multi-step tool use can have significant delays between operations
- You don’t want evaluations running on half-complete interactions
- Runs matching evaluations against it
- Checks if it should be added to any live annotation queues
- Classifies it against system annotation queues
Filtering and Search
The trace dashboard provides rich filtering through a shared filter system. You can filter by:| Filter | Description |
|---|---|
| Status | Error, OK, or unset |
| Name | Root span name |
| Session ID | Filter to a specific session |
| User ID | Filter to a specific user |
| Tags | Custom tags attached to spans |
| Models | Which LLM models were used |
| Providers | Which providers were called |
| Cost | Total cost of the trace |
| Duration | End-to-end duration |
| Span count | Number of spans in the trace |
| Error count | Number of errored spans |
| Token usage | Input and output tokens |
| Custom metadata | Any metadata.* fields you send |