Sign in

AI monitoring

See what your agents call, and what it costs.

Unhandled reads the OpenTelemetry GenAI spans your AI features already emit. Every model call, tool call and evaluation score lands next to its trace, and token usage is priced from published list prices, so you can see which model and which conversation the money went to.

Sign inRead the docs
Estimated cost
Per model, priced from list prices
Conversations
Each agent loop, step by step
Scorers
Evaluation results per scorer and label
MCP monitoring
Tools, clients and users of your server
#models

Cost and tokens per model

The Models tab has one row per model with its calls, average and p95 latency, input, output and cached tokens, cost and errors. Two charts stack cost and tokens for the five busiest models over the range.

  • Prices apply when you open the page, so a model added to the price table later is priced for its earlier calls too
  • Model ids match with or without a provider prefix or date suffix, so Bedrock and OpenRouter ids find their price
  • A span that reports gen_ai.usage.cost keeps that cost, and a model with no known price shows No price instead of $0
#setup

Send spans from the SDK you already use

Install the Vercel AI SDK or Mastra instrumentation and register it with your agent. Any other span counts as AI when it carries the GenAI operation or provider attribute, so OpenLLMetry spans work as they are.

  • The Vercel AI SDK package traces generateText, streamText, generateObject, streamObject, embed, embedMany and rerank
  • Prompts and responses are recorded only when you set recordInputs and recordOutputs, and tokens and cost arrive either way
  • Recorded content is scrubbed of secrets and email addresses before it is stored
config/unhandled.ts
import env from '#start/env'
import { defineConfig } from '@unhandled/agent-adonisjs'
import { vercelAi } from '@unhandled/instrumentation-vercel-ai/integration'

export default defineConfig({
  dsn: env.get('UNHANDLED_DSN'),
  integrations: [vercelAi({ recordInputs: true, recordOutputs: false })],
})
#conversations

Replay a conversation step by step

Spans that share a gen_ai.conversation.id form a conversation. Its page lists every step in order with its offset, operation, model or tool, tokens, cost, duration and status, with the scores it received beside it.

  • The Conversations tab lists the 100 most recent with agent, models, messages, tool calls, tokens, cost and score
  • The overview ranks the five most expensive and the five slowest conversations in the range
  • Every trace ID the conversation touched is listed, ready to copy into the trace view
#scorers

Evaluation scores next to the calls they judge

Add a gen_ai.evaluation.result event to any span, or let the Mastra instrumentation send its scorer results. Each scorer gets its average score, a distribution of scores from 0 to 1, a trend and counts per label and source.

  • Labels are pass, warning and fail, and the pass rate counts only results that have a label
  • Each scorer counts its live, offline and manual results apart, and shows its version
  • The 50 newest results show their explanation and link to the conversation they scored
#mcp

Watch the MCP server you ship

If your application exposes a Model Context Protocol server, MCP monitoring shows which tools get called, by which clients and users, how fast they answer and how often they fail.

  • Tools show p50, p95 and p99 latency and error rate, and clients show each version they connected with
  • A call counts as failed when the tool returns isError, the span status is error, or the request gets a JSON-RPC error
  • Each tool has its own page with its 50 most recent calls, their errors and their trace IDs
#mcp-setup

One call before connect

Wrap your server with the MCP instrumentation before it connects. It works with @modelcontextprotocol/sdk 1.0.0 and later, and spans from any other tracer show up too when they carry the MCP attributes.

  • Set user.id from your tool handler to fill the Users table and the unique users count
  • The Methods table counts every JSON-RPC method, such as initialize, tools/list and resources/read
  • This is about your server. To let an assistant read your Unhandled data, use the Unhandled MCP server
start/mcp.ts
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js'
import { instrumentMcpServer } from '@unhandled/instrumentation-mcp-server'

const server = new McpServer({ name: 'shop-assistant', version: '2.3.0' })
instrumentMcpServer(server, { recordInputs: true })

Sources

Where the spans come from

#faq

Questions

In the docs

How it works, in detail

Every feature

Try it on your own stack.

Every feature is on every plan, priced by the telemetry you send.

Sign inRead the docs