docs

Troubleshooting

Why telemetry does not reach Unhandled or lands in the wrong place, and how to check each cause in order.

Work through this page top to bottom when a service sends nothing, sends to the wrong environment, or shows gaps. Each section rules out one layer: the app, the agent, the network request, then what Unhandled does with the data. The examples use the AdonisJS agent, but the ingest responses apply to every SDK.

Is the app producing spans?

Turn on debug in config/unhandled.ts:

ts
import env from '#start/env'
import { defineConfig } from '@unhandled/agent-adonisjs'

export default defineConfig({
  dev: env.get('NODE_ENV') === 'development',
  dsn: env.get('UNHANDLED_DSN'),
  environment: env.get('APP_ENV'),
  debug: true,
})

Restart and send a request. The agent prints every span to the console, even when export is off.

  • Nothing prints. The problem is in the app. Read the next two sections.
  • Spans print but never show up in Unhandled. The problem is between the agent and Unhandled. Skip to What the ingest endpoint answered.

Turn debug off again afterwards.

Is the agent turned on?

The agent decides at boot whether to export. It stays off, without a warning, in these cases:

CauseFix
UNHANDLED_DSN is empty or unset in the deployed environmentSet it where the app runs, not only in your local .env.
dev is true. The generated config sets it when NODE_ENV=development, and dev mode ignores the DSNDeploy with NODE_ENV=production, or set dev: false while you test locally.
NODE_ENV=test and enabled is not setExpected. Tests never send telemetry. Set enabled: true to override.
enabled: false in the configRemove it.

When the DSN is set but malformed, the agent disables export and emits a Node.js warning with code UNHANDLED_INVALID_DSN:

Invalid UNHANDLED_DSN: invalid api key. Telemetry export is disabled

The text after the colon names the problem: invalid url, unsupported protocol, missing api key, invalid api key, password not allowed, query not allowed, fragment not allowed, or a path that is not a service UUID. Most of the time the DSN was cut short while copying. You cannot see the full DSN again, so rotate the key from Settings, Services and copy the new one. The format is described under DSN.

An invalid environment does not fail silently. The agent throws UnhandledConfigError at boot with Invalid Unhandled environment "<value>". Use production, staging or development.

Is otel.ts the first import?

import '../otel.js' must be the first import of bin/server.ts, and of bin/console.ts for Ace commands. OpenTelemetry patches modules as they load. Anything imported before otel.ts, such as the HTTP server or Lucid, stays unpatched and produces no spans.

node ace add @unhandled/agent-adonisjs adds the import for you. Check that a later edit or an import sorter did not move it.

Does the dev server crash with "must default export defineConfig()"?

New AdonisJS 7 apps compile TypeScript with @poppinss/ts-exec 1.5, which hooks Node's module loader in the main thread. The agent hooks the loader the same way on Node.js 24.11.1 and later. On an older Node.js 24 release it has to hook a separate loader thread, and TypeScript files then load with no exports: node ace serve and every Ace command fail with config/unhandled.js must default export defineConfig(). Upgrade Node.js to 24.11.1 or later. If you cannot, pin @poppinss/ts-exec to ~1.4.4 in devDependencies. Built apps (node build/bin/server.js) are not affected.

Does a TypeScript app exit at once under @poppinss/ts-exec?

@unhandled/express/register, @unhandled/fastify/register, @unhandled/hono/register, @unhandled/nestjs/register and @unhandled/node/register hook Node's module loader in the main thread on Node.js 24.11.1 and later, which @poppinss/ts-exec 1.5 needs. On an older Node.js 24 release, node --import @poppinss/ts-exec --import @unhandled/express/register server.ts fails with ERR_INVALID_RETURN_PROPERTY_VALUE. Upgrade Node.js to 24.11.1 or later, or pin @poppinss/ts-exec to ~1.4.4. tsx, Node's own type stripping and plain JavaScript work on every Node.js 24 release.

What the ingest endpoint answered

Every rejection is a JSON body with the status and a message, for example {"code":401,"message":"Invalid or revoked API key"}. If you can see the export responses, match them here:

StatusMessageCause
401Missing x-api-key or x-unhandled-service-idThe request has no key or no service ID. This is what the agent's legacy apiKey and endpoint setup gets from Unhandled. Use a DSN.
400x-unhandled-env must be production, staging or developmentThe environment header is missing or not one of the three names. prod and local are rejected.
401Invalid or revoked API keyThe key was rotated or revoked, or its service was deleted.
403API key does not belong to this serviceThe key is valid but the service ID in the DSN belongs to a different service. Copy the DSN again rather than editing it by hand.
403Origin is not allowed for this serviceA browser sent the data from a site the service does not allow.
415Use application/json or application/x-protobufWrong content-type.
415Unsupported content-encoding "..."Only gzip and no encoding are accepted.
413OTLP request bodies are limited to 10 MBLower the batch size with the agent's batch option.
413Decompressed OTLP body exceeds 16 MBSame fix.
429Ingest is limited to 500 requests per second per serviceBatch more items per request. The response carries retry-after.
429This organization is sending more bytes than its plan's ingest rate, retry laterThe organization's services together send faster than its byte rate. SDKs retry after retry-after. Sustained, it means an SDK loop or a plan that is too small.
429This organization reached its ingest hard cap for the billing period; ingestion resumes at <time>Usage passed five times the plan's allowance. Ingestion resumes when the period ends or the plan changes.
429Ingest is at capacity, retry laterThe ingest server is briefly full. Retry after retry-after.
408The request body was not received within 30 sThe connection is too slow or stalled mid-upload.
413Envelopes are limited to 100 itemsA Sentry envelope holds more than 100 items, counting the ones Unhandled does not store.
403This organization is suspended, so its telemetry is not accepted. Contact support to restore it.Staff suspended the organization.
400Body is not valid gzipThe body claimed gzip but was not.
503Telemetry storage is unavailable, retry laterA temporary storage failure. The response carries retry-after. Sentry endpoints answer 429 instead, because Sentry SDKs drop events on a 5xx.
400resourceSpans[0] is not an object (or another path)A list in the OTLP JSON holds null or a plain value where an object belongs.

A revoked key is refused on the next request, and so is a key whose organization was suspended. After Rotate key, the old key keeps working for the grace period you picked in the rotate dialog. A new key works right away.

If export succeeds but some records are missing, Unhandled accepted the batch and rejected individual items. OTLP reports those in the partialSuccess part of the response.

Data is in the wrong environment

Each page shows one environment at a time, and pages open on Production. Switch the Env selector in the top bar before you assume data is missing.

The agent picks the environment in this order:

  1. the environment option, which the generated config reads from APP_ENV,
  2. NODE_ENV, when it is production, staging or development,
  3. development.

The configure hook writes APP_ENV=development to your local .env. If that file ships with the app, or the deployment never overrides it, production traffic lands under Development. Set APP_ENV per deployment. See environments.

Data is under the wrong service

With a DSN, data always goes to the DSN's service, under its telemetry name. serviceName, OTEL_SERVICE_NAME or APP_NAME do not change that. If two apps show up as one service, they share a DSN. Create a service per deployable and give each app its own DSN. See services.

Requests or commands are missing

Some traffic is never traced, on purpose.

  • Health and well-known URLs. @unhandled/node ignores /health, /healthz, /internal/healthz, /ready, /readiness, /metrics, /internal/metrics, /favicon.ico, /robots.txt, /sitemap.xml, /manifest.json, /site.webmanifest, /browserconfig.xml and /ads.txt. Your http.ignoredUrls globs add to that list.
  • Static files. Requests whose last path segment ends in an asset extension such as .js, .css, .png or .woff2.
  • CORS preflights. OPTIONS requests, while ignoreOptionsRequests is true (the default).
  • Ace commands. make:*, generate:*, queue:work and queue:listen are excluded by default. Long-running commands are never traced.

Change these in configuration.

Traces are partial or missing

  • Head sampling. samplingRatio below 1 drops that share of traces before they start.
  • Job sampling. A sampling callback can drop a job's spans entirely, or keep them only on error. When a failed job kept by keepErrors({ maxSpans }) passes maxSpans, only its root span is exported, tagged unhandled.sampling.truncated=true and unhandled.sampling.dropped_spans. See job sampling.
  • Queries with no parent. A query that runs outside a request, job or command starts its own trace. See background spans.

Still stuck

Collect the agent version from your lockfile, your config/unhandled.ts with the DSN removed, and the console output with debug: true, then reach out through the channels on Support.