Metrics

Metrics lists every OpenTelemetry metric the selected environment received, from runtime gauges to the counters and histograms you record yourself. Use it to check a number that no built-in dashboard shows, or to find the right metric before you put it on a custom dashboard.

The catalog lives at /o/:org/projects/:project/metrics and one metric at /o/:org/projects/:project/metrics/:name.

Where the data comes from

Metrics arrive over OTLP from any SDK. The AdonisJS agent sends runtime and host metrics by default. You add your own with the OpenTelemetry metrics API, which the agent exports with everything else:

import { metrics } from '@opentelemetry/api'

const meter = metrics.getMeter('shop')
const checkoutDuration = meter.createHistogram('shop.checkout.duration', {
  unit: 'ms',
  description: 'Time to place an order',
})
const ordersPlaced = meter.createCounter('shop.orders.placed')

export const recordCheckout = (durationMs: number, paymentMethod: string) => {
  checkoutDuration.record(durationMs, { 'payment.method': paymentMethod })
  ordersPlaced.add(1, { 'payment.method': paymentMethod })
}

Gauges, sums, histograms and summaries are stored. Exponential histograms are converted to explicit buckets on one shared grid, about 9% wide, so points from different SDKs and scales line up. A point flagged as having no recorded value is dropped, and a point whose value is NaN or infinite is rejected through partial success instead of being stored as zero.

Each series is one OpenTelemetry stream: the service, its resource attributes, the instrumentation scope and the data-point attributes. Two processes on one host stay apart as long as service.instance.id or another resource attribute differs. The agent sets service.instance.id to the HOSTNAME environment variable plus the process id, for example web-1-4821, or to a random ID when HOSTNAME is not set.

A metric keeps at most 2000 series per hour. Past that, a data point of a new series loses its attributes and instrumentation scope and joins its writer's overflow series, tagged otel.metric.overflow=true. Each writer, meaning each set of resource attributes such as service.instance.id or host.name, gets its own overflow series. That keeps every counter and histogram a single process's running total, so increases stay exact. A metric that has overflowed admits no new series until a whole clock hour passes without overflow, so a series never moves between its own series and the overflow series. A metric holds at most 500 overflow series. Points from further writers are dropped, and the count is reported in your own metrics as unhandled.metrics.dropped_points, with the dropped metric's name in the metric.name attribute. The catalog marks an overflowed metric with a warning icon, and its charts say so under the chart.

The catalog

Metrics are grouped by namespace, the part of the name before the first dot, so shop.checkout.duration and shop.orders.placed sit together under shop.

The list covers metrics seen in the last 90 days, while the trend follows the selected range. Search matches name or description, and the type filter offers All, Gauge, Sum, Histogram and Summary. Both are kept in the URL as q and type.

A warning icon marks a metric that services report with different types or units. Those cannot share an axis, so its chart stays blocked until you select a single service in the top bar.

Exploring a metric

The metric page charts the series over the selected range, with Latest, Average, Min and Max for the visible series above the chart.

Aggregation depends on the type:

Delta and cumulative data read the same. A cumulative point contributes its change since the previous point of the same series, looking back up to 15 minutes before the range, so a counter exported once a minute still shows its increase in one-minute buckets. A drop, or a start time later than the previous point, is a restart, and the new value counts in full. A series that started inside the range counts from zero, as Prometheus does. Unlike Prometheus, increases are not extrapolated to the bucket edges: each increase lands in the bucket of the point that reported it.

For a counter, Sum is the increase within each bucket and Rate divides it by the bucket length in seconds. For an up-down counter, Sum adds the level of each series. Cumulative histograms and summaries are turned into per-bucket counts the same way, so P95 describes the bucket, not the whole process lifetime.

Histogram percentiles merge the bucket counts of each time bucket and interpolate linearly inside the bucket that holds the rank, so their precision depends on the bucket boundaries your SDK uses. When the data points carry min and max, the percentile is clamped to them; a cumulative histogram's min and max cover the process lifetime, so its Min and Max stay empty. Histograms with different bucket boundaries in one bucket are re-bucketed onto the union of their boundaries, and a note says so. Summary percentiles show the quantiles the SDK reported; across several series the highest one is shown, and a quantile the SDK did not report stays empty.

Group splits the chart by a data-point or resource attribute. The 10 series with the largest values are drawn, and a table under the chart lists them with a trend, Latest, Average, Min and Max. Click a row to hide a series, double-click to show only that one.

Filters narrow to attribute values. Pick an attribute, toggle between = and !=, and type a value or choose one of the most frequent values. Click the operator on a filter chip to flip it.

Chart type switches between Line, Area and Bars. Drag across the chart to zoom the time range.

Add to dashboard saves the chart as a time series widget with this metric, aggregation, grouping and filters, on an existing custom dashboard or a new one. It is disabled while the chart is blocked.

Aggregation, group, filters and chart type live in the URL as agg, group, where and chart, so a copied link reopens the same view. where uses the search syntax, limited to = and !=:

payment.method=card service.name!=worker

Units and gaps

Values are formatted from the unit: ms, s and us read as durations, By as bytes, and 1 as a percentage when the name contains utilization, ratio, percent or fraction. Count and Rate always show plain numbers.

A bucket with no data is a gap for gauges, histograms and summaries and zero for sums, because a counter that did not move added nothing.

Answering "which payment method is slowest?"

1. Open shop.checkout.duration from the shop namespace.
2. Pick P95 and group by payment.method.
3. Read the series table for the method with the highest Average, then double-click it to isolate its line.
4. Click Add to dashboard to watch the chart on a custom dashboard after each deploy.