Skip to content

WIP: Set up OpenTelemetry instrumentation#4789

Draft
robinjam wants to merge 6 commits intomainfrom
jar-otel
Draft

WIP: Set up OpenTelemetry instrumentation#4789
robinjam wants to merge 6 commits intomainfrom
jar-otel

Conversation

@robinjam
Copy link
Copy Markdown
Contributor

@robinjam robinjam commented Mar 23, 2026

https://trello.com/c/25A3gpXh/1358-replace-statsd-instrumentation-across-notify-with-otel

Adds the following metrics, instrumented using OpenTelemetry (optional attributes are denoted like [this]):

  • notification.send.duration{[error.type], key.type, notification.type, [notification.sms.country_code], [notification.sms.international], provider.name}: The elapsed time between when a notification was created and when it was sent to a provider.
  • notification.deliver.duration{key.type, notification.status, notification.type, [notification.sms.country_code], [notification.sms.international], provider.name}: Elapsed time between sending a notification to a provider and receiving a callback from that provider.
  • provider.request.duration{[error.type], notification.type, [notification.sms.country_code], [notification.sms.international], provider.name}: The total time taken to make a request to a provider.

When these are pushed to Prometheus by the OTel sidecar, their names and the names of their attributes will automatically be converted to match Prometheus naming conventions, e.g. notification.send.duration{notification.type=...} becomes notification_send_duration_seconds_bucket{notification_type=...}.

These metrics are ultimately intended to replace all of the metrics we currently collect via StatsD (though this will require updating a bunch of dashboards).

@robinjam robinjam force-pushed the jar-otel branch 24 times, most recently from 4fadbf2 to c0b9237 Compare March 30, 2026 17:05
@robinjam robinjam force-pushed the jar-otel branch 6 times, most recently from 24c5c05 to 30b9b15 Compare March 31, 2026 12:28
@robinjam robinjam force-pushed the jar-otel branch 11 times, most recently from 7fc3523 to 7f548fc Compare March 31, 2026 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant