Skip to content

fix(api): install import-in-the-middle hook for ESM auto-instrumentation#11

Merged
alextnetto merged 1 commit into
mainfrom
fix/api-otel-esm-loader-hook
May 11, 2026
Merged

fix(api): install import-in-the-middle hook for ESM auto-instrumentation#11
alextnetto merged 1 commit into
mainfrom
fix/api-otel-esm-loader-hook

Conversation

@alextnetto
Copy link
Copy Markdown
Member

What's broken

After deploying #5, metrics and logs flowed to Grafana fine, but Tempo received zero spans:

  • otelcol_receiver_accepted_spans_total was empty
  • The x-trace-id response header was absent on every API response
  • Pino log records had service_name=pagent-api but no trace_id/span_id correlation

Root cause

@opentelemetry/auto-instrumentations-node patches modules at import time. With Node 22 + ESM + --experimental-strip-types, --import ./tracing.ts evaluates our SDK config before server.ts, but the import-in-the-middle loader hook has to be installed via Node's module hook registry — registering it from inside an already-loaded module is too late for built-ins like node:http. So getNodeAutoInstrumentations loaded but silently never wrapped anything.

The OTel team's standard solution is node --import @opentelemetry/auto-instrumentations-node/register, but that bootstraps its own NodeSDK (configured via env vars), which would conflict with our custom-configured SDK in tracing.ts (we wire metric reader + log processor + pino correlation).

Fix

A four-line apps/api/instrument.mjs that registers the loader hook only — leaves SDK setup to tracing.ts:

import { register } from 'node:module';
register('import-in-the-middle/hook.mjs', import.meta.url);

Wired in via a second --import flag (instrument.mjs runs before tracing.ts):

node --experimental-strip-types --import ./instrument.mjs --import ./tracing.ts server.ts

Verification

  • npm test (162 tests pass)
  • npm run typecheck (clean)
  • After merge + Railway redeploy: confirm otelcol_receiver_accepted_spans_total > 0 in Grafana, and x-trace-id header is present on curl -i https://pagent.up.railway.app/health.

OTel's auto-instrumentation patches modules at import/require time. With
Node 22 + ESM + --experimental-strip-types, `--import ./tracing.ts` runs
our SDK config before server.ts, but the import-in-the-middle loader
hook needs to be installed via Node's module hook registry — registering
it from inside an already-loaded module is too late for built-ins like
node:http.

Result of the gap: HTTP/postgres auto-instrumentation loaded but never
wrapped anything in prod, so Tempo received zero spans (even though
metrics + logs flowed fine). Trace IDs were missing from response
headers and from pino log records.

Fix: add apps/api/instrument.mjs that calls `register('import-in-the-middle/hook.mjs')`
and load it via a second --import flag before tracing.ts.
@vercel
Copy link
Copy Markdown

vercel Bot commented May 11, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
pagent Ready Ready Preview, Comment May 11, 2026 0:35am

Request Review

@alextnetto alextnetto merged commit b7281cc into main May 11, 2026
3 checks passed
@alextnetto alextnetto deleted the fix/api-otel-esm-loader-hook branch May 11, 2026 00:36
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: efe50c5b28

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/api/instrument.mjs
// Usage: node --import ./instrument.mjs --import ./tracing.ts server.ts
import { register } from 'node:module';

register('import-in-the-middle/hook.mjs', import.meta.url);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Declare import-in-the-middle as a direct dependency

instrument.mjs now imports import-in-the-middle/hook.mjs, but apps/api/package.json does not declare import-in-the-middle directly. This currently works only when the package manager hoists that transitive dependency; in non-hoisted or isolated installs (common in CI/deploy/container builds), startup will fail with ERR_MODULE_NOT_FOUND before the API boots. Please add import-in-the-middle to dependencies so the new boot path is deterministic across install environments.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant