DI: coordinated sampling for Live Debugger snapshots (runtime_id, trace_id_source)

[p-datadog]

What does this PR do?

Adds coordinated sampling for Live Debugger snapshots, plus two snapshot-envelope
fields, on the DI producer side.

• Coordinated sampling (Datadog::DI::ExecutionUnit + Datadog::DI::Correlation):
  one emit/drop decision per execution unit — the active APM trace — shared
  across every probe that fires within that unit. Within an emitting unit, a
  per-probe-per-span cap bounds each probe to one snapshot so a probe inside a
  high-iteration loop cannot starve sibling probes. When no trace is active the
  hit is not correlated (an independent per-probe decision). The gate runs at
  the top of the probe-hit path, before any capture work, so a dropped probe
  costs one lookup.
• runtimeId in the snapshot envelope: distinguishes snapshots from before and
  after a restart inside the same container (host/container tags cannot). Same
  value already sent in probe status diagnostics.
• trace_id_source ("apm" | "none"): tells a consumer when the trace id in
  the envelope is a valid join key to APM.

The unit is resolved from existing tracer context only (the active trace/span);
no new context mechanism is introduced. The sampling decision reuses the probe's
existing rate limiter (the first probe in a unit decides, siblings inherit), so
this introduces no new sampling-rate contract on the wire or in configuration.
The gate fails open to the previous per-probe rate-limiting behavior if
correlation is unavailable or errors, so a correlation bug cannot silence all
snapshots.

Motivation:

Independent per-probe sampling fragments chains of related snapshots: the system
can emit one probe's snapshot while dropping a sibling, parent, or child probe
that would explain it, with no signal to the user. This implements the
producer-side track of the "Improving Correlation for Live Debugger Snapshots"
RFC for Ruby.

Change log entry

Yes. Live Debugger snapshots from probes in the same trace now share one sampling
decision so related snapshots arrive together instead of fragmenting, and each
snapshot carries the process runtime id.

Additional Notes:

Correlation uses existing tracer context only; there is deliberately no new
context mechanism, so probes firing outside an active trace are not correlated.

Deferred (not in this PR): emitting a non-null thread_id and the associated
generation token (Ruby emits thread_id: null today); and a global per-session
events/sec ceiling.

How to test the change?

• spec/datadog/di/execution_unit_spec.rb — unit resolution from the active
  trace/span, and the no-trace (:none) case.
• spec/datadog/di/correlation_spec.rb — the sampler: coordinated decision,
  per-span cap, LRU eviction.
• spec/datadog/di/integration/correlation_integration_spec.rb — end-to-end
  through real instrumentation: tier-1 coordination, per-span cap, no-trace
  independent decision, runtime id on the wire, and fail-open.
• spec/datadog/di/probe_notification_builder_spec.rb — envelope fields.

Full rake spec:di:di_with_ext is green (1069 examples). Standard and Steep are
clean.

[dd-octo-sts]

Typing analysis

Note: Ignored files are excluded from the next sections.

steep:ignore comments

This PR introduces 11 steep:ignore comments, and clears 11 steep:ignore comments.

steep:ignore comments (+11-11)

❌ Introduced:

lib/datadog/di/instrumenter.rb:199
lib/datadog/di/instrumenter.rb:204
lib/datadog/di/instrumenter.rb:208
lib/datadog/di/instrumenter.rb:213
lib/datadog/di/instrumenter.rb:217
lib/datadog/di/instrumenter.rb:236
lib/datadog/di/instrumenter.rb:297
lib/datadog/di/instrumenter.rb:299
lib/datadog/di/instrumenter.rb:925
lib/datadog/di/probe_notification_builder.rb:181
lib/datadog/di/probe_notification_builder.rb:425

✅ Cleared:

lib/datadog/di/instrumenter.rb:196
lib/datadog/di/instrumenter.rb:201
lib/datadog/di/instrumenter.rb:205
lib/datadog/di/instrumenter.rb:210
lib/datadog/di/instrumenter.rb:214
lib/datadog/di/instrumenter.rb:233
lib/datadog/di/instrumenter.rb:294
lib/datadog/di/instrumenter.rb:296
lib/datadog/di/instrumenter.rb:903
lib/datadog/di/probe_notification_builder.rb:180
lib/datadog/di/probe_notification_builder.rb:416

Untyped methods

This PR introduces 10 partially typed methods, and clears 10 partially typed methods. It increases the percentage of typed methods from 69.55% to 69.67% (+0.12%).

Partially typed methods (+10-10)

❌ Introduced:

sig/datadog/di/component.rbs:74
└── def parse_probe_spec_and_notify: (Hash[String, untyped] probe_spec) -> Probe
sig/datadog/di/instrumenter.rbs:48
└── def hook_method: (Probe probe, untyped responder) -> void
sig/datadog/di/instrumenter.rbs:51
└── def hook_line: (Probe probe, untyped responder) -> bool?
sig/datadog/di/instrumenter.rbs:55
└── def hook: (Probe probe, untyped responder) -> void
sig/datadog/di/instrumenter.rbs:59
└── def self.get_local_variables: (TracePoint trace_point) -> Hash[Symbol, untyped]
sig/datadog/di/instrumenter.rbs:60
└── def self.get_instance_variables: (Object self) -> Hash[Symbol, untyped]
sig/datadog/di/instrumenter.rbs:67
└── def run_method_probe: (::Array[untyped] args, ::Hash[::Symbol, untyped] kwargs, ::Proc? target_block, Object target_self, Probe probe, untyped responder, [::String, ::Integer]? loc, ::String method_name) { () -> untyped } -> untyped
sig/datadog/di/instrumenter.rbs:71
└── def kwargs_from_splat: (::Array[untyped] args) -> [::Array[untyped], ::Hash[untyped, untyped]]
sig/datadog/di/instrumenter.rbs:79
└── def line_trace_point_callback: (Probe probe, RubyVM::InstructionSequence? iseq, untyped responder, TracePoint tp) -> void
sig/datadog/di/instrumenter.rbs:83
└── def check_and_disable_if_exceeded: (Probe probe, untyped responder, Float di_start_time, ?Float accumulated_duration) -> void

✅ Cleared:

sig/datadog/di/component.rbs:70
└── def parse_probe_spec_and_notify: (Hash[String, untyped] probe_spec) -> Probe
sig/datadog/di/instrumenter.rbs:44
└── def hook_method: (Probe probe, untyped responder) -> void
sig/datadog/di/instrumenter.rbs:47
└── def hook_line: (Probe probe, untyped responder) -> bool?
sig/datadog/di/instrumenter.rbs:51
└── def hook: (Probe probe, untyped responder) -> void
sig/datadog/di/instrumenter.rbs:55
└── def self.get_local_variables: (TracePoint trace_point) -> Hash[Symbol, untyped]
sig/datadog/di/instrumenter.rbs:56
└── def self.get_instance_variables: (Object self) -> Hash[Symbol, untyped]
sig/datadog/di/instrumenter.rbs:63
└── def run_method_probe: (::Array[untyped] args, ::Hash[::Symbol, untyped] kwargs, ::Proc? target_block, Object target_self, Probe probe, untyped responder, [::String, ::Integer]? loc, ::String method_name) { () -> untyped } -> untyped
sig/datadog/di/instrumenter.rbs:67
└── def kwargs_from_splat: (::Array[untyped] args) -> [::Array[untyped], ::Hash[untyped, untyped]]
sig/datadog/di/instrumenter.rbs:73
└── def line_trace_point_callback: (Probe probe, RubyVM::InstructionSequence? iseq, untyped responder, TracePoint tp) -> void
sig/datadog/di/instrumenter.rbs:77
└── def check_and_disable_if_exceeded: (Probe probe, untyped responder, Float di_start_time, ?Float accumulated_duration) -> void

If you believe a method or an attribute is rightfully untyped or partially typed, you can add # untyped:accept on the line before the definition to remove it from the stats.

[datadog-prod-us1-6]

 

⚠️ Warnings

🚦 9 Pipeline jobs failed

Unit Tests | Ruby 2.6 / build & test (standard) [0]     

Unit Tests | Ruby 2.7 / build & test (standard) [0]     

Unit Tests | Ruby 3.0 / build & test (standard) [0]     

View all 9 failed jobs.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: af741e4 | Docs | Datadog PR Page | Give us feedback!}

[pr-commenter]

Benchmarks

Benchmark execution time: 2026-07-29 18:04:12

Comparing candidate commit af741e4 in PR branch casual-correlation with baseline commit e1e1961 in branch master.

📊 Benchmarking dashboard

Found 1 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 1 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

• 🟩 = significantly better candidate vs. baseline
• 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:profiling - Allocations ()

• 🟩 throughput [+330937.525op/s; +373985.104op/s] or [+11.363%; +12.841%]

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:tracing - trace.to_digest - Continue

• unstable throughput [-1884.734op/s; +1168.658op/s] or [-6.440%; +3.993%]