Skip to content

feat(data-pipeline): add meta struct blob setter - #2303

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 7 commits into
mainfrom
lloeki/native-meta-struct-m1
Aug 4, 2026
Merged

feat(data-pipeline): add meta struct blob setter#2303
gh-worker-dd-mergequeue-cf854d[bot] merged 7 commits into
mainfrom
lloeki/native-meta-struct-m1

Conversation

@lloeki

@lloeki lloeki commented Jul 30, 2026

Copy link
Copy Markdown
Member

AI-generated code disclosure: this PR was implemented with substantial AI assistance and manually reviewed and validated.

What does this PR do?

Adds an opaque per-key meta_struct blob setter to TracerSpan. Keys are validated as UTF-8, while values are copied as uninterpreted binary data. This transitional API lets consumers pass their existing per-key MessagePack blobs without changing the Rust trace model.

Motivation

Native exporter consumers cannot currently populate meta_struct, which blocks payload parity tracked by APMSP-3219. The opaque setter provides the safe first stage before the language-neutral structured-value APIs.

Additional Notes

Stack position: L1, current PR: DataDog/libdatadog PR 2303.

These open L0-L5 changes are not included in libdatadog v38.0.0. They are planned to ship together in a single libdatadog v39 release, with the common dd-trace-rb uptake tracked by R3 DataDog/dd-trace-rb PR 6143.

Complete expected libdatadog review and merge order for that single v39 release:

Level libdatadog PR Matching dd-trace-rb consumer
L0 DataDog/libdatadog PR 2302 Release prerequisite; no payload API consumer
L1 (current) DataDog/libdatadog PR 2303 R4 DataDog/dd-trace-rb PR 6130
L2 DataDog/libdatadog PR 2305 R5 DataDog/dd-trace-rb PR 6129
L3 DataDog/libdatadog PR 2301 R6 DataDog/dd-trace-rb PR 6134
L4 DataDog/libdatadog PR 2304 R7 DataDog/dd-trace-rb PR 6132
L5 DataDog/libdatadog PR 2300 R8 DataDog/dd-trace-rb PR 6133

L0-L3 are independent and can be reviewed in parallel. Merging them in the listed order is still recommended because it reduces integration and release-branch conflicts, and lands panic containment before the new payload surfaces. L4 is stacked on L1, and L5 is stacked on L4.

The matching payload consumers are R4 DataDog/dd-trace-rb PR 6130, R5 DataDog/dd-trace-rb PR 6129, R6 DataDog/dd-trace-rb PR 6134, R7 DataDog/dd-trace-rb PR 6132, and R8 DataDog/dd-trace-rb PR 6133. All depend through the common v39 uptake in R3 DataDog/dd-trace-rb PR 6143. The downstream review and merge order after R3 is R4, R5, R6, R7, then R8; R7 also depends on R4, and R8 depends on R7.

DataDog/libdatadog-rb v37 PR 65 and DataDog/libdatadog-rb v38 PR 66 are separate ordered prerequisite releases only for R2 DataDog/dd-trace-rb PR 6131 and its ForkSafeRuntime uptake. They do not contain the open L0-L5 changes or payload APIs. R3 is stacked on R2, but R3, rather than either prerequisite package, is the common v39 payload-API uptake.

L0 tracks APMSP-3830; L1-L5 implement the payload work tracked by APMSP-3219.

How to test the change?

Prior local validation completed successfully:

  • nix develop -c cargo test -p libdd-data-pipeline-ffi passed, including arbitrary binary values, overwrite semantics, null handles, and invalid UTF-8 keys.
  • nix develop -c cargo clippy -p libdd-data-pipeline-ffi --all-targets -- -D warnings passed.
  • nix develop .#nightly -c cargo fmt --all -- --check passed.
  • The generated C header was rebuilt through nix develop -c cargo run --bin release -- --out /tmp/opencode/native-meta-struct-libdatadog/x86_64-linux and consumed successfully by the matching dd-trace-rb branch.
  • nix develop .#ruby34 --command bundle exec rspec spec/datadog/tracing/transport/native/tracer_span_spec.rb spec/datadog/tracing/transport/native/conformance_spec.rb spec/datadog/tracing/transport/native/transport_spec.rb spec/datadog/appsec/actions_handler/serializable_backtrace_spec.rb passed 116 examples, including AppSec and AI Guard-shaped wire conformance.
  • Targeted Standard and Steep checks for the native transport passed.

Expose a C setter that copies an opaque binary value into a span's
`meta_struct` map. Native tracers can now supply pre-encoded
MessagePack without transferring ownership or requiring Rust to
interpret it.

Reject null span handles and non-UTF-8 keys. Reusing a key replaces its
value, and the copied bytes remain owned by the span after the call.
@lloeki
lloeki requested a review from a team as a code owner July 30, 2026 11:02
@lloeki lloeki added the AI Generated PR largely written by AI tools label Jul 30, 2026
@pr-commenter

pr-commenter Bot commented Jul 30, 2026

Copy link
Copy Markdown

Benchmarks

Comparison

Candidate

Candidate benchmark details

Baseline

Baseline benchmark details

@datadog-prod-us1-6

datadog-prod-us1-6 Bot commented Jul 30, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 92.16%
Overall Coverage: 74.83% (+0.02%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 220d6f8 | Docs | Datadog PR Page | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 87.32 MB 87.33 MB +.01% (+11.89 KB) 🔍
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.01 MB 8.01 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.78 MB 10.78 MB +0% (+336 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 98.55 MB 98.56 MB +.01% (+12.62 KB) 🔍
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 26.05 MB 26.05 MB +.01% (+3.50 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 89.60 KB 89.92 KB +.35% (+324 B) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 188.07 MB 188.08 MB +0% (+16.00 KB) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 980.68 MB 980.71 MB +0% (+31.04 KB) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 8.49 MB 8.49 MB +.02% (+2.50 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 89.60 KB 89.92 KB +.35% (+324 B) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 25.10 MB 25.10 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 49.92 MB 49.93 MB +.02% (+10.68 KB) 🔍
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 22.69 MB 22.69 MB +.01% (+2.50 KB) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 91.00 KB 91.33 KB +.35% (+330 B) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 192.83 MB 192.85 MB +.01% (+24.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 969.32 MB 969.35 MB +0% (+30.15 KB) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.56 MB 6.56 MB +.02% (+2.00 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 91.00 KB 91.33 KB +.35% (+330 B) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 26.97 MB 26.98 MB +.02% (+8.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 47.53 MB 47.54 MB +.02% (+9.75 KB) 🔍
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 77.87 MB 77.88 MB +.01% (+11.60 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.93 MB 8.93 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 93.36 MB 93.37 MB +.01% (+12.02 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.87 MB 10.87 MB +0% (+248 B) 👌

@lloeki

lloeki commented Jul 30, 2026

Copy link
Copy Markdown
Member Author

Draft dd-trace-rb example consumer: DataDog/dd-trace-rb#6130.

Comment thread libdd-data-pipeline-ffi/src/tracer.rs Outdated
};
span.0
.meta_struct
.insert(key, Bytes::copy_from_slice(value.as_bytes()));

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

value.as_bytes() can panic when there is invalid input (example: null pointer, non-zero length). You can use try_as_bytes instead to get a more meaningful error.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made the change on 10bfcb8

Comment thread libdd-data-pipeline-ffi/src/tracer.rs Outdated
/// The `key` and opaque binary `value` are copied into the span. The value is
/// not interpreted or validated as MessagePack.
///
/// Repeating a `key` appends a new entry rather than replacing the previous one. The last value

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This exposes an internal implementation detail, so maybe shouldn't be here? I figure it's worth it since it could have performance implications for meta_struct

@lloeki lloeki Aug 3, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your comment was confusing to me until I read the following line:

/// written for a key is the one read back and the one serialized

And with that, agreed: it just appears to behave as a regular setter to the API consumer.

@ekump ekump left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but someone else should review since I also pushed commits

@hoolioh hoolioh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, a little too much verbosity regarding comments though.

gh-worker-dd-mergequeue-cf854d Bot pushed a commit that referenced this pull request Aug 4, 2026
AI-generated code disclosure: this PR was implemented with substantial AI assistance and manually reviewed and validated.

# What does this PR do?

Ensures the combined libdatadog profiling artifact enables `catch_panic` for data-pipeline FFI entry points. It also adds a packaged C regression that triggers capacity overflow and verifies that the FFI returns an error instead of aborting the process.

# Motivation

The standalone data-pipeline FFI crate enables panic containment by default, but the combined profiling artifact disables dependency defaults and did not re-enable it. This closes [APMSP-3830](https://datadoghq.atlassian.net/browse/APMSP-3830) before the native trace exporter payload APIs ship together.

# Additional Notes

**Stack position: L0, current PR: [DataDog/libdatadog PR 2302](#2302

These open L0-L5 changes are not included in [libdatadog v38.0.0](https://github.com/DataDog/libdatadog/releases/tag/v38.0.0). They are planned to ship together in a single libdatadog v39 release, with the common dd-trace-rb uptake tracked by [R3 DataDog/dd-trace-rb PR 6143](DataDog/dd-trace-rb#6143).

Complete expected libdatadog review and merge order for that single v39 release:

| Level | libdatadog PR | Matching dd-trace-rb consumer |
| --- | --- | --- |
| **L0 (current)** | **[DataDog/libdatadog PR 2302](#2302 | Release prerequisite; no payload API consumer |
| L1 | [DataDog/libdatadog PR 2303](#2303) | [R4 DataDog/dd-trace-rb PR 6130](DataDog/dd-trace-rb#6130) |
| L2 | [DataDog/libdatadog PR 2305](#2305) | [R5 DataDog/dd-trace-rb PR 6129](DataDog/dd-trace-rb#6129) |
| L3 | [DataDog/libdatadog PR 2301](#2301) | [R6 DataDog/dd-trace-rb PR 6134](DataDog/dd-trace-rb#6134) |
| L4 | [DataDog/libdatadog PR 2304](#2304) | [R7 DataDog/dd-trace-rb PR 6132](DataDog/dd-trace-rb#6132) |
| L5 | [DataDog/libdatadog PR 2300](#2300) | [R8 DataDog/dd-trace-rb PR 6133](DataDog/dd-trace-rb#6133) |

L0-L3 are independent and can be reviewed in parallel. Merging them in the listed order is still recommended because it reduces integration and release-branch conflicts, and lands panic containment before the new payload surfaces. L4 is stacked on L1, and L5 is stacked on L4.

The matching payload consumers are [R4 DataDog/dd-trace-rb PR 6130](DataDog/dd-trace-rb#6130), [R5 DataDog/dd-trace-rb PR 6129](DataDog/dd-trace-rb#6129), [R6 DataDog/dd-trace-rb PR 6134](DataDog/dd-trace-rb#6134), [R7 DataDog/dd-trace-rb PR 6132](DataDog/dd-trace-rb#6132), and [R8 DataDog/dd-trace-rb PR 6133](DataDog/dd-trace-rb#6133). All depend through the common v39 uptake in [R3 DataDog/dd-trace-rb PR 6143](DataDog/dd-trace-rb#6143). The downstream review and merge order after R3 is R4, R5, R6, R7, then R8; R7 also depends on R4, and R8 depends on R7.

[DataDog/libdatadog-rb v37 PR 65](DataDog/libdatadog-rb#65) and [DataDog/libdatadog-rb v38 PR 66](DataDog/libdatadog-rb#66) are separate ordered prerequisite releases only for [R2 DataDog/dd-trace-rb PR 6131](DataDog/dd-trace-rb#6131) and its `ForkSafeRuntime` uptake. They do not contain the open L0-L5 changes or payload APIs. R3 is stacked on R2, but R3, rather than either prerequisite package, is the common v39 payload-API uptake.

L0 tracks [APMSP-3830](https://datadoghq.atlassian.net/browse/APMSP-3830); L1-L5 implement the payload work tracked by [APMSP-3219](https://datadoghq.atlassian.net/browse/APMSP-3219).

# How to test the change?

Prior local validation completed successfully:

- `nix develop -c cargo test -p libdd-profiling-ffi --features data-pipeline-ffi` passed the combined profiling/data-pipeline FFI suite: 57 tests.
- `nix develop -c cargo test -p libdd-data-pipeline-ffi --features catch_panic trace_chunks_new_with_overflow_capacity_returns_panic_error` passed.
- `nix develop -c cargo test -p libdd-data-pipeline-ffi --features catch_panic begin_chunk_with_overflow_capacity_returns_panic_error` passed.
- `nix develop -c cargo ffi-test --filter trace_exporter_panic_containment` built and passed the packaged C panic-containment regression.
- `nix develop .#nightly --command cargo fmt --all -- --check` passed.
- `nix develop -c cargo clippy -p libdd-profiling-ffi --all-targets --features data-pipeline-ffi -- -D warnings` passed.




[APMSP-3830]: https://datadoghq.atlassian.net/browse/APMSP-3830?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[APMSP-3830]: https://datadoghq.atlassian.net/browse/APMSP-3830?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ


Co-authored-by: ekump <edmund.kump@datadoghq.com>
Co-authored-by: hoolioh <julio.gonzalez@datadoghq.com>
@lloeki

lloeki commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Aug 4, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-08-04 11:03:53 UTC ℹ️ Start processing command /merge


2026-08-04 11:03:57 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 53m (p90).


2026-08-04 11:48:46 UTC ℹ️ MergeQueue: This merge request was merged

@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot merged commit 38a1987 into main Aug 4, 2026
91 checks passed
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot deleted the lloeki/native-meta-struct-m1 branch August 4, 2026 11:48
hoolioh added a commit that referenced this pull request Aug 5, 2026
AI-generated code disclosure: this PR was implemented with substantial
AI assistance and manually reviewed and validated.

# What does this PR do?

Adds owned span-event construction and atomic attachment through the
data-pipeline FFI. It supports typed scalar attributes, homogeneous
typed arrays, empty arrays, event ordering, and UTF-8 validation.

# Motivation

The Rust trace model supports span events, but native exporter consumers
cannot populate them. This fills the span-event portion of the payload
parity work tracked by
[APMSP-3219](https://datadoghq.atlassian.net/browse/APMSP-3219).

# Additional Notes

**Stack position: L3, current PR: [DataDog/libdatadog PR
2301](#2301

These open L0-L5 changes are not included in [libdatadog
v38.0.0](https://github.com/DataDog/libdatadog/releases/tag/v38.0.0).
They are planned to ship together in a single libdatadog v39 release,
with the common dd-trace-rb uptake tracked by [R3 DataDog/dd-trace-rb PR
6143](DataDog/dd-trace-rb#6143).

Complete expected libdatadog review and merge order for that single v39
release:

| Level | libdatadog PR | Matching dd-trace-rb consumer |
| --- | --- | --- |
| L0 | [DataDog/libdatadog PR
2302](#2302) | Release
prerequisite; no payload API consumer |
| L1 | [DataDog/libdatadog PR
2303](#2303) | [R4
DataDog/dd-trace-rb PR
6130](DataDog/dd-trace-rb#6130) |
| L2 | [DataDog/libdatadog PR
2305](#2305) | [R5
DataDog/dd-trace-rb PR
6129](DataDog/dd-trace-rb#6129) |
| **L3 (current)** | **[DataDog/libdatadog PR
2301](#2301 | **[R6
DataDog/dd-trace-rb PR
6134](DataDog/dd-trace-rb#6134 |
| L4 | [DataDog/libdatadog PR
2304](#2304) | [R7
DataDog/dd-trace-rb PR
6132](DataDog/dd-trace-rb#6132) |
| L5 | [DataDog/libdatadog PR
2300](#2300) | [R8
DataDog/dd-trace-rb PR
6133](DataDog/dd-trace-rb#6133) |

L0-L3 are independent and can be reviewed in parallel. Merging them in
the listed order is still recommended because it reduces integration and
release-branch conflicts, and lands panic containment before the new
payload surfaces. L4 is stacked on L1, and L5 is stacked on L4.

The matching payload consumers are [R4 DataDog/dd-trace-rb PR
6130](DataDog/dd-trace-rb#6130), [R5
DataDog/dd-trace-rb PR
6129](DataDog/dd-trace-rb#6129), [R6
DataDog/dd-trace-rb PR
6134](DataDog/dd-trace-rb#6134), [R7
DataDog/dd-trace-rb PR
6132](DataDog/dd-trace-rb#6132), and [R8
DataDog/dd-trace-rb PR
6133](DataDog/dd-trace-rb#6133). All depend
through the common v39 uptake in [R3 DataDog/dd-trace-rb PR
6143](DataDog/dd-trace-rb#6143). The downstream
review and merge order after R3 is R4, R5, R6, R7, then R8; R7 also
depends on R4, and R8 depends on R7.

[DataDog/libdatadog-rb v37 PR
65](DataDog/libdatadog-rb#65) and
[DataDog/libdatadog-rb v38 PR
66](DataDog/libdatadog-rb#66) are separate
ordered prerequisite releases only for [R2 DataDog/dd-trace-rb PR
6131](DataDog/dd-trace-rb#6131) and its
`ForkSafeRuntime` uptake. They do not contain the open L0-L5 changes or
payload APIs. R3 is stacked on R2, but R3, rather than either
prerequisite package, is the common v39 payload-API uptake.

L0 tracks
[APMSP-3830](https://datadoghq.atlassian.net/browse/APMSP-3830); L1-L5
implement the payload work tracked by
[APMSP-3219](https://datadoghq.atlassian.net/browse/APMSP-3219).

# How to test the change?

Prior local validation completed successfully:

- `nix develop --command cargo test -p libdd-data-pipeline-ffi` passed
58 tests, including scalar values, homogeneous typed arrays, empty
arrays, ordering, ownership, and atomic failure coverage.
- `nix develop --command cargo clippy -p libdd-data-pipeline-ffi
--all-targets -- -D warnings` passed.
- `nix develop .#nightly --command cargo fmt --all -- --check` passed.
- `nix develop --command env
LIBDATADOG_VENDOR_OVERRIDE=/tmp/opencode/native-span-events-ruby bundle
exec rspec spec/datadog/tracing/transport/native/tracer_span_spec.rb
spec/datadog/tracing/transport/native/send_traces_spec.rb
spec/datadog/tracing/transport/native/transport_spec.rb
spec/datadog/tracing/transport/traces_spec.rb
spec/datadog/tracing/span_event_spec.rb
spec/datadog/tracing/span_operation_spec.rb` passed 123 native and
producer examples, including native wire output and legacy JSON
fallback.
- Targeted Standard and Steep checks for the native transport and
span-event paths passed.




[APMSP-3219]:
https://datadoghq.atlassian.net/browse/APMSP-3219?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[APMSP-3830]:
https://datadoghq.atlassian.net/browse/APMSP-3830?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

---------

Co-authored-by: Julio <julio.gonzalez@datadoghq.com>
Co-authored-by: Edmund Kump <edmund.kump@datadoghq.com>
Co-authored-by: Julio Gonzalez <107922352+hoolioh@users.noreply.github.com>
hoolioh added a commit that referenced this pull request Aug 5, 2026
AI-generated code disclosure: this PR was implemented with substantial
AI assistance and manually reviewed and validated.

# What does this PR do?

Adds an atomic batched FFI for complete span links, including 128-bit
trace IDs, span IDs, flags, tracestate, attributes, and ordering. All
borrowed input is validated and copied before insertion.

# Motivation

The Rust trace model supports span links, but native exporter consumers
cannot populate it. This fills the span-link portion of the payload
parity work tracked by
[APMSP-3219](https://datadoghq.atlassian.net/browse/APMSP-3219).

# Additional Notes

`dropped_attributes_count` is intentionally deferred by additive commit
[961044e](961044e).
Neither the Agent v0.4 nor indexed V1 span-link schema defines the
field, and current Datadog tracers do not produce non-zero counts. The
original implementation remains in the preceding commit and can be
restored by reverting the deferral commit.

**Stack position: L2, current PR: [DataDog/libdatadog PR
2305](#2305

These open L0-L5 changes are not included in [libdatadog
v38.0.0](https://github.com/DataDog/libdatadog/releases/tag/v38.0.0).
They are planned to ship together in a single libdatadog v39 release,
with the common dd-trace-rb uptake tracked by [R3 DataDog/dd-trace-rb PR
6143](DataDog/dd-trace-rb#6143).

Complete expected libdatadog review and merge order for that single v39
release:

| Level | libdatadog PR | Matching dd-trace-rb consumer |
| --- | --- | --- |
| L0 | [DataDog/libdatadog PR
2302](#2302) | Release
prerequisite; no payload API consumer |
| L1 | [DataDog/libdatadog PR
2303](#2303) | [R4
DataDog/dd-trace-rb PR
6130](DataDog/dd-trace-rb#6130) |
| **L2 (current)** | **[DataDog/libdatadog PR
2305](#2305 | **[R5
DataDog/dd-trace-rb PR
6129](DataDog/dd-trace-rb#6129 |
| L3 | [DataDog/libdatadog PR
2301](#2301) | [R6
DataDog/dd-trace-rb PR
6134](DataDog/dd-trace-rb#6134) |
| L4 | [DataDog/libdatadog PR
2304](#2304) | [R7
DataDog/dd-trace-rb PR
6132](DataDog/dd-trace-rb#6132) |
| L5 | [DataDog/libdatadog PR
2300](#2300) | [R8
DataDog/dd-trace-rb PR
6133](DataDog/dd-trace-rb#6133) |

L0-L3 are independent and can be reviewed in parallel. Merging them in
the listed order is still recommended because it reduces integration and
release-branch conflicts, and lands panic containment before the new
payload surfaces. L4 is stacked on L1, and L5 is stacked on L4.

The matching payload consumers are [R4 DataDog/dd-trace-rb PR
6130](DataDog/dd-trace-rb#6130), [R5
DataDog/dd-trace-rb PR
6129](DataDog/dd-trace-rb#6129), [R6
DataDog/dd-trace-rb PR
6134](DataDog/dd-trace-rb#6134), [R7
DataDog/dd-trace-rb PR
6132](DataDog/dd-trace-rb#6132), and [R8
DataDog/dd-trace-rb PR
6133](DataDog/dd-trace-rb#6133). All depend
through the common v39 uptake in [R3 DataDog/dd-trace-rb PR
6143](DataDog/dd-trace-rb#6143). The downstream
review and merge order after R3 is R4, R5, R6, R7, then R8; R7 also
depends on R4, and R8 depends on R7.

[DataDog/libdatadog-rb v37 PR
65](DataDog/libdatadog-rb#65) and
[DataDog/libdatadog-rb v38 PR
66](DataDog/libdatadog-rb#66) are separate
ordered prerequisite releases only for [R2 DataDog/dd-trace-rb PR
6131](DataDog/dd-trace-rb#6131) and its
`ForkSafeRuntime` uptake. They do not contain the open L0-L5 changes or
payload APIs. R3 is stacked on R2, but R3, rather than either
prerequisite package, is the common v39 payload-API uptake.

L0 tracks
[APMSP-3830](https://datadoghq.atlassian.net/browse/APMSP-3830); L1-L5
implement the payload work tracked by
[APMSP-3219](https://datadoghq.atlassian.net/browse/APMSP-3219).

# How to test the change?

Prior local validation completed successfully:

- `nix develop -c cargo nextest run -p libdd-data-pipeline-ffi` passed
59 tests, including atomic invalid-UTF-8 failure, ordering, and
ownership coverage.
- `nix develop -c cargo test -p libdd-trace-utils` passed the unit,
integration, and doc-test coverage for all affected encoders and
decoders.
- `nix develop -c cargo clippy -p libdd-trace-utils -p
libdd-data-pipeline-ffi --all-targets -- -D warnings` passed.
- `nix develop .#nightly --command cargo fmt --all -- --check` passed.
- `nix develop --command bundle exec rspec
spec/datadog/tracing/transport/native/tracer_span_spec.rb
spec/datadog/tracing/transport/native/transport_spec.rb
spec/datadog/tracing/transport/native/conformance_spec.rb` passed 119
native transport examples against the locally built artifact.
- `nix develop --command bundle exec rspec
spec/datadog/tracing/span_link_spec.rb
spec/datadog/tracing/distributed/propagation_spec.rb` passed 69
distributed-propagation examples with 1 existing pending example.
- `nix develop --command env
BUNDLE_GEMFILE=gemfiles/ruby_4.0_opentelemetry.gemfile bundle exec rspec
spec/datadog/opentelemetry_spec.rb` passed 164 OpenTelemetry examples.
- Full Standard and Steep checks passed on the matching consumer branch.




[APMSP-3219]:
https://datadoghq.atlassian.net/browse/APMSP-3219?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[APMSP-3830]:
https://datadoghq.atlassian.net/browse/APMSP-3830?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

---------

Co-authored-by: Edmund Kump <edmund.kump@datadoghq.com>
Co-authored-by: Julio <julio.gonzalez@datadoghq.com>
Co-authored-by: Julio Gonzalez <107922352+hoolioh@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants