Skip to content

fix(tools): collapse intra-file duplicate typedefs in base header - #2149

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 2 commits into
mainfrom
lloeki/fix-dedup-headers-intra-file-typedefs
Jul 24, 2026
Merged

fix(tools): collapse intra-file duplicate typedefs in base header#2149
gh-worker-dd-mergequeue-cf854d[bot] merged 2 commits into
mainfrom
lloeki/fix-dedup-headers-intra-file-typedefs

Conversation

@lloeki

@lloeki lloeki commented Jun 22, 2026

Copy link
Copy Markdown
Member

What

Enhance the dedup_headers dev tool so the generated/bundled
include/datadog/common.h no longer contains duplicate typedefs.

Why

dedup_headers only removed definitions from child headers that were
byte-identical to ones already present in the base header (common.h).
It never deduplicated definitions within the base header. When cbindgen
emits the same profiling type from two crate boundaries (e.g. via
after_includes forward declarations in libdd-profiling-ffi/cbindgen.toml
plus the regular body definition), the merged common.h ends up with
duplicate typedefs.

These are fatal for consumers compiling with -Werror -Wtypedef-redefinition
(C11). Two distinct classes were observed in the v36.0.0 artifacts:

  1. Forward + full-struct collision — a typedef struct X X; forward
    declaration coexisting with the full typedef struct X { ... } X;:
    ddog_prof_EncodedProfile, ddog_prof_StringId, OpaqueStringId.
  2. Exact-duplicate pointer typedefs emitted twice, identical except that
    one carries a doc comment (so the existing exact-string dedup keeps both):
    ddog_prof_StringId2, ddog_prof_MappingId2, ddog_prof_FunctionId2.

How

Add a final pass (dedup_base_typedefs) over the assembled base header that:

  • drops a bare forward typedef struct/union/enum X X; when a full-body
    definition of the same name X exists elsewhere in the file (keeping the
    body, regardless of ordering), and
  • drops later duplicates of an identical typedef statement, comparing the
    statement text with any leading doc comment stripped.

Opaque forward declarations (no body) and genuine aliases
(typedef struct A B; with A != B) are preserved.

This makes common.h clean by construction and obsoletes downstream
post-processing workarounds (e.g. the one in libdatadog-rb).

Validation

Headers were generated via the FFI crates' cbindgen build scripts and run
through dedup_headers exactly as builder invokes it.

Before — each of the six types appears twice; clang fails:

$ clang -std=gnu99 -Werror -Wtypedef-redefinition -I<out>/include -fsyntax-only t.c
common.h:541: error: redefinition of typedef 'ddog_prof_EncodedProfile' ...
common.h:909: error: redefinition of typedef 'ddog_prof_StringId2' ...
common.h:936: error: redefinition of typedef 'ddog_prof_MappingId2' ...
common.h:965: error: redefinition of typedef 'ddog_prof_FunctionId2' ...
common.h:1166: error: redefinition of typedef 'OpaqueStringId' ...
common.h:1516: error: redefinition of typedef 'ddog_prof_StringId' ...
6 errors generated.

After — each type appears exactly once; clang passes (exit 0):

ddog_prof_EncodedProfile : 1
ddog_prof_StringId       : 1
OpaqueStringId           : 1
ddog_prof_StringId2      : 1
ddog_prof_MappingId2     : 1
ddog_prof_FunctionId2    : 1
  • cargo test -p tools --lib — 20 passed (5 new tests for the dedup pass)
  • cargo clippy -p tools --all-targets --all-features -- -D warnings — clean
  • cargo fmt -p tools -- --check — clean

Note

One of three coordinated changes for the libdatadog v36 duplicate-typedef header issue (increasing order of permanence):

  • dd-trace-rb#5928 — immediate CI mitigation: bump dd-trace-rb to v36 plus a temporary -Wno-error=typedef-redefinition stopgap.
  • libdatadog-rb#62 — gem-level fix: strip the duplicate typedefs during vendoring and ship 36.0.0.1.1, without waiting for a libdatadog release.
  • This PR (libdatadog#2149) — upstream fix in dedup_headers: makes common.h clean by construction and obsoletes the libdatadog-rb post-processing once released.

@datadog-official

datadog-official Bot commented Jun 22, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 95.00%
Overall Coverage: 74.67% (+0.02%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 68316e4 | Docs | Datadog PR Page | Give us feedback!

@lloeki
lloeki marked this pull request as ready for review June 22, 2026 15:58
@lloeki
lloeki requested a review from a team as a code owner June 22, 2026 15:59
@lloeki
lloeki force-pushed the lloeki/fix-dedup-headers-intra-file-typedefs branch from ab502c6 to 77f60a7 Compare June 22, 2026 16:05
Comment thread tools/src/lib.rs Outdated
Comment thread tools/src/lib.rs Outdated
Comment thread tools/src/lib.rs Outdated
@dd-octo-sts

dd-octo-sts Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.13 MB 8.13 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 88.33 MB 88.33 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 99.62 MB 99.62 MB 0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.94 MB 10.94 MB 0% (0 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 26.65 MB 26.65 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 89.18 KB 89.18 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 191.80 MB 191.80 MB +0% (+8.00 KB) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 1.04 GB 1.04 GB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 8.68 MB 8.68 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 89.18 KB 89.18 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 25.40 MB 25.40 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 50.65 MB 50.65 MB 0% (0 B) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 23.21 MB 23.21 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 90.58 KB 90.58 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 196.51 MB 196.50 MB -0% (-8.00 KB) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 1.03 GB 1.03 GB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.70 MB 6.70 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 90.58 KB 90.58 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 27.32 MB 27.32 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 48.23 MB 48.23 MB 0% (0 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 78.78 MB 78.78 MB 0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 9.07 MB 9.07 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 94.28 MB 94.28 MB 0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.04 MB 11.04 MB 0% (0 B) 👌

@lloeki
lloeki requested a review from hoolioh July 17, 2026 13:30

@hoolioh hoolioh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending on Yann's suggestions, it looks good to me.

@hoolioh
hoolioh force-pushed the lloeki/fix-dedup-headers-intra-file-typedefs branch 2 times, most recently from f0aae42 to 00f6661 Compare July 24, 2026 10:16
dedup_headers only removed definitions from child headers that were
byte-identical to ones already present in the base header. It never
deduplicated definitions within the base header itself, so cbindgen
output that emits the same profiling type from two crate boundaries
left duplicate typedefs in the merged common.h.

Two cases survived and broke consumers compiling with
-Werror -Wtypedef-redefinition (C11):

1. A bare forward declaration "typedef struct X X;" coexisting with the
   full-body definition "typedef struct X { ... } X;" (e.g.
   ddog_prof_EncodedProfile, ddog_prof_StringId, OpaqueStringId).
2. An identical pointer typedef emitted twice whose doc comments differ,
   so the existing exact-string dedup kept both (ddog_prof_StringId2,
   ddog_prof_MappingId2, ddog_prof_FunctionId2).

Add a final pass over the assembled base header that drops a forward
struct/union/enum declaration when a full-body definition of the same
name exists, and removes later duplicates of an identical typedef
statement regardless of differing comments. This makes the generated
common.h clean by construction and removes the need for downstream
post-processing workarounds.
@hoolioh
hoolioh force-pushed the lloeki/fix-dedup-headers-intra-file-typedefs branch from 00f6661 to d322fd4 Compare July 24, 2026 10:18
@hoolioh

hoolioh commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Jul 24, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-07-24 10:18:55 UTC ℹ️ Start processing command /merge


2026-07-24 10:19:02 UTC ℹ️ MergeQueue: waiting for PR to be ready

This pull request is not mergeable according to GitHub. Common reasons include pending required checks, missing approvals, or merge conflicts — but it could also be blocked by other repository rules or settings.
It will be added to the queue as soon as checks pass and/or get approvals. View in MergeQueue UI.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.


2026-07-24 10:28:20 UTC ⚠️ MergeQueue: This merge request was unqueued

julio.gonzalez@datadoghq.com unqueued this merge request

@hoolioh
hoolioh force-pushed the lloeki/fix-dedup-headers-intra-file-typedefs branch from d322fd4 to 89c2c06 Compare July 24, 2026 10:20
@pr-commenter

pr-commenter Bot commented Jul 24, 2026

Copy link
Copy Markdown

Benchmarks

Comparison

Candidate

Candidate benchmark details

Baseline

Baseline benchmark details

@hoolioh
hoolioh force-pushed the lloeki/fix-dedup-headers-intra-file-typedefs branch from 89c2c06 to d322fd4 Compare July 24, 2026 10:24
@hoolioh

hoolioh commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Jul 24, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-07-24 10:26:30 UTC ℹ️ Start processing command /merge


2026-07-24 10:26:40 UTC ❌ MergeQueue

PR already in the queue with status waiting

@hoolioh

hoolioh commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

/merge -c

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Jul 24, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-07-24 10:28:14 UTC ℹ️ Start processing command /merge -c

@hoolioh

hoolioh commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Jul 24, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-07-24 10:28:36 UTC ℹ️ Start processing command /merge


2026-07-24 10:28:43 UTC ℹ️ MergeQueue: waiting for PR to be ready

This pull request is not mergeable according to GitHub. Common reasons include pending required checks, missing approvals, or merge conflicts — but it could also be blocked by other repository rules or settings.
It will be added to the queue as soon as checks pass and/or get approvals. View in MergeQueue UI.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.


2026-07-24 11:04:08 UTC ℹ️ MergeQueue: merge request added to the queue

The expected merge time in main is approximately 1h (p90).


2026-07-24 11:45:52 UTCMergeQueue: The checks failed on this merge request

Tests failed on this commit 68423be:

What to do next?

  • Investigate the failures and when ready, re-add your pull request to the queue!
  • If your PR checks are green, try to rebase/merge. It might be because the CI run is a bit old.
  • Any question, go check the FAQ.

@hoolioh

hoolioh commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Jul 24, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-07-24 12:48:14 UTC ℹ️ Start processing command /merge


2026-07-24 12:48:18 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 1h (p90).


2026-07-24 13:35:14 UTC ℹ️ MergeQueue: This merge request was merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants