Skip to content

Merge ThreadContext (MDC) into event ContextData in Log4j2Logger#6

Merged
merlimat merged 2 commits into
mainfrom
fix-mdc-propagation
Apr 25, 2026
Merged

Merge ThreadContext (MDC) into event ContextData in Log4j2Logger#6
merlimat merged 2 commits into
mainfrom
fix-mdc-propagation

Conversation

@merlimat
Copy link
Copy Markdown
Owner

Summary

Log4j2Logger.buildContextData builds the StringMap from only slog's contextAttrs and event attrs — it ignores the caller thread's log4j2 ThreadContext. Layouts using %X{key} and appenders that read event.getContextData().getValue(key) see nothing, even when the thread had MDC entries set at the time of the log call.

The standard log4j2 logger handles this via ContextDataInjector inside LogEventFactory. Slog's Log4j2Logger builds the MutableLogEvent manually for performance, and so must do the equivalent injection itself.

Reproduction (pre-fix)

ThreadContext.put("req", "abc123");
SLF4J_LOG.info("via SLF4J");        // → [req=abc123] via SLF4J
SLOG_LOG.info("via slog short");    // → [req=]       via slog short  ❌
SLOG_LOG.info().log("via slog event"); // → [req=]    via slog event  ❌

Fix

In Log4j2Logger.buildContextData, after clearing the thread-local StringMap, copy the current ThreadContext entries in first via ThreadContext.getImmutableContext().forEach(map::putValue), then layer slog's contextAttrs / event attrs / duration on top so slog overrides on key collision.

Performance

The fast path is preserved with one extra check:

MDC state slog attrs Cost added
Empty None One ThreadContext.isEmpty() check (single field read on the static map). Returns the same emptyFrozenContextData() singleton — no allocation, no map clear.
Empty Some Same as before (zero new cost in the inner loop).
Non-empty Either One getImmutableContext() (returns a backing-map reference, no copy) + a forEach loop to copy MDC entries into the existing thread-local StringMap. The copy has to happen anyway for the appender to see those entries.

ThreadContext.getImmutableContext() returns the actual immutable map reference from the underlying ThreadContextMap — it does not allocate a copy. Map.forEach iterates without per-entry allocation.

Tests

Added Log4j2MdcPropagationTest with 7 cases covering:

  • MDC propagation through plain info(String) and fluent info().log(...) paths
  • Override semantics: slog event attrs and derived-logger context attrs both override MDC on key collision
  • MDC entries with no slog override are preserved
  • slog event attrs do not leak into ThreadContext after the log call
  • The empty-MDC + empty-attrs fast path still produces a non-null empty ContextData
  • Pooled thread-local StringMap doesn't retain stale entries between calls

Without the fix, 4 of 7 tests fail. With it, all 7 pass.

Log4j2Logger.buildContextData built the StringMap from only the slog
contextAttrs and event attrs, completely ignoring the caller thread's
log4j2 ThreadContext. Layouts using %X{key} and appenders that read
event.getContextData().getValue(key) saw nothing — even though the
thread had MDC entries set when the log call ran. The standard log4j2
logger handles this via ContextDataInjector inside the LogEventFactory;
slog builds the event manually and so must do the equivalent.

Fix: in buildContextData, after clearing the thread-local StringMap,
copy the current ThreadContext entries in first via
ThreadContext.getImmutableContext().forEach(map::putValue), then layer
slog's contextAttrs / event attrs / duration on top so slog overrides
on key collision.

Performance: when MDC is empty (the dominant case), the only added
work is one ThreadContext.isEmpty() check — no allocation, no map
clear, the existing emptyFrozenContextData() fast path is preserved.
When MDC is non-empty, getImmutableContext() returns a backing-map
reference (no copy) and forEach iterates without allocating
intermediate entries.
@merlimat merlimat merged commit 7c67aa8 into main Apr 25, 2026
7 checks passed
merlimat added a commit to merlimat/bookkeeper that referenced this pull request Apr 25, 2026
Pulls in the MDC propagation fix from merlimat/slog#6, which makes
log4j2 ThreadContext entries visible on slog events emitted via the
Log4j2Logger backend (so %X{key} layouts and appenders that read
event.getContextData().getValue(key) see the caller's MDC).
merlimat added a commit to merlimat/bookkeeper that referenced this pull request Apr 25, 2026
Pulls in the MDC propagation fix from merlimat/slog#6, which makes
log4j2 ThreadContext entries visible on slog events emitted via the
Log4j2Logger backend (so %X{key} layouts and appenders that read
event.getContextData().getValue(key) see the caller's MDC).
merlimat added a commit to merlimat/bookkeeper that referenced this pull request Apr 25, 2026
Pulls in the MDC propagation fix from merlimat/slog#6, which makes
log4j2 ThreadContext entries visible on slog events emitted via the
Log4j2Logger backend (so %X{key} layouts and appenders that read
event.getContextData().getValue(key) see the caller's MDC).
merlimat added a commit to merlimat/bookkeeper that referenced this pull request Apr 25, 2026
Pulls in the MDC propagation fix from merlimat/slog#6, which makes
log4j2 ThreadContext entries visible on slog events emitted via the
Log4j2Logger backend (so %X{key} layouts and appenders that read
event.getContextData().getValue(key) see the caller's MDC).
merlimat added a commit to apache/bookkeeper that referenced this pull request Apr 28, 2026
* BP-69: Convert bookkeeper-common from SLF4J to slog

First phase of the slog migration for BP-69. Adds the slog dependency,
Lombok @CustomLog configuration, and converts the bookkeeper-common module.

* Added license header

* BP-69 common: fix checkstyle ImportOrder violations

CI flagged 4 ImportOrder violations in the Phase 1 conversion:
- SafeRunnable.java: io.github.merlimat.slog.Logger placed after java.*
- OrderedExecutor.java: lombok.CustomLog placed after org.*
- AbstractLifecycleComponent.java: same
- TestOrderedExecutorDecorators.java: same

Move each import to its correct alphabetical position per the project's
checkstyle rule (`io.` < `java.`; `lombok.` < `org.`).

* BP-69 common: add slog-0.9.7 to bookkeeper-dist LICENSE files

CI check-binary-license flagged the new io.github.merlimat.slog-slog-0.9.7.jar
as unaccounted for. Add it to the Apache-2.0 bundled-jars list in both
LICENSE-all.bin.txt and LICENSE-bkctl.bin.txt, with a new numbered source
reference pointing at https://github.com/merlimat/slog/tree/v0.9.7.

* BP-69 common: also add slog-0.9.7 to LICENSE-server.bin.txt

The previous LICENSE fix only updated LICENSE-all.bin.txt and
LICENSE-bkctl.bin.txt, but the bin-server.xml assembly descriptor
packages LICENSE-server.bin.txt into the server tarball that the CI
license-check script inspects.

* BP-69 common: remove TestOrderedExecutorDecorators and MdcContextTest

Both tests exercise SLF4J MDC context propagation through the
OrderedExecutor / OrderedScheduler decorators. With the slog migration
the logger API is different and these tests are not straightforward to
keep working against the new logger while still testing MDC behaviour
as seen by the underlying SLF4J backend.

Drop them for now. The MDC propagation code itself (MdcUtils,
OrderedExecutor's mdcContextMap snapshot/restore) is unchanged, so
end-to-end MDC-in-logs behaviour under Logback/Log4j2 backends is
preserved.

* Revert "BP-69 common: remove TestOrderedExecutorDecorators and MdcContextTest"

This reverts commit 790d495.

* Bump slog dependency to 0.9.8

Pulls in the MDC propagation fix from merlimat/slog#6, which makes
log4j2 ThreadContext entries visible on slog events emitted via the
Log4j2Logger backend (so %X{key} layouts and appenders that read
event.getContextData().getValue(key) see the caller's MDC).
merlimat added a commit to apache/bookkeeper that referenced this pull request Apr 28, 2026
* BP-69 base: add slog dependency and LICENSE entries

Minimal scaffolding commit for the BP-69 slog migration series:

- Add `io.github.merlimat.slog:slog:0.9.7` to the root pom.xml
  dependencyManagement and to the global compile classpath alongside
  the existing SLF4J API (which stays as the rendering backend).
- Add lombok.config at the repo root so `@CustomLog` generates a slog
  `Logger` instead of an SLF4J one.
- Register the slog jar in LICENSE-all.bin.txt, LICENSE-server.bin.txt
  and LICENSE-bkctl.bin.txt so the CI check-binary-license script
  finds it accounted for in the bundled-jars list.

No actual Java file is converted in this commit. Individual module
migrations stack on top of this one.

* BP-69: Convert stats and allocator modules from SLF4J to slog

Part of BP-69. Converts the stats-api, codahale/otel/prometheus metrics
providers, stats-utils, and bookkeeper-common-allocator modules.

* Fix checkstyle ImportOrder violations

* Bump slog dependency to 0.9.8

Pulls in the MDC propagation fix from merlimat/slog#6, which makes
log4j2 ThreadContext entries visible on slog events emitted via the
Log4j2Logger backend (so %X{key} layouts and appenders that read
event.getContextData().getValue(key) see the caller's MDC).
merlimat added a commit to apache/bookkeeper that referenced this pull request Apr 28, 2026
* BP-69 base: add slog dependency and LICENSE entries

Minimal scaffolding commit for the BP-69 slog migration series:

- Add `io.github.merlimat.slog:slog:0.9.7` to the root pom.xml
  dependencyManagement and to the global compile classpath alongside
  the existing SLF4J API (which stays as the rendering backend).
- Add lombok.config at the repo root so `@CustomLog` generates a slog
  `Logger` instead of an SLF4J one.
- Register the slog jar in LICENSE-all.bin.txt, LICENSE-server.bin.txt
  and LICENSE-bkctl.bin.txt so the CI check-binary-license script
  finds it accounted for in the bundled-jars list.

No actual Java file is converted in this commit. Individual module
migrations stack on top of this one.

* BP-69: Convert stream module from SLF4J to slog

* Bump slog dependency to 0.9.8

Pulls in the MDC propagation fix from merlimat/slog#6, which makes
log4j2 ThreadContext entries visible on slog events emitted via the
Log4j2Logger backend (so %X{key} layouts and appenders that read
event.getContextData().getValue(key) see the caller's MDC).
merlimat added a commit to apache/bookkeeper that referenced this pull request Apr 28, 2026
…hase 4+6) (#4758)

* BP-69 base: add slog dependency and LICENSE entries

Minimal scaffolding commit for the BP-69 slog migration series:

- Add `io.github.merlimat.slog:slog:0.9.7` to the root pom.xml
  dependencyManagement and to the global compile classpath alongside
  the existing SLF4J API (which stays as the rendering backend).
- Add lombok.config at the repo root so `@CustomLog` generates a slog
  `Logger` instead of an SLF4J one.
- Register the slog jar in LICENSE-all.bin.txt, LICENSE-server.bin.txt
  and LICENSE-bkctl.bin.txt so the CI check-binary-license script
  finds it accounted for in the bundled-jars list.

No actual Java file is converted in this commit. Individual module
migrations stack on top of this one.

* BP-69: Convert http, tools, benchmark and metadata-drivers modules to slog

* Bump slog dependency to 0.9.8

Pulls in the MDC propagation fix from merlimat/slog#6, which makes
log4j2 ThreadContext entries visible on slog events emitted via the
Log4j2Logger backend (so %X{key} layouts and appenders that read
event.getContextData().getValue(key) see the caller's MDC).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant