feat(agent-config): allow extensible configuration via ConfigExtension trait by duncanista · Pull Request #111 · DataDog/serverless-components

duncanista · 2026-04-02T20:00:45Z

Summary

Introduces a generic Config<E: ConfigExtension = NoExtension> type that lets consumers define additional configuration fields without modifying or copy-pasting the core crate
Adds a unified Source type for dual extraction from both env vars and YAML, and a merge_fields! macro to reduce merge boilerplate
Moves Lambda-specific fields out of the core Config struct so non-Lambda consumers don't carry unused fields
Restructures the crate to use a conventional src/ layout and adds a README documenting the extension API

Test plan

All 71 existing + new unit tests pass (cargo test -p datadog-agent-config)
Clippy passes with -D warnings on lib and tests
Full workspace build passes (cargo build --workspace)
Extension mechanism tested: NoExtension works, extension receives env/yaml fields, env overrides yaml, defaults when not set, extension does not interfere with core fields

Copilot

Copilot reviewed 5 out of 12 changed files in this pull request and generated no comments.

duncanpharvey · 2026-04-06T16:31:35Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7c6eb51564

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

crates/datadog-agent-config/src/lib.rs

…n trait Introduces a generic `Config<E: ConfigExtension>` type that lets consumers define additional configuration fields without modifying or copy-pasting the core crate. Includes a unified `Source` type for dual extraction from both env vars and YAML, a `merge_fields!` macro to reduce merge boilerplate, and moves Lambda-specific fields out of the core Config struct. Also restructures the crate to use a conventional `src/` layout and adds a README documenting the extension API.

…s/ modules Move config source implementations (env, yaml) into `src/sources/` and type definitions with custom deserialization into `src/deserializers/`. Re-exports at the crate root preserve all existing import paths.

…zers/helpers.rs Extracts all generic deserializer functions (deserialize_optional_string, deserialize_with_default, duration parsers, key-value parsers, etc.) from lib.rs into src/deserializers/helpers.rs. Re-exported at the crate root so all existing import paths continue to work.

Reorganize lib.rs so an engineer opening the file immediately sees the Config struct and its fields, followed by the loading entry points, then the extension trait, builder, and macros. Sections are separated with headers for quick scanning.

Change NoExtensionSource from a unit struct to an empty struct so serde accepts map-shaped data from figment (env vars / YAML) instead of expecting null/unit. Prevents spurious warning logs on every get_config() call when no extension is used.

duncanpharvey

Approved with a few notes!

duncanpharvey · 2026-04-08T16:03:27Z

crates/datadog-agent-config/README.md

+### Flat fields only
+
+The single `Source` type is used for both env var and YAML extraction. This works when extension fields are top-level (flat) in the YAML file, which is the common case. If you need nested YAML structures that differ from the flat env var layout, implement `merge_from` with a nested source struct and handle the mapping manually.


Would having separate merge_from_env and merge_from_yaml resolve this issue for flat fields?

Not really — the flat-field limitation isn't about the merge step, it's about the deserialization step. Figment uses a single key-value namespace per provider, so a flat Source struct works naturally for both env vars (DD_CUSTOM_FLAG=true) and YAML (custom_flag: true). Splitting into merge_from_env/merge_from_yaml wouldn't change the deserialization model.

The issue only shows up if you need nested YAML (e.g., lambda: { enhanced_metrics: true }) that maps to a flat env var (DD_ENHANCED_METRICS). For that case you'd need two different Source structs — one flat for env, one nested for YAML — which would mean two associated types on the trait. That's extra complexity we don't need yet since all current extension fields are flat.

If we hit that case later, we can add EnvSource/YamlSource associated types in a backward-compatible way. For now I'll update the doc to make this clearer.

duncanpharvey · 2026-04-08T16:09:18Z

crates/datadog-agent-config/README.md

+}
+
+/// Source struct for deserialization. Must use #[serde(default)] and
+/// graceful deserializers so one bad field doesn't fail the whole extraction.


Are there warning if #[serde(default)] or graceful deserializers aren't used? How can we make this clear to someone using the config extension in case they miss this note in the readme?

Good call — there are runtime warnings but no compile-time enforcement. Here's what happens:

Missing #[serde(default)]: figment's extract() fails because serde won't fill in defaults for absent fields → hits the tracing::warn! branch → extension falls back to E::default(). All extension fields silently get defaults.

Missing graceful deserializers: one malformed value (e.g., "yes" for a bool) causes the whole extraction to fail → same warn + fallback behavior.

Both cases are silent in the sense that the extension consumer gets defaults without error, just a warning log. The README note is easy to miss.

I'll improve discoverability by:

Adding a prominent # Safety / requirements section to the Source associated type doc (which IDEs surface on hover)

Explaining the failure mode directly ("extraction will fall back to defaults with a warning")

We can't enforce #[serde(default)] at compile time, but making the consequence clear in the type-level doc should help.

duncanpharvey · 2026-04-08T16:11:30Z

crates/datadog-agent-config/src/sources/yaml.rs


 #[allow(clippy::too_many_lines)]
-fn merge_config(config: &mut Config, yaml_config: &YamlConfig) {
+fn merge_config<E: ConfigExtension>(config: &mut Config<E>, yaml_config: &YamlConfig) {


Are there any consequences to having the same field defined in the extension as one that is already in the core config?

No harmful consequences — the core and extension are extracted independently from the same figment, so both get their own copy of the value. The core field drives behavior as usual, and the extension field just has a duplicate. No errors, no data loss, no interference.

The main risk is confusion: someone might shadow api_key in their extension and expect it to override the core, but it wouldn't — config.api_key and config.ext.api_key would be independent copies.

I'll add a note about this in the trait doc.

Address reviewer feedback: document field name collision behavior, clarify Source type requirements and their runtime failure modes, and expand the README with collision and flat-field explanations.

Copilot AI review requested due to automatic review settings April 2, 2026 20:00

duncanista requested review from a team as code owners April 2, 2026 20:00

duncanista requested review from Lewis-E and jchrostek-dd and removed request for a team April 2, 2026 20:00

Copilot started reviewing on behalf of duncanista April 2, 2026 20:01 View session

Copilot AI reviewed Apr 2, 2026

View reviewed changes

duncanista force-pushed the jordan.gonzalez/config/allow-extensible-configuration branch from 9fe85fb to 7c6eb51 Compare April 2, 2026 20:26

Lewis-E requested a review from duncanpharvey April 2, 2026 21:12

chatgpt-codex-connector bot reviewed Apr 6, 2026

View reviewed changes

crates/datadog-agent-config/src/lib.rs Outdated Show resolved Hide resolved

duncanista added 5 commits April 7, 2026 10:19

duncanista force-pushed the jordan.gonzalez/config/allow-extensible-configuration branch from 3c8af4d to e8ba905 Compare April 7, 2026 14:22

duncanpharvey approved these changes Apr 8, 2026

View reviewed changes

duncanpharvey mentioned this pull request Apr 8, 2026

feat: add DD_ORG_UUID to datadog-agent-config crate #112

Open

2 tasks

docs(agent-config): improve ConfigExtension trait and Source docs

6c948f5

Address reviewer feedback: document field name collision behavior, clarify Source type requirements and their runtime failure modes, and expand the README with collision and flat-field explanations.

duncanista merged commit 1eb4556 into main Apr 9, 2026
27 checks passed

duncanista deleted the jordan.gonzalez/config/allow-extensible-configuration branch April 9, 2026 20:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(agent-config): allow extensible configuration via ConfigExtension trait#111

feat(agent-config): allow extensible configuration via ConfigExtension trait#111
duncanista merged 6 commits intomainfrom
jordan.gonzalez/config/allow-extensible-configuration

duncanista commented Apr 2, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

duncanpharvey commented Apr 6, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

Uh oh!

duncanpharvey left a comment

Uh oh!

duncanpharvey Apr 8, 2026

Uh oh!

duncanista Apr 8, 2026

Uh oh!

duncanpharvey Apr 8, 2026

Uh oh!

duncanista Apr 8, 2026

Uh oh!

duncanpharvey Apr 8, 2026

Uh oh!

duncanista Apr 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		### Flat fields only

		The single `Source` type is used for both env var and YAML extraction. This works when extension fields are top-level (flat) in the YAML file, which is the common case. If you need nested YAML structures that differ from the flat env var layout, implement `merge_from` with a nested source struct and handle the mapping manually.

Conversation

duncanista commented Apr 2, 2026

Summary

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

duncanpharvey commented Apr 6, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

duncanpharvey left a comment

Choose a reason for hiding this comment

Uh oh!

duncanpharvey Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

duncanista Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

duncanpharvey Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

duncanista Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

duncanpharvey Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

duncanista Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants