Skip to content

Enable config sync by default when CNM direct send is enabled - #3368

Draft
rahulkaukuntla wants to merge 2 commits into
mainfrom
fix/cnm-direct-send-config-sync
Draft

Enable config sync by default when CNM direct send is enabled#3368
rahulkaukuntla wants to merge 2 commits into
mainfrom
fix/cnm-direct-send-config-sync

Conversation

@rahulkaukuntla

Copy link
Copy Markdown
Contributor

What does this PR do?

When CNM direct send is enabled for NPM or USM, also set DD_AGENT_IPC_PORT=5009 and DD_AGENT_IPC_CONFIG_REFRESH_INTERVAL=60 on the core agent, process-agent and system-probe containers, so that config sync is enabled by default alongside direct send.

Both env vars are applied with a merge func that keeps any value the user already provided, so an explicit override.nodeAgent.env entry still wins.

Motivation

Direct send makes system-probe submit network payloads itself instead of routing them through process-agent. system-probe wires the no-op secrets component, so it cannot resolve an ENC[...] secret handle — if api_key comes from a secret backend, system-probe receives the handle verbatim and the direct sender fails to start, taking the whole network_tracer module (NPM and USM) down with it. Config sync is the mechanism that hands system-probe the api_key already resolved by the core agent.

Config sync is off by default and needs two independent settings: agent_ipc.config_refresh_interval must be positive, and there must be a transport (agent_ipc.port positive, or agent_ipc.use_socket). Both config_refresh_interval and port default to 0, so neither is satisfied out of the box.

This matters here because the Operator enables direct send by default for NPM and USM on agents >= 7.81.0 (agent.datadoghq.com/cnm-direct-send-enabled is an opt-out, not an opt-in). Any customer on a secret-backed api_key with NPM or USM therefore loses network monitoring unless they know to turn config sync on themselves. Enabling it here by default removes that footgun.

Related agent-side change: DataDog/datadog-agent#55014, which turns the resulting failure into an explicit error naming the config sync settings, and scopes it to the direct send path so a secret-backed api_key no longer breaks unrelated system-probe modules.

Additional Notes

The port and interval values match what the otelcollector and hostprofiler features already set, for exactly the same secrets reason. That consistency is deliberate: all of these land on the shared core agent container, so a mismatch would make the effective IPC port depend on feature ordering. The shared helper lives in feature/utils rather than being duplicated a third and fourth time.

Clusters that use socket-based IPC instead (agent_ipc.use_socket: true) are unaffected by the port var — both the config sync client (comp/core/configsync/impl/module.go) and the IPC listener (comp/api/api/apiimpl/listener/common.go) check use_socket first and never read agent_ipc.port when it is set, so no extra TCP listener is opened. The refresh interval is still required in that setup, since it is gated independently of the transport.

One precedence caveat for consumers that configure the agent through a datadog.yaml values file: env vars outrank file config, so DD_AGENT_IPC_CONFIG_REFRESH_INTERVAL=60 will override an agent_ipc.config_refresh_interval set that way. Tuning the interval now requires override.nodeAgent.env. This is pre-existing behaviour for the otelcollector and hostprofiler features.

An equivalent change is needed in DataDog/helm-charts, which also enables direct send by default.

Minimum Agent Versions

No new minimum. This only takes effect where direct send is already enabled, which the Operator gates on Agent >= 7.81.0.

  • Agent: v7.81.0
  • Cluster Agent: n/a

Describe your test plan

Unit tests in feature/npm and feature/usm cover both directions: with direct send enabled (the default on a supported agent version) the two env vars are expected on the system-probe container and asserted present on the core agent container; with direct send disabled by annotation, or on an agent below 7.81.0, they are asserted absent.

go test ./internal/... ./api/... and make lint both pass locally.

For manual verification, deploy a DatadogAgent with NPM enabled and global.credentials.apiSecret pointing at a secret-backend-resolved api_key, then confirm system-probe logs configsync enabled (agent_ipc 'localhost:5009' ...) and that the network_tracer module starts rather than failing with an unresolved secret handle.

Checklist

  • PR has at least one valid label: bug, enhancement, refactoring, documentation, tooling, and/or dependencies
  • PR has a milestone or the qa/skip-qa label
  • All commits are signed (see: signing commits)

Direct send is on by default for NPM and USM on agents >= 7.81.0, which makes system-probe send network payloads itself. system-probe wires the no-op secrets component, so an ENC[...] api_key is never resolved there and the direct sender fails to start. Config sync is what supplies the resolved value from the core agent, and both of its settings default to off.

Set DD_AGENT_IPC_PORT and DD_AGENT_IPC_CONFIG_REFRESH_INTERVAL on the core agent, process-agent and system-probe containers whenever direct send is enabled, reusing the values the otelcollector and hostprofiler features already set so the shared core agent container cannot end up with conflicting ports. User-provided values still win.
@datadog-official

datadog-official Bot commented Aug 18, 2026

Copy link
Copy Markdown

Pipelines  Code Coverage

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 50.32% (+0.02%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 786530c | Docs | View more details | Give us feedback!

The helper's statements were only exercised indirectly from the npm and usm packages, so per-package patch coverage counted all of them as uncovered.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant