feat(azure): add vm_id to KVP telemetry event keys #6551

peytonr18 · 2025-10-30T16:33:43Z

Proposed Commit Message

feat(azure): add vm_id to KVP telemetry event keys

Add VM ID to the KVP (Key-Value Pair) event key format to improve
telemetry tracking and debugging capabilities for Azure/Hyper-V
deployments.

Additional Context

The VM ID is now queried during HyperVKvpReportingHandler initialization and appended to each event key.
Changes include:

Query vm_id during HyperVKvpReportingHandler initialization
Store vm_id as instance attribute with fallback to zero-guid string
Update _event_key_format_parts() to include vm_id in key structure
Update event key format to:
CLOUD_INIT|<incarnation>|<event_type>|<event_name>|<uuid>|<vm_id>[|subevent_index]
Added test_vm_id_fallback_to_zero_guid unit test to verify fallback logic.
Production test shows vm_id in key, where vm_id is 26bd6c2f-ecc6-4b04-9f79-e500096ee45b:

CLOUD_INIT|1762458786|finish|modules-final|b8398a61-459c-4e35-95fd-c3a099396a02|26bd6c2f-ecc6-4b04-9f79-e500096ee45b{"name":"modules-final","type":"finish","ts":"2025-11-06T19:53:30.941724+00:00","result":"SUCCESS","msg":"running modules for final"}

Merge Type

Squash merge using "Proposed Commit Message"

…ails; adding unit test to cover scenario

cloudinit/reporting/handlers.py

cjp256 · 2025-11-11T14:15:56Z

cloudinit/reporting/handlers.py

        )
+
+        try:
+            from cloudinit.sources.azure.identity import query_vm_id


was this done late to avoid circular imports?

Yeah that was exactly why - I kept getting an error message about that. Let me know if there is a better way to avoid that circular import!

Generally a circular import error indicates a lack of clear hierarchy in the code. The reporting code is currently not coupled to any particular cloud, but after this change that would no longer be true.

I haven't put much thought into it, but since query_vm_id is currently not used anywhere it could just get relocated to this module.

let's break the circular dependency here by removing the import for:

from cloudinit.sources.helpers.azure import report_diagnostic_event

A little refactoring will be required to do so.

…rder Relocate query_vm_id() and helper functions from azure.identity to reporting.handlers to eliminate circular import. Also fix event key format to place vm_id before uuid in order to prevent breaking RdAgent.

peytonr18 · 2025-11-18T01:32:14Z

A request was made outside of this PR to re-order uuid and vm_id to ensure RdAgent can distinguish the key, which is the reason for the change in the most recent commit.

github-actions · 2025-12-03T00:28:53Z

Hello! Thank you for this proposed change to cloud-init. This pull request is now marked as stale as it has not seen any activity in 14 days. If no activity occurs within the next 7 days, this pull request will automatically close.

If you are waiting for code review and you are seeing this message, apologies! Please reply, tagging blackboxsw, and he will ensure that someone takes a look soon.

(If the pull request is closed and you would like to continue working on it, please do tag blackboxsw to reopen it.)

Move vm_id lookup out of HyperVKvpReportingHandler and into azure/kvp.py. The handler now defaults vm_id to ZERO_GUID, and get_kvp_handler() injects the real vm_id via identity.query_vm_id(). This removes ~65 lines of duplicate Azure identity code from cloudinit/reporting/handlers.py and keeps azure/identity.py as the single source of truth for vm_id derivation. Updated tests to reflect the simplified handler init.

peytonr18 · 2025-12-10T20:55:48Z

Per offline discussion,
There was concern about pulling the same code in azure.identity.py into the reporter, as well as importing the code from the azure module (which is what was done previously).

The current suggestion is to instead have the datasource set vm_id via azure's get_kvp_handler() method, with the eventual goal of eventually caching the vm_id in a shared location and moving all KVP logic out of the HyperVReportingHandler.

Allow HyperVKvpReportingHandler to resolve vm_id at encode time via an injected provider. On the default Hyper-V KVP pool path, fall back to DMI system-uuid when no provider is set to avoid ZERO_GUID keys. Azure KVP helper now installs identity.query_vm_id as the provider. Add unit tests for the fallback behavior.

cjp256 · 2025-12-15T23:08:50Z

cloudinit/reporting/handlers.py

        )
+
+        self._vm_id = self.ZERO_GUID
+        self._vm_id_provider = None


this feels like overkill as opposed to breaking the dependency cycle and querying it directly, let's sync on this to discuss

Per offline discussion:
rather than doing it this way, going to try removing the imports that lead to the circular dependency issue and sort out how to do the necessary imports for the kvp_handler (+ the change made to request it when required)

Include vm_id in KVP telemetry event keys for Azure host-side VM correlation. Break circular import by removing report_diagnostic_event from identity.py - Azure's kvp.py now sets vm_id via identity module when get_kvp_handler() is called.

github-actions · 2025-12-31T00:31:28Z

Hello! Thank you for this proposed change to cloud-init. This pull request is now marked as stale as it has not seen any activity in 14 days. If no activity occurs within the next 7 days, this pull request will automatically close.

If you are waiting for code review and you are seeing this message, apologies! Please reply, tagging blackboxsw, and he will ensure that someone takes a look soon.

(If the pull request is closed and you would like to continue working on it, please do tag blackboxsw to reopen it.)

holmanb · 2026-01-05T14:26:32Z

I just removed the stale-pr label, but I haven't taken a look at this yet because CI is still failing. @peytonr18 do you expect to continue working on this PR?

peytonr18 · 2026-01-05T21:44:27Z

Hi @holmanb, I didn't realize it was failing the CI before I went on holiday. I'll resolve the mypy error and work to get approvals from my team on the PR so that it'll be ready for your approval.

Thanks!

anhvoms

LGTM!

peytonr18 added 4 commits October 29, 2025 17:33

Add vm_id to key value for KVP telemetry

be0671a

Fallback to zero-guid instead of empty string if querying the vm id f…

269f2b3

…ails; adding unit test to cover scenario

Lazy import to resolve circular dependency error

0dc6776

Resolving subprocess unit test errors by mocking query_vm_id()

3329fc6

peytonr18 marked this pull request as ready for review November 6, 2025 19:28

Run tox formatting for CI tests

8f1f71f

KsenijaS reviewed Nov 11, 2025

View reviewed changes

cloudinit/reporting/handlers.py Outdated Show resolved Hide resolved

cjp256 reviewed Nov 11, 2025

View reviewed changes

peytonr18 force-pushed the probertson-kvp-vm-id branch from 5e5a253 to 7b66c56 Compare November 18, 2025 01:28

github-actions bot added stale-pr Pull request is stale; will be auto-closed soon and removed stale-pr Pull request is stale; will be auto-closed soon labels Dec 3, 2025

holmanb self-assigned this Dec 8, 2025

peytonr18 force-pushed the probertson-kvp-vm-id branch from 9d0e689 to fa77a7a Compare December 9, 2025 23:33

peytonr18 force-pushed the probertson-kvp-vm-id branch from a2a7dcd to 0e516d7 Compare December 12, 2025 23:19

peytonr18 force-pushed the probertson-kvp-vm-id branch from 0e516d7 to 8a09dbf Compare December 15, 2025 20:39

cjp256 reviewed Dec 15, 2025

View reviewed changes

feat(azure): Add vm_id to HyperV KVP event keys

aa18e0b

Include vm_id in KVP telemetry event keys for Azure host-side VM correlation. Break circular import by removing report_diagnostic_event from identity.py - Azure's kvp.py now sets vm_id via identity module when get_kvp_handler() is called.

peytonr18 force-pushed the probertson-kvp-vm-id branch from 80da91b to aa18e0b Compare December 16, 2025 20:30

github-actions bot added the stale-pr Pull request is stale; will be auto-closed soon label Dec 31, 2025

holmanb removed the stale-pr Pull request is stale; will be auto-closed soon label Jan 5, 2026

Resolving mypy error

9464ce4

anhvoms approved these changes Jan 6, 2026

View reviewed changes

feat(azure): add vm_id to KVP telemetry event keys #6551

Are you sure you want to change the base?

feat(azure): add vm_id to KVP telemetry event keys #6551

Conversation

peytonr18 commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed Commit Message

Additional Context

Merge Type

Uh oh!

Uh oh!

cjp256 Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

peytonr18 Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

holmanb Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

cjp256 Dec 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

peytonr18 commented Nov 18, 2025

Uh oh!

github-actions bot commented Dec 3, 2025

Uh oh!

peytonr18 commented Dec 10, 2025

Uh oh!

cjp256 Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

peytonr18 Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 31, 2025

Uh oh!

holmanb commented Jan 5, 2026

Uh oh!

peytonr18 commented Jan 5, 2026

Uh oh!

anhvoms left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

peytonr18 commented Oct 30, 2025 •

edited

Loading

cjp256 Dec 5, 2025 •

edited

Loading

cjp256 Dec 15, 2025 •

edited

Loading