feat: add PET version and build ID telemetry properties to enhance performance tracking#1548
Closed
eleanorjboyd wants to merge 4 commits into
Closed
feat: add PET version and build ID telemetry properties to enhance performance tracking#1548eleanorjboyd wants to merge 4 commits into
eleanorjboyd wants to merge 4 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR augments existing PET (Python Environment Tools) performance/health telemetry by fetching PET’s version/build identifier via a new info JSON-RPC call and attaching those properties to key PET events, enabling performance tracking across PET versions.
Changes:
- Add a fire-and-forget PET
inforequest and cache the response to stamp subsequent telemetry withpetVersion/petBuildId. - Include
petVersionandpetBuildIdproperties onpet.refresh,pet.resolve, andpet.process_restarttelemetry events. - Register the new telemetry properties in the GDPR mapping/types.
Show a summary per file
| File | Description |
|---|---|
| src/managers/common/nativePythonFinder.ts | Fetches/caches PET info and adds petVersion/petBuildId to PET telemetry payloads. |
| src/common/telemetry/constants.ts | Declares the new telemetry properties and adds GDPR metadata/type annotations for relevant PET events. |
Copilot's findings
- Files reviewed: 2/2 changed files
- Comments generated: 1
StellaHuang95
force-pushed
the
excess-marten
branch
from
e20801f to
81a2f56
Compare
…rformance tracking
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
StellaHuang95
force-pushed
the
excess-marten
branch
from
81a2f56 to
01ddf56
Compare
eleanorjboyd
pushed a commit
to microsoft/python-environment-tools
that referenced
this pull request
Jun 9, 2026
…ression tracking (#473) ## Background PET (Python Environment Tools) is a Rust-based JSONRPC server used by the VS Code Python extension to discover Python environments. PET does not send its own telemetry — instead, it returns metadata to the calling extension (vscode-python-environments), which forwards it via telemetry. To help catch regressions, we need a way to know exactly which PET build a user is running by querying telemetry. [PR #470](#470) added a JSONRPC `info` request that returns PET's version and an optional CI build ID. However, the build number alone isn't enough to pinpoint the exact source code — we also need the git commit hash. This PR extends the `info` response to include the **git commit SHA** baked into the binary at build time. On the extension side, the companion PR ([vscode-python-environments#1548](microsoft/vscode-python-environments#1548)) passes this information along to telemetry, so we can correlate user-reported issues or regressions back to the specific PET commit. ## What Changed ### 1. Build script (`crates/pet/build.rs`) - Added support for three new environment variables that CI systems set automatically: - `PET_COMMIT_SHA` — explicit override (highest priority) - `BUILD_SOURCEVERSION` — set by Azure Pipelines - `GITHUB_SHA` — set by GitHub Actions - When any of these is present and non-empty, the value is embedded into the binary as a compile-time constant via `cargo:rustc-env=PET_COMMIT_SHA=...`. ### 2. JSONRPC response (`crates/pet/src/jsonrpc.rs`) - Added a new optional `commit_sha` field to the `InfoResponse` struct. - `InfoResponse::current()` populates it from the compile-time `PET_COMMIT_SHA` env var (same pattern used for `build_id`). - For local dev builds where no CI env var is set, the field is `None` and omitted from the JSON response. ### 3. JSONRPC documentation (`docs/JSONRPC.md`) - Updated the `InfoResponse` TypeScript interface to document the new `commitSha?: string` field, including where the value is sourced from. ### 4. Tests (`crates/pet/src/jsonrpc.rs`, `crates/pet/tests/jsonrpc_server_test.rs`, `crates/pet/tests/jsonrpc_client.rs`) - Added `commit_sha` field to the test client's `PetInfoResponse` struct. - Added assertions that `commit_sha`, when present, is non-empty (mirrors existing `build_id` assertion pattern). - Added clarifying comments explaining that both `build_id` and `commit_sha` are `None` in local dev builds and only populated in CI. ## How It Works ``` CI build (Azure Pipelines / GitHub Actions) ↓ sets BUILD_SOURCEVERSION or GITHUB_SHA build.rs captures it → bakes into binary as PET_COMMIT_SHA ↓ JSONRPC `info` request → returns { petVersion, buildId, commitSha } ↓ vscode-python-environments extension → includes commitSha in telemetry ↓ Team queries telemetry → correlates regressions to exact PET commit ``` ## Related PRs - [PET PR #470](#470) — Added the `info` JSONRPC request with version and build ID (merged) - [vscode-python-environments PR #1548](microsoft/vscode-python-environments#1548) — Extension-side: passes PET version/build/commit info to telemetry
Contributor
|
This looks good, I am going to add the commit hash as another property to the telemetry and combine all the changes into a single PR. |
StellaHuang95
added a commit
that referenced
this pull request
Jun 10, 2026
## Context PET (Python Environment Tools) is the Rust JSON-RPC service this extension uses to discover Python environments. PET ships independently of the extension and is bundled as a binary inside the VSIX, so the PET version a user is running can drift from the extension version. When we see a performance regression or a failure in PET telemetry today, we have no way to map it back to the **exact PET source code** the user is running — we know `pet --version` (already telemetered via `PET.VERSION`) but a single PET version line can correspond to many commits across multiple builds. This PR closes that gap by stamping every PET telemetry event with the binary's `petVersion`, `petBuildId`, and `petCommitSha`, sourced directly from the running PET process via a new `info` JSON-RPC request. We can then `summarize by petCommitSha` in Kusto and join straight to git log to find the offending change. ## Related PRs PET side (both merged): - microsoft/python-environment-tools#470 — adds the `info` JSON-RPC request returning `petVersion` and optional `buildId` - microsoft/python-environment-tools#473 — extends the `info` response with optional `commitSha` baked in from CI env vars (`PET_COMMIT_SHA` / `BUILD_SOURCEVERSION` / `GITHUB_SHA`) Supersedes / replaces: - #1548 — earlier draft that only added `petVersion` + `petBuildId` and accidentally corrupted the enum docstring for `PET_RESOLVE` / `MIGRATION_SYSTEM_ENV_MANAGER`. Please close. ## What this PR does 1. **Defines a typed `NativePetInfo` interface** matching PET's `info` response shape (`petVersion: string`, `buildId?: string`, `commitSha?: string`). 2. **Sends one `info` RPC per PET process start** in `kickoffInfoFetch(connection)`, called immediately after `connection.listen()` inside `start()`. The call is **fire-and-forget** with a 2 s timeout — `start()` does not await it, so discovery is never blocked. The response is cached in `this.petInfo` for the lifetime of that PET process. `this.petInfo` is reset to `undefined` on every `start()` (initial spawn + every crash-recovery restart). 3. **Guards against stale responses** via `connection !== this.connection` checks in both `.then` and `.catch`, so a late reply from a previous PET process can't clobber the cache of a newer one after a restart. 4. **Spreads `getPetInfoProperties()` into the six existing PET telemetry call sites** (success + error paths of `PET_RESOLVE`, `PET_REFRESH`, `PET_PROCESS_RESTART`). The helper always returns concrete strings, defaulting each field to `'unknown'`, so Kusto group-bys work without null handling. 5. **Adds GDPR comments + TypeScript types** for the three new fields on `PET_RESOLVE`, `PET_REFRESH`, `PET_PROCESS_RESTART`. All classified as `SystemMetaData` / `PerformanceAndHealth`. ## Files changed | File | Why | |---|---| | `src/managers/common/nativePythonFinder.ts` | `INFO_TIMEOUT_MS` constant, `NativePetInfo` interface, `petInfo` field, `kickoffInfoFetch` + `getPetInfoProperties` helpers, kickoff wiring in `start()`, payload spread at six telemetry sites | | `src/common/telemetry/constants.ts` | New `petVersion` / `petBuildId` / `petCommitSha` properties on `PET_RESOLVE`, `PET_REFRESH`, `PET_PROCESS_RESTART` (GDPR blocks + TS types) | ## Compatibility with older PET binaries The extension currently ships PET as a bundled binary (downloaded by the Azure pipelines from PET CI artifacts). Until the PET release branch picks up #470 / #473, the bundled binary won't have the `info` handler. In that case: - PET responds with JSON-RPC error code `-1` (`Failed to find handler for request info`) - `sendRequestWithTimeout` rejects → `.catch` swallows → `petInfo` stays `undefined` - All three telemetry properties report `'unknown'` - One harmless `[pet] Failed to find handler for method: info` line surfaces from PET's stderr into the Python Environments output channel Discovery, refresh, resolve, and restart all continue to work normally. No crash, no functional regression — just `'unknown'` values in telemetry until PET catches up. ## Crash attribution semantics A subtle but important detail of where the spread is placed: the crashing PET's commit hash **is** captured in `PET_REFRESH` / `PET_RESOLVE` error events because we call `sendTelemetryEvent(..., ...this.getPetInfoProperties())` **before** killing the process and resetting the cache. So when a user reports "PET crashed during refresh", we can identify which exact commit was running. The `PET_PROCESS_RESTART` success event itself reports `'unknown'` for the **new** PET (its `info` reply usually hasn't landed in the few ms between `start()` and the telemetry call), but the new binary's identity surfaces on the very next refresh/resolve. ## Performance impact - One extra JSON-RPC roundtrip **per PET process lifetime** (typically once per VS Code session), not per telemetry event - ~3 string allocations per telemetry event from the spread — invisible against existing payload assembly - `kickoffInfoFetch` returns `void` synchronously; the response runs on the microtask queue and never blocks refresh/resolve - 2 s timeout caps the worst case if PET hangs entirely ## Validation - `npm run lint` ✅ - `npx tsc -p . --noEmit` ✅ - `npm run compile-tests` ✅ - `npm run unittest` — 1141 passing, 2 pending (unchanged from baseline) ✅ - `npm run compile` (webpack production bundle) ✅ ## Manual testing To get non-`'unknown'` values locally, build PET from main and drop the binary into `python-env-tools/bin/`: ```powershell # In the PET repo: $env:PET_COMMIT_SHA = (git rev-parse HEAD) $env:PET_BUILD_ID = "local-dev" cargo build --release --package pet # In this repo: Copy-Item <pet-repo>\target\release\pet.exe .\python-env-tools\bin\pet.exe -Force ``` Then F5 → open the Python Environments view → run `Python Environments: Refresh Environments`. Set the Python Environments output channel to Debug level to see the `[pet] info: { petVersion, buildId, commitSha }` line confirming the cache was populated. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Eleanor Boyd <26030610+eleanorjboyd@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
goes along with microsoft/python-environment-tools#470 to support getting PET versions as telemetry to track performance regression and improvements across specific version numbers