Skip to content

HELD: 000159 legs 1-2 -- flake instrumentation, 5.1 SARIF validation, ManifestConsistency multi-name export fix (prep v1.27.2) - #108

Merged
manderse21 merged 5 commits into
mainfrom
dispatch/000159-legs-1-2
Jul 29, 2026
Merged

HELD: 000159 legs 1-2 -- flake instrumentation, 5.1 SARIF validation, ManifestConsistency multi-name export fix (prep v1.27.2)#108
manderse21 merged 5 commits into
mainfrom
dispatch/000159-legs-1-2

Conversation

@manderse21

Copy link
Copy Markdown
Owner

HELD for Mike. Do not merge until reviewed; every merge and every pipeline trigger stays yours.

Dispatch 000159, legs 1 and 2. Prepped for the v1.27.2 PATCH cut.

What is here

Leg Change Bump
1a Flake instrumentation: rescue isolated data-root logs into the CI artifact; split exited-empty-stdout from killed-at-cap none (test-infra)
1b Schema-validate the SARIF Windows PowerShell 5.1 emits, from a pwsh step none (CI)
2 ManifestConsistency reads multi-name Export-ModuleMember lists PATCH -> 1.27.2

Leg 1a -- instrument the flake, do not theorise about it

000156 leg 4 falsified the standing explanation (recorded It duration 3.3167s against a 25000ms
cap -- nothing was killed at a cap) and then could go no further, because both things it needed were
unobservable. Steps 1 and 2 of its recorded fix shape, and only those:

  • The logs. Isolated roots live under the OS temp dir while CI uploads only psls-test-data/**,
    so they were outside the uploaded tree to begin with -- and several teardowns then discarded
    them. Two barriers, not the one the charter described. The rescue runs first in each
    teardown, ahead of the session-file cleanup that was itself discarding session/<sid>.json.
  • The outcome. Invoke-PluginHook collapsed three distinct failures into one empty string.
    The reason is now recorded on a side channel; the return value is untouched, so no behaviour or
    timing changes. Closed across all 12 collapsing hooks.

Both exclusions are earned, not asserted: Invoke-CaptureU and Invoke-CaptureC spawn with a cap
but return a hashtable carrying ExitCode = -999 / Err = 'timeout', so they already discriminate.
The guard proves that rather than trusting it.

Step 3 (bounded retry / widened window) is deliberately NOT built. The evidence to choose
between them does not exist yet; producing it is the point. No Start-Sleep anywhere, including
fixtures -- killed-at-cap is forced with a child that blocks on an unset ManualResetEventSlim and
never exits, so the cap expires deterministically rather than probabilistically.

Leg 1b -- the riskiest host was the unchecked one

Three conformance cases skip on 5.1 because Test-Json -Schema does not exist there. That skip is
legitimate and stays. What it left behind: SARIF emitted under 5.1 was never schema-validated
anywhere, and 5.1's ConvertTo-Json is the serializer most likely to deviate. The JSON is already
produced; only the validator needed a modern host. Emission stays on 5.1, validation moves to pwsh.

-RequireHost 5 is the vacuity guard and is the point, not a nicety: a leg that silently emitted
nothing would otherwise validate zero files and report success.

Leg 2 -- the fix, and an accepted deviation

Multi-name lists parse as one node in either idiomatic form (ArrayLiteralAst /
ArrayExpressionAst), so both were skipped whole, the collected set stayed empty, and the caller
read that as "no explicit Export-ModuleMember" and assumed export-all -- reporting every
private function as under-declared.

  • dogfood-reader.psm1: 13 false warnings -> 0, modelled surface matching Import-Module
    ground truth exactly (25 defined - 12 exported = the 13).
  • -Cmdlet was measured to share the collection path and is fixed with it. -Alias measured
    not to -- alias names stay out of the function set (BurntToast shape, 000128).
  • Mixed lists degrade rather than half-resolving: a partial set read as complete would be worse
    than the silence it replaced.

Accepted deviation (Mike's ruling)

The 0%-FP oracle bar is scoped to the chartered class, and that class measures clean:
0 of 910 remaining hits are attributable to a multi-name list. PowerShellGet's lists now resolve
correctly (38 names); every other module with remaining hits has no Export-ModuleMember at all.

A second, unrelated class is accepted as a recorded deviation and not fixed here: a
manifest's FunctionsToExport is the final export gate, so a defined-but-unlisted function is
simply not exported, yet is still reported as under-declared. Live oracle: 155 manifests enumerated,
26 with a resolvable RootModule, hits 1088 -> 910, of which 909 of 910 are confirmed false
positives with 0 true positives
. Pester is the clean case: 419 functions defined, manifest lists
26, PowerShell exports exactly 26, and all 393 hits name functions it does not export. Chartered as
its own PATCH dispatch in the outbox.

Also recorded, pinned-not-fixed: the dot-source degrade in the same helper is inert -- it matches
CommandElements[0] -eq '.', but PowerShell carries the dot as the InvocationOperator. Same trap
000156 leg 3 banked after hitting it in a guard, never swept from shipped code. A characterization
test pins it so it cannot change silently in either direction.

Verification

Every proof below was re-run in this session against the adopted commits rather than trusted
from their messages (Hub Rule 7).

Proof Result
Leg 1a suite 17/17
Leg 1b real run (5.1 emits -> pwsh validates) exit 0, 2 artifacts conform
Leg 1b RED: empty dir / missing dir / no-5.1-artifact / non-conformant exit 1 each
Leg 2 reproducer at base 2068b2b RED 8/14
Leg 2 reproducer at work GREEN 14/14
dogfood-reader 13 -> 0, ground truth matched
PowerShellLsp.Release.Tests.ps1 39/39
Version lockstep plugin.json == marketplace.json == 1.27.2 == dated CHANGELOG heading
ASCII (0x00-0x7F), per file clean on every touched file

After merging

  1. Confirm the push CI on main is green and headSha-matched to the merge commit (000081).
  2. Trigger the release pipeline for 1.27.2 -- dry-run first, then the producing run.
    No tag is cut here: the bump helper's manual tag pair is a broken-pipeline fallback and was
    deliberately not run (a hand-cut tag is unsigned and would make Gate 2 refuse the pipeline).

PR-B (leg 3) stacks on this branch and assumes it merges first.

https://claude.ai/code/session_01NCXh8Xms8ZKhdfKRBqSmaQ

… split empty-stdout from killed-at-cap (leg 1a)

Steps 1 and 2 of the fix shape 000156 leg 4 recorded, and only those. Step 3
(bounded retry / widened window) stays deliberately unbuilt: the evidence needed
to choose between them does not exist yet, and producing it is the point.

Step 1 -- the logs. Several Describes mint an isolated data root under the OS temp
dir while CI uploads only psls-test-data/**, so those roots were outside the
uploaded tree to begin with, and several AfterAll blocks then discarded them. Two
barriers, not the one the charter described. Save-IsolatedDataRootLog copies each
root's logs/ and session/ into the uploaded tree, wired FIRST in each AfterAll --
ahead of the session-file cleanup, which was itself discarding session/<sid>.json
from those roots.

Step 2 -- the outcome. Invoke-PluginHook collapsed three distinct failures into one
empty string. New-PluginHookOutcome records which, on a side channel; the return
value is untouched, so no behavior or timing changes. Closed across all 12
collapsing hooks, not one instance.

Both exclusions are earned rather than asserted: Invoke-CaptureU and Invoke-CaptureC
spawn with a cap but return a hashtable carrying ExitCode -999 and Err 'timeout',
so they already discriminate. Proven in the guard, not assumed.

No Start-Sleep anywhere, including the fixtures: killed-at-cap is forced with a
child that blocks on an unset ManualResetEventSlim and NEVER exits, so WaitForExit
times out deterministically rather than probabilistically.

Also fixes a gap the instrumentation itself exposed: 000024 and 000039 never
dot-sourced Integration.Common.ps1, so the new call failed at run time in exactly
the blocks that skip locally. A guard now asserts every instrumented hook's
Describe dot-sources the helper it calls.

Claude-Session: https://claude.ai/code/session_01NCXh8Xms8ZKhdfKRBqSmaQ
…eg 1b)

The cheapest fix shape 000157 leg 4 named, and nothing more.

THE GAP. Three conformance cases skip on -Skip:(PSVersion.Major -lt 6) because
they call Test-Json -Schema, measured absent on 5.1 and present on pwsh 7. The
skip is legitimate and STAYS -- the test physically cannot run there. What it left
behind is narrow and real: SARIF emitted UNDER 5.1 was never schema-validated
anywhere, and 5.1's ConvertTo-Json is exactly the serializer most likely to
deviate. The riskiest host was the unchecked one.

THE FIX. The JSON is already produced; only the validator needs a modern host. The
5.1 leg now writes what it emitted (Save-EmittedSarif, gated on an env var so local
runs write nothing) and a pwsh CI step validates those artifacts against the SAME
vendored 2.1.0 schema. Emission stays on 5.1; only validation moves. The in-suite
skips are untouched.

-RequireHost 5 is the vacuity guard, and it is the point rather than a nicety: a
leg that silently emitted nothing would otherwise validate zero files and report
success -- green has to mean "5.1's own SARIF was checked", not "nothing was
checked". #Requires -Version 6.0 on the validator is load-bearing for the same
reason: it must refuse to run on the host that cannot validate, never degrade to
a pass.

Measured, not argued: 5.1 emitted ps5-helpers-report.sarif and
ps5-not-analyzed.sarif; pwsh 7.6.3 validated both against the vendored schema --
the first time 5.1's serializer output has ever been checked. Four RED proofs:
empty dir, missing dir, artifacts present but none from host 5, and a
non-conformant payload (rejected naming '/runs' as the offending pointer). All
exit 1; the real run exits 0.

Sarif.Common.ps1 exists because Pester 5 runs discovery and run in separate
scopes: the first cut defined the helper at the top of the .Tests.ps1 and every
test in the block failed with "Save-EmittedSarif is not recognized".

Claude-Session: https://claude.ai/code/session_01NCXh8Xms8ZKhdfKRBqSmaQ
…sistency (leg 2)

The export-name collector accepted only individual StringConstantExpressionAst
elements, so a multi-name list parsed as ONE node in either idiomatic form --
'A', 'B' (ArrayLiteralAst) and @('A','B') (ArrayExpressionAst) -- and was skipped
whole. The collected set stayed empty, the caller read that as 'no explicit
Export-ModuleMember' and assumed export-all, and every PRIVATE function was then
reported as an under-declared export.

Measured, both ends pinned. RED 8/14 at base 2068b2b, GREEN 14/14 at work.
dogfood-reader.psm1: 13 false warnings -> 0, with the modelled 12-export surface
matching Import-Module ground truth exactly (25 defined - 12 exported = the 13).

Class, not instance: -Cmdlet was MEASURED to share the collection path and is
fixed by the same change. -Alias is measured NOT to share it -- alias names are
still deliberately kept out of the function set (BurntToast shape, 000128) and
tests pin that the list forms do not reopen it.

Mixed lists degrade rather than half-resolving. This was the hazard the fix could
have created: collecting only the literal half leaves a PARTIAL set that reads as
COMPLETE, which is worse than the silence it replaced. Pre-fix a mixed list
silently assumed export-all with no degrade recorded; it now degrades.

ACCEPTANCE (b) IS NOT MET, and that is recorded rather than worked around. On the
live oracle (155 manifests enumerated, 26 with a resolvable RootModule) hits fall
1088 -> 910, but hand-triage against Import-Module ground truth confirms 909 of
the 910 remaining are FALSE positives with ZERO true positives. The cause is a
SECOND, unrelated class this dispatch did not charter: a manifest's
FunctionsToExport is the final export gate, so a defined-but-unlisted function is
simply not exported, yet is still reported as under-declared. Pester is the clear
case -- 419 functions defined, manifest lists 26, PowerShell exports exactly 26,
and all 393 hits name functions it does not export.

Also found and NOT fixed: the dot-source degrade in this same helper is inert --
it matches CommandElements[0] -eq '.', but PowerShell carries the dot as the
CommandAst InvocationOperator, the same trap 000156 leg 3 banked after hitting it
in a guard and never swept from shipped code. Pinned as a characterization test so
it cannot change silently in either direction.

Claude-Session: https://claude.ai/code/session_01NCXh8Xms8ZKhdfKRBqSmaQ
Lockstep bump via scripts/bump-version.ps1 -Apply, and the [Unreleased] entry
moved under a dated [1.27.2] heading per docs/RELEASING.md.

Proven at this head: plugin.json == marketplace.json == 1.27.2 == the dated
CHANGELOG heading, with [Unreleased] left empty. tests/PowerShellLsp.Release.Tests.ps1
39/39 green, including the published-parity guard at 1.27.2.

No tag is cut here. The bump helper prints a manual tag pair as a FALLBACK for a
broken pipeline; it was deliberately not run, since a hand-cut tag is unsigned and
unattested and Gate 2 would refuse the pipeline until it were deleted. The release
pipeline cuts the tag itself, and triggering it stays Mike's.

Claude-Session: https://claude.ai/code/session_01NCXh8Xms8ZKhdfKRBqSmaQ
manderse21 added a commit that referenced this pull request Jul 29, 2026
Docs/close-out: true the ROADMAP to the 000159 outcome, with PR #108 still HELD (000160 leg 3)
@manderse21
manderse21 merged commit 49ce894 into main Jul 29, 2026
4 checks passed
@manderse21
manderse21 deleted the dispatch/000159-legs-1-2 branch July 29, 2026 17:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants