[UITests] Add UITest.Next suites (Image Resizer, Peek, File Explorer add‑ons) + local‑VM tooling and CI test‑signing - #49671
[UITests] Add UITest.Next suites (Image Resizer, Peek, File Explorer add‑ons) + local‑VM tooling and CI test‑signing#49671khmyznikov wants to merge 48 commits into
Conversation
khmyznikov
added
Area-Tests
| & "$(build.sourcesdirectory)\.pipelines\signSparsePackages.ps1" -PackageRoot @( | ||
| "$(Pipeline.Workspace)\$(TestArtifactsName)", | ||
| "$env:ProgramFiles\PowerToys\WinUI3Apps", | ||
| "$env:LOCALAPPDATA\PowerToys\WinUI3Apps") |
There was a problem hiding this comment.
Make sparse-package setup a required precondition for Image Resizer jobs
Severity: high
This step treats signing/trust failures as best-effort, but the new Windows 11 Image Resizer test explicitly has no classic-menu fallback and there is no ModernRegistered() guard in the suite. A signing failure is therefore reported later as a misleading product/test failure; additionally, the installed roots passed here omit ImageResizerContextMenuPackage.msix, which is installed at the PowerToys root rather than under WinUI3Apps. Make the Image Resizer/all-modules jobs require a successfully trusted ImageResizerContextMenuPackage.msix, search the PowerToys install roots recursively, and have signSparsePackages.ps1 throw when the required package cannot be signed or trusted.
| [int]$WinRmPort = 15987, | ||
| [switch]$UseHttp, | ||
| [string]$CredentialPath = (Join-Path $env:LOCALAPPDATA 'PowerToysUiTestVm-Win11\admin.credential.xml') | ||
| ) | ||
|
|
||
| $ErrorActionPreference = 'Stop' | ||
| if (-not (Test-Path $CredentialPath)) { | ||
| throw "Credential file not found: $CredentialPath. Point -CredentialPath at the VM's admin.credential.xml." | ||
| } | ||
| $credential = Import-Clixml $CredentialPath | ||
|
|
||
| if ($UseHttp) { | ||
| $session = New-PSSession -ConnectionUri "http://127.0.0.1:$WinRmPort/wsman" -Authentication Basic -Credential $credential | ||
| } | ||
| else { | ||
| $sessionOption = New-PSSessionOption -SkipCACheck -SkipCNCheck -SkipRevocationCheck | ||
| $session = New-PSSession -ConnectionUri "https://127.0.0.1:$WinRmPort/wsman" -Authentication Basic -Credential $credential -SessionOption $sessionOption | ||
| } |
There was a problem hiding this comment.
Align the guest helper with the shipped VM connection settings
Severity: medium
The scaffold/controller default to HTTPS on port 15986, while this helper defaults to 15987; its HTTP path also uses Basic authentication even though the controller and documentation use Negotiate. As written, the helper fails against a stock scaffold and the migration fallback uses the wrong authentication mode. Align the default port and session construction with Invoke-LocalVmUiTest.ps1.
| [int]$WinRmPort = 15987, | |
| [switch]$UseHttp, | |
| [string]$CredentialPath = (Join-Path $env:LOCALAPPDATA 'PowerToysUiTestVm-Win11\admin.credential.xml') | |
| ) | |
| $ErrorActionPreference = 'Stop' | |
| if (-not (Test-Path $CredentialPath)) { | |
| throw "Credential file not found: $CredentialPath. Point -CredentialPath at the VM's admin.credential.xml." | |
| } | |
| $credential = Import-Clixml $CredentialPath | |
| if ($UseHttp) { | |
| $session = New-PSSession -ConnectionUri "http://127.0.0.1:$WinRmPort/wsman" -Authentication Basic -Credential $credential | |
| } | |
| else { | |
| $sessionOption = New-PSSessionOption -SkipCACheck -SkipCNCheck -SkipRevocationCheck | |
| $session = New-PSSession -ConnectionUri "https://127.0.0.1:$WinRmPort/wsman" -Authentication Basic -Credential $credential -SessionOption $sessionOption | |
| } | |
| param( | |
| [Parameter(Mandatory)][scriptblock]$ScriptBlock, | |
| [object[]]$ArgumentList = @(), | |
| [int]$WinRmPort = 15986, | |
| [switch]$UseHttp, | |
| [string]$CredentialPath = (Join-Path $env:LOCALAPPDATA 'PowerToysUiTestVm-Win11\admin.credential.xml') | |
| ) | |
| $ErrorActionPreference = 'Stop' | |
| if (-not (Test-Path $CredentialPath)) { | |
| throw "Credential file not found: $CredentialPath. Point -CredentialPath at the VM's admin.credential.xml." | |
| } | |
| $credential = Import-Clixml $CredentialPath | |
| $scheme = if ($UseHttp) { 'http' } else { 'https' } | |
| $authentication = if ($UseHttp) { 'Negotiate' } else { 'Basic' } | |
| $sessionOption = if ($UseHttp) { | |
| New-PSSessionOption | |
| } | |
| else { | |
| New-PSSessionOption -SkipCACheck -SkipCNCheck -SkipRevocationCheck | |
| } | |
| $session = New-PSSession ` | |
| -ConnectionUri "${scheme}://127.0.0.1:$WinRmPort/wsman" ` | |
| -Authentication $authentication ` | |
| -Credential $credential ` | |
| -SessionOption $sessionOption |
| - [UI-tests migration skill](../../../.github/skills/ui-tests-migration/SKILL.md): create new | ||
| `.Next` test projects, port legacy WinAppDriver tests, design stable selectors/waits/lifecycle, and | ||
| prepare tests for CI. | ||
| - [Windows Sandbox UI-tests skill](../../../.github/skills/windows-sandbox-ui-tests/SKILL.md): enable |
There was a problem hiding this comment.
Replace the non-existent Sandbox workflow with the shipped local-VM skill
Severity: medium
This guide links to .github/skills/windows-sandbox-ui-tests, but that skill and its scripts do not exist in this PR. The repository actually adds ui-tests-local-vm, so contributors following the documented .Next workflow hit missing paths at both this link and the later Sandbox commands. Rewrite this section and the “Running .Next tests” walkthrough to use ui-tests-local-vm/SKILL.md, Initialize-LocalVm.ps1, and Invoke-LocalVmUiTest.ps1.
| toggle = SetModuleEnabled(toggle, true); | ||
| explorer = OpenExplorer(folder); | ||
| AssertContextMenuPresence(explorer, new[] { fixture }, expected: true); |
There was a problem hiding this comment.
Refresh Explorer after re-enabling the context-menu handler
Severity: medium
EnsureContextMenuHandlerLoaded() documents that an existing Explorer process does not see runtime handler registration until the shell restarts, but its one-time guard has already been consumed by the disabled-state assertion. The re-enabled assertion can therefore fail even when Image Resizer registers correctly. Reset the guard so OpenExplorer performs a second refresh for the new registration state.
| toggle = SetModuleEnabled(toggle, true); | |
| explorer = OpenExplorer(folder); | |
| AssertContextMenuPresence(explorer, new[] { fixture }, expected: true); | |
| toggle = SetModuleEnabled(toggle, true); | |
| contextMenuExplorerRefreshed = false; | |
| explorer = OpenExplorer(folder); | |
| AssertContextMenuPresence(explorer, new[] { fixture }, expected: true); |
| { | ||
| ForcePipelineLightTheme(); | ||
|
|
||
| SettingsConfigHelper.UpdateModuleSettings( |
There was a problem hiding this comment.
Restore module settings after the new UI-test suites finish
Severity: medium
This static setup rewrites the real Peek settings file, but class cleanup restores only the theme registry value. FileExplorerAddonsTests has the same issue for the File Explorer settings file. Because UI-test executables run sequentially in the same user profile—and local VMs are persistent—these changes leak into later suites and reruns, creating order-dependent behavior. Snapshot each module's settings.json before mutation and restore/delete it in class cleanup; a reusable SettingsConfigHelper.PreserveModuleSettings scope with unit coverage can serve both suites, matching the existing Image Resizer cleanup pattern.
| { | ||
| try | ||
| { | ||
| if (browser.HWND != explorerWindow.ToInt64() || browser.Document is not Shell32.IShellFolderViewDual2 folderView) |
There was a problem hiding this comment.
Release the Shell folder-view COM objects used by polling helpers
Severity: medium
The selection/view helpers release each IWebBrowserApp, but browser.Document creates a separate folder-view RCW that is never released. These methods are called repeatedly by stable waits and recovery loops, so the leaked RCWs can accumulate and eventually cause flaky Shell COM failures. Capture browser.Document in a local variable and release it in the same finally block before releasing browser; apply the same pattern in TryGetSelection, TryGetView, TrySetView, and TrySetSelection (around lines 136, 226, 264, and 306).
| <TextBlock | ||
| x:Name="PreviewStateAutomationPeer" | ||
| Width="1" | ||
| Height="1" | ||
| HorizontalAlignment="Left" | ||
| VerticalAlignment="Top" | ||
| AutomationProperties.AutomationId="PreviewStateAutomationPeer" | ||
| IsHitTestVisible="False" | ||
| Opacity="0" | ||
| Text="{x:Bind GetPreviewStateText(Previewer.State), Mode=OneWay}" /> |
There was a problem hiding this comment.
Keep the hidden test probe out of the screen-reader control view
Severity: medium
The transparent 1×1 TextBlock remains visible in the accessibility control view, so assistive technology can announce test-only preview-state text in production. Mark it as Raw so UI Automation tests can still query it without adding spoken content for users.
| <TextBlock | |
| x:Name="PreviewStateAutomationPeer" | |
| Width="1" | |
| Height="1" | |
| HorizontalAlignment="Left" | |
| VerticalAlignment="Top" | |
| AutomationProperties.AutomationId="PreviewStateAutomationPeer" | |
| IsHitTestVisible="False" | |
| Opacity="0" | |
| Text="{x:Bind GetPreviewStateText(Previewer.State), Mode=OneWay}" /> | |
| <TextBlock | |
| x:Name="PreviewStateAutomationPeer" | |
| Width="1" | |
| Height="1" | |
| HorizontalAlignment="Left" | |
| VerticalAlignment="Top" | |
| AutomationProperties.AccessibilityView="Raw" | |
| AutomationProperties.AutomationId="PreviewStateAutomationPeer" | |
| IsHitTestVisible="False" | |
| Opacity="0" | |
| Text="{x:Bind GetPreviewStateText(Previewer.State), Mode=OneWay}" /> |
| public static void MoveWindow(IntPtr hWnd, int x, int y) | ||
| { | ||
| var (left, top, right, bottom) = GetWindowBounds(hWnd); | ||
| SetWindowPos(hWnd, IntPtr.Zero, x, y, right - left, bottom - top, SWP_NOZORDER | SWP_NOACTIVATE); | ||
| } |
There was a problem hiding this comment.
Preserve window size without depending on a successful bounds query
Severity: medium
GetWindowBounds returns (0,0,0,0) when GetWindowRect fails, so this helper can unexpectedly resize the target window to 0×0 while trying only to move it. SetWindowPos already supports preserving size directly; use SWP_NOSIZE and avoid the fragile read entirely.
| public static void MoveWindow(IntPtr hWnd, int x, int y) | |
| { | |
| var (left, top, right, bottom) = GetWindowBounds(hWnd); | |
| SetWindowPos(hWnd, IntPtr.Zero, x, y, right - left, bottom - top, SWP_NOZORDER | SWP_NOACTIVATE); | |
| } | |
| /// <summary>Move a window to explicit screen coordinates while preserving its current size.</summary> | |
| public static void MoveWindow(IntPtr hWnd, int x, int y) => | |
| SetWindowPos(hWnd, IntPtr.Zero, x, y, 0, 0, SWP_NOSIZE | SWP_NOZORDER | SWP_NOACTIVATE); |
2 participants
Summary
Adds end‑to‑end UI tests on the
Microsoft.PowerToys.UITest.Next(winappcli) framework for threemodules, grows the shared
.Nexttest framework with the helpers those suites needed, and adds theCI plumbing that lets shell‑extension tests exercise the real Windows 11 modern context menu.
Also ships two agent skills that document how to write and run these tests.
Product runtime behavior is unchanged — the only product edits are test‑observability hooks in Peek
and a unit‑test project exclude.
Closes: #40660 #49424 #40661
What's added
New UI test suites
src/modules/imageresizer/tests/ImageResizer.UITests: context‑menu enable/disabletracking, the resize dialog, custom presets, every fit mode, every unit, filename format, keep‑date,
shrink‑only, replace‑in‑place, and orientation.
src/modules/peek/Peek.UITests.Next: file‑preview coverage across image/text/archive/markdown types with per‑arch visual baselines.
src/modules/previewpane/PreviewPane.UITests: Preview Pane handlers andthumbnail providers.
UITestAutomation.NextframeworkExplorerShell(Shell selection/view‑mode interop),WaitHelper(structured stablewaits),
WindowControl(foreground/context‑menu/process control),VisualAssert(image compare),WindowHelper.Session,UITestBase,SettingsConfigHelper,WinappCli.UITestAutomation.Next.UnitTestsproject covering the new wait/settings/CLI helpers.CI — sign sparse MSIX so the modern menu registers
.pipelines/signSparsePackages.ps1— self‑signs each sparse context‑menu MSIX with apublisher‑matching test certificate and force‑trusts it (machine stores), so
AddPackageByUriAsyncsucceeds on otherwise‑unsigned PR builds. Robustsigntooldiscovery with aNuGet fallback; test‑only trust that asserts no security.
.pipelines/v2/templates/job-test-project.ymlas a best‑effort step covering therun‑in‑place, machine‑install, and per‑user‑install locations. Signs nothing it can't (skips
already‑signed packages) and never fails the job.
Product changes (test observability only)
FilePreview.xaml/.xaml.cs— a namedLoadingIndicatorand a hidden automation peerthat exposes the current preview state as text, so tests can read load state deterministically. No
runtime behavior change.
ImageResizer.UnitTests.csproj— exclude the siblingImageResizer.UITests\**folder from theunit‑test compilation.
Agent skills & docs
ui-tests-local-vmskill — run.Nextsuites in persistent dockur/windows VMs: setup,agentic loop, image customization, troubleshooting, the shell‑extension signing reference, plus
controller/guest scripts and VM templates.
ui-tests-migrationskill — WinAppDriver/Selenium →.Nextporting guidance(CI stability, Explorer/shell‑extension test design, patterns & pitfalls).
doc/devdocs/development/ui-tests.md— updated for the.Nextworkflow.Testing
and per‑user install legs).
Reviewer notes
best‑effort, so it can only add modern‑menu coverage and never regress the job.