Skip to content

Add affected test selection to dotnet test - #55574

Merged
Evangelink merged 2 commits into
mainfrom
dev/amauryleve/plan-cts-dotnet-test
Aug 5, 2026
Merged

Add affected test selection to dotnet test#55574
Evangelink merged 2 commits into
mainfrom
dev/amauryleve/plan-cts-dotnet-test

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Summary

Adds an experimental, Microsoft.Testing.Platform-only affected-test workflow to dotnet test:

set DOTNET_CLI_ENABLE_AFFECTED_TESTS=1
dotnet test --collect-test-map
dotnet test --affected-tests

The SDK owns only the command-line and multi-module orchestration boundary. Repository analysis, test-map collection/storage, and affected-test filtering remain in a separately distributed MTP extension.

Relates to #55405. The composable MTP execution-filter API used by the extension was added by microsoft/testfx#10235.

Behavior

  • Adds gated, localized --collect-test-map and --affected-tests options for the MTP runner.
  • Keeps the operations mutually exclusive and rejects incompatible device, parallel-module, and minimum-test policies.
  • Runs test-map collection one module at a time.
  • Preserves forwarded argument ordering and validates recursive response files using each test application's effective working directory.
  • Requires response-file activation to resolve consistently across all test applications.
  • Marks SDK-launched children with DOTNET_CLI_TEST_AFFECTED_TESTS_MODE so the companion extension can reject activation injected through launch profiles, project run arguments, or test configuration.
  • Treats a valid empty affected selection as success while preserving failure for all-skipped runs and unexpected process failures.
  • Leaves ordinary dotnet test behavior unchanged when neither operation is requested.

Validation

  • src/Cli/dotnet/dotnet.csproj builds with 0 warnings and 0 errors.
  • test/dotnet.Tests/dotnet.Tests.csproj builds with 0 warnings and 0 errors.
  • 123 targeted parser, launch-contract, terminal-reporting, and related command tests pass.
  • Localization files regenerated with UpdateXlf.
  • git diff --check passes.
  • Six independent bug-focused review rounds were completed and all actionable findings were fixed.

Copilot AI review requested due to automatic review settings August 3, 2026 16:45
@Evangelink
Evangelink requested review from a team as code owners August 3, 2026 16:45
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
2 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an experimental, Microsoft.Testing.Platform-only affected-test workflow to dotnet test, gated behind DOTNET_CLI_ENABLE_AFFECTED_TESTS, with supporting validation, process-launch signaling, response-file detection, and terminal reporting changes.

Changes:

  • Adds --collect-test-map / --affected-tests options (hidden unless enabled), with mutual-exclusion and incompatibility validation.
  • Extends MTP orchestration to detect affected-test activation in forwarded response files and adjust behavior (e.g., collection forces single-module execution).
  • Updates terminal reporting + exit-code behavior to allow “valid empty selection” in affected-test mode, with targeted regression tests.
Show a summary per file
File Description
test/dotnet.Tests/CommandTests/Test/TestCommandParserTests.cs Adds parser/normalization/response-file detection unit coverage for affected-test options.
test/dotnet.Tests/CommandTests/Test/TestApplicationLaunchTests.cs Adds tests for affected-test option propagation and SDK “mode marker” environment variable behavior.
test/dotnet.Tests/CommandTests/Test/TerminalTestReporterTests.cs Adds terminal-summary tests for allowed-zero-test behavior in affected-test mode.
test/dotnet.Tests/CommandTests/Test/MTPHelpSnapshotTests.cs Stabilizes help snapshot by explicitly disabling affected-tests feature flag.
test/dotnet.Tests/CommandTests/Test/GivenDotnetTestSelectsDevice.cs Adds validation test for --list-devices incompatibility with affected-test operations.
src/Cli/Microsoft.DotNet.Cli.Definitions/Commands/Test/TestCommandDefinition.MicrosoftTestingPlatform.cs Introduces the two new options, feature gating, and definition-level validation.
src/Cli/Microsoft.DotNet.Cli.Definitions/CommandDefinitionStrings.resx Adds localized source strings for new definition-level help/validation messages.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.zh-Hant.xlf Regenerated localization entries for new definition strings (zh-Hant).
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.zh-Hans.xlf Regenerated localization entries for new definition strings (zh-Hans).
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.tr.xlf Regenerated localization entries for new definition strings (tr).
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.ru.xlf Regenerated localization entries for new definition strings (ru).
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.pt-BR.xlf Regenerated localization entries for new definition strings (pt-BR).
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.pl.xlf Regenerated localization entries for new definition strings (pl).
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.ko.xlf Regenerated localization entries for new definition strings (ko).
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.ja.xlf Regenerated localization entries for new definition strings (ja).
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.it.xlf Regenerated localization entries for new definition strings (it).
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.fr.xlf Regenerated localization entries for new definition strings (fr).
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.es.xlf Regenerated localization entries for new definition strings (es).
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.de.xlf Regenerated localization entries for new definition strings (de).
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.cs.xlf Regenerated localization entries for new definition strings (cs).
src/Cli/dotnet/Commands/Test/MTP/ITestHandler.cs Extends handler contract to expose test-application working directories for response-file validation.
src/Cli/dotnet/Commands/Test/MTP/MSBuildHandler.cs Implements working-directory enumeration for MSBuild-resolved test modules.
src/Cli/dotnet/Commands/Test/MTP/TestModulesFilterHandler.cs Implements working-directory enumeration (fallback) for --test-modules flow.
src/Cli/dotnet/Commands/Test/MTP/Options.cs Extends TestOptions with affected-test flags and forwarded-activation markers.
src/Cli/dotnet/Commands/Test/MTP/TestApplication.cs Marks child processes with affected-test mode env var and preserves forwarded argument ordering.
src/Cli/dotnet/Commands/Test/MTP/MicrosoftTestingPlatformTestCommand.cs Adds orchestration: validation, response-file detection, parallelism override for collection, and adjusted exit-code logic.
src/Cli/dotnet/Commands/Test/MTP/Terminal/TerminalTestReporterOptions.cs Adds AllowZeroTests option to control summary semantics in affected-test mode.
src/Cli/dotnet/Commands/Test/MTP/Terminal/TerminalTestReporter.cs Updates summary + per-assembly reporting to support allowed-zero-test runs and unexpected exit-code handling.
src/Cli/dotnet/Commands/CliCommandStrings.resx Adds user-facing CLI validation strings for affected-test mode + response-file consistency.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.zh-Hant.xlf Regenerated localization entries for new CLI strings (zh-Hant).
src/Cli/dotnet/Commands/xlf/CliCommandStrings.zh-Hans.xlf Regenerated localization entries for new CLI strings (zh-Hans).
src/Cli/dotnet/Commands/xlf/CliCommandStrings.tr.xlf Regenerated localization entries for new CLI strings (tr).
src/Cli/dotnet/Commands/xlf/CliCommandStrings.ru.xlf Regenerated localization entries for new CLI strings (ru).
src/Cli/dotnet/Commands/xlf/CliCommandStrings.pt-BR.xlf Regenerated localization entries for new CLI strings (pt-BR).
src/Cli/dotnet/Commands/xlf/CliCommandStrings.pl.xlf Regenerated localization entries for new CLI strings (pl).
src/Cli/dotnet/Commands/xlf/CliCommandStrings.ko.xlf Regenerated localization entries for new CLI strings (ko).
src/Cli/dotnet/Commands/xlf/CliCommandStrings.ja.xlf Regenerated localization entries for new CLI strings (ja).
src/Cli/dotnet/Commands/xlf/CliCommandStrings.it.xlf Regenerated localization entries for new CLI strings (it).
src/Cli/dotnet/Commands/xlf/CliCommandStrings.fr.xlf Regenerated localization entries for new CLI strings (fr).
src/Cli/dotnet/Commands/xlf/CliCommandStrings.es.xlf Regenerated localization entries for new CLI strings (es).
src/Cli/dotnet/Commands/xlf/CliCommandStrings.de.xlf Regenerated localization entries for new CLI strings (de).
src/Cli/dotnet/Commands/xlf/CliCommandStrings.cs.xlf Regenerated localization entries for new CLI strings (cs).

Copilot's findings

  • Files reviewed: 42/42 changed files
  • Comments generated: 0

@Evangelink
Evangelink enabled auto-merge August 3, 2026 17:04
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3c3b90fe-16c9-4122-9d31-caba5e1f8972
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3c3b90fe-16c9-4122-9d31-caba5e1f8972
@Evangelink
Evangelink merged commit 1c9fb67 into main Aug 5, 2026
21 checks passed
@Evangelink
Evangelink deleted the dev/amauryleve/plan-cts-dotnet-test branch August 5, 2026 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants