diff --git a/docs/core/testing/selective-unit-tests.md b/docs/core/testing/selective-unit-tests.md index 7a5d0eee4ea68..8f6dd5633b163 100644 --- a/docs/core/testing/selective-unit-tests.md +++ b/docs/core/testing/selective-unit-tests.md @@ -2,7 +2,8 @@ title: Run selected unit tests description: How to use a filter expression to run selected unit tests with the dotnet test command in .NET Core. author: smadala -ms.date: 10/29/2021 +ms.date: 07/19/2026 +ai-usage: ai-assisted zone_pivot_groups: unit-testing-framework-set-one ms.topic: reference --- @@ -31,7 +32,7 @@ dotnet test --filter | -------------- | -------------------- | | MSTest | `FullyQualifiedName`
`Name`
`ClassName`
`Priority`
`TestCategory` | | xUnit | `FullyQualifiedName`
`DisplayName`
`Traits` | - | Nunit | `FullyQualifiedName`
`Name`
`Priority`
`TestCategory` | + | NUnit | `FullyQualifiedName`
`Name`
`Priority`
`TestCategory`
`Category`
`Property` | * **Operators** @@ -44,6 +45,21 @@ dotnet test --filter ## Character escaping +Use escape sequences to represent filter operator characters in a filter value. These escape sequences are separate from any escaping your shell requires. + +| Escape sequence | Represents | +| --------------- | -------------------- | +| `\\` | `\` | +| `\(` | `(` | +| `\)` | `)` | +| `\&` | `&` | +| `\|` | | | +| `\=` | `=` | +| `\!` | `!` | +| `\~` | `~` | + +To escape filter values programmatically, use the `Microsoft.VisualStudio.TestPlatform.ObjectModel.Utilities.FilterHelper.Escape` API from the `Microsoft.VisualStudio.TestPlatform.ObjectModel` NuGet package. + To use an exclamation mark (`!`) in a filter expression, you have to escape it in some Linux or macOS shells by putting a backslash in front of it (`\!`). For example, the following filter skips all tests in a namespace that contains `IntegrationTests`: ```dotnetcli @@ -56,6 +72,8 @@ For `FullyQualifiedName` values that include a comma for generic type parameters dotnet test --filter "FullyQualifiedName=MyNamespace.MyTestsClass.MyTestMethod" ``` +Because your shell also parses the filter expression, quote the entire `--filter` value whenever it contains characters your shell treats specially, such as `<`, `>`, or `,`. Quoting is required in PowerShell, where the comma is the array operator. + For `Name` or `DisplayName`, use the URL encoding for the special characters. For example, to run a test with the name `MyTestMethod` and a string value `"text"`, use the following filter: ```dotnetcli @@ -228,7 +246,7 @@ To run tests that have either a `** (Available since .NET 5.0 SDK) The type of crash dump to be collected. Supported dump types are `full` (default), and `mini`. Implies `--blame-crash`. @@ -146,11 +148,49 @@ Where `Microsoft.NET.Test.Sdk` is the test host, `xunit` is the test framework. When no unit is used (for example, 5400000), the value is assumed to be in milliseconds. When used together with data driven tests, the timeout behavior depends on the test adapter used. For xUnit, NUnit, and MSTest 2.2.4+, the timeout is renewed after every test case. For MSTest before version 2.2.4, the timeout is used for all test cases. This option is supported on Windows with `netcoreapp2.1` and later, on Linux with `netcoreapp3.1` and later, and on macOS with `net5.0` or later. Implies `--blame` and `--blame-hang`. + You can also configure blame in a `.runsettings` file and pass it with `--settings`. The `.runsettings` file supports the same blame behavior and additional keys that aren't exposed as top-level `dotnet test` switches, such as `CollectDumpOnTestSessionHang` and `MonitorPostmortemDebugger`. For more information, see [Blame data collector](https://github.com/microsoft/vstest/blob/main/docs/extensions/blame-datacollector.md). + + ```xml + + + + + + + + + + + + + + ``` + + The following tables map the blame options to their `dotnet test` switches and `.runsettings` elements. For the complete blame collector reference, see [Blame data collector](https://github.com/microsoft/vstest/blob/main/docs/extensions/blame-datacollector.md). + + Crash dump options: + + | Behavior | `dotnet test` switch | `.runsettings` element | + | --- | --- | --- | + | Collect a crash dump | `--blame-crash` | `` | + | Collect even on a clean exit | `--blame-crash-collect-always` | `` | + | Dump type (`mini`, `full`; default `full`) | `--blame-crash-dump-type` | `` | + + Hang dump options: + + | Behavior | `dotnet test` switch | `.runsettings` element | + | --- | --- | --- | + | Collect a hang dump | `--blame-hang` | `` | + | Timeout before the hang dump (default `1h`) | `--blame-hang-timeout` | `` | + | Hang dump type (`mini`, `full`, `none`) | `--blame-hang-dump-type` | `` | + + For Microsoft.Testing.Platform (MTP) test apps, the `--blame-*` switches and the blame data collector don't apply. MTP uses `--crashdump`, `--hangdump`, and `--hangdump-timeout` from the `Microsoft.Testing.Extensions.CrashDump` and `Microsoft.Testing.Extensions.HangDump` packages. For more information, see [dotnet test with MTP](dotnet-test-mtp.md). + - [!INCLUDE [configuration](includes/cli-configuration.md)] - **`--collect `** - Enables data collector for the test run. For more information, see [Monitor and analyze test run](https://aka.ms/vstest-collect). + Enables a data collector for the test run. For more information, including the Event Log data collector and guidance for authoring your own data collector, see [Monitor and analyze test run](https://aka.ms/vstest-collect). For example you can collect code coverage by using the `--collect "Code Coverage"` option. For more information, see [Use code coverage](/visualstudio/test/using-code-coverage-to-determine-how-much-code-is-being-tested), [Customize code coverage analysis](/visualstudio/test/customizing-code-coverage-analysis), and [GitHub issue dotnet/docs#34479](https://github.com/dotnet/docs/issues/34479). @@ -160,6 +200,8 @@ Where `Microsoft.NET.Test.Sdk` is the test host, `xunit` is the test framework. Enables diagnostic mode for the test platform and writes diagnostic messages to the specified file and to files next to it. The process that is logging the messages determines which files are created, such as `*.host_.txt` for test host log, and `*.datacollector_.txt` for data collector log. + To set the trace level, append `;tracelevel=` to the log file name, for example `--diag log.txt;tracelevel=verbose`. The allowed values for `tracelevel` are `off`, `error`, `warning`, `info`, and `verbose`. The default value is `verbose`. + - [!INCLUDE [disable-build-servers](includes/cli-disable-build-servers.md)] - **`-e|--environment `** @@ -254,7 +296,9 @@ Where `Microsoft.NET.Test.Sdk` is the test host, `xunit` is the test framework. Example: `dotnet test -- MSTest.DeploymentEnabled=false MSTest.MapInconclusiveToFailed=True` - For more information, see [Passing RunSettings arguments through command line](https://github.com/Microsoft/vstest-docs/blob/main/docs/RunSettingsArguments.md). + Starting with the .NET 5 SDK, you can also set `TestRunParameters` from the command line, for example: `dotnet test -- TestRunParameters.Parameter(name="myParam", value="value")`. `RunSettings` arguments take precedence over values from a `.runsettings` file. + + For more information, see [Passing RunSettings arguments through command line](https://github.com/microsoft/vstest/blob/main/docs/RunSettingsArguments.md). ## Examples @@ -357,8 +401,10 @@ Where `Microsoft.NET.Test.Sdk` is the test host, `xunit` is the test framework. | Test Framework | Supported properties | | -------------- | --------------------------------------------------------------------------------------------------------- | | MSTest |
  • FullyQualifiedName
  • Name
  • ClassName
  • Priority
  • TestCategory
| -| xUnit |
  • FullyQualifiedName
  • DisplayName
  • Category
| -| NUnit |
  • FullyQualifiedName
  • Name
  • Category
  • Priority
| +| xUnit |
  • FullyQualifiedName
  • DisplayName
  • Traits
| +| NUnit |
  • FullyQualifiedName
  • Name
  • Priority
  • TestCategory
  • Category
  • Property
| + +For xUnit, a trait defined with `[Trait("key", "value")]` is filtered by its key (for example, `[Trait("Category", "bvt")]` is matched with `--filter Category=bvt`). For NUnit, `TestCategory` and `Category` are equivalent, and a property defined with `[Property("key", "value")]` is filtered by its key. The `` describes the relationship between the property and the value: @@ -384,10 +430,24 @@ You can enclose expressions in parenthesis when using conditional operators (for For more information and examples on how to use selective unit test filtering, see [Running selective unit tests](../testing/selective-unit-tests.md). +## Exit codes + +When you run tests through the VSTest path, `dotnet test` reports the outcome with one of two exit codes: + +| Exit code | Meaning | +| --------- | ------- | +| `0` | Success. The requested operation completed and, for a test run, all executed tests passed. | +| `1` | Failure. For example, one or more tests failed, a run error was reported, the command line was invalid, a test source couldn't be loaded, or the run was aborted or canceled. | + +The underlying `vstest.console` process never returns any other value. + +When discovery finds no matching tests, the run prints a warning rather than an error and still returns `0` by default. To make a run that discovers or selects zero tests return `1` instead, set `RunConfiguration.TreatNoTestsAsError` to `true` in a `.runsettings` file. + ## See also - [Frameworks and Targets](../../standard/frameworks.md) - [.NET Runtime Identifier (RID) catalog](../rid-catalog.md) - [Passing runsettings arguments through commandline](https://github.com/microsoft/vstest/blob/main/docs/RunSettingsArguments.md) +- [VSTest environment variables (microsoft/vstest)](https://github.com/microsoft/vstest/blob/main/docs/environment-variables.md) - [dotnet test](dotnet-test.md) - [dotnet test with MTP](dotnet-test-mtp.md) diff --git a/docs/core/tools/dotnet-vstest.md b/docs/core/tools/dotnet-vstest.md index 1a29ffb81e98b..a9b1ec04f04f5 100644 --- a/docs/core/tools/dotnet-vstest.md +++ b/docs/core/tools/dotnet-vstest.md @@ -1,7 +1,8 @@ --- title: dotnet vstest command description: The dotnet vstest command builds a project and all of its dependencies. -ms.date: 10/28/2025 +ms.date: 07/19/2026 +ai-usage: ai-assisted --- # dotnet vstest @@ -17,20 +18,20 @@ ms.date: 10/28/2025 ## Synopsis ```dotnetcli -dotnet vstest [] [--Blame] [--Diag ] +dotnet vstest [] [--Blame] [--Collect ] + [--Diag ] [-e|--Environment ] [--Framework ] [--InIsolation] [-lt|--ListTests ] [--logger ] [--Parallel] - [--ParentProcessId ] [--Platform] - [--Port ] [--ResultsDirectory] [--Settings ] - [--TestAdapterPath ] [--TestCaseFilter ] - [--Tests ] [[--] ...] + [--Platform ] [--ResultsDirectory ] + [--Settings ] [--TestAdapterPath ] + [--TestCaseFilter ] [--Tests ] [[--] ...] dotnet vstest -?|--Help ``` ## Description -The `dotnet vstest` command runs the `VSTest.Console` command-line application to run automated unit tests. +The `dotnet vstest` command runs the `VSTest.Console` command-line application to run automated unit tests. The options in this article are forwarded to `vstest.console`. For the complete list of command-line options, see [vstest.console.exe command line options](https://github.com/microsoft/vstest/blob/main/docs/commandline.md). ## Arguments @@ -44,10 +45,18 @@ The `dotnet vstest` command runs the `VSTest.Console` command-line application t Runs the tests in blame mode. This option is helpful in isolating the problematic tests causing test host to crash. It creates an output file in the current directory as *Sequence.xml* that captures the order of tests execution before the crash. +- **`--Collect `** + + Enables a data collector for the test run. For example, `--Collect "Code Coverage"` collects code coverage with the built-in Visual Studio collector, and `--Collect "XPlat Code Coverage"` uses the cross-platform Coverlet collector. For more information, see [Monitor and analyze test run](https://aka.ms/vstest-collect). + - **`--Diag `** Enables verbose logs for the test platform. Logs are written to the provided file. +- **`-e|--Environment `** + + Sets the value of an environment variable for the test host process. Creates the variable if it doesn't exist, overrides it if it does. Using this option forces the tests to run in an isolated process. Specify the option multiple times to set multiple variables. + - **`--Framework `** Target .NET Framework version used for test execution. An example of a valid value is `.NETFramework,Version=v7.0`. Other supported values are `Framework40`, `Framework45`, `FrameworkCore10`, and `FrameworkUap10`. @@ -86,19 +95,11 @@ The `dotnet vstest` command runs the `VSTest.Console` command-line application t Run tests in parallel. By default, all available cores on the machine are available for use. Specify an explicit number of cores by setting the `MaxCpuCount` property under the `RunConfiguration` node in the *runsettings* file. -- **`--ParentProcessId `** - - Process ID of the parent process responsible for launching the current process. - - **`--Platform `** - Target platform architecture used for test execution. Valid values are `x86`, `x64`, and `ARM`. - -- **`--Port `** - - Specifies the port for the socket connection and receiving the event messages. + Target platform architecture used for test execution. Valid values are `x86`, `x64`, `ARM`, `ARM64`, `S390x`, `Ppc64le`, `RiscV64`, and `LoongArch64`. -- **`--ResultsDirectory:`** +- **`--ResultsDirectory `** Test results directory will be created in specified path if not exists. @@ -112,7 +113,7 @@ The `dotnet vstest` command runs the `VSTest.Console` command-line application t - **`--TestCaseFilter `** - Run tests that match the given expression. `` is of the format `Operator[|&]`, where Operator is one of `=`, `!=`, or `~`. Operator `~` has 'contains' semantics and is applicable for string properties like `DisplayName`. Parentheses `()` are used to group subexpressions. For more information, see [TestCase filter](https://github.com/Microsoft/vstest-docs/blob/main/docs/filter.md). + Run tests that match the given expression. `` is of the format `Operator[|&]`, where Operator is one of `=`, `!=`, or `~`. Operator `~` has 'contains' semantics and is applicable for string properties like `DisplayName`. Parentheses `()` are used to group subexpressions. For more information, see [TestCase filter](https://github.com/microsoft/vstest/blob/main/docs/filter.md). - **`--Tests `** @@ -164,4 +165,6 @@ dotnet vstest /Tests:TestMethod1,TestMethod2 ## See also -- [VSTest.Console.exe command-line options](/visualstudio/test/vstest-console-options) +- [VSTest.Console.exe command-line options](/visualstudio/test/vstest-console-options) - the official Microsoft Learn reference for the `VSTest.Console.exe` tool, including options that `dotnet vstest` doesn't expose. Start here for stable, curated documentation. +- [vstest.console.exe command line options (microsoft/vstest)](https://github.com/microsoft/vstest/blob/main/docs/commandline.md) - the source-derived reference in the `microsoft/vstest` repo. Use it for the most up-to-date option details and internal switches, plus links to related runsettings, filtering, and diagnostics docs. +- [VSTest environment variables (microsoft/vstest)](https://github.com/microsoft/vstest/blob/main/docs/environment-variables.md) - environment variables that VSTest and the test host recognize, such as `VSTEST_DUMP_FORCEPROCDUMP` and the `VSTEST_DOTNET_ROOT_*` variables the .NET SDK sets when it launches `vstest.console`.