Support for granular OOP control of domain-layered test

[gregorFeigel]

Motivation

More complex systems might include domain layers where higher-level tests are only meaningful if lower-level functionality has succeeded. For example, access-permission tests depend on user/entity CRUD tests; running permission tests when CRUD has already failed produces non-test related errors, might be interpreted as misleading failures, and require unnecessary execution time.

The current framework executes tests in alphabetical order, and one has to rely on workarounds such as name-prefixing (a_, b_, …) or nested extensions to force execution order. This is fragile: renaming tests can break logical sequencing, and large (extended) suites become difficult to maintain. There is yet no supported way to declare that a suite or test should run only if another domain (suite/test) has completed.

Proposed solution

The introduction of a lightweight, opt-in mechanism for test-domain dependencies similar to existing .enabled(if:) semantics but driven by prior test results. Tests or suites could declare that they depend on the success of one or more lower-level domains.

@Suite("User CRUD", .tags(.crud))
struct UserCRUD { ... }

@Suite("Content CRUD", .tags(.crud))
struct ContentCRUD { ... }

@Test("PermissionCheck", .requires(.tag(.crud)))
func testPermission() { ... }

If the dependency domain fails, dependent tests are skipped automatically. If the dependency domain completes, dependent tests become eligible for execution in parallel like any other (unit) test.

This would:

• allows for predictable domain level sequencing without relying on name-ordering.
• clean the overall output: higher-level tests are skipped instead of generating cascading failures.
• reduced boilerplate: no need to recreate domain-level state defensively in each layer, since it would logically make no sense to the test them at all if the lower domain had failed.
• preserve parallelism and the concept of test isolation: the mechanism affects scheduling eligibility, not execution semantics.

The system could be implemented via tags, traits, or an explicit dependency field in the test plan document.

Alternatives considered

• Alphabetical prefixing (a_, b_, c_): Fragile to renaming, and scales poorly.
• Deep nesting of suites/extensions: Hard to read in large projects, obscures domain structure, and still only -merely express semantic dependencies.
• Runtime guards / setup checks inside tests: Adds boilerplate and duplicates logic across suites.

Additional information

No response

[grynspan]

Generally speaking, test state proceeds "inward" from a suite to its subsuites to their test functions to their test cases, and sharing state across tree branches like this is not something we want to support (although there are exceptions to every rule).

Could you structure such tests using .enabled(if:) instead so that the dependencies are modelled outside the test graph itself? e.g.

@Test(.enabled(if: CRUD.isAvailable))
func `test that requires CRUD`() { ... }

[grynspan]

See also https://forums.swift.org/t/pre-pitch-data-dependent-test-serialization/81251/