Skip to content

[Testing] Add PHPUnit Test Impact Analysis via jasonmccreary/phpunit-tia - #8305

Draft
TomasVotruba wants to merge 1 commit into
mainfrom
phpunit-tia
Draft

[Testing] Add PHPUnit Test Impact Analysis via jasonmccreary/phpunit-tia#8305
TomasVotruba wants to merge 1 commit into
mainfrom
phpunit-tia

Conversation

@TomasVotruba

Copy link
Copy Markdown
Member

Ports jasonmccreary/phpunit-tia (Pest's Test Impact Analysis engine, for PHPUnit 13) into the test suite. TIA records a test -> source coverage graph, then skips tests that no changed file can reach.

Wiring

The extension is registered in phpunit.xml, and the RunWithTia trait goes into AbstractLazyTestCase so every Rector test case - including AbstractRectorTestCase - picks it up:

 abstract class AbstractLazyTestCase extends TestCase
 {
+    // skips tests unaffected by the changed files, based on the recorded coverage graph
+    use RunWithTia {
+        RunWithTia::setUp as tiaSetUp;
+    }
+
     protected function setUp(): void
     {
+        $this->tiaSetUp();
+
         $this->includePreloadFilesAndScoperAutoload();
     }

Graph storage is global (~/.phpunit-tia/<project>-<hash>/), so no .gitignore entry is needed.

Fixtures are invisible to a coverage graph

This is the part that needed work. A fixture-only change - the most common change in this repo - is exactly what TIA gets wrong out of the box:

  • *.php.inc is never executed, so it gets no coverage edge and no .php suffix match
  • Source/ and Expected/ files are loaded by reflection, not by the test
  • the package's generic sibling-directory fallback misses too, because fixtures sit in Fixture/ while the edges point at rules/

So SomeRectorTest would be skipped as "unaffected" after editing its own fixture. phpunit-tia.php registers a FixtureResolver that maps any changed file below a test directory to the closest test class above it:

rules-tests/CodeQuality/Rector/If_/SomeRector/Fixture/nested/some_fixture.php.inc
  -> rules-tests/CodeQuality/Rector/If_/SomeRector/SomeRectorTest.php

False positives here only cost a re-run; a false negative silently skips a real test. Covered by tests/Tia/FixtureResolverTest.php.

Please read before merging

src/Testing ships. .gitattributes export-ignores tests and rules-tests, but not src/Testing. Since jasonmccreary/phpunit-tia is require-dev, every repo that extends AbstractLazyTestCase / AbstractRectorTestCase without that package installed will fatal on the missing trait. That means rector-doctrine, rector-symfony, rector-phpunit, rector-downgrade-php, and any user with custom rule tests. Each needs composer require --dev jasonmccreary/phpunit-tia - this should land as a coordinated set of PRs, which is why this is a draft.

Moving the package to require instead is not an option: it declares "conflict": {"phpunit/phpunit": "<13.2.6"}, which would block install for every Rector user not on PHPUnit 13.2.6+.

composer-dependency-analyser flags the dev-dep-in-prod-code, ignored the same way phpunit/phpunit already is.

Inert until a coverage driver exists. Recording needs pcov or Xdebug. tests.yaml sets coverage: none, so CI records nothing and skips nothing - it only prints one line to stderr per run. This is a local-dev speedup until someone decides whether a pcov job is worth it. Replay is decoupled from recording, so a recorded graph still works on a machine without a driver.

Verified

Full suite green (5332 tests, 6838 assertions), same single pre-existing warning as main. No measurable overhead: 34.3s on this branch vs 35.3s on main. ECS, PHPStan, Rector and composer-dependency-analyser all clean.

Not verified: the actual record -> skip cycle. No coverage extension is installed on this machine and none is available in extension_dir, so TIA stayed inactive for every local run. The wiring, the resolver and the no-driver fallback path are exercised; the graph replay itself is not.

Only re-run tests hit by changed files. The RunWithTia trait goes into
AbstractLazyTestCase, so every Rector test case picks it up.

Fixture files never show up in the recorded coverage graph: *.php.inc is
not executed code, and Source/ or Expected/ files are loaded by
reflection. A FixtureResolver maps any changed file below a test
directory to the closest test class above it, so a fixture-only change
is not mistaken for 'affects nothing'.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant