[Testing] Add PHPUnit Test Impact Analysis via jasonmccreary/phpunit-tia - #8305
Draft
TomasVotruba wants to merge 1 commit into
Draft
[Testing] Add PHPUnit Test Impact Analysis via jasonmccreary/phpunit-tia#8305TomasVotruba wants to merge 1 commit into
TomasVotruba wants to merge 1 commit into
Conversation
Only re-run tests hit by changed files. The RunWithTia trait goes into AbstractLazyTestCase, so every Rector test case picks it up. Fixture files never show up in the recorded coverage graph: *.php.inc is not executed code, and Source/ or Expected/ files are loaded by reflection. A FixtureResolver maps any changed file below a test directory to the closest test class above it, so a fixture-only change is not mistaken for 'affects nothing'.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports jasonmccreary/phpunit-tia (Pest's Test Impact Analysis engine, for PHPUnit 13) into the test suite. TIA records a test -> source coverage graph, then skips tests that no changed file can reach.
Wiring
The extension is registered in
phpunit.xml, and theRunWithTiatrait goes intoAbstractLazyTestCaseso every Rector test case - includingAbstractRectorTestCase- picks it up:Graph storage is
global(~/.phpunit-tia/<project>-<hash>/), so no.gitignoreentry is needed.Fixtures are invisible to a coverage graph
This is the part that needed work. A fixture-only change - the most common change in this repo - is exactly what TIA gets wrong out of the box:
*.php.incis never executed, so it gets no coverage edge and no.phpsuffix matchSource/andExpected/files are loaded by reflection, not by the testFixture/while the edges point atrules/So
SomeRectorTestwould be skipped as "unaffected" after editing its own fixture.phpunit-tia.phpregisters aFixtureResolverthat maps any changed file below a test directory to the closest test class above it:False positives here only cost a re-run; a false negative silently skips a real test. Covered by
tests/Tia/FixtureResolverTest.php.Please read before merging
src/Testingships..gitattributesexport-ignorestestsandrules-tests, but notsrc/Testing. Sincejasonmccreary/phpunit-tiaisrequire-dev, every repo that extendsAbstractLazyTestCase/AbstractRectorTestCasewithout that package installed will fatal on the missing trait. That meansrector-doctrine,rector-symfony,rector-phpunit,rector-downgrade-php, and any user with custom rule tests. Each needscomposer require --dev jasonmccreary/phpunit-tia- this should land as a coordinated set of PRs, which is why this is a draft.Moving the package to
requireinstead is not an option: it declares"conflict": {"phpunit/phpunit": "<13.2.6"}, which would block install for every Rector user not on PHPUnit 13.2.6+.composer-dependency-analyserflags the dev-dep-in-prod-code, ignored the same wayphpunit/phpunitalready is.Inert until a coverage driver exists. Recording needs pcov or Xdebug.
tests.yamlsetscoverage: none, so CI records nothing and skips nothing - it only prints one line to stderr per run. This is a local-dev speedup until someone decides whether a pcov job is worth it. Replay is decoupled from recording, so a recorded graph still works on a machine without a driver.Verified
Full suite green (5332 tests, 6838 assertions), same single pre-existing warning as
main. No measurable overhead: 34.3s on this branch vs 35.3s onmain. ECS, PHPStan, Rector andcomposer-dependency-analyserall clean.Not verified: the actual record -> skip cycle. No coverage extension is installed on this machine and none is available in
extension_dir, so TIA stayed inactive for every local run. The wiring, the resolver and the no-driver fallback path are exercised; the graph replay itself is not.