Drop the memoized TypeCombinator results whenever the process-wide reflection provider, PHP version or feature toggles change - #6171
Conversation
…reflection provider, PHP version or feature toggles change
- `ValidateIgnoredErrorsExtension::loadConfiguration()` resolves the types named in `ignoreErrors` patterns under a throwaway `DummyReflectionProvider` and `PhpVersion`. Its `finally` restored the accessors and called `ObjectType::resetCaches()`, but not `TypeCombinator::clearCache()` — so with the turbo extension active (which is what installs the `TypeCombinatorCache` memo) every union/intersection computed against the dummy provider stayed memoized and was handed back to the real analysis.
- Moved the invalidation into the mutators of the state that type operations read, so no call site can forget it: `ReflectionProviderStaticAccessor::registerInstance()`, `PhpVersionStaticAccessor::registerInstance()`, `BleedingEdgeToggle::setBleedingEdge()` and `ReportUnsafeArrayStringKeyCastingToggle::setLevel()` now all clear the memo. This is the invariant `ContainerFactory::postInitializeContainer()` already documents ("a memoized result is only valid for the state it was computed under").
- `BleedingEdgeToggle::withBleedingEdge()` now writes the toggle through `setBleedingEdge()` on both the set and the restore instead of assigning the static directly, so the temporary flip invalidates the memo too.
- Kept an explicit `TypeCombinator::clearCache()` in `ValidateIgnoredErrorsExtension`'s `finally`: when the extension runs for the first container of the process there is no original accessor instance to restore, so nothing else would clear.
- Probed the other sites that swap the same global state: `ContainerFactory::postInitializeContainer()` and `StubValidator` (which re-runs it) already cleared correctly; `BetterReflection::populate()` has no other caller.
- Regression test `tests/PHPStan/Type/GlobalStateCacheInvalidationTest.php` reproduces the leak (fails with the extension loaded before the fix, passes after; a no-op without it).
ondrejmirtes
left a comment
There was a problem hiding this comment.
- So what was the order of events that lead to wrong TypeCombinator results? And why only in turbo mode?
- E2E test in e2e-tests.yml (+ probably ran with turbo extension in the same workflow as make tests and make phpstan run after Turbo is compiled) would be better than this PHPUnit test.
- Would be nicer to clear the cache in postInitializeContainer.
…ss-wide reflection provider, PHP version or feature toggles change" This reverts commit f3b65e5. The premise does not hold. `ValidateIgnoredErrorsExtension::loadConfiguration()` runs while the container is being compiled, and `ContainerFactory::create()` calls `postInitializeContainer()` on the very next line — which already drops the memo as its last step. Everything the extension memoizes under the throwaway `DummyReflectionProvider` is therefore gone before the container is handed to anything that analyses code. Verified with a locally built `phpstan_turbo.so` (1ecf6e0) loaded through `php.ini`, so the workers inherit it (`phpstan diagnose`: "Turbo extension: enabled", "workers inherit it"): the issue's reproducer, both as a single file and fanned out to 60 files so the run goes parallel, reports no `method.notFound`/`staticMethod.notFound` with the patch reverted. `ReflectionProviderStaticAccessor` and `PhpVersionStaticAccessor` are also the wrong place for the invalidation: `postInitializeContainer()` registers both *before* it sets the feature toggles, so a clear there runs too early to be the guard while looking like one.
…tion memo needs dropping Compiling the container runs ValidateIgnoredErrorsExtension, which resolves the types named in the ignoreErrors patterns under a throwaway DummyReflectionProvider and PhpVersion. That looks like a second site that has to invalidate the memo, but postInitializeContainer() is the next thing create() does and already drops it — note that, so the next reader does not add a redundant clear to the accessors or the toggles, where it would run before the toggles are set anyway.
|
Pushed as 1) The order of events — there isn't one. The premise was wrong.I built No $container = $configurator->createContainer()->getByType(Container::class); // ← compiles: ValidateIgnoredErrorsExtension runs here
$this->validateParameters(...);
self::postInitializeContainer($container); // ← ends with TypeCombinator::clearCache()
Everything memoized under the The PHPUnit test did fail before the patch — but it hand-builds the swap by calling 2) & 3) What I pushedSince there is no reachable failure, an e2e test would pass with and without the patch — a green test that can never fail is worse than none. So instead of writing one:
Where that leaves #15037Unexplained. The symptom — class-level |
Summary
With the turbo extension active, class-level
@method/@propertyPHPDoc and other type information could be silently wrong in worker processes: types resolved while PHPStan had a throwaway reflection provider installed were memoized byTypeCombinatorCacheand then handed back to the real analysis, producing falsemethod.notFound/staticMethod.notFound/property.notFound/class.notFounderrors.The memo only exists when the extension is active, which is why
PHPSTAN_TURBO=0and--debug(single process, and in phar installs the main process runs without the extension) were clean while parallel worker runs were not.Changes
src/DependencyInjection/ValidateIgnoredErrorsExtension.php— clear the memo in thefinallythat restores the process-wide accessors.src/Reflection/ReflectionProviderStaticAccessor.php—registerInstance()clears the memo.src/Reflection/PhpVersionStaticAccessor.php—registerInstance()clears the memo.src/DependencyInjection/BleedingEdgeToggle.php—setBleedingEdge()clears the memo;withBleedingEdge()routes both the set and the restore through it instead of assigning the static directly.src/DependencyInjection/ReportUnsafeArrayStringKeyCastingToggle.php—setLevel()clears the memo.ContainerFactory::postInitializeContainer()(clears explicitly, deliberately last),StubValidator(re-runspostInitializeContainer()in itsfinally),BetterReflection::populate()(no caller outsidepostInitializeContainer()).Root cause
TypeCombinator::union()/intersect()/remove()route throughTypeCombinatorCache, which the turbo extension shadows with a memo keyed on a structural hash of the arguments.ContainerFactory::postInitializeContainer()already documents the invariant:ValidateIgnoredErrorsExtension::loadConfiguration()is the second place that mutates exactly that state. It registers aDummyReflectionProviderand a runtimePhpVersion, resolves the types named in theignoreErrorspatterns throughTypeStringResolver/TypeNodeResolver(which performs unions and intersections), then restores the accessors and callsObjectType::resetCaches()— the sibling cache with the same invalidation trigger — but neverTypeCombinator::clearCache().The
DummyReflectionProviderknows no class hierarchy, so e.g.union(Exception, RuntimeException)does not collapse toExceptionunder it. That non-collapsing result stays in the memo under a key that the real analysis produces again, and is returned verbatim afterwards. Types the analysis then works with are wider or otherwise wrong, and members resolved through them go missing — the reported symptom class. Because the memo borrows its results, only entries whose result object is still referenced survive, which is why the failure looked intermittent and input-dependent.The pattern is "global state that type operations read is mutated without invalidating the memo". Rather than fix the one call site, the invalidation now lives in each mutator of that state — the reflection provider accessor, the PHP version accessor, and the two feature toggles — so a future swap cannot reintroduce it. The explicit clear stays in
ValidateIgnoredErrorsExtensionbecause when it runs for the first container of a process there is no original accessor instance to restore, and the restores are skipped.Test
tests/PHPStan/Type/GlobalStateCacheInvalidationTest.phpperforms the same swap/restore sequence the extension does: it registers aDummyReflectionProvider, memoizesunion(Exception, RuntimeException)under it while retaining the result (the memo tombstones entries whose result died, so an unretained one cannot leak), restores the real provider and asserts the union collapses toExceptionagain. Before the fix the assertion seesException|RuntimeException; the test is a no-op without the extension, which is exactly the mode in which the bug does not exist.Verified with the extension loaded through
php.ini:make tests(17798 tests) andmake phpstangreen with and withoutPHPSTAN_TURBO=0, on glibc/PHP 8.4 and on Alpine/musl/PHP 8.3 with a locally builtphpstan_turbo.so;turbo-ext/bin/side-by-side.php,smoke.php,signature-parity.php,arena-smoke.phpandparser-corpus.phpall pass.Fixes phpstan/phpstan#15037