fix: stop model removal from invalidating prompt caches (#304857)#304984
Open
Wolfe1 wants to merge 1 commit intomicrosoft:mainfrom
Open
fix: stop model removal from invalidating prompt caches (#304857)#304984Wolfe1 wants to merge 1 commit intomicrosoft:mainfrom
Wolfe1 wants to merge 1 commit intomicrosoft:mainfrom
Conversation
When BoundModelReferenceCollection evicts models (50-reference cap), ModelChangeTracker was treating this as a content change, firing onDidPromptModelChange. This caused a feedback loop: 1. Extensions parse agents via openTextDocument (creating model refs) 2. 50+ refs trigger eviction of oldest models 3. Model removal fired onDidPromptModelChange 4. Cache invalidation triggered onDidChangeCustomAgents 5. Extensions re-parse all agents → GOTO 1 Fix: ModelChangeTracker.onModelRemoved now only cleans up the per-model content listener without firing a change signal. Model removal is a memory management concern, not a content change. parseNew() already falls back to fileService.readFile() when the model is gone, and file-system changes are covered by getFileLocatorEvent(). Fixes microsoft#304857
Contributor
|
@Wolfe1 Thanks, I think its easier if I make the fix (together with the LLM), I understand the problem now. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR addresses the high CPU usage reported in #304857 when multiple extensions register
chatAgent/chatSkillprompt files.Root Cause Analysis
Through investigation, we believe the high CPU stems from a feedback loop in the prompt caching system, specifically involving
ModelChangeTrackerinPromptsService:$acceptCustomAgentspushes agents to the extension host, firingonDidChangeCustomAgentson the APIChatCustomAgentsService) respond by callingvscode.workspace.openTextDocument()for each agent URI to parse themBoundModelReferenceCollectioninmainThreadDocuments.tshas a 50-document cap (_maxSize = 50). When 60+ agent/skill documents accumulate, the oldest references are evictedIModelService.onModelRemoved, whichModelChangeTrackerwas treating as a content change signal — firingonDidPromptModelChangeCachedPromiseinvalidates the agents/skills cache on this signalonDidChangeCustomAgentsagain → sends$acceptCustomAgentsto the extension host → GOTO 1The key insight is that model removal (step 4) is a memory management concern, not a content change. When a model is evicted, the underlying file hasn't changed —
parseNew()already falls back tofileService.readFile()when no in-memory model exists, and actual file-system changes are separately covered by the file locator events viagetFileLocatorEvent().Fix
ModelChangeTracker'sonModelRemovedhandler no longer firesonDidPromptModelChange. It only cleans up the per-model content-change listener and removes the entry from its tracking map. This breaks the feedback loop at step 4→5.What still works:
getFileLocatorEvent()✅Testing
ModelService, verifies content edits DO triggeronDidChangeCustomAgents, then verifiesdestroyModeldoes NOTPromptsServicetests pass, zero regressionsCaveat
While this fix resolves the issue in our testing and we believe the reasoning is sound, we acknowledge this may not be the direction the team would prefer. There may be edge cases we haven't considered where model removal should signal a cache refresh, or the team may prefer to address this at a different layer (e.g., deduplication in
_pushCustomAgents, increasing theBoundModelReferenceCollectioncap, or debouncing the cache invalidation). We're happy to adjust the approach based on feedback.Fixes #304857