-
Notifications
You must be signed in to change notification settings - Fork 3.4k
feat: timeout diagnostics #33023
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
lucgonp
wants to merge
21
commits into
cypress-io:develop
Choose a base branch
from
lucgonp:feat/timeout-diagnostics
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
feat: timeout diagnostics #33023
Changes from 11 commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
605636c
feat: add intelligent timeout diagnostics system
lucgonp 3b1f9de
fix: address Cursor Bot security and correctness issues
lucgonp 0d31eeb
docs(driver): translate remaining README sections to English
lucgonp 1bbdaf1
fix: escape selector in timeout suggestions and remove duplicate READ…
lucgonp 454b5db
fix(driver): correct action command timeout hints
lucgonp 77deef7
fix(driver): harden timeout diagnostics suggestions
lucgonp cd2738b
Merge branch 'develop' into feat/timeout-diagnostics
lucgonp 4e60321
chore: update changelog for timeout diagnostics feature
lucgonp 1337ed6
Merge branch 'develop' into feat/timeout-diagnostics
lucgonp 9e3cc66
Merge branch 'develop' into feat/timeout-diagnostics
lucgonp 03e50e1
Merge branch 'develop' into feat/timeout-diagnostics
lucgonp e9c620d
fix: escape double quotes in diagnostic suggestion formatting
lucgonp 7f48063
Merge branch 'develop' into feat/timeout-diagnostics
lucgonp 2d94454
Merge branch 'develop' into feat/timeout-diagnostics
lucgonp c12ef35
chore: update release date for version 15.7.1 in CHANGELOG.md
lucgonp 3e4b28a
chore: update release date for version 15.7.1 in CHANGELOG.md to pending
lucgonp b899279
chore: update release date for version 15.7.1 in CHANGELOG.md
lucgonp d6b727a
Merge branch 'develop' into feat/timeout-diagnostics
lucgonp 5025fa8
feat: add actionable suggestions for timeout diagnostics in error mes…
lucgonp 5bea5c6
Merge branch 'develop' into feat/timeout-diagnostics
lucgonp c84894f
Merge branch 'develop' into feat/timeout-diagnostics
lucgonp File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
112 changes: 112 additions & 0 deletions
112
packages/driver/src/cypress/TIMEOUT_DIAGNOSTICS_README.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,112 @@ | ||
| <!-- NOTE: This file documents a work-in-progress experimental feature. It is NOT integrated into the main Cypress error pipeline yet. --> | ||
| # Timeout Diagnostics — Smart Timeout Error Messages (WIP) | ||
|
|
||
| Status: Work‑in‑progress (not yet integrated into Cypress). This document describes an experimental timeout diagnostics system that analyzes command context (selectors, network, animations, DOM mutations) and produces contextual recommendations to help users address flaky or timing-related failures. | ||
|
|
||
| ## Objective | ||
|
|
||
| Improve the developer experience for timeout errors by producing actionable, context-aware suggestions when a command times out. Instead of a generic message like `cy.get() timed out waiting 4000ms`, diagnostics should indicate likely causes (dynamic content, pending network requests, animations, etc.) and suggest concrete fixes. | ||
|
|
||
| ## Motivation | ||
|
|
||
| Timeout errors are a frequent source of confusion in end-to-end tests. The goal of this feature is to reduce debugging time and teach best practices inline by providing targeted suggestions and examples. | ||
|
|
||
| ## Example Enhanced Output | ||
|
|
||
| ``` | ||
| cy.get('.user-list') timed out waiting 4000ms | ||
|
|
||
| Diagnostic suggestions: | ||
|
|
||
| 1) Selector likely targets dynamic content | ||
| • Wait for the loading state: cy.get('.loading-spinner').should('not.exist') | ||
| • Prefer stable data attributes: e.g. [data-cy="user-list"] | ||
| • Consider intercepting the API call that populates this content | ||
|
|
||
| 2) Network requests pending (8) | ||
| • Use cy.intercept('GET', '/api/users').as('getUsers') and cy.wait('@getUsers') | ||
| • If requests are expected to be slow, either wait for the specific request or increase the timeout | ||
|
|
||
| Learn more: https://on.cypress.io/intercept | ||
| ``` | ||
|
|
||
| ## Features | ||
|
|
||
| - Problematic selector detection (loading spinners, skeletons, dynamic IDs) | ||
| - Network analysis (pending requests, slow/long-running requests) | ||
| - Animation detection and guidance to reduce timing flakiness | ||
| - Detection of excessive DOM mutations and recommendations to wait for stability | ||
| - Command-specific suggestions (get, click, type, etc.) and doc links | ||
|
|
||
| ## Project Structure (Implementation Sketch) | ||
|
|
||
| ``` | ||
| packages/driver/src/cypress/ | ||
| timeout_diagnostics.ts — core analysis + formatting (WIP) | ||
|
|
||
| packages/driver/test/unit/cypress/ | ||
| timeout_diagnostics.spec.ts — unit tests | ||
| ``` | ||
|
|
||
| ## API Sketch | ||
|
|
||
| ```ts | ||
| // analyze context and produce suggestions | ||
| const suggestions = TimeoutDiagnostics.analyze({ | ||
| command: 'get', | ||
| selector: '.loading-spinner', | ||
| timeout: 4000, | ||
| networkRequests: 5, | ||
| animationsRunning: true, | ||
| }) | ||
|
|
||
| const formatted = TimeoutDiagnostics.formatSuggestions(suggestions) | ||
|
|
||
| // enhance base timeout message with suggestions | ||
| const enhanced = TimeoutDiagnostics.enhanceTimeoutError('cy.get() timed out', context) | ||
| ``` | ||
|
|
||
| ## Integration Notes | ||
|
|
||
| To integrate this into Cypress's existing error pipeline we would: | ||
|
|
||
| 1. Extend `error_utils` (or the error creation path) to provide additional context (selector, pending network count, DOM mutation rate, animation state) when creating a timeout error. | ||
| 2. Call `TimeoutDiagnostics.analyze(context)` and append the returned suggestions to the error message (optionally behind a feature flag). | ||
| 3. Add a configuration option to opt in/out of diagnostics in CI or local environments. | ||
|
|
||
| Integration example (pseudo): | ||
| ```ts | ||
| import TimeoutDiagnostics from './timeout_diagnostics' | ||
|
|
||
| function createTimeoutError(cmd, ms) { | ||
| const context = { | ||
| command: cmd.get('name'), | ||
| selector: cmd.get('selector'), | ||
| timeout: ms, | ||
| networkRequests: getNetworkMonitor().pendingCount(), | ||
| animationsRunning: hasRunningAnimations(), | ||
| domMutations: getDOMMutationCount(), | ||
| } | ||
|
|
||
| const baseMessage = `cy.${cmd.get('name')}() timed out waiting ${ms}ms` | ||
| return TimeoutDiagnostics.enhanceTimeoutError(baseMessage, context) | ||
| } | ||
| ``` | ||
|
|
||
| ## Tests | ||
|
|
||
| Run unit tests for the diagnostics module: | ||
|
|
||
| ```powershell | ||
| cd packages/driver | ||
| yarn test timeout_diagnostics.spec.ts | ||
| ``` | ||
|
|
||
| ## Notes & Next Steps | ||
|
|
||
| - This README documents the experimental design and examples. The feature is intentionally left out of the main error flow until heuristics and runtime metric collection are reviewed. | ||
| - Next: finalize analysis heuristics, expand unit tests, and implement an opt-in integration path in `error_utils`. | ||
|
|
||
| ## License | ||
|
|
||
| MIT — consistent with the Cypress project | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.