From b4ce3ee8eeb620b25cc1a5e260de9cb7348f3e6a Mon Sep 17 00:00:00 2001 From: Svetozar Lashin Date: Fri, 22 May 2026 10:45:52 +0300 Subject: [PATCH 1/3] merge AQA & TestGen versions from other PR --- .../api-qa-flow-api-spec-analysis.md | 204 +++++ .../workflows/api-qa-flow-data-collection.md | 60 ++ ...i-qa-flow-execution-and-report-analysis.md | 63 ++ ...flow-gap-and-requirements-clarification.md | 68 ++ .../api-qa-flow-project-config-loading.md | 52 ++ .../api-qa-flow-test-case-specification.md | 121 +++ .../workflows/api-qa-flow-test-correction.md | 74 ++ instructions/r3/core/workflows/api-qa-flow.md | 164 ++++ .../core/workflows/aqa-flow-code-analysis.md | 370 ++------- .../workflows/aqa-flow-data-collection.md | 209 ++--- .../aqa-flow-requirements-clarification.md | 213 ++--- .../aqa-flow-selector-identification.md | 406 ++-------- .../aqa-flow-selector-implementation.md | 317 ++------ .../workflows/aqa-flow-test-correction.md | 328 ++------ .../workflows/aqa-flow-test-implementation.md | 459 ++--------- .../aqa-flow-test-report-analysis.md | 316 ++------ instructions/r3/core/workflows/aqa-flow.md | 286 ++++--- .../workflows/testgen-flow-data-collection.md | 330 ++------ ...gen-flow-gap-and-contradiction-analysis.md | 385 ++------- .../testgen-flow-project-config-loading.md | 199 ++--- .../testgen-flow-question-generation.md | 425 +++------- ...n-flow-requirements-document-generation.md | 559 ++----------- .../testgen-flow-test-case-export.md | 497 ++---------- .../testgen-flow-test-case-generation.md | 764 ++++-------------- .../r3/core/workflows/testgen-flow.md | 332 ++++---- 25 files changed, 2167 insertions(+), 5034 deletions(-) create mode 100644 instructions/r3/core/workflows/api-qa-flow-api-spec-analysis.md create mode 100644 instructions/r3/core/workflows/api-qa-flow-data-collection.md create mode 100644 instructions/r3/core/workflows/api-qa-flow-execution-and-report-analysis.md create mode 100644 instructions/r3/core/workflows/api-qa-flow-gap-and-requirements-clarification.md create mode 100644 instructions/r3/core/workflows/api-qa-flow-project-config-loading.md create mode 100644 instructions/r3/core/workflows/api-qa-flow-test-case-specification.md create mode 100644 instructions/r3/core/workflows/api-qa-flow-test-correction.md create mode 100644 instructions/r3/core/workflows/api-qa-flow.md diff --git a/instructions/r3/core/workflows/api-qa-flow-api-spec-analysis.md b/instructions/r3/core/workflows/api-qa-flow-api-spec-analysis.md new file mode 100644 index 00000000..5b849a3f --- /dev/null +++ b/instructions/r3/core/workflows/api-qa-flow-api-spec-analysis.md @@ -0,0 +1,204 @@ +--- +name: api-qa-flow-api-spec-analysis +description: Phase 2 of API QA workflow - Swagger/OpenAPI Spec Analysis +alwaysApply: false +tags: [] +baseSchema: docs/schemas/phase.md +--- + + + + +Analyze Swagger/OpenAPI specification or codebase API definitions to extract endpoint contracts, auth requirements, and data dependencies. + + + +- Phase 2 of 8 in `api-qa-flow` +- Input: raw data from Phase 1 + project config (Swagger URL if available) +- Output: `agents/api-qa/{IDENTIFIER}/api-analysis.md` with endpoint contracts, auth details, data dependencies +- Prerequisite: Phase 1 complete, `raw-data.md` exists with identified endpoints + + + +1. Determine API specification source +2. Execute API spec analysis using skill +3. Produce API analysis document +4. Validate and update state + + + + +Determine `{backend-source-path}` from Phase 1 raw data "Backend Source Code Analysis" section, or from project config "Backend Source Code" section, or from Rosetta docs at `RefSrc/{project-name}/docs/` (see `api-qa-data-collection` skill, step 4 for full discovery logic). If Rosetta docs exist for the backend project, read `ARCHITECTURE.md` and `CODEMAP.md` from `RefSrc/{project-name}/docs/` to understand API architecture before searching source code. + +Determine spec source in order: + +1. **Swagger URL from project config** (`api-qa-project-config.md`) +2. **Swagger/OpenAPI in backend source** (if `{backend-source-path}` configured): + - Search within `{backend-source-path}` for: `swagger.json`, `swagger.yaml`, `openapi.json`, `openapi.yaml`, `api-docs` + - If `{backend-source-path}` is NOT configured, search entire codebase instead. +3. **API route definitions in backend source** (if `{backend-source-path}` configured): + - Search within `{backend-source-path}` for framework-specific route patterns + - If `{backend-source-path}` is NOT configured, search entire codebase instead. +4. **If none found**: Proceed with documentation from Phase 1 only; ask user for endpoint details + +Decision point: Swagger available -> full spec analysis. No Swagger -> code-based analysis + user input. + + + + + +1. USE SKILL `swagger-contracts-analysis` with spec source determined in step 2.1 and target endpoints identified in Phase 1 test cases +2. The skill extracts: endpoint contracts, auth requirements, data dependencies + + + + + +Create `agents/api-qa/{IDENTIFIER}/api-analysis.md` using the following template: + +```markdown +# API Analysis - [IDENTIFIER] + +**Analyzed**: [DateTime] +**Phase**: 2 - API Spec Analysis +**Spec Source**: [Swagger URL / Code Analysis / Documentation / Combined] + +--- + +## API Overview + +- **Base URL**: [Base URL or env variable] +- **API Version**: [v1, v2, etc.] +- **Auth Mechanism**: [Type] +- **Content Type**: [application/json, etc.] + +--- + +## Endpoints Under Test + +### Endpoint 1: [METHOD] [PATH] + +**Summary**: [What it does] +**Tags**: [API group/category] + +#### Request +- **Content-Type**: [Type] +- **Auth Required**: [Yes/No — scheme name] +- **Path Parameters**: + | Name | Type | Required | Description | + |------|------|----------|-------------| + | [name] | [type] | [Yes/No] | [desc] | + +- **Query Parameters**: + | Name | Type | Required | Default | Description | + |------|------|----------|---------|-------------| + | [name] | [type] | [Yes/No] | [value] | [desc] | + +- **Request Body Schema**: + ```json + { + "field1": "string (required, max 255)", + "field2": "integer (optional, min 0)" + } + ``` + +- **Example Request**: + ```json + { + "field1": "example value", + "field2": 42 + } + ``` + +#### Responses + +| Status | Description | Schema | +|--------|-------------|--------| +| 200 | Success | `{ id, field1, field2, createdAt }` | +| 400 | Validation error | `{ error, message, details[] }` | +| 401 | Unauthorized | `{ error, message }` | +| 404 | Not found | `{ error, message }` | +| 500 | Server error | `{ error, message }` | + +#### Constraints +- [Rate limiting, max payload size, required permissions] + +--- + +[Repeat for each endpoint] + +--- + +## Authentication Details + +### Auth Mechanism: [Type] +- **Token Endpoint**: [URL if applicable] +- **Token Type**: [Bearer / API Key / etc.] +- **Token Location**: [Header / Query / Cookie] +- **Header Name**: [Authorization / X-API-Key / etc.] + +### Auth for Tests +- **Strategy**: [Test credentials / Mock auth / Service account] +- **Existing Pattern**: [How current tests do it — from Phase 1] +- **Setup Required**: [Token acquisition steps] + +--- + +## Data Dependencies + +### Preconditions +- [Entity 1 must exist before endpoint X can be tested] + +### Creation Order +1. [Create entity A first] +2. [Create entity B (depends on A)] +3. [Test target endpoint (depends on A and B)] + +### Cleanup Considerations +- [Data created by tests should be cleaned up] +- [Cascade delete behavior] + +--- + +## Analysis Summary + +- **Endpoints Analyzed**: [Count] +- **HTTP Methods**: GET: [N], POST: [N], PUT: [N], DELETE: [N], PATCH: [N] +- **Auth Required Endpoints**: [Count] +- **Public Endpoints**: [Count] +- **Request Schemas Extracted**: [Count] +- **Response Schemas Extracted**: [Count] +- **Data Dependencies Found**: [Count] +- **Spec Coverage**: [% of test case endpoints covered by spec] +``` + + + + +1. Confirm all endpoints from test cases analyzed +2. Confirm request/response schemas documented +3. Confirm auth requirements identified +4. Confirm data dependencies mapped + + + +1. Update `agents/api-qa-state.md`: + - Endpoints Analyzed: [count] + - HTTP Methods: [GET/POST/PUT/DELETE/PATCH counts] + - Auth Required Endpoints: [count] + - Spec Source: [Swagger / Code / Docs / Combined] + - Backend Source Path: [path or N/A] + - Phase 2 completion timestamp +2. Mark Phase 2 complete, Phase 3 current + + + +- All endpoints from test cases analyzed +- Request/response schemas documented (from spec or code) +- Auth requirements identified +- Data dependencies mapped +- Backend source code analyzed for route definitions (if path configured) +- `api-analysis.md` created with all sections + + + diff --git a/instructions/r3/core/workflows/api-qa-flow-data-collection.md b/instructions/r3/core/workflows/api-qa-flow-data-collection.md new file mode 100644 index 00000000..80d90ef8 --- /dev/null +++ b/instructions/r3/core/workflows/api-qa-flow-data-collection.md @@ -0,0 +1,60 @@ +--- +name: api-qa-flow-data-collection +description: Phase 1 of API QA workflow - Data Collection from test cases, documentation, and codebase +alwaysApply: false +tags: [] +baseSchema: docs/schemas/phase.md +--- + + + + +Gather test case details from TMS, search documentation, and discover existing API test patterns in the codebase to establish baseline for automation. + + + +- Phase 1 of 8 in `api-qa-flow` +- Input: project config + initial data from Phase 0 +- Output: `agents/api-qa/{IDENTIFIER}/raw-data.md` with test cases, documentation, and existing test patterns +- Prerequisite: Phase 0 complete, `api-qa-project-config.md` and `initial-data.md` exist + + + +1. Confirm data sources from project config +2. Execute data collection +3. Validate and update state + + + +1. Verify project config loaded with data source information +2. Verify initial data file exists with test case reference +3. Identify TMS, documentation, and codebase sources to query + + + +1. USE SKILL `api-qa-data-collection` +2. Verify raw data file created at `agents/api-qa/{IDENTIFIER}/raw-data.md` + + + +1. Update `agents/api-qa-state.md`: + - Test Cases Retrieved: [count] + - Documentation Pages Found: [count] + - API Endpoints Identified: [count] + - Existing Test Files Found: [count] + - Test Framework: [name] + - Backend Source: [path or N/A] + - Phase 1 completion timestamp +2. Mark Phase 1 complete, Phase 2 current + + + +- Test case data retrieved and documented +- Documentation searched (results found OR user confirmed skip) +- Existing test patterns analyzed +- Backend source code searched (if path configured in project config) +- `raw-data.md` created with all sections populated +- API endpoints identified from test cases + + + diff --git a/instructions/r3/core/workflows/api-qa-flow-execution-and-report-analysis.md b/instructions/r3/core/workflows/api-qa-flow-execution-and-report-analysis.md new file mode 100644 index 00000000..1982ab5c --- /dev/null +++ b/instructions/r3/core/workflows/api-qa-flow-execution-and-report-analysis.md @@ -0,0 +1,63 @@ +--- +name: api-qa-flow-execution-and-report-analysis +description: Phase 6 of API QA workflow - Test Execution and Report Analysis (USER INTERACTION REQUIRED) +alwaysApply: false +tags: [] +baseSchema: docs/schemas/phase.md +--- + + + + +Analyze test execution results provided by user. Identify failures, categorize root causes, and prepare actionable fix recommendations. + + + +- Phase 6 of 8 in `api-qa-flow` +- Input: test execution report or output (user-provided or from `agents/user-instructions/`) +- Output: `agents/api-qa/{IDENTIFIER}/execution-report.md` with failure analysis and recommendations +- Prerequisite: Phase 5 complete, tests executed by user +- HITL: may need to ask user for test execution results + + + +1. Obtain test execution results +2. Execute report analysis (Part A of skill) +3. Review findings and update state + + + +1. USE SKILL `debugging` +2. USE SKILL `api-qa-test-debugging` +3. Execute Part A (Report Analysis) only +4. If test report location unknown, ask user +5. **WAIT** for user to provide results if not found in `agents/user-instructions/` + + + +1. Verify all failures categorized +2. Verify root causes identified +3. Verify patterns analyzed across failures +4. Confirm recommendations are actionable + + + +1. Update `agents/api-qa-state.md`: + - Tests Executed: [count] + - Tests Passed: [count] + - Tests Failed: [count] + - Root Causes: [list by category] + - Phase 6 completion timestamp +2. Mark Phase 6 complete, Phase 7 current + + + +- Test execution results obtained from user +- All results parsed and categorized +- Root causes analyzed for each failure +- Patterns identified across failures +- `execution-report.md` created with all sections +- Clear recommendations for Phase 7 + + + diff --git a/instructions/r3/core/workflows/api-qa-flow-gap-and-requirements-clarification.md b/instructions/r3/core/workflows/api-qa-flow-gap-and-requirements-clarification.md new file mode 100644 index 00000000..e3aeb0f7 --- /dev/null +++ b/instructions/r3/core/workflows/api-qa-flow-gap-and-requirements-clarification.md @@ -0,0 +1,68 @@ +--- +name: api-qa-flow-gap-and-requirements-clarification +description: Phase 3 of API QA workflow - Gap Analysis and Requirements Clarification (USER INTERACTION REQUIRED) +alwaysApply: false +tags: [] +baseSchema: docs/schemas/phase.md +--- + + + + +Cross-reference test cases, documentation, and API spec to identify gaps, contradictions, and ambiguities. Clarify all unknowns with user before test specification. + + + +- Phase 3 of 8 in `api-qa-flow` +- Input: raw data (Phase 1) + API analysis (Phase 2) + project config +- Output: `agents/api-qa/{IDENTIFIER}/analysis.md` with gaps resolved, user answers documented +- Prerequisite: Phases 1 and 2 complete +- HITL: user answers required before Phase 4 + + + +1. Execute gap analysis +2. Present questions and wait for user answers +3. Document clarifications and update state + + + +1. USE SKILL `api-qa-gap-analysis` +2. USE SKILL `gap-and-contradiction-analysis` +3. USE SKILL `aqa-requirements-elicitation` +4. Prepare prioritized list of gaps, contradictions, ambiguities + + + +1. USE SKILL `questioning` +2. Present structured questions to user (Critical / Important / Optional) +3. **STOP AND WAIT** for user to provide all answers +4. If user doesn't know an answer, mark as assumption and document + + + +1. Process user answers +2. Update analysis document with questions, answers, and resolved items +3. Verify `agents/api-qa/{IDENTIFIER}/analysis.md` created + + + +1. Update `agents/api-qa-state.md`: + - Gaps Found: [count] + - Contradictions Found: [count] + - Questions Asked: [count] + - Answers Received: [count] + - Open Assumptions: [count] + - Phase 3 completion timestamp +2. Mark Phase 3 complete, Phase 4 current + + + +- Cross-reference analysis completed +- All gaps, contradictions, and ambiguities documented +- Questions presented to user +- User answers received and documented +- `analysis.md` created with all sections + + + diff --git a/instructions/r3/core/workflows/api-qa-flow-project-config-loading.md b/instructions/r3/core/workflows/api-qa-flow-project-config-loading.md new file mode 100644 index 00000000..4ce85977 --- /dev/null +++ b/instructions/r3/core/workflows/api-qa-flow-project-config-loading.md @@ -0,0 +1,52 @@ +--- +name: api-qa-flow-project-config-loading +description: Phase 0 of API QA workflow - Project Config Loading (USER INTERACTION CONDITIONALLY REQUIRED) +alwaysApply: false +tags: [] +baseSchema: docs/schemas/phase.md +--- + + + + +Initialize API QA session directory, load existing project config or collect project-specific information from user for backend API test automation. + + + +- Phase 0 of 8 in `api-qa-flow` +- Input: user request with test case reference (TestRail ID, Jira ticket, or direct description) +- Output: `agents/api-qa/{IDENTIFIER}/` directory with `initial-data.md` and `api-qa-project-config.md`; `agents/api-qa-state.md` initialized +- Prerequisite: starting new API QA flow +- HITL: conditional — only if project config does not already exist + + + +1. Parse user input and setup session directory +2. Load or create project config +3. Create initial data file and update state + + + +1. USE SKILL `api-qa-project-config` +2. Verify `agents/api-qa/{IDENTIFIER}/` directory created +3. Verify `api-qa-project-config.md` exists with non-empty content +4. **ASK USER** for project info only if config does not already exist + + + +1. Update `agents/api-qa-state.md`: + - Test Case Source: [TestRail ID / Jira key / Manual] + - Config Source: [Existing / User provided / Discovered] + - Files Created: initial-data.md, api-qa-state.md + - Phase 0 completion timestamp +2. Mark Phase 0 complete, Phase 1 current + + + +- `agents/api-qa/{IDENTIFIER}/` directory exists +- `api-qa-project-config.md` exists with non-empty content +- `initial-data.md` created with initial prompt and config reference +- `agents/api-qa-state.md` created with Phase 0 marked complete + + + diff --git a/instructions/r3/core/workflows/api-qa-flow-test-case-specification.md b/instructions/r3/core/workflows/api-qa-flow-test-case-specification.md new file mode 100644 index 00000000..04687f14 --- /dev/null +++ b/instructions/r3/core/workflows/api-qa-flow-test-case-specification.md @@ -0,0 +1,121 @@ +--- +name: api-qa-flow-test-case-specification +description: Phase 4 of API QA workflow - Detailed API Test Case Specification (HITL APPROVAL GATE) +alwaysApply: false +tags: [] +baseSchema: docs/schemas/phase.md +--- + + + + +Convert test cases into detailed, implementation-ready API test specifications using Given-When-Then format. User approval required before implementation. + + + +- Phase 4 of 8 in `api-qa-flow` +- Input: all phase 1-3 outputs (raw data, API analysis, gap analysis) +- Output: `agents/api-qa/{IDENTIFIER}/test-specs.md` with Given-When-Then scenarios, file mapping, shared utilities +- Prerequisite: Phase 3 complete, all user clarifications received +- HITL: explicit user approval required before Phase 5 + + + +1. Load all previous phase outputs +2. Execute test specification authoring +3. Produce test specs document +4. Present for user approval +5. Update state + + + + +Read completely: +1. `agents/api-qa/{IDENTIFIER}/raw-data.md` — original test cases and patterns +2. `agents/api-qa/{IDENTIFIER}/api-analysis.md` — endpoint contracts +3. `agents/api-qa/{IDENTIFIER}/analysis.md` — clarifications and resolved gaps + + + + + +1. USE SKILL `api-test-spec-authoring` with all loaded inputs from step 4.1 +2. The skill generates: test scenarios, GWT specs, file mapping, shared utilities, execution order + + + + + +Create `agents/api-qa/{IDENTIFIER}/test-specs.md` using the following template: + +```markdown +# API QA Test Specifications - [IDENTIFIER] + +**Created**: [DateTime] +**Phase**: 4 - Test Case Specification +**Source Test Cases**: [List source references] + +--- + +## Summary + +- **Total Test Scenarios**: [Count] +- **Priority Breakdown**: P0: [N], P1: [N], P2: [N], P3: [N] +- **Type Breakdown**: Happy Path: [N], Negative: [N], Auth: [N], Edge Case: [N] +- **Endpoints Covered**: [Count] +- **Test Files Planned**: [Count] + +--- + +## Test Scenarios + +### Endpoint: [METHOD] [PATH] + +[All ATC-NNN specifications for this endpoint] + +--- + +## Test File Mapping +[From skill step 4] + +## Shared Utilities +[From skill step 5] + +## Execution Order +[From skill step 6] + +## Assumptions +[List any assumptions from Phase 3 that affect these specs] +``` + + + + +1. Present summary to user: total scenarios, priority breakdown, endpoints covered +2. **WAIT FOR USER APPROVAL** — "Yes", "Approve", or similar +3. If user requests modifications: update specs and re-present +4. **DO NOT PROCEED** to Phase 5 without explicit approval + + + +1. Update `agents/api-qa-state.md`: + - Test Cases Specified: [count] + - Priority Breakdown: P0: [N], P1: [N], P2: [N], P3: [N] + - Endpoints Covered: [count] + - User Approval: [datetime] + - Phase 4 completion timestamp +2. Mark Phase 4 complete, Phase 5 current + + + +- All source test cases converted to detailed specifications +- Given-When-Then format used for every scenario +- Exact request values specified (no placeholders) +- Exact response assertions defined +- Auth and error scenarios covered +- Test file mapping defined +- Shared utilities identified +- User approval received + + + diff --git a/instructions/r3/core/workflows/api-qa-flow-test-correction.md b/instructions/r3/core/workflows/api-qa-flow-test-correction.md new file mode 100644 index 00000000..c7bdde66 --- /dev/null +++ b/instructions/r3/core/workflows/api-qa-flow-test-correction.md @@ -0,0 +1,74 @@ +--- +name: api-qa-flow-test-correction +description: Phase 7 of API QA workflow - Test Corrections (USER APPROVAL REQUIRED) +alwaysApply: false +tags: [] +baseSchema: docs/schemas/phase.md +--- + + + + +Fix identified test failures based on Phase 6 analysis. Requires explicit user approval before applying changes. + + + +- Phase 7 of 8 in `api-qa-flow` +- Input: execution report from Phase 6 +- Output: corrected test code, ready for re-testing +- Prerequisite: Phase 6 complete +- HITL: explicit user approval required before applying changes + + + +1. Execute correction preparation (Part B of skill) +2. Present changes for approval +3. Apply approved changes +4. Update state + + + +1. USE SKILL `debugging` +2. USE SKILL `coding` +3. USE SKILL `api-qa-test-debugging` +4. Execute Part B (Corrections) — prepare proposed changes only +5. Do NOT apply changes yet + + + +1. Present all proposed changes with before/after code +2. **WAIT** for explicit user approval +3. User must type "approved" or "yes" — do not assume approval +4. If user requests modifications: update proposals, re-present +5. If user rejects specific changes: remove from plan + + + +1. Apply approved changes one at a time +2. Validate linting after each change +3. Verify changes address root causes + + + +1. Update `agents/api-qa-state.md`: + - Issues Fixed: [count] + - Changes Applied: [count] + - User Approval: [datetime] + - Files Modified: [list] + - Status: Ready for re-testing + - Phase 7 completion timestamp +2. Mark Phase 7 complete +3. Inform user to re-run tests +4. If tests still fail: return to Phase 6 + + + +- Phase 6 analysis reviewed +- Proposed changes prepared with before/after code +- User approval explicitly received +- All approved changes applied +- Linting errors checked and fixed +- Changes address identified root causes + + + diff --git a/instructions/r3/core/workflows/api-qa-flow.md b/instructions/r3/core/workflows/api-qa-flow.md new file mode 100644 index 00000000..91541044 --- /dev/null +++ b/instructions/r3/core/workflows/api-qa-flow.md @@ -0,0 +1,164 @@ +--- +name: api-qa-flow +tags: ["workflow"] +baseSchema: docs/schemas/workflow.md +--- + + + + + +End-to-end backend API test automation from test case input to working automated tests. Uses test case management systems (TestRail, Jira), Swagger/OpenAPI specs, Confluence documentation, and project code to create automated API tests following existing architecture and coding standards. + + + + + +- Rosetta prep steps completed +- MUST FOLLOW THIS WORKFLOW ENTIRELY AND FULLY, ALL PHASES ARE SEQUENTIAL. +- ONE PHASE AT A TIME: Acquire phase file, execute, update state, move to next. +- DO NOT SKIP PHASES: Each builds on previous. +- NO ASSUMPTIONS: Never assume endpoints, payloads, auth mechanisms, or response schemas. Always ask the user if information is missing. +- STATE TRACKING: Update `agents/api-qa-state.md` after each phase. +- MUST use todo tasks for tracking progress. Prioritize ACCURACY over SPEED. +- If user did not specify preferences, perform all steps except optional. +- User CAN customize: specific phases, already-done phases, specific goals, specific cases — LISTEN and ADOPT. +- Prefer existing test files and utilities over creating new ones. Use `project_description.md`, `CONTEXT.md` + `ARCHITECTURE.md` + `IMPLEMENTATION.md` as source of truth for coding standards. + + + +1. ACQUIRE `api-qa-flow-project-config-loading.md` FROM KB +2. Execute phase instructions. +3. Input: user request. Output: project config file, initial data file, session directory at `agents/api-qa/{IDENTIFIER}/`. +4. **ASK USER FOR PROJECT INFO** if config does not already exist. +5. Recommended skills: `api-qa-project-config` +6. Update `agents/api-qa-state.md` + + + + + +1. ACQUIRE `api-qa-flow-data-collection.md` FROM KB +2. Execute phase instructions. +3. Input: project config + initial data. Output: raw data document at `agents/api-qa/{IDENTIFIER}/raw-data.md` with test cases, documentation, and existing test patterns. +4. Recommended skills: `api-qa-data-collection` +5. Update `agents/api-qa-state.md` + + + + + +1. ACQUIRE `api-qa-flow-api-spec-analysis.md` FROM KB +2. Execute phase instructions. +3. Input: raw data + project config. Output: API analysis document at `agents/api-qa/{IDENTIFIER}/api-analysis.md` with endpoint contracts, auth requirements, data dependencies. +4. Recommended skills: `swagger-contracts-analysis` +5. Update `agents/api-qa-state.md` + + + + + +1. ACQUIRE `api-qa-flow-gap-and-requirements-clarification.md` FROM KB +2. Execute phase instructions. +3. Input: raw data + API analysis. Output: analysis document at `agents/api-qa/{IDENTIFIER}/analysis.md` with gaps, contradictions, ambiguities resolved. +4. **WAIT FOR USER ANSWERS** before Phase 4. +5. Recommended skills: `api-qa-gap-analysis`, `gap-and-contradiction-analysis`, `aqa-requirements-elicitation`, `questioning` +6. Update `agents/api-qa-state.md` + + + + + +1. ACQUIRE `api-qa-flow-test-case-specification.md` FROM KB +2. Execute phase instructions. +3. Input: all phase 1-3 outputs. Output: test specifications at `agents/api-qa/{IDENTIFIER}/test-specs.md` with Given-When-Then scenarios. +4. **WAIT FOR USER APPROVAL** before Phase 5. +5. Recommended skills: `api-test-spec-authoring` +6. Update `agents/api-qa-state.md` + + + + + +1. ACQUIRE `api-qa-flow-test-implementation.md` FROM KB +2. Execute phase instructions. +3. Input: approved test specs + existing patterns + API analysis. Output: implemented test files. +4. **STOP AND WAIT** for user to execute tests. +5. Recommended skills: `coding`, `testing`, `api-qa-test-implementation` +6. Update `agents/api-qa-state.md` + + + + + +1. ACQUIRE `api-qa-flow-execution-and-report-analysis.md` FROM KB +2. Execute phase instructions. +3. Input: test execution report (user-provided or from `agents/user-instructions/`). Output: execution report at `agents/api-qa/{IDENTIFIER}/execution-report.md` with failure analysis. +4. **WAIT FOR USER TO PROVIDE TEST EXECUTION RESULTS**. +5. Recommended skills: `debugging`, `api-qa-test-debugging` (Part A) +6. Update `agents/api-qa-state.md` + + + + + +1. ACQUIRE `api-qa-flow-test-correction.md` FROM KB +2. Execute phase instructions. +3. Input: execution report + test files + test specs. Output: corrected test files. +4. **WAIT FOR USER APPROVAL** before applying changes. +5. Recommended skills: `debugging`, `coding`, `api-qa-test-debugging` (Part B) +6. Update `agents/api-qa-state.md` + + + + + + + +Create/update `agents/api-qa-state.md` after each phase: + +```markdown +# API QA State - + +**Last Updated**: [DateTime] +**Current Phase**: [0-7 or COMPLETE] +**Test Case Source**: [TestRail ID / Jira Ticket / Manual] +**Feature**: [Feature Name] +**API Base URL**: [Base URL if known] + +## Phase Completion Status + +- [ ] Phase 0: Project Config Loading +- [ ] Phase 1: Data Collection +- [ ] Phase 2: API Spec Analysis +- [ ] Phase 3: Gap & Requirements Clarification +- [ ] Phase 4: Test Case Specification +- [ ] Phase 5: Test Implementation +- [ ] Phase 6: Execution & Report Analysis +- [ ] Phase 7: Test Corrections +``` + + + + + +Subagents: +- `discoverer` (Lightweight): external MCP data gathering, codebase analysis, API spec extraction +- `architect` (Full): test requirements specification, gap analysis, test case design +- `engineer` (Full): test implementation, debugging, corrections +- `executor` (Lightweight): optional for mechanical actions (builds, installs) + +Skills: +- `questioning`, `coding`, `testing`, `debugging` +- `api-qa-project-config`, `api-qa-data-collection`, `swagger-contracts-analysis`, `api-qa-gap-analysis`, `api-test-spec-authoring`, `api-qa-test-implementation`, `api-qa-test-debugging` +- `aqa-requirements-elicitation`, `gap-and-contradiction-analysis` + +Note: `api-qa-test-debugging` is a standalone ad-hoc skill (no dedicated workflow file). It is invoked on-demand during Phase 6 (failure analysis) and Phase 7 (corrections). + +MCPs: +- `TestRail` — test case management +- `Jira MCP` — Jira issues + Confluence documentation + + + + diff --git a/instructions/r3/core/workflows/aqa-flow-code-analysis.md b/instructions/r3/core/workflows/aqa-flow-code-analysis.md index 4f7fc0d7..ae781357 100644 --- a/instructions/r3/core/workflows/aqa-flow-code-analysis.md +++ b/instructions/r3/core/workflows/aqa-flow-code-analysis.md @@ -2,323 +2,61 @@ name: aqa-flow-code-analysis description: Phase 3 of AQA workflow - Code Analysis and Architecture Understanding alwaysApply: false -baseSchema: docs/schemas/rule.md +tags: [] +baseSchema: docs/schemas/phase.md --- -# Phase 3: Code Analysis - -## Objective + + Understand existing test architecture, identify reusable components, and determine where new test should be integrated. - -## Prerequisites - -- Phase 1 and 2 completed -- Test plan file updated with assertions and clarifications -- User answers received - -## Phase Tasks - -### Task 1: Read Project Description - -**Actions**: -1. Locate and read `agents/user-app/project_description.md` file -2. Extract key information: - - **Test Framework**: What testing framework is used? (e.g., Playwright, Selenium, Cypress) - - **Language**: Programming language (e.g., Python, JavaScript, TypeScript, Java) - - **Project Structure**: How are tests organized? - - Test directories - - Page Object locations - - Utility/helper locations - - Test data locations - - **Coding Standards**: - - Naming conventions (files, classes, methods, variables) - - Code formatting rules - - Import organization - - Comment style - - **Test Patterns**: - - How tests are structured (AAA, Given-When-Then, etc.) - - Setup/teardown patterns - - Assertion patterns - - **Dependencies**: Required libraries and utilities -3. Document findings in test plan - -**Expected Output**: Understanding of project standards and structure. - -### Task 1.5: Read and Understand Common User Instructions - -**Actions**: -1. Locate and read all files in `agents/user-instructions/` directory -2. Extract common user instructions and preferences from all files: - - **Test Creation Guidelines**: Specific rules or patterns for creating tests - - **Code Style Preferences**: Any user-specific coding style requirements - - **Test Data Handling**: How test data should be managed or generated - - **Assertion Patterns**: Preferred assertion styles or custom matchers - - **Setup/Teardown Requirements**: Specific setup or cleanup procedures - - **Naming Conventions**: User-specific naming requirements beyond project standards - - **Error Handling**: How errors or failures should be handled in tests - - **Documentation Requirements**: Any specific documentation needs - - **Integration Patterns**: How tests should integrate with other systems - - **Performance Considerations**: Any performance-related requirements -3. Categorize instructions: - - **Must Follow**: Critical instructions that must be applied - - **Should Follow**: Important preferences that should be applied when possible - - **Nice to Have**: Optional preferences -4. Document extracted instructions in test plan -5. **Apply these instructions** throughout the test creation process: - - When identifying Page Objects (Task 2) - - When analyzing similar tests (Task 3) - - When identifying utilities (Task 4) - - When updating test plan (Task 5) - - Ensure instructions are referenced in Phase 6 (Test Implementation) - -**Expected Output**: Extracted user instructions documented and ready to apply in test creation. - -**Note**: If `agents/user-instructions/` directory does not exist or is empty, skip this task and proceed to Task 2. Document that no user instructions files were found. - -### Task 2: Analyze Frontend Source Code (if available) - -**Actions**: -1. Check if frontend source code is available: - ``` - Use: Glob to check for RefSrc/tools-st-frontend/ - ``` -2. If frontend code exists, analyze UI structure: - - Search for React components related to the feature under test - - Identify component file structure in `RefSrc/tools-st-frontend/src/` - - Note component props, interfaces, and data-testid attributes - - Document UI flow and component hierarchy - - Identify API calls and data models used -3. Extract selector candidates: - - Look for `data-testid`, `data-test`, or `test-id` attributes - - Identify stable `id` and `className` patterns - - Note ARIA labels and semantic HTML -4. Document findings: - ```markdown - ### Frontend Code Analysis - - #### Component: DashboardComponent (RefSrc/tools-st-frontend/src/features/dashboard/Dashboard.tsx) - - data-testid attributes: "welcome-message", "dashboard-title" - - Props: { userName: string, notifications: number } - - API calls: fetchDashboardData() - - Related components: NotificationBell, UserProfile - - #### Component: SettingsPage (RefSrc/tools-st-frontend/src/features/settings/SettingsPage.tsx) - - data-testid attributes: "email-input", "save-button" - - Form fields: email, notifications, preferences - ``` -5. If frontend code NOT available, skip to Task 3 - -**Expected Output**: Understanding of UI implementation and available test identifiers. - -### Task 3: Identify Existing Page Objects - -**Actions**: -1. Search for Page Object files in the test automation codebase: - ``` - Use: Glob or Grep to find Page Object files - Example patterns: "**/pages/**", "**/page-objects/**", "**/*Page.*" - ``` -2. For each relevant Page Object, analyze: - - What page/component does it represent? - - What selectors are already defined? - - What methods/actions are available? - - How are selectors organized (constants, getters, properties)? - - What naming patterns are used? -3. Identify which Page Objects are relevant to this test: - - Which pages will the test interact with? - - Do Page Objects exist for all required pages? - - Which Page Objects need to be extended? -4. Document findings: - ```markdown - ### Existing Page Objects - - #### LoginPage (src/pages/LoginPage.ts) - - Selectors: username, password, loginButton, errorMessage - - Methods: login(), isErrorDisplayed() - - Relevance: Needed for test setup - - #### DashboardPage (src/pages/DashboardPage.ts) - - Selectors: welcomeMessage, menuButton, userProfile - - Methods: navigateToProfile(), getWelcomeText() - - Relevance: Main test target - - #### Missing Page Objects: - - SettingsPage (needed for test, does not exist) - ``` - -**Expected Output**: Complete inventory of relevant Page Objects and gaps. - -### Task 4: Search for Similar Tests - -**Actions**: -1. Search for tests covering similar features or flows: - ``` - Use: Grep or SemanticSearch to find related tests - Search for: feature names, page names, similar actions - ``` -2. For each similar test found, analyze: - - What does it test? - - How is it structured? - - What patterns does it use? - - Where is it located? - - What utilities does it import? - - How are assertions written? -3. Identify the most similar tests (closest match to new test) -4. Determine best location for new test: - - **Add to existing file**: If test is very similar and file is not too large - - **Create new file**: If test covers new area or existing file is too large -5. Document findings: - ```markdown - ### Similar Tests - - #### tests/auth/login.test.ts - - Tests: User login flow - - Pattern: Setup -> Action -> Assert -> Cleanup - - Uses: LoginPage, DashboardPage - - Similarity: Uses same pages, similar flow - - #### tests/dashboard/navigation.test.ts - - Tests: Dashboard navigation - - Pattern: Login setup -> Multiple navigation assertions - - Uses: DashboardPage, utility helpers - - Similarity: Similar assertion style - - ### Recommended Test Location - - File: tests/dashboard/user-profile.test.ts (new file) - - Reason: New feature area, logical grouping - - Alternative: Add to tests/dashboard/navigation.test.ts if test is small - ``` - -**Expected Output**: Understanding of existing test patterns and determined location for new test. - -### Task 5: Identify Reusable Utilities - -**Actions**: -1. Search for utility/helper files: - ``` - Use: Glob to find utility files - Patterns: "**/utils/**", "**/helpers/**", "**/lib/**" - ``` -2. Identify reusable components: - - Test setup helpers (login, navigation, data creation) - - Assertion utilities (custom matchers, wait helpers) - - Data generators (test data factories) - - Configuration utilities -3. Document relevant utilities: - ```markdown - ### Reusable Utilities - - - `utils/test-helpers.ts` - - `loginAsUser(username, password)`: Automates login - - `waitForPageLoad()`: Smart page load wait - - - `utils/assertions.ts` - - `expectElementVisible(selector)`: Custom visibility assertion - - `expectTextContains(element, text)`: Text assertion helper - - - `utils/test-data.ts` - - `generateUser()`: Creates test user data - ``` - -**Expected Output**: List of utilities that should be reused in new test. - -### Task 6: Update Test Plan with Analysis - -**Actions**: -1. Add Phase 3 section to test plan: - ```markdown - ## Phase 3: Code Analysis - - ### Project Information - - Framework: [e.g., Playwright with TypeScript] - - Test Location: [Directory path] - - Naming Convention: [Pattern] - - ### Frontend Code Analysis (if available) - - Frontend Source: RefSrc/tools-st-frontend/ - - Components Analyzed: [List] - - Available data-testid attributes: [List] - - Component Props: [Relevant props] - - UI Flow: [Brief description] - - ### Common User Instructions - - Source: `agents/user-instructions/` (all files) - - Must Follow: [List critical instructions] - - Should Follow: [List important preferences] - - Nice to Have: [List optional preferences] - - Application: These instructions MUST be applied during test implementation (Phase 6) - - ### Existing Page Objects - [List with relevance] - - ### Page Objects to Create/Extend - - [List missing Page Objects] - - [List Page Objects needing new selectors] - - ### Similar Tests - [List with file paths and similarity notes] - - ### Recommended Test Location - - File: [Path] - - Reason: [Why] - - ### Reusable Utilities - [List utilities to import and use] - - ### Coding Patterns to Follow - - Test structure: [Pattern] - - Naming: [Convention] - - Assertions: [Style] - - User Instructions: [Apply user instructions from agents/user-instructions/] - ``` - -**Expected Output**: Test plan enhanced with architecture understanding. - -## Completion Criteria - -- [ ] `agents/user-app/project_description.md` read and understood -- [ ] All files in `agents/user-instructions/` read and understood (if directory exists) -- [ ] Common user instructions extracted and categorized -- [ ] User instructions documented in test plan -- [ ] All relevant Page Objects identified and analyzed -- [ ] Similar tests found and patterns understood -- [ ] Test location determined (new file vs. existing file) -- [ ] Reusable utilities identified -- [ ] Coding standards and conventions documented -- [ ] Test plan updated with Phase 3 information including user instructions -- [ ] `agents/aqa-state.md` updated with Phase 3 completion - -## Update State File - -After completing Phase 3, update `agents/aqa-state.md`: - -```markdown -### Phase 3: Code Analysis -- Completed: [DateTime] -- User Instructions Directory: [Found/Not Found, files list if found] -- User Instructions Applied: [Yes/No, summary if yes] -- Existing Page Objects: [Count and list] -- Page Objects to Create: [Count and list] -- Similar Tests: [File paths] -- Test Location: [Directory/File decision] -- Framework: [Name and version] -``` - -Mark Phase 3 as completed and Phase 4 as current. - -## Next Phase - -Proceed to **Phase 4: Selector Identification** by reading: -``` -agents/instructions/agents/advanced/r1/aqa-phase4.md -``` - -## Important Notes - -- **Architecture First**: Understanding existing structure prevents duplication -- **Pattern Consistency**: New test must match existing patterns -- **Reuse Over Reinvent**: Use existing utilities and Page Objects -- **User Instructions**: Common user instructions from all files in `agents/user-instructions/` MUST be applied during test implementation (Phase 6) -- **Document Decisions**: Record why specific location/approach was chosen -- **No Assumptions**: If project structure is unclear, ask user for clarification + + + +- Phase 3 of 8 in `aqa-flow` +- Input: test plan with assertions and clarifications +- Output: architecture analysis, page object inventory, test location decision +- Prerequisite: Phases 1 and 2 complete + + + +1. Read project description +2. Execute codebase analysis +2. Validate findings +3. Update state + + + +1. USE SKILL `aqa-codebase-analysis` +2. Verify test plan updated with architecture findings + + + +1. Confirm project description read +2. Confirm user instructions extracted (if directory exists) +3. Confirm page objects inventoried +4. Confirm test location decided + + + +1. Update `agents/aqa-state.md`: + - User Instructions: [found/not found] + - Existing Page Objects: [count and list] + - Page Objects to Create: [count and list] + - Similar Tests: [paths] + - Test Location: [directory/file] + - Framework: [name] + - Phase 3 completion timestamp +2. Mark Phase 3 complete, Phase 4 current + + + +- Project description read and standards documented +- User instructions extracted and categorized (if available) +- All relevant page objects identified +- Similar tests found and patterns documented +- Test location determined with rationale +- Reusable utilities identified + + + diff --git a/instructions/r3/core/workflows/aqa-flow-data-collection.md b/instructions/r3/core/workflows/aqa-flow-data-collection.md index 34fc0898..790e86d5 100644 --- a/instructions/r3/core/workflows/aqa-flow-data-collection.md +++ b/instructions/r3/core/workflows/aqa-flow-data-collection.md @@ -1,78 +1,54 @@ --- name: aqa-flow-data-collection -description: Phase 1 of AQA workflow - Data Collection -alwaysApply: false +description: Phase 1 of AQA workflow - Data Collection from TestRail and Confluence +tags: ["aqa", "phase"] baseSchema: docs/schemas/phase.md --- -# Phase 1: Data Collection - -## Objective - -Gather all required information from external sources (TestRail and Confluence) to understand test requirements and expected behavior. - -## Prerequisites - -- TestRail MCP configured and accessible -- Atlassian (Confluence) MCP configured and accessible -- Test case ID or requirement provided by user - -## Phase Tasks - -### Task 1: Read TestRail Test Case - -**Actions**: -1. Ask user for TestRail test case ID if not provided -2. Use TestRail MCP to retrieve test case details: - ``` - Use: user-testrail-get_case with case_id - ``` -3. Extract key information: - - Test case ID and title - - Test description - - Preconditions - - Test steps (step-by-step actions) - - Expected results for each step - - Overall test goal - - Priority and test type -4. Document findings in test plan file - -**Expected Output**: Complete understanding of what needs to be tested according to TestRail. - -### Task 2: Read Confluence Documentation - -**Actions**: -1. Ask user for Confluence page ID/URL or search terms if not provided -2. Use Atlassian Confluence MCP to find related documentation: - ``` - Use: user-mcp-atlassian-confluence_search with query - Or: user-mcp-atlassian-confluence_get_page with page_id - ``` -3. Extract relevant information: - - Feature description and purpose - - Business context and user flows - - Technical specifications - - UI/UX requirements - - Integration points - - Known limitations or constraints -4. Cross-reference with TestRail test case -5. Document findings in test plan file - -**Expected Output**: Business and technical context for the feature being tested. - -### Task 3: Create Initial Test Plan Document - -**Actions**: -1. Create `agents/plans/aqa-.md` file with: - - Test case reference (TestRail ID and link) - - Feature name and description - - Test goal - - Expected results summary - - Confluence references - - Initial understanding of test scope -2. Structure document for additions in subsequent phases - -**Template**: + + + +Gather test case details from TestRail and feature context from Confluence, cross-reference, and produce initial test plan document. + + + +- Phase 1 of 8 in `aqa-flow` +- Input: TestRail case ID or URL, Confluence page ID or search terms (from user) +- Output: `agents/plans/aqa-.md` with test case info and feature context +- Skills: `mcp-testrail-data-collection`, `mcp-confluence-data-collection` +- Prerequisite: TestRail and Atlassian MCPs configured + + + +1. Confirm inputs from user +2. Gather TestRail data +3. Gather Confluence data +4. Cross-reference and assemble test plan +5. Validate and update state + + + +1. Verify TestRail case ID or URL provided (ask user if missing) +2. Verify Confluence page ID or search terms provided (ask user if missing) + + + +1. USE SKILL `mcp-testrail-data-collection` +2. Extract: case ID, title, description, preconditions, step-by-step actions with expected results, test goal, priority, test type + + + +1. USE SKILL `mcp-confluence-data-collection` +2. Extract: feature description and purpose, business context, user flows, technical specifications, UI/UX requirements, integration points, known limitations + + + +1. Validate TestRail steps against Confluence feature context — note gaps or contradictions +2. Create `agents/plans/aqa-.md` using the template below +3. Verify test plan file created + +Output template for `agents/plans/aqa-.md`: + ```markdown # AQA Test Plan - @@ -98,7 +74,6 @@ Gather all required information from external sources (TestRail and Confluence) - Expected: [Result] 2. [Step 2] - Expected: [Result] -... ### Expected Overall Result [Final expected outcome] @@ -106,69 +81,43 @@ Gather all required information from external sources (TestRail and Confluence) ## Feature Context ### Business Purpose -[From Confluence - why this feature exists] +[From Confluence] ### Technical Details -[From Confluence - how it works] +[From Confluence] ### User Flow -[From Confluence - user journey] - -## Notes -- [Any observations or questions] - ---- -## Phase 2: Requirements Clarification -[To be filled in Phase 2] - -## Phase 3: Code Analysis -[To be filled in Phase 3] - -## Phase 4: Selector Identification -[To be filled in Phase 4] - -## Phase 5: Selector Implementation -[To be filled in Phase 5] - -## Phase 6: Test Implementation -[To be filled in Phase 6] -``` - -## Completion Criteria +[From Confluence] -- [ ] TestRail test case retrieved and documented -- [ ] Confluence documentation retrieved and documented -- [ ] Test plan file created with all Phase 1 information -- [ ] Test goal clearly understood -- [ ] Expected results documented -- [ ] `agents/aqa-state.md` updated with Phase 1 completion - -## Update State File - -After completing Phase 1, update `agents/aqa-state.md`: - -```markdown -### Phase 1: Data Collection -- Completed: [DateTime] -- TestRail Case: [ID/URL] -- Confluence Pages: [URLs] -- Test Goal: [Brief description] -- Expected Result: [Brief description] -- Test Plan File: agents/plans/aqa-.md -``` - -Mark Phase 1 as completed and Phase 2 as current. - -## Next Phase - -Proceed to **Phase 2: Requirements Clarification** by executing: +## Cross-Reference Notes +- [Gaps, contradictions, or observations between TestRail and Confluence] ``` -ACQUIRE aqa-phase2-md FROM KB -``` - -## Important Notes -- **No Assumptions**: If TestRail or Confluence data is incomplete, note it in the test plan -- **Ask Questions**: If user hasn't provided IDs/URLs, ask for them -- **Document Everything**: Capture all details even if they seem minor -- **Cross-Reference**: Ensure TestRail and Confluence information aligns + + + +1. Update `agents/aqa-state.md`: + - TestRail Case: [ID/URL] + - Confluence Pages: [URLs] + - Test Goal: [brief] + - Test Plan File: [path] + - Phase 1 completion timestamp +2. Mark Phase 1 complete, Phase 2 current + + + +- TestRail test case retrieved and documented +- Confluence documentation retrieved and documented +- Cross-reference between TestRail and Confluence completed +- Test plan file created with all Phase 1 information +- Test goal clearly understood +- Expected results documented + + + +- Assuming test data when TestRail or Confluence data is incomplete — note gaps instead +- Skipping cross-reference between TestRail and Confluence +- Not asking user for IDs/URLs when missing + + + diff --git a/instructions/r3/core/workflows/aqa-flow-requirements-clarification.md b/instructions/r3/core/workflows/aqa-flow-requirements-clarification.md index f0d21565..9b5f3d54 100644 --- a/instructions/r3/core/workflows/aqa-flow-requirements-clarification.md +++ b/instructions/r3/core/workflows/aqa-flow-requirements-clarification.md @@ -1,100 +1,43 @@ --- name: aqa-flow-requirements-clarification -description: Phase 2 of AQA workflow - Requirements Clarification and Assertion Definition +description: Phase 2 of AQA workflow - Requirements Clarification and Assertion Definition (USER INTERACTION REQUIRED) alwaysApply: false +tags: [] baseSchema: docs/schemas/phase.md --- -# Phase 2: Requirements Clarification - -## Objective - -Fill gaps in understanding, clarify unknowns, and define explicit assertions before implementation. This phase requires **USER INTERACTION**. - -## Prerequisites - -- Phase 1 completed -- Test plan file created with TestRail and Confluence data -- Initial understanding of test requirements - -## Phase Tasks - -### Task 1: Review Gathered Information for Gaps - -**Actions**: -1. Read the test plan file from Phase 1 -2. Analyze information for completeness: - - Are test steps clear and unambiguous? - - Are expected results specific and measurable? - - Is test data defined? - - Are edge cases identified? - - Are success criteria explicit? -3. Create list of unknowns and ambiguities -4. Identify areas requiring clarification - -**Expected Output**: List of gaps and questions that need user input. - -### Task 2: Define Explicit Assertions - -**Actions**: -1. For each test step, define what will be verified: - - UI element states (visible, enabled, disabled, checked) - - Text content (exact match, contains, pattern) - - Data values (equals, greater than, within range) - - Navigation (URL, page title, breadcrumbs) - - Error messages or success notifications -2. Specify assertion types: - - Presence assertions (element exists) - - State assertions (element state matches expected) - - Content assertions (text/value matches expected) - - Behavioral assertions (action triggers expected response) -3. Document all assertions in test plan - -**Expected Output**: Complete list of explicit, measurable assertions for each test step. - -### Task 3: Prepare Questions for User - -**Actions**: -1. Formulate specific questions about: - - **Test Coverage**: What exactly should be tested? Are there specific scenarios? - - **Success Criteria**: How do we know the test passed? What defines success? - - **Edge Cases**: What unusual conditions should be covered? What can go wrong? - - **Test Data**: What specific data should be used? Any special values? - - **Expected Behavior**: What should happen in each step? Any timing considerations? - - **Out of Scope**: What should NOT be tested in this test case? -2. Group questions logically -3. Prioritize questions (critical vs. nice-to-have) - -**Example Questions**: -``` -Critical Questions: -1. When clicking [Button X], should we verify only [Element Y] appears, - or also check that [Element Z] disappears? -2. For the success message, should we match exact text "Success!" - or just verify message contains "Success"? -3. What test data should be used for [Field A]? Any specific format? - -Edge Cases: -4. What should happen if [Condition X] occurs during the test? -5. Should we test with empty/invalid data, or only valid data? - -Test Flow: -6. Are there any timing dependencies (waits, delays)? -7. Should this test clean up data after execution? -``` - -**Expected Output**: Organized list of specific questions for user. - -### Task 4: Ask User and Wait for Answers - -**Actions**: -1. Present questions to user in clear, organized format -2. Explain why each question is important -3. **WAIT** for user to provide all answers -4. **DO NOT PROCEED** to Phase 3 until answers received -5. Document user responses in test plan - -**User Interaction Format**: + + + +Fill gaps in understanding, clarify unknowns. Requires user interaction. + + + +- Phase 2 of 8 in `aqa-flow` +- Input: test plan file `agents/plans/aqa-.md` from Phase 1 +- Output: user answers, updated test plan +- Prerequisite: Phase 1 complete +- HITL: user answers required before Phase 3 + + + +1. Define gaps in test case understanding +2. Ask user for clarification gaps +3. Wait for user answers +4. Update test plan file `agents/plans/aqa-.md` according to clarifications from user +5. Document and update state + + + +1. USE SKILL `aqa-requirements-elicitation` +2. Preprate list unknowns and ambiguities + + + +1. USE SKILL `questioning` +2. Present structured questions to user + + ``` I need clarification on the following to ensure accurate test implementation: @@ -115,14 +58,18 @@ I need clarification on the following to ensure accurate test implementation: Please provide answers so I can proceed with test implementation. ``` + + -**Expected Output**: Complete answers from user to all questions. -### Task 5: Update Test Plan with Clarifications + +1. **STOP AND WAIT** for user to provide all answers + -**Actions**: -1. Add new section to test plan: - ```markdown + +1. Process user answers +2. Add new section to test plan file `agents/plans/aqa-.md` according to clarifications from user according to the format: +``markdown ## Phase 2: Requirements Clarification ### Questions Asked @@ -131,16 +78,6 @@ Please provide answers so I can proceed with test implementation. ### User Responses [Documented answers] - ### Defined Assertions - #### Step 1: [Action] - - Assert: [Explicit assertion] - - Verification: [How to verify] - - #### Step 2: [Action] - - Assert: [Explicit assertion] - - Verification: [How to verify] - ... - ### Edge Cases to Cover - [Edge case 1] - [Edge case 2] @@ -151,49 +88,21 @@ Please provide answers so I can proceed with test implementation. - [Data requirement 2] ... ``` -2. Update test steps with explicit assertions -3. Add edge case scenarios if applicable -4. Document test data requirements - -**Expected Output**: Enhanced test plan with all clarifications and explicit assertions documented. - -## Completion Criteria - -- [ ] All gaps in understanding identified -- [ ] Explicit assertions defined for each test step -- [ ] Questions prepared and presented to user -- [ ] **User answers received and documented** -- [ ] Test plan updated with Phase 2 information -- [ ] Edge cases identified and documented -- [ ] Test data requirements specified -- [ ] `agents/aqa-state.md` updated with Phase 2 completion - -## Update State File - -After completing Phase 2, update `agents/aqa-state.md`: - -```markdown -### Phase 2: Requirements Clarification -- Completed: [DateTime] -- Questions Asked: [Count] -- Assertions Defined: [Count] -- Edge Cases: [List] -- User Responses: Documented in test plan -``` - -Mark Phase 2 as completed and Phase 3 as current. - -## Next Phase - -After user provides all answers, proceed to **Phase 3: Code Analysis** by executing: -``` -ACQUIRE aqa-phase3-md FROM KB -``` - -## Important Notes - -- **CRITICAL**: DO NOT proceed to Phase 3 without user answers -- **No Assumptions**: Never assume answers - always ask user -- **Explicit Over Implicit**: Every assertion must be measurable and verifiable -- **User Authority**: User has final say on requirements and expected behavior -- **Document Everything**: Record all questions and answers for traceability + + + +1. Update `agents/aqa-state.md`: + - Questions Asked: [count] + - User Responses: Documented in test plan file +2. Mark Phase 2 complete, Phase 3 current + + + +- All gaps identified and questions prepared +- User answers received and documented +- Test plan updated with clarifications +- Edge cases identified +- Test data requirements specified + + + diff --git a/instructions/r3/core/workflows/aqa-flow-selector-identification.md b/instructions/r3/core/workflows/aqa-flow-selector-identification.md index cfb05c64..7f782640 100644 --- a/instructions/r3/core/workflows/aqa-flow-selector-identification.md +++ b/instructions/r3/core/workflows/aqa-flow-selector-identification.md @@ -1,357 +1,61 @@ --- name: aqa-flow-selector-identification -description: Phase 4 of AQA workflow - Selector Identification and Page Source Request +description: Phase 4 of AQA workflow - Selector Identification (USER INTERACTION CONDITIONALLY REQUIRED) alwaysApply: false +tags: [] baseSchema: docs/schemas/phase.md --- -# Phase 4: Selector Identification - -## Objective - -Identify missing selectors needed for test implementation. First attempt to find selectors from frontend source code. If frontend code is unavailable or selectors cannot be found, request page source from user. - -## Prerequisites - -- Phase 1, 2, and 3 completed -- Test plan updated with assertions and code analysis -- Understanding of existing Page Objects -- Understanding of test requirements -- Frontend code analysis completed (if available) - -## Phase Tasks - -### Task 1: Map Test Steps to Required Interactions - -**Actions**: -1. Review test plan - each test step and assertion -2. For each step, list all UI interactions needed: - - Elements to click (buttons, links, tabs, etc.) - - Elements to type into (input fields, textareas) - - Elements to select from (dropdowns, radio buttons, checkboxes) - - Elements to verify (text, images, status indicators) - - Elements to wait for (loading spinners, notifications) -3. Create interaction map: - ```markdown - ### Test Step 1: Navigate to Login Page - Required Interactions: - - Click: "Login" navigation link - - Verify: Login page heading "Sign In" - - ### Test Step 2: Enter Credentials - Required Interactions: - - Type: Username field - - Type: Password field - - Click: "Login" button - - ### Test Step 3: Verify Dashboard - Required Interactions: - - Verify: Welcome message text - - Verify: User profile icon visible - - Verify: Dashboard title "My Dashboard" - ``` - -**Expected Output**: Complete list of all required UI interactions. - -### Task 2: Check Existing Page Objects for Selectors - -**Actions**: -1. For each required interaction, check if selector already exists in Page Objects: - ```markdown - ### Selector Availability Check - - ✅ LoginPage.usernameInput - EXISTS - ✅ LoginPage.passwordInput - EXISTS - ✅ LoginPage.loginButton - EXISTS - ❌ DashboardPage.welcomeMessage - MISSING - ❌ DashboardPage.dashboardTitle - MISSING - ✅ DashboardPage.userProfileIcon - EXISTS - ``` -2. Categorize findings: - - **Available**: Selectors that already exist - - **Missing**: Selectors that need to be added - - **Uncertain**: Selectors that might exist under different names -3. For each missing selector, note: - - Which Page Object should contain it - - What element it represents - - How it will be used (click, verify, type) - -**Expected Output**: Clear list of missing selectors with their intended Page Objects. - -### Task 3: Search Frontend Source Code for Selectors (if available) - -**Actions**: -1. Check if frontend source code is available -2. If available, search for missing selectors: - ``` - Use: Grep or SemanticSearch - Search for: data-testid, data-test, component names, feature names - ``` -3. For each missing selector, search relevant component files: - - Look for `data-testid="selector-name"` attributes - - Check component props and interfaces - - Identify stable `id`, `className`, or ARIA attributes - - Note element types (button, input, div, etc.) -4. Document found selectors: - ```markdown - ### Selectors Found in Frontend Code - - #### DashboardPage - - Welcome Message: `data-testid="welcome-message"` (h2 element, line 45) - - Dashboard Title: `data-testid="dashboard-title"` (h1 element, line 38) - - Notification Bell: `data-testid="notification-bell"` (button element, line 52) - - #### SettingsPage - - Email Input: `data-testid="email-input"` (input type="email", line 67) - - Save Button: `data-testid="save-settings-btn"` (button element, line 89) - ``` -5. Categorize findings: - - **Found in Frontend**: Selectors identified in source code - - **Still Missing**: Selectors not found (need user page source) -6. If ALL selectors found, skip Task 4 and proceed to Task 5 -7. If frontend code NOT available or selectors still missing, proceed to Task 4 - -**Expected Output**: Selectors found from frontend code OR list of selectors still needing page source. - -### Task 4: Prepare Page Source Request (if needed) - -**Actions**: -1. **ONLY execute this task if**: - - Frontend source code is not available, OR - - Some selectors could not be found in frontend code -2. Group missing selectors by page/component: - ```markdown - ### Missing Selectors by Page - - #### Dashboard Page - - Welcome message (text element showing "Welcome, [username]") - - Dashboard title (heading with "My Dashboard") - - Notification bell icon (clickable icon in header) - - #### Settings Page - - Email input field (editable field for email) - - Save button (button to save settings) - - Success notification (message shown after save) - ``` -2. For each page, specify what HTML is needed: - - Specific elements to locate - - Surrounding context (parent elements, siblings) - - Any attributes to note (id, class, data-testid, aria-label) -3. Create detailed request for user: - ``` - I need page source HTML to identify the correct selectors. Please provide: - - ### Dashboard Page HTML - Please save the HTML for these elements: - - The welcome message element (showing "Welcome, [username]") - - The dashboard title/heading - - The notification bell icon - - To capture: - 1. Open browser Developer Tools (F12) - 2. Right-click the element → Inspect - 3. In Elements tab, right-click the element → Copy → Copy outerHTML - 4. Include parent containers for context (2-3 levels up) - 5. Save HTML in files in `agents/aqa/{TICKET-KEY}/page-sources/` directory - (e.g., dashboard-page.html, settings-page.html) - - ### Settings Page HTML - [Similar instructions for Settings page elements] - ``` - -**Expected Output**: Clear, specific request for user with instructions on how to provide HTML. - -**Note**: If all selectors were found in Task 3 (frontend code), skip this task entirely. - -### Task 5: Create Directory and Wait for User to Add Page Sources (if needed) - -**Actions**: -1. Create directory for page sources using Shell tool: - ```bash - mkdir -p agents/aqa/{TICKET-KEY}/page-sources/ - ``` -2. Present request to user in clear, actionable format -3. Explain why page source is needed and where to save files -4. Provide clear instructions with file naming convention -5. **WAIT** for user to add page source files to the directory -6. **DO NOT PROCEED** to Task 5 until user confirms files are added -7. List directory contents using LS tool to verify files exist -8. Ask clarifying questions if provided HTML is unclear or incomplete - -**User Interaction Format**: -``` -To implement the test accurately, I need to identify the correct selectors. - -I've created a directory for page sources at: `agents/aqa/{TICKET-KEY}/page-sources/` - -## Missing Selectors -I need HTML for the following elements: - -### Dashboard Page -- [Element 1 description] -- [Element 2 description] - -### Settings Page -- [Element 1 description] -- [Element 2 description] - -## How to Provide Page Sources - -1. Open the application and navigate to each page -2. For each page, create a separate HTML file in the `page-sources/` directory: - - `dashboard-page.html` for Dashboard Page elements - - `settings-page.html` for Settings Page elements -3. Use this naming convention: `{page-name}.html` in kebab-case -4. In each HTML file, include the HTML for all relevant elements with surrounding context -5. To capture HTML: - - Open Developer Tools (F12 or Right-click → Inspect) - - Right-click the element → Inspect - - In Elements/Inspector tab, find the element - - Right-click the HTML → Copy → Copy outerHTML - - Include 2-3 parent levels for context - - Paste into the appropriate `.html` file - -**Please add the page source files to the `agents/aqa/{TICKET-KEY}/page-sources/` directory and let me know when ready.** -``` - -**Expected Output**: User adds HTML files to the `page-sources/` directory and confirms. - -**Note**: If all selectors were found in Task 3 (frontend code), skip this task entirely. - -### Task 6: Analyze Provided HTML and Document Selectors (if needed) - -**Actions**: -1. List files in `agents/aqa/{TICKET-KEY}/page-sources/` directory using LS tool, then read each page source file using Read tool -2. For each missing selector, determine best selector strategy: - - **Preferred**: `data-testid` or `data-test` attributes - - **Good**: Unique `id` attributes - - **Acceptable**: Specific `class` names (if stable) - - **Last Resort**: CSS selectors by structure or XPath -3. Document selected selectors: - ```markdown - ### Identified Selectors - - #### DashboardPage Selectors - - **Welcome Message** - - HTML: `

` - - Selector: `[data-testid="welcome-message"]` - - Type: CSS - - Usage: Text verification - - **Dashboard Title** - - HTML: `

My Dashboard

` - - Selector: `#dashboard-title` - - Type: CSS (ID) - - Usage: Text verification - - **Notification Bell** - - HTML: `