diff --git a/docs/web/docs/aqa-flow.md b/docs/web/docs/aqa-flow.md index 4670a6c3..8bb9e6d9 100644 --- a/docs/web/docs/aqa-flow.md +++ b/docs/web/docs/aqa-flow.md @@ -10,15 +10,17 @@ permalink: /docs/aqa-flow/ ## TL;DR -Use AQA Flow when you need Rosetta-guided automated UI test work tied to a real TestRail case or QA scenario. The workflow gathers TestRail and Confluence context, clarifies assertions, analyzes existing test architecture, identifies selectors without guessing, implements the test, then stops so you can run it and return the report. +Use AQA Flow when you need Rosetta-guided automated UI test work: either **integrated** (TestRail + Confluence as primary sources when you use that path) or **minimal-input / agent-led** (you supply URLs, page source or a selector map, scenario summary, and expectations so the agent can implement without pulling case data from integrations). In both cases the workflow clarifies assertions, analyzes existing test architecture, identifies selectors without guessing, implements the test, then stops so you can run it and return the report. -This is a strict sequential workflow. Phases build on each other, `agents/aqa-state.md` is updated after each phase, and the coding agent must not skip ahead. Mandatory user interaction happens in Phase 2, Phase 6, Phase 7, and Phase 8. Phase 4 asks for page HTML only when frontend code or stable selectors are not available. +Phase 1 records the chosen **execution mode** in the test plan and `agents/aqa-state.md`. Later phases still run in order; only what Phase 1 captures changes. + +This is a strict sequential workflow. Phases build on each other, `agents/aqa-state.md` is updated after each phase, and the coding agent must not skip ahead. Mandatory user interaction happens in **Phase 1** (**Mandatory** until mode is explicit in text or after your reply to the mode question—see Phase 1 row in the table below), Phase 2, Phase 6, Phase 7, and Phase 8. Phase 4 asks for page HTML only when frontend code or stable selectors are not available. ## When To Use This Workflow -- Automate a TestRail case against an existing UI. +- Automate a TestRail case against an existing UI (**integrated** path), or implement UI automation from **explicit** user-supplied pages, DOM or selector maps, and scenario context (**minimal-input / agent-led** path). - Add or update UI automation while reusing existing Page Objects and helpers. -- Build a test from TestRail steps plus Confluence context instead of starting from code guesses. +- Build a test from TestRail steps plus Confluence context instead of starting from code guesses (integrated), or from a documented minimal-input package you provide (minimal-input). - Diagnose failing automated test output and prepare grounded fixes. - Add selectors to Page Objects before test implementation when the current test layer is incomplete. @@ -31,15 +33,30 @@ This is a strict sequential workflow. Phases build on each other, `agents/aqa-st ## Before You Start -Prepare the inputs this workflow explicitly depends on: +Prepare inputs by **execution mode** (the agent confirms this in Phase 1 if you have not already stated it): + +**Integrated AQA** + +- A TestRail case ID or another precise QA target the agent will read via integrations. +- Confluence page IDs, URLs, or search terms for the feature under test (when Confluence is in scope). + +**Minimal-input / agent-led** + +- A clear scenario goal (what the test must prove). +- Entry URLs or navigation to the UI under test. +- UI grounding you will not let the agent invent: e.g. page source or DOM snapshots, a selector map, pointers to existing Page Objects in the repo, and/or frontend paths the agent will use in Phase 3. Anything deferred here must be resolved or explicitly re-deferred in Phase 2 with your text. + +**All modes** -- A TestRail case ID or another precise QA target. -- Confluence page IDs, URLs, or search terms for the feature under test. - Access to the target repository test code. - Access to existing Page Objects, similar tests, helper utilities, and project test conventions. - `agents/user-app/project_description.md` if the project uses it as the coding-standards source. - Any files under `agents/user-instructions/` that define test creation rules, test report locations, or team-specific conventions. +**Execution mode.** Phase 1 **must not infer** integrated vs minimal-input from context (missing TestRail, attachments, etc.). The agent **must not** create `agents/plans/aqa-*.md` or write `**AQA execution mode**` until your message contains `integrated` or `minimal-input` (or spelled equivalents) **or** you have answered the agent’s one-time mode question. If your first message does not **literally** include those keywords, the agent **asks once** and **waits** until you pick. Add one of those words to your opening message when you already know the mode and want to skip the question. + +**Integrated Path A and MCP routing.** On the integrated path, live pulls from TestRail, Confluence, or Jira are gated by **`agents/mcp-capability.yaml`** plus optional **`agents/user-instructions/mcp-guidance.md`** and any **user override** in your message (see [`mcp-capability-interaction.md`](https://github.com/griddynamics/rosetta/blob/main/instructions/r2/core/workflows/mcp-capability-interaction.md) and the example [`mcp-capability.example.yaml`](https://github.com/griddynamics/rosetta/blob/main/instructions/r2/core/templates/mcp-capability.example.yaml)). When an integration is **questionnaire**, you paste or answer for that source instead of MCP calls. + You also get better results when the project already has strong shared Rosetta context. Keep shared setup in [Usage Guide](/rosetta/docs/usage-guide/#customization), especially `docs/CONTEXT.md`, `docs/ARCHITECTURE.md`, and `docs/TECHSTACK.md`. ## How To Start @@ -62,6 +79,10 @@ Analyze this failing automated test report for case C9012 and prepare correction Extend the existing checkout automation with a new TestRail scenario and reuse current Page Objects. ``` +```text +Minimal-input: automate checkout success on staging. I will provide the URL, a selector map, and saved HTML under agents/aqa/.../page-sources/. No TestRail MCP. +``` + ## How Rosetta Shapes This Workflow Rosetta provides the instructions. The coding agent executes them. Rosetta itself does not read your source code or test data. @@ -73,12 +94,13 @@ For this workflow, the always-active Rosetta behavior changes the user experienc - Human review is built into the workflow, not added later. The agent must stop for answers, test execution, report handoff, and approval before corrections. - The workflow is state-driven. After each phase, the agent updates `agents/aqa-state.md` so the work can resume cleanly after interruptions. - Existing project architecture wins over convenience. The agent must inspect existing tests, Page Objects, utilities, and coding standards before writing new automation. +- Phase 1 establishes **execution mode**; the agent must not silently assume TestRail/Confluence when you chose minimal-input, must not invent pages or selectors you did not supply or defer with consent, and must **not infer** mode when your message lacks explicit `integrated` or `minimal-input` keywords (or spelled equivalents)—it **asks and waits** instead. ## Workflow At A Glance | Phase | What you provide | What the coding agent does | What you get | Mandatory workflow stop | |---|---|---|---|---| -| 1. Data Collection | TestRail case, Confluence reference | Reads external QA/business context and creates the test plan | `agents/plans/aqa-.md`, initial `agents/aqa-state.md` | None | +| 1. Data Collection | **Integrated:** TestRail case, Confluence reference. **Minimal-input:** scenario, URLs/navigation, UI grounding (or explicit deferral to Phase 2). **Either:** include `integrated` or `minimal-input` (or spelled equivalents) in your first message if you already know the mode | If those keywords are missing, **asks once** for integrated vs minimal-input and **waits**; then pulls TestRail/Confluence **or** records your minimal-input package | `agents/plans/aqa-.md` (includes `AQA execution mode`), initial `agents/aqa-state.md` | **Mandatory** until mode is explicit in text or after your reply to the mode question | | 2. Requirements Clarification | Answers about assertions, data, edge cases, scope | Turns vague steps into explicit, measurable assertions | Updated test plan with assertions, edge cases, test data rules | Mandatory user answers before Phase 3 | | 3. Code Analysis | Repository test code, project docs, user instruction files | Analyzes framework, conventions, Page Objects, similar tests, helpers, optional frontend code | Updated test plan with architecture findings and target test location | None | | 4. Selector Identification | Frontend code if available, otherwise page HTML when requested | Maps test steps to UI elements and identifies missing selectors without guessing | Selector map, page-source request if needed, updated plan/state | Mandatory user input only if selectors cannot be grounded from code | @@ -94,7 +116,7 @@ Recommended review still matters throughout the workflow, but those checks are a ```mermaid flowchart TD - A[Start AQA request]:::start --> B[Phase 1 Data Collection]:::phase + A[Start AQA request]:::start --> B[Phase 1: execution mode + data collection]:::phase B --> C[Phase 2 Requirements Clarification]:::phase C --> C1{User answered?}:::hitl C1 -- No --> C2[Wait for answers]:::action @@ -132,7 +154,12 @@ sequenceDiagram U->>A: Request automated QA work R-->>A: Enforce sequential phases, no assumptions, state tracking - A->>X: Read TestRail case and Confluence context + alt Integrated AQA + A->>X: Read TestRail case and Confluence context + else Minimal-input + A->>U: Confirm mode; collect URLs, page source or selector map, scenario + U->>A: Provide agreed minimal-input package + end A->>F: Create agents/plans/aqa-test-name.md and agents/aqa-state.md A->>U: Ask clarification questions for assertions, scope, data, edge cases U->>A: Provide answers @@ -160,36 +187,39 @@ sequenceDiagram ### Phase 1: Data Collection Goal: -- Gather the external requirement sources before any code analysis starts. +- Record **execution mode** and gather everything later phases need: either **integrated** (TestRail + Confluence) or **minimal-input** (your explicit package), before code analysis starts. What you provide: -- TestRail case ID if it was not already in the request. -- Confluence page ID, URL, or search terms if they were not already in the request. +- **Integrated:** TestRail case ID if not already in the request; Confluence page ID, URL, or search terms if not already in the request. +- **Minimal-input:** Scenario goal; entry URLs or navigation; UI grounding (page source paths, selector map, repo pointers to Page Objects/components, and/or frontend paths for Phase 3). Optional TestRail/Confluence IDs for traceability only, without pretending MCP data exists when you did not pull it. +- If the agent asks you to choose a mode, reply with **`integrated`** or **`minimal-input`** (or the spelled labels). If you already know the mode, put one of those words in your **first** message so Phase 1 can skip the question. What the coding agent does: -- Reads the TestRail case through TestRail MCP. -- Reads Confluence documentation through Atlassian Confluence MCP. -- Extracts test steps, expected results, business context, user flow, and technical details. +- If the opening request lacks explicit mode keywords (`integrated`, `minimal-input`, `minimal input`, `Integrated AQA`, or `minimal-input / agent-led`), **asks once**, **stops**, and **waits**—**does not infer** mode from missing TestRail, attachments, or environment. +- Confirms **integrated** vs **minimal-input** and documents `AQA execution mode` in the plan and state. +- **Integrated:** reads TestRail via TestRail MCP and Confluence via Atlassian MCP; extracts steps, expected results, and feature context. +- **Minimal-input:** records your checklist in the plan (no inventing pages or selectors); may defer specific items to Phase 2 only with your explicit consent, noted in the plan. - Creates `agents/plans/aqa-.md` as the main working test plan. -- Starts `agents/aqa-state.md`. +- Starts or updates `agents/aqa-state.md`. Artifacts: - `agents/plans/aqa-.md` - `agents/aqa-state.md` Recommended review: -- Check that the right TestRail case and Confluence sources were used. -- Check that the test goal and expected result summary reflect the real scenario. +- **Integrated:** Check that the right TestRail case and Confluence sources were used and that the goal matches the scenario. +- **Minimal-input:** Check that the plan’s checklist is complete enough that later phases will not force the agent to guess DOM or flows; resolve or approve deferrals before Phase 3. ### Phase 2: Requirements Clarification Goal: -- Turn collected test steps into explicit, measurable assertions before implementation starts. +- Turn collected test steps (or draft steps from minimal-input) into explicit, measurable assertions before implementation starts. What you provide: - Answers about success criteria, exact assertions, test data, timing constraints, edge cases, and out-of-scope cases. What the coding agent does: +- Reads **AQA execution mode** from the plan. In **minimal-input**, closes gaps in the Phase 1 checklist (especially UI grounding and test data) instead of re-demanding TestRail/Confluence unless you want traceability. - Reviews Phase 1 for gaps and ambiguities. - Defines assertion types for each step: presence, state, content, and behavior. - Prepares targeted clarification questions. @@ -291,10 +321,11 @@ Goal: - Implement the automated test using the approved assertions, Page Objects, and local project conventions. What you provide: -- Usually no new business input if earlier phases were complete. +- Usually no new business input if earlier phases were complete (for **integrated**, full 1–5; for **minimal-input**, checklist + Phase 2 assertions satisfied per the instruction workflow). - Later in the phase, you must execute the test yourself. What the coding agent does: +- Re-checks execution mode and, for **minimal-input**, the documented checklist and Phase 2 assertions before coding; stops and asks if material inputs are still missing. - Reviews the full test plan. - Chooses the target test file or confirms the planned location. - Sets up imports, suite structure, hooks, setup steps, actions, assertions, and cleanup using the current project style. @@ -368,7 +399,7 @@ Recommended review: Review each handoff like a QA and test-automation lead, not like a passive approver. These are recommended review checkpoints, not additional mandatory workflow stops beyond the ones listed in the summary table. -- After Phase 1, verify the workflow is anchored to the right TestRail case and the right Confluence context. +- After Phase 1, verify **execution mode** is recorded correctly. **Integrated:** right TestRail case and Confluence context. **Minimal-input:** the plan’s checklist (URLs, UI grounding, scenario) matches what you intend; no silent substitutes for missing sources. - After Phase 2, verify assertions are measurable and complete. Weak assertions or missing edge cases here will corrupt every later phase. - After Phase 3, verify the agent found the right Page Objects, similar tests, helpers, and coding standards. - After Phase 4, verify selectors came from real frontend code or supplied HTML. Reject any invented locator strategy. @@ -383,6 +414,7 @@ If the page-source request, test plan, or correction proposal is vague, stop the These customizations materially improve AQA Flow: +- State **integrated** or **minimal-input** in your **first** message when you already know the mode (saves one round-trip); otherwise the agent **must** ask before Path A or Path B. - Keep `agents/user-app/project_description.md` accurate. This workflow treats it as the single source of truth for coding standards. - Use `agents/user-instructions/` for team-specific test creation rules, report locations, naming conventions, setup rules, and assertion preferences. - If frontend source code is available, keep it accessible so Phase 3 and Phase 4 can ground selectors there first. @@ -402,7 +434,7 @@ Common artifacts from this workflow: Common artifact content: -- external requirement capture from TestRail and Confluence +- external requirement capture from TestRail and Confluence (**integrated**), or a documented **minimal-input package** in the plan (**minimal-input**) - explicit assertions and edge cases - framework and project-structure analysis - selector inventory and source tracking @@ -417,7 +449,7 @@ Conditional outputs: ## Common Mistakes -- Starting without a real TestRail case or equivalent QA source. +- Starting **integrated** AQA without a real TestRail case (or equivalent) and Confluence context when those are the agreed sources, or starting **minimal-input** without enough UI grounding and expecting the agent to invent selectors. - Letting Phase 2 stay vague and then expecting correct assertions later. - Skipping `agents/user-instructions/` even though it may define report locations or team-specific rules. - Guessing selectors instead of grounding them from frontend code or provided HTML. diff --git a/docs/web/docs/testgen-flow.md b/docs/web/docs/testgen-flow.md index a21b5275..a298327a 100644 --- a/docs/web/docs/testgen-flow.md +++ b/docs/web/docs/testgen-flow.md @@ -35,8 +35,8 @@ Prepare the minimum inputs that materially affect output quality: - A Jira ticket key or Jira ticket URL. - Confluence page URLs if you already know the relevant pages. This lets the workflow skip weak auto-search results. -- Access to the Jira and Confluence retrieval path used in your project. -- If you want Phase 6, access to TestRail and the ability to provide or create the target section and share its `section_id`. Project and suite details may also be needed when your setup cannot detect them from the current ticket and user profile. +- Access to the Jira and Confluence retrieval path used in your project **when Phase 1 is guided**; otherwise be ready to paste ticket and wiki content (questionnaire path). +- If you want Phase 6 with **guided** TestRail export, TestRail MCP access and the ability to provide or create the target section and share its `section_id`. For **questionnaire** export, be ready to supply `project_id`, `suite_id`, and `section_id`, then import manually or paste back TestRail case IDs. - Time to review every phase result, especially `questions.md`, `requirements.md`, and `test-scenarios.md`. Workflow-specific preparation that improves results: @@ -48,6 +48,16 @@ Workflow-specific preparation that improves results: For shared Rosetta setup and project-context customization, see [Usage Guide](/rosetta/docs/usage-guide/). +## MCP capability file and questionnaire path + +Some workspaces do not attach Jira, Confluence, or TestRail MCP servers to the agent. TestGen (and AQA integrated Path A) still run if you add **`agents/mcp-capability.yaml`** at the repo root (copy the shape from [`mcp-capability.example.yaml`](https://github.com/griddynamics/rosetta/blob/main/instructions/r2/core/templates/mcp-capability.example.yaml)) and optionally **`agents/user-instructions/mcp-guidance.md`** for stable IDs and naming rules. + +- **`mcp.mode: absent`** — use structured questionnaires in Phase 1 and Phase 6; no live MCP calls. +- **`mcp.mode: capable`** with per-integration `true`/`false` — mix guided and questionnaire per system (for example Jira guided, Confluence paste-only). +- **User override** in the task text (for example “no MCP” or “use MCP”) wins for that run over the YAML file. + +Workflow details live in [`mcp-capability-interaction.md`](https://github.com/griddynamics/rosetta/blob/main/instructions/r2/core/workflows/mcp-capability-interaction.md) (routing), [`mcp-tool-resolution.md`](https://github.com/griddynamics/rosetta/blob/main/instructions/r2/core/workflows/mcp-tool-resolution.md) and [`agent-action-resolution.md`](https://github.com/griddynamics/rosetta/blob/main/instructions/r2/core/workflows/agent-action-resolution.md) (placeholder binding), [`testgen-flow-data-collection.md`](https://github.com/griddynamics/rosetta/blob/main/instructions/r2/core/workflows/testgen-flow-data-collection.md), and [`testgen-flow-test-case-export.md`](https://github.com/griddynamics/rosetta/blob/main/instructions/r2/core/workflows/testgen-flow-test-case-export.md). + ## How To Start Example prompts that match the workflow: diff --git a/gain.json b/gain.json index f560df38..0f59958f 100644 --- a/gain.json +++ b/gain.json @@ -46,7 +46,11 @@ "vfs": "Virtual File System, after stripping release and org prefix from the physical file path", "bundling": "Multiple documents at the same VFS resource path are merged into a single XML response" }, + "rosettaInstructionUpdates": { + "aqaFlowPhase1ExecutionMode": "AQA workflow: new Phase 1 Task 0 locks execution mode to integrated (TestRail+Confluence Path A) or minimal-input / agent-led (Path B). Until the user message contains explicit mode keywords (integrated, minimal-input, Integrated AQA, or minimal-input / agent-led) or the user answers the one-time mode question, the agent must not create agents/plans/aqa-*.md, must not write **AQA execution mode**, and must not mark Phase 1 complete in agents/aqa-state.md. Inferring minimal-input from missing TestRail or local HTML alone is forbidden. Phase 2 and Phase 6 instructions aligned; docs/web/docs/aqa-flow.md updated; mirrored under plugins (Cursor, Copilot, Codex).", + "mcpCapabilityBPlusATestgenAqa": "Shared fragment mcp-capability-interaction.md: workspace agents/mcp-capability.yaml (see instructions/r2/core/templates/mcp-capability.example.yaml) plus user-message overrides route guided MCP vs questionnaire. AQA Path A and TestGen Phases 1 and 6 acquire this fragment; TestGen adds Step 4Q (Confluence questionnaire), Phase 6 Step 0/1Q/5Q for TestRail without MCP; plugin mirrors updated." + }, "versions": { - "rosetta": "2.0.19" + "rosetta": "2.0.20" } } diff --git a/instructions/r2/core/templates/mcp-capability.example.yaml b/instructions/r2/core/templates/mcp-capability.example.yaml new file mode 100644 index 00000000..a268c86c --- /dev/null +++ b/instructions/r2/core/templates/mcp-capability.example.yaml @@ -0,0 +1,20 @@ +# Copy to workspace root as: agents/mcp-capability.yaml +# See: instructions/r2/core/workflows/mcp-capability-interaction.md +# Placeholder binding (when workflows use tokens): mcp-tool-resolution.md, agent-action-resolution.md + +mcp: + # "capable" = agent may call MCPs for true integrations after reading mcp-guidance.md + # "absent" = use questionnaire + user paste; do not call external MCPs + mode: capable + + # Per-integration toggles (optional). If omitted, agent may still try when mode is capable + # and workflow needs that integration—unless workflow says otherwise. + jira: true + # Confluence: one product, two YAML key names used by different workflows (not two layers): + # - TestGen (`testgen-flow-data-collection.md`, etc.) reads `confluence`. + # - AQA (`aqa-flow-data-collection.md`) reads `atlassian_confluence` (Atlassian MCP naming). + # If you use both workflow families, keep both booleans in sync. If you use only one family, + # you may omit the key the other family reads (omitted + capable → guided per that workflow). + confluence: true + testrail: true + atlassian_confluence: true diff --git a/instructions/r2/core/workflows/agent-action-resolution.md b/instructions/r2/core/workflows/agent-action-resolution.md new file mode 100644 index 00000000..75fb60a9 --- /dev/null +++ b/instructions/r2/core/workflows/agent-action-resolution.md @@ -0,0 +1,18 @@ +--- +name: agent-action-resolution +description: Maps `{agent-action:…}` tokens to host IDE or agent-runtime file tools (read, write, patch). +alwaysApply: false +baseSchema: docs/schemas/phase.md +--- + +# `{agent-action:…}` resolution (host file tools) + +Use when a workflow uses **`{agent-action:read-file}`**, **`{agent-action:write-file}`**, or **`{agent-action:patch-file}`**. These are **not** remote MCP calls and are **not** covered by **`mcp-tool-resolution.md`**. + +| Token | Intent | +|-------|--------| +| `{agent-action:read-file}` | Read a file from the workspace via the host agent’s file-read tool | +| `{agent-action:write-file}` | Create or overwrite a workspace file via the host agent’s file-write tool | +| `{agent-action:patch-file}` | Apply targeted edits via the host agent’s patch / search-replace tool | + +**Resolution:** Map each token to the host IDE’s or agent runtime’s **documented** file operation (names differ: e.g. `Read`, `Write`, `StrReplace`, `apply_patch`). If the host exposes no matching capability, **STOP**, tell the user, and **WAIT** — do not substitute an MCP tool. diff --git a/instructions/r2/core/workflows/aqa-flow-data-collection.md b/instructions/r2/core/workflows/aqa-flow-data-collection.md index 34fc0898..a9e7253d 100644 --- a/instructions/r2/core/workflows/aqa-flow-data-collection.md +++ b/instructions/r2/core/workflows/aqa-flow-data-collection.md @@ -9,19 +9,61 @@ baseSchema: docs/schemas/phase.md ## Objective -Gather all required information from external sources (TestRail and Confluence) to understand test requirements and expected behavior. +Gather everything needed for later phases: either from **TestRail and Confluence** (**integrated AQA**) or from **user-supplied artifacts** (**minimal-input / agent-led**). Always record the chosen **execution mode** in the test plan and state file. ## Prerequisites -- TestRail MCP configured and accessible -- Atlassian (Confluence) MCP configured and accessible -- Test case ID or requirement provided by user +- User intent is clear enough to start Phase 1 (IDs, URLs, or a plain-language automation goal). +- **Integrated AQA path**: For **guided** MCP interaction (see Task 0b), TestRail MCP and Atlassian (Confluence) MCP must be usable when you run Path A MCP steps. For **questionnaire** interaction, do **not** call those MCPs; collect the same facts via user answers and paste into the test plan. +- **Minimal-input path**: No MCP requirement for TestRail/Confluence; Task 0b **does not apply** to Path B for those integrations. If something on the checklist is missing, **ask** — do not guess. + +## MCP capability reference + +Resolve **guided vs questionnaire** per **`mcp-capability-interaction.md`** (ACQUIRE FROM KB). Config file: **`agents/mcp-capability.yaml`** (copy from `instructions/r2/core/templates/mcp-capability.example.yaml` in Rosetta repo). Optional guidance: **`agents/user-instructions/mcp-guidance.md`**. ## Phase Tasks -### Task 1: Read TestRail Test Case +### Task 0: Confirm execution mode + +**Actions**: +1. Check whether the **current task text** (or a prior user message in the same session the user instructs you to treat as binding) already names the mode **explicitly**, using any of these (case-insensitive): `integrated`, `minimal-input`, `minimal input`, `Integrated AQA`, or `minimal-input / agent-led`. The token must be **user-authored** in that message (not something you paraphrase or infer). If yes, set the mode to `integrated` or `minimal-input` accordingly and **skip steps 2–3**; go directly to **Path A** or **Path B** matching that mode. +2. If **not** explicit: **Do not infer** mode from clues such as missing TestRail IDs, presence of HTML attachments, “sandbox,” “no MCP,” or vague phrases like “I don’t have TestRail yet.” **Forbidden:** choosing `minimal-input` because TestRail was omitted, the repo looks small, or the user only supplied local HTML—those are **not** mode labels. Informal phrases (“lightweight,” “without TestRail,” “bring my own selectors”) **do not** count as explicit; they still require steps 2–3. **Ask once** with both options, for example: + - **Integrated AQA**: "We pull the official case and docs from TestRail/Confluence and follow the full chain." + - **Minimal-input / agent-led**: "You provide URLs, page source or a selector map, scenario and expectations; we still run phases 2–8 in order but Phase 1 records your package instead of MCP pulls." +3. **STOP** and **WAIT** until the user replies with a clear **integrated** or **minimal-input** (or the spelled labels in step 1). If the reply is still ambiguous, ask **one** narrowing question that still requires those labels; **do not** choose a path without those words from the user. +4. **Hard gate — before Path A or Path B:** Do **not** create or update `agents/plans/aqa-*.md`, do **not** write `**AQA execution mode**` in any artifact, and do **not** mark Phase 1 complete in `agents/aqa-state.md` until step **1** matched or step **3** completed. If you already started Path A/B without satisfying Task 0, **stop**, discard partial mode lines, run steps 2–3, then continue. +5. Ensure the Phase 1 test plan created in Path A or Path B includes `**AQA execution mode**` and that `agents/aqa-state.md` reflects the mode when Phase 1 is marked complete (see **Update State File** below). + +**Expected Output**: Mode is either read from **explicit user keywords** (step 1) or obtained only after **mandatory** ask-and-wait (steps 2–3); then proceed with **Path A** or **Path B** (not both). + +--- + +### Task 0b: Resolve MCP interaction (Path A only) + +**Skip this entire task for Path B (minimal-input).** **Actions**: +1. ACQUIRE **mcp-capability-interaction.md** FROM KB (if not already loaded this phase). +2. Read **`agents/mcp-capability.yaml`** if it exists; apply **user override** rules from that fragment (task text beats file). If file missing and user has not overridden, ask **one** yes/no: “Use live MCP for TestRail/Confluence in this workspace?” — **WAIT**; treat **No** as questionnaire, **Yes** as guided for both integrations until a YAML file exists. +3. Derive **per integration** (TestRail vs Confluence) whether the source is **guided** (MCP allowed) or **questionnaire** (no MCP): if `mcp.mode` is `absent`, both are **questionnaire**; if `capable`, use `mcp.testrail` / `mcp.atlassian_confluence` when present (boolean); if a key is **omitted** under `capable`, default that integration to **guided**. +4. Record in the test plan and `agents/aqa-state.md` Phase 1 notes: `TestRail source: guided|questionnaire`, `Confluence source: guided|questionnaire`, plus `MCP interaction source:` (`agents/mcp-capability.yaml` | user override | default question`). +5. **Questionnaire leg:** For each integration still **questionnaire** and not yet documented in the plan, run **one** numbered questionnaire (combine TestRail + Confluence in a single message if both need data). **STOP** and **WAIT**. Merge answers under **User-provided (MCP absent) — TestRail** and/or **User-provided (MCP absent) — Confluence** in the working plan (or append to the plan draft before Task A3). +6. **Guided leg:** If **`agents/user-instructions/mcp-guidance.md`** exists and at least one integration is **guided**, read it before the first MCP call for a **guided** integration. Then continue to Task A1 / A2 and follow the **guided** branches there only. + +**Expected Output**: TestRail and Confluence are each classified **guided** or **questionnaire**; questionnaire content is captured before MCP calls; no MCP call runs for a **questionnaire** integration. + +--- + +### Path A — Integrated AQA (TestRail + Confluence) + +**Prerequisite:** Task 0 already set execution mode to **integrated** (step 1 keyword match or step 3 user reply). Do not enter Path A without that. + +### Task A1: Read TestRail Test Case + +**Actions**: +- **If Task 0b resolved questionnaire for TestRail:** Ensure the test plan already contains **User-provided (MCP absent) — TestRail** content from Task 0b. If missing, **STOP**, ask numbered questions (case ID for traceability, title, steps, expected results, preconditions), **WAIT**, merge into the plan. **Do not** call `user-testrail-get_case`. Skip the guided-only steps below for TestRail. +- **If Task 0b resolved guided for TestRail:** Perform steps 1–4. + 1. Ask user for TestRail test case ID if not provided 2. Use TestRail MCP to retrieve test case details: ``` @@ -39,9 +81,12 @@ Gather all required information from external sources (TestRail and Confluence) **Expected Output**: Complete understanding of what needs to be tested according to TestRail. -### Task 2: Read Confluence Documentation +### Task A2: Read Confluence Documentation **Actions**: +- **If Task 0b resolved questionnaire for Confluence:** Ensure the test plan contains **User-provided (MCP absent) — Confluence** content from Task 0b. If missing, **STOP**, ask numbered questions (URLs, pasted excerpts, or summaries the user authorizes), **WAIT**, merge into the plan. **Do not** call `user-mcp-atlassian-confluence_*`. Skip guided-only steps below for Confluence. +- **If Task 0b resolved guided for Confluence:** Perform steps 1–5. + 1. Ask user for Confluence page ID/URL or search terms if not provided 2. Use Atlassian Confluence MCP to find related documentation: ``` @@ -60,10 +105,11 @@ Gather all required information from external sources (TestRail and Confluence) **Expected Output**: Business and technical context for the feature being tested. -### Task 3: Create Initial Test Plan Document +### Task A3: Create initial test plan document (integrated) **Actions**: 1. Create `agents/plans/aqa-.md` file with: + - **AQA execution mode**: `integrated` - Test case reference (TestRail ID and link) - Feature name and description - Test goal @@ -77,6 +123,7 @@ Gather all required information from external sources (TestRail and Confluence) # AQA Test Plan - **Created**: [DateTime] +**AQA execution mode**: integrated **TestRail Case**: [ID/URL] **Feature**: [Feature Name] **Status**: Phase 1 Complete @@ -134,14 +181,112 @@ Gather all required information from external sources (TestRail and Confluence) [To be filled in Phase 6] ``` +--- + +### Path B — Minimal-input / agent-led + +**Prerequisite:** Task 0 already set execution mode to **minimal-input** (step 1 keyword match or step 3 user reply). Do not enter Path B because TestRail was missing or files were attached—only because the user chose **minimal-input** in text. + +### Task B1: Offer agent-led implementation and required artifacts + +**Actions**: +1. Tell the user clearly that the agent **can** implement the test **without** TestRail/Confluence **if** they provide enough grounded detail — but the agent **must not** invent pages, selectors, or flows. +2. List what you still need at minimum (see Task B2 checklist). **WAIT** until the user either supplies items or explicitly defers an item to Phase 2 (then note "open in Phase 2" in the plan). + +**Expected Output**: User understands the tradeoff and what to supply (or what is deferred to Phase 2 with explicit consent). + +### Task B2: Collect minimal-input package + +**Actions**: +1. Capture the following in the test plan (fill or mark *deferred to Phase 2* with user agreement): + - **Scenario / goal**: What behavior the automated test must prove (plain language). + - **Entry / URLs**: Starting URL(s) or navigation path to reach the UI under test. + - **UI grounding** (at least one required before ending Phase 1 unless user explicitly defers to Phase 2): e.g. saved page source or DOM snapshot paths, a **selector map** (purpose → locator), pointers to existing Page Objects in the repo, and/or frontend file paths the agent will use in Phase 3. + - **Preconditions and test data**: Accounts, feature flags, seed data — or *unknown — Phase 2*. + - **Optional**: TestRail/Confluence IDs "for traceability only" without MCP — record as references, not as substitute for missing UI grounding. +2. **Do not** mark Phase 1 complete if UI grounding and scenario goal are both missing and the user has not agreed to defer. + +**Expected Output**: Minimal-input checklist documented in the test plan. + +### Task B3: Create initial test plan document (minimal-input) + +**Actions**: +1. Create `agents/plans/aqa-.md` using the template below (adapt section headings if the scenario is small). +2. For TestRail/Confluence fields, use `N/A` or optional reference links — do not pretend MCP data exists. + +**Template**: +```markdown +# AQA Test Plan - + +**Created**: [DateTime] +**AQA execution mode**: minimal-input +**TestRail Case**: [N/A or ID for traceability only] +**Confluence**: [N/A or links for traceability only] +**Feature**: [Feature Name] +**Status**: Phase 1 Complete + +## Minimal-input package + +### Scenario / goal +[What the test must validate] + +### Entry and navigation +- URLs: [...] +- Steps to reach the UI: [...] + +### UI grounding +- Page source / DOM snapshots: [paths or attach instructions] +- Selector map or locators: [...] +- Repo pointers (Page Objects / components): [...] + +### Preconditions and test data +[Or: deferred to Phase 2 — user confirmed] + +### Notes and open questions +- [...] + +## Test Case Information (synthetic) + +### Test Steps (draft) +1. [Step — can be refined in Phase 2] +... + +### Expected Overall Result +[Draft — refined in Phase 2] + +--- +## Phase 2: Requirements Clarification +[To be filled in Phase 2] + +## Phase 3: Code Analysis +[To be filled in Phase 3] + +## Phase 4: Selector Identification +[To be filled in Phase 4] + +## Phase 5: Selector Implementation +[To be filled in Phase 5] + +## Phase 6: Test Implementation +[To be filled in Phase 6] +``` + ## Completion Criteria +**All modes** +- [ ] Task 0 complete: execution mode is **either** named by explicit user keywords (`integrated` / `minimal-input` / spelled equivalents) **or** obtained after the mandatory ask-and-wait; mode recorded in test plan and state +- [ ] Test plan file `agents/plans/aqa-.md` created +- [ ] Test goal clearly understood (or explicitly deferred to Phase 2 with user consent) +- [ ] `agents/aqa-state.md` updated with Phase 1 completion + +**Integrated AQA only** - [ ] TestRail test case retrieved and documented - [ ] Confluence documentation retrieved and documented -- [ ] Test plan file created with all Phase 1 information -- [ ] Test goal clearly understood -- [ ] Expected results documented -- [ ] `agents/aqa-state.md` updated with Phase 1 completion +- [ ] Expected results from sources documented + +**Minimal-input only** +- [ ] Minimal-input checklist captured in the test plan +- [ ] UI grounding present or explicitly deferred to Phase 2 with user approval ## Update State File @@ -150,8 +295,10 @@ After completing Phase 1, update `agents/aqa-state.md`: ```markdown ### Phase 1: Data Collection - Completed: [DateTime] -- TestRail Case: [ID/URL] -- Confluence Pages: [URLs] +- Execution mode: [integrated | minimal-input] +- TestRail Case: [ID/URL or N/A] +- Confluence Pages: [URLs or N/A] +- Minimal-input checklist: [brief summary or N/A] - Test Goal: [Brief description] - Expected Result: [Brief description] - Test Plan File: agents/plans/aqa-.md @@ -168,7 +315,9 @@ ACQUIRE aqa-phase2-md FROM KB ## Important Notes -- **No Assumptions**: If TestRail or Confluence data is incomplete, note it in the test plan -- **Ask Questions**: If user hasn't provided IDs/URLs, ask for them -- **Document Everything**: Capture all details even if they seem minor -- **Cross-Reference**: Ensure TestRail and Confluence information aligns +- **No Assumptions**: If data is incomplete for the chosen mode, note it in the test plan and ask — never guess selectors or flows. +- **Ask Questions**: If user hasn't provided IDs/URLs (integrated) or checklist items (minimal-input), ask for them. +- **Document Everything**: Capture all details even if they seem minor. +- **Integrated path only**: Cross-reference TestRail and Confluence information for alignment. +- **Minimal-input path**: Prefer an explicit **selector map** or page source over vague descriptions; deferrals must be user-visible in the plan. +- **Task 0**: Never infer execution mode from context; only explicit keywords or a user reply after the mandatory question count. diff --git a/instructions/r2/core/workflows/aqa-flow-requirements-clarification.md b/instructions/r2/core/workflows/aqa-flow-requirements-clarification.md index f0d21565..f7fb9a9b 100644 --- a/instructions/r2/core/workflows/aqa-flow-requirements-clarification.md +++ b/instructions/r2/core/workflows/aqa-flow-requirements-clarification.md @@ -13,16 +13,23 @@ Fill gaps in understanding, clarify unknowns, and define explicit assertions bef ## Prerequisites -- Phase 1 completed -- Test plan file created with TestRail and Confluence data -- Initial understanding of test requirements +- Phase 1 completed with a test plan file +- Phase 1 documents either **integrated** sources (TestRail + Confluence as used) **or** **minimal-input** execution mode with the checklist from Phase 1 (scenario, URLs, UI grounding or explicit deferral) +- Initial understanding of test requirements (may be draft in minimal-input mode) ## Phase Tasks +### Minimal-input mode (read first) + +If the test plan shows **`AQA execution mode: minimal-input`**: +- Treat Phase 1's checklist as authoritative for what the user already supplied; **do not** re-demand TestRail/Confluence data unless the user wants it for traceability. +- Still run gap analysis: anything deferred from Phase 1 **must** be resolved or explicitly re-deferred here with user text. +- Assertions must still be explicit and measurable before Phase 3; **never** invent selectors the user did not supply, defer, or point to in repo/page source. + ### Task 1: Review Gathered Information for Gaps **Actions**: -1. Read the test plan file from Phase 1 +1. Read the test plan file from Phase 1 and note **AQA execution mode** (`integrated` vs `minimal-input`). 2. Analyze information for completeness: - Are test steps clear and unambiguous? - Are expected results specific and measurable? @@ -197,3 +204,4 @@ ACQUIRE aqa-phase3-md FROM KB - **Explicit Over Implicit**: Every assertion must be measurable and verifiable - **User Authority**: User has final say on requirements and expected behavior - **Document Everything**: Record all questions and answers for traceability +- **Minimal-input mode**: Questions should close gaps in the Phase 1 checklist (especially UI grounding and test data), not re-litigate integrated AQA unless the user switches mode explicitly diff --git a/instructions/r2/core/workflows/aqa-flow-test-implementation.md b/instructions/r2/core/workflows/aqa-flow-test-implementation.md index a16d294e..e92ab613 100644 --- a/instructions/r2/core/workflows/aqa-flow-test-implementation.md +++ b/instructions/r2/core/workflows/aqa-flow-test-implementation.md @@ -13,13 +13,16 @@ Create automated test following project standards, integrating all Page Objects ## Prerequisites -- All previous phases (1-5) completed -- Test plan fully documented with all details -- Page Objects updated with all required selectors +- All previous phases (1-5) completed **for the chosen execution mode** (see below) +- Test plan fully documented with all details **including** `AQA execution mode` and, if `minimal-input`, the **minimal-input checklist** from Phase 1 (and any Phase 2 resolutions of deferrals) +- Page Objects updated with all required selectors **or** a clear plan from Phases 4–5 that matches user-supplied grounding (minimal-input) - Explicit assertions defined - Code architecture understood -## Phase Tasks +### Execution modes and prerequisites + +- **Integrated AQA**: The bullets above mean phases 1–5 were completed in full with TestRail/Confluence-backed Phase 1 where applicable. +- **Minimal-input / agent-led** (recorded in Phase 1): Phases 1–5 count as satisfied only if the test plan contains the **minimal-input checklist** (scenario, URLs/navigation, UI grounding or approved Phase 2 deferrals) and **explicit assertions from Phase 2**. **Before writing code**, re-read that checklist and Phase 2 assertions; if anything material is still missing, **STOP**, ask the user, and do not invent selectors, pages, or flows. ### Task 1: Review Test Implementation Plan diff --git a/instructions/r2/core/workflows/aqa-flow.md b/instructions/r2/core/workflows/aqa-flow.md index 00f7c725..f7931333 100644 --- a/instructions/r2/core/workflows/aqa-flow.md +++ b/instructions/r2/core/workflows/aqa-flow.md @@ -10,20 +10,23 @@ baseSchema: docs/schemas/workflow.md ## Context -This agent handles end-to-end test automation from requirements gathering to test implementation. It uses TestRail, Confluence, and project documentation to create automated tests following existing architecture and coding standards. +This agent handles end-to-end test automation from requirements gathering to test implementation. It uses TestRail, Confluence, and project documentation when you run **integrated AQA**; in **minimal-input / agent-led** mode the user supplies pages, DOM or selector details, and scenario context instead (see Phase 1). In all modes, tests follow existing architecture and coding standards. ### Critical Requirements - **ONE PHASE AT A TIME**: Read phase file, execute, update state, move to next. - **DO NOT SKIP PHASES**: Each builds on previous. - **NO ASSUMPTIONS**: Never assume selectors, flows, or data. Always ask the user if information is missing. -- **USER INTERACTION**: Wait for user responses when questions are asked or files are requested. Phase 2, 6, 7, and 8 always require user input. Phase 4 requires user input ONLY if frontend code is unavailable or selectors cannot be found +- **MCP ROUTING (B + A)**: Before calling external MCPs in integrated Path A, follow **`mcp-capability-interaction`** using `agents/mcp-capability.yaml` and any user override in the task text. **Guided** = use MCP after optional `agents/user-instructions/mcp-guidance.md`. **Questionnaire** = no MCP calls for that integration; user answers structured questions; record answers in the plan. ACQUIRE `mcp-capability-interaction.md` FROM KB when executing Phase 1 Path A. +- **EXECUTION MODE (Phase 1, first)**: The user must **explicitly** choose **integrated** or **minimal-input** (case-insensitive), or use the spelled labels **Integrated AQA** or **minimal-input / agent-led**, in **their** message before you record a mode. If none of those appear, **ask once** with both options, **STOP**, and **WAIT**—**do not infer** mode from missing TestRail, attachments, sandbox, or “no MCP.” **Do not** write `agents/plans/aqa-*.md` or set `**AQA execution mode**` until that keyword match or the user’s reply to the question. **Forbidden:** assigning `minimal-input` because TestRail was omitted or the project looks like a demo. After the user answers, document the mode in the Phase 1 test plan and `agents/aqa-state.md`. Do not silently invent missing integrations or missing UI grounding. User customizations elsewhere do **not** waive this rule when mode was not explicitly named. +- **USER INTERACTION**: Wait for user responses when questions are asked or files are requested. **Phase 1** requires user input until execution mode is explicit or the user answers the one-time mode question (see **EXECUTION MODE**). Phase 2, 6, 7, and 8 always require user input. Phase 4 requires user input ONLY if frontend code is unavailable or selectors cannot be found - **STATE TRACKING**: Update `agents/aqa-state.md` after each phase. - **MUST** use todo tasks for tracking progress. - Prioritize ACCURACY over SPEED! ### User Customizations +- The **execution mode** rule cannot be waived by “only run some phases” unless the user message already contains the mode keywords or the user answers the one-time mode question. - If user did not specify any preferences perform all steps except optional. - User CAN customize and ask only for specific phases OR phases could have been done already OR towards specific goal OR for specific case, in this case LISTEN and ADOPT to the user. @@ -31,9 +34,10 @@ This agent handles end-to-end test automation from requirements gathering to tes **Phase 1: Data Collection** [aqa-flow-data-collection.md] 1. ACQUIRE aqa-flow-data-collection.md FROM KB -2. Execute phase instructions -3. Update `agents/aqa-state.md` -4. Validate gathered data +2. ACQUIRE mcp-capability-interaction.md FROM KB (resolve MCP guided vs questionnaire before Path A MCP calls) +3. Execute phase instructions (**start with execution mode confirmation** — integrated vs minimal-input) +4. Update `agents/aqa-state.md` +5. Validate gathered data (checklist for chosen mode is complete before Phase 2) **Phase 2: Requirements Clarification** [aqa-flow-requirements-clarification.md] ⭐ **USER INTERACTION REQUIRED** 1. ACQUIRE aqa-flow-requirements-clarification.md FROM KB @@ -91,7 +95,8 @@ Create/update `agents/aqa-state.md` after each phase: **Last Updated**: [DateTime] **Current Phase**: [1-8 or COMPLETE] -**TestRail Case**: [Test Case ID/URL] +**AQA execution mode**: [integrated | minimal-input] +**TestRail Case**: [Test Case ID/URL or N/A in minimal-input] **Feature**: [Feature Name] ## Phase Completion Status @@ -109,8 +114,10 @@ Create/update `agents/aqa-state.md` after each phase: ### Phase 1: Data Collection - Completed: [DateTime] -- TestRail Case: [ID/URL] -- Confluence Pages: [URLs] +- Execution mode: [integrated | minimal-input] +- TestRail Case: [ID/URL or N/A] +- Confluence Pages: [URLs or N/A] +- Minimal-input checklist: [summary or N/A — see Phase 1 test plan] - Test Goal: [Brief description] - Expected Result: [Brief description] @@ -158,6 +165,7 @@ Create/update `agents/aqa-state.md` after each phase: ## Important Notes - **Sequential Execution**: Phases build on each other, must execute in order. +- **Execution modes**: **Integrated AQA** uses TestRail and Confluence in Phase 1 when those tools are in scope. **Minimal-input / agent-led** uses the Phase 1 checklist (pages, DOM or selectors, scenario context); later phases still run in order, but Phase 1 documents what substitutes for external case management. Without an **explicit** user mode label (`integrated` or `minimal-input` / spelled equivalents), Phase 1 must obtain that label by asking—never infer from context alone. - **No Assumptions Rule**: Always ask user when information is missing - never guess selectors, flows, or test data. - **Architecture First**: Always analyze existing code structure before implementing new tests. - **Reuse Over Creation**: Prefer adding to existing files and using existing Page Objects over creating new ones. diff --git a/instructions/r2/core/workflows/mcp-capability-interaction.md b/instructions/r2/core/workflows/mcp-capability-interaction.md new file mode 100644 index 00000000..4f4fae4d --- /dev/null +++ b/instructions/r2/core/workflows/mcp-capability-interaction.md @@ -0,0 +1,64 @@ +--- +name: mcp-capability-interaction +description: Routes guided vs questionnaire MCP usage from workspace YAML and user text, records interaction source, and defines where capability files live. Placeholder binding uses sibling workflows. +alwaysApply: false +baseSchema: docs/schemas/phase.md +--- + +# MCP capability interaction (shared) + +Use when a workflow **may** call **remote MCPs** (Jira, Confluence, TestRail, Atlassian). It does **not** bind **`{integration-action:…}`** (see **`mcp-tool-resolution.md`**) or **`{agent-action:…}`** (see **`agent-action-resolution.md`**). **AQA** Path A and **TestGen** Phases 1 and 6 load this for **routing** (guided vs questionnaire) only. + +## B — `agents/mcp-capability.yaml` (workspace root) + +Read before the first MCP call. Keys optional; unknown keys ignored. + +```yaml +mcp: + mode: capable # required if file exists: capable | absent + jira: true + confluence: true + testrail: true + atlassian_confluence: true +``` + +- **`confluence` vs `atlassian_confluence`:** **Not aliases** — both refer to **Confluence** MCP on/off for **guided vs questionnaire**, but **different workflows read different keys** (TestGen uses `confluence`; AQA data collection uses `atlassian_confluence`). If you use **both** families, set **both** to the same boolean. If you use **one** family, you may omit the key the other workflow reads. Template: `instructions/r2/core/templates/mcp-capability.example.yaml`. + +- **`mode: capable`** — MCP allowed. Per integration: **`true`** or key omitted → **guided**. **`false`** → **questionnaire** (no MCP; user text). Read **`agents/user-instructions/mcp-guidance.md`** first when it exists. Do not re-ask values already in that file unless missing. +- **`mode: absent`** — Do not invoke MCP. Parent workflow runs questionnaire (**STOP**, **WAIT**), merges into plan / `raw-data.md` / agreed artifact. Never fabricate MCP output. +- **File missing** — Apply **A** if the user message overrides; else **one** question whose **scope** (which integrations) is defined by the **parent workflow**, not this fragment. **Example — TestGen Phase 1:** when `agents/mcp-capability.yaml` is absent, **Step 2b** in `testgen-flow-data-collection.md` asks once for **Jira and Confluence together**; **TestRail** is resolved again in **TestGen Phase 6**. **No** → questionnaire for the integrations that question covers. **Yes** → capable for that run; recommend adding YAML. + +## A — user message (wins over file) + +Record in state/plan. **Absent:** `MCP absent`, `no MCP`, `without MCP`, `questionnaire only`, `paste-only`. **Capable:** `MCP capable`, `use MCP`, `MCP enabled`, `pull from Jira` when clearly a live pull. User wins on conflict. + +## Capable mode without guidance + +Use task text and YAML only; questionnaire **only** for missing fields. Do not invent secrets. + +## Questionnaire (`absent` or integration `false`) + +Parent supplies **numbered** questions for fields MCP would return. **STOP**; **WAIT**. Merge into the same artifacts as the MCP path. Label **user-provided (no MCP)**. + +## Recording + +When routing is resolved, write into the phase plan or state file: either **one** line `MCP interaction: guided | questionnaire (source: agents/mcp-capability.yaml | user override | default question)` when a single scope applies, or **one line per integration** (or per parent-defined field such as TestGen `Jira source` / `Confluence source` / Phase 6 TestRail export), each with the same `(source: ...)` options. + +**Before leaving this fragment:** Re-read the plan or state lines you wrote for MCP routing and confirm: **(a)** each line lists exactly one source among `agents/mcp-capability.yaml`, `user override`, and `default question`; **(b)** using one aggregate line vs one line per integration (or per parent-defined field such as TestGen `Jira source` / `Confluence source`) matches what the **parent workflow** requires. Correct the artifact if a check fails. + +## Path conventions (Rosetta vs target workspace) + +Use whenever a workflow mentions both Rosetta artifacts and workspace files. + +| Location | Path | Notes | +|----------|------|--------| +| **Rosetta repo / distribution only** | `instructions/r2/core/templates/mcp-capability.example.yaml` | Template for **authoring** `mcp-capability.yaml`. Not expected inside an arbitrary customer application checkout unless copied from Rosetta. | +| **Target workspace** | `agents/mcp-capability.yaml` | Optional capability file; **expected at workspace root** only. | +| **Target workspace** | `agents/user-instructions/mcp-guidance.md` | Optional per-workspace MCP hints. | + +If **`agents/mcp-capability.yaml`** is missing at workspace root, treat capability config as **missing** for routing: follow the parent workflow’s questionnaire / default-question path and record **MCP interaction source** accordingly (do not search the customer repo for Rosetta template paths). + +## Related workflows (placeholder binding) + +- **`mcp-tool-resolution.md`** — Map **`{integration-action:…}`** to MCP tool ids when **guided**. +- **`agent-action-resolution.md`** — Map **`{agent-action:…}`** to host read / write / patch tools. diff --git a/instructions/r2/core/workflows/mcp-tool-resolution.md b/instructions/r2/core/workflows/mcp-tool-resolution.md new file mode 100644 index 00000000..8695ba68 --- /dev/null +++ b/instructions/r2/core/workflows/mcp-tool-resolution.md @@ -0,0 +1,34 @@ +--- +name: mcp-tool-resolution +description: Binds `{integration-action:…}` tokens to MCP tool identifiers in guided runs using a single-pass scan of the session tool list. +alwaysApply: false +baseSchema: docs/schemas/phase.md +--- + +# `{integration-action:…}` resolution (MCP tools) + +Use when a parent workflow is **guided** for an integration and must turn a **`{integration-action:…}`** token into a concrete MCP tool call. **ACQUIRE only when needed** (not for questionnaire-only paths). Routing (guided vs questionnaire) is defined in **`mcp-capability-interaction.md`**. + +Parent workflows define token spellings and parameter blocks. **Questionnaire** integrations must **not** use this file to call MCP. + +## Deterministic tool selection (single pass over MCP list) + +Let `T` = the list of MCP tool names (or ids) available in **this** session. Perform **exactly one** linear scan over `T` (no second pass, no “try again” loops). + +1. **Required substrings** — From the token after `integration-action:`, take segments split by `-` (e.g. `jira-get-issue` → `jira`, `get`, `issue`). Build candidate set **C** = tools in `T` whose identifier **contains every remaining segment** (case-insensitive). If **C** is empty, drop segments in this order until **C** is non-empty or one segment remains: drop **`get`**, then **`search`**, then **`list`** or **`pages`**, then repeatedly drop the **rightmost** remaining segment. +2. **If C has exactly one tool** — use it. +3. **If C has more than one tool** — Prefer tools explicitly allowed or named in **`agents/mcp-capability.yaml`** and **`agents/user-instructions/mcp-guidance.md`** for that integration. If still more than one, **STOP** and ask **one** disambiguation question using the **format below** (no guess). +4. **If C is empty after step 1 reductions** — **Do not invent** a tool. Record resolution failure in state, then follow **Questionnaire** rules for that integration in **`mcp-capability-interaction.md`** (**STOP**, **WAIT**), then continue per the parent workflow’s routing table. + +Map **Parameters** from the phase `Action` blocks to the chosen tool’s schema (adapt snake_case vs camelCase per tool docs). + +## Disambiguation question format + +When step **3** leaves multiple candidates, use exactly this shape: + +1. One line stating the token and that multiple MCP tools matched. +2. Numbered lines **`1`** through **`N`**: each line is **one verbatim tool id** from the remaining candidate set (same spelling as in `T`). +3. Line **`N+1`**: `Cancel — use questionnaire for this integration instead`. +4. **Parse the reply:** accept only an integer **k**. If **k = N+1**, do not call MCP for that integration; follow the parent’s questionnaire path. If **1 ≤ k ≤ N**, select the **k**-th listed tool in order. Any other reply → ask again once with the same list. + +Vendor tool names in examples elsewhere are **illustrations**; **`T`** is authoritative. diff --git a/instructions/r2/core/workflows/testgen-flow-data-collection.md b/instructions/r2/core/workflows/testgen-flow-data-collection.md index 4696f7d5..3e14221a 100644 --- a/instructions/r2/core/workflows/testgen-flow-data-collection.md +++ b/instructions/r2/core/workflows/testgen-flow-data-collection.md @@ -9,9 +9,12 @@ baseSchema: docs/schemas/phase.md ## Prerequisites -- MUST be starting new test generation flow -- User provided Jira ticket key or URL -- Jira MCP configured and accessible +- MUST be starting new test generation flow. +- User provided Jira ticket key or URL. +- **Acquire workflows:** ACQUIRE **`mcp-capability-interaction.md`** FROM KB (MCP guided vs questionnaire; always for Phase 1). Before the first guided **`{integration-action:*}`** call in Steps 3–4, ACQUIRE **`mcp-tool-resolution.md`**. Before **Persist** blocks using **`{agent-action:*}`** in Steps 5–6, ACQUIRE **`agent-action-resolution.md`**. If an alias fails, use canonical paths under **`instructions/r2/core/workflows/`** with the same basename. +- **Step 2b only:** Run the numbered **Step 2b checklist** in this file (not the whole fragment for later steps unless needed). +- **Before MCP calls:** Do **not** call Jira/Confluence MCPs until Step **2b** completes. +- **URL placeholders:** `{jira-host}` and `{confluence-host}` in examples mean the customer’s real web hostnames (including subdomain), not fixed vendor domains. ## Objective @@ -19,35 +22,29 @@ Extract all relevant data from Jira ticket and related Confluence documentation ## Requirements +**Token resolution (canonical):** **`{integration-action:*}`** → **`mcp-tool-resolution.md`**. **`{agent-action:*}`** → **`agent-action-resolution.md`**. MCP guided vs questionnaire and recording → **`mcp-capability-interaction.md`** (ACQUIRE per **Prerequisites**). Later steps say **see Token resolution** instead of repeating this. + ### Step 1: Parse Initial User Input **Extract from user's initial prompt**: -1. **Jira ticket**: Key or URL (REQUIRED) -2. **Confluence URLs**: List of URLs (REQUIRED) +1. **Jira ticket**: Key or URL (REQUIRED). +2. **Confluence URLs** (optional): Parse and store for later steps. How they affect routing is defined **only** in the **Step 2b** matrix (after `Jira source` / `Confluence source` exist). **Supported formats**: ``` "Analyze requirements for PROJ-123" -"Analyze requirements for PROJ-123 with Confluence: https://confluence.com/display/PROJ/Page" +"Analyze requirements for PROJ-123 with Confluence: https://{confluence-host}/display/PROJ/Page" "Analyze PROJ-123, Confluence pages: URL1, URL2, URL3" -"PROJ-123 + https://confluence.com/display/PROJ/Auth" +"PROJ-123 + https://{confluence-host}/display/PROJ/Auth" ``` **Parse Confluence URLs**: - Extract from patterns: "with Confluence", "Confluence:", "Confluence docs:", "Confluence pages:" - Accept comma-separated or line-separated URLs - URLs may be: - - Display format: `https://confluence.company.com/display/SPACE/Page+Title` - - Direct format: `https://confluence.company.com/pages/viewpage.action?pageId=123456` - - Short format: `https://confluence.company.com/x/AbCdEf` - -**If Confluence URLs provided**: -- Extract page IDs from URLs -- Skip automatic search (Step 3) -- Go directly to retrieving specified pages - -**If no Confluence URLs provided**: -- Proceed with automatic search (Step 3) + - Display format: `https://{confluence-host}/display/SPACE/Page+Title` + - Direct format: `https://{confluence-host}/pages/viewpage.action?pageId=123456` + - Short format: `https://{confluence-host}/x/AbCdEf` ### Step 2: Setup Output Directory @@ -57,48 +54,122 @@ agents/testgen/{TICKET-KEY}/ └── testgen-state.md (initialize) ``` +### Step 2b: Resolve MCP routing (user message vs capability file) + +**Precedence:** **A first** — fragment **section A** (user / task text **wins** over the file). **B second** — fragment **section B** (`agents/mcp-capability.yaml`). Full rules: **`mcp-capability-interaction.md`** (ACQUIRE per **Prerequisites**). + +**Checklist (execute in order before any Jira/Confluence MCP call in Phase 1):** + +1. Confirm **Prerequisites** ACQUIRE list for this phase is loaded (**`mcp-capability-interaction.md`**, and before guided MCP / Persist: **`mcp-tool-resolution.md`**, **`agent-action-resolution.md`** as listed there). +2. Read **`agents/mcp-capability.yaml`** at workspace root if present; if absent, treat capability config as **missing** (see **`mcp-capability-interaction.md`** Path conventions). +3. Apply fragment **§ A — user message** (task text wins over file). +4. If YAML is **missing** and the task text does not override MCP usage: ask **one** question — “Use live MCP for Jira and Confluence in this workspace?” — **STOP**, **WAIT**. **No** → both integrations **questionnaire** for this run. **Yes** → both **guided**; recommend adding `agents/mcp-capability.yaml`. +5. Derive **`Jira source`** and **`Confluence source`** using fragment **§ B** (`mode`, `mcp.jira`, `mcp.confluence`, omitted-key defaults). +6. Record in `agents/testgen/{TICKET-KEY}/testgen-state.md` under Phase 1: `Jira source`, `Confluence source`, and **MCP interaction source:** `agents/mcp-capability.yaml` **or** `user override` **or** `default question`. +7. If **`agents/user-instructions/mcp-guidance.md`** exists and at least one source is **guided**, read it before the first Jira/Confluence MCP call in Phase 1. + +#### Routing matrix (after Step 2b) + +Use `Jira source` and `Confluence source` from state. **Step labels:** **3Q** = Jira questionnaire (no Jira MCP); **3A** = Jira guided MCP; **4Q** = Confluence questionnaire; **4A** = Confluence guided MCP. **“URLs in Step 1?”** = Confluence URL(s) were parsed from the **initial** user prompt in Step 1. + +| Jira source | Confluence source | URLs in Step 1? | Execute | +|-------------|-------------------|-----------------|---------| +| guided | guided | yes | **3A** → **4A** Option A | +| guided | guided | no | **3A** → **4A** Option B | +| guided | questionnaire | — | **3A** → **4Q** | +| questionnaire | guided | yes | **3Q** → **4A** Option A | +| questionnaire | guided | no | **3Q** → **4A** Option B | +| questionnaire | questionnaire | — | **3Q** → **4Q** | + +#### `{integration-action:…}` intent reference (Phase 1) + +Use with **Token resolution** and **`mcp-tool-resolution.md`** when mapping tokens to host MCP tools. + +| Token | Intent | +|-------|--------| +| `{integration-action:jira-get-issue}` | Load one issue by key with field list, expand flags, comment limit | +| `{integration-action:jira-search-fields}` | Discover Jira field IDs / API names (custom fields) | +| `{integration-action:confluence-get-page}` | Load one Confluence page by id (markdown/metadata options) | +| `{integration-action:confluence-search-pages}` | Search Confluence pages (query + limit) | +| `{integration-action:confluence-list-child-pages}` | List child pages for a parent (content options) | + ### Step 3: Extract Jira Ticket Data -**Use**: Jira and/or Confluence MCPs respectively, snippets below will contain example pseudo-function calls for better understanding `mcp_Jira_MCP_jira_get_issue`, `mcp_Jira_MCP_confluence_get_page`, `mcp_Jira_MCP_confluence_search`, `mcp_Jira_MCP_confluence_get_page_children`. +Pick **exactly one** subsection below using the **Step 2b** matrix (`Jira source` column). **Do not** read, summarize, or execute the other subsection (**progressive disclosure**). + +#### Step 3Q: Jira questionnaire (no MCP) + +**Jira MCP:** **off** — no **`{integration-action:jira-*}`** calls; user-provided content only. -**Extract ticket key** from user input: -- Format: "PROJ-123" or URL "https://jira.company.com/browse/PROJ-123" -- Parse key from URL if needed +1. Ask **numbered** questions so you can build the Jira section of `raw-data.md`: ticket summary, description (or ask user to paste export), status, priority, labels, components, key links, and any acceptance criteria they rely on. +2. **STOP** and **WAIT** for answers. +3. Write `agents/testgen/{TICKET-KEY}/raw-data.md` (create if needed) with `## Jira (user-provided, MCP absent)` containing merged answers. For any field the user could not supply or was unsure about, add an explicit **`unknown`**, **`unverified`**, or short **why missing** note next to that item (do not imply MCP-grade certainty). Then continue per the **Step 2b** matrix to Step 4. -**Retrieve issue** with comprehensive fields: -```python -mcp_Jira_MCP_jira_get_issue( - issue_key="PROJ-123", - fields="summary,description,status,issuetype,assignee,priority,reporter,labels,components,created,updated", - expand="renderedFields", - comment_limit=10 -) +#### Step 3A: Guided Jira (MCP) + +1. **Extract ticket key** from user input: + - Format: "PROJ-123" or URL "https://{jira-host}/browse/PROJ-123" + - Parse key from URL if needed + +2. **Retrieve issue** — see **Token resolution**, **`mcp-tool-resolution.md`**, and **Intent reference** above for **`{integration-action:jira-get-issue}`**. Then call the resolved tool; logical parameters: +```text +Action: {integration-action:jira-get-issue} +Parameters: + issue_key: PROJ-123 + fields: summary,description,status,issuetype,assignee,priority,reporter,labels,components,created,updated + expand: renderedFields + comment_limit: 10 ``` -**Capture**: -- Summary, description (both raw and rendered) -- Issue type, status, priority -- Labels, components -- Assignee, reporter -- Comments (up to 10 recent) -- Created/updated dates -- Custom fields if present (epic link, story points, etc.) +**Example (mapping only):** On a common Atlassian host the resolved tool is `mcp__atlassian__jira_get_issue`; rename keys from the block above as required (e.g. `issue_key` → `issueKey`, `comment_limit` → `commentLimit`). Other tokens: see **Token resolution**. + +3. **Capture** (for Step 5 / `raw-data.md`): + - Summary, description (both raw and rendered) + - Issue type, status, priority + - Labels, components + - Assignee, reporter + - Comments (up to 10 recent) + - Created/updated dates + - Custom fields if present (epic link, story points, etc.) + +#### Step 3A.4 — Jira field schema (optional) + +Run **only** when substep **3** leaves custom-field **API names or ids** ambiguous for `raw-data.md`. + +1. **Invoke** **`{integration-action:jira-search-fields}`** **once** (see **Token resolution**). +2. **Output:** merge the returned field metadata into working notes and into **`### Custom Fields`** (and related) when executing **Step 5** (`raw-data.md`). If substep 3 was sufficient, **skip** this entire **3A.4** subsection. ### Step 4: Get Confluence Documentation -**Decision Point**: Did user provide Confluence URLs in initial prompt? +Pick **exactly one** subsection below using the **Step 2b** matrix (`Confluence source` column; for **4A**, use the **URLs in Step 1?** column for Option A vs B). **Do not** read, summarize, or execute the other subsection (**progressive disclosure**). + +#### Step 4Q: Confluence questionnaire (no MCP) + +**Confluence MCP:** **off** — no **`{integration-action:confluence-*}`** calls; user-provided content only. -#### Option A: User Provided Confluence URLs +1. Ask **numbered** questions: paste each page body (or export), or provide URLs plus authorized excerpts, parent/child relationships if relevant. +2. **STOP** and **WAIT**. +3. Append to `agents/testgen/{TICKET-KEY}/raw-data.md` under `## Confluence (user-provided, MCP absent)` with merged answers. Mark gaps, pasted excerpts of unknown completeness, or user-stated uncertainty the same way as Step 3Q (**`unknown` / `unverified` / reason**). If Jira questionnaire already created the file, append; otherwise create the file with this section. +4. Continue to **Step 5** so `raw-data.md` matches the full template (headings, metadata). Then run **Validation**. + +#### Step 4A: Guided Confluence (MCP) + +**Option A vs B:** From the **Step 2b** matrix (**URLs in Step 1?** column). + +Resolve each **`{integration-action:confluence-*}`** (see **Token resolution**; e.g. `confluence-get-page` → `mcp__atlassian__confluence_get_page` with `pageId` / `convertToMarkdown` when the host uses that idiom). + +##### Option A: User-provided Confluence URLs -**If URLs provided in initial prompt**: 1. Parse page IDs from URLs 2. For each URL, extract: - Page ID from URL parameters (pageId=123456) - Or use space + title from display URL -3. Retrieve pages directly using `mcp_Jira_MCP_confluence_get_page()` +3. Retrieve pages using **`{integration-action:confluence-get-page}`** 4. Check each page for child pages (REQUIRED) 5. Skip automatic search +**Illustrative resolved call (Atlassian-style host, not mandatory):** `mcp__atlassian__confluence_get_page` with arguments such as `{ "pageId": "123456", "convertToMarkdown": true }` (exact keys per host mapping). + **Tell user**: ``` ✅ Using provided Confluence pages: @@ -107,9 +178,9 @@ mcp_Jira_MCP_jira_get_issue( 🔍 Checking for child pages... ``` -#### Option B: No URLs Provided - Auto-Search +##### Option B: No URLs — auto-search -**Use**: `mcp_Jira_MCP_confluence_search()` +**Use**: **`{integration-action:confluence-search-pages}`** **Extract search terms** from Jira ticket: - Project key (from ticket key) @@ -123,35 +194,37 @@ type=page AND space={PROJECT_KEY} AND (text ~ "{term1}" OR text ~ "{term2}") ``` **Search Confluence**: -```python -mcp_Jira_MCP_confluence_search( - query=cql_query, - limit=10 -) +```text +Action: {integration-action:confluence-search-pages} +Parameters: + query: + limit: 10 ``` +**Illustrative resolved call (Atlassian-style host, not mandatory):** `mcp__atlassian__confluence_search_pages` (or host-equivalent) with CQL in the query argument, e.g. `type=page AND space=PROJ AND (text ~ "auth" OR text ~ "login")`. + **Rank results** by relevance: - Title matches ticket terms - Labels match ticket labels - Content matches key terms **Get top 3-5 pages**: -```python -mcp_Jira_MCP_confluence_get_page( - page_id=page_id, - convert_to_markdown=True, - include_metadata=True -) +```text +Action: {integration-action:confluence-get-page} +Parameters: + page_id: + convert_to_markdown: true + include_metadata: true ``` **IMPORTANT: Check for child pages** (nested documents often missed by search): -```python -mcp_Jira_MCP_confluence_get_page_children( - parent_id=page_id, - include_content=True, - convert_to_markdown=True, - limit=10 -) +```text +Action: {integration-action:confluence-list-child-pages} +Parameters: + parent_id: + include_content: true + convert_to_markdown: true + limit: 10 ``` For each parent page found: @@ -186,6 +259,19 @@ For each parent page found: **File**: `agents/testgen/{TICKET-KEY}/raw-data.md` +**Persist** — see **Token resolution** for **`{agent-action:write-file}`** / **`{agent-action:patch-file}`**: + +```text +Action: {agent-action:write-file} +Parameters: + path: agents/testgen/{TICKET-KEY}/raw-data.md + content: +``` + +**Illustrative host mapping (Cursor-style agent, not mandatory):** tool `Write` with `file_path` / `contents` (or host-equivalent keys after resolution). + +If the file already exists and you are only appending, you may use **`{agent-action:patch-file}`** instead (see **Token resolution**). + **Format**: ```markdown # Raw Data - [TICKET-KEY] @@ -193,6 +279,7 @@ For each parent page found: **Extracted**: [DateTime] **Phase**: 1 - Data Collection **Confluence Source**: [User-provided URLs / Auto-search / User-provided after search / Skipped] +**Confidence / Unknowns**: [If either source was questionnaire: list unverified, missing, or user-uncertain items; else `MCP-backed` or `n/a`] --- @@ -279,12 +366,22 @@ For each parent page found: - **Total Content Size**: [Approximate word count] - **Search Terms Used**: [List] - **Notes**: [Any issues during extraction] +- **Confidence / Unknowns**: [Same as header line; required when any questionnaire path ran] ``` ### Step 6: Update State File **File**: `agents/testgen/{TICKET-KEY}/testgen-state.md` +**Persist** — same as Step 5 (**see Token resolution**): + +```text +Action: {agent-action:write-file} +Parameters: + path: agents/testgen/{TICKET-KEY}/testgen-state.md + content: +``` + **Create initial state**: ```markdown # Test Generation State - [TICKET-KEY] @@ -329,18 +426,26 @@ For each parent page found: Before completing Phase 1, verify: - ✅ `agents/testgen/{TICKET-KEY}/` directory exists -- ✅ `raw-data.md` created with Jira section populated +- ✅ Phase 1 block in `testgen-state.md` contains lines **`Jira source`**, **`Confluence source`**, and **`MCP interaction source`** +- ✅ If **`Jira source` is `questionnaire`**: `raw-data.md` contains `## Jira (user-provided, MCP absent)` +- ✅ If **`Confluence source` is `questionnaire`**: `raw-data.md` contains `## Confluence (user-provided, MCP absent)` +- ✅ If any **questionnaire** path ran: `raw-data.md` **Data Collection Summary** includes **`Confidence / Unknowns`** +- ✅ `raw-data.md` created with Jira section populated (MCP or questionnaire path) - ✅ Confluence section has at least 1 page OR user confirmed skip - ✅ `testgen-state.md` created with Phase 1 marked complete - ✅ All key Jira fields captured (summary, description, status, priority) ## Tools Used -- `mcp_Jira_MCP_jira_get_issue()` - Jira ticket extraction -- `mcp_Jira_MCP_confluence_search()` - Confluence page search -- `mcp_Jira_MCP_confluence_get_page()` - Confluence page content retrieval -- `mcp_Jira_MCP_confluence_get_page_children()` - Confluence child page discovery -- `write()` - File creation +| Token / use | Where | +|---------------|--------| +| `{integration-action:jira-get-issue}` | Step 3A.2 | +| `{integration-action:jira-search-fields}` | Step 3A.4 (conditional) | +| `{integration-action:confluence-search-pages}` | Step 4A Option B | +| `{integration-action:confluence-get-page}` | Step 4A Options A / B | +| `{integration-action:confluence-list-child-pages}` | Step 4A Option B | +| `{agent-action:write-file}` | Steps 5–6 (**Persist** blocks) | +| `{agent-action:patch-file}` | Steps 5–6 (optional append / edit) | ## Common Issues @@ -354,10 +459,10 @@ Before completing Phase 1, verify: **Solution**: Include first 5000 words, note truncation in raw-data.md **Issue**: Custom fields not recognized -**Solution**: Use `mcp_Jira_MCP_jira_search_fields()` to discover field names +**Solution**: Run **Step 3A.4** — **`{integration-action:jira-search-fields}`** (or equivalent) to discover field names **Issue**: Confluence search finds parent but misses child pages -**Solution**: Always check for child pages using `confluence_get_page_children()` for each found page +**Solution**: Always check for child pages using **`{integration-action:confluence-list-child-pages}`** for each found page **Issue**: User provided invalid Confluence URL **Solution**: Try to parse page ID, if fails ask user for correct URL or page ID @@ -378,8 +483,6 @@ After Phase 1 completion: - Confluence search may need tuning based on organization's Confluence structure - Some Jira instances have custom fields - capture all available - Confluence pages may be in different spaces - search broadly initially -- **User can provide Confluence URLs in initial prompt** - this skips auto-search -- If user provides specific page URLs/IDs, use those directly instead of search - **CRITICAL**: Always check for child pages - nested documentation often contains the most relevant details - Example: "Job Post" parent may have children "Create a Job Post", "Edit a Job Post", etc. - Retrieve up to 10 child pages per parent, prioritize by relevance to ticket diff --git a/instructions/r2/core/workflows/testgen-flow-test-case-export.md b/instructions/r2/core/workflows/testgen-flow-test-case-export.md index e2cce06c..7e9164fc 100644 --- a/instructions/r2/core/workflows/testgen-flow-test-case-export.md +++ b/instructions/r2/core/workflows/testgen-flow-test-case-export.md @@ -12,26 +12,70 @@ baseSchema: docs/schemas/phase.md - Phase 5 MUST be complete - `agents/testgen/{TICKET-KEY}/test-scenarios.md` exists with test cases - User has reviewed and edited test cases -- TestRail MCP configured and accessible -- TestRail project_id and suite_id known +- **TestRail export path (B + A):** Either **guided** (TestRail MCP allowed per `mcp-capability-interaction.md` and `agents/mcp-capability.yaml`) or **questionnaire** (no TestRail MCP; user supplies IDs and performs or confirms manual import). Optional **`agents/user-instructions/mcp-guidance.md`** when guided. +- **Placeholder binding:** Before first guided **`{integration-action:*}`** call, ACQUIRE **`mcp-tool-resolution.md`**. For **`{agent-action:*}`** file steps, ACQUIRE **`agent-action-resolution.md`**. Canonical: **`instructions/r2/core/workflows/.md`** if KB alias fails. +- For **guided** runs: TestRail MCP configured and accessible; `project_id` and `suite_id` known (from guidance, state, or user). ## Objective -Export test cases from test-scenarios.md to TestRail using MCP, creating a new section (folder) for the Jira ticket and adding all test cases. -If MCP is not available, ask user how to proceed and provide possible options. +Export test cases from `test-scenarios.md` to TestRail. + +- **Guided:** Use TestRail MCP to verify the project, resolve `section_id`, and create cases. +- **Questionnaire:** Do **not** run any **`{integration-action:testrail-*}`** call. Collect `project_id`, `suite_id`, and `section_id` from the user, write a **manual export pack** (copy-paste / CSV-friendly), and record completion when the user confirms import or pastes back TestRail case IDs. + +If the interaction mode is unclear, resolve it in **Step 0** before any MCP call. ## TestRail Configuration **Default Configuration**: detect using current ticket and user profile. **Section Name**: [TICKET-KEY] (e.g., "PROJ-123") +**Web URL placeholders:** `{testrail-suite-view-url}`, `{testrail-case-view-url}`, and `{testrail-suite-cases-grouped-url}` stand in for full TestRail browser links (`https://` + host + `index.php?` paths + ids). Compose from `{TestRail instance host}`, `suite_id`, case id, and section grouping as in Step 1Q or `mcp-guidance.md`. + +### Guided TestRail and workspace file actions (placeholders) + +| Token | Intent | +|-------|--------| +| `{integration-action:testrail-get-project}` | Verify TestRail project access by `project_id` | +| `{integration-action:testrail-get-cases}` | List cases in a project/suite (optional dedup before add) | +| `{integration-action:testrail-add-case}` | Create a test case in a `section_id` with title, priority, type, refs, separated steps | +| `{agent-action:read-file}` | Read a workspace file (host-specific file tool) | +| `{agent-action:write-file}` | Create or overwrite a workspace file | +| `{agent-action:patch-file}` | Apply targeted edits to a workspace file (includes search-and-replace style edits) | + +Resolve **`{integration-action:…}`** using **`mcp-tool-resolution.md`** (guided only). Resolve **`{agent-action:…}`** using **`agent-action-resolution.md`**. ## Requirements -### Step 1: Verify TestRail Connection +### Step 0: Resolve TestRail MCP interaction (B + A) -**Test connection** using: -```python -mcp_testrail_get_project(project_id=69) +1. ACQUIRE **`mcp-capability-interaction.md`** FROM KB. ACQUIRE **`mcp-tool-resolution.md`** before any **`{integration-action:*}`** MCP call (when **guided**). ACQUIRE **`agent-action-resolution.md`** before any **`{agent-action:*}`** workspace step. If KB alias fails, use **`instructions/r2/core/workflows/.md`**. +2. Read **`agents/mcp-capability.yaml`** if present; apply **user override** from the current task text (same rules as Phase 1). +3. Derive **`TestRail export: guided | questionnaire`**: + - **`mcp.mode: absent`** → questionnaire. + - **`mcp.mode: capable`** and **`mcp.testrail: false`** (when the key is present) → questionnaire. + - **`mcp.mode: capable`** and (`testrail` omitted or `true`) → guided. + - Missing YAML: use **user override** if any; else **one** short question (“Use TestRail MCP for export?”). **No** → questionnaire; **Yes** → guided and recommend adding `agents/mcp-capability.yaml`. +4. Record in `agents/testgen/{TICKET-KEY}/testgen-state.md` (Phase 6): + - `TestRail export: guided | questionnaire` + - `MCP interaction source:` (`agents/mcp-capability.yaml` | user override | default question`) +5. If **guided** and **`agents/user-instructions/mcp-guidance.md`** exists, read it before the first **`{integration-action:testrail-*}`** call. + +### Step 1: Verify TestRail connection (guided only) + +**If `TestRail export` is `questionnaire`:** Run **Step 1Q** instead of this step. + +#### Step 1Q: Questionnaire — no MCP verification + +1. Ask **numbered** questions: `project_id`, `suite_id` (for links and documentation), and confirm the user will import cases manually or paste back IDs after import. +2. **STOP** and **WAIT**. +3. Skip to **Step 2** (section_id flow is unchanged; user may already know `section_id`). + +**Guided path — test connection** using (replace `project_id` with value from guidance/state/user; **do not** hard-code example IDs unless the user confirmed them): + +```text +Action: {integration-action:testrail-get-project} +Parameters: + project_id: ``` If fails, tell user: @@ -39,7 +83,7 @@ If fails, tell user: ❌ TestRail connection failed. Please verify: 1. TestRail MCP is configured 2. Credentials are correct -3. Project ID 69 exists and you have access +3. Project ID exists and you have access ``` ### Step 2: Create Section in TestRail @@ -57,7 +101,7 @@ If you already have a section, provide the section_id. You can find it in the URL when viewing a section (e.g., group_id=94686 or section_id=94686) **Option B: Create new section** -1. Go to: https://griddynamics.testrail.io/index.php?/suites/view/3300 +1. Go to: {testrail-suite-view-url} 2. Click "Add Section" 3. Name it: [TICKET-KEY] (e.g., "PROJ-456") 4. After creating, find the section_id in the URL or section details @@ -224,19 +268,40 @@ Execute this test case for EACH row in the table below: - Just include original preconditions normally - No "Execute for EACH row" note -### Step 5: Add Test Cases to TestRail +### Step 5: Add Test Cases to TestRail (guided only) -**For each mapped test case**: +**If `TestRail export` is `questionnaire`:** Run **Step 5Q** only; **do not** run **`{integration-action:testrail-add-case}`**. -```python -result = mcp_testrail_add_case( - section_id=section_id, - title=testrail_case["title"], - priority_id=testrail_case["priority_id"], - type_id=testrail_case["type_id"], - refs=testrail_case["refs"], - custom_steps_separated=testrail_case["custom_steps_separated"] -) +#### Step 5Q: Manual export pack (no MCP) + +1. After **Step 4** mapping, write **`agents/testgen/{TICKET-KEY}/testrail-manual-export.md`** containing: + - User-supplied `project_id`, `suite_id`, `section_id` (if known) + - One block per test case: title, priority, type, refs, preconditions text, and steps (content + expected) in plain text or markdown tables suitable for copy-paste into TestRail. +2. Optionally add **`agents/testgen/{TICKET-KEY}/testrail-import-hints.csv`** (title, priority, type, refs, preconditions, steps_json) if the team uses CSV import. +3. **STOP** and **WAIT**: ask the user to confirm either (a) they created cases manually / imported the pack, or (b) they paste back a mapping `TC-001 → C12345` (or URLs). +4. **After the user replies**, execute **Step 6** to merge IDs into `test-scenarios.md`. Until then you may leave placeholders (e.g. `Pending`) only if the workflow must save intermediate files; do **not** mark Phase 6 complete in state until Step 6 reflects the user’s answer. + +**Optional (guided only) — list existing cases before add:** If duplicate titles, re-exports, or unclear existing coverage are a concern, run **`{integration-action:testrail-get-cases}`** once against the target `project_id` / `suite_id` (and use section or other filters when your MCP supports them) to compare with parsed markdown. **Not required** for a first-time export or when the user accepts duplicates. + +```text +Action: {integration-action:testrail-get-cases} +Parameters: + project_id: + suite_id: +``` + +**Guided path — for each mapped test case**: + +```text +Action: {integration-action:testrail-add-case} +Parameters: + section_id: + title: + priority_id: + type_id: + refs: + custom_steps_separated: +Outcome: capture returned case id for results tracking ``` **Track results**: @@ -261,12 +326,14 @@ results = { **Update**: `agents/testgen/{TICKET-KEY}/test-scenarios.md` +For **questionnaire** exports, use user-pasted IDs when available; otherwise keep **`TestRail ID`**: `Manual (see testrail-manual-export.md)`** until the user confirms. Use real `project` / `suite` / base URL values from Step 1Q or guidance — **do not** invent instance hostnames. + **Add TestRail IDs to each test case**: ```markdown ### TC-001: User Login with Valid Credentials (Happy Path) **TestRail ID**: C12345 ✅ -**TestRail Link**: https://griddynamics.testrail.io/index.php?/cases/view/12345 +**TestRail Link**: {testrail-case-view-url} **Related Requirement**: US-1, FR-1 ... ``` @@ -283,11 +350,11 @@ results = { ## TestRail Export Summary **Exported**: [DateTime] -**Project**: 69 -**Suite**: 3300 +**Project**: {project_id} +**Suite**: {suite_id} **Section**: [TICKET-KEY] (ID: [section_id]) **Total Exported**: [X] test cases -**TestRail Link**: https://griddynamics.testrail.io/index.php?/suites/view/3300&group_by=cases:section_id +**TestRail Link**: {testrail-suite-cases-grouped-url} | TC ID | TestRail ID | Status | |-------|-------------|--------| @@ -326,8 +393,8 @@ results = { ### Phase 6: TestRail Export - **Completed**: [DateTime] -- **Project ID**: 69 -- **Suite ID**: 3300 +- **Project ID**: {project_id} +- **Suite ID**: {suite_id} - **Section ID**: [section_id] - **Section Name**: [TICKET-KEY] - **Test Cases Created**: [Count] @@ -339,21 +406,30 @@ results = { ## Validation Before completing Phase 6, verify: -- ✅ TestRail connection successful -- ✅ Section exists in TestRail + +**Guided path:** +- ✅ TestRail connection successful (`{integration-action:testrail-get-project}` or equivalent) +- ✅ Section exists in TestRail (or user provided valid `section_id`) - ✅ All test cases parsed from markdown -- ✅ At least 80% of test cases exported successfully -- ✅ test-scenarios.md updated with TestRail IDs +- ✅ At least 80% of test cases exported successfully via MCP (or failures documented) +- ✅ `test-scenarios.md` updated with TestRail IDs - ✅ State file updated with Phase 6 complete - ✅ TestRail link provided to user +**Questionnaire path:** +- ✅ Step 0 recorded `TestRail export: questionnaire` and source line +- ✅ User confirmed `project_id` / `suite_id` / `section_id` as needed +- ✅ `testrail-manual-export.md` (and optional CSV) written +- ✅ User confirmed manual import or pasted ID mapping +- ✅ `test-scenarios.md` updated with IDs or explicit “manual / pending” notes +- ✅ State file updated with Phase 6 complete + ## Tools Used -- `mcp_testrail_get_project(project_id)` - Verify connection -- `mcp_testrail_get_cases(project_id, suite_id)` - Check existing cases -- `mcp_testrail_add_case(section_id, title, ...)` - Create test cases -- `read_file()` - Read test-scenarios.md -- `write()` / `search_replace()` - Update files +- **Guided:** `{integration-action:testrail-get-project}` — verify connection; `{integration-action:testrail-get-cases}` — optional dedup; `{integration-action:testrail-add-case}` — create test cases +- **Questionnaire:** `{agent-action:read-file}` / `{agent-action:write-file}` / `{agent-action:patch-file}` only — build `testrail-manual-export.md` and update docs; **no** `{integration-action:testrail-*}` calls +- `{agent-action:read-file}` — read `test-scenarios.md` +- `{agent-action:write-file}` / `{agent-action:patch-file}` — update files ## Common Issues @@ -375,6 +451,9 @@ Before completing Phase 6, verify: **Issue**: Rate limit exceeded **Solution**: Add delay between API calls, batch requests +**Issue**: Risk of duplicate titles or unclear existing coverage +**Solution**: Optionally run **`{integration-action:testrail-get-cases}`** (Step 5 optional block) before **`{integration-action:testrail-add-case}`** — optional discovery of existing cases, not a mandatory gate. + **Issue**: Test case already exists **Solution**: Create anyway (TestRail allows duplicates), note in report @@ -393,7 +472,7 @@ All 6 phases finished successfully: ✅ Phase 6: TestRail Export ([R] cases exported) **TestRail Section**: [TICKET-KEY] -**TestRail Link**: https://griddynamics.testrail.io/index.php?/suites/view/3300 +**TestRail Link**: {testrail-suite-view-url} **Deliverables**: 📄 requirements.md - Use for implementation @@ -411,8 +490,8 @@ All 6 phases finished successfully: - TestRail MCP currently lacks section creation - user must create manually - Test case IDs in TestRail are prefixed with "C" (e.g., C12345) -- Suite ID 3300 is from user's TestRail URL -- Project ID 69 is user's default project +- `{suite_id}` comes from the user, `mcp-guidance.md`, or the suite view URL on their instance +- `{project_id}` comes from the user, `mcp-guidance.md`, or the TestRail project settings - Parameterized test data is included in step content or expected results - Re-running export creates duplicate test cases (by design, to preserve history) - Consider creating test run after export for immediate execution diff --git a/instructions/r2/core/workflows/testgen-flow.md b/instructions/r2/core/workflows/testgen-flow.md index 1e334ca6..de88e4c8 100644 --- a/instructions/r2/core/workflows/testgen-flow.md +++ b/instructions/r2/core/workflows/testgen-flow.md @@ -14,6 +14,7 @@ Systematic requirements analysis from Jira tickets and Confluence documentation ### Critical Requirements +- **MCP ROUTING (B + A)**: Phases 1 and 6 may call external MCPs only when **`agents/mcp-capability.yaml`** and **`mcp-capability-interaction.md`** resolve **guided** for that integration. Otherwise use **questionnaire** or manual export paths documented in those phases. Optional **`agents/user-instructions/mcp-guidance.md`** for IDs and guardrails when guided. **`{integration-action:*}`** / **`{agent-action:*}`** binding: follow each phase file’s **Prerequisites** (**`mcp-tool-resolution.md`**, **`agent-action-resolution.md`**). - **ONE PHASE AT A TIME**: Read phase file, execute, update state, move to next. - **DO NOT SKIP PHASES**: Each builds on previous. - **STATE TRACKING**: Update `agents/testgen/{TICKET-KEY}/testgen-state.md` after each phase. @@ -31,9 +32,10 @@ Systematic requirements analysis from Jira tickets and Confluence documentation **Phase 1: Data Collection** [testgen-flow-data-collection.md] 1. ACQUIRE testgen-flow-data-collection.md FROM KB -2. Execute phase instructions -3. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` -4. Validate by listing raw-data.md file +2. ACQUIRE **`mcp-capability-interaction.md`**, and per that phase’s **Prerequisites** also **`mcp-tool-resolution.md`** / **`agent-action-resolution.md`** when placeholders are used +3. Execute phase instructions +4. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` +5. Validate by listing raw-data.md file **Phase 2: Gap & Contradiction Analysis** [testgen-flow-gap-and-contradiction-analysis.md] 1. ACQUIRE testgen-flow-gap-and-contradiction-analysis.md FROM KB @@ -61,8 +63,9 @@ Systematic requirements analysis from Jira tickets and Confluence documentation **Phase 6: Test Case Export** [testgen-flow-test-case-export.md] ⭐ 1. ACQUIRE testgen-flow-test-case-export.md FROM KB -4. Execute phase instructions -5. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` +2. ACQUIRE **`mcp-capability-interaction.md`** and per that phase’s **Step 0** / **Prerequisites** also **`mcp-tool-resolution.md`** / **`agent-action-resolution.md`** when placeholders are used (guided vs questionnaire for TestRail export) +3. Execute phase instructions +4. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` ## State File Format @@ -136,9 +139,9 @@ agents/testgen/{TICKET-KEY}/ ## Prerequisites -- **Jira MCP**: Configured and accessible -- **Jira Ticket**: User provides ticket key or link -- **Confluence Access**: Via Jira MCP (same authentication) +- **MCP routing (B + A):** `agents/mcp-capability.yaml` (see `instructions/r2/core/templates/mcp-capability.example.yaml`) plus optional `agents/user-instructions/mcp-guidance.md` control **guided** vs **questionnaire** per integration. Phases 1 and 6 embed the branching rules. +- **Jira ticket**: User provides ticket key or link (Phase 1) +- **Confluence**: Either MCP-guided discovery (Phase 1) or user-provided content when **questionnaire** - **Output Directory**: Created at start (`agents/testgen/{TICKET-KEY}/`) ## Common Patterns diff --git a/plugins/core-codex/.agents/workflows/agent-action-resolution.md b/plugins/core-codex/.agents/workflows/agent-action-resolution.md new file mode 100644 index 00000000..75fb60a9 --- /dev/null +++ b/plugins/core-codex/.agents/workflows/agent-action-resolution.md @@ -0,0 +1,18 @@ +--- +name: agent-action-resolution +description: Maps `{agent-action:…}` tokens to host IDE or agent-runtime file tools (read, write, patch). +alwaysApply: false +baseSchema: docs/schemas/phase.md +--- + +# `{agent-action:…}` resolution (host file tools) + +Use when a workflow uses **`{agent-action:read-file}`**, **`{agent-action:write-file}`**, or **`{agent-action:patch-file}`**. These are **not** remote MCP calls and are **not** covered by **`mcp-tool-resolution.md`**. + +| Token | Intent | +|-------|--------| +| `{agent-action:read-file}` | Read a file from the workspace via the host agent’s file-read tool | +| `{agent-action:write-file}` | Create or overwrite a workspace file via the host agent’s file-write tool | +| `{agent-action:patch-file}` | Apply targeted edits via the host agent’s patch / search-replace tool | + +**Resolution:** Map each token to the host IDE’s or agent runtime’s **documented** file operation (names differ: e.g. `Read`, `Write`, `StrReplace`, `apply_patch`). If the host exposes no matching capability, **STOP**, tell the user, and **WAIT** — do not substitute an MCP tool. diff --git a/plugins/core-codex/.agents/workflows/aqa-flow-data-collection.md b/plugins/core-codex/.agents/workflows/aqa-flow-data-collection.md index 34fc0898..a9e7253d 100644 --- a/plugins/core-codex/.agents/workflows/aqa-flow-data-collection.md +++ b/plugins/core-codex/.agents/workflows/aqa-flow-data-collection.md @@ -9,19 +9,61 @@ baseSchema: docs/schemas/phase.md ## Objective -Gather all required information from external sources (TestRail and Confluence) to understand test requirements and expected behavior. +Gather everything needed for later phases: either from **TestRail and Confluence** (**integrated AQA**) or from **user-supplied artifacts** (**minimal-input / agent-led**). Always record the chosen **execution mode** in the test plan and state file. ## Prerequisites -- TestRail MCP configured and accessible -- Atlassian (Confluence) MCP configured and accessible -- Test case ID or requirement provided by user +- User intent is clear enough to start Phase 1 (IDs, URLs, or a plain-language automation goal). +- **Integrated AQA path**: For **guided** MCP interaction (see Task 0b), TestRail MCP and Atlassian (Confluence) MCP must be usable when you run Path A MCP steps. For **questionnaire** interaction, do **not** call those MCPs; collect the same facts via user answers and paste into the test plan. +- **Minimal-input path**: No MCP requirement for TestRail/Confluence; Task 0b **does not apply** to Path B for those integrations. If something on the checklist is missing, **ask** — do not guess. + +## MCP capability reference + +Resolve **guided vs questionnaire** per **`mcp-capability-interaction.md`** (ACQUIRE FROM KB). Config file: **`agents/mcp-capability.yaml`** (copy from `instructions/r2/core/templates/mcp-capability.example.yaml` in Rosetta repo). Optional guidance: **`agents/user-instructions/mcp-guidance.md`**. ## Phase Tasks -### Task 1: Read TestRail Test Case +### Task 0: Confirm execution mode + +**Actions**: +1. Check whether the **current task text** (or a prior user message in the same session the user instructs you to treat as binding) already names the mode **explicitly**, using any of these (case-insensitive): `integrated`, `minimal-input`, `minimal input`, `Integrated AQA`, or `minimal-input / agent-led`. The token must be **user-authored** in that message (not something you paraphrase or infer). If yes, set the mode to `integrated` or `minimal-input` accordingly and **skip steps 2–3**; go directly to **Path A** or **Path B** matching that mode. +2. If **not** explicit: **Do not infer** mode from clues such as missing TestRail IDs, presence of HTML attachments, “sandbox,” “no MCP,” or vague phrases like “I don’t have TestRail yet.” **Forbidden:** choosing `minimal-input` because TestRail was omitted, the repo looks small, or the user only supplied local HTML—those are **not** mode labels. Informal phrases (“lightweight,” “without TestRail,” “bring my own selectors”) **do not** count as explicit; they still require steps 2–3. **Ask once** with both options, for example: + - **Integrated AQA**: "We pull the official case and docs from TestRail/Confluence and follow the full chain." + - **Minimal-input / agent-led**: "You provide URLs, page source or a selector map, scenario and expectations; we still run phases 2–8 in order but Phase 1 records your package instead of MCP pulls." +3. **STOP** and **WAIT** until the user replies with a clear **integrated** or **minimal-input** (or the spelled labels in step 1). If the reply is still ambiguous, ask **one** narrowing question that still requires those labels; **do not** choose a path without those words from the user. +4. **Hard gate — before Path A or Path B:** Do **not** create or update `agents/plans/aqa-*.md`, do **not** write `**AQA execution mode**` in any artifact, and do **not** mark Phase 1 complete in `agents/aqa-state.md` until step **1** matched or step **3** completed. If you already started Path A/B without satisfying Task 0, **stop**, discard partial mode lines, run steps 2–3, then continue. +5. Ensure the Phase 1 test plan created in Path A or Path B includes `**AQA execution mode**` and that `agents/aqa-state.md` reflects the mode when Phase 1 is marked complete (see **Update State File** below). + +**Expected Output**: Mode is either read from **explicit user keywords** (step 1) or obtained only after **mandatory** ask-and-wait (steps 2–3); then proceed with **Path A** or **Path B** (not both). + +--- + +### Task 0b: Resolve MCP interaction (Path A only) + +**Skip this entire task for Path B (minimal-input).** **Actions**: +1. ACQUIRE **mcp-capability-interaction.md** FROM KB (if not already loaded this phase). +2. Read **`agents/mcp-capability.yaml`** if it exists; apply **user override** rules from that fragment (task text beats file). If file missing and user has not overridden, ask **one** yes/no: “Use live MCP for TestRail/Confluence in this workspace?” — **WAIT**; treat **No** as questionnaire, **Yes** as guided for both integrations until a YAML file exists. +3. Derive **per integration** (TestRail vs Confluence) whether the source is **guided** (MCP allowed) or **questionnaire** (no MCP): if `mcp.mode` is `absent`, both are **questionnaire**; if `capable`, use `mcp.testrail` / `mcp.atlassian_confluence` when present (boolean); if a key is **omitted** under `capable`, default that integration to **guided**. +4. Record in the test plan and `agents/aqa-state.md` Phase 1 notes: `TestRail source: guided|questionnaire`, `Confluence source: guided|questionnaire`, plus `MCP interaction source:` (`agents/mcp-capability.yaml` | user override | default question`). +5. **Questionnaire leg:** For each integration still **questionnaire** and not yet documented in the plan, run **one** numbered questionnaire (combine TestRail + Confluence in a single message if both need data). **STOP** and **WAIT**. Merge answers under **User-provided (MCP absent) — TestRail** and/or **User-provided (MCP absent) — Confluence** in the working plan (or append to the plan draft before Task A3). +6. **Guided leg:** If **`agents/user-instructions/mcp-guidance.md`** exists and at least one integration is **guided**, read it before the first MCP call for a **guided** integration. Then continue to Task A1 / A2 and follow the **guided** branches there only. + +**Expected Output**: TestRail and Confluence are each classified **guided** or **questionnaire**; questionnaire content is captured before MCP calls; no MCP call runs for a **questionnaire** integration. + +--- + +### Path A — Integrated AQA (TestRail + Confluence) + +**Prerequisite:** Task 0 already set execution mode to **integrated** (step 1 keyword match or step 3 user reply). Do not enter Path A without that. + +### Task A1: Read TestRail Test Case + +**Actions**: +- **If Task 0b resolved questionnaire for TestRail:** Ensure the test plan already contains **User-provided (MCP absent) — TestRail** content from Task 0b. If missing, **STOP**, ask numbered questions (case ID for traceability, title, steps, expected results, preconditions), **WAIT**, merge into the plan. **Do not** call `user-testrail-get_case`. Skip the guided-only steps below for TestRail. +- **If Task 0b resolved guided for TestRail:** Perform steps 1–4. + 1. Ask user for TestRail test case ID if not provided 2. Use TestRail MCP to retrieve test case details: ``` @@ -39,9 +81,12 @@ Gather all required information from external sources (TestRail and Confluence) **Expected Output**: Complete understanding of what needs to be tested according to TestRail. -### Task 2: Read Confluence Documentation +### Task A2: Read Confluence Documentation **Actions**: +- **If Task 0b resolved questionnaire for Confluence:** Ensure the test plan contains **User-provided (MCP absent) — Confluence** content from Task 0b. If missing, **STOP**, ask numbered questions (URLs, pasted excerpts, or summaries the user authorizes), **WAIT**, merge into the plan. **Do not** call `user-mcp-atlassian-confluence_*`. Skip guided-only steps below for Confluence. +- **If Task 0b resolved guided for Confluence:** Perform steps 1–5. + 1. Ask user for Confluence page ID/URL or search terms if not provided 2. Use Atlassian Confluence MCP to find related documentation: ``` @@ -60,10 +105,11 @@ Gather all required information from external sources (TestRail and Confluence) **Expected Output**: Business and technical context for the feature being tested. -### Task 3: Create Initial Test Plan Document +### Task A3: Create initial test plan document (integrated) **Actions**: 1. Create `agents/plans/aqa-.md` file with: + - **AQA execution mode**: `integrated` - Test case reference (TestRail ID and link) - Feature name and description - Test goal @@ -77,6 +123,7 @@ Gather all required information from external sources (TestRail and Confluence) # AQA Test Plan - **Created**: [DateTime] +**AQA execution mode**: integrated **TestRail Case**: [ID/URL] **Feature**: [Feature Name] **Status**: Phase 1 Complete @@ -134,14 +181,112 @@ Gather all required information from external sources (TestRail and Confluence) [To be filled in Phase 6] ``` +--- + +### Path B — Minimal-input / agent-led + +**Prerequisite:** Task 0 already set execution mode to **minimal-input** (step 1 keyword match or step 3 user reply). Do not enter Path B because TestRail was missing or files were attached—only because the user chose **minimal-input** in text. + +### Task B1: Offer agent-led implementation and required artifacts + +**Actions**: +1. Tell the user clearly that the agent **can** implement the test **without** TestRail/Confluence **if** they provide enough grounded detail — but the agent **must not** invent pages, selectors, or flows. +2. List what you still need at minimum (see Task B2 checklist). **WAIT** until the user either supplies items or explicitly defers an item to Phase 2 (then note "open in Phase 2" in the plan). + +**Expected Output**: User understands the tradeoff and what to supply (or what is deferred to Phase 2 with explicit consent). + +### Task B2: Collect minimal-input package + +**Actions**: +1. Capture the following in the test plan (fill or mark *deferred to Phase 2* with user agreement): + - **Scenario / goal**: What behavior the automated test must prove (plain language). + - **Entry / URLs**: Starting URL(s) or navigation path to reach the UI under test. + - **UI grounding** (at least one required before ending Phase 1 unless user explicitly defers to Phase 2): e.g. saved page source or DOM snapshot paths, a **selector map** (purpose → locator), pointers to existing Page Objects in the repo, and/or frontend file paths the agent will use in Phase 3. + - **Preconditions and test data**: Accounts, feature flags, seed data — or *unknown — Phase 2*. + - **Optional**: TestRail/Confluence IDs "for traceability only" without MCP — record as references, not as substitute for missing UI grounding. +2. **Do not** mark Phase 1 complete if UI grounding and scenario goal are both missing and the user has not agreed to defer. + +**Expected Output**: Minimal-input checklist documented in the test plan. + +### Task B3: Create initial test plan document (minimal-input) + +**Actions**: +1. Create `agents/plans/aqa-.md` using the template below (adapt section headings if the scenario is small). +2. For TestRail/Confluence fields, use `N/A` or optional reference links — do not pretend MCP data exists. + +**Template**: +```markdown +# AQA Test Plan - + +**Created**: [DateTime] +**AQA execution mode**: minimal-input +**TestRail Case**: [N/A or ID for traceability only] +**Confluence**: [N/A or links for traceability only] +**Feature**: [Feature Name] +**Status**: Phase 1 Complete + +## Minimal-input package + +### Scenario / goal +[What the test must validate] + +### Entry and navigation +- URLs: [...] +- Steps to reach the UI: [...] + +### UI grounding +- Page source / DOM snapshots: [paths or attach instructions] +- Selector map or locators: [...] +- Repo pointers (Page Objects / components): [...] + +### Preconditions and test data +[Or: deferred to Phase 2 — user confirmed] + +### Notes and open questions +- [...] + +## Test Case Information (synthetic) + +### Test Steps (draft) +1. [Step — can be refined in Phase 2] +... + +### Expected Overall Result +[Draft — refined in Phase 2] + +--- +## Phase 2: Requirements Clarification +[To be filled in Phase 2] + +## Phase 3: Code Analysis +[To be filled in Phase 3] + +## Phase 4: Selector Identification +[To be filled in Phase 4] + +## Phase 5: Selector Implementation +[To be filled in Phase 5] + +## Phase 6: Test Implementation +[To be filled in Phase 6] +``` + ## Completion Criteria +**All modes** +- [ ] Task 0 complete: execution mode is **either** named by explicit user keywords (`integrated` / `minimal-input` / spelled equivalents) **or** obtained after the mandatory ask-and-wait; mode recorded in test plan and state +- [ ] Test plan file `agents/plans/aqa-.md` created +- [ ] Test goal clearly understood (or explicitly deferred to Phase 2 with user consent) +- [ ] `agents/aqa-state.md` updated with Phase 1 completion + +**Integrated AQA only** - [ ] TestRail test case retrieved and documented - [ ] Confluence documentation retrieved and documented -- [ ] Test plan file created with all Phase 1 information -- [ ] Test goal clearly understood -- [ ] Expected results documented -- [ ] `agents/aqa-state.md` updated with Phase 1 completion +- [ ] Expected results from sources documented + +**Minimal-input only** +- [ ] Minimal-input checklist captured in the test plan +- [ ] UI grounding present or explicitly deferred to Phase 2 with user approval ## Update State File @@ -150,8 +295,10 @@ After completing Phase 1, update `agents/aqa-state.md`: ```markdown ### Phase 1: Data Collection - Completed: [DateTime] -- TestRail Case: [ID/URL] -- Confluence Pages: [URLs] +- Execution mode: [integrated | minimal-input] +- TestRail Case: [ID/URL or N/A] +- Confluence Pages: [URLs or N/A] +- Minimal-input checklist: [brief summary or N/A] - Test Goal: [Brief description] - Expected Result: [Brief description] - Test Plan File: agents/plans/aqa-.md @@ -168,7 +315,9 @@ ACQUIRE aqa-phase2-md FROM KB ## Important Notes -- **No Assumptions**: If TestRail or Confluence data is incomplete, note it in the test plan -- **Ask Questions**: If user hasn't provided IDs/URLs, ask for them -- **Document Everything**: Capture all details even if they seem minor -- **Cross-Reference**: Ensure TestRail and Confluence information aligns +- **No Assumptions**: If data is incomplete for the chosen mode, note it in the test plan and ask — never guess selectors or flows. +- **Ask Questions**: If user hasn't provided IDs/URLs (integrated) or checklist items (minimal-input), ask for them. +- **Document Everything**: Capture all details even if they seem minor. +- **Integrated path only**: Cross-reference TestRail and Confluence information for alignment. +- **Minimal-input path**: Prefer an explicit **selector map** or page source over vague descriptions; deferrals must be user-visible in the plan. +- **Task 0**: Never infer execution mode from context; only explicit keywords or a user reply after the mandatory question count. diff --git a/plugins/core-codex/.agents/workflows/aqa-flow-requirements-clarification.md b/plugins/core-codex/.agents/workflows/aqa-flow-requirements-clarification.md index f0d21565..f7fb9a9b 100644 --- a/plugins/core-codex/.agents/workflows/aqa-flow-requirements-clarification.md +++ b/plugins/core-codex/.agents/workflows/aqa-flow-requirements-clarification.md @@ -13,16 +13,23 @@ Fill gaps in understanding, clarify unknowns, and define explicit assertions bef ## Prerequisites -- Phase 1 completed -- Test plan file created with TestRail and Confluence data -- Initial understanding of test requirements +- Phase 1 completed with a test plan file +- Phase 1 documents either **integrated** sources (TestRail + Confluence as used) **or** **minimal-input** execution mode with the checklist from Phase 1 (scenario, URLs, UI grounding or explicit deferral) +- Initial understanding of test requirements (may be draft in minimal-input mode) ## Phase Tasks +### Minimal-input mode (read first) + +If the test plan shows **`AQA execution mode: minimal-input`**: +- Treat Phase 1's checklist as authoritative for what the user already supplied; **do not** re-demand TestRail/Confluence data unless the user wants it for traceability. +- Still run gap analysis: anything deferred from Phase 1 **must** be resolved or explicitly re-deferred here with user text. +- Assertions must still be explicit and measurable before Phase 3; **never** invent selectors the user did not supply, defer, or point to in repo/page source. + ### Task 1: Review Gathered Information for Gaps **Actions**: -1. Read the test plan file from Phase 1 +1. Read the test plan file from Phase 1 and note **AQA execution mode** (`integrated` vs `minimal-input`). 2. Analyze information for completeness: - Are test steps clear and unambiguous? - Are expected results specific and measurable? @@ -197,3 +204,4 @@ ACQUIRE aqa-phase3-md FROM KB - **Explicit Over Implicit**: Every assertion must be measurable and verifiable - **User Authority**: User has final say on requirements and expected behavior - **Document Everything**: Record all questions and answers for traceability +- **Minimal-input mode**: Questions should close gaps in the Phase 1 checklist (especially UI grounding and test data), not re-litigate integrated AQA unless the user switches mode explicitly diff --git a/plugins/core-codex/.agents/workflows/aqa-flow-test-implementation.md b/plugins/core-codex/.agents/workflows/aqa-flow-test-implementation.md index a16d294e..e92ab613 100644 --- a/plugins/core-codex/.agents/workflows/aqa-flow-test-implementation.md +++ b/plugins/core-codex/.agents/workflows/aqa-flow-test-implementation.md @@ -13,13 +13,16 @@ Create automated test following project standards, integrating all Page Objects ## Prerequisites -- All previous phases (1-5) completed -- Test plan fully documented with all details -- Page Objects updated with all required selectors +- All previous phases (1-5) completed **for the chosen execution mode** (see below) +- Test plan fully documented with all details **including** `AQA execution mode` and, if `minimal-input`, the **minimal-input checklist** from Phase 1 (and any Phase 2 resolutions of deferrals) +- Page Objects updated with all required selectors **or** a clear plan from Phases 4–5 that matches user-supplied grounding (minimal-input) - Explicit assertions defined - Code architecture understood -## Phase Tasks +### Execution modes and prerequisites + +- **Integrated AQA**: The bullets above mean phases 1–5 were completed in full with TestRail/Confluence-backed Phase 1 where applicable. +- **Minimal-input / agent-led** (recorded in Phase 1): Phases 1–5 count as satisfied only if the test plan contains the **minimal-input checklist** (scenario, URLs/navigation, UI grounding or approved Phase 2 deferrals) and **explicit assertions from Phase 2**. **Before writing code**, re-read that checklist and Phase 2 assertions; if anything material is still missing, **STOP**, ask the user, and do not invent selectors, pages, or flows. ### Task 1: Review Test Implementation Plan diff --git a/plugins/core-codex/.agents/workflows/aqa-flow.md b/plugins/core-codex/.agents/workflows/aqa-flow.md index 00f7c725..f7931333 100644 --- a/plugins/core-codex/.agents/workflows/aqa-flow.md +++ b/plugins/core-codex/.agents/workflows/aqa-flow.md @@ -10,20 +10,23 @@ baseSchema: docs/schemas/workflow.md ## Context -This agent handles end-to-end test automation from requirements gathering to test implementation. It uses TestRail, Confluence, and project documentation to create automated tests following existing architecture and coding standards. +This agent handles end-to-end test automation from requirements gathering to test implementation. It uses TestRail, Confluence, and project documentation when you run **integrated AQA**; in **minimal-input / agent-led** mode the user supplies pages, DOM or selector details, and scenario context instead (see Phase 1). In all modes, tests follow existing architecture and coding standards. ### Critical Requirements - **ONE PHASE AT A TIME**: Read phase file, execute, update state, move to next. - **DO NOT SKIP PHASES**: Each builds on previous. - **NO ASSUMPTIONS**: Never assume selectors, flows, or data. Always ask the user if information is missing. -- **USER INTERACTION**: Wait for user responses when questions are asked or files are requested. Phase 2, 6, 7, and 8 always require user input. Phase 4 requires user input ONLY if frontend code is unavailable or selectors cannot be found +- **MCP ROUTING (B + A)**: Before calling external MCPs in integrated Path A, follow **`mcp-capability-interaction`** using `agents/mcp-capability.yaml` and any user override in the task text. **Guided** = use MCP after optional `agents/user-instructions/mcp-guidance.md`. **Questionnaire** = no MCP calls for that integration; user answers structured questions; record answers in the plan. ACQUIRE `mcp-capability-interaction.md` FROM KB when executing Phase 1 Path A. +- **EXECUTION MODE (Phase 1, first)**: The user must **explicitly** choose **integrated** or **minimal-input** (case-insensitive), or use the spelled labels **Integrated AQA** or **minimal-input / agent-led**, in **their** message before you record a mode. If none of those appear, **ask once** with both options, **STOP**, and **WAIT**—**do not infer** mode from missing TestRail, attachments, sandbox, or “no MCP.” **Do not** write `agents/plans/aqa-*.md` or set `**AQA execution mode**` until that keyword match or the user’s reply to the question. **Forbidden:** assigning `minimal-input` because TestRail was omitted or the project looks like a demo. After the user answers, document the mode in the Phase 1 test plan and `agents/aqa-state.md`. Do not silently invent missing integrations or missing UI grounding. User customizations elsewhere do **not** waive this rule when mode was not explicitly named. +- **USER INTERACTION**: Wait for user responses when questions are asked or files are requested. **Phase 1** requires user input until execution mode is explicit or the user answers the one-time mode question (see **EXECUTION MODE**). Phase 2, 6, 7, and 8 always require user input. Phase 4 requires user input ONLY if frontend code is unavailable or selectors cannot be found - **STATE TRACKING**: Update `agents/aqa-state.md` after each phase. - **MUST** use todo tasks for tracking progress. - Prioritize ACCURACY over SPEED! ### User Customizations +- The **execution mode** rule cannot be waived by “only run some phases” unless the user message already contains the mode keywords or the user answers the one-time mode question. - If user did not specify any preferences perform all steps except optional. - User CAN customize and ask only for specific phases OR phases could have been done already OR towards specific goal OR for specific case, in this case LISTEN and ADOPT to the user. @@ -31,9 +34,10 @@ This agent handles end-to-end test automation from requirements gathering to tes **Phase 1: Data Collection** [aqa-flow-data-collection.md] 1. ACQUIRE aqa-flow-data-collection.md FROM KB -2. Execute phase instructions -3. Update `agents/aqa-state.md` -4. Validate gathered data +2. ACQUIRE mcp-capability-interaction.md FROM KB (resolve MCP guided vs questionnaire before Path A MCP calls) +3. Execute phase instructions (**start with execution mode confirmation** — integrated vs minimal-input) +4. Update `agents/aqa-state.md` +5. Validate gathered data (checklist for chosen mode is complete before Phase 2) **Phase 2: Requirements Clarification** [aqa-flow-requirements-clarification.md] ⭐ **USER INTERACTION REQUIRED** 1. ACQUIRE aqa-flow-requirements-clarification.md FROM KB @@ -91,7 +95,8 @@ Create/update `agents/aqa-state.md` after each phase: **Last Updated**: [DateTime] **Current Phase**: [1-8 or COMPLETE] -**TestRail Case**: [Test Case ID/URL] +**AQA execution mode**: [integrated | minimal-input] +**TestRail Case**: [Test Case ID/URL or N/A in minimal-input] **Feature**: [Feature Name] ## Phase Completion Status @@ -109,8 +114,10 @@ Create/update `agents/aqa-state.md` after each phase: ### Phase 1: Data Collection - Completed: [DateTime] -- TestRail Case: [ID/URL] -- Confluence Pages: [URLs] +- Execution mode: [integrated | minimal-input] +- TestRail Case: [ID/URL or N/A] +- Confluence Pages: [URLs or N/A] +- Minimal-input checklist: [summary or N/A — see Phase 1 test plan] - Test Goal: [Brief description] - Expected Result: [Brief description] @@ -158,6 +165,7 @@ Create/update `agents/aqa-state.md` after each phase: ## Important Notes - **Sequential Execution**: Phases build on each other, must execute in order. +- **Execution modes**: **Integrated AQA** uses TestRail and Confluence in Phase 1 when those tools are in scope. **Minimal-input / agent-led** uses the Phase 1 checklist (pages, DOM or selectors, scenario context); later phases still run in order, but Phase 1 documents what substitutes for external case management. Without an **explicit** user mode label (`integrated` or `minimal-input` / spelled equivalents), Phase 1 must obtain that label by asking—never infer from context alone. - **No Assumptions Rule**: Always ask user when information is missing - never guess selectors, flows, or test data. - **Architecture First**: Always analyze existing code structure before implementing new tests. - **Reuse Over Creation**: Prefer adding to existing files and using existing Page Objects over creating new ones. diff --git a/plugins/core-codex/.agents/workflows/mcp-capability-interaction.md b/plugins/core-codex/.agents/workflows/mcp-capability-interaction.md new file mode 100644 index 00000000..4f4fae4d --- /dev/null +++ b/plugins/core-codex/.agents/workflows/mcp-capability-interaction.md @@ -0,0 +1,64 @@ +--- +name: mcp-capability-interaction +description: Routes guided vs questionnaire MCP usage from workspace YAML and user text, records interaction source, and defines where capability files live. Placeholder binding uses sibling workflows. +alwaysApply: false +baseSchema: docs/schemas/phase.md +--- + +# MCP capability interaction (shared) + +Use when a workflow **may** call **remote MCPs** (Jira, Confluence, TestRail, Atlassian). It does **not** bind **`{integration-action:…}`** (see **`mcp-tool-resolution.md`**) or **`{agent-action:…}`** (see **`agent-action-resolution.md`**). **AQA** Path A and **TestGen** Phases 1 and 6 load this for **routing** (guided vs questionnaire) only. + +## B — `agents/mcp-capability.yaml` (workspace root) + +Read before the first MCP call. Keys optional; unknown keys ignored. + +```yaml +mcp: + mode: capable # required if file exists: capable | absent + jira: true + confluence: true + testrail: true + atlassian_confluence: true +``` + +- **`confluence` vs `atlassian_confluence`:** **Not aliases** — both refer to **Confluence** MCP on/off for **guided vs questionnaire**, but **different workflows read different keys** (TestGen uses `confluence`; AQA data collection uses `atlassian_confluence`). If you use **both** families, set **both** to the same boolean. If you use **one** family, you may omit the key the other workflow reads. Template: `instructions/r2/core/templates/mcp-capability.example.yaml`. + +- **`mode: capable`** — MCP allowed. Per integration: **`true`** or key omitted → **guided**. **`false`** → **questionnaire** (no MCP; user text). Read **`agents/user-instructions/mcp-guidance.md`** first when it exists. Do not re-ask values already in that file unless missing. +- **`mode: absent`** — Do not invoke MCP. Parent workflow runs questionnaire (**STOP**, **WAIT**), merges into plan / `raw-data.md` / agreed artifact. Never fabricate MCP output. +- **File missing** — Apply **A** if the user message overrides; else **one** question whose **scope** (which integrations) is defined by the **parent workflow**, not this fragment. **Example — TestGen Phase 1:** when `agents/mcp-capability.yaml` is absent, **Step 2b** in `testgen-flow-data-collection.md` asks once for **Jira and Confluence together**; **TestRail** is resolved again in **TestGen Phase 6**. **No** → questionnaire for the integrations that question covers. **Yes** → capable for that run; recommend adding YAML. + +## A — user message (wins over file) + +Record in state/plan. **Absent:** `MCP absent`, `no MCP`, `without MCP`, `questionnaire only`, `paste-only`. **Capable:** `MCP capable`, `use MCP`, `MCP enabled`, `pull from Jira` when clearly a live pull. User wins on conflict. + +## Capable mode without guidance + +Use task text and YAML only; questionnaire **only** for missing fields. Do not invent secrets. + +## Questionnaire (`absent` or integration `false`) + +Parent supplies **numbered** questions for fields MCP would return. **STOP**; **WAIT**. Merge into the same artifacts as the MCP path. Label **user-provided (no MCP)**. + +## Recording + +When routing is resolved, write into the phase plan or state file: either **one** line `MCP interaction: guided | questionnaire (source: agents/mcp-capability.yaml | user override | default question)` when a single scope applies, or **one line per integration** (or per parent-defined field such as TestGen `Jira source` / `Confluence source` / Phase 6 TestRail export), each with the same `(source: ...)` options. + +**Before leaving this fragment:** Re-read the plan or state lines you wrote for MCP routing and confirm: **(a)** each line lists exactly one source among `agents/mcp-capability.yaml`, `user override`, and `default question`; **(b)** using one aggregate line vs one line per integration (or per parent-defined field such as TestGen `Jira source` / `Confluence source`) matches what the **parent workflow** requires. Correct the artifact if a check fails. + +## Path conventions (Rosetta vs target workspace) + +Use whenever a workflow mentions both Rosetta artifacts and workspace files. + +| Location | Path | Notes | +|----------|------|--------| +| **Rosetta repo / distribution only** | `instructions/r2/core/templates/mcp-capability.example.yaml` | Template for **authoring** `mcp-capability.yaml`. Not expected inside an arbitrary customer application checkout unless copied from Rosetta. | +| **Target workspace** | `agents/mcp-capability.yaml` | Optional capability file; **expected at workspace root** only. | +| **Target workspace** | `agents/user-instructions/mcp-guidance.md` | Optional per-workspace MCP hints. | + +If **`agents/mcp-capability.yaml`** is missing at workspace root, treat capability config as **missing** for routing: follow the parent workflow’s questionnaire / default-question path and record **MCP interaction source** accordingly (do not search the customer repo for Rosetta template paths). + +## Related workflows (placeholder binding) + +- **`mcp-tool-resolution.md`** — Map **`{integration-action:…}`** to MCP tool ids when **guided**. +- **`agent-action-resolution.md`** — Map **`{agent-action:…}`** to host read / write / patch tools. diff --git a/plugins/core-codex/.agents/workflows/mcp-tool-resolution.md b/plugins/core-codex/.agents/workflows/mcp-tool-resolution.md new file mode 100644 index 00000000..8695ba68 --- /dev/null +++ b/plugins/core-codex/.agents/workflows/mcp-tool-resolution.md @@ -0,0 +1,34 @@ +--- +name: mcp-tool-resolution +description: Binds `{integration-action:…}` tokens to MCP tool identifiers in guided runs using a single-pass scan of the session tool list. +alwaysApply: false +baseSchema: docs/schemas/phase.md +--- + +# `{integration-action:…}` resolution (MCP tools) + +Use when a parent workflow is **guided** for an integration and must turn a **`{integration-action:…}`** token into a concrete MCP tool call. **ACQUIRE only when needed** (not for questionnaire-only paths). Routing (guided vs questionnaire) is defined in **`mcp-capability-interaction.md`**. + +Parent workflows define token spellings and parameter blocks. **Questionnaire** integrations must **not** use this file to call MCP. + +## Deterministic tool selection (single pass over MCP list) + +Let `T` = the list of MCP tool names (or ids) available in **this** session. Perform **exactly one** linear scan over `T` (no second pass, no “try again” loops). + +1. **Required substrings** — From the token after `integration-action:`, take segments split by `-` (e.g. `jira-get-issue` → `jira`, `get`, `issue`). Build candidate set **C** = tools in `T` whose identifier **contains every remaining segment** (case-insensitive). If **C** is empty, drop segments in this order until **C** is non-empty or one segment remains: drop **`get`**, then **`search`**, then **`list`** or **`pages`**, then repeatedly drop the **rightmost** remaining segment. +2. **If C has exactly one tool** — use it. +3. **If C has more than one tool** — Prefer tools explicitly allowed or named in **`agents/mcp-capability.yaml`** and **`agents/user-instructions/mcp-guidance.md`** for that integration. If still more than one, **STOP** and ask **one** disambiguation question using the **format below** (no guess). +4. **If C is empty after step 1 reductions** — **Do not invent** a tool. Record resolution failure in state, then follow **Questionnaire** rules for that integration in **`mcp-capability-interaction.md`** (**STOP**, **WAIT**), then continue per the parent workflow’s routing table. + +Map **Parameters** from the phase `Action` blocks to the chosen tool’s schema (adapt snake_case vs camelCase per tool docs). + +## Disambiguation question format + +When step **3** leaves multiple candidates, use exactly this shape: + +1. One line stating the token and that multiple MCP tools matched. +2. Numbered lines **`1`** through **`N`**: each line is **one verbatim tool id** from the remaining candidate set (same spelling as in `T`). +3. Line **`N+1`**: `Cancel — use questionnaire for this integration instead`. +4. **Parse the reply:** accept only an integer **k**. If **k = N+1**, do not call MCP for that integration; follow the parent’s questionnaire path. If **1 ≤ k ≤ N**, select the **k**-th listed tool in order. Any other reply → ask again once with the same list. + +Vendor tool names in examples elsewhere are **illustrations**; **`T`** is authoritative. diff --git a/plugins/core-codex/.agents/workflows/testgen-flow-data-collection.md b/plugins/core-codex/.agents/workflows/testgen-flow-data-collection.md index 4696f7d5..3e14221a 100644 --- a/plugins/core-codex/.agents/workflows/testgen-flow-data-collection.md +++ b/plugins/core-codex/.agents/workflows/testgen-flow-data-collection.md @@ -9,9 +9,12 @@ baseSchema: docs/schemas/phase.md ## Prerequisites -- MUST be starting new test generation flow -- User provided Jira ticket key or URL -- Jira MCP configured and accessible +- MUST be starting new test generation flow. +- User provided Jira ticket key or URL. +- **Acquire workflows:** ACQUIRE **`mcp-capability-interaction.md`** FROM KB (MCP guided vs questionnaire; always for Phase 1). Before the first guided **`{integration-action:*}`** call in Steps 3–4, ACQUIRE **`mcp-tool-resolution.md`**. Before **Persist** blocks using **`{agent-action:*}`** in Steps 5–6, ACQUIRE **`agent-action-resolution.md`**. If an alias fails, use canonical paths under **`instructions/r2/core/workflows/`** with the same basename. +- **Step 2b only:** Run the numbered **Step 2b checklist** in this file (not the whole fragment for later steps unless needed). +- **Before MCP calls:** Do **not** call Jira/Confluence MCPs until Step **2b** completes. +- **URL placeholders:** `{jira-host}` and `{confluence-host}` in examples mean the customer’s real web hostnames (including subdomain), not fixed vendor domains. ## Objective @@ -19,35 +22,29 @@ Extract all relevant data from Jira ticket and related Confluence documentation ## Requirements +**Token resolution (canonical):** **`{integration-action:*}`** → **`mcp-tool-resolution.md`**. **`{agent-action:*}`** → **`agent-action-resolution.md`**. MCP guided vs questionnaire and recording → **`mcp-capability-interaction.md`** (ACQUIRE per **Prerequisites**). Later steps say **see Token resolution** instead of repeating this. + ### Step 1: Parse Initial User Input **Extract from user's initial prompt**: -1. **Jira ticket**: Key or URL (REQUIRED) -2. **Confluence URLs**: List of URLs (REQUIRED) +1. **Jira ticket**: Key or URL (REQUIRED). +2. **Confluence URLs** (optional): Parse and store for later steps. How they affect routing is defined **only** in the **Step 2b** matrix (after `Jira source` / `Confluence source` exist). **Supported formats**: ``` "Analyze requirements for PROJ-123" -"Analyze requirements for PROJ-123 with Confluence: https://confluence.com/display/PROJ/Page" +"Analyze requirements for PROJ-123 with Confluence: https://{confluence-host}/display/PROJ/Page" "Analyze PROJ-123, Confluence pages: URL1, URL2, URL3" -"PROJ-123 + https://confluence.com/display/PROJ/Auth" +"PROJ-123 + https://{confluence-host}/display/PROJ/Auth" ``` **Parse Confluence URLs**: - Extract from patterns: "with Confluence", "Confluence:", "Confluence docs:", "Confluence pages:" - Accept comma-separated or line-separated URLs - URLs may be: - - Display format: `https://confluence.company.com/display/SPACE/Page+Title` - - Direct format: `https://confluence.company.com/pages/viewpage.action?pageId=123456` - - Short format: `https://confluence.company.com/x/AbCdEf` - -**If Confluence URLs provided**: -- Extract page IDs from URLs -- Skip automatic search (Step 3) -- Go directly to retrieving specified pages - -**If no Confluence URLs provided**: -- Proceed with automatic search (Step 3) + - Display format: `https://{confluence-host}/display/SPACE/Page+Title` + - Direct format: `https://{confluence-host}/pages/viewpage.action?pageId=123456` + - Short format: `https://{confluence-host}/x/AbCdEf` ### Step 2: Setup Output Directory @@ -57,48 +54,122 @@ agents/testgen/{TICKET-KEY}/ └── testgen-state.md (initialize) ``` +### Step 2b: Resolve MCP routing (user message vs capability file) + +**Precedence:** **A first** — fragment **section A** (user / task text **wins** over the file). **B second** — fragment **section B** (`agents/mcp-capability.yaml`). Full rules: **`mcp-capability-interaction.md`** (ACQUIRE per **Prerequisites**). + +**Checklist (execute in order before any Jira/Confluence MCP call in Phase 1):** + +1. Confirm **Prerequisites** ACQUIRE list for this phase is loaded (**`mcp-capability-interaction.md`**, and before guided MCP / Persist: **`mcp-tool-resolution.md`**, **`agent-action-resolution.md`** as listed there). +2. Read **`agents/mcp-capability.yaml`** at workspace root if present; if absent, treat capability config as **missing** (see **`mcp-capability-interaction.md`** Path conventions). +3. Apply fragment **§ A — user message** (task text wins over file). +4. If YAML is **missing** and the task text does not override MCP usage: ask **one** question — “Use live MCP for Jira and Confluence in this workspace?” — **STOP**, **WAIT**. **No** → both integrations **questionnaire** for this run. **Yes** → both **guided**; recommend adding `agents/mcp-capability.yaml`. +5. Derive **`Jira source`** and **`Confluence source`** using fragment **§ B** (`mode`, `mcp.jira`, `mcp.confluence`, omitted-key defaults). +6. Record in `agents/testgen/{TICKET-KEY}/testgen-state.md` under Phase 1: `Jira source`, `Confluence source`, and **MCP interaction source:** `agents/mcp-capability.yaml` **or** `user override` **or** `default question`. +7. If **`agents/user-instructions/mcp-guidance.md`** exists and at least one source is **guided**, read it before the first Jira/Confluence MCP call in Phase 1. + +#### Routing matrix (after Step 2b) + +Use `Jira source` and `Confluence source` from state. **Step labels:** **3Q** = Jira questionnaire (no Jira MCP); **3A** = Jira guided MCP; **4Q** = Confluence questionnaire; **4A** = Confluence guided MCP. **“URLs in Step 1?”** = Confluence URL(s) were parsed from the **initial** user prompt in Step 1. + +| Jira source | Confluence source | URLs in Step 1? | Execute | +|-------------|-------------------|-----------------|---------| +| guided | guided | yes | **3A** → **4A** Option A | +| guided | guided | no | **3A** → **4A** Option B | +| guided | questionnaire | — | **3A** → **4Q** | +| questionnaire | guided | yes | **3Q** → **4A** Option A | +| questionnaire | guided | no | **3Q** → **4A** Option B | +| questionnaire | questionnaire | — | **3Q** → **4Q** | + +#### `{integration-action:…}` intent reference (Phase 1) + +Use with **Token resolution** and **`mcp-tool-resolution.md`** when mapping tokens to host MCP tools. + +| Token | Intent | +|-------|--------| +| `{integration-action:jira-get-issue}` | Load one issue by key with field list, expand flags, comment limit | +| `{integration-action:jira-search-fields}` | Discover Jira field IDs / API names (custom fields) | +| `{integration-action:confluence-get-page}` | Load one Confluence page by id (markdown/metadata options) | +| `{integration-action:confluence-search-pages}` | Search Confluence pages (query + limit) | +| `{integration-action:confluence-list-child-pages}` | List child pages for a parent (content options) | + ### Step 3: Extract Jira Ticket Data -**Use**: Jira and/or Confluence MCPs respectively, snippets below will contain example pseudo-function calls for better understanding `mcp_Jira_MCP_jira_get_issue`, `mcp_Jira_MCP_confluence_get_page`, `mcp_Jira_MCP_confluence_search`, `mcp_Jira_MCP_confluence_get_page_children`. +Pick **exactly one** subsection below using the **Step 2b** matrix (`Jira source` column). **Do not** read, summarize, or execute the other subsection (**progressive disclosure**). + +#### Step 3Q: Jira questionnaire (no MCP) + +**Jira MCP:** **off** — no **`{integration-action:jira-*}`** calls; user-provided content only. -**Extract ticket key** from user input: -- Format: "PROJ-123" or URL "https://jira.company.com/browse/PROJ-123" -- Parse key from URL if needed +1. Ask **numbered** questions so you can build the Jira section of `raw-data.md`: ticket summary, description (or ask user to paste export), status, priority, labels, components, key links, and any acceptance criteria they rely on. +2. **STOP** and **WAIT** for answers. +3. Write `agents/testgen/{TICKET-KEY}/raw-data.md` (create if needed) with `## Jira (user-provided, MCP absent)` containing merged answers. For any field the user could not supply or was unsure about, add an explicit **`unknown`**, **`unverified`**, or short **why missing** note next to that item (do not imply MCP-grade certainty). Then continue per the **Step 2b** matrix to Step 4. -**Retrieve issue** with comprehensive fields: -```python -mcp_Jira_MCP_jira_get_issue( - issue_key="PROJ-123", - fields="summary,description,status,issuetype,assignee,priority,reporter,labels,components,created,updated", - expand="renderedFields", - comment_limit=10 -) +#### Step 3A: Guided Jira (MCP) + +1. **Extract ticket key** from user input: + - Format: "PROJ-123" or URL "https://{jira-host}/browse/PROJ-123" + - Parse key from URL if needed + +2. **Retrieve issue** — see **Token resolution**, **`mcp-tool-resolution.md`**, and **Intent reference** above for **`{integration-action:jira-get-issue}`**. Then call the resolved tool; logical parameters: +```text +Action: {integration-action:jira-get-issue} +Parameters: + issue_key: PROJ-123 + fields: summary,description,status,issuetype,assignee,priority,reporter,labels,components,created,updated + expand: renderedFields + comment_limit: 10 ``` -**Capture**: -- Summary, description (both raw and rendered) -- Issue type, status, priority -- Labels, components -- Assignee, reporter -- Comments (up to 10 recent) -- Created/updated dates -- Custom fields if present (epic link, story points, etc.) +**Example (mapping only):** On a common Atlassian host the resolved tool is `mcp__atlassian__jira_get_issue`; rename keys from the block above as required (e.g. `issue_key` → `issueKey`, `comment_limit` → `commentLimit`). Other tokens: see **Token resolution**. + +3. **Capture** (for Step 5 / `raw-data.md`): + - Summary, description (both raw and rendered) + - Issue type, status, priority + - Labels, components + - Assignee, reporter + - Comments (up to 10 recent) + - Created/updated dates + - Custom fields if present (epic link, story points, etc.) + +#### Step 3A.4 — Jira field schema (optional) + +Run **only** when substep **3** leaves custom-field **API names or ids** ambiguous for `raw-data.md`. + +1. **Invoke** **`{integration-action:jira-search-fields}`** **once** (see **Token resolution**). +2. **Output:** merge the returned field metadata into working notes and into **`### Custom Fields`** (and related) when executing **Step 5** (`raw-data.md`). If substep 3 was sufficient, **skip** this entire **3A.4** subsection. ### Step 4: Get Confluence Documentation -**Decision Point**: Did user provide Confluence URLs in initial prompt? +Pick **exactly one** subsection below using the **Step 2b** matrix (`Confluence source` column; for **4A**, use the **URLs in Step 1?** column for Option A vs B). **Do not** read, summarize, or execute the other subsection (**progressive disclosure**). + +#### Step 4Q: Confluence questionnaire (no MCP) + +**Confluence MCP:** **off** — no **`{integration-action:confluence-*}`** calls; user-provided content only. -#### Option A: User Provided Confluence URLs +1. Ask **numbered** questions: paste each page body (or export), or provide URLs plus authorized excerpts, parent/child relationships if relevant. +2. **STOP** and **WAIT**. +3. Append to `agents/testgen/{TICKET-KEY}/raw-data.md` under `## Confluence (user-provided, MCP absent)` with merged answers. Mark gaps, pasted excerpts of unknown completeness, or user-stated uncertainty the same way as Step 3Q (**`unknown` / `unverified` / reason**). If Jira questionnaire already created the file, append; otherwise create the file with this section. +4. Continue to **Step 5** so `raw-data.md` matches the full template (headings, metadata). Then run **Validation**. + +#### Step 4A: Guided Confluence (MCP) + +**Option A vs B:** From the **Step 2b** matrix (**URLs in Step 1?** column). + +Resolve each **`{integration-action:confluence-*}`** (see **Token resolution**; e.g. `confluence-get-page` → `mcp__atlassian__confluence_get_page` with `pageId` / `convertToMarkdown` when the host uses that idiom). + +##### Option A: User-provided Confluence URLs -**If URLs provided in initial prompt**: 1. Parse page IDs from URLs 2. For each URL, extract: - Page ID from URL parameters (pageId=123456) - Or use space + title from display URL -3. Retrieve pages directly using `mcp_Jira_MCP_confluence_get_page()` +3. Retrieve pages using **`{integration-action:confluence-get-page}`** 4. Check each page for child pages (REQUIRED) 5. Skip automatic search +**Illustrative resolved call (Atlassian-style host, not mandatory):** `mcp__atlassian__confluence_get_page` with arguments such as `{ "pageId": "123456", "convertToMarkdown": true }` (exact keys per host mapping). + **Tell user**: ``` ✅ Using provided Confluence pages: @@ -107,9 +178,9 @@ mcp_Jira_MCP_jira_get_issue( 🔍 Checking for child pages... ``` -#### Option B: No URLs Provided - Auto-Search +##### Option B: No URLs — auto-search -**Use**: `mcp_Jira_MCP_confluence_search()` +**Use**: **`{integration-action:confluence-search-pages}`** **Extract search terms** from Jira ticket: - Project key (from ticket key) @@ -123,35 +194,37 @@ type=page AND space={PROJECT_KEY} AND (text ~ "{term1}" OR text ~ "{term2}") ``` **Search Confluence**: -```python -mcp_Jira_MCP_confluence_search( - query=cql_query, - limit=10 -) +```text +Action: {integration-action:confluence-search-pages} +Parameters: + query: + limit: 10 ``` +**Illustrative resolved call (Atlassian-style host, not mandatory):** `mcp__atlassian__confluence_search_pages` (or host-equivalent) with CQL in the query argument, e.g. `type=page AND space=PROJ AND (text ~ "auth" OR text ~ "login")`. + **Rank results** by relevance: - Title matches ticket terms - Labels match ticket labels - Content matches key terms **Get top 3-5 pages**: -```python -mcp_Jira_MCP_confluence_get_page( - page_id=page_id, - convert_to_markdown=True, - include_metadata=True -) +```text +Action: {integration-action:confluence-get-page} +Parameters: + page_id: + convert_to_markdown: true + include_metadata: true ``` **IMPORTANT: Check for child pages** (nested documents often missed by search): -```python -mcp_Jira_MCP_confluence_get_page_children( - parent_id=page_id, - include_content=True, - convert_to_markdown=True, - limit=10 -) +```text +Action: {integration-action:confluence-list-child-pages} +Parameters: + parent_id: + include_content: true + convert_to_markdown: true + limit: 10 ``` For each parent page found: @@ -186,6 +259,19 @@ For each parent page found: **File**: `agents/testgen/{TICKET-KEY}/raw-data.md` +**Persist** — see **Token resolution** for **`{agent-action:write-file}`** / **`{agent-action:patch-file}`**: + +```text +Action: {agent-action:write-file} +Parameters: + path: agents/testgen/{TICKET-KEY}/raw-data.md + content: +``` + +**Illustrative host mapping (Cursor-style agent, not mandatory):** tool `Write` with `file_path` / `contents` (or host-equivalent keys after resolution). + +If the file already exists and you are only appending, you may use **`{agent-action:patch-file}`** instead (see **Token resolution**). + **Format**: ```markdown # Raw Data - [TICKET-KEY] @@ -193,6 +279,7 @@ For each parent page found: **Extracted**: [DateTime] **Phase**: 1 - Data Collection **Confluence Source**: [User-provided URLs / Auto-search / User-provided after search / Skipped] +**Confidence / Unknowns**: [If either source was questionnaire: list unverified, missing, or user-uncertain items; else `MCP-backed` or `n/a`] --- @@ -279,12 +366,22 @@ For each parent page found: - **Total Content Size**: [Approximate word count] - **Search Terms Used**: [List] - **Notes**: [Any issues during extraction] +- **Confidence / Unknowns**: [Same as header line; required when any questionnaire path ran] ``` ### Step 6: Update State File **File**: `agents/testgen/{TICKET-KEY}/testgen-state.md` +**Persist** — same as Step 5 (**see Token resolution**): + +```text +Action: {agent-action:write-file} +Parameters: + path: agents/testgen/{TICKET-KEY}/testgen-state.md + content: +``` + **Create initial state**: ```markdown # Test Generation State - [TICKET-KEY] @@ -329,18 +426,26 @@ For each parent page found: Before completing Phase 1, verify: - ✅ `agents/testgen/{TICKET-KEY}/` directory exists -- ✅ `raw-data.md` created with Jira section populated +- ✅ Phase 1 block in `testgen-state.md` contains lines **`Jira source`**, **`Confluence source`**, and **`MCP interaction source`** +- ✅ If **`Jira source` is `questionnaire`**: `raw-data.md` contains `## Jira (user-provided, MCP absent)` +- ✅ If **`Confluence source` is `questionnaire`**: `raw-data.md` contains `## Confluence (user-provided, MCP absent)` +- ✅ If any **questionnaire** path ran: `raw-data.md` **Data Collection Summary** includes **`Confidence / Unknowns`** +- ✅ `raw-data.md` created with Jira section populated (MCP or questionnaire path) - ✅ Confluence section has at least 1 page OR user confirmed skip - ✅ `testgen-state.md` created with Phase 1 marked complete - ✅ All key Jira fields captured (summary, description, status, priority) ## Tools Used -- `mcp_Jira_MCP_jira_get_issue()` - Jira ticket extraction -- `mcp_Jira_MCP_confluence_search()` - Confluence page search -- `mcp_Jira_MCP_confluence_get_page()` - Confluence page content retrieval -- `mcp_Jira_MCP_confluence_get_page_children()` - Confluence child page discovery -- `write()` - File creation +| Token / use | Where | +|---------------|--------| +| `{integration-action:jira-get-issue}` | Step 3A.2 | +| `{integration-action:jira-search-fields}` | Step 3A.4 (conditional) | +| `{integration-action:confluence-search-pages}` | Step 4A Option B | +| `{integration-action:confluence-get-page}` | Step 4A Options A / B | +| `{integration-action:confluence-list-child-pages}` | Step 4A Option B | +| `{agent-action:write-file}` | Steps 5–6 (**Persist** blocks) | +| `{agent-action:patch-file}` | Steps 5–6 (optional append / edit) | ## Common Issues @@ -354,10 +459,10 @@ Before completing Phase 1, verify: **Solution**: Include first 5000 words, note truncation in raw-data.md **Issue**: Custom fields not recognized -**Solution**: Use `mcp_Jira_MCP_jira_search_fields()` to discover field names +**Solution**: Run **Step 3A.4** — **`{integration-action:jira-search-fields}`** (or equivalent) to discover field names **Issue**: Confluence search finds parent but misses child pages -**Solution**: Always check for child pages using `confluence_get_page_children()` for each found page +**Solution**: Always check for child pages using **`{integration-action:confluence-list-child-pages}`** for each found page **Issue**: User provided invalid Confluence URL **Solution**: Try to parse page ID, if fails ask user for correct URL or page ID @@ -378,8 +483,6 @@ After Phase 1 completion: - Confluence search may need tuning based on organization's Confluence structure - Some Jira instances have custom fields - capture all available - Confluence pages may be in different spaces - search broadly initially -- **User can provide Confluence URLs in initial prompt** - this skips auto-search -- If user provides specific page URLs/IDs, use those directly instead of search - **CRITICAL**: Always check for child pages - nested documentation often contains the most relevant details - Example: "Job Post" parent may have children "Create a Job Post", "Edit a Job Post", etc. - Retrieve up to 10 child pages per parent, prioritize by relevance to ticket diff --git a/plugins/core-codex/.agents/workflows/testgen-flow-test-case-export.md b/plugins/core-codex/.agents/workflows/testgen-flow-test-case-export.md index e2cce06c..7e9164fc 100644 --- a/plugins/core-codex/.agents/workflows/testgen-flow-test-case-export.md +++ b/plugins/core-codex/.agents/workflows/testgen-flow-test-case-export.md @@ -12,26 +12,70 @@ baseSchema: docs/schemas/phase.md - Phase 5 MUST be complete - `agents/testgen/{TICKET-KEY}/test-scenarios.md` exists with test cases - User has reviewed and edited test cases -- TestRail MCP configured and accessible -- TestRail project_id and suite_id known +- **TestRail export path (B + A):** Either **guided** (TestRail MCP allowed per `mcp-capability-interaction.md` and `agents/mcp-capability.yaml`) or **questionnaire** (no TestRail MCP; user supplies IDs and performs or confirms manual import). Optional **`agents/user-instructions/mcp-guidance.md`** when guided. +- **Placeholder binding:** Before first guided **`{integration-action:*}`** call, ACQUIRE **`mcp-tool-resolution.md`**. For **`{agent-action:*}`** file steps, ACQUIRE **`agent-action-resolution.md`**. Canonical: **`instructions/r2/core/workflows/.md`** if KB alias fails. +- For **guided** runs: TestRail MCP configured and accessible; `project_id` and `suite_id` known (from guidance, state, or user). ## Objective -Export test cases from test-scenarios.md to TestRail using MCP, creating a new section (folder) for the Jira ticket and adding all test cases. -If MCP is not available, ask user how to proceed and provide possible options. +Export test cases from `test-scenarios.md` to TestRail. + +- **Guided:** Use TestRail MCP to verify the project, resolve `section_id`, and create cases. +- **Questionnaire:** Do **not** run any **`{integration-action:testrail-*}`** call. Collect `project_id`, `suite_id`, and `section_id` from the user, write a **manual export pack** (copy-paste / CSV-friendly), and record completion when the user confirms import or pastes back TestRail case IDs. + +If the interaction mode is unclear, resolve it in **Step 0** before any MCP call. ## TestRail Configuration **Default Configuration**: detect using current ticket and user profile. **Section Name**: [TICKET-KEY] (e.g., "PROJ-123") +**Web URL placeholders:** `{testrail-suite-view-url}`, `{testrail-case-view-url}`, and `{testrail-suite-cases-grouped-url}` stand in for full TestRail browser links (`https://` + host + `index.php?` paths + ids). Compose from `{TestRail instance host}`, `suite_id`, case id, and section grouping as in Step 1Q or `mcp-guidance.md`. + +### Guided TestRail and workspace file actions (placeholders) + +| Token | Intent | +|-------|--------| +| `{integration-action:testrail-get-project}` | Verify TestRail project access by `project_id` | +| `{integration-action:testrail-get-cases}` | List cases in a project/suite (optional dedup before add) | +| `{integration-action:testrail-add-case}` | Create a test case in a `section_id` with title, priority, type, refs, separated steps | +| `{agent-action:read-file}` | Read a workspace file (host-specific file tool) | +| `{agent-action:write-file}` | Create or overwrite a workspace file | +| `{agent-action:patch-file}` | Apply targeted edits to a workspace file (includes search-and-replace style edits) | + +Resolve **`{integration-action:…}`** using **`mcp-tool-resolution.md`** (guided only). Resolve **`{agent-action:…}`** using **`agent-action-resolution.md`**. ## Requirements -### Step 1: Verify TestRail Connection +### Step 0: Resolve TestRail MCP interaction (B + A) -**Test connection** using: -```python -mcp_testrail_get_project(project_id=69) +1. ACQUIRE **`mcp-capability-interaction.md`** FROM KB. ACQUIRE **`mcp-tool-resolution.md`** before any **`{integration-action:*}`** MCP call (when **guided**). ACQUIRE **`agent-action-resolution.md`** before any **`{agent-action:*}`** workspace step. If KB alias fails, use **`instructions/r2/core/workflows/.md`**. +2. Read **`agents/mcp-capability.yaml`** if present; apply **user override** from the current task text (same rules as Phase 1). +3. Derive **`TestRail export: guided | questionnaire`**: + - **`mcp.mode: absent`** → questionnaire. + - **`mcp.mode: capable`** and **`mcp.testrail: false`** (when the key is present) → questionnaire. + - **`mcp.mode: capable`** and (`testrail` omitted or `true`) → guided. + - Missing YAML: use **user override** if any; else **one** short question (“Use TestRail MCP for export?”). **No** → questionnaire; **Yes** → guided and recommend adding `agents/mcp-capability.yaml`. +4. Record in `agents/testgen/{TICKET-KEY}/testgen-state.md` (Phase 6): + - `TestRail export: guided | questionnaire` + - `MCP interaction source:` (`agents/mcp-capability.yaml` | user override | default question`) +5. If **guided** and **`agents/user-instructions/mcp-guidance.md`** exists, read it before the first **`{integration-action:testrail-*}`** call. + +### Step 1: Verify TestRail connection (guided only) + +**If `TestRail export` is `questionnaire`:** Run **Step 1Q** instead of this step. + +#### Step 1Q: Questionnaire — no MCP verification + +1. Ask **numbered** questions: `project_id`, `suite_id` (for links and documentation), and confirm the user will import cases manually or paste back IDs after import. +2. **STOP** and **WAIT**. +3. Skip to **Step 2** (section_id flow is unchanged; user may already know `section_id`). + +**Guided path — test connection** using (replace `project_id` with value from guidance/state/user; **do not** hard-code example IDs unless the user confirmed them): + +```text +Action: {integration-action:testrail-get-project} +Parameters: + project_id: ``` If fails, tell user: @@ -39,7 +83,7 @@ If fails, tell user: ❌ TestRail connection failed. Please verify: 1. TestRail MCP is configured 2. Credentials are correct -3. Project ID 69 exists and you have access +3. Project ID exists and you have access ``` ### Step 2: Create Section in TestRail @@ -57,7 +101,7 @@ If you already have a section, provide the section_id. You can find it in the URL when viewing a section (e.g., group_id=94686 or section_id=94686) **Option B: Create new section** -1. Go to: https://griddynamics.testrail.io/index.php?/suites/view/3300 +1. Go to: {testrail-suite-view-url} 2. Click "Add Section" 3. Name it: [TICKET-KEY] (e.g., "PROJ-456") 4. After creating, find the section_id in the URL or section details @@ -224,19 +268,40 @@ Execute this test case for EACH row in the table below: - Just include original preconditions normally - No "Execute for EACH row" note -### Step 5: Add Test Cases to TestRail +### Step 5: Add Test Cases to TestRail (guided only) -**For each mapped test case**: +**If `TestRail export` is `questionnaire`:** Run **Step 5Q** only; **do not** run **`{integration-action:testrail-add-case}`**. -```python -result = mcp_testrail_add_case( - section_id=section_id, - title=testrail_case["title"], - priority_id=testrail_case["priority_id"], - type_id=testrail_case["type_id"], - refs=testrail_case["refs"], - custom_steps_separated=testrail_case["custom_steps_separated"] -) +#### Step 5Q: Manual export pack (no MCP) + +1. After **Step 4** mapping, write **`agents/testgen/{TICKET-KEY}/testrail-manual-export.md`** containing: + - User-supplied `project_id`, `suite_id`, `section_id` (if known) + - One block per test case: title, priority, type, refs, preconditions text, and steps (content + expected) in plain text or markdown tables suitable for copy-paste into TestRail. +2. Optionally add **`agents/testgen/{TICKET-KEY}/testrail-import-hints.csv`** (title, priority, type, refs, preconditions, steps_json) if the team uses CSV import. +3. **STOP** and **WAIT**: ask the user to confirm either (a) they created cases manually / imported the pack, or (b) they paste back a mapping `TC-001 → C12345` (or URLs). +4. **After the user replies**, execute **Step 6** to merge IDs into `test-scenarios.md`. Until then you may leave placeholders (e.g. `Pending`) only if the workflow must save intermediate files; do **not** mark Phase 6 complete in state until Step 6 reflects the user’s answer. + +**Optional (guided only) — list existing cases before add:** If duplicate titles, re-exports, or unclear existing coverage are a concern, run **`{integration-action:testrail-get-cases}`** once against the target `project_id` / `suite_id` (and use section or other filters when your MCP supports them) to compare with parsed markdown. **Not required** for a first-time export or when the user accepts duplicates. + +```text +Action: {integration-action:testrail-get-cases} +Parameters: + project_id: + suite_id: +``` + +**Guided path — for each mapped test case**: + +```text +Action: {integration-action:testrail-add-case} +Parameters: + section_id: + title: + priority_id: + type_id: + refs: + custom_steps_separated: +Outcome: capture returned case id for results tracking ``` **Track results**: @@ -261,12 +326,14 @@ results = { **Update**: `agents/testgen/{TICKET-KEY}/test-scenarios.md` +For **questionnaire** exports, use user-pasted IDs when available; otherwise keep **`TestRail ID`**: `Manual (see testrail-manual-export.md)`** until the user confirms. Use real `project` / `suite` / base URL values from Step 1Q or guidance — **do not** invent instance hostnames. + **Add TestRail IDs to each test case**: ```markdown ### TC-001: User Login with Valid Credentials (Happy Path) **TestRail ID**: C12345 ✅ -**TestRail Link**: https://griddynamics.testrail.io/index.php?/cases/view/12345 +**TestRail Link**: {testrail-case-view-url} **Related Requirement**: US-1, FR-1 ... ``` @@ -283,11 +350,11 @@ results = { ## TestRail Export Summary **Exported**: [DateTime] -**Project**: 69 -**Suite**: 3300 +**Project**: {project_id} +**Suite**: {suite_id} **Section**: [TICKET-KEY] (ID: [section_id]) **Total Exported**: [X] test cases -**TestRail Link**: https://griddynamics.testrail.io/index.php?/suites/view/3300&group_by=cases:section_id +**TestRail Link**: {testrail-suite-cases-grouped-url} | TC ID | TestRail ID | Status | |-------|-------------|--------| @@ -326,8 +393,8 @@ results = { ### Phase 6: TestRail Export - **Completed**: [DateTime] -- **Project ID**: 69 -- **Suite ID**: 3300 +- **Project ID**: {project_id} +- **Suite ID**: {suite_id} - **Section ID**: [section_id] - **Section Name**: [TICKET-KEY] - **Test Cases Created**: [Count] @@ -339,21 +406,30 @@ results = { ## Validation Before completing Phase 6, verify: -- ✅ TestRail connection successful -- ✅ Section exists in TestRail + +**Guided path:** +- ✅ TestRail connection successful (`{integration-action:testrail-get-project}` or equivalent) +- ✅ Section exists in TestRail (or user provided valid `section_id`) - ✅ All test cases parsed from markdown -- ✅ At least 80% of test cases exported successfully -- ✅ test-scenarios.md updated with TestRail IDs +- ✅ At least 80% of test cases exported successfully via MCP (or failures documented) +- ✅ `test-scenarios.md` updated with TestRail IDs - ✅ State file updated with Phase 6 complete - ✅ TestRail link provided to user +**Questionnaire path:** +- ✅ Step 0 recorded `TestRail export: questionnaire` and source line +- ✅ User confirmed `project_id` / `suite_id` / `section_id` as needed +- ✅ `testrail-manual-export.md` (and optional CSV) written +- ✅ User confirmed manual import or pasted ID mapping +- ✅ `test-scenarios.md` updated with IDs or explicit “manual / pending” notes +- ✅ State file updated with Phase 6 complete + ## Tools Used -- `mcp_testrail_get_project(project_id)` - Verify connection -- `mcp_testrail_get_cases(project_id, suite_id)` - Check existing cases -- `mcp_testrail_add_case(section_id, title, ...)` - Create test cases -- `read_file()` - Read test-scenarios.md -- `write()` / `search_replace()` - Update files +- **Guided:** `{integration-action:testrail-get-project}` — verify connection; `{integration-action:testrail-get-cases}` — optional dedup; `{integration-action:testrail-add-case}` — create test cases +- **Questionnaire:** `{agent-action:read-file}` / `{agent-action:write-file}` / `{agent-action:patch-file}` only — build `testrail-manual-export.md` and update docs; **no** `{integration-action:testrail-*}` calls +- `{agent-action:read-file}` — read `test-scenarios.md` +- `{agent-action:write-file}` / `{agent-action:patch-file}` — update files ## Common Issues @@ -375,6 +451,9 @@ Before completing Phase 6, verify: **Issue**: Rate limit exceeded **Solution**: Add delay between API calls, batch requests +**Issue**: Risk of duplicate titles or unclear existing coverage +**Solution**: Optionally run **`{integration-action:testrail-get-cases}`** (Step 5 optional block) before **`{integration-action:testrail-add-case}`** — optional discovery of existing cases, not a mandatory gate. + **Issue**: Test case already exists **Solution**: Create anyway (TestRail allows duplicates), note in report @@ -393,7 +472,7 @@ All 6 phases finished successfully: ✅ Phase 6: TestRail Export ([R] cases exported) **TestRail Section**: [TICKET-KEY] -**TestRail Link**: https://griddynamics.testrail.io/index.php?/suites/view/3300 +**TestRail Link**: {testrail-suite-view-url} **Deliverables**: 📄 requirements.md - Use for implementation @@ -411,8 +490,8 @@ All 6 phases finished successfully: - TestRail MCP currently lacks section creation - user must create manually - Test case IDs in TestRail are prefixed with "C" (e.g., C12345) -- Suite ID 3300 is from user's TestRail URL -- Project ID 69 is user's default project +- `{suite_id}` comes from the user, `mcp-guidance.md`, or the suite view URL on their instance +- `{project_id}` comes from the user, `mcp-guidance.md`, or the TestRail project settings - Parameterized test data is included in step content or expected results - Re-running export creates duplicate test cases (by design, to preserve history) - Consider creating test run after export for immediate execution diff --git a/plugins/core-codex/.agents/workflows/testgen-flow.md b/plugins/core-codex/.agents/workflows/testgen-flow.md index 1e334ca6..de88e4c8 100644 --- a/plugins/core-codex/.agents/workflows/testgen-flow.md +++ b/plugins/core-codex/.agents/workflows/testgen-flow.md @@ -14,6 +14,7 @@ Systematic requirements analysis from Jira tickets and Confluence documentation ### Critical Requirements +- **MCP ROUTING (B + A)**: Phases 1 and 6 may call external MCPs only when **`agents/mcp-capability.yaml`** and **`mcp-capability-interaction.md`** resolve **guided** for that integration. Otherwise use **questionnaire** or manual export paths documented in those phases. Optional **`agents/user-instructions/mcp-guidance.md`** for IDs and guardrails when guided. **`{integration-action:*}`** / **`{agent-action:*}`** binding: follow each phase file’s **Prerequisites** (**`mcp-tool-resolution.md`**, **`agent-action-resolution.md`**). - **ONE PHASE AT A TIME**: Read phase file, execute, update state, move to next. - **DO NOT SKIP PHASES**: Each builds on previous. - **STATE TRACKING**: Update `agents/testgen/{TICKET-KEY}/testgen-state.md` after each phase. @@ -31,9 +32,10 @@ Systematic requirements analysis from Jira tickets and Confluence documentation **Phase 1: Data Collection** [testgen-flow-data-collection.md] 1. ACQUIRE testgen-flow-data-collection.md FROM KB -2. Execute phase instructions -3. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` -4. Validate by listing raw-data.md file +2. ACQUIRE **`mcp-capability-interaction.md`**, and per that phase’s **Prerequisites** also **`mcp-tool-resolution.md`** / **`agent-action-resolution.md`** when placeholders are used +3. Execute phase instructions +4. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` +5. Validate by listing raw-data.md file **Phase 2: Gap & Contradiction Analysis** [testgen-flow-gap-and-contradiction-analysis.md] 1. ACQUIRE testgen-flow-gap-and-contradiction-analysis.md FROM KB @@ -61,8 +63,9 @@ Systematic requirements analysis from Jira tickets and Confluence documentation **Phase 6: Test Case Export** [testgen-flow-test-case-export.md] ⭐ 1. ACQUIRE testgen-flow-test-case-export.md FROM KB -4. Execute phase instructions -5. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` +2. ACQUIRE **`mcp-capability-interaction.md`** and per that phase’s **Step 0** / **Prerequisites** also **`mcp-tool-resolution.md`** / **`agent-action-resolution.md`** when placeholders are used (guided vs questionnaire for TestRail export) +3. Execute phase instructions +4. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` ## State File Format @@ -136,9 +139,9 @@ agents/testgen/{TICKET-KEY}/ ## Prerequisites -- **Jira MCP**: Configured and accessible -- **Jira Ticket**: User provides ticket key or link -- **Confluence Access**: Via Jira MCP (same authentication) +- **MCP routing (B + A):** `agents/mcp-capability.yaml` (see `instructions/r2/core/templates/mcp-capability.example.yaml`) plus optional `agents/user-instructions/mcp-guidance.md` control **guided** vs **questionnaire** per integration. Phases 1 and 6 embed the branching rules. +- **Jira ticket**: User provides ticket key or link (Phase 1) +- **Confluence**: Either MCP-guided discovery (Phase 1) or user-provided content when **questionnaire** - **Output Directory**: Created at start (`agents/testgen/{TICKET-KEY}/`) ## Common Patterns diff --git a/plugins/core-copilot-standalone/.github/prompts/agent-action-resolution.prompt.md b/plugins/core-copilot-standalone/.github/prompts/agent-action-resolution.prompt.md new file mode 100644 index 00000000..75fb60a9 --- /dev/null +++ b/plugins/core-copilot-standalone/.github/prompts/agent-action-resolution.prompt.md @@ -0,0 +1,18 @@ +--- +name: agent-action-resolution +description: Maps `{agent-action:…}` tokens to host IDE or agent-runtime file tools (read, write, patch). +alwaysApply: false +baseSchema: docs/schemas/phase.md +--- + +# `{agent-action:…}` resolution (host file tools) + +Use when a workflow uses **`{agent-action:read-file}`**, **`{agent-action:write-file}`**, or **`{agent-action:patch-file}`**. These are **not** remote MCP calls and are **not** covered by **`mcp-tool-resolution.md`**. + +| Token | Intent | +|-------|--------| +| `{agent-action:read-file}` | Read a file from the workspace via the host agent’s file-read tool | +| `{agent-action:write-file}` | Create or overwrite a workspace file via the host agent’s file-write tool | +| `{agent-action:patch-file}` | Apply targeted edits via the host agent’s patch / search-replace tool | + +**Resolution:** Map each token to the host IDE’s or agent runtime’s **documented** file operation (names differ: e.g. `Read`, `Write`, `StrReplace`, `apply_patch`). If the host exposes no matching capability, **STOP**, tell the user, and **WAIT** — do not substitute an MCP tool. diff --git a/plugins/core-copilot-standalone/.github/prompts/aqa-flow-data-collection.prompt.md b/plugins/core-copilot-standalone/.github/prompts/aqa-flow-data-collection.prompt.md index 34fc0898..a9e7253d 100644 --- a/plugins/core-copilot-standalone/.github/prompts/aqa-flow-data-collection.prompt.md +++ b/plugins/core-copilot-standalone/.github/prompts/aqa-flow-data-collection.prompt.md @@ -9,19 +9,61 @@ baseSchema: docs/schemas/phase.md ## Objective -Gather all required information from external sources (TestRail and Confluence) to understand test requirements and expected behavior. +Gather everything needed for later phases: either from **TestRail and Confluence** (**integrated AQA**) or from **user-supplied artifacts** (**minimal-input / agent-led**). Always record the chosen **execution mode** in the test plan and state file. ## Prerequisites -- TestRail MCP configured and accessible -- Atlassian (Confluence) MCP configured and accessible -- Test case ID or requirement provided by user +- User intent is clear enough to start Phase 1 (IDs, URLs, or a plain-language automation goal). +- **Integrated AQA path**: For **guided** MCP interaction (see Task 0b), TestRail MCP and Atlassian (Confluence) MCP must be usable when you run Path A MCP steps. For **questionnaire** interaction, do **not** call those MCPs; collect the same facts via user answers and paste into the test plan. +- **Minimal-input path**: No MCP requirement for TestRail/Confluence; Task 0b **does not apply** to Path B for those integrations. If something on the checklist is missing, **ask** — do not guess. + +## MCP capability reference + +Resolve **guided vs questionnaire** per **`mcp-capability-interaction.md`** (ACQUIRE FROM KB). Config file: **`agents/mcp-capability.yaml`** (copy from `instructions/r2/core/templates/mcp-capability.example.yaml` in Rosetta repo). Optional guidance: **`agents/user-instructions/mcp-guidance.md`**. ## Phase Tasks -### Task 1: Read TestRail Test Case +### Task 0: Confirm execution mode + +**Actions**: +1. Check whether the **current task text** (or a prior user message in the same session the user instructs you to treat as binding) already names the mode **explicitly**, using any of these (case-insensitive): `integrated`, `minimal-input`, `minimal input`, `Integrated AQA`, or `minimal-input / agent-led`. The token must be **user-authored** in that message (not something you paraphrase or infer). If yes, set the mode to `integrated` or `minimal-input` accordingly and **skip steps 2–3**; go directly to **Path A** or **Path B** matching that mode. +2. If **not** explicit: **Do not infer** mode from clues such as missing TestRail IDs, presence of HTML attachments, “sandbox,” “no MCP,” or vague phrases like “I don’t have TestRail yet.” **Forbidden:** choosing `minimal-input` because TestRail was omitted, the repo looks small, or the user only supplied local HTML—those are **not** mode labels. Informal phrases (“lightweight,” “without TestRail,” “bring my own selectors”) **do not** count as explicit; they still require steps 2–3. **Ask once** with both options, for example: + - **Integrated AQA**: "We pull the official case and docs from TestRail/Confluence and follow the full chain." + - **Minimal-input / agent-led**: "You provide URLs, page source or a selector map, scenario and expectations; we still run phases 2–8 in order but Phase 1 records your package instead of MCP pulls." +3. **STOP** and **WAIT** until the user replies with a clear **integrated** or **minimal-input** (or the spelled labels in step 1). If the reply is still ambiguous, ask **one** narrowing question that still requires those labels; **do not** choose a path without those words from the user. +4. **Hard gate — before Path A or Path B:** Do **not** create or update `agents/plans/aqa-*.md`, do **not** write `**AQA execution mode**` in any artifact, and do **not** mark Phase 1 complete in `agents/aqa-state.md` until step **1** matched or step **3** completed. If you already started Path A/B without satisfying Task 0, **stop**, discard partial mode lines, run steps 2–3, then continue. +5. Ensure the Phase 1 test plan created in Path A or Path B includes `**AQA execution mode**` and that `agents/aqa-state.md` reflects the mode when Phase 1 is marked complete (see **Update State File** below). + +**Expected Output**: Mode is either read from **explicit user keywords** (step 1) or obtained only after **mandatory** ask-and-wait (steps 2–3); then proceed with **Path A** or **Path B** (not both). + +--- + +### Task 0b: Resolve MCP interaction (Path A only) + +**Skip this entire task for Path B (minimal-input).** **Actions**: +1. ACQUIRE **mcp-capability-interaction.md** FROM KB (if not already loaded this phase). +2. Read **`agents/mcp-capability.yaml`** if it exists; apply **user override** rules from that fragment (task text beats file). If file missing and user has not overridden, ask **one** yes/no: “Use live MCP for TestRail/Confluence in this workspace?” — **WAIT**; treat **No** as questionnaire, **Yes** as guided for both integrations until a YAML file exists. +3. Derive **per integration** (TestRail vs Confluence) whether the source is **guided** (MCP allowed) or **questionnaire** (no MCP): if `mcp.mode` is `absent`, both are **questionnaire**; if `capable`, use `mcp.testrail` / `mcp.atlassian_confluence` when present (boolean); if a key is **omitted** under `capable`, default that integration to **guided**. +4. Record in the test plan and `agents/aqa-state.md` Phase 1 notes: `TestRail source: guided|questionnaire`, `Confluence source: guided|questionnaire`, plus `MCP interaction source:` (`agents/mcp-capability.yaml` | user override | default question`). +5. **Questionnaire leg:** For each integration still **questionnaire** and not yet documented in the plan, run **one** numbered questionnaire (combine TestRail + Confluence in a single message if both need data). **STOP** and **WAIT**. Merge answers under **User-provided (MCP absent) — TestRail** and/or **User-provided (MCP absent) — Confluence** in the working plan (or append to the plan draft before Task A3). +6. **Guided leg:** If **`agents/user-instructions/mcp-guidance.md`** exists and at least one integration is **guided**, read it before the first MCP call for a **guided** integration. Then continue to Task A1 / A2 and follow the **guided** branches there only. + +**Expected Output**: TestRail and Confluence are each classified **guided** or **questionnaire**; questionnaire content is captured before MCP calls; no MCP call runs for a **questionnaire** integration. + +--- + +### Path A — Integrated AQA (TestRail + Confluence) + +**Prerequisite:** Task 0 already set execution mode to **integrated** (step 1 keyword match or step 3 user reply). Do not enter Path A without that. + +### Task A1: Read TestRail Test Case + +**Actions**: +- **If Task 0b resolved questionnaire for TestRail:** Ensure the test plan already contains **User-provided (MCP absent) — TestRail** content from Task 0b. If missing, **STOP**, ask numbered questions (case ID for traceability, title, steps, expected results, preconditions), **WAIT**, merge into the plan. **Do not** call `user-testrail-get_case`. Skip the guided-only steps below for TestRail. +- **If Task 0b resolved guided for TestRail:** Perform steps 1–4. + 1. Ask user for TestRail test case ID if not provided 2. Use TestRail MCP to retrieve test case details: ``` @@ -39,9 +81,12 @@ Gather all required information from external sources (TestRail and Confluence) **Expected Output**: Complete understanding of what needs to be tested according to TestRail. -### Task 2: Read Confluence Documentation +### Task A2: Read Confluence Documentation **Actions**: +- **If Task 0b resolved questionnaire for Confluence:** Ensure the test plan contains **User-provided (MCP absent) — Confluence** content from Task 0b. If missing, **STOP**, ask numbered questions (URLs, pasted excerpts, or summaries the user authorizes), **WAIT**, merge into the plan. **Do not** call `user-mcp-atlassian-confluence_*`. Skip guided-only steps below for Confluence. +- **If Task 0b resolved guided for Confluence:** Perform steps 1–5. + 1. Ask user for Confluence page ID/URL or search terms if not provided 2. Use Atlassian Confluence MCP to find related documentation: ``` @@ -60,10 +105,11 @@ Gather all required information from external sources (TestRail and Confluence) **Expected Output**: Business and technical context for the feature being tested. -### Task 3: Create Initial Test Plan Document +### Task A3: Create initial test plan document (integrated) **Actions**: 1. Create `agents/plans/aqa-.md` file with: + - **AQA execution mode**: `integrated` - Test case reference (TestRail ID and link) - Feature name and description - Test goal @@ -77,6 +123,7 @@ Gather all required information from external sources (TestRail and Confluence) # AQA Test Plan - **Created**: [DateTime] +**AQA execution mode**: integrated **TestRail Case**: [ID/URL] **Feature**: [Feature Name] **Status**: Phase 1 Complete @@ -134,14 +181,112 @@ Gather all required information from external sources (TestRail and Confluence) [To be filled in Phase 6] ``` +--- + +### Path B — Minimal-input / agent-led + +**Prerequisite:** Task 0 already set execution mode to **minimal-input** (step 1 keyword match or step 3 user reply). Do not enter Path B because TestRail was missing or files were attached—only because the user chose **minimal-input** in text. + +### Task B1: Offer agent-led implementation and required artifacts + +**Actions**: +1. Tell the user clearly that the agent **can** implement the test **without** TestRail/Confluence **if** they provide enough grounded detail — but the agent **must not** invent pages, selectors, or flows. +2. List what you still need at minimum (see Task B2 checklist). **WAIT** until the user either supplies items or explicitly defers an item to Phase 2 (then note "open in Phase 2" in the plan). + +**Expected Output**: User understands the tradeoff and what to supply (or what is deferred to Phase 2 with explicit consent). + +### Task B2: Collect minimal-input package + +**Actions**: +1. Capture the following in the test plan (fill or mark *deferred to Phase 2* with user agreement): + - **Scenario / goal**: What behavior the automated test must prove (plain language). + - **Entry / URLs**: Starting URL(s) or navigation path to reach the UI under test. + - **UI grounding** (at least one required before ending Phase 1 unless user explicitly defers to Phase 2): e.g. saved page source or DOM snapshot paths, a **selector map** (purpose → locator), pointers to existing Page Objects in the repo, and/or frontend file paths the agent will use in Phase 3. + - **Preconditions and test data**: Accounts, feature flags, seed data — or *unknown — Phase 2*. + - **Optional**: TestRail/Confluence IDs "for traceability only" without MCP — record as references, not as substitute for missing UI grounding. +2. **Do not** mark Phase 1 complete if UI grounding and scenario goal are both missing and the user has not agreed to defer. + +**Expected Output**: Minimal-input checklist documented in the test plan. + +### Task B3: Create initial test plan document (minimal-input) + +**Actions**: +1. Create `agents/plans/aqa-.md` using the template below (adapt section headings if the scenario is small). +2. For TestRail/Confluence fields, use `N/A` or optional reference links — do not pretend MCP data exists. + +**Template**: +```markdown +# AQA Test Plan - + +**Created**: [DateTime] +**AQA execution mode**: minimal-input +**TestRail Case**: [N/A or ID for traceability only] +**Confluence**: [N/A or links for traceability only] +**Feature**: [Feature Name] +**Status**: Phase 1 Complete + +## Minimal-input package + +### Scenario / goal +[What the test must validate] + +### Entry and navigation +- URLs: [...] +- Steps to reach the UI: [...] + +### UI grounding +- Page source / DOM snapshots: [paths or attach instructions] +- Selector map or locators: [...] +- Repo pointers (Page Objects / components): [...] + +### Preconditions and test data +[Or: deferred to Phase 2 — user confirmed] + +### Notes and open questions +- [...] + +## Test Case Information (synthetic) + +### Test Steps (draft) +1. [Step — can be refined in Phase 2] +... + +### Expected Overall Result +[Draft — refined in Phase 2] + +--- +## Phase 2: Requirements Clarification +[To be filled in Phase 2] + +## Phase 3: Code Analysis +[To be filled in Phase 3] + +## Phase 4: Selector Identification +[To be filled in Phase 4] + +## Phase 5: Selector Implementation +[To be filled in Phase 5] + +## Phase 6: Test Implementation +[To be filled in Phase 6] +``` + ## Completion Criteria +**All modes** +- [ ] Task 0 complete: execution mode is **either** named by explicit user keywords (`integrated` / `minimal-input` / spelled equivalents) **or** obtained after the mandatory ask-and-wait; mode recorded in test plan and state +- [ ] Test plan file `agents/plans/aqa-.md` created +- [ ] Test goal clearly understood (or explicitly deferred to Phase 2 with user consent) +- [ ] `agents/aqa-state.md` updated with Phase 1 completion + +**Integrated AQA only** - [ ] TestRail test case retrieved and documented - [ ] Confluence documentation retrieved and documented -- [ ] Test plan file created with all Phase 1 information -- [ ] Test goal clearly understood -- [ ] Expected results documented -- [ ] `agents/aqa-state.md` updated with Phase 1 completion +- [ ] Expected results from sources documented + +**Minimal-input only** +- [ ] Minimal-input checklist captured in the test plan +- [ ] UI grounding present or explicitly deferred to Phase 2 with user approval ## Update State File @@ -150,8 +295,10 @@ After completing Phase 1, update `agents/aqa-state.md`: ```markdown ### Phase 1: Data Collection - Completed: [DateTime] -- TestRail Case: [ID/URL] -- Confluence Pages: [URLs] +- Execution mode: [integrated | minimal-input] +- TestRail Case: [ID/URL or N/A] +- Confluence Pages: [URLs or N/A] +- Minimal-input checklist: [brief summary or N/A] - Test Goal: [Brief description] - Expected Result: [Brief description] - Test Plan File: agents/plans/aqa-.md @@ -168,7 +315,9 @@ ACQUIRE aqa-phase2-md FROM KB ## Important Notes -- **No Assumptions**: If TestRail or Confluence data is incomplete, note it in the test plan -- **Ask Questions**: If user hasn't provided IDs/URLs, ask for them -- **Document Everything**: Capture all details even if they seem minor -- **Cross-Reference**: Ensure TestRail and Confluence information aligns +- **No Assumptions**: If data is incomplete for the chosen mode, note it in the test plan and ask — never guess selectors or flows. +- **Ask Questions**: If user hasn't provided IDs/URLs (integrated) or checklist items (minimal-input), ask for them. +- **Document Everything**: Capture all details even if they seem minor. +- **Integrated path only**: Cross-reference TestRail and Confluence information for alignment. +- **Minimal-input path**: Prefer an explicit **selector map** or page source over vague descriptions; deferrals must be user-visible in the plan. +- **Task 0**: Never infer execution mode from context; only explicit keywords or a user reply after the mandatory question count. diff --git a/plugins/core-copilot-standalone/.github/prompts/aqa-flow-requirements-clarification.prompt.md b/plugins/core-copilot-standalone/.github/prompts/aqa-flow-requirements-clarification.prompt.md index f0d21565..f7fb9a9b 100644 --- a/plugins/core-copilot-standalone/.github/prompts/aqa-flow-requirements-clarification.prompt.md +++ b/plugins/core-copilot-standalone/.github/prompts/aqa-flow-requirements-clarification.prompt.md @@ -13,16 +13,23 @@ Fill gaps in understanding, clarify unknowns, and define explicit assertions bef ## Prerequisites -- Phase 1 completed -- Test plan file created with TestRail and Confluence data -- Initial understanding of test requirements +- Phase 1 completed with a test plan file +- Phase 1 documents either **integrated** sources (TestRail + Confluence as used) **or** **minimal-input** execution mode with the checklist from Phase 1 (scenario, URLs, UI grounding or explicit deferral) +- Initial understanding of test requirements (may be draft in minimal-input mode) ## Phase Tasks +### Minimal-input mode (read first) + +If the test plan shows **`AQA execution mode: minimal-input`**: +- Treat Phase 1's checklist as authoritative for what the user already supplied; **do not** re-demand TestRail/Confluence data unless the user wants it for traceability. +- Still run gap analysis: anything deferred from Phase 1 **must** be resolved or explicitly re-deferred here with user text. +- Assertions must still be explicit and measurable before Phase 3; **never** invent selectors the user did not supply, defer, or point to in repo/page source. + ### Task 1: Review Gathered Information for Gaps **Actions**: -1. Read the test plan file from Phase 1 +1. Read the test plan file from Phase 1 and note **AQA execution mode** (`integrated` vs `minimal-input`). 2. Analyze information for completeness: - Are test steps clear and unambiguous? - Are expected results specific and measurable? @@ -197,3 +204,4 @@ ACQUIRE aqa-phase3-md FROM KB - **Explicit Over Implicit**: Every assertion must be measurable and verifiable - **User Authority**: User has final say on requirements and expected behavior - **Document Everything**: Record all questions and answers for traceability +- **Minimal-input mode**: Questions should close gaps in the Phase 1 checklist (especially UI grounding and test data), not re-litigate integrated AQA unless the user switches mode explicitly diff --git a/plugins/core-copilot-standalone/.github/prompts/aqa-flow-test-implementation.prompt.md b/plugins/core-copilot-standalone/.github/prompts/aqa-flow-test-implementation.prompt.md index a16d294e..e92ab613 100644 --- a/plugins/core-copilot-standalone/.github/prompts/aqa-flow-test-implementation.prompt.md +++ b/plugins/core-copilot-standalone/.github/prompts/aqa-flow-test-implementation.prompt.md @@ -13,13 +13,16 @@ Create automated test following project standards, integrating all Page Objects ## Prerequisites -- All previous phases (1-5) completed -- Test plan fully documented with all details -- Page Objects updated with all required selectors +- All previous phases (1-5) completed **for the chosen execution mode** (see below) +- Test plan fully documented with all details **including** `AQA execution mode` and, if `minimal-input`, the **minimal-input checklist** from Phase 1 (and any Phase 2 resolutions of deferrals) +- Page Objects updated with all required selectors **or** a clear plan from Phases 4–5 that matches user-supplied grounding (minimal-input) - Explicit assertions defined - Code architecture understood -## Phase Tasks +### Execution modes and prerequisites + +- **Integrated AQA**: The bullets above mean phases 1–5 were completed in full with TestRail/Confluence-backed Phase 1 where applicable. +- **Minimal-input / agent-led** (recorded in Phase 1): Phases 1–5 count as satisfied only if the test plan contains the **minimal-input checklist** (scenario, URLs/navigation, UI grounding or approved Phase 2 deferrals) and **explicit assertions from Phase 2**. **Before writing code**, re-read that checklist and Phase 2 assertions; if anything material is still missing, **STOP**, ask the user, and do not invent selectors, pages, or flows. ### Task 1: Review Test Implementation Plan diff --git a/plugins/core-copilot-standalone/.github/prompts/aqa-flow.prompt.md b/plugins/core-copilot-standalone/.github/prompts/aqa-flow.prompt.md index 00f7c725..f7931333 100644 --- a/plugins/core-copilot-standalone/.github/prompts/aqa-flow.prompt.md +++ b/plugins/core-copilot-standalone/.github/prompts/aqa-flow.prompt.md @@ -10,20 +10,23 @@ baseSchema: docs/schemas/workflow.md ## Context -This agent handles end-to-end test automation from requirements gathering to test implementation. It uses TestRail, Confluence, and project documentation to create automated tests following existing architecture and coding standards. +This agent handles end-to-end test automation from requirements gathering to test implementation. It uses TestRail, Confluence, and project documentation when you run **integrated AQA**; in **minimal-input / agent-led** mode the user supplies pages, DOM or selector details, and scenario context instead (see Phase 1). In all modes, tests follow existing architecture and coding standards. ### Critical Requirements - **ONE PHASE AT A TIME**: Read phase file, execute, update state, move to next. - **DO NOT SKIP PHASES**: Each builds on previous. - **NO ASSUMPTIONS**: Never assume selectors, flows, or data. Always ask the user if information is missing. -- **USER INTERACTION**: Wait for user responses when questions are asked or files are requested. Phase 2, 6, 7, and 8 always require user input. Phase 4 requires user input ONLY if frontend code is unavailable or selectors cannot be found +- **MCP ROUTING (B + A)**: Before calling external MCPs in integrated Path A, follow **`mcp-capability-interaction`** using `agents/mcp-capability.yaml` and any user override in the task text. **Guided** = use MCP after optional `agents/user-instructions/mcp-guidance.md`. **Questionnaire** = no MCP calls for that integration; user answers structured questions; record answers in the plan. ACQUIRE `mcp-capability-interaction.md` FROM KB when executing Phase 1 Path A. +- **EXECUTION MODE (Phase 1, first)**: The user must **explicitly** choose **integrated** or **minimal-input** (case-insensitive), or use the spelled labels **Integrated AQA** or **minimal-input / agent-led**, in **their** message before you record a mode. If none of those appear, **ask once** with both options, **STOP**, and **WAIT**—**do not infer** mode from missing TestRail, attachments, sandbox, or “no MCP.” **Do not** write `agents/plans/aqa-*.md` or set `**AQA execution mode**` until that keyword match or the user’s reply to the question. **Forbidden:** assigning `minimal-input` because TestRail was omitted or the project looks like a demo. After the user answers, document the mode in the Phase 1 test plan and `agents/aqa-state.md`. Do not silently invent missing integrations or missing UI grounding. User customizations elsewhere do **not** waive this rule when mode was not explicitly named. +- **USER INTERACTION**: Wait for user responses when questions are asked or files are requested. **Phase 1** requires user input until execution mode is explicit or the user answers the one-time mode question (see **EXECUTION MODE**). Phase 2, 6, 7, and 8 always require user input. Phase 4 requires user input ONLY if frontend code is unavailable or selectors cannot be found - **STATE TRACKING**: Update `agents/aqa-state.md` after each phase. - **MUST** use todo tasks for tracking progress. - Prioritize ACCURACY over SPEED! ### User Customizations +- The **execution mode** rule cannot be waived by “only run some phases” unless the user message already contains the mode keywords or the user answers the one-time mode question. - If user did not specify any preferences perform all steps except optional. - User CAN customize and ask only for specific phases OR phases could have been done already OR towards specific goal OR for specific case, in this case LISTEN and ADOPT to the user. @@ -31,9 +34,10 @@ This agent handles end-to-end test automation from requirements gathering to tes **Phase 1: Data Collection** [aqa-flow-data-collection.md] 1. ACQUIRE aqa-flow-data-collection.md FROM KB -2. Execute phase instructions -3. Update `agents/aqa-state.md` -4. Validate gathered data +2. ACQUIRE mcp-capability-interaction.md FROM KB (resolve MCP guided vs questionnaire before Path A MCP calls) +3. Execute phase instructions (**start with execution mode confirmation** — integrated vs minimal-input) +4. Update `agents/aqa-state.md` +5. Validate gathered data (checklist for chosen mode is complete before Phase 2) **Phase 2: Requirements Clarification** [aqa-flow-requirements-clarification.md] ⭐ **USER INTERACTION REQUIRED** 1. ACQUIRE aqa-flow-requirements-clarification.md FROM KB @@ -91,7 +95,8 @@ Create/update `agents/aqa-state.md` after each phase: **Last Updated**: [DateTime] **Current Phase**: [1-8 or COMPLETE] -**TestRail Case**: [Test Case ID/URL] +**AQA execution mode**: [integrated | minimal-input] +**TestRail Case**: [Test Case ID/URL or N/A in minimal-input] **Feature**: [Feature Name] ## Phase Completion Status @@ -109,8 +114,10 @@ Create/update `agents/aqa-state.md` after each phase: ### Phase 1: Data Collection - Completed: [DateTime] -- TestRail Case: [ID/URL] -- Confluence Pages: [URLs] +- Execution mode: [integrated | minimal-input] +- TestRail Case: [ID/URL or N/A] +- Confluence Pages: [URLs or N/A] +- Minimal-input checklist: [summary or N/A — see Phase 1 test plan] - Test Goal: [Brief description] - Expected Result: [Brief description] @@ -158,6 +165,7 @@ Create/update `agents/aqa-state.md` after each phase: ## Important Notes - **Sequential Execution**: Phases build on each other, must execute in order. +- **Execution modes**: **Integrated AQA** uses TestRail and Confluence in Phase 1 when those tools are in scope. **Minimal-input / agent-led** uses the Phase 1 checklist (pages, DOM or selectors, scenario context); later phases still run in order, but Phase 1 documents what substitutes for external case management. Without an **explicit** user mode label (`integrated` or `minimal-input` / spelled equivalents), Phase 1 must obtain that label by asking—never infer from context alone. - **No Assumptions Rule**: Always ask user when information is missing - never guess selectors, flows, or test data. - **Architecture First**: Always analyze existing code structure before implementing new tests. - **Reuse Over Creation**: Prefer adding to existing files and using existing Page Objects over creating new ones. diff --git a/plugins/core-copilot-standalone/.github/prompts/mcp-capability-interaction.prompt.md b/plugins/core-copilot-standalone/.github/prompts/mcp-capability-interaction.prompt.md new file mode 100644 index 00000000..4f4fae4d --- /dev/null +++ b/plugins/core-copilot-standalone/.github/prompts/mcp-capability-interaction.prompt.md @@ -0,0 +1,64 @@ +--- +name: mcp-capability-interaction +description: Routes guided vs questionnaire MCP usage from workspace YAML and user text, records interaction source, and defines where capability files live. Placeholder binding uses sibling workflows. +alwaysApply: false +baseSchema: docs/schemas/phase.md +--- + +# MCP capability interaction (shared) + +Use when a workflow **may** call **remote MCPs** (Jira, Confluence, TestRail, Atlassian). It does **not** bind **`{integration-action:…}`** (see **`mcp-tool-resolution.md`**) or **`{agent-action:…}`** (see **`agent-action-resolution.md`**). **AQA** Path A and **TestGen** Phases 1 and 6 load this for **routing** (guided vs questionnaire) only. + +## B — `agents/mcp-capability.yaml` (workspace root) + +Read before the first MCP call. Keys optional; unknown keys ignored. + +```yaml +mcp: + mode: capable # required if file exists: capable | absent + jira: true + confluence: true + testrail: true + atlassian_confluence: true +``` + +- **`confluence` vs `atlassian_confluence`:** **Not aliases** — both refer to **Confluence** MCP on/off for **guided vs questionnaire**, but **different workflows read different keys** (TestGen uses `confluence`; AQA data collection uses `atlassian_confluence`). If you use **both** families, set **both** to the same boolean. If you use **one** family, you may omit the key the other workflow reads. Template: `instructions/r2/core/templates/mcp-capability.example.yaml`. + +- **`mode: capable`** — MCP allowed. Per integration: **`true`** or key omitted → **guided**. **`false`** → **questionnaire** (no MCP; user text). Read **`agents/user-instructions/mcp-guidance.md`** first when it exists. Do not re-ask values already in that file unless missing. +- **`mode: absent`** — Do not invoke MCP. Parent workflow runs questionnaire (**STOP**, **WAIT**), merges into plan / `raw-data.md` / agreed artifact. Never fabricate MCP output. +- **File missing** — Apply **A** if the user message overrides; else **one** question whose **scope** (which integrations) is defined by the **parent workflow**, not this fragment. **Example — TestGen Phase 1:** when `agents/mcp-capability.yaml` is absent, **Step 2b** in `testgen-flow-data-collection.md` asks once for **Jira and Confluence together**; **TestRail** is resolved again in **TestGen Phase 6**. **No** → questionnaire for the integrations that question covers. **Yes** → capable for that run; recommend adding YAML. + +## A — user message (wins over file) + +Record in state/plan. **Absent:** `MCP absent`, `no MCP`, `without MCP`, `questionnaire only`, `paste-only`. **Capable:** `MCP capable`, `use MCP`, `MCP enabled`, `pull from Jira` when clearly a live pull. User wins on conflict. + +## Capable mode without guidance + +Use task text and YAML only; questionnaire **only** for missing fields. Do not invent secrets. + +## Questionnaire (`absent` or integration `false`) + +Parent supplies **numbered** questions for fields MCP would return. **STOP**; **WAIT**. Merge into the same artifacts as the MCP path. Label **user-provided (no MCP)**. + +## Recording + +When routing is resolved, write into the phase plan or state file: either **one** line `MCP interaction: guided | questionnaire (source: agents/mcp-capability.yaml | user override | default question)` when a single scope applies, or **one line per integration** (or per parent-defined field such as TestGen `Jira source` / `Confluence source` / Phase 6 TestRail export), each with the same `(source: ...)` options. + +**Before leaving this fragment:** Re-read the plan or state lines you wrote for MCP routing and confirm: **(a)** each line lists exactly one source among `agents/mcp-capability.yaml`, `user override`, and `default question`; **(b)** using one aggregate line vs one line per integration (or per parent-defined field such as TestGen `Jira source` / `Confluence source`) matches what the **parent workflow** requires. Correct the artifact if a check fails. + +## Path conventions (Rosetta vs target workspace) + +Use whenever a workflow mentions both Rosetta artifacts and workspace files. + +| Location | Path | Notes | +|----------|------|--------| +| **Rosetta repo / distribution only** | `instructions/r2/core/templates/mcp-capability.example.yaml` | Template for **authoring** `mcp-capability.yaml`. Not expected inside an arbitrary customer application checkout unless copied from Rosetta. | +| **Target workspace** | `agents/mcp-capability.yaml` | Optional capability file; **expected at workspace root** only. | +| **Target workspace** | `agents/user-instructions/mcp-guidance.md` | Optional per-workspace MCP hints. | + +If **`agents/mcp-capability.yaml`** is missing at workspace root, treat capability config as **missing** for routing: follow the parent workflow’s questionnaire / default-question path and record **MCP interaction source** accordingly (do not search the customer repo for Rosetta template paths). + +## Related workflows (placeholder binding) + +- **`mcp-tool-resolution.md`** — Map **`{integration-action:…}`** to MCP tool ids when **guided**. +- **`agent-action-resolution.md`** — Map **`{agent-action:…}`** to host read / write / patch tools. diff --git a/plugins/core-copilot-standalone/.github/prompts/mcp-tool-resolution.prompt.md b/plugins/core-copilot-standalone/.github/prompts/mcp-tool-resolution.prompt.md new file mode 100644 index 00000000..8695ba68 --- /dev/null +++ b/plugins/core-copilot-standalone/.github/prompts/mcp-tool-resolution.prompt.md @@ -0,0 +1,34 @@ +--- +name: mcp-tool-resolution +description: Binds `{integration-action:…}` tokens to MCP tool identifiers in guided runs using a single-pass scan of the session tool list. +alwaysApply: false +baseSchema: docs/schemas/phase.md +--- + +# `{integration-action:…}` resolution (MCP tools) + +Use when a parent workflow is **guided** for an integration and must turn a **`{integration-action:…}`** token into a concrete MCP tool call. **ACQUIRE only when needed** (not for questionnaire-only paths). Routing (guided vs questionnaire) is defined in **`mcp-capability-interaction.md`**. + +Parent workflows define token spellings and parameter blocks. **Questionnaire** integrations must **not** use this file to call MCP. + +## Deterministic tool selection (single pass over MCP list) + +Let `T` = the list of MCP tool names (or ids) available in **this** session. Perform **exactly one** linear scan over `T` (no second pass, no “try again” loops). + +1. **Required substrings** — From the token after `integration-action:`, take segments split by `-` (e.g. `jira-get-issue` → `jira`, `get`, `issue`). Build candidate set **C** = tools in `T` whose identifier **contains every remaining segment** (case-insensitive). If **C** is empty, drop segments in this order until **C** is non-empty or one segment remains: drop **`get`**, then **`search`**, then **`list`** or **`pages`**, then repeatedly drop the **rightmost** remaining segment. +2. **If C has exactly one tool** — use it. +3. **If C has more than one tool** — Prefer tools explicitly allowed or named in **`agents/mcp-capability.yaml`** and **`agents/user-instructions/mcp-guidance.md`** for that integration. If still more than one, **STOP** and ask **one** disambiguation question using the **format below** (no guess). +4. **If C is empty after step 1 reductions** — **Do not invent** a tool. Record resolution failure in state, then follow **Questionnaire** rules for that integration in **`mcp-capability-interaction.md`** (**STOP**, **WAIT**), then continue per the parent workflow’s routing table. + +Map **Parameters** from the phase `Action` blocks to the chosen tool’s schema (adapt snake_case vs camelCase per tool docs). + +## Disambiguation question format + +When step **3** leaves multiple candidates, use exactly this shape: + +1. One line stating the token and that multiple MCP tools matched. +2. Numbered lines **`1`** through **`N`**: each line is **one verbatim tool id** from the remaining candidate set (same spelling as in `T`). +3. Line **`N+1`**: `Cancel — use questionnaire for this integration instead`. +4. **Parse the reply:** accept only an integer **k**. If **k = N+1**, do not call MCP for that integration; follow the parent’s questionnaire path. If **1 ≤ k ≤ N**, select the **k**-th listed tool in order. Any other reply → ask again once with the same list. + +Vendor tool names in examples elsewhere are **illustrations**; **`T`** is authoritative. diff --git a/plugins/core-copilot-standalone/.github/prompts/testgen-flow-data-collection.prompt.md b/plugins/core-copilot-standalone/.github/prompts/testgen-flow-data-collection.prompt.md index 4696f7d5..3e14221a 100644 --- a/plugins/core-copilot-standalone/.github/prompts/testgen-flow-data-collection.prompt.md +++ b/plugins/core-copilot-standalone/.github/prompts/testgen-flow-data-collection.prompt.md @@ -9,9 +9,12 @@ baseSchema: docs/schemas/phase.md ## Prerequisites -- MUST be starting new test generation flow -- User provided Jira ticket key or URL -- Jira MCP configured and accessible +- MUST be starting new test generation flow. +- User provided Jira ticket key or URL. +- **Acquire workflows:** ACQUIRE **`mcp-capability-interaction.md`** FROM KB (MCP guided vs questionnaire; always for Phase 1). Before the first guided **`{integration-action:*}`** call in Steps 3–4, ACQUIRE **`mcp-tool-resolution.md`**. Before **Persist** blocks using **`{agent-action:*}`** in Steps 5–6, ACQUIRE **`agent-action-resolution.md`**. If an alias fails, use canonical paths under **`instructions/r2/core/workflows/`** with the same basename. +- **Step 2b only:** Run the numbered **Step 2b checklist** in this file (not the whole fragment for later steps unless needed). +- **Before MCP calls:** Do **not** call Jira/Confluence MCPs until Step **2b** completes. +- **URL placeholders:** `{jira-host}` and `{confluence-host}` in examples mean the customer’s real web hostnames (including subdomain), not fixed vendor domains. ## Objective @@ -19,35 +22,29 @@ Extract all relevant data from Jira ticket and related Confluence documentation ## Requirements +**Token resolution (canonical):** **`{integration-action:*}`** → **`mcp-tool-resolution.md`**. **`{agent-action:*}`** → **`agent-action-resolution.md`**. MCP guided vs questionnaire and recording → **`mcp-capability-interaction.md`** (ACQUIRE per **Prerequisites**). Later steps say **see Token resolution** instead of repeating this. + ### Step 1: Parse Initial User Input **Extract from user's initial prompt**: -1. **Jira ticket**: Key or URL (REQUIRED) -2. **Confluence URLs**: List of URLs (REQUIRED) +1. **Jira ticket**: Key or URL (REQUIRED). +2. **Confluence URLs** (optional): Parse and store for later steps. How they affect routing is defined **only** in the **Step 2b** matrix (after `Jira source` / `Confluence source` exist). **Supported formats**: ``` "Analyze requirements for PROJ-123" -"Analyze requirements for PROJ-123 with Confluence: https://confluence.com/display/PROJ/Page" +"Analyze requirements for PROJ-123 with Confluence: https://{confluence-host}/display/PROJ/Page" "Analyze PROJ-123, Confluence pages: URL1, URL2, URL3" -"PROJ-123 + https://confluence.com/display/PROJ/Auth" +"PROJ-123 + https://{confluence-host}/display/PROJ/Auth" ``` **Parse Confluence URLs**: - Extract from patterns: "with Confluence", "Confluence:", "Confluence docs:", "Confluence pages:" - Accept comma-separated or line-separated URLs - URLs may be: - - Display format: `https://confluence.company.com/display/SPACE/Page+Title` - - Direct format: `https://confluence.company.com/pages/viewpage.action?pageId=123456` - - Short format: `https://confluence.company.com/x/AbCdEf` - -**If Confluence URLs provided**: -- Extract page IDs from URLs -- Skip automatic search (Step 3) -- Go directly to retrieving specified pages - -**If no Confluence URLs provided**: -- Proceed with automatic search (Step 3) + - Display format: `https://{confluence-host}/display/SPACE/Page+Title` + - Direct format: `https://{confluence-host}/pages/viewpage.action?pageId=123456` + - Short format: `https://{confluence-host}/x/AbCdEf` ### Step 2: Setup Output Directory @@ -57,48 +54,122 @@ agents/testgen/{TICKET-KEY}/ └── testgen-state.md (initialize) ``` +### Step 2b: Resolve MCP routing (user message vs capability file) + +**Precedence:** **A first** — fragment **section A** (user / task text **wins** over the file). **B second** — fragment **section B** (`agents/mcp-capability.yaml`). Full rules: **`mcp-capability-interaction.md`** (ACQUIRE per **Prerequisites**). + +**Checklist (execute in order before any Jira/Confluence MCP call in Phase 1):** + +1. Confirm **Prerequisites** ACQUIRE list for this phase is loaded (**`mcp-capability-interaction.md`**, and before guided MCP / Persist: **`mcp-tool-resolution.md`**, **`agent-action-resolution.md`** as listed there). +2. Read **`agents/mcp-capability.yaml`** at workspace root if present; if absent, treat capability config as **missing** (see **`mcp-capability-interaction.md`** Path conventions). +3. Apply fragment **§ A — user message** (task text wins over file). +4. If YAML is **missing** and the task text does not override MCP usage: ask **one** question — “Use live MCP for Jira and Confluence in this workspace?” — **STOP**, **WAIT**. **No** → both integrations **questionnaire** for this run. **Yes** → both **guided**; recommend adding `agents/mcp-capability.yaml`. +5. Derive **`Jira source`** and **`Confluence source`** using fragment **§ B** (`mode`, `mcp.jira`, `mcp.confluence`, omitted-key defaults). +6. Record in `agents/testgen/{TICKET-KEY}/testgen-state.md` under Phase 1: `Jira source`, `Confluence source`, and **MCP interaction source:** `agents/mcp-capability.yaml` **or** `user override` **or** `default question`. +7. If **`agents/user-instructions/mcp-guidance.md`** exists and at least one source is **guided**, read it before the first Jira/Confluence MCP call in Phase 1. + +#### Routing matrix (after Step 2b) + +Use `Jira source` and `Confluence source` from state. **Step labels:** **3Q** = Jira questionnaire (no Jira MCP); **3A** = Jira guided MCP; **4Q** = Confluence questionnaire; **4A** = Confluence guided MCP. **“URLs in Step 1?”** = Confluence URL(s) were parsed from the **initial** user prompt in Step 1. + +| Jira source | Confluence source | URLs in Step 1? | Execute | +|-------------|-------------------|-----------------|---------| +| guided | guided | yes | **3A** → **4A** Option A | +| guided | guided | no | **3A** → **4A** Option B | +| guided | questionnaire | — | **3A** → **4Q** | +| questionnaire | guided | yes | **3Q** → **4A** Option A | +| questionnaire | guided | no | **3Q** → **4A** Option B | +| questionnaire | questionnaire | — | **3Q** → **4Q** | + +#### `{integration-action:…}` intent reference (Phase 1) + +Use with **Token resolution** and **`mcp-tool-resolution.md`** when mapping tokens to host MCP tools. + +| Token | Intent | +|-------|--------| +| `{integration-action:jira-get-issue}` | Load one issue by key with field list, expand flags, comment limit | +| `{integration-action:jira-search-fields}` | Discover Jira field IDs / API names (custom fields) | +| `{integration-action:confluence-get-page}` | Load one Confluence page by id (markdown/metadata options) | +| `{integration-action:confluence-search-pages}` | Search Confluence pages (query + limit) | +| `{integration-action:confluence-list-child-pages}` | List child pages for a parent (content options) | + ### Step 3: Extract Jira Ticket Data -**Use**: Jira and/or Confluence MCPs respectively, snippets below will contain example pseudo-function calls for better understanding `mcp_Jira_MCP_jira_get_issue`, `mcp_Jira_MCP_confluence_get_page`, `mcp_Jira_MCP_confluence_search`, `mcp_Jira_MCP_confluence_get_page_children`. +Pick **exactly one** subsection below using the **Step 2b** matrix (`Jira source` column). **Do not** read, summarize, or execute the other subsection (**progressive disclosure**). + +#### Step 3Q: Jira questionnaire (no MCP) + +**Jira MCP:** **off** — no **`{integration-action:jira-*}`** calls; user-provided content only. -**Extract ticket key** from user input: -- Format: "PROJ-123" or URL "https://jira.company.com/browse/PROJ-123" -- Parse key from URL if needed +1. Ask **numbered** questions so you can build the Jira section of `raw-data.md`: ticket summary, description (or ask user to paste export), status, priority, labels, components, key links, and any acceptance criteria they rely on. +2. **STOP** and **WAIT** for answers. +3. Write `agents/testgen/{TICKET-KEY}/raw-data.md` (create if needed) with `## Jira (user-provided, MCP absent)` containing merged answers. For any field the user could not supply or was unsure about, add an explicit **`unknown`**, **`unverified`**, or short **why missing** note next to that item (do not imply MCP-grade certainty). Then continue per the **Step 2b** matrix to Step 4. -**Retrieve issue** with comprehensive fields: -```python -mcp_Jira_MCP_jira_get_issue( - issue_key="PROJ-123", - fields="summary,description,status,issuetype,assignee,priority,reporter,labels,components,created,updated", - expand="renderedFields", - comment_limit=10 -) +#### Step 3A: Guided Jira (MCP) + +1. **Extract ticket key** from user input: + - Format: "PROJ-123" or URL "https://{jira-host}/browse/PROJ-123" + - Parse key from URL if needed + +2. **Retrieve issue** — see **Token resolution**, **`mcp-tool-resolution.md`**, and **Intent reference** above for **`{integration-action:jira-get-issue}`**. Then call the resolved tool; logical parameters: +```text +Action: {integration-action:jira-get-issue} +Parameters: + issue_key: PROJ-123 + fields: summary,description,status,issuetype,assignee,priority,reporter,labels,components,created,updated + expand: renderedFields + comment_limit: 10 ``` -**Capture**: -- Summary, description (both raw and rendered) -- Issue type, status, priority -- Labels, components -- Assignee, reporter -- Comments (up to 10 recent) -- Created/updated dates -- Custom fields if present (epic link, story points, etc.) +**Example (mapping only):** On a common Atlassian host the resolved tool is `mcp__atlassian__jira_get_issue`; rename keys from the block above as required (e.g. `issue_key` → `issueKey`, `comment_limit` → `commentLimit`). Other tokens: see **Token resolution**. + +3. **Capture** (for Step 5 / `raw-data.md`): + - Summary, description (both raw and rendered) + - Issue type, status, priority + - Labels, components + - Assignee, reporter + - Comments (up to 10 recent) + - Created/updated dates + - Custom fields if present (epic link, story points, etc.) + +#### Step 3A.4 — Jira field schema (optional) + +Run **only** when substep **3** leaves custom-field **API names or ids** ambiguous for `raw-data.md`. + +1. **Invoke** **`{integration-action:jira-search-fields}`** **once** (see **Token resolution**). +2. **Output:** merge the returned field metadata into working notes and into **`### Custom Fields`** (and related) when executing **Step 5** (`raw-data.md`). If substep 3 was sufficient, **skip** this entire **3A.4** subsection. ### Step 4: Get Confluence Documentation -**Decision Point**: Did user provide Confluence URLs in initial prompt? +Pick **exactly one** subsection below using the **Step 2b** matrix (`Confluence source` column; for **4A**, use the **URLs in Step 1?** column for Option A vs B). **Do not** read, summarize, or execute the other subsection (**progressive disclosure**). + +#### Step 4Q: Confluence questionnaire (no MCP) + +**Confluence MCP:** **off** — no **`{integration-action:confluence-*}`** calls; user-provided content only. -#### Option A: User Provided Confluence URLs +1. Ask **numbered** questions: paste each page body (or export), or provide URLs plus authorized excerpts, parent/child relationships if relevant. +2. **STOP** and **WAIT**. +3. Append to `agents/testgen/{TICKET-KEY}/raw-data.md` under `## Confluence (user-provided, MCP absent)` with merged answers. Mark gaps, pasted excerpts of unknown completeness, or user-stated uncertainty the same way as Step 3Q (**`unknown` / `unverified` / reason**). If Jira questionnaire already created the file, append; otherwise create the file with this section. +4. Continue to **Step 5** so `raw-data.md` matches the full template (headings, metadata). Then run **Validation**. + +#### Step 4A: Guided Confluence (MCP) + +**Option A vs B:** From the **Step 2b** matrix (**URLs in Step 1?** column). + +Resolve each **`{integration-action:confluence-*}`** (see **Token resolution**; e.g. `confluence-get-page` → `mcp__atlassian__confluence_get_page` with `pageId` / `convertToMarkdown` when the host uses that idiom). + +##### Option A: User-provided Confluence URLs -**If URLs provided in initial prompt**: 1. Parse page IDs from URLs 2. For each URL, extract: - Page ID from URL parameters (pageId=123456) - Or use space + title from display URL -3. Retrieve pages directly using `mcp_Jira_MCP_confluence_get_page()` +3. Retrieve pages using **`{integration-action:confluence-get-page}`** 4. Check each page for child pages (REQUIRED) 5. Skip automatic search +**Illustrative resolved call (Atlassian-style host, not mandatory):** `mcp__atlassian__confluence_get_page` with arguments such as `{ "pageId": "123456", "convertToMarkdown": true }` (exact keys per host mapping). + **Tell user**: ``` ✅ Using provided Confluence pages: @@ -107,9 +178,9 @@ mcp_Jira_MCP_jira_get_issue( 🔍 Checking for child pages... ``` -#### Option B: No URLs Provided - Auto-Search +##### Option B: No URLs — auto-search -**Use**: `mcp_Jira_MCP_confluence_search()` +**Use**: **`{integration-action:confluence-search-pages}`** **Extract search terms** from Jira ticket: - Project key (from ticket key) @@ -123,35 +194,37 @@ type=page AND space={PROJECT_KEY} AND (text ~ "{term1}" OR text ~ "{term2}") ``` **Search Confluence**: -```python -mcp_Jira_MCP_confluence_search( - query=cql_query, - limit=10 -) +```text +Action: {integration-action:confluence-search-pages} +Parameters: + query: + limit: 10 ``` +**Illustrative resolved call (Atlassian-style host, not mandatory):** `mcp__atlassian__confluence_search_pages` (or host-equivalent) with CQL in the query argument, e.g. `type=page AND space=PROJ AND (text ~ "auth" OR text ~ "login")`. + **Rank results** by relevance: - Title matches ticket terms - Labels match ticket labels - Content matches key terms **Get top 3-5 pages**: -```python -mcp_Jira_MCP_confluence_get_page( - page_id=page_id, - convert_to_markdown=True, - include_metadata=True -) +```text +Action: {integration-action:confluence-get-page} +Parameters: + page_id: + convert_to_markdown: true + include_metadata: true ``` **IMPORTANT: Check for child pages** (nested documents often missed by search): -```python -mcp_Jira_MCP_confluence_get_page_children( - parent_id=page_id, - include_content=True, - convert_to_markdown=True, - limit=10 -) +```text +Action: {integration-action:confluence-list-child-pages} +Parameters: + parent_id: + include_content: true + convert_to_markdown: true + limit: 10 ``` For each parent page found: @@ -186,6 +259,19 @@ For each parent page found: **File**: `agents/testgen/{TICKET-KEY}/raw-data.md` +**Persist** — see **Token resolution** for **`{agent-action:write-file}`** / **`{agent-action:patch-file}`**: + +```text +Action: {agent-action:write-file} +Parameters: + path: agents/testgen/{TICKET-KEY}/raw-data.md + content: +``` + +**Illustrative host mapping (Cursor-style agent, not mandatory):** tool `Write` with `file_path` / `contents` (or host-equivalent keys after resolution). + +If the file already exists and you are only appending, you may use **`{agent-action:patch-file}`** instead (see **Token resolution**). + **Format**: ```markdown # Raw Data - [TICKET-KEY] @@ -193,6 +279,7 @@ For each parent page found: **Extracted**: [DateTime] **Phase**: 1 - Data Collection **Confluence Source**: [User-provided URLs / Auto-search / User-provided after search / Skipped] +**Confidence / Unknowns**: [If either source was questionnaire: list unverified, missing, or user-uncertain items; else `MCP-backed` or `n/a`] --- @@ -279,12 +366,22 @@ For each parent page found: - **Total Content Size**: [Approximate word count] - **Search Terms Used**: [List] - **Notes**: [Any issues during extraction] +- **Confidence / Unknowns**: [Same as header line; required when any questionnaire path ran] ``` ### Step 6: Update State File **File**: `agents/testgen/{TICKET-KEY}/testgen-state.md` +**Persist** — same as Step 5 (**see Token resolution**): + +```text +Action: {agent-action:write-file} +Parameters: + path: agents/testgen/{TICKET-KEY}/testgen-state.md + content: +``` + **Create initial state**: ```markdown # Test Generation State - [TICKET-KEY] @@ -329,18 +426,26 @@ For each parent page found: Before completing Phase 1, verify: - ✅ `agents/testgen/{TICKET-KEY}/` directory exists -- ✅ `raw-data.md` created with Jira section populated +- ✅ Phase 1 block in `testgen-state.md` contains lines **`Jira source`**, **`Confluence source`**, and **`MCP interaction source`** +- ✅ If **`Jira source` is `questionnaire`**: `raw-data.md` contains `## Jira (user-provided, MCP absent)` +- ✅ If **`Confluence source` is `questionnaire`**: `raw-data.md` contains `## Confluence (user-provided, MCP absent)` +- ✅ If any **questionnaire** path ran: `raw-data.md` **Data Collection Summary** includes **`Confidence / Unknowns`** +- ✅ `raw-data.md` created with Jira section populated (MCP or questionnaire path) - ✅ Confluence section has at least 1 page OR user confirmed skip - ✅ `testgen-state.md` created with Phase 1 marked complete - ✅ All key Jira fields captured (summary, description, status, priority) ## Tools Used -- `mcp_Jira_MCP_jira_get_issue()` - Jira ticket extraction -- `mcp_Jira_MCP_confluence_search()` - Confluence page search -- `mcp_Jira_MCP_confluence_get_page()` - Confluence page content retrieval -- `mcp_Jira_MCP_confluence_get_page_children()` - Confluence child page discovery -- `write()` - File creation +| Token / use | Where | +|---------------|--------| +| `{integration-action:jira-get-issue}` | Step 3A.2 | +| `{integration-action:jira-search-fields}` | Step 3A.4 (conditional) | +| `{integration-action:confluence-search-pages}` | Step 4A Option B | +| `{integration-action:confluence-get-page}` | Step 4A Options A / B | +| `{integration-action:confluence-list-child-pages}` | Step 4A Option B | +| `{agent-action:write-file}` | Steps 5–6 (**Persist** blocks) | +| `{agent-action:patch-file}` | Steps 5–6 (optional append / edit) | ## Common Issues @@ -354,10 +459,10 @@ Before completing Phase 1, verify: **Solution**: Include first 5000 words, note truncation in raw-data.md **Issue**: Custom fields not recognized -**Solution**: Use `mcp_Jira_MCP_jira_search_fields()` to discover field names +**Solution**: Run **Step 3A.4** — **`{integration-action:jira-search-fields}`** (or equivalent) to discover field names **Issue**: Confluence search finds parent but misses child pages -**Solution**: Always check for child pages using `confluence_get_page_children()` for each found page +**Solution**: Always check for child pages using **`{integration-action:confluence-list-child-pages}`** for each found page **Issue**: User provided invalid Confluence URL **Solution**: Try to parse page ID, if fails ask user for correct URL or page ID @@ -378,8 +483,6 @@ After Phase 1 completion: - Confluence search may need tuning based on organization's Confluence structure - Some Jira instances have custom fields - capture all available - Confluence pages may be in different spaces - search broadly initially -- **User can provide Confluence URLs in initial prompt** - this skips auto-search -- If user provides specific page URLs/IDs, use those directly instead of search - **CRITICAL**: Always check for child pages - nested documentation often contains the most relevant details - Example: "Job Post" parent may have children "Create a Job Post", "Edit a Job Post", etc. - Retrieve up to 10 child pages per parent, prioritize by relevance to ticket diff --git a/plugins/core-copilot-standalone/.github/prompts/testgen-flow-test-case-export.prompt.md b/plugins/core-copilot-standalone/.github/prompts/testgen-flow-test-case-export.prompt.md index e2cce06c..7e9164fc 100644 --- a/plugins/core-copilot-standalone/.github/prompts/testgen-flow-test-case-export.prompt.md +++ b/plugins/core-copilot-standalone/.github/prompts/testgen-flow-test-case-export.prompt.md @@ -12,26 +12,70 @@ baseSchema: docs/schemas/phase.md - Phase 5 MUST be complete - `agents/testgen/{TICKET-KEY}/test-scenarios.md` exists with test cases - User has reviewed and edited test cases -- TestRail MCP configured and accessible -- TestRail project_id and suite_id known +- **TestRail export path (B + A):** Either **guided** (TestRail MCP allowed per `mcp-capability-interaction.md` and `agents/mcp-capability.yaml`) or **questionnaire** (no TestRail MCP; user supplies IDs and performs or confirms manual import). Optional **`agents/user-instructions/mcp-guidance.md`** when guided. +- **Placeholder binding:** Before first guided **`{integration-action:*}`** call, ACQUIRE **`mcp-tool-resolution.md`**. For **`{agent-action:*}`** file steps, ACQUIRE **`agent-action-resolution.md`**. Canonical: **`instructions/r2/core/workflows/.md`** if KB alias fails. +- For **guided** runs: TestRail MCP configured and accessible; `project_id` and `suite_id` known (from guidance, state, or user). ## Objective -Export test cases from test-scenarios.md to TestRail using MCP, creating a new section (folder) for the Jira ticket and adding all test cases. -If MCP is not available, ask user how to proceed and provide possible options. +Export test cases from `test-scenarios.md` to TestRail. + +- **Guided:** Use TestRail MCP to verify the project, resolve `section_id`, and create cases. +- **Questionnaire:** Do **not** run any **`{integration-action:testrail-*}`** call. Collect `project_id`, `suite_id`, and `section_id` from the user, write a **manual export pack** (copy-paste / CSV-friendly), and record completion when the user confirms import or pastes back TestRail case IDs. + +If the interaction mode is unclear, resolve it in **Step 0** before any MCP call. ## TestRail Configuration **Default Configuration**: detect using current ticket and user profile. **Section Name**: [TICKET-KEY] (e.g., "PROJ-123") +**Web URL placeholders:** `{testrail-suite-view-url}`, `{testrail-case-view-url}`, and `{testrail-suite-cases-grouped-url}` stand in for full TestRail browser links (`https://` + host + `index.php?` paths + ids). Compose from `{TestRail instance host}`, `suite_id`, case id, and section grouping as in Step 1Q or `mcp-guidance.md`. + +### Guided TestRail and workspace file actions (placeholders) + +| Token | Intent | +|-------|--------| +| `{integration-action:testrail-get-project}` | Verify TestRail project access by `project_id` | +| `{integration-action:testrail-get-cases}` | List cases in a project/suite (optional dedup before add) | +| `{integration-action:testrail-add-case}` | Create a test case in a `section_id` with title, priority, type, refs, separated steps | +| `{agent-action:read-file}` | Read a workspace file (host-specific file tool) | +| `{agent-action:write-file}` | Create or overwrite a workspace file | +| `{agent-action:patch-file}` | Apply targeted edits to a workspace file (includes search-and-replace style edits) | + +Resolve **`{integration-action:…}`** using **`mcp-tool-resolution.md`** (guided only). Resolve **`{agent-action:…}`** using **`agent-action-resolution.md`**. ## Requirements -### Step 1: Verify TestRail Connection +### Step 0: Resolve TestRail MCP interaction (B + A) -**Test connection** using: -```python -mcp_testrail_get_project(project_id=69) +1. ACQUIRE **`mcp-capability-interaction.md`** FROM KB. ACQUIRE **`mcp-tool-resolution.md`** before any **`{integration-action:*}`** MCP call (when **guided**). ACQUIRE **`agent-action-resolution.md`** before any **`{agent-action:*}`** workspace step. If KB alias fails, use **`instructions/r2/core/workflows/.md`**. +2. Read **`agents/mcp-capability.yaml`** if present; apply **user override** from the current task text (same rules as Phase 1). +3. Derive **`TestRail export: guided | questionnaire`**: + - **`mcp.mode: absent`** → questionnaire. + - **`mcp.mode: capable`** and **`mcp.testrail: false`** (when the key is present) → questionnaire. + - **`mcp.mode: capable`** and (`testrail` omitted or `true`) → guided. + - Missing YAML: use **user override** if any; else **one** short question (“Use TestRail MCP for export?”). **No** → questionnaire; **Yes** → guided and recommend adding `agents/mcp-capability.yaml`. +4. Record in `agents/testgen/{TICKET-KEY}/testgen-state.md` (Phase 6): + - `TestRail export: guided | questionnaire` + - `MCP interaction source:` (`agents/mcp-capability.yaml` | user override | default question`) +5. If **guided** and **`agents/user-instructions/mcp-guidance.md`** exists, read it before the first **`{integration-action:testrail-*}`** call. + +### Step 1: Verify TestRail connection (guided only) + +**If `TestRail export` is `questionnaire`:** Run **Step 1Q** instead of this step. + +#### Step 1Q: Questionnaire — no MCP verification + +1. Ask **numbered** questions: `project_id`, `suite_id` (for links and documentation), and confirm the user will import cases manually or paste back IDs after import. +2. **STOP** and **WAIT**. +3. Skip to **Step 2** (section_id flow is unchanged; user may already know `section_id`). + +**Guided path — test connection** using (replace `project_id` with value from guidance/state/user; **do not** hard-code example IDs unless the user confirmed them): + +```text +Action: {integration-action:testrail-get-project} +Parameters: + project_id: ``` If fails, tell user: @@ -39,7 +83,7 @@ If fails, tell user: ❌ TestRail connection failed. Please verify: 1. TestRail MCP is configured 2. Credentials are correct -3. Project ID 69 exists and you have access +3. Project ID exists and you have access ``` ### Step 2: Create Section in TestRail @@ -57,7 +101,7 @@ If you already have a section, provide the section_id. You can find it in the URL when viewing a section (e.g., group_id=94686 or section_id=94686) **Option B: Create new section** -1. Go to: https://griddynamics.testrail.io/index.php?/suites/view/3300 +1. Go to: {testrail-suite-view-url} 2. Click "Add Section" 3. Name it: [TICKET-KEY] (e.g., "PROJ-456") 4. After creating, find the section_id in the URL or section details @@ -224,19 +268,40 @@ Execute this test case for EACH row in the table below: - Just include original preconditions normally - No "Execute for EACH row" note -### Step 5: Add Test Cases to TestRail +### Step 5: Add Test Cases to TestRail (guided only) -**For each mapped test case**: +**If `TestRail export` is `questionnaire`:** Run **Step 5Q** only; **do not** run **`{integration-action:testrail-add-case}`**. -```python -result = mcp_testrail_add_case( - section_id=section_id, - title=testrail_case["title"], - priority_id=testrail_case["priority_id"], - type_id=testrail_case["type_id"], - refs=testrail_case["refs"], - custom_steps_separated=testrail_case["custom_steps_separated"] -) +#### Step 5Q: Manual export pack (no MCP) + +1. After **Step 4** mapping, write **`agents/testgen/{TICKET-KEY}/testrail-manual-export.md`** containing: + - User-supplied `project_id`, `suite_id`, `section_id` (if known) + - One block per test case: title, priority, type, refs, preconditions text, and steps (content + expected) in plain text or markdown tables suitable for copy-paste into TestRail. +2. Optionally add **`agents/testgen/{TICKET-KEY}/testrail-import-hints.csv`** (title, priority, type, refs, preconditions, steps_json) if the team uses CSV import. +3. **STOP** and **WAIT**: ask the user to confirm either (a) they created cases manually / imported the pack, or (b) they paste back a mapping `TC-001 → C12345` (or URLs). +4. **After the user replies**, execute **Step 6** to merge IDs into `test-scenarios.md`. Until then you may leave placeholders (e.g. `Pending`) only if the workflow must save intermediate files; do **not** mark Phase 6 complete in state until Step 6 reflects the user’s answer. + +**Optional (guided only) — list existing cases before add:** If duplicate titles, re-exports, or unclear existing coverage are a concern, run **`{integration-action:testrail-get-cases}`** once against the target `project_id` / `suite_id` (and use section or other filters when your MCP supports them) to compare with parsed markdown. **Not required** for a first-time export or when the user accepts duplicates. + +```text +Action: {integration-action:testrail-get-cases} +Parameters: + project_id: + suite_id: +``` + +**Guided path — for each mapped test case**: + +```text +Action: {integration-action:testrail-add-case} +Parameters: + section_id: + title: + priority_id: + type_id: + refs: + custom_steps_separated: +Outcome: capture returned case id for results tracking ``` **Track results**: @@ -261,12 +326,14 @@ results = { **Update**: `agents/testgen/{TICKET-KEY}/test-scenarios.md` +For **questionnaire** exports, use user-pasted IDs when available; otherwise keep **`TestRail ID`**: `Manual (see testrail-manual-export.md)`** until the user confirms. Use real `project` / `suite` / base URL values from Step 1Q or guidance — **do not** invent instance hostnames. + **Add TestRail IDs to each test case**: ```markdown ### TC-001: User Login with Valid Credentials (Happy Path) **TestRail ID**: C12345 ✅ -**TestRail Link**: https://griddynamics.testrail.io/index.php?/cases/view/12345 +**TestRail Link**: {testrail-case-view-url} **Related Requirement**: US-1, FR-1 ... ``` @@ -283,11 +350,11 @@ results = { ## TestRail Export Summary **Exported**: [DateTime] -**Project**: 69 -**Suite**: 3300 +**Project**: {project_id} +**Suite**: {suite_id} **Section**: [TICKET-KEY] (ID: [section_id]) **Total Exported**: [X] test cases -**TestRail Link**: https://griddynamics.testrail.io/index.php?/suites/view/3300&group_by=cases:section_id +**TestRail Link**: {testrail-suite-cases-grouped-url} | TC ID | TestRail ID | Status | |-------|-------------|--------| @@ -326,8 +393,8 @@ results = { ### Phase 6: TestRail Export - **Completed**: [DateTime] -- **Project ID**: 69 -- **Suite ID**: 3300 +- **Project ID**: {project_id} +- **Suite ID**: {suite_id} - **Section ID**: [section_id] - **Section Name**: [TICKET-KEY] - **Test Cases Created**: [Count] @@ -339,21 +406,30 @@ results = { ## Validation Before completing Phase 6, verify: -- ✅ TestRail connection successful -- ✅ Section exists in TestRail + +**Guided path:** +- ✅ TestRail connection successful (`{integration-action:testrail-get-project}` or equivalent) +- ✅ Section exists in TestRail (or user provided valid `section_id`) - ✅ All test cases parsed from markdown -- ✅ At least 80% of test cases exported successfully -- ✅ test-scenarios.md updated with TestRail IDs +- ✅ At least 80% of test cases exported successfully via MCP (or failures documented) +- ✅ `test-scenarios.md` updated with TestRail IDs - ✅ State file updated with Phase 6 complete - ✅ TestRail link provided to user +**Questionnaire path:** +- ✅ Step 0 recorded `TestRail export: questionnaire` and source line +- ✅ User confirmed `project_id` / `suite_id` / `section_id` as needed +- ✅ `testrail-manual-export.md` (and optional CSV) written +- ✅ User confirmed manual import or pasted ID mapping +- ✅ `test-scenarios.md` updated with IDs or explicit “manual / pending” notes +- ✅ State file updated with Phase 6 complete + ## Tools Used -- `mcp_testrail_get_project(project_id)` - Verify connection -- `mcp_testrail_get_cases(project_id, suite_id)` - Check existing cases -- `mcp_testrail_add_case(section_id, title, ...)` - Create test cases -- `read_file()` - Read test-scenarios.md -- `write()` / `search_replace()` - Update files +- **Guided:** `{integration-action:testrail-get-project}` — verify connection; `{integration-action:testrail-get-cases}` — optional dedup; `{integration-action:testrail-add-case}` — create test cases +- **Questionnaire:** `{agent-action:read-file}` / `{agent-action:write-file}` / `{agent-action:patch-file}` only — build `testrail-manual-export.md` and update docs; **no** `{integration-action:testrail-*}` calls +- `{agent-action:read-file}` — read `test-scenarios.md` +- `{agent-action:write-file}` / `{agent-action:patch-file}` — update files ## Common Issues @@ -375,6 +451,9 @@ Before completing Phase 6, verify: **Issue**: Rate limit exceeded **Solution**: Add delay between API calls, batch requests +**Issue**: Risk of duplicate titles or unclear existing coverage +**Solution**: Optionally run **`{integration-action:testrail-get-cases}`** (Step 5 optional block) before **`{integration-action:testrail-add-case}`** — optional discovery of existing cases, not a mandatory gate. + **Issue**: Test case already exists **Solution**: Create anyway (TestRail allows duplicates), note in report @@ -393,7 +472,7 @@ All 6 phases finished successfully: ✅ Phase 6: TestRail Export ([R] cases exported) **TestRail Section**: [TICKET-KEY] -**TestRail Link**: https://griddynamics.testrail.io/index.php?/suites/view/3300 +**TestRail Link**: {testrail-suite-view-url} **Deliverables**: 📄 requirements.md - Use for implementation @@ -411,8 +490,8 @@ All 6 phases finished successfully: - TestRail MCP currently lacks section creation - user must create manually - Test case IDs in TestRail are prefixed with "C" (e.g., C12345) -- Suite ID 3300 is from user's TestRail URL -- Project ID 69 is user's default project +- `{suite_id}` comes from the user, `mcp-guidance.md`, or the suite view URL on their instance +- `{project_id}` comes from the user, `mcp-guidance.md`, or the TestRail project settings - Parameterized test data is included in step content or expected results - Re-running export creates duplicate test cases (by design, to preserve history) - Consider creating test run after export for immediate execution diff --git a/plugins/core-copilot-standalone/.github/prompts/testgen-flow.prompt.md b/plugins/core-copilot-standalone/.github/prompts/testgen-flow.prompt.md index 1e334ca6..de88e4c8 100644 --- a/plugins/core-copilot-standalone/.github/prompts/testgen-flow.prompt.md +++ b/plugins/core-copilot-standalone/.github/prompts/testgen-flow.prompt.md @@ -14,6 +14,7 @@ Systematic requirements analysis from Jira tickets and Confluence documentation ### Critical Requirements +- **MCP ROUTING (B + A)**: Phases 1 and 6 may call external MCPs only when **`agents/mcp-capability.yaml`** and **`mcp-capability-interaction.md`** resolve **guided** for that integration. Otherwise use **questionnaire** or manual export paths documented in those phases. Optional **`agents/user-instructions/mcp-guidance.md`** for IDs and guardrails when guided. **`{integration-action:*}`** / **`{agent-action:*}`** binding: follow each phase file’s **Prerequisites** (**`mcp-tool-resolution.md`**, **`agent-action-resolution.md`**). - **ONE PHASE AT A TIME**: Read phase file, execute, update state, move to next. - **DO NOT SKIP PHASES**: Each builds on previous. - **STATE TRACKING**: Update `agents/testgen/{TICKET-KEY}/testgen-state.md` after each phase. @@ -31,9 +32,10 @@ Systematic requirements analysis from Jira tickets and Confluence documentation **Phase 1: Data Collection** [testgen-flow-data-collection.md] 1. ACQUIRE testgen-flow-data-collection.md FROM KB -2. Execute phase instructions -3. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` -4. Validate by listing raw-data.md file +2. ACQUIRE **`mcp-capability-interaction.md`**, and per that phase’s **Prerequisites** also **`mcp-tool-resolution.md`** / **`agent-action-resolution.md`** when placeholders are used +3. Execute phase instructions +4. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` +5. Validate by listing raw-data.md file **Phase 2: Gap & Contradiction Analysis** [testgen-flow-gap-and-contradiction-analysis.md] 1. ACQUIRE testgen-flow-gap-and-contradiction-analysis.md FROM KB @@ -61,8 +63,9 @@ Systematic requirements analysis from Jira tickets and Confluence documentation **Phase 6: Test Case Export** [testgen-flow-test-case-export.md] ⭐ 1. ACQUIRE testgen-flow-test-case-export.md FROM KB -4. Execute phase instructions -5. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` +2. ACQUIRE **`mcp-capability-interaction.md`** and per that phase’s **Step 0** / **Prerequisites** also **`mcp-tool-resolution.md`** / **`agent-action-resolution.md`** when placeholders are used (guided vs questionnaire for TestRail export) +3. Execute phase instructions +4. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` ## State File Format @@ -136,9 +139,9 @@ agents/testgen/{TICKET-KEY}/ ## Prerequisites -- **Jira MCP**: Configured and accessible -- **Jira Ticket**: User provides ticket key or link -- **Confluence Access**: Via Jira MCP (same authentication) +- **MCP routing (B + A):** `agents/mcp-capability.yaml` (see `instructions/r2/core/templates/mcp-capability.example.yaml`) plus optional `agents/user-instructions/mcp-guidance.md` control **guided** vs **questionnaire** per integration. Phases 1 and 6 embed the branching rules. +- **Jira ticket**: User provides ticket key or link (Phase 1) +- **Confluence**: Either MCP-guided discovery (Phase 1) or user-provided content when **questionnaire** - **Output Directory**: Created at start (`agents/testgen/{TICKET-KEY}/`) ## Common Patterns diff --git a/plugins/core-copilot/commands/agent-action-resolution.md b/plugins/core-copilot/commands/agent-action-resolution.md new file mode 100644 index 00000000..75fb60a9 --- /dev/null +++ b/plugins/core-copilot/commands/agent-action-resolution.md @@ -0,0 +1,18 @@ +--- +name: agent-action-resolution +description: Maps `{agent-action:…}` tokens to host IDE or agent-runtime file tools (read, write, patch). +alwaysApply: false +baseSchema: docs/schemas/phase.md +--- + +# `{agent-action:…}` resolution (host file tools) + +Use when a workflow uses **`{agent-action:read-file}`**, **`{agent-action:write-file}`**, or **`{agent-action:patch-file}`**. These are **not** remote MCP calls and are **not** covered by **`mcp-tool-resolution.md`**. + +| Token | Intent | +|-------|--------| +| `{agent-action:read-file}` | Read a file from the workspace via the host agent’s file-read tool | +| `{agent-action:write-file}` | Create or overwrite a workspace file via the host agent’s file-write tool | +| `{agent-action:patch-file}` | Apply targeted edits via the host agent’s patch / search-replace tool | + +**Resolution:** Map each token to the host IDE’s or agent runtime’s **documented** file operation (names differ: e.g. `Read`, `Write`, `StrReplace`, `apply_patch`). If the host exposes no matching capability, **STOP**, tell the user, and **WAIT** — do not substitute an MCP tool. diff --git a/plugins/core-copilot/commands/aqa-flow-data-collection.md b/plugins/core-copilot/commands/aqa-flow-data-collection.md index 34fc0898..a9e7253d 100644 --- a/plugins/core-copilot/commands/aqa-flow-data-collection.md +++ b/plugins/core-copilot/commands/aqa-flow-data-collection.md @@ -9,19 +9,61 @@ baseSchema: docs/schemas/phase.md ## Objective -Gather all required information from external sources (TestRail and Confluence) to understand test requirements and expected behavior. +Gather everything needed for later phases: either from **TestRail and Confluence** (**integrated AQA**) or from **user-supplied artifacts** (**minimal-input / agent-led**). Always record the chosen **execution mode** in the test plan and state file. ## Prerequisites -- TestRail MCP configured and accessible -- Atlassian (Confluence) MCP configured and accessible -- Test case ID or requirement provided by user +- User intent is clear enough to start Phase 1 (IDs, URLs, or a plain-language automation goal). +- **Integrated AQA path**: For **guided** MCP interaction (see Task 0b), TestRail MCP and Atlassian (Confluence) MCP must be usable when you run Path A MCP steps. For **questionnaire** interaction, do **not** call those MCPs; collect the same facts via user answers and paste into the test plan. +- **Minimal-input path**: No MCP requirement for TestRail/Confluence; Task 0b **does not apply** to Path B for those integrations. If something on the checklist is missing, **ask** — do not guess. + +## MCP capability reference + +Resolve **guided vs questionnaire** per **`mcp-capability-interaction.md`** (ACQUIRE FROM KB). Config file: **`agents/mcp-capability.yaml`** (copy from `instructions/r2/core/templates/mcp-capability.example.yaml` in Rosetta repo). Optional guidance: **`agents/user-instructions/mcp-guidance.md`**. ## Phase Tasks -### Task 1: Read TestRail Test Case +### Task 0: Confirm execution mode + +**Actions**: +1. Check whether the **current task text** (or a prior user message in the same session the user instructs you to treat as binding) already names the mode **explicitly**, using any of these (case-insensitive): `integrated`, `minimal-input`, `minimal input`, `Integrated AQA`, or `minimal-input / agent-led`. The token must be **user-authored** in that message (not something you paraphrase or infer). If yes, set the mode to `integrated` or `minimal-input` accordingly and **skip steps 2–3**; go directly to **Path A** or **Path B** matching that mode. +2. If **not** explicit: **Do not infer** mode from clues such as missing TestRail IDs, presence of HTML attachments, “sandbox,” “no MCP,” or vague phrases like “I don’t have TestRail yet.” **Forbidden:** choosing `minimal-input` because TestRail was omitted, the repo looks small, or the user only supplied local HTML—those are **not** mode labels. Informal phrases (“lightweight,” “without TestRail,” “bring my own selectors”) **do not** count as explicit; they still require steps 2–3. **Ask once** with both options, for example: + - **Integrated AQA**: "We pull the official case and docs from TestRail/Confluence and follow the full chain." + - **Minimal-input / agent-led**: "You provide URLs, page source or a selector map, scenario and expectations; we still run phases 2–8 in order but Phase 1 records your package instead of MCP pulls." +3. **STOP** and **WAIT** until the user replies with a clear **integrated** or **minimal-input** (or the spelled labels in step 1). If the reply is still ambiguous, ask **one** narrowing question that still requires those labels; **do not** choose a path without those words from the user. +4. **Hard gate — before Path A or Path B:** Do **not** create or update `agents/plans/aqa-*.md`, do **not** write `**AQA execution mode**` in any artifact, and do **not** mark Phase 1 complete in `agents/aqa-state.md` until step **1** matched or step **3** completed. If you already started Path A/B without satisfying Task 0, **stop**, discard partial mode lines, run steps 2–3, then continue. +5. Ensure the Phase 1 test plan created in Path A or Path B includes `**AQA execution mode**` and that `agents/aqa-state.md` reflects the mode when Phase 1 is marked complete (see **Update State File** below). + +**Expected Output**: Mode is either read from **explicit user keywords** (step 1) or obtained only after **mandatory** ask-and-wait (steps 2–3); then proceed with **Path A** or **Path B** (not both). + +--- + +### Task 0b: Resolve MCP interaction (Path A only) + +**Skip this entire task for Path B (minimal-input).** **Actions**: +1. ACQUIRE **mcp-capability-interaction.md** FROM KB (if not already loaded this phase). +2. Read **`agents/mcp-capability.yaml`** if it exists; apply **user override** rules from that fragment (task text beats file). If file missing and user has not overridden, ask **one** yes/no: “Use live MCP for TestRail/Confluence in this workspace?” — **WAIT**; treat **No** as questionnaire, **Yes** as guided for both integrations until a YAML file exists. +3. Derive **per integration** (TestRail vs Confluence) whether the source is **guided** (MCP allowed) or **questionnaire** (no MCP): if `mcp.mode` is `absent`, both are **questionnaire**; if `capable`, use `mcp.testrail` / `mcp.atlassian_confluence` when present (boolean); if a key is **omitted** under `capable`, default that integration to **guided**. +4. Record in the test plan and `agents/aqa-state.md` Phase 1 notes: `TestRail source: guided|questionnaire`, `Confluence source: guided|questionnaire`, plus `MCP interaction source:` (`agents/mcp-capability.yaml` | user override | default question`). +5. **Questionnaire leg:** For each integration still **questionnaire** and not yet documented in the plan, run **one** numbered questionnaire (combine TestRail + Confluence in a single message if both need data). **STOP** and **WAIT**. Merge answers under **User-provided (MCP absent) — TestRail** and/or **User-provided (MCP absent) — Confluence** in the working plan (or append to the plan draft before Task A3). +6. **Guided leg:** If **`agents/user-instructions/mcp-guidance.md`** exists and at least one integration is **guided**, read it before the first MCP call for a **guided** integration. Then continue to Task A1 / A2 and follow the **guided** branches there only. + +**Expected Output**: TestRail and Confluence are each classified **guided** or **questionnaire**; questionnaire content is captured before MCP calls; no MCP call runs for a **questionnaire** integration. + +--- + +### Path A — Integrated AQA (TestRail + Confluence) + +**Prerequisite:** Task 0 already set execution mode to **integrated** (step 1 keyword match or step 3 user reply). Do not enter Path A without that. + +### Task A1: Read TestRail Test Case + +**Actions**: +- **If Task 0b resolved questionnaire for TestRail:** Ensure the test plan already contains **User-provided (MCP absent) — TestRail** content from Task 0b. If missing, **STOP**, ask numbered questions (case ID for traceability, title, steps, expected results, preconditions), **WAIT**, merge into the plan. **Do not** call `user-testrail-get_case`. Skip the guided-only steps below for TestRail. +- **If Task 0b resolved guided for TestRail:** Perform steps 1–4. + 1. Ask user for TestRail test case ID if not provided 2. Use TestRail MCP to retrieve test case details: ``` @@ -39,9 +81,12 @@ Gather all required information from external sources (TestRail and Confluence) **Expected Output**: Complete understanding of what needs to be tested according to TestRail. -### Task 2: Read Confluence Documentation +### Task A2: Read Confluence Documentation **Actions**: +- **If Task 0b resolved questionnaire for Confluence:** Ensure the test plan contains **User-provided (MCP absent) — Confluence** content from Task 0b. If missing, **STOP**, ask numbered questions (URLs, pasted excerpts, or summaries the user authorizes), **WAIT**, merge into the plan. **Do not** call `user-mcp-atlassian-confluence_*`. Skip guided-only steps below for Confluence. +- **If Task 0b resolved guided for Confluence:** Perform steps 1–5. + 1. Ask user for Confluence page ID/URL or search terms if not provided 2. Use Atlassian Confluence MCP to find related documentation: ``` @@ -60,10 +105,11 @@ Gather all required information from external sources (TestRail and Confluence) **Expected Output**: Business and technical context for the feature being tested. -### Task 3: Create Initial Test Plan Document +### Task A3: Create initial test plan document (integrated) **Actions**: 1. Create `agents/plans/aqa-.md` file with: + - **AQA execution mode**: `integrated` - Test case reference (TestRail ID and link) - Feature name and description - Test goal @@ -77,6 +123,7 @@ Gather all required information from external sources (TestRail and Confluence) # AQA Test Plan - **Created**: [DateTime] +**AQA execution mode**: integrated **TestRail Case**: [ID/URL] **Feature**: [Feature Name] **Status**: Phase 1 Complete @@ -134,14 +181,112 @@ Gather all required information from external sources (TestRail and Confluence) [To be filled in Phase 6] ``` +--- + +### Path B — Minimal-input / agent-led + +**Prerequisite:** Task 0 already set execution mode to **minimal-input** (step 1 keyword match or step 3 user reply). Do not enter Path B because TestRail was missing or files were attached—only because the user chose **minimal-input** in text. + +### Task B1: Offer agent-led implementation and required artifacts + +**Actions**: +1. Tell the user clearly that the agent **can** implement the test **without** TestRail/Confluence **if** they provide enough grounded detail — but the agent **must not** invent pages, selectors, or flows. +2. List what you still need at minimum (see Task B2 checklist). **WAIT** until the user either supplies items or explicitly defers an item to Phase 2 (then note "open in Phase 2" in the plan). + +**Expected Output**: User understands the tradeoff and what to supply (or what is deferred to Phase 2 with explicit consent). + +### Task B2: Collect minimal-input package + +**Actions**: +1. Capture the following in the test plan (fill or mark *deferred to Phase 2* with user agreement): + - **Scenario / goal**: What behavior the automated test must prove (plain language). + - **Entry / URLs**: Starting URL(s) or navigation path to reach the UI under test. + - **UI grounding** (at least one required before ending Phase 1 unless user explicitly defers to Phase 2): e.g. saved page source or DOM snapshot paths, a **selector map** (purpose → locator), pointers to existing Page Objects in the repo, and/or frontend file paths the agent will use in Phase 3. + - **Preconditions and test data**: Accounts, feature flags, seed data — or *unknown — Phase 2*. + - **Optional**: TestRail/Confluence IDs "for traceability only" without MCP — record as references, not as substitute for missing UI grounding. +2. **Do not** mark Phase 1 complete if UI grounding and scenario goal are both missing and the user has not agreed to defer. + +**Expected Output**: Minimal-input checklist documented in the test plan. + +### Task B3: Create initial test plan document (minimal-input) + +**Actions**: +1. Create `agents/plans/aqa-.md` using the template below (adapt section headings if the scenario is small). +2. For TestRail/Confluence fields, use `N/A` or optional reference links — do not pretend MCP data exists. + +**Template**: +```markdown +# AQA Test Plan - + +**Created**: [DateTime] +**AQA execution mode**: minimal-input +**TestRail Case**: [N/A or ID for traceability only] +**Confluence**: [N/A or links for traceability only] +**Feature**: [Feature Name] +**Status**: Phase 1 Complete + +## Minimal-input package + +### Scenario / goal +[What the test must validate] + +### Entry and navigation +- URLs: [...] +- Steps to reach the UI: [...] + +### UI grounding +- Page source / DOM snapshots: [paths or attach instructions] +- Selector map or locators: [...] +- Repo pointers (Page Objects / components): [...] + +### Preconditions and test data +[Or: deferred to Phase 2 — user confirmed] + +### Notes and open questions +- [...] + +## Test Case Information (synthetic) + +### Test Steps (draft) +1. [Step — can be refined in Phase 2] +... + +### Expected Overall Result +[Draft — refined in Phase 2] + +--- +## Phase 2: Requirements Clarification +[To be filled in Phase 2] + +## Phase 3: Code Analysis +[To be filled in Phase 3] + +## Phase 4: Selector Identification +[To be filled in Phase 4] + +## Phase 5: Selector Implementation +[To be filled in Phase 5] + +## Phase 6: Test Implementation +[To be filled in Phase 6] +``` + ## Completion Criteria +**All modes** +- [ ] Task 0 complete: execution mode is **either** named by explicit user keywords (`integrated` / `minimal-input` / spelled equivalents) **or** obtained after the mandatory ask-and-wait; mode recorded in test plan and state +- [ ] Test plan file `agents/plans/aqa-.md` created +- [ ] Test goal clearly understood (or explicitly deferred to Phase 2 with user consent) +- [ ] `agents/aqa-state.md` updated with Phase 1 completion + +**Integrated AQA only** - [ ] TestRail test case retrieved and documented - [ ] Confluence documentation retrieved and documented -- [ ] Test plan file created with all Phase 1 information -- [ ] Test goal clearly understood -- [ ] Expected results documented -- [ ] `agents/aqa-state.md` updated with Phase 1 completion +- [ ] Expected results from sources documented + +**Minimal-input only** +- [ ] Minimal-input checklist captured in the test plan +- [ ] UI grounding present or explicitly deferred to Phase 2 with user approval ## Update State File @@ -150,8 +295,10 @@ After completing Phase 1, update `agents/aqa-state.md`: ```markdown ### Phase 1: Data Collection - Completed: [DateTime] -- TestRail Case: [ID/URL] -- Confluence Pages: [URLs] +- Execution mode: [integrated | minimal-input] +- TestRail Case: [ID/URL or N/A] +- Confluence Pages: [URLs or N/A] +- Minimal-input checklist: [brief summary or N/A] - Test Goal: [Brief description] - Expected Result: [Brief description] - Test Plan File: agents/plans/aqa-.md @@ -168,7 +315,9 @@ ACQUIRE aqa-phase2-md FROM KB ## Important Notes -- **No Assumptions**: If TestRail or Confluence data is incomplete, note it in the test plan -- **Ask Questions**: If user hasn't provided IDs/URLs, ask for them -- **Document Everything**: Capture all details even if they seem minor -- **Cross-Reference**: Ensure TestRail and Confluence information aligns +- **No Assumptions**: If data is incomplete for the chosen mode, note it in the test plan and ask — never guess selectors or flows. +- **Ask Questions**: If user hasn't provided IDs/URLs (integrated) or checklist items (minimal-input), ask for them. +- **Document Everything**: Capture all details even if they seem minor. +- **Integrated path only**: Cross-reference TestRail and Confluence information for alignment. +- **Minimal-input path**: Prefer an explicit **selector map** or page source over vague descriptions; deferrals must be user-visible in the plan. +- **Task 0**: Never infer execution mode from context; only explicit keywords or a user reply after the mandatory question count. diff --git a/plugins/core-copilot/commands/aqa-flow-requirements-clarification.md b/plugins/core-copilot/commands/aqa-flow-requirements-clarification.md index f0d21565..f7fb9a9b 100644 --- a/plugins/core-copilot/commands/aqa-flow-requirements-clarification.md +++ b/plugins/core-copilot/commands/aqa-flow-requirements-clarification.md @@ -13,16 +13,23 @@ Fill gaps in understanding, clarify unknowns, and define explicit assertions bef ## Prerequisites -- Phase 1 completed -- Test plan file created with TestRail and Confluence data -- Initial understanding of test requirements +- Phase 1 completed with a test plan file +- Phase 1 documents either **integrated** sources (TestRail + Confluence as used) **or** **minimal-input** execution mode with the checklist from Phase 1 (scenario, URLs, UI grounding or explicit deferral) +- Initial understanding of test requirements (may be draft in minimal-input mode) ## Phase Tasks +### Minimal-input mode (read first) + +If the test plan shows **`AQA execution mode: minimal-input`**: +- Treat Phase 1's checklist as authoritative for what the user already supplied; **do not** re-demand TestRail/Confluence data unless the user wants it for traceability. +- Still run gap analysis: anything deferred from Phase 1 **must** be resolved or explicitly re-deferred here with user text. +- Assertions must still be explicit and measurable before Phase 3; **never** invent selectors the user did not supply, defer, or point to in repo/page source. + ### Task 1: Review Gathered Information for Gaps **Actions**: -1. Read the test plan file from Phase 1 +1. Read the test plan file from Phase 1 and note **AQA execution mode** (`integrated` vs `minimal-input`). 2. Analyze information for completeness: - Are test steps clear and unambiguous? - Are expected results specific and measurable? @@ -197,3 +204,4 @@ ACQUIRE aqa-phase3-md FROM KB - **Explicit Over Implicit**: Every assertion must be measurable and verifiable - **User Authority**: User has final say on requirements and expected behavior - **Document Everything**: Record all questions and answers for traceability +- **Minimal-input mode**: Questions should close gaps in the Phase 1 checklist (especially UI grounding and test data), not re-litigate integrated AQA unless the user switches mode explicitly diff --git a/plugins/core-copilot/commands/aqa-flow-test-implementation.md b/plugins/core-copilot/commands/aqa-flow-test-implementation.md index a16d294e..e92ab613 100644 --- a/plugins/core-copilot/commands/aqa-flow-test-implementation.md +++ b/plugins/core-copilot/commands/aqa-flow-test-implementation.md @@ -13,13 +13,16 @@ Create automated test following project standards, integrating all Page Objects ## Prerequisites -- All previous phases (1-5) completed -- Test plan fully documented with all details -- Page Objects updated with all required selectors +- All previous phases (1-5) completed **for the chosen execution mode** (see below) +- Test plan fully documented with all details **including** `AQA execution mode` and, if `minimal-input`, the **minimal-input checklist** from Phase 1 (and any Phase 2 resolutions of deferrals) +- Page Objects updated with all required selectors **or** a clear plan from Phases 4–5 that matches user-supplied grounding (minimal-input) - Explicit assertions defined - Code architecture understood -## Phase Tasks +### Execution modes and prerequisites + +- **Integrated AQA**: The bullets above mean phases 1–5 were completed in full with TestRail/Confluence-backed Phase 1 where applicable. +- **Minimal-input / agent-led** (recorded in Phase 1): Phases 1–5 count as satisfied only if the test plan contains the **minimal-input checklist** (scenario, URLs/navigation, UI grounding or approved Phase 2 deferrals) and **explicit assertions from Phase 2**. **Before writing code**, re-read that checklist and Phase 2 assertions; if anything material is still missing, **STOP**, ask the user, and do not invent selectors, pages, or flows. ### Task 1: Review Test Implementation Plan diff --git a/plugins/core-copilot/commands/aqa-flow.md b/plugins/core-copilot/commands/aqa-flow.md index 00f7c725..f7931333 100644 --- a/plugins/core-copilot/commands/aqa-flow.md +++ b/plugins/core-copilot/commands/aqa-flow.md @@ -10,20 +10,23 @@ baseSchema: docs/schemas/workflow.md ## Context -This agent handles end-to-end test automation from requirements gathering to test implementation. It uses TestRail, Confluence, and project documentation to create automated tests following existing architecture and coding standards. +This agent handles end-to-end test automation from requirements gathering to test implementation. It uses TestRail, Confluence, and project documentation when you run **integrated AQA**; in **minimal-input / agent-led** mode the user supplies pages, DOM or selector details, and scenario context instead (see Phase 1). In all modes, tests follow existing architecture and coding standards. ### Critical Requirements - **ONE PHASE AT A TIME**: Read phase file, execute, update state, move to next. - **DO NOT SKIP PHASES**: Each builds on previous. - **NO ASSUMPTIONS**: Never assume selectors, flows, or data. Always ask the user if information is missing. -- **USER INTERACTION**: Wait for user responses when questions are asked or files are requested. Phase 2, 6, 7, and 8 always require user input. Phase 4 requires user input ONLY if frontend code is unavailable or selectors cannot be found +- **MCP ROUTING (B + A)**: Before calling external MCPs in integrated Path A, follow **`mcp-capability-interaction`** using `agents/mcp-capability.yaml` and any user override in the task text. **Guided** = use MCP after optional `agents/user-instructions/mcp-guidance.md`. **Questionnaire** = no MCP calls for that integration; user answers structured questions; record answers in the plan. ACQUIRE `mcp-capability-interaction.md` FROM KB when executing Phase 1 Path A. +- **EXECUTION MODE (Phase 1, first)**: The user must **explicitly** choose **integrated** or **minimal-input** (case-insensitive), or use the spelled labels **Integrated AQA** or **minimal-input / agent-led**, in **their** message before you record a mode. If none of those appear, **ask once** with both options, **STOP**, and **WAIT**—**do not infer** mode from missing TestRail, attachments, sandbox, or “no MCP.” **Do not** write `agents/plans/aqa-*.md` or set `**AQA execution mode**` until that keyword match or the user’s reply to the question. **Forbidden:** assigning `minimal-input` because TestRail was omitted or the project looks like a demo. After the user answers, document the mode in the Phase 1 test plan and `agents/aqa-state.md`. Do not silently invent missing integrations or missing UI grounding. User customizations elsewhere do **not** waive this rule when mode was not explicitly named. +- **USER INTERACTION**: Wait for user responses when questions are asked or files are requested. **Phase 1** requires user input until execution mode is explicit or the user answers the one-time mode question (see **EXECUTION MODE**). Phase 2, 6, 7, and 8 always require user input. Phase 4 requires user input ONLY if frontend code is unavailable or selectors cannot be found - **STATE TRACKING**: Update `agents/aqa-state.md` after each phase. - **MUST** use todo tasks for tracking progress. - Prioritize ACCURACY over SPEED! ### User Customizations +- The **execution mode** rule cannot be waived by “only run some phases” unless the user message already contains the mode keywords or the user answers the one-time mode question. - If user did not specify any preferences perform all steps except optional. - User CAN customize and ask only for specific phases OR phases could have been done already OR towards specific goal OR for specific case, in this case LISTEN and ADOPT to the user. @@ -31,9 +34,10 @@ This agent handles end-to-end test automation from requirements gathering to tes **Phase 1: Data Collection** [aqa-flow-data-collection.md] 1. ACQUIRE aqa-flow-data-collection.md FROM KB -2. Execute phase instructions -3. Update `agents/aqa-state.md` -4. Validate gathered data +2. ACQUIRE mcp-capability-interaction.md FROM KB (resolve MCP guided vs questionnaire before Path A MCP calls) +3. Execute phase instructions (**start with execution mode confirmation** — integrated vs minimal-input) +4. Update `agents/aqa-state.md` +5. Validate gathered data (checklist for chosen mode is complete before Phase 2) **Phase 2: Requirements Clarification** [aqa-flow-requirements-clarification.md] ⭐ **USER INTERACTION REQUIRED** 1. ACQUIRE aqa-flow-requirements-clarification.md FROM KB @@ -91,7 +95,8 @@ Create/update `agents/aqa-state.md` after each phase: **Last Updated**: [DateTime] **Current Phase**: [1-8 or COMPLETE] -**TestRail Case**: [Test Case ID/URL] +**AQA execution mode**: [integrated | minimal-input] +**TestRail Case**: [Test Case ID/URL or N/A in minimal-input] **Feature**: [Feature Name] ## Phase Completion Status @@ -109,8 +114,10 @@ Create/update `agents/aqa-state.md` after each phase: ### Phase 1: Data Collection - Completed: [DateTime] -- TestRail Case: [ID/URL] -- Confluence Pages: [URLs] +- Execution mode: [integrated | minimal-input] +- TestRail Case: [ID/URL or N/A] +- Confluence Pages: [URLs or N/A] +- Minimal-input checklist: [summary or N/A — see Phase 1 test plan] - Test Goal: [Brief description] - Expected Result: [Brief description] @@ -158,6 +165,7 @@ Create/update `agents/aqa-state.md` after each phase: ## Important Notes - **Sequential Execution**: Phases build on each other, must execute in order. +- **Execution modes**: **Integrated AQA** uses TestRail and Confluence in Phase 1 when those tools are in scope. **Minimal-input / agent-led** uses the Phase 1 checklist (pages, DOM or selectors, scenario context); later phases still run in order, but Phase 1 documents what substitutes for external case management. Without an **explicit** user mode label (`integrated` or `minimal-input` / spelled equivalents), Phase 1 must obtain that label by asking—never infer from context alone. - **No Assumptions Rule**: Always ask user when information is missing - never guess selectors, flows, or test data. - **Architecture First**: Always analyze existing code structure before implementing new tests. - **Reuse Over Creation**: Prefer adding to existing files and using existing Page Objects over creating new ones. diff --git a/plugins/core-copilot/commands/mcp-capability-interaction.md b/plugins/core-copilot/commands/mcp-capability-interaction.md new file mode 100644 index 00000000..4f4fae4d --- /dev/null +++ b/plugins/core-copilot/commands/mcp-capability-interaction.md @@ -0,0 +1,64 @@ +--- +name: mcp-capability-interaction +description: Routes guided vs questionnaire MCP usage from workspace YAML and user text, records interaction source, and defines where capability files live. Placeholder binding uses sibling workflows. +alwaysApply: false +baseSchema: docs/schemas/phase.md +--- + +# MCP capability interaction (shared) + +Use when a workflow **may** call **remote MCPs** (Jira, Confluence, TestRail, Atlassian). It does **not** bind **`{integration-action:…}`** (see **`mcp-tool-resolution.md`**) or **`{agent-action:…}`** (see **`agent-action-resolution.md`**). **AQA** Path A and **TestGen** Phases 1 and 6 load this for **routing** (guided vs questionnaire) only. + +## B — `agents/mcp-capability.yaml` (workspace root) + +Read before the first MCP call. Keys optional; unknown keys ignored. + +```yaml +mcp: + mode: capable # required if file exists: capable | absent + jira: true + confluence: true + testrail: true + atlassian_confluence: true +``` + +- **`confluence` vs `atlassian_confluence`:** **Not aliases** — both refer to **Confluence** MCP on/off for **guided vs questionnaire**, but **different workflows read different keys** (TestGen uses `confluence`; AQA data collection uses `atlassian_confluence`). If you use **both** families, set **both** to the same boolean. If you use **one** family, you may omit the key the other workflow reads. Template: `instructions/r2/core/templates/mcp-capability.example.yaml`. + +- **`mode: capable`** — MCP allowed. Per integration: **`true`** or key omitted → **guided**. **`false`** → **questionnaire** (no MCP; user text). Read **`agents/user-instructions/mcp-guidance.md`** first when it exists. Do not re-ask values already in that file unless missing. +- **`mode: absent`** — Do not invoke MCP. Parent workflow runs questionnaire (**STOP**, **WAIT**), merges into plan / `raw-data.md` / agreed artifact. Never fabricate MCP output. +- **File missing** — Apply **A** if the user message overrides; else **one** question whose **scope** (which integrations) is defined by the **parent workflow**, not this fragment. **Example — TestGen Phase 1:** when `agents/mcp-capability.yaml` is absent, **Step 2b** in `testgen-flow-data-collection.md` asks once for **Jira and Confluence together**; **TestRail** is resolved again in **TestGen Phase 6**. **No** → questionnaire for the integrations that question covers. **Yes** → capable for that run; recommend adding YAML. + +## A — user message (wins over file) + +Record in state/plan. **Absent:** `MCP absent`, `no MCP`, `without MCP`, `questionnaire only`, `paste-only`. **Capable:** `MCP capable`, `use MCP`, `MCP enabled`, `pull from Jira` when clearly a live pull. User wins on conflict. + +## Capable mode without guidance + +Use task text and YAML only; questionnaire **only** for missing fields. Do not invent secrets. + +## Questionnaire (`absent` or integration `false`) + +Parent supplies **numbered** questions for fields MCP would return. **STOP**; **WAIT**. Merge into the same artifacts as the MCP path. Label **user-provided (no MCP)**. + +## Recording + +When routing is resolved, write into the phase plan or state file: either **one** line `MCP interaction: guided | questionnaire (source: agents/mcp-capability.yaml | user override | default question)` when a single scope applies, or **one line per integration** (or per parent-defined field such as TestGen `Jira source` / `Confluence source` / Phase 6 TestRail export), each with the same `(source: ...)` options. + +**Before leaving this fragment:** Re-read the plan or state lines you wrote for MCP routing and confirm: **(a)** each line lists exactly one source among `agents/mcp-capability.yaml`, `user override`, and `default question`; **(b)** using one aggregate line vs one line per integration (or per parent-defined field such as TestGen `Jira source` / `Confluence source`) matches what the **parent workflow** requires. Correct the artifact if a check fails. + +## Path conventions (Rosetta vs target workspace) + +Use whenever a workflow mentions both Rosetta artifacts and workspace files. + +| Location | Path | Notes | +|----------|------|--------| +| **Rosetta repo / distribution only** | `instructions/r2/core/templates/mcp-capability.example.yaml` | Template for **authoring** `mcp-capability.yaml`. Not expected inside an arbitrary customer application checkout unless copied from Rosetta. | +| **Target workspace** | `agents/mcp-capability.yaml` | Optional capability file; **expected at workspace root** only. | +| **Target workspace** | `agents/user-instructions/mcp-guidance.md` | Optional per-workspace MCP hints. | + +If **`agents/mcp-capability.yaml`** is missing at workspace root, treat capability config as **missing** for routing: follow the parent workflow’s questionnaire / default-question path and record **MCP interaction source** accordingly (do not search the customer repo for Rosetta template paths). + +## Related workflows (placeholder binding) + +- **`mcp-tool-resolution.md`** — Map **`{integration-action:…}`** to MCP tool ids when **guided**. +- **`agent-action-resolution.md`** — Map **`{agent-action:…}`** to host read / write / patch tools. diff --git a/plugins/core-copilot/commands/mcp-tool-resolution.md b/plugins/core-copilot/commands/mcp-tool-resolution.md new file mode 100644 index 00000000..8695ba68 --- /dev/null +++ b/plugins/core-copilot/commands/mcp-tool-resolution.md @@ -0,0 +1,34 @@ +--- +name: mcp-tool-resolution +description: Binds `{integration-action:…}` tokens to MCP tool identifiers in guided runs using a single-pass scan of the session tool list. +alwaysApply: false +baseSchema: docs/schemas/phase.md +--- + +# `{integration-action:…}` resolution (MCP tools) + +Use when a parent workflow is **guided** for an integration and must turn a **`{integration-action:…}`** token into a concrete MCP tool call. **ACQUIRE only when needed** (not for questionnaire-only paths). Routing (guided vs questionnaire) is defined in **`mcp-capability-interaction.md`**. + +Parent workflows define token spellings and parameter blocks. **Questionnaire** integrations must **not** use this file to call MCP. + +## Deterministic tool selection (single pass over MCP list) + +Let `T` = the list of MCP tool names (or ids) available in **this** session. Perform **exactly one** linear scan over `T` (no second pass, no “try again” loops). + +1. **Required substrings** — From the token after `integration-action:`, take segments split by `-` (e.g. `jira-get-issue` → `jira`, `get`, `issue`). Build candidate set **C** = tools in `T` whose identifier **contains every remaining segment** (case-insensitive). If **C** is empty, drop segments in this order until **C** is non-empty or one segment remains: drop **`get`**, then **`search`**, then **`list`** or **`pages`**, then repeatedly drop the **rightmost** remaining segment. +2. **If C has exactly one tool** — use it. +3. **If C has more than one tool** — Prefer tools explicitly allowed or named in **`agents/mcp-capability.yaml`** and **`agents/user-instructions/mcp-guidance.md`** for that integration. If still more than one, **STOP** and ask **one** disambiguation question using the **format below** (no guess). +4. **If C is empty after step 1 reductions** — **Do not invent** a tool. Record resolution failure in state, then follow **Questionnaire** rules for that integration in **`mcp-capability-interaction.md`** (**STOP**, **WAIT**), then continue per the parent workflow’s routing table. + +Map **Parameters** from the phase `Action` blocks to the chosen tool’s schema (adapt snake_case vs camelCase per tool docs). + +## Disambiguation question format + +When step **3** leaves multiple candidates, use exactly this shape: + +1. One line stating the token and that multiple MCP tools matched. +2. Numbered lines **`1`** through **`N`**: each line is **one verbatim tool id** from the remaining candidate set (same spelling as in `T`). +3. Line **`N+1`**: `Cancel — use questionnaire for this integration instead`. +4. **Parse the reply:** accept only an integer **k**. If **k = N+1**, do not call MCP for that integration; follow the parent’s questionnaire path. If **1 ≤ k ≤ N**, select the **k**-th listed tool in order. Any other reply → ask again once with the same list. + +Vendor tool names in examples elsewhere are **illustrations**; **`T`** is authoritative. diff --git a/plugins/core-copilot/commands/testgen-flow-data-collection.md b/plugins/core-copilot/commands/testgen-flow-data-collection.md index 4696f7d5..3e14221a 100644 --- a/plugins/core-copilot/commands/testgen-flow-data-collection.md +++ b/plugins/core-copilot/commands/testgen-flow-data-collection.md @@ -9,9 +9,12 @@ baseSchema: docs/schemas/phase.md ## Prerequisites -- MUST be starting new test generation flow -- User provided Jira ticket key or URL -- Jira MCP configured and accessible +- MUST be starting new test generation flow. +- User provided Jira ticket key or URL. +- **Acquire workflows:** ACQUIRE **`mcp-capability-interaction.md`** FROM KB (MCP guided vs questionnaire; always for Phase 1). Before the first guided **`{integration-action:*}`** call in Steps 3–4, ACQUIRE **`mcp-tool-resolution.md`**. Before **Persist** blocks using **`{agent-action:*}`** in Steps 5–6, ACQUIRE **`agent-action-resolution.md`**. If an alias fails, use canonical paths under **`instructions/r2/core/workflows/`** with the same basename. +- **Step 2b only:** Run the numbered **Step 2b checklist** in this file (not the whole fragment for later steps unless needed). +- **Before MCP calls:** Do **not** call Jira/Confluence MCPs until Step **2b** completes. +- **URL placeholders:** `{jira-host}` and `{confluence-host}` in examples mean the customer’s real web hostnames (including subdomain), not fixed vendor domains. ## Objective @@ -19,35 +22,29 @@ Extract all relevant data from Jira ticket and related Confluence documentation ## Requirements +**Token resolution (canonical):** **`{integration-action:*}`** → **`mcp-tool-resolution.md`**. **`{agent-action:*}`** → **`agent-action-resolution.md`**. MCP guided vs questionnaire and recording → **`mcp-capability-interaction.md`** (ACQUIRE per **Prerequisites**). Later steps say **see Token resolution** instead of repeating this. + ### Step 1: Parse Initial User Input **Extract from user's initial prompt**: -1. **Jira ticket**: Key or URL (REQUIRED) -2. **Confluence URLs**: List of URLs (REQUIRED) +1. **Jira ticket**: Key or URL (REQUIRED). +2. **Confluence URLs** (optional): Parse and store for later steps. How they affect routing is defined **only** in the **Step 2b** matrix (after `Jira source` / `Confluence source` exist). **Supported formats**: ``` "Analyze requirements for PROJ-123" -"Analyze requirements for PROJ-123 with Confluence: https://confluence.com/display/PROJ/Page" +"Analyze requirements for PROJ-123 with Confluence: https://{confluence-host}/display/PROJ/Page" "Analyze PROJ-123, Confluence pages: URL1, URL2, URL3" -"PROJ-123 + https://confluence.com/display/PROJ/Auth" +"PROJ-123 + https://{confluence-host}/display/PROJ/Auth" ``` **Parse Confluence URLs**: - Extract from patterns: "with Confluence", "Confluence:", "Confluence docs:", "Confluence pages:" - Accept comma-separated or line-separated URLs - URLs may be: - - Display format: `https://confluence.company.com/display/SPACE/Page+Title` - - Direct format: `https://confluence.company.com/pages/viewpage.action?pageId=123456` - - Short format: `https://confluence.company.com/x/AbCdEf` - -**If Confluence URLs provided**: -- Extract page IDs from URLs -- Skip automatic search (Step 3) -- Go directly to retrieving specified pages - -**If no Confluence URLs provided**: -- Proceed with automatic search (Step 3) + - Display format: `https://{confluence-host}/display/SPACE/Page+Title` + - Direct format: `https://{confluence-host}/pages/viewpage.action?pageId=123456` + - Short format: `https://{confluence-host}/x/AbCdEf` ### Step 2: Setup Output Directory @@ -57,48 +54,122 @@ agents/testgen/{TICKET-KEY}/ └── testgen-state.md (initialize) ``` +### Step 2b: Resolve MCP routing (user message vs capability file) + +**Precedence:** **A first** — fragment **section A** (user / task text **wins** over the file). **B second** — fragment **section B** (`agents/mcp-capability.yaml`). Full rules: **`mcp-capability-interaction.md`** (ACQUIRE per **Prerequisites**). + +**Checklist (execute in order before any Jira/Confluence MCP call in Phase 1):** + +1. Confirm **Prerequisites** ACQUIRE list for this phase is loaded (**`mcp-capability-interaction.md`**, and before guided MCP / Persist: **`mcp-tool-resolution.md`**, **`agent-action-resolution.md`** as listed there). +2. Read **`agents/mcp-capability.yaml`** at workspace root if present; if absent, treat capability config as **missing** (see **`mcp-capability-interaction.md`** Path conventions). +3. Apply fragment **§ A — user message** (task text wins over file). +4. If YAML is **missing** and the task text does not override MCP usage: ask **one** question — “Use live MCP for Jira and Confluence in this workspace?” — **STOP**, **WAIT**. **No** → both integrations **questionnaire** for this run. **Yes** → both **guided**; recommend adding `agents/mcp-capability.yaml`. +5. Derive **`Jira source`** and **`Confluence source`** using fragment **§ B** (`mode`, `mcp.jira`, `mcp.confluence`, omitted-key defaults). +6. Record in `agents/testgen/{TICKET-KEY}/testgen-state.md` under Phase 1: `Jira source`, `Confluence source`, and **MCP interaction source:** `agents/mcp-capability.yaml` **or** `user override` **or** `default question`. +7. If **`agents/user-instructions/mcp-guidance.md`** exists and at least one source is **guided**, read it before the first Jira/Confluence MCP call in Phase 1. + +#### Routing matrix (after Step 2b) + +Use `Jira source` and `Confluence source` from state. **Step labels:** **3Q** = Jira questionnaire (no Jira MCP); **3A** = Jira guided MCP; **4Q** = Confluence questionnaire; **4A** = Confluence guided MCP. **“URLs in Step 1?”** = Confluence URL(s) were parsed from the **initial** user prompt in Step 1. + +| Jira source | Confluence source | URLs in Step 1? | Execute | +|-------------|-------------------|-----------------|---------| +| guided | guided | yes | **3A** → **4A** Option A | +| guided | guided | no | **3A** → **4A** Option B | +| guided | questionnaire | — | **3A** → **4Q** | +| questionnaire | guided | yes | **3Q** → **4A** Option A | +| questionnaire | guided | no | **3Q** → **4A** Option B | +| questionnaire | questionnaire | — | **3Q** → **4Q** | + +#### `{integration-action:…}` intent reference (Phase 1) + +Use with **Token resolution** and **`mcp-tool-resolution.md`** when mapping tokens to host MCP tools. + +| Token | Intent | +|-------|--------| +| `{integration-action:jira-get-issue}` | Load one issue by key with field list, expand flags, comment limit | +| `{integration-action:jira-search-fields}` | Discover Jira field IDs / API names (custom fields) | +| `{integration-action:confluence-get-page}` | Load one Confluence page by id (markdown/metadata options) | +| `{integration-action:confluence-search-pages}` | Search Confluence pages (query + limit) | +| `{integration-action:confluence-list-child-pages}` | List child pages for a parent (content options) | + ### Step 3: Extract Jira Ticket Data -**Use**: Jira and/or Confluence MCPs respectively, snippets below will contain example pseudo-function calls for better understanding `mcp_Jira_MCP_jira_get_issue`, `mcp_Jira_MCP_confluence_get_page`, `mcp_Jira_MCP_confluence_search`, `mcp_Jira_MCP_confluence_get_page_children`. +Pick **exactly one** subsection below using the **Step 2b** matrix (`Jira source` column). **Do not** read, summarize, or execute the other subsection (**progressive disclosure**). + +#### Step 3Q: Jira questionnaire (no MCP) + +**Jira MCP:** **off** — no **`{integration-action:jira-*}`** calls; user-provided content only. -**Extract ticket key** from user input: -- Format: "PROJ-123" or URL "https://jira.company.com/browse/PROJ-123" -- Parse key from URL if needed +1. Ask **numbered** questions so you can build the Jira section of `raw-data.md`: ticket summary, description (or ask user to paste export), status, priority, labels, components, key links, and any acceptance criteria they rely on. +2. **STOP** and **WAIT** for answers. +3. Write `agents/testgen/{TICKET-KEY}/raw-data.md` (create if needed) with `## Jira (user-provided, MCP absent)` containing merged answers. For any field the user could not supply or was unsure about, add an explicit **`unknown`**, **`unverified`**, or short **why missing** note next to that item (do not imply MCP-grade certainty). Then continue per the **Step 2b** matrix to Step 4. -**Retrieve issue** with comprehensive fields: -```python -mcp_Jira_MCP_jira_get_issue( - issue_key="PROJ-123", - fields="summary,description,status,issuetype,assignee,priority,reporter,labels,components,created,updated", - expand="renderedFields", - comment_limit=10 -) +#### Step 3A: Guided Jira (MCP) + +1. **Extract ticket key** from user input: + - Format: "PROJ-123" or URL "https://{jira-host}/browse/PROJ-123" + - Parse key from URL if needed + +2. **Retrieve issue** — see **Token resolution**, **`mcp-tool-resolution.md`**, and **Intent reference** above for **`{integration-action:jira-get-issue}`**. Then call the resolved tool; logical parameters: +```text +Action: {integration-action:jira-get-issue} +Parameters: + issue_key: PROJ-123 + fields: summary,description,status,issuetype,assignee,priority,reporter,labels,components,created,updated + expand: renderedFields + comment_limit: 10 ``` -**Capture**: -- Summary, description (both raw and rendered) -- Issue type, status, priority -- Labels, components -- Assignee, reporter -- Comments (up to 10 recent) -- Created/updated dates -- Custom fields if present (epic link, story points, etc.) +**Example (mapping only):** On a common Atlassian host the resolved tool is `mcp__atlassian__jira_get_issue`; rename keys from the block above as required (e.g. `issue_key` → `issueKey`, `comment_limit` → `commentLimit`). Other tokens: see **Token resolution**. + +3. **Capture** (for Step 5 / `raw-data.md`): + - Summary, description (both raw and rendered) + - Issue type, status, priority + - Labels, components + - Assignee, reporter + - Comments (up to 10 recent) + - Created/updated dates + - Custom fields if present (epic link, story points, etc.) + +#### Step 3A.4 — Jira field schema (optional) + +Run **only** when substep **3** leaves custom-field **API names or ids** ambiguous for `raw-data.md`. + +1. **Invoke** **`{integration-action:jira-search-fields}`** **once** (see **Token resolution**). +2. **Output:** merge the returned field metadata into working notes and into **`### Custom Fields`** (and related) when executing **Step 5** (`raw-data.md`). If substep 3 was sufficient, **skip** this entire **3A.4** subsection. ### Step 4: Get Confluence Documentation -**Decision Point**: Did user provide Confluence URLs in initial prompt? +Pick **exactly one** subsection below using the **Step 2b** matrix (`Confluence source` column; for **4A**, use the **URLs in Step 1?** column for Option A vs B). **Do not** read, summarize, or execute the other subsection (**progressive disclosure**). + +#### Step 4Q: Confluence questionnaire (no MCP) + +**Confluence MCP:** **off** — no **`{integration-action:confluence-*}`** calls; user-provided content only. -#### Option A: User Provided Confluence URLs +1. Ask **numbered** questions: paste each page body (or export), or provide URLs plus authorized excerpts, parent/child relationships if relevant. +2. **STOP** and **WAIT**. +3. Append to `agents/testgen/{TICKET-KEY}/raw-data.md` under `## Confluence (user-provided, MCP absent)` with merged answers. Mark gaps, pasted excerpts of unknown completeness, or user-stated uncertainty the same way as Step 3Q (**`unknown` / `unverified` / reason**). If Jira questionnaire already created the file, append; otherwise create the file with this section. +4. Continue to **Step 5** so `raw-data.md` matches the full template (headings, metadata). Then run **Validation**. + +#### Step 4A: Guided Confluence (MCP) + +**Option A vs B:** From the **Step 2b** matrix (**URLs in Step 1?** column). + +Resolve each **`{integration-action:confluence-*}`** (see **Token resolution**; e.g. `confluence-get-page` → `mcp__atlassian__confluence_get_page` with `pageId` / `convertToMarkdown` when the host uses that idiom). + +##### Option A: User-provided Confluence URLs -**If URLs provided in initial prompt**: 1. Parse page IDs from URLs 2. For each URL, extract: - Page ID from URL parameters (pageId=123456) - Or use space + title from display URL -3. Retrieve pages directly using `mcp_Jira_MCP_confluence_get_page()` +3. Retrieve pages using **`{integration-action:confluence-get-page}`** 4. Check each page for child pages (REQUIRED) 5. Skip automatic search +**Illustrative resolved call (Atlassian-style host, not mandatory):** `mcp__atlassian__confluence_get_page` with arguments such as `{ "pageId": "123456", "convertToMarkdown": true }` (exact keys per host mapping). + **Tell user**: ``` ✅ Using provided Confluence pages: @@ -107,9 +178,9 @@ mcp_Jira_MCP_jira_get_issue( 🔍 Checking for child pages... ``` -#### Option B: No URLs Provided - Auto-Search +##### Option B: No URLs — auto-search -**Use**: `mcp_Jira_MCP_confluence_search()` +**Use**: **`{integration-action:confluence-search-pages}`** **Extract search terms** from Jira ticket: - Project key (from ticket key) @@ -123,35 +194,37 @@ type=page AND space={PROJECT_KEY} AND (text ~ "{term1}" OR text ~ "{term2}") ``` **Search Confluence**: -```python -mcp_Jira_MCP_confluence_search( - query=cql_query, - limit=10 -) +```text +Action: {integration-action:confluence-search-pages} +Parameters: + query: + limit: 10 ``` +**Illustrative resolved call (Atlassian-style host, not mandatory):** `mcp__atlassian__confluence_search_pages` (or host-equivalent) with CQL in the query argument, e.g. `type=page AND space=PROJ AND (text ~ "auth" OR text ~ "login")`. + **Rank results** by relevance: - Title matches ticket terms - Labels match ticket labels - Content matches key terms **Get top 3-5 pages**: -```python -mcp_Jira_MCP_confluence_get_page( - page_id=page_id, - convert_to_markdown=True, - include_metadata=True -) +```text +Action: {integration-action:confluence-get-page} +Parameters: + page_id: + convert_to_markdown: true + include_metadata: true ``` **IMPORTANT: Check for child pages** (nested documents often missed by search): -```python -mcp_Jira_MCP_confluence_get_page_children( - parent_id=page_id, - include_content=True, - convert_to_markdown=True, - limit=10 -) +```text +Action: {integration-action:confluence-list-child-pages} +Parameters: + parent_id: + include_content: true + convert_to_markdown: true + limit: 10 ``` For each parent page found: @@ -186,6 +259,19 @@ For each parent page found: **File**: `agents/testgen/{TICKET-KEY}/raw-data.md` +**Persist** — see **Token resolution** for **`{agent-action:write-file}`** / **`{agent-action:patch-file}`**: + +```text +Action: {agent-action:write-file} +Parameters: + path: agents/testgen/{TICKET-KEY}/raw-data.md + content: +``` + +**Illustrative host mapping (Cursor-style agent, not mandatory):** tool `Write` with `file_path` / `contents` (or host-equivalent keys after resolution). + +If the file already exists and you are only appending, you may use **`{agent-action:patch-file}`** instead (see **Token resolution**). + **Format**: ```markdown # Raw Data - [TICKET-KEY] @@ -193,6 +279,7 @@ For each parent page found: **Extracted**: [DateTime] **Phase**: 1 - Data Collection **Confluence Source**: [User-provided URLs / Auto-search / User-provided after search / Skipped] +**Confidence / Unknowns**: [If either source was questionnaire: list unverified, missing, or user-uncertain items; else `MCP-backed` or `n/a`] --- @@ -279,12 +366,22 @@ For each parent page found: - **Total Content Size**: [Approximate word count] - **Search Terms Used**: [List] - **Notes**: [Any issues during extraction] +- **Confidence / Unknowns**: [Same as header line; required when any questionnaire path ran] ``` ### Step 6: Update State File **File**: `agents/testgen/{TICKET-KEY}/testgen-state.md` +**Persist** — same as Step 5 (**see Token resolution**): + +```text +Action: {agent-action:write-file} +Parameters: + path: agents/testgen/{TICKET-KEY}/testgen-state.md + content: +``` + **Create initial state**: ```markdown # Test Generation State - [TICKET-KEY] @@ -329,18 +426,26 @@ For each parent page found: Before completing Phase 1, verify: - ✅ `agents/testgen/{TICKET-KEY}/` directory exists -- ✅ `raw-data.md` created with Jira section populated +- ✅ Phase 1 block in `testgen-state.md` contains lines **`Jira source`**, **`Confluence source`**, and **`MCP interaction source`** +- ✅ If **`Jira source` is `questionnaire`**: `raw-data.md` contains `## Jira (user-provided, MCP absent)` +- ✅ If **`Confluence source` is `questionnaire`**: `raw-data.md` contains `## Confluence (user-provided, MCP absent)` +- ✅ If any **questionnaire** path ran: `raw-data.md` **Data Collection Summary** includes **`Confidence / Unknowns`** +- ✅ `raw-data.md` created with Jira section populated (MCP or questionnaire path) - ✅ Confluence section has at least 1 page OR user confirmed skip - ✅ `testgen-state.md` created with Phase 1 marked complete - ✅ All key Jira fields captured (summary, description, status, priority) ## Tools Used -- `mcp_Jira_MCP_jira_get_issue()` - Jira ticket extraction -- `mcp_Jira_MCP_confluence_search()` - Confluence page search -- `mcp_Jira_MCP_confluence_get_page()` - Confluence page content retrieval -- `mcp_Jira_MCP_confluence_get_page_children()` - Confluence child page discovery -- `write()` - File creation +| Token / use | Where | +|---------------|--------| +| `{integration-action:jira-get-issue}` | Step 3A.2 | +| `{integration-action:jira-search-fields}` | Step 3A.4 (conditional) | +| `{integration-action:confluence-search-pages}` | Step 4A Option B | +| `{integration-action:confluence-get-page}` | Step 4A Options A / B | +| `{integration-action:confluence-list-child-pages}` | Step 4A Option B | +| `{agent-action:write-file}` | Steps 5–6 (**Persist** blocks) | +| `{agent-action:patch-file}` | Steps 5–6 (optional append / edit) | ## Common Issues @@ -354,10 +459,10 @@ Before completing Phase 1, verify: **Solution**: Include first 5000 words, note truncation in raw-data.md **Issue**: Custom fields not recognized -**Solution**: Use `mcp_Jira_MCP_jira_search_fields()` to discover field names +**Solution**: Run **Step 3A.4** — **`{integration-action:jira-search-fields}`** (or equivalent) to discover field names **Issue**: Confluence search finds parent but misses child pages -**Solution**: Always check for child pages using `confluence_get_page_children()` for each found page +**Solution**: Always check for child pages using **`{integration-action:confluence-list-child-pages}`** for each found page **Issue**: User provided invalid Confluence URL **Solution**: Try to parse page ID, if fails ask user for correct URL or page ID @@ -378,8 +483,6 @@ After Phase 1 completion: - Confluence search may need tuning based on organization's Confluence structure - Some Jira instances have custom fields - capture all available - Confluence pages may be in different spaces - search broadly initially -- **User can provide Confluence URLs in initial prompt** - this skips auto-search -- If user provides specific page URLs/IDs, use those directly instead of search - **CRITICAL**: Always check for child pages - nested documentation often contains the most relevant details - Example: "Job Post" parent may have children "Create a Job Post", "Edit a Job Post", etc. - Retrieve up to 10 child pages per parent, prioritize by relevance to ticket diff --git a/plugins/core-copilot/commands/testgen-flow-test-case-export.md b/plugins/core-copilot/commands/testgen-flow-test-case-export.md index e2cce06c..7e9164fc 100644 --- a/plugins/core-copilot/commands/testgen-flow-test-case-export.md +++ b/plugins/core-copilot/commands/testgen-flow-test-case-export.md @@ -12,26 +12,70 @@ baseSchema: docs/schemas/phase.md - Phase 5 MUST be complete - `agents/testgen/{TICKET-KEY}/test-scenarios.md` exists with test cases - User has reviewed and edited test cases -- TestRail MCP configured and accessible -- TestRail project_id and suite_id known +- **TestRail export path (B + A):** Either **guided** (TestRail MCP allowed per `mcp-capability-interaction.md` and `agents/mcp-capability.yaml`) or **questionnaire** (no TestRail MCP; user supplies IDs and performs or confirms manual import). Optional **`agents/user-instructions/mcp-guidance.md`** when guided. +- **Placeholder binding:** Before first guided **`{integration-action:*}`** call, ACQUIRE **`mcp-tool-resolution.md`**. For **`{agent-action:*}`** file steps, ACQUIRE **`agent-action-resolution.md`**. Canonical: **`instructions/r2/core/workflows/.md`** if KB alias fails. +- For **guided** runs: TestRail MCP configured and accessible; `project_id` and `suite_id` known (from guidance, state, or user). ## Objective -Export test cases from test-scenarios.md to TestRail using MCP, creating a new section (folder) for the Jira ticket and adding all test cases. -If MCP is not available, ask user how to proceed and provide possible options. +Export test cases from `test-scenarios.md` to TestRail. + +- **Guided:** Use TestRail MCP to verify the project, resolve `section_id`, and create cases. +- **Questionnaire:** Do **not** run any **`{integration-action:testrail-*}`** call. Collect `project_id`, `suite_id`, and `section_id` from the user, write a **manual export pack** (copy-paste / CSV-friendly), and record completion when the user confirms import or pastes back TestRail case IDs. + +If the interaction mode is unclear, resolve it in **Step 0** before any MCP call. ## TestRail Configuration **Default Configuration**: detect using current ticket and user profile. **Section Name**: [TICKET-KEY] (e.g., "PROJ-123") +**Web URL placeholders:** `{testrail-suite-view-url}`, `{testrail-case-view-url}`, and `{testrail-suite-cases-grouped-url}` stand in for full TestRail browser links (`https://` + host + `index.php?` paths + ids). Compose from `{TestRail instance host}`, `suite_id`, case id, and section grouping as in Step 1Q or `mcp-guidance.md`. + +### Guided TestRail and workspace file actions (placeholders) + +| Token | Intent | +|-------|--------| +| `{integration-action:testrail-get-project}` | Verify TestRail project access by `project_id` | +| `{integration-action:testrail-get-cases}` | List cases in a project/suite (optional dedup before add) | +| `{integration-action:testrail-add-case}` | Create a test case in a `section_id` with title, priority, type, refs, separated steps | +| `{agent-action:read-file}` | Read a workspace file (host-specific file tool) | +| `{agent-action:write-file}` | Create or overwrite a workspace file | +| `{agent-action:patch-file}` | Apply targeted edits to a workspace file (includes search-and-replace style edits) | + +Resolve **`{integration-action:…}`** using **`mcp-tool-resolution.md`** (guided only). Resolve **`{agent-action:…}`** using **`agent-action-resolution.md`**. ## Requirements -### Step 1: Verify TestRail Connection +### Step 0: Resolve TestRail MCP interaction (B + A) -**Test connection** using: -```python -mcp_testrail_get_project(project_id=69) +1. ACQUIRE **`mcp-capability-interaction.md`** FROM KB. ACQUIRE **`mcp-tool-resolution.md`** before any **`{integration-action:*}`** MCP call (when **guided**). ACQUIRE **`agent-action-resolution.md`** before any **`{agent-action:*}`** workspace step. If KB alias fails, use **`instructions/r2/core/workflows/.md`**. +2. Read **`agents/mcp-capability.yaml`** if present; apply **user override** from the current task text (same rules as Phase 1). +3. Derive **`TestRail export: guided | questionnaire`**: + - **`mcp.mode: absent`** → questionnaire. + - **`mcp.mode: capable`** and **`mcp.testrail: false`** (when the key is present) → questionnaire. + - **`mcp.mode: capable`** and (`testrail` omitted or `true`) → guided. + - Missing YAML: use **user override** if any; else **one** short question (“Use TestRail MCP for export?”). **No** → questionnaire; **Yes** → guided and recommend adding `agents/mcp-capability.yaml`. +4. Record in `agents/testgen/{TICKET-KEY}/testgen-state.md` (Phase 6): + - `TestRail export: guided | questionnaire` + - `MCP interaction source:` (`agents/mcp-capability.yaml` | user override | default question`) +5. If **guided** and **`agents/user-instructions/mcp-guidance.md`** exists, read it before the first **`{integration-action:testrail-*}`** call. + +### Step 1: Verify TestRail connection (guided only) + +**If `TestRail export` is `questionnaire`:** Run **Step 1Q** instead of this step. + +#### Step 1Q: Questionnaire — no MCP verification + +1. Ask **numbered** questions: `project_id`, `suite_id` (for links and documentation), and confirm the user will import cases manually or paste back IDs after import. +2. **STOP** and **WAIT**. +3. Skip to **Step 2** (section_id flow is unchanged; user may already know `section_id`). + +**Guided path — test connection** using (replace `project_id` with value from guidance/state/user; **do not** hard-code example IDs unless the user confirmed them): + +```text +Action: {integration-action:testrail-get-project} +Parameters: + project_id: ``` If fails, tell user: @@ -39,7 +83,7 @@ If fails, tell user: ❌ TestRail connection failed. Please verify: 1. TestRail MCP is configured 2. Credentials are correct -3. Project ID 69 exists and you have access +3. Project ID exists and you have access ``` ### Step 2: Create Section in TestRail @@ -57,7 +101,7 @@ If you already have a section, provide the section_id. You can find it in the URL when viewing a section (e.g., group_id=94686 or section_id=94686) **Option B: Create new section** -1. Go to: https://griddynamics.testrail.io/index.php?/suites/view/3300 +1. Go to: {testrail-suite-view-url} 2. Click "Add Section" 3. Name it: [TICKET-KEY] (e.g., "PROJ-456") 4. After creating, find the section_id in the URL or section details @@ -224,19 +268,40 @@ Execute this test case for EACH row in the table below: - Just include original preconditions normally - No "Execute for EACH row" note -### Step 5: Add Test Cases to TestRail +### Step 5: Add Test Cases to TestRail (guided only) -**For each mapped test case**: +**If `TestRail export` is `questionnaire`:** Run **Step 5Q** only; **do not** run **`{integration-action:testrail-add-case}`**. -```python -result = mcp_testrail_add_case( - section_id=section_id, - title=testrail_case["title"], - priority_id=testrail_case["priority_id"], - type_id=testrail_case["type_id"], - refs=testrail_case["refs"], - custom_steps_separated=testrail_case["custom_steps_separated"] -) +#### Step 5Q: Manual export pack (no MCP) + +1. After **Step 4** mapping, write **`agents/testgen/{TICKET-KEY}/testrail-manual-export.md`** containing: + - User-supplied `project_id`, `suite_id`, `section_id` (if known) + - One block per test case: title, priority, type, refs, preconditions text, and steps (content + expected) in plain text or markdown tables suitable for copy-paste into TestRail. +2. Optionally add **`agents/testgen/{TICKET-KEY}/testrail-import-hints.csv`** (title, priority, type, refs, preconditions, steps_json) if the team uses CSV import. +3. **STOP** and **WAIT**: ask the user to confirm either (a) they created cases manually / imported the pack, or (b) they paste back a mapping `TC-001 → C12345` (or URLs). +4. **After the user replies**, execute **Step 6** to merge IDs into `test-scenarios.md`. Until then you may leave placeholders (e.g. `Pending`) only if the workflow must save intermediate files; do **not** mark Phase 6 complete in state until Step 6 reflects the user’s answer. + +**Optional (guided only) — list existing cases before add:** If duplicate titles, re-exports, or unclear existing coverage are a concern, run **`{integration-action:testrail-get-cases}`** once against the target `project_id` / `suite_id` (and use section or other filters when your MCP supports them) to compare with parsed markdown. **Not required** for a first-time export or when the user accepts duplicates. + +```text +Action: {integration-action:testrail-get-cases} +Parameters: + project_id: + suite_id: +``` + +**Guided path — for each mapped test case**: + +```text +Action: {integration-action:testrail-add-case} +Parameters: + section_id: + title: + priority_id: + type_id: + refs: + custom_steps_separated: +Outcome: capture returned case id for results tracking ``` **Track results**: @@ -261,12 +326,14 @@ results = { **Update**: `agents/testgen/{TICKET-KEY}/test-scenarios.md` +For **questionnaire** exports, use user-pasted IDs when available; otherwise keep **`TestRail ID`**: `Manual (see testrail-manual-export.md)`** until the user confirms. Use real `project` / `suite` / base URL values from Step 1Q or guidance — **do not** invent instance hostnames. + **Add TestRail IDs to each test case**: ```markdown ### TC-001: User Login with Valid Credentials (Happy Path) **TestRail ID**: C12345 ✅ -**TestRail Link**: https://griddynamics.testrail.io/index.php?/cases/view/12345 +**TestRail Link**: {testrail-case-view-url} **Related Requirement**: US-1, FR-1 ... ``` @@ -283,11 +350,11 @@ results = { ## TestRail Export Summary **Exported**: [DateTime] -**Project**: 69 -**Suite**: 3300 +**Project**: {project_id} +**Suite**: {suite_id} **Section**: [TICKET-KEY] (ID: [section_id]) **Total Exported**: [X] test cases -**TestRail Link**: https://griddynamics.testrail.io/index.php?/suites/view/3300&group_by=cases:section_id +**TestRail Link**: {testrail-suite-cases-grouped-url} | TC ID | TestRail ID | Status | |-------|-------------|--------| @@ -326,8 +393,8 @@ results = { ### Phase 6: TestRail Export - **Completed**: [DateTime] -- **Project ID**: 69 -- **Suite ID**: 3300 +- **Project ID**: {project_id} +- **Suite ID**: {suite_id} - **Section ID**: [section_id] - **Section Name**: [TICKET-KEY] - **Test Cases Created**: [Count] @@ -339,21 +406,30 @@ results = { ## Validation Before completing Phase 6, verify: -- ✅ TestRail connection successful -- ✅ Section exists in TestRail + +**Guided path:** +- ✅ TestRail connection successful (`{integration-action:testrail-get-project}` or equivalent) +- ✅ Section exists in TestRail (or user provided valid `section_id`) - ✅ All test cases parsed from markdown -- ✅ At least 80% of test cases exported successfully -- ✅ test-scenarios.md updated with TestRail IDs +- ✅ At least 80% of test cases exported successfully via MCP (or failures documented) +- ✅ `test-scenarios.md` updated with TestRail IDs - ✅ State file updated with Phase 6 complete - ✅ TestRail link provided to user +**Questionnaire path:** +- ✅ Step 0 recorded `TestRail export: questionnaire` and source line +- ✅ User confirmed `project_id` / `suite_id` / `section_id` as needed +- ✅ `testrail-manual-export.md` (and optional CSV) written +- ✅ User confirmed manual import or pasted ID mapping +- ✅ `test-scenarios.md` updated with IDs or explicit “manual / pending” notes +- ✅ State file updated with Phase 6 complete + ## Tools Used -- `mcp_testrail_get_project(project_id)` - Verify connection -- `mcp_testrail_get_cases(project_id, suite_id)` - Check existing cases -- `mcp_testrail_add_case(section_id, title, ...)` - Create test cases -- `read_file()` - Read test-scenarios.md -- `write()` / `search_replace()` - Update files +- **Guided:** `{integration-action:testrail-get-project}` — verify connection; `{integration-action:testrail-get-cases}` — optional dedup; `{integration-action:testrail-add-case}` — create test cases +- **Questionnaire:** `{agent-action:read-file}` / `{agent-action:write-file}` / `{agent-action:patch-file}` only — build `testrail-manual-export.md` and update docs; **no** `{integration-action:testrail-*}` calls +- `{agent-action:read-file}` — read `test-scenarios.md` +- `{agent-action:write-file}` / `{agent-action:patch-file}` — update files ## Common Issues @@ -375,6 +451,9 @@ Before completing Phase 6, verify: **Issue**: Rate limit exceeded **Solution**: Add delay between API calls, batch requests +**Issue**: Risk of duplicate titles or unclear existing coverage +**Solution**: Optionally run **`{integration-action:testrail-get-cases}`** (Step 5 optional block) before **`{integration-action:testrail-add-case}`** — optional discovery of existing cases, not a mandatory gate. + **Issue**: Test case already exists **Solution**: Create anyway (TestRail allows duplicates), note in report @@ -393,7 +472,7 @@ All 6 phases finished successfully: ✅ Phase 6: TestRail Export ([R] cases exported) **TestRail Section**: [TICKET-KEY] -**TestRail Link**: https://griddynamics.testrail.io/index.php?/suites/view/3300 +**TestRail Link**: {testrail-suite-view-url} **Deliverables**: 📄 requirements.md - Use for implementation @@ -411,8 +490,8 @@ All 6 phases finished successfully: - TestRail MCP currently lacks section creation - user must create manually - Test case IDs in TestRail are prefixed with "C" (e.g., C12345) -- Suite ID 3300 is from user's TestRail URL -- Project ID 69 is user's default project +- `{suite_id}` comes from the user, `mcp-guidance.md`, or the suite view URL on their instance +- `{project_id}` comes from the user, `mcp-guidance.md`, or the TestRail project settings - Parameterized test data is included in step content or expected results - Re-running export creates duplicate test cases (by design, to preserve history) - Consider creating test run after export for immediate execution diff --git a/plugins/core-copilot/commands/testgen-flow.md b/plugins/core-copilot/commands/testgen-flow.md index 1e334ca6..de88e4c8 100644 --- a/plugins/core-copilot/commands/testgen-flow.md +++ b/plugins/core-copilot/commands/testgen-flow.md @@ -14,6 +14,7 @@ Systematic requirements analysis from Jira tickets and Confluence documentation ### Critical Requirements +- **MCP ROUTING (B + A)**: Phases 1 and 6 may call external MCPs only when **`agents/mcp-capability.yaml`** and **`mcp-capability-interaction.md`** resolve **guided** for that integration. Otherwise use **questionnaire** or manual export paths documented in those phases. Optional **`agents/user-instructions/mcp-guidance.md`** for IDs and guardrails when guided. **`{integration-action:*}`** / **`{agent-action:*}`** binding: follow each phase file’s **Prerequisites** (**`mcp-tool-resolution.md`**, **`agent-action-resolution.md`**). - **ONE PHASE AT A TIME**: Read phase file, execute, update state, move to next. - **DO NOT SKIP PHASES**: Each builds on previous. - **STATE TRACKING**: Update `agents/testgen/{TICKET-KEY}/testgen-state.md` after each phase. @@ -31,9 +32,10 @@ Systematic requirements analysis from Jira tickets and Confluence documentation **Phase 1: Data Collection** [testgen-flow-data-collection.md] 1. ACQUIRE testgen-flow-data-collection.md FROM KB -2. Execute phase instructions -3. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` -4. Validate by listing raw-data.md file +2. ACQUIRE **`mcp-capability-interaction.md`**, and per that phase’s **Prerequisites** also **`mcp-tool-resolution.md`** / **`agent-action-resolution.md`** when placeholders are used +3. Execute phase instructions +4. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` +5. Validate by listing raw-data.md file **Phase 2: Gap & Contradiction Analysis** [testgen-flow-gap-and-contradiction-analysis.md] 1. ACQUIRE testgen-flow-gap-and-contradiction-analysis.md FROM KB @@ -61,8 +63,9 @@ Systematic requirements analysis from Jira tickets and Confluence documentation **Phase 6: Test Case Export** [testgen-flow-test-case-export.md] ⭐ 1. ACQUIRE testgen-flow-test-case-export.md FROM KB -4. Execute phase instructions -5. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` +2. ACQUIRE **`mcp-capability-interaction.md`** and per that phase’s **Step 0** / **Prerequisites** also **`mcp-tool-resolution.md`** / **`agent-action-resolution.md`** when placeholders are used (guided vs questionnaire for TestRail export) +3. Execute phase instructions +4. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` ## State File Format @@ -136,9 +139,9 @@ agents/testgen/{TICKET-KEY}/ ## Prerequisites -- **Jira MCP**: Configured and accessible -- **Jira Ticket**: User provides ticket key or link -- **Confluence Access**: Via Jira MCP (same authentication) +- **MCP routing (B + A):** `agents/mcp-capability.yaml` (see `instructions/r2/core/templates/mcp-capability.example.yaml`) plus optional `agents/user-instructions/mcp-guidance.md` control **guided** vs **questionnaire** per integration. Phases 1 and 6 embed the branching rules. +- **Jira ticket**: User provides ticket key or link (Phase 1) +- **Confluence**: Either MCP-guided discovery (Phase 1) or user-provided content when **questionnaire** - **Output Directory**: Created at start (`agents/testgen/{TICKET-KEY}/`) ## Common Patterns diff --git a/plugins/core-cursor-standalone/.cursor/commands/agent-action-resolution.md b/plugins/core-cursor-standalone/.cursor/commands/agent-action-resolution.md new file mode 100644 index 00000000..75fb60a9 --- /dev/null +++ b/plugins/core-cursor-standalone/.cursor/commands/agent-action-resolution.md @@ -0,0 +1,18 @@ +--- +name: agent-action-resolution +description: Maps `{agent-action:…}` tokens to host IDE or agent-runtime file tools (read, write, patch). +alwaysApply: false +baseSchema: docs/schemas/phase.md +--- + +# `{agent-action:…}` resolution (host file tools) + +Use when a workflow uses **`{agent-action:read-file}`**, **`{agent-action:write-file}`**, or **`{agent-action:patch-file}`**. These are **not** remote MCP calls and are **not** covered by **`mcp-tool-resolution.md`**. + +| Token | Intent | +|-------|--------| +| `{agent-action:read-file}` | Read a file from the workspace via the host agent’s file-read tool | +| `{agent-action:write-file}` | Create or overwrite a workspace file via the host agent’s file-write tool | +| `{agent-action:patch-file}` | Apply targeted edits via the host agent’s patch / search-replace tool | + +**Resolution:** Map each token to the host IDE’s or agent runtime’s **documented** file operation (names differ: e.g. `Read`, `Write`, `StrReplace`, `apply_patch`). If the host exposes no matching capability, **STOP**, tell the user, and **WAIT** — do not substitute an MCP tool. diff --git a/plugins/core-cursor-standalone/.cursor/commands/aqa-flow-data-collection.md b/plugins/core-cursor-standalone/.cursor/commands/aqa-flow-data-collection.md index 34fc0898..a9e7253d 100644 --- a/plugins/core-cursor-standalone/.cursor/commands/aqa-flow-data-collection.md +++ b/plugins/core-cursor-standalone/.cursor/commands/aqa-flow-data-collection.md @@ -9,19 +9,61 @@ baseSchema: docs/schemas/phase.md ## Objective -Gather all required information from external sources (TestRail and Confluence) to understand test requirements and expected behavior. +Gather everything needed for later phases: either from **TestRail and Confluence** (**integrated AQA**) or from **user-supplied artifacts** (**minimal-input / agent-led**). Always record the chosen **execution mode** in the test plan and state file. ## Prerequisites -- TestRail MCP configured and accessible -- Atlassian (Confluence) MCP configured and accessible -- Test case ID or requirement provided by user +- User intent is clear enough to start Phase 1 (IDs, URLs, or a plain-language automation goal). +- **Integrated AQA path**: For **guided** MCP interaction (see Task 0b), TestRail MCP and Atlassian (Confluence) MCP must be usable when you run Path A MCP steps. For **questionnaire** interaction, do **not** call those MCPs; collect the same facts via user answers and paste into the test plan. +- **Minimal-input path**: No MCP requirement for TestRail/Confluence; Task 0b **does not apply** to Path B for those integrations. If something on the checklist is missing, **ask** — do not guess. + +## MCP capability reference + +Resolve **guided vs questionnaire** per **`mcp-capability-interaction.md`** (ACQUIRE FROM KB). Config file: **`agents/mcp-capability.yaml`** (copy from `instructions/r2/core/templates/mcp-capability.example.yaml` in Rosetta repo). Optional guidance: **`agents/user-instructions/mcp-guidance.md`**. ## Phase Tasks -### Task 1: Read TestRail Test Case +### Task 0: Confirm execution mode + +**Actions**: +1. Check whether the **current task text** (or a prior user message in the same session the user instructs you to treat as binding) already names the mode **explicitly**, using any of these (case-insensitive): `integrated`, `minimal-input`, `minimal input`, `Integrated AQA`, or `minimal-input / agent-led`. The token must be **user-authored** in that message (not something you paraphrase or infer). If yes, set the mode to `integrated` or `minimal-input` accordingly and **skip steps 2–3**; go directly to **Path A** or **Path B** matching that mode. +2. If **not** explicit: **Do not infer** mode from clues such as missing TestRail IDs, presence of HTML attachments, “sandbox,” “no MCP,” or vague phrases like “I don’t have TestRail yet.” **Forbidden:** choosing `minimal-input` because TestRail was omitted, the repo looks small, or the user only supplied local HTML—those are **not** mode labels. Informal phrases (“lightweight,” “without TestRail,” “bring my own selectors”) **do not** count as explicit; they still require steps 2–3. **Ask once** with both options, for example: + - **Integrated AQA**: "We pull the official case and docs from TestRail/Confluence and follow the full chain." + - **Minimal-input / agent-led**: "You provide URLs, page source or a selector map, scenario and expectations; we still run phases 2–8 in order but Phase 1 records your package instead of MCP pulls." +3. **STOP** and **WAIT** until the user replies with a clear **integrated** or **minimal-input** (or the spelled labels in step 1). If the reply is still ambiguous, ask **one** narrowing question that still requires those labels; **do not** choose a path without those words from the user. +4. **Hard gate — before Path A or Path B:** Do **not** create or update `agents/plans/aqa-*.md`, do **not** write `**AQA execution mode**` in any artifact, and do **not** mark Phase 1 complete in `agents/aqa-state.md` until step **1** matched or step **3** completed. If you already started Path A/B without satisfying Task 0, **stop**, discard partial mode lines, run steps 2–3, then continue. +5. Ensure the Phase 1 test plan created in Path A or Path B includes `**AQA execution mode**` and that `agents/aqa-state.md` reflects the mode when Phase 1 is marked complete (see **Update State File** below). + +**Expected Output**: Mode is either read from **explicit user keywords** (step 1) or obtained only after **mandatory** ask-and-wait (steps 2–3); then proceed with **Path A** or **Path B** (not both). + +--- + +### Task 0b: Resolve MCP interaction (Path A only) + +**Skip this entire task for Path B (minimal-input).** **Actions**: +1. ACQUIRE **mcp-capability-interaction.md** FROM KB (if not already loaded this phase). +2. Read **`agents/mcp-capability.yaml`** if it exists; apply **user override** rules from that fragment (task text beats file). If file missing and user has not overridden, ask **one** yes/no: “Use live MCP for TestRail/Confluence in this workspace?” — **WAIT**; treat **No** as questionnaire, **Yes** as guided for both integrations until a YAML file exists. +3. Derive **per integration** (TestRail vs Confluence) whether the source is **guided** (MCP allowed) or **questionnaire** (no MCP): if `mcp.mode` is `absent`, both are **questionnaire**; if `capable`, use `mcp.testrail` / `mcp.atlassian_confluence` when present (boolean); if a key is **omitted** under `capable`, default that integration to **guided**. +4. Record in the test plan and `agents/aqa-state.md` Phase 1 notes: `TestRail source: guided|questionnaire`, `Confluence source: guided|questionnaire`, plus `MCP interaction source:` (`agents/mcp-capability.yaml` | user override | default question`). +5. **Questionnaire leg:** For each integration still **questionnaire** and not yet documented in the plan, run **one** numbered questionnaire (combine TestRail + Confluence in a single message if both need data). **STOP** and **WAIT**. Merge answers under **User-provided (MCP absent) — TestRail** and/or **User-provided (MCP absent) — Confluence** in the working plan (or append to the plan draft before Task A3). +6. **Guided leg:** If **`agents/user-instructions/mcp-guidance.md`** exists and at least one integration is **guided**, read it before the first MCP call for a **guided** integration. Then continue to Task A1 / A2 and follow the **guided** branches there only. + +**Expected Output**: TestRail and Confluence are each classified **guided** or **questionnaire**; questionnaire content is captured before MCP calls; no MCP call runs for a **questionnaire** integration. + +--- + +### Path A — Integrated AQA (TestRail + Confluence) + +**Prerequisite:** Task 0 already set execution mode to **integrated** (step 1 keyword match or step 3 user reply). Do not enter Path A without that. + +### Task A1: Read TestRail Test Case + +**Actions**: +- **If Task 0b resolved questionnaire for TestRail:** Ensure the test plan already contains **User-provided (MCP absent) — TestRail** content from Task 0b. If missing, **STOP**, ask numbered questions (case ID for traceability, title, steps, expected results, preconditions), **WAIT**, merge into the plan. **Do not** call `user-testrail-get_case`. Skip the guided-only steps below for TestRail. +- **If Task 0b resolved guided for TestRail:** Perform steps 1–4. + 1. Ask user for TestRail test case ID if not provided 2. Use TestRail MCP to retrieve test case details: ``` @@ -39,9 +81,12 @@ Gather all required information from external sources (TestRail and Confluence) **Expected Output**: Complete understanding of what needs to be tested according to TestRail. -### Task 2: Read Confluence Documentation +### Task A2: Read Confluence Documentation **Actions**: +- **If Task 0b resolved questionnaire for Confluence:** Ensure the test plan contains **User-provided (MCP absent) — Confluence** content from Task 0b. If missing, **STOP**, ask numbered questions (URLs, pasted excerpts, or summaries the user authorizes), **WAIT**, merge into the plan. **Do not** call `user-mcp-atlassian-confluence_*`. Skip guided-only steps below for Confluence. +- **If Task 0b resolved guided for Confluence:** Perform steps 1–5. + 1. Ask user for Confluence page ID/URL or search terms if not provided 2. Use Atlassian Confluence MCP to find related documentation: ``` @@ -60,10 +105,11 @@ Gather all required information from external sources (TestRail and Confluence) **Expected Output**: Business and technical context for the feature being tested. -### Task 3: Create Initial Test Plan Document +### Task A3: Create initial test plan document (integrated) **Actions**: 1. Create `agents/plans/aqa-.md` file with: + - **AQA execution mode**: `integrated` - Test case reference (TestRail ID and link) - Feature name and description - Test goal @@ -77,6 +123,7 @@ Gather all required information from external sources (TestRail and Confluence) # AQA Test Plan - **Created**: [DateTime] +**AQA execution mode**: integrated **TestRail Case**: [ID/URL] **Feature**: [Feature Name] **Status**: Phase 1 Complete @@ -134,14 +181,112 @@ Gather all required information from external sources (TestRail and Confluence) [To be filled in Phase 6] ``` +--- + +### Path B — Minimal-input / agent-led + +**Prerequisite:** Task 0 already set execution mode to **minimal-input** (step 1 keyword match or step 3 user reply). Do not enter Path B because TestRail was missing or files were attached—only because the user chose **minimal-input** in text. + +### Task B1: Offer agent-led implementation and required artifacts + +**Actions**: +1. Tell the user clearly that the agent **can** implement the test **without** TestRail/Confluence **if** they provide enough grounded detail — but the agent **must not** invent pages, selectors, or flows. +2. List what you still need at minimum (see Task B2 checklist). **WAIT** until the user either supplies items or explicitly defers an item to Phase 2 (then note "open in Phase 2" in the plan). + +**Expected Output**: User understands the tradeoff and what to supply (or what is deferred to Phase 2 with explicit consent). + +### Task B2: Collect minimal-input package + +**Actions**: +1. Capture the following in the test plan (fill or mark *deferred to Phase 2* with user agreement): + - **Scenario / goal**: What behavior the automated test must prove (plain language). + - **Entry / URLs**: Starting URL(s) or navigation path to reach the UI under test. + - **UI grounding** (at least one required before ending Phase 1 unless user explicitly defers to Phase 2): e.g. saved page source or DOM snapshot paths, a **selector map** (purpose → locator), pointers to existing Page Objects in the repo, and/or frontend file paths the agent will use in Phase 3. + - **Preconditions and test data**: Accounts, feature flags, seed data — or *unknown — Phase 2*. + - **Optional**: TestRail/Confluence IDs "for traceability only" without MCP — record as references, not as substitute for missing UI grounding. +2. **Do not** mark Phase 1 complete if UI grounding and scenario goal are both missing and the user has not agreed to defer. + +**Expected Output**: Minimal-input checklist documented in the test plan. + +### Task B3: Create initial test plan document (minimal-input) + +**Actions**: +1. Create `agents/plans/aqa-.md` using the template below (adapt section headings if the scenario is small). +2. For TestRail/Confluence fields, use `N/A` or optional reference links — do not pretend MCP data exists. + +**Template**: +```markdown +# AQA Test Plan - + +**Created**: [DateTime] +**AQA execution mode**: minimal-input +**TestRail Case**: [N/A or ID for traceability only] +**Confluence**: [N/A or links for traceability only] +**Feature**: [Feature Name] +**Status**: Phase 1 Complete + +## Minimal-input package + +### Scenario / goal +[What the test must validate] + +### Entry and navigation +- URLs: [...] +- Steps to reach the UI: [...] + +### UI grounding +- Page source / DOM snapshots: [paths or attach instructions] +- Selector map or locators: [...] +- Repo pointers (Page Objects / components): [...] + +### Preconditions and test data +[Or: deferred to Phase 2 — user confirmed] + +### Notes and open questions +- [...] + +## Test Case Information (synthetic) + +### Test Steps (draft) +1. [Step — can be refined in Phase 2] +... + +### Expected Overall Result +[Draft — refined in Phase 2] + +--- +## Phase 2: Requirements Clarification +[To be filled in Phase 2] + +## Phase 3: Code Analysis +[To be filled in Phase 3] + +## Phase 4: Selector Identification +[To be filled in Phase 4] + +## Phase 5: Selector Implementation +[To be filled in Phase 5] + +## Phase 6: Test Implementation +[To be filled in Phase 6] +``` + ## Completion Criteria +**All modes** +- [ ] Task 0 complete: execution mode is **either** named by explicit user keywords (`integrated` / `minimal-input` / spelled equivalents) **or** obtained after the mandatory ask-and-wait; mode recorded in test plan and state +- [ ] Test plan file `agents/plans/aqa-.md` created +- [ ] Test goal clearly understood (or explicitly deferred to Phase 2 with user consent) +- [ ] `agents/aqa-state.md` updated with Phase 1 completion + +**Integrated AQA only** - [ ] TestRail test case retrieved and documented - [ ] Confluence documentation retrieved and documented -- [ ] Test plan file created with all Phase 1 information -- [ ] Test goal clearly understood -- [ ] Expected results documented -- [ ] `agents/aqa-state.md` updated with Phase 1 completion +- [ ] Expected results from sources documented + +**Minimal-input only** +- [ ] Minimal-input checklist captured in the test plan +- [ ] UI grounding present or explicitly deferred to Phase 2 with user approval ## Update State File @@ -150,8 +295,10 @@ After completing Phase 1, update `agents/aqa-state.md`: ```markdown ### Phase 1: Data Collection - Completed: [DateTime] -- TestRail Case: [ID/URL] -- Confluence Pages: [URLs] +- Execution mode: [integrated | minimal-input] +- TestRail Case: [ID/URL or N/A] +- Confluence Pages: [URLs or N/A] +- Minimal-input checklist: [brief summary or N/A] - Test Goal: [Brief description] - Expected Result: [Brief description] - Test Plan File: agents/plans/aqa-.md @@ -168,7 +315,9 @@ ACQUIRE aqa-phase2-md FROM KB ## Important Notes -- **No Assumptions**: If TestRail or Confluence data is incomplete, note it in the test plan -- **Ask Questions**: If user hasn't provided IDs/URLs, ask for them -- **Document Everything**: Capture all details even if they seem minor -- **Cross-Reference**: Ensure TestRail and Confluence information aligns +- **No Assumptions**: If data is incomplete for the chosen mode, note it in the test plan and ask — never guess selectors or flows. +- **Ask Questions**: If user hasn't provided IDs/URLs (integrated) or checklist items (minimal-input), ask for them. +- **Document Everything**: Capture all details even if they seem minor. +- **Integrated path only**: Cross-reference TestRail and Confluence information for alignment. +- **Minimal-input path**: Prefer an explicit **selector map** or page source over vague descriptions; deferrals must be user-visible in the plan. +- **Task 0**: Never infer execution mode from context; only explicit keywords or a user reply after the mandatory question count. diff --git a/plugins/core-cursor-standalone/.cursor/commands/aqa-flow-requirements-clarification.md b/plugins/core-cursor-standalone/.cursor/commands/aqa-flow-requirements-clarification.md index f0d21565..f7fb9a9b 100644 --- a/plugins/core-cursor-standalone/.cursor/commands/aqa-flow-requirements-clarification.md +++ b/plugins/core-cursor-standalone/.cursor/commands/aqa-flow-requirements-clarification.md @@ -13,16 +13,23 @@ Fill gaps in understanding, clarify unknowns, and define explicit assertions bef ## Prerequisites -- Phase 1 completed -- Test plan file created with TestRail and Confluence data -- Initial understanding of test requirements +- Phase 1 completed with a test plan file +- Phase 1 documents either **integrated** sources (TestRail + Confluence as used) **or** **minimal-input** execution mode with the checklist from Phase 1 (scenario, URLs, UI grounding or explicit deferral) +- Initial understanding of test requirements (may be draft in minimal-input mode) ## Phase Tasks +### Minimal-input mode (read first) + +If the test plan shows **`AQA execution mode: minimal-input`**: +- Treat Phase 1's checklist as authoritative for what the user already supplied; **do not** re-demand TestRail/Confluence data unless the user wants it for traceability. +- Still run gap analysis: anything deferred from Phase 1 **must** be resolved or explicitly re-deferred here with user text. +- Assertions must still be explicit and measurable before Phase 3; **never** invent selectors the user did not supply, defer, or point to in repo/page source. + ### Task 1: Review Gathered Information for Gaps **Actions**: -1. Read the test plan file from Phase 1 +1. Read the test plan file from Phase 1 and note **AQA execution mode** (`integrated` vs `minimal-input`). 2. Analyze information for completeness: - Are test steps clear and unambiguous? - Are expected results specific and measurable? @@ -197,3 +204,4 @@ ACQUIRE aqa-phase3-md FROM KB - **Explicit Over Implicit**: Every assertion must be measurable and verifiable - **User Authority**: User has final say on requirements and expected behavior - **Document Everything**: Record all questions and answers for traceability +- **Minimal-input mode**: Questions should close gaps in the Phase 1 checklist (especially UI grounding and test data), not re-litigate integrated AQA unless the user switches mode explicitly diff --git a/plugins/core-cursor-standalone/.cursor/commands/aqa-flow-test-implementation.md b/plugins/core-cursor-standalone/.cursor/commands/aqa-flow-test-implementation.md index a16d294e..e92ab613 100644 --- a/plugins/core-cursor-standalone/.cursor/commands/aqa-flow-test-implementation.md +++ b/plugins/core-cursor-standalone/.cursor/commands/aqa-flow-test-implementation.md @@ -13,13 +13,16 @@ Create automated test following project standards, integrating all Page Objects ## Prerequisites -- All previous phases (1-5) completed -- Test plan fully documented with all details -- Page Objects updated with all required selectors +- All previous phases (1-5) completed **for the chosen execution mode** (see below) +- Test plan fully documented with all details **including** `AQA execution mode` and, if `minimal-input`, the **minimal-input checklist** from Phase 1 (and any Phase 2 resolutions of deferrals) +- Page Objects updated with all required selectors **or** a clear plan from Phases 4–5 that matches user-supplied grounding (minimal-input) - Explicit assertions defined - Code architecture understood -## Phase Tasks +### Execution modes and prerequisites + +- **Integrated AQA**: The bullets above mean phases 1–5 were completed in full with TestRail/Confluence-backed Phase 1 where applicable. +- **Minimal-input / agent-led** (recorded in Phase 1): Phases 1–5 count as satisfied only if the test plan contains the **minimal-input checklist** (scenario, URLs/navigation, UI grounding or approved Phase 2 deferrals) and **explicit assertions from Phase 2**. **Before writing code**, re-read that checklist and Phase 2 assertions; if anything material is still missing, **STOP**, ask the user, and do not invent selectors, pages, or flows. ### Task 1: Review Test Implementation Plan diff --git a/plugins/core-cursor-standalone/.cursor/commands/aqa-flow.md b/plugins/core-cursor-standalone/.cursor/commands/aqa-flow.md index 00f7c725..f7931333 100644 --- a/plugins/core-cursor-standalone/.cursor/commands/aqa-flow.md +++ b/plugins/core-cursor-standalone/.cursor/commands/aqa-flow.md @@ -10,20 +10,23 @@ baseSchema: docs/schemas/workflow.md ## Context -This agent handles end-to-end test automation from requirements gathering to test implementation. It uses TestRail, Confluence, and project documentation to create automated tests following existing architecture and coding standards. +This agent handles end-to-end test automation from requirements gathering to test implementation. It uses TestRail, Confluence, and project documentation when you run **integrated AQA**; in **minimal-input / agent-led** mode the user supplies pages, DOM or selector details, and scenario context instead (see Phase 1). In all modes, tests follow existing architecture and coding standards. ### Critical Requirements - **ONE PHASE AT A TIME**: Read phase file, execute, update state, move to next. - **DO NOT SKIP PHASES**: Each builds on previous. - **NO ASSUMPTIONS**: Never assume selectors, flows, or data. Always ask the user if information is missing. -- **USER INTERACTION**: Wait for user responses when questions are asked or files are requested. Phase 2, 6, 7, and 8 always require user input. Phase 4 requires user input ONLY if frontend code is unavailable or selectors cannot be found +- **MCP ROUTING (B + A)**: Before calling external MCPs in integrated Path A, follow **`mcp-capability-interaction`** using `agents/mcp-capability.yaml` and any user override in the task text. **Guided** = use MCP after optional `agents/user-instructions/mcp-guidance.md`. **Questionnaire** = no MCP calls for that integration; user answers structured questions; record answers in the plan. ACQUIRE `mcp-capability-interaction.md` FROM KB when executing Phase 1 Path A. +- **EXECUTION MODE (Phase 1, first)**: The user must **explicitly** choose **integrated** or **minimal-input** (case-insensitive), or use the spelled labels **Integrated AQA** or **minimal-input / agent-led**, in **their** message before you record a mode. If none of those appear, **ask once** with both options, **STOP**, and **WAIT**—**do not infer** mode from missing TestRail, attachments, sandbox, or “no MCP.” **Do not** write `agents/plans/aqa-*.md` or set `**AQA execution mode**` until that keyword match or the user’s reply to the question. **Forbidden:** assigning `minimal-input` because TestRail was omitted or the project looks like a demo. After the user answers, document the mode in the Phase 1 test plan and `agents/aqa-state.md`. Do not silently invent missing integrations or missing UI grounding. User customizations elsewhere do **not** waive this rule when mode was not explicitly named. +- **USER INTERACTION**: Wait for user responses when questions are asked or files are requested. **Phase 1** requires user input until execution mode is explicit or the user answers the one-time mode question (see **EXECUTION MODE**). Phase 2, 6, 7, and 8 always require user input. Phase 4 requires user input ONLY if frontend code is unavailable or selectors cannot be found - **STATE TRACKING**: Update `agents/aqa-state.md` after each phase. - **MUST** use todo tasks for tracking progress. - Prioritize ACCURACY over SPEED! ### User Customizations +- The **execution mode** rule cannot be waived by “only run some phases” unless the user message already contains the mode keywords or the user answers the one-time mode question. - If user did not specify any preferences perform all steps except optional. - User CAN customize and ask only for specific phases OR phases could have been done already OR towards specific goal OR for specific case, in this case LISTEN and ADOPT to the user. @@ -31,9 +34,10 @@ This agent handles end-to-end test automation from requirements gathering to tes **Phase 1: Data Collection** [aqa-flow-data-collection.md] 1. ACQUIRE aqa-flow-data-collection.md FROM KB -2. Execute phase instructions -3. Update `agents/aqa-state.md` -4. Validate gathered data +2. ACQUIRE mcp-capability-interaction.md FROM KB (resolve MCP guided vs questionnaire before Path A MCP calls) +3. Execute phase instructions (**start with execution mode confirmation** — integrated vs minimal-input) +4. Update `agents/aqa-state.md` +5. Validate gathered data (checklist for chosen mode is complete before Phase 2) **Phase 2: Requirements Clarification** [aqa-flow-requirements-clarification.md] ⭐ **USER INTERACTION REQUIRED** 1. ACQUIRE aqa-flow-requirements-clarification.md FROM KB @@ -91,7 +95,8 @@ Create/update `agents/aqa-state.md` after each phase: **Last Updated**: [DateTime] **Current Phase**: [1-8 or COMPLETE] -**TestRail Case**: [Test Case ID/URL] +**AQA execution mode**: [integrated | minimal-input] +**TestRail Case**: [Test Case ID/URL or N/A in minimal-input] **Feature**: [Feature Name] ## Phase Completion Status @@ -109,8 +114,10 @@ Create/update `agents/aqa-state.md` after each phase: ### Phase 1: Data Collection - Completed: [DateTime] -- TestRail Case: [ID/URL] -- Confluence Pages: [URLs] +- Execution mode: [integrated | minimal-input] +- TestRail Case: [ID/URL or N/A] +- Confluence Pages: [URLs or N/A] +- Minimal-input checklist: [summary or N/A — see Phase 1 test plan] - Test Goal: [Brief description] - Expected Result: [Brief description] @@ -158,6 +165,7 @@ Create/update `agents/aqa-state.md` after each phase: ## Important Notes - **Sequential Execution**: Phases build on each other, must execute in order. +- **Execution modes**: **Integrated AQA** uses TestRail and Confluence in Phase 1 when those tools are in scope. **Minimal-input / agent-led** uses the Phase 1 checklist (pages, DOM or selectors, scenario context); later phases still run in order, but Phase 1 documents what substitutes for external case management. Without an **explicit** user mode label (`integrated` or `minimal-input` / spelled equivalents), Phase 1 must obtain that label by asking—never infer from context alone. - **No Assumptions Rule**: Always ask user when information is missing - never guess selectors, flows, or test data. - **Architecture First**: Always analyze existing code structure before implementing new tests. - **Reuse Over Creation**: Prefer adding to existing files and using existing Page Objects over creating new ones. diff --git a/plugins/core-cursor-standalone/.cursor/commands/mcp-capability-interaction.md b/plugins/core-cursor-standalone/.cursor/commands/mcp-capability-interaction.md new file mode 100644 index 00000000..4f4fae4d --- /dev/null +++ b/plugins/core-cursor-standalone/.cursor/commands/mcp-capability-interaction.md @@ -0,0 +1,64 @@ +--- +name: mcp-capability-interaction +description: Routes guided vs questionnaire MCP usage from workspace YAML and user text, records interaction source, and defines where capability files live. Placeholder binding uses sibling workflows. +alwaysApply: false +baseSchema: docs/schemas/phase.md +--- + +# MCP capability interaction (shared) + +Use when a workflow **may** call **remote MCPs** (Jira, Confluence, TestRail, Atlassian). It does **not** bind **`{integration-action:…}`** (see **`mcp-tool-resolution.md`**) or **`{agent-action:…}`** (see **`agent-action-resolution.md`**). **AQA** Path A and **TestGen** Phases 1 and 6 load this for **routing** (guided vs questionnaire) only. + +## B — `agents/mcp-capability.yaml` (workspace root) + +Read before the first MCP call. Keys optional; unknown keys ignored. + +```yaml +mcp: + mode: capable # required if file exists: capable | absent + jira: true + confluence: true + testrail: true + atlassian_confluence: true +``` + +- **`confluence` vs `atlassian_confluence`:** **Not aliases** — both refer to **Confluence** MCP on/off for **guided vs questionnaire**, but **different workflows read different keys** (TestGen uses `confluence`; AQA data collection uses `atlassian_confluence`). If you use **both** families, set **both** to the same boolean. If you use **one** family, you may omit the key the other workflow reads. Template: `instructions/r2/core/templates/mcp-capability.example.yaml`. + +- **`mode: capable`** — MCP allowed. Per integration: **`true`** or key omitted → **guided**. **`false`** → **questionnaire** (no MCP; user text). Read **`agents/user-instructions/mcp-guidance.md`** first when it exists. Do not re-ask values already in that file unless missing. +- **`mode: absent`** — Do not invoke MCP. Parent workflow runs questionnaire (**STOP**, **WAIT**), merges into plan / `raw-data.md` / agreed artifact. Never fabricate MCP output. +- **File missing** — Apply **A** if the user message overrides; else **one** question whose **scope** (which integrations) is defined by the **parent workflow**, not this fragment. **Example — TestGen Phase 1:** when `agents/mcp-capability.yaml` is absent, **Step 2b** in `testgen-flow-data-collection.md` asks once for **Jira and Confluence together**; **TestRail** is resolved again in **TestGen Phase 6**. **No** → questionnaire for the integrations that question covers. **Yes** → capable for that run; recommend adding YAML. + +## A — user message (wins over file) + +Record in state/plan. **Absent:** `MCP absent`, `no MCP`, `without MCP`, `questionnaire only`, `paste-only`. **Capable:** `MCP capable`, `use MCP`, `MCP enabled`, `pull from Jira` when clearly a live pull. User wins on conflict. + +## Capable mode without guidance + +Use task text and YAML only; questionnaire **only** for missing fields. Do not invent secrets. + +## Questionnaire (`absent` or integration `false`) + +Parent supplies **numbered** questions for fields MCP would return. **STOP**; **WAIT**. Merge into the same artifacts as the MCP path. Label **user-provided (no MCP)**. + +## Recording + +When routing is resolved, write into the phase plan or state file: either **one** line `MCP interaction: guided | questionnaire (source: agents/mcp-capability.yaml | user override | default question)` when a single scope applies, or **one line per integration** (or per parent-defined field such as TestGen `Jira source` / `Confluence source` / Phase 6 TestRail export), each with the same `(source: ...)` options. + +**Before leaving this fragment:** Re-read the plan or state lines you wrote for MCP routing and confirm: **(a)** each line lists exactly one source among `agents/mcp-capability.yaml`, `user override`, and `default question`; **(b)** using one aggregate line vs one line per integration (or per parent-defined field such as TestGen `Jira source` / `Confluence source`) matches what the **parent workflow** requires. Correct the artifact if a check fails. + +## Path conventions (Rosetta vs target workspace) + +Use whenever a workflow mentions both Rosetta artifacts and workspace files. + +| Location | Path | Notes | +|----------|------|--------| +| **Rosetta repo / distribution only** | `instructions/r2/core/templates/mcp-capability.example.yaml` | Template for **authoring** `mcp-capability.yaml`. Not expected inside an arbitrary customer application checkout unless copied from Rosetta. | +| **Target workspace** | `agents/mcp-capability.yaml` | Optional capability file; **expected at workspace root** only. | +| **Target workspace** | `agents/user-instructions/mcp-guidance.md` | Optional per-workspace MCP hints. | + +If **`agents/mcp-capability.yaml`** is missing at workspace root, treat capability config as **missing** for routing: follow the parent workflow’s questionnaire / default-question path and record **MCP interaction source** accordingly (do not search the customer repo for Rosetta template paths). + +## Related workflows (placeholder binding) + +- **`mcp-tool-resolution.md`** — Map **`{integration-action:…}`** to MCP tool ids when **guided**. +- **`agent-action-resolution.md`** — Map **`{agent-action:…}`** to host read / write / patch tools. diff --git a/plugins/core-cursor-standalone/.cursor/commands/mcp-tool-resolution.md b/plugins/core-cursor-standalone/.cursor/commands/mcp-tool-resolution.md new file mode 100644 index 00000000..8695ba68 --- /dev/null +++ b/plugins/core-cursor-standalone/.cursor/commands/mcp-tool-resolution.md @@ -0,0 +1,34 @@ +--- +name: mcp-tool-resolution +description: Binds `{integration-action:…}` tokens to MCP tool identifiers in guided runs using a single-pass scan of the session tool list. +alwaysApply: false +baseSchema: docs/schemas/phase.md +--- + +# `{integration-action:…}` resolution (MCP tools) + +Use when a parent workflow is **guided** for an integration and must turn a **`{integration-action:…}`** token into a concrete MCP tool call. **ACQUIRE only when needed** (not for questionnaire-only paths). Routing (guided vs questionnaire) is defined in **`mcp-capability-interaction.md`**. + +Parent workflows define token spellings and parameter blocks. **Questionnaire** integrations must **not** use this file to call MCP. + +## Deterministic tool selection (single pass over MCP list) + +Let `T` = the list of MCP tool names (or ids) available in **this** session. Perform **exactly one** linear scan over `T` (no second pass, no “try again” loops). + +1. **Required substrings** — From the token after `integration-action:`, take segments split by `-` (e.g. `jira-get-issue` → `jira`, `get`, `issue`). Build candidate set **C** = tools in `T` whose identifier **contains every remaining segment** (case-insensitive). If **C** is empty, drop segments in this order until **C** is non-empty or one segment remains: drop **`get`**, then **`search`**, then **`list`** or **`pages`**, then repeatedly drop the **rightmost** remaining segment. +2. **If C has exactly one tool** — use it. +3. **If C has more than one tool** — Prefer tools explicitly allowed or named in **`agents/mcp-capability.yaml`** and **`agents/user-instructions/mcp-guidance.md`** for that integration. If still more than one, **STOP** and ask **one** disambiguation question using the **format below** (no guess). +4. **If C is empty after step 1 reductions** — **Do not invent** a tool. Record resolution failure in state, then follow **Questionnaire** rules for that integration in **`mcp-capability-interaction.md`** (**STOP**, **WAIT**), then continue per the parent workflow’s routing table. + +Map **Parameters** from the phase `Action` blocks to the chosen tool’s schema (adapt snake_case vs camelCase per tool docs). + +## Disambiguation question format + +When step **3** leaves multiple candidates, use exactly this shape: + +1. One line stating the token and that multiple MCP tools matched. +2. Numbered lines **`1`** through **`N`**: each line is **one verbatim tool id** from the remaining candidate set (same spelling as in `T`). +3. Line **`N+1`**: `Cancel — use questionnaire for this integration instead`. +4. **Parse the reply:** accept only an integer **k**. If **k = N+1**, do not call MCP for that integration; follow the parent’s questionnaire path. If **1 ≤ k ≤ N**, select the **k**-th listed tool in order. Any other reply → ask again once with the same list. + +Vendor tool names in examples elsewhere are **illustrations**; **`T`** is authoritative. diff --git a/plugins/core-cursor-standalone/.cursor/commands/testgen-flow-data-collection.md b/plugins/core-cursor-standalone/.cursor/commands/testgen-flow-data-collection.md index 4696f7d5..3e14221a 100644 --- a/plugins/core-cursor-standalone/.cursor/commands/testgen-flow-data-collection.md +++ b/plugins/core-cursor-standalone/.cursor/commands/testgen-flow-data-collection.md @@ -9,9 +9,12 @@ baseSchema: docs/schemas/phase.md ## Prerequisites -- MUST be starting new test generation flow -- User provided Jira ticket key or URL -- Jira MCP configured and accessible +- MUST be starting new test generation flow. +- User provided Jira ticket key or URL. +- **Acquire workflows:** ACQUIRE **`mcp-capability-interaction.md`** FROM KB (MCP guided vs questionnaire; always for Phase 1). Before the first guided **`{integration-action:*}`** call in Steps 3–4, ACQUIRE **`mcp-tool-resolution.md`**. Before **Persist** blocks using **`{agent-action:*}`** in Steps 5–6, ACQUIRE **`agent-action-resolution.md`**. If an alias fails, use canonical paths under **`instructions/r2/core/workflows/`** with the same basename. +- **Step 2b only:** Run the numbered **Step 2b checklist** in this file (not the whole fragment for later steps unless needed). +- **Before MCP calls:** Do **not** call Jira/Confluence MCPs until Step **2b** completes. +- **URL placeholders:** `{jira-host}` and `{confluence-host}` in examples mean the customer’s real web hostnames (including subdomain), not fixed vendor domains. ## Objective @@ -19,35 +22,29 @@ Extract all relevant data from Jira ticket and related Confluence documentation ## Requirements +**Token resolution (canonical):** **`{integration-action:*}`** → **`mcp-tool-resolution.md`**. **`{agent-action:*}`** → **`agent-action-resolution.md`**. MCP guided vs questionnaire and recording → **`mcp-capability-interaction.md`** (ACQUIRE per **Prerequisites**). Later steps say **see Token resolution** instead of repeating this. + ### Step 1: Parse Initial User Input **Extract from user's initial prompt**: -1. **Jira ticket**: Key or URL (REQUIRED) -2. **Confluence URLs**: List of URLs (REQUIRED) +1. **Jira ticket**: Key or URL (REQUIRED). +2. **Confluence URLs** (optional): Parse and store for later steps. How they affect routing is defined **only** in the **Step 2b** matrix (after `Jira source` / `Confluence source` exist). **Supported formats**: ``` "Analyze requirements for PROJ-123" -"Analyze requirements for PROJ-123 with Confluence: https://confluence.com/display/PROJ/Page" +"Analyze requirements for PROJ-123 with Confluence: https://{confluence-host}/display/PROJ/Page" "Analyze PROJ-123, Confluence pages: URL1, URL2, URL3" -"PROJ-123 + https://confluence.com/display/PROJ/Auth" +"PROJ-123 + https://{confluence-host}/display/PROJ/Auth" ``` **Parse Confluence URLs**: - Extract from patterns: "with Confluence", "Confluence:", "Confluence docs:", "Confluence pages:" - Accept comma-separated or line-separated URLs - URLs may be: - - Display format: `https://confluence.company.com/display/SPACE/Page+Title` - - Direct format: `https://confluence.company.com/pages/viewpage.action?pageId=123456` - - Short format: `https://confluence.company.com/x/AbCdEf` - -**If Confluence URLs provided**: -- Extract page IDs from URLs -- Skip automatic search (Step 3) -- Go directly to retrieving specified pages - -**If no Confluence URLs provided**: -- Proceed with automatic search (Step 3) + - Display format: `https://{confluence-host}/display/SPACE/Page+Title` + - Direct format: `https://{confluence-host}/pages/viewpage.action?pageId=123456` + - Short format: `https://{confluence-host}/x/AbCdEf` ### Step 2: Setup Output Directory @@ -57,48 +54,122 @@ agents/testgen/{TICKET-KEY}/ └── testgen-state.md (initialize) ``` +### Step 2b: Resolve MCP routing (user message vs capability file) + +**Precedence:** **A first** — fragment **section A** (user / task text **wins** over the file). **B second** — fragment **section B** (`agents/mcp-capability.yaml`). Full rules: **`mcp-capability-interaction.md`** (ACQUIRE per **Prerequisites**). + +**Checklist (execute in order before any Jira/Confluence MCP call in Phase 1):** + +1. Confirm **Prerequisites** ACQUIRE list for this phase is loaded (**`mcp-capability-interaction.md`**, and before guided MCP / Persist: **`mcp-tool-resolution.md`**, **`agent-action-resolution.md`** as listed there). +2. Read **`agents/mcp-capability.yaml`** at workspace root if present; if absent, treat capability config as **missing** (see **`mcp-capability-interaction.md`** Path conventions). +3. Apply fragment **§ A — user message** (task text wins over file). +4. If YAML is **missing** and the task text does not override MCP usage: ask **one** question — “Use live MCP for Jira and Confluence in this workspace?” — **STOP**, **WAIT**. **No** → both integrations **questionnaire** for this run. **Yes** → both **guided**; recommend adding `agents/mcp-capability.yaml`. +5. Derive **`Jira source`** and **`Confluence source`** using fragment **§ B** (`mode`, `mcp.jira`, `mcp.confluence`, omitted-key defaults). +6. Record in `agents/testgen/{TICKET-KEY}/testgen-state.md` under Phase 1: `Jira source`, `Confluence source`, and **MCP interaction source:** `agents/mcp-capability.yaml` **or** `user override` **or** `default question`. +7. If **`agents/user-instructions/mcp-guidance.md`** exists and at least one source is **guided**, read it before the first Jira/Confluence MCP call in Phase 1. + +#### Routing matrix (after Step 2b) + +Use `Jira source` and `Confluence source` from state. **Step labels:** **3Q** = Jira questionnaire (no Jira MCP); **3A** = Jira guided MCP; **4Q** = Confluence questionnaire; **4A** = Confluence guided MCP. **“URLs in Step 1?”** = Confluence URL(s) were parsed from the **initial** user prompt in Step 1. + +| Jira source | Confluence source | URLs in Step 1? | Execute | +|-------------|-------------------|-----------------|---------| +| guided | guided | yes | **3A** → **4A** Option A | +| guided | guided | no | **3A** → **4A** Option B | +| guided | questionnaire | — | **3A** → **4Q** | +| questionnaire | guided | yes | **3Q** → **4A** Option A | +| questionnaire | guided | no | **3Q** → **4A** Option B | +| questionnaire | questionnaire | — | **3Q** → **4Q** | + +#### `{integration-action:…}` intent reference (Phase 1) + +Use with **Token resolution** and **`mcp-tool-resolution.md`** when mapping tokens to host MCP tools. + +| Token | Intent | +|-------|--------| +| `{integration-action:jira-get-issue}` | Load one issue by key with field list, expand flags, comment limit | +| `{integration-action:jira-search-fields}` | Discover Jira field IDs / API names (custom fields) | +| `{integration-action:confluence-get-page}` | Load one Confluence page by id (markdown/metadata options) | +| `{integration-action:confluence-search-pages}` | Search Confluence pages (query + limit) | +| `{integration-action:confluence-list-child-pages}` | List child pages for a parent (content options) | + ### Step 3: Extract Jira Ticket Data -**Use**: Jira and/or Confluence MCPs respectively, snippets below will contain example pseudo-function calls for better understanding `mcp_Jira_MCP_jira_get_issue`, `mcp_Jira_MCP_confluence_get_page`, `mcp_Jira_MCP_confluence_search`, `mcp_Jira_MCP_confluence_get_page_children`. +Pick **exactly one** subsection below using the **Step 2b** matrix (`Jira source` column). **Do not** read, summarize, or execute the other subsection (**progressive disclosure**). + +#### Step 3Q: Jira questionnaire (no MCP) + +**Jira MCP:** **off** — no **`{integration-action:jira-*}`** calls; user-provided content only. -**Extract ticket key** from user input: -- Format: "PROJ-123" or URL "https://jira.company.com/browse/PROJ-123" -- Parse key from URL if needed +1. Ask **numbered** questions so you can build the Jira section of `raw-data.md`: ticket summary, description (or ask user to paste export), status, priority, labels, components, key links, and any acceptance criteria they rely on. +2. **STOP** and **WAIT** for answers. +3. Write `agents/testgen/{TICKET-KEY}/raw-data.md` (create if needed) with `## Jira (user-provided, MCP absent)` containing merged answers. For any field the user could not supply or was unsure about, add an explicit **`unknown`**, **`unverified`**, or short **why missing** note next to that item (do not imply MCP-grade certainty). Then continue per the **Step 2b** matrix to Step 4. -**Retrieve issue** with comprehensive fields: -```python -mcp_Jira_MCP_jira_get_issue( - issue_key="PROJ-123", - fields="summary,description,status,issuetype,assignee,priority,reporter,labels,components,created,updated", - expand="renderedFields", - comment_limit=10 -) +#### Step 3A: Guided Jira (MCP) + +1. **Extract ticket key** from user input: + - Format: "PROJ-123" or URL "https://{jira-host}/browse/PROJ-123" + - Parse key from URL if needed + +2. **Retrieve issue** — see **Token resolution**, **`mcp-tool-resolution.md`**, and **Intent reference** above for **`{integration-action:jira-get-issue}`**. Then call the resolved tool; logical parameters: +```text +Action: {integration-action:jira-get-issue} +Parameters: + issue_key: PROJ-123 + fields: summary,description,status,issuetype,assignee,priority,reporter,labels,components,created,updated + expand: renderedFields + comment_limit: 10 ``` -**Capture**: -- Summary, description (both raw and rendered) -- Issue type, status, priority -- Labels, components -- Assignee, reporter -- Comments (up to 10 recent) -- Created/updated dates -- Custom fields if present (epic link, story points, etc.) +**Example (mapping only):** On a common Atlassian host the resolved tool is `mcp__atlassian__jira_get_issue`; rename keys from the block above as required (e.g. `issue_key` → `issueKey`, `comment_limit` → `commentLimit`). Other tokens: see **Token resolution**. + +3. **Capture** (for Step 5 / `raw-data.md`): + - Summary, description (both raw and rendered) + - Issue type, status, priority + - Labels, components + - Assignee, reporter + - Comments (up to 10 recent) + - Created/updated dates + - Custom fields if present (epic link, story points, etc.) + +#### Step 3A.4 — Jira field schema (optional) + +Run **only** when substep **3** leaves custom-field **API names or ids** ambiguous for `raw-data.md`. + +1. **Invoke** **`{integration-action:jira-search-fields}`** **once** (see **Token resolution**). +2. **Output:** merge the returned field metadata into working notes and into **`### Custom Fields`** (and related) when executing **Step 5** (`raw-data.md`). If substep 3 was sufficient, **skip** this entire **3A.4** subsection. ### Step 4: Get Confluence Documentation -**Decision Point**: Did user provide Confluence URLs in initial prompt? +Pick **exactly one** subsection below using the **Step 2b** matrix (`Confluence source` column; for **4A**, use the **URLs in Step 1?** column for Option A vs B). **Do not** read, summarize, or execute the other subsection (**progressive disclosure**). + +#### Step 4Q: Confluence questionnaire (no MCP) + +**Confluence MCP:** **off** — no **`{integration-action:confluence-*}`** calls; user-provided content only. -#### Option A: User Provided Confluence URLs +1. Ask **numbered** questions: paste each page body (or export), or provide URLs plus authorized excerpts, parent/child relationships if relevant. +2. **STOP** and **WAIT**. +3. Append to `agents/testgen/{TICKET-KEY}/raw-data.md` under `## Confluence (user-provided, MCP absent)` with merged answers. Mark gaps, pasted excerpts of unknown completeness, or user-stated uncertainty the same way as Step 3Q (**`unknown` / `unverified` / reason**). If Jira questionnaire already created the file, append; otherwise create the file with this section. +4. Continue to **Step 5** so `raw-data.md` matches the full template (headings, metadata). Then run **Validation**. + +#### Step 4A: Guided Confluence (MCP) + +**Option A vs B:** From the **Step 2b** matrix (**URLs in Step 1?** column). + +Resolve each **`{integration-action:confluence-*}`** (see **Token resolution**; e.g. `confluence-get-page` → `mcp__atlassian__confluence_get_page` with `pageId` / `convertToMarkdown` when the host uses that idiom). + +##### Option A: User-provided Confluence URLs -**If URLs provided in initial prompt**: 1. Parse page IDs from URLs 2. For each URL, extract: - Page ID from URL parameters (pageId=123456) - Or use space + title from display URL -3. Retrieve pages directly using `mcp_Jira_MCP_confluence_get_page()` +3. Retrieve pages using **`{integration-action:confluence-get-page}`** 4. Check each page for child pages (REQUIRED) 5. Skip automatic search +**Illustrative resolved call (Atlassian-style host, not mandatory):** `mcp__atlassian__confluence_get_page` with arguments such as `{ "pageId": "123456", "convertToMarkdown": true }` (exact keys per host mapping). + **Tell user**: ``` ✅ Using provided Confluence pages: @@ -107,9 +178,9 @@ mcp_Jira_MCP_jira_get_issue( 🔍 Checking for child pages... ``` -#### Option B: No URLs Provided - Auto-Search +##### Option B: No URLs — auto-search -**Use**: `mcp_Jira_MCP_confluence_search()` +**Use**: **`{integration-action:confluence-search-pages}`** **Extract search terms** from Jira ticket: - Project key (from ticket key) @@ -123,35 +194,37 @@ type=page AND space={PROJECT_KEY} AND (text ~ "{term1}" OR text ~ "{term2}") ``` **Search Confluence**: -```python -mcp_Jira_MCP_confluence_search( - query=cql_query, - limit=10 -) +```text +Action: {integration-action:confluence-search-pages} +Parameters: + query: + limit: 10 ``` +**Illustrative resolved call (Atlassian-style host, not mandatory):** `mcp__atlassian__confluence_search_pages` (or host-equivalent) with CQL in the query argument, e.g. `type=page AND space=PROJ AND (text ~ "auth" OR text ~ "login")`. + **Rank results** by relevance: - Title matches ticket terms - Labels match ticket labels - Content matches key terms **Get top 3-5 pages**: -```python -mcp_Jira_MCP_confluence_get_page( - page_id=page_id, - convert_to_markdown=True, - include_metadata=True -) +```text +Action: {integration-action:confluence-get-page} +Parameters: + page_id: + convert_to_markdown: true + include_metadata: true ``` **IMPORTANT: Check for child pages** (nested documents often missed by search): -```python -mcp_Jira_MCP_confluence_get_page_children( - parent_id=page_id, - include_content=True, - convert_to_markdown=True, - limit=10 -) +```text +Action: {integration-action:confluence-list-child-pages} +Parameters: + parent_id: + include_content: true + convert_to_markdown: true + limit: 10 ``` For each parent page found: @@ -186,6 +259,19 @@ For each parent page found: **File**: `agents/testgen/{TICKET-KEY}/raw-data.md` +**Persist** — see **Token resolution** for **`{agent-action:write-file}`** / **`{agent-action:patch-file}`**: + +```text +Action: {agent-action:write-file} +Parameters: + path: agents/testgen/{TICKET-KEY}/raw-data.md + content: +``` + +**Illustrative host mapping (Cursor-style agent, not mandatory):** tool `Write` with `file_path` / `contents` (or host-equivalent keys after resolution). + +If the file already exists and you are only appending, you may use **`{agent-action:patch-file}`** instead (see **Token resolution**). + **Format**: ```markdown # Raw Data - [TICKET-KEY] @@ -193,6 +279,7 @@ For each parent page found: **Extracted**: [DateTime] **Phase**: 1 - Data Collection **Confluence Source**: [User-provided URLs / Auto-search / User-provided after search / Skipped] +**Confidence / Unknowns**: [If either source was questionnaire: list unverified, missing, or user-uncertain items; else `MCP-backed` or `n/a`] --- @@ -279,12 +366,22 @@ For each parent page found: - **Total Content Size**: [Approximate word count] - **Search Terms Used**: [List] - **Notes**: [Any issues during extraction] +- **Confidence / Unknowns**: [Same as header line; required when any questionnaire path ran] ``` ### Step 6: Update State File **File**: `agents/testgen/{TICKET-KEY}/testgen-state.md` +**Persist** — same as Step 5 (**see Token resolution**): + +```text +Action: {agent-action:write-file} +Parameters: + path: agents/testgen/{TICKET-KEY}/testgen-state.md + content: +``` + **Create initial state**: ```markdown # Test Generation State - [TICKET-KEY] @@ -329,18 +426,26 @@ For each parent page found: Before completing Phase 1, verify: - ✅ `agents/testgen/{TICKET-KEY}/` directory exists -- ✅ `raw-data.md` created with Jira section populated +- ✅ Phase 1 block in `testgen-state.md` contains lines **`Jira source`**, **`Confluence source`**, and **`MCP interaction source`** +- ✅ If **`Jira source` is `questionnaire`**: `raw-data.md` contains `## Jira (user-provided, MCP absent)` +- ✅ If **`Confluence source` is `questionnaire`**: `raw-data.md` contains `## Confluence (user-provided, MCP absent)` +- ✅ If any **questionnaire** path ran: `raw-data.md` **Data Collection Summary** includes **`Confidence / Unknowns`** +- ✅ `raw-data.md` created with Jira section populated (MCP or questionnaire path) - ✅ Confluence section has at least 1 page OR user confirmed skip - ✅ `testgen-state.md` created with Phase 1 marked complete - ✅ All key Jira fields captured (summary, description, status, priority) ## Tools Used -- `mcp_Jira_MCP_jira_get_issue()` - Jira ticket extraction -- `mcp_Jira_MCP_confluence_search()` - Confluence page search -- `mcp_Jira_MCP_confluence_get_page()` - Confluence page content retrieval -- `mcp_Jira_MCP_confluence_get_page_children()` - Confluence child page discovery -- `write()` - File creation +| Token / use | Where | +|---------------|--------| +| `{integration-action:jira-get-issue}` | Step 3A.2 | +| `{integration-action:jira-search-fields}` | Step 3A.4 (conditional) | +| `{integration-action:confluence-search-pages}` | Step 4A Option B | +| `{integration-action:confluence-get-page}` | Step 4A Options A / B | +| `{integration-action:confluence-list-child-pages}` | Step 4A Option B | +| `{agent-action:write-file}` | Steps 5–6 (**Persist** blocks) | +| `{agent-action:patch-file}` | Steps 5–6 (optional append / edit) | ## Common Issues @@ -354,10 +459,10 @@ Before completing Phase 1, verify: **Solution**: Include first 5000 words, note truncation in raw-data.md **Issue**: Custom fields not recognized -**Solution**: Use `mcp_Jira_MCP_jira_search_fields()` to discover field names +**Solution**: Run **Step 3A.4** — **`{integration-action:jira-search-fields}`** (or equivalent) to discover field names **Issue**: Confluence search finds parent but misses child pages -**Solution**: Always check for child pages using `confluence_get_page_children()` for each found page +**Solution**: Always check for child pages using **`{integration-action:confluence-list-child-pages}`** for each found page **Issue**: User provided invalid Confluence URL **Solution**: Try to parse page ID, if fails ask user for correct URL or page ID @@ -378,8 +483,6 @@ After Phase 1 completion: - Confluence search may need tuning based on organization's Confluence structure - Some Jira instances have custom fields - capture all available - Confluence pages may be in different spaces - search broadly initially -- **User can provide Confluence URLs in initial prompt** - this skips auto-search -- If user provides specific page URLs/IDs, use those directly instead of search - **CRITICAL**: Always check for child pages - nested documentation often contains the most relevant details - Example: "Job Post" parent may have children "Create a Job Post", "Edit a Job Post", etc. - Retrieve up to 10 child pages per parent, prioritize by relevance to ticket diff --git a/plugins/core-cursor-standalone/.cursor/commands/testgen-flow-test-case-export.md b/plugins/core-cursor-standalone/.cursor/commands/testgen-flow-test-case-export.md index e2cce06c..7e9164fc 100644 --- a/plugins/core-cursor-standalone/.cursor/commands/testgen-flow-test-case-export.md +++ b/plugins/core-cursor-standalone/.cursor/commands/testgen-flow-test-case-export.md @@ -12,26 +12,70 @@ baseSchema: docs/schemas/phase.md - Phase 5 MUST be complete - `agents/testgen/{TICKET-KEY}/test-scenarios.md` exists with test cases - User has reviewed and edited test cases -- TestRail MCP configured and accessible -- TestRail project_id and suite_id known +- **TestRail export path (B + A):** Either **guided** (TestRail MCP allowed per `mcp-capability-interaction.md` and `agents/mcp-capability.yaml`) or **questionnaire** (no TestRail MCP; user supplies IDs and performs or confirms manual import). Optional **`agents/user-instructions/mcp-guidance.md`** when guided. +- **Placeholder binding:** Before first guided **`{integration-action:*}`** call, ACQUIRE **`mcp-tool-resolution.md`**. For **`{agent-action:*}`** file steps, ACQUIRE **`agent-action-resolution.md`**. Canonical: **`instructions/r2/core/workflows/.md`** if KB alias fails. +- For **guided** runs: TestRail MCP configured and accessible; `project_id` and `suite_id` known (from guidance, state, or user). ## Objective -Export test cases from test-scenarios.md to TestRail using MCP, creating a new section (folder) for the Jira ticket and adding all test cases. -If MCP is not available, ask user how to proceed and provide possible options. +Export test cases from `test-scenarios.md` to TestRail. + +- **Guided:** Use TestRail MCP to verify the project, resolve `section_id`, and create cases. +- **Questionnaire:** Do **not** run any **`{integration-action:testrail-*}`** call. Collect `project_id`, `suite_id`, and `section_id` from the user, write a **manual export pack** (copy-paste / CSV-friendly), and record completion when the user confirms import or pastes back TestRail case IDs. + +If the interaction mode is unclear, resolve it in **Step 0** before any MCP call. ## TestRail Configuration **Default Configuration**: detect using current ticket and user profile. **Section Name**: [TICKET-KEY] (e.g., "PROJ-123") +**Web URL placeholders:** `{testrail-suite-view-url}`, `{testrail-case-view-url}`, and `{testrail-suite-cases-grouped-url}` stand in for full TestRail browser links (`https://` + host + `index.php?` paths + ids). Compose from `{TestRail instance host}`, `suite_id`, case id, and section grouping as in Step 1Q or `mcp-guidance.md`. + +### Guided TestRail and workspace file actions (placeholders) + +| Token | Intent | +|-------|--------| +| `{integration-action:testrail-get-project}` | Verify TestRail project access by `project_id` | +| `{integration-action:testrail-get-cases}` | List cases in a project/suite (optional dedup before add) | +| `{integration-action:testrail-add-case}` | Create a test case in a `section_id` with title, priority, type, refs, separated steps | +| `{agent-action:read-file}` | Read a workspace file (host-specific file tool) | +| `{agent-action:write-file}` | Create or overwrite a workspace file | +| `{agent-action:patch-file}` | Apply targeted edits to a workspace file (includes search-and-replace style edits) | + +Resolve **`{integration-action:…}`** using **`mcp-tool-resolution.md`** (guided only). Resolve **`{agent-action:…}`** using **`agent-action-resolution.md`**. ## Requirements -### Step 1: Verify TestRail Connection +### Step 0: Resolve TestRail MCP interaction (B + A) -**Test connection** using: -```python -mcp_testrail_get_project(project_id=69) +1. ACQUIRE **`mcp-capability-interaction.md`** FROM KB. ACQUIRE **`mcp-tool-resolution.md`** before any **`{integration-action:*}`** MCP call (when **guided**). ACQUIRE **`agent-action-resolution.md`** before any **`{agent-action:*}`** workspace step. If KB alias fails, use **`instructions/r2/core/workflows/.md`**. +2. Read **`agents/mcp-capability.yaml`** if present; apply **user override** from the current task text (same rules as Phase 1). +3. Derive **`TestRail export: guided | questionnaire`**: + - **`mcp.mode: absent`** → questionnaire. + - **`mcp.mode: capable`** and **`mcp.testrail: false`** (when the key is present) → questionnaire. + - **`mcp.mode: capable`** and (`testrail` omitted or `true`) → guided. + - Missing YAML: use **user override** if any; else **one** short question (“Use TestRail MCP for export?”). **No** → questionnaire; **Yes** → guided and recommend adding `agents/mcp-capability.yaml`. +4. Record in `agents/testgen/{TICKET-KEY}/testgen-state.md` (Phase 6): + - `TestRail export: guided | questionnaire` + - `MCP interaction source:` (`agents/mcp-capability.yaml` | user override | default question`) +5. If **guided** and **`agents/user-instructions/mcp-guidance.md`** exists, read it before the first **`{integration-action:testrail-*}`** call. + +### Step 1: Verify TestRail connection (guided only) + +**If `TestRail export` is `questionnaire`:** Run **Step 1Q** instead of this step. + +#### Step 1Q: Questionnaire — no MCP verification + +1. Ask **numbered** questions: `project_id`, `suite_id` (for links and documentation), and confirm the user will import cases manually or paste back IDs after import. +2. **STOP** and **WAIT**. +3. Skip to **Step 2** (section_id flow is unchanged; user may already know `section_id`). + +**Guided path — test connection** using (replace `project_id` with value from guidance/state/user; **do not** hard-code example IDs unless the user confirmed them): + +```text +Action: {integration-action:testrail-get-project} +Parameters: + project_id: ``` If fails, tell user: @@ -39,7 +83,7 @@ If fails, tell user: ❌ TestRail connection failed. Please verify: 1. TestRail MCP is configured 2. Credentials are correct -3. Project ID 69 exists and you have access +3. Project ID exists and you have access ``` ### Step 2: Create Section in TestRail @@ -57,7 +101,7 @@ If you already have a section, provide the section_id. You can find it in the URL when viewing a section (e.g., group_id=94686 or section_id=94686) **Option B: Create new section** -1. Go to: https://griddynamics.testrail.io/index.php?/suites/view/3300 +1. Go to: {testrail-suite-view-url} 2. Click "Add Section" 3. Name it: [TICKET-KEY] (e.g., "PROJ-456") 4. After creating, find the section_id in the URL or section details @@ -224,19 +268,40 @@ Execute this test case for EACH row in the table below: - Just include original preconditions normally - No "Execute for EACH row" note -### Step 5: Add Test Cases to TestRail +### Step 5: Add Test Cases to TestRail (guided only) -**For each mapped test case**: +**If `TestRail export` is `questionnaire`:** Run **Step 5Q** only; **do not** run **`{integration-action:testrail-add-case}`**. -```python -result = mcp_testrail_add_case( - section_id=section_id, - title=testrail_case["title"], - priority_id=testrail_case["priority_id"], - type_id=testrail_case["type_id"], - refs=testrail_case["refs"], - custom_steps_separated=testrail_case["custom_steps_separated"] -) +#### Step 5Q: Manual export pack (no MCP) + +1. After **Step 4** mapping, write **`agents/testgen/{TICKET-KEY}/testrail-manual-export.md`** containing: + - User-supplied `project_id`, `suite_id`, `section_id` (if known) + - One block per test case: title, priority, type, refs, preconditions text, and steps (content + expected) in plain text or markdown tables suitable for copy-paste into TestRail. +2. Optionally add **`agents/testgen/{TICKET-KEY}/testrail-import-hints.csv`** (title, priority, type, refs, preconditions, steps_json) if the team uses CSV import. +3. **STOP** and **WAIT**: ask the user to confirm either (a) they created cases manually / imported the pack, or (b) they paste back a mapping `TC-001 → C12345` (or URLs). +4. **After the user replies**, execute **Step 6** to merge IDs into `test-scenarios.md`. Until then you may leave placeholders (e.g. `Pending`) only if the workflow must save intermediate files; do **not** mark Phase 6 complete in state until Step 6 reflects the user’s answer. + +**Optional (guided only) — list existing cases before add:** If duplicate titles, re-exports, or unclear existing coverage are a concern, run **`{integration-action:testrail-get-cases}`** once against the target `project_id` / `suite_id` (and use section or other filters when your MCP supports them) to compare with parsed markdown. **Not required** for a first-time export or when the user accepts duplicates. + +```text +Action: {integration-action:testrail-get-cases} +Parameters: + project_id: + suite_id: +``` + +**Guided path — for each mapped test case**: + +```text +Action: {integration-action:testrail-add-case} +Parameters: + section_id: + title: + priority_id: + type_id: + refs: + custom_steps_separated: +Outcome: capture returned case id for results tracking ``` **Track results**: @@ -261,12 +326,14 @@ results = { **Update**: `agents/testgen/{TICKET-KEY}/test-scenarios.md` +For **questionnaire** exports, use user-pasted IDs when available; otherwise keep **`TestRail ID`**: `Manual (see testrail-manual-export.md)`** until the user confirms. Use real `project` / `suite` / base URL values from Step 1Q or guidance — **do not** invent instance hostnames. + **Add TestRail IDs to each test case**: ```markdown ### TC-001: User Login with Valid Credentials (Happy Path) **TestRail ID**: C12345 ✅ -**TestRail Link**: https://griddynamics.testrail.io/index.php?/cases/view/12345 +**TestRail Link**: {testrail-case-view-url} **Related Requirement**: US-1, FR-1 ... ``` @@ -283,11 +350,11 @@ results = { ## TestRail Export Summary **Exported**: [DateTime] -**Project**: 69 -**Suite**: 3300 +**Project**: {project_id} +**Suite**: {suite_id} **Section**: [TICKET-KEY] (ID: [section_id]) **Total Exported**: [X] test cases -**TestRail Link**: https://griddynamics.testrail.io/index.php?/suites/view/3300&group_by=cases:section_id +**TestRail Link**: {testrail-suite-cases-grouped-url} | TC ID | TestRail ID | Status | |-------|-------------|--------| @@ -326,8 +393,8 @@ results = { ### Phase 6: TestRail Export - **Completed**: [DateTime] -- **Project ID**: 69 -- **Suite ID**: 3300 +- **Project ID**: {project_id} +- **Suite ID**: {suite_id} - **Section ID**: [section_id] - **Section Name**: [TICKET-KEY] - **Test Cases Created**: [Count] @@ -339,21 +406,30 @@ results = { ## Validation Before completing Phase 6, verify: -- ✅ TestRail connection successful -- ✅ Section exists in TestRail + +**Guided path:** +- ✅ TestRail connection successful (`{integration-action:testrail-get-project}` or equivalent) +- ✅ Section exists in TestRail (or user provided valid `section_id`) - ✅ All test cases parsed from markdown -- ✅ At least 80% of test cases exported successfully -- ✅ test-scenarios.md updated with TestRail IDs +- ✅ At least 80% of test cases exported successfully via MCP (or failures documented) +- ✅ `test-scenarios.md` updated with TestRail IDs - ✅ State file updated with Phase 6 complete - ✅ TestRail link provided to user +**Questionnaire path:** +- ✅ Step 0 recorded `TestRail export: questionnaire` and source line +- ✅ User confirmed `project_id` / `suite_id` / `section_id` as needed +- ✅ `testrail-manual-export.md` (and optional CSV) written +- ✅ User confirmed manual import or pasted ID mapping +- ✅ `test-scenarios.md` updated with IDs or explicit “manual / pending” notes +- ✅ State file updated with Phase 6 complete + ## Tools Used -- `mcp_testrail_get_project(project_id)` - Verify connection -- `mcp_testrail_get_cases(project_id, suite_id)` - Check existing cases -- `mcp_testrail_add_case(section_id, title, ...)` - Create test cases -- `read_file()` - Read test-scenarios.md -- `write()` / `search_replace()` - Update files +- **Guided:** `{integration-action:testrail-get-project}` — verify connection; `{integration-action:testrail-get-cases}` — optional dedup; `{integration-action:testrail-add-case}` — create test cases +- **Questionnaire:** `{agent-action:read-file}` / `{agent-action:write-file}` / `{agent-action:patch-file}` only — build `testrail-manual-export.md` and update docs; **no** `{integration-action:testrail-*}` calls +- `{agent-action:read-file}` — read `test-scenarios.md` +- `{agent-action:write-file}` / `{agent-action:patch-file}` — update files ## Common Issues @@ -375,6 +451,9 @@ Before completing Phase 6, verify: **Issue**: Rate limit exceeded **Solution**: Add delay between API calls, batch requests +**Issue**: Risk of duplicate titles or unclear existing coverage +**Solution**: Optionally run **`{integration-action:testrail-get-cases}`** (Step 5 optional block) before **`{integration-action:testrail-add-case}`** — optional discovery of existing cases, not a mandatory gate. + **Issue**: Test case already exists **Solution**: Create anyway (TestRail allows duplicates), note in report @@ -393,7 +472,7 @@ All 6 phases finished successfully: ✅ Phase 6: TestRail Export ([R] cases exported) **TestRail Section**: [TICKET-KEY] -**TestRail Link**: https://griddynamics.testrail.io/index.php?/suites/view/3300 +**TestRail Link**: {testrail-suite-view-url} **Deliverables**: 📄 requirements.md - Use for implementation @@ -411,8 +490,8 @@ All 6 phases finished successfully: - TestRail MCP currently lacks section creation - user must create manually - Test case IDs in TestRail are prefixed with "C" (e.g., C12345) -- Suite ID 3300 is from user's TestRail URL -- Project ID 69 is user's default project +- `{suite_id}` comes from the user, `mcp-guidance.md`, or the suite view URL on their instance +- `{project_id}` comes from the user, `mcp-guidance.md`, or the TestRail project settings - Parameterized test data is included in step content or expected results - Re-running export creates duplicate test cases (by design, to preserve history) - Consider creating test run after export for immediate execution diff --git a/plugins/core-cursor-standalone/.cursor/commands/testgen-flow.md b/plugins/core-cursor-standalone/.cursor/commands/testgen-flow.md index 1e334ca6..de88e4c8 100644 --- a/plugins/core-cursor-standalone/.cursor/commands/testgen-flow.md +++ b/plugins/core-cursor-standalone/.cursor/commands/testgen-flow.md @@ -14,6 +14,7 @@ Systematic requirements analysis from Jira tickets and Confluence documentation ### Critical Requirements +- **MCP ROUTING (B + A)**: Phases 1 and 6 may call external MCPs only when **`agents/mcp-capability.yaml`** and **`mcp-capability-interaction.md`** resolve **guided** for that integration. Otherwise use **questionnaire** or manual export paths documented in those phases. Optional **`agents/user-instructions/mcp-guidance.md`** for IDs and guardrails when guided. **`{integration-action:*}`** / **`{agent-action:*}`** binding: follow each phase file’s **Prerequisites** (**`mcp-tool-resolution.md`**, **`agent-action-resolution.md`**). - **ONE PHASE AT A TIME**: Read phase file, execute, update state, move to next. - **DO NOT SKIP PHASES**: Each builds on previous. - **STATE TRACKING**: Update `agents/testgen/{TICKET-KEY}/testgen-state.md` after each phase. @@ -31,9 +32,10 @@ Systematic requirements analysis from Jira tickets and Confluence documentation **Phase 1: Data Collection** [testgen-flow-data-collection.md] 1. ACQUIRE testgen-flow-data-collection.md FROM KB -2. Execute phase instructions -3. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` -4. Validate by listing raw-data.md file +2. ACQUIRE **`mcp-capability-interaction.md`**, and per that phase’s **Prerequisites** also **`mcp-tool-resolution.md`** / **`agent-action-resolution.md`** when placeholders are used +3. Execute phase instructions +4. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` +5. Validate by listing raw-data.md file **Phase 2: Gap & Contradiction Analysis** [testgen-flow-gap-and-contradiction-analysis.md] 1. ACQUIRE testgen-flow-gap-and-contradiction-analysis.md FROM KB @@ -61,8 +63,9 @@ Systematic requirements analysis from Jira tickets and Confluence documentation **Phase 6: Test Case Export** [testgen-flow-test-case-export.md] ⭐ 1. ACQUIRE testgen-flow-test-case-export.md FROM KB -4. Execute phase instructions -5. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` +2. ACQUIRE **`mcp-capability-interaction.md`** and per that phase’s **Step 0** / **Prerequisites** also **`mcp-tool-resolution.md`** / **`agent-action-resolution.md`** when placeholders are used (guided vs questionnaire for TestRail export) +3. Execute phase instructions +4. Update `agents/testgen/{TICKET-KEY}/testgen-state.md` ## State File Format @@ -136,9 +139,9 @@ agents/testgen/{TICKET-KEY}/ ## Prerequisites -- **Jira MCP**: Configured and accessible -- **Jira Ticket**: User provides ticket key or link -- **Confluence Access**: Via Jira MCP (same authentication) +- **MCP routing (B + A):** `agents/mcp-capability.yaml` (see `instructions/r2/core/templates/mcp-capability.example.yaml`) plus optional `agents/user-instructions/mcp-guidance.md` control **guided** vs **questionnaire** per integration. Phases 1 and 6 embed the branching rules. +- **Jira ticket**: User provides ticket key or link (Phase 1) +- **Confluence**: Either MCP-guided discovery (Phase 1) or user-provided content when **questionnaire** - **Output Directory**: Created at start (`agents/testgen/{TICKET-KEY}/`) ## Common Patterns diff --git a/test-library/testgen/trigger.txt b/test-library/testgen/trigger.txt index 28677fef..10cec4d1 100644 --- a/test-library/testgen/trigger.txt +++ b/test-library/testgen/trigger.txt @@ -4,3 +4,8 @@ instructions/agents/core/r1/testgen-phase2.md instructions/agents/core/r1/testgen-phase3.md instructions/agents/core/r1/testgen-phase4.md instructions/agents/core/r1/testgen-phase5.md +# R2 TestGen + MCP capability workflows (scenario comparison when these change) +instructions/r2/core/workflows/testgen-flow +instructions/r2/core/workflows/mcp-capability-interaction +instructions/r2/core/workflows/mcp-tool-resolution +instructions/r2/core/workflows/agent-action-resolution