Following are changes details,#8
Open
naveedsaleem001 wants to merge 2 commits into
Open
Conversation
- add qa-specialist agent with manual and automated testing expertise - add QA framework with exploratory testing and documentation standards - add complete QA framework with environment, data, and metrics skills
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a QA-focused agent plus a set of QA/testing skills and rules intended to standardize manual testing, exploratory testing, automation guidance, environment/data management, and quality metrics across the repository.
Changes:
- Added a new
qa-specialistagent with guidance for manual + automated QA workflows. - Added multiple QA skill documents (automation, data strategy, environment management, metrics, manual/exploratory testing).
- Added/updated QA rules and contribution guidance (web testing pyramid, accessibility testing, test documentation standards, CONTRIBUTING QA section).
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 21 comments.
Show a summary per file
| File | Description |
|---|---|
| Claude/skills/test-environment-management.md | New environment management skill/framework (configs, isolation, CI, provisioning). |
| Claude/skills/test-data-strategy.md | New test data management strategy/framework (synthetic data, masking, factories, validation, dashboards). |
| Claude/skills/test-automation.md | New automation framework guidance (architecture, patterns, CI examples). |
| Claude/skills/quality-metrics.md | New quality metrics framework (KPIs, collection workflow, processing script, dashboard examples). |
| Claude/skills/manual-testing.md | New manual testing guide (planning + charter template). |
| Claude/skills/exploratory-testing.md | New exploratory testing guide (charters, session structure/reporting, techniques). |
| Claude/rules/web/test-pyramid.md | New web-specific test pyramid rule. |
| Claude/rules/web/accessibility-testing.md | New web accessibility testing rule (WCAG 2.2 AA guidance). |
| Claude/rules/test-documentation.md | New test documentation standards (cases, reporting, defects, metrics). |
| Claude/agents/qa-specialist.md | New QA agent definition and example workflows. |
| CONTRIBUTING.md | Added QA contribution guidance + updated checklist bullets. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| lastName = faker.person.commonLastName(); | ||
| } | ||
|
|
||
| users.push(this.create({ firstName, lastName })); |
Comment on lines
+1
to
+12
| --- | ||
| type: skill | ||
| name: test-data-strategy | ||
| description: Comprehensive framework for test data management including provisioning, anonymization, synthetic data generation, and lifecycle management. | ||
| origin: ECC | ||
| --- | ||
|
|
||
| # Test Data Strategy Framework | ||
|
|
||
| Strategic approach to managing test data across the development lifecycle, ensuring data quality, privacy compliance, and test reliability. | ||
|
|
||
| ## Data Provisioning Patterns |
Comment on lines
+756
to
+759
| return \ | ||
| {year}-W\ | ||
| {week.toString().padStart(2, '0')}\ | ||
| `; |
| await driver.elementById('login-button').click() | ||
|
|
||
| // Verify login success | ||
| const welcomeText = await driver elementById('welcome-message').text() |
Comment on lines
+14
to
+64
| ### Defect Management Metrics | ||
|
|
||
| ```markdown | ||
| # Defect Management Dashboard | ||
|
|
||
| ## Key Metrics | ||
|
|
||
| | Metric | Formula | Target | Current | Status | | ||
| |--------|---------|--------|---------|--------| | ||
| | Defect Density | Total Defects / KLOC | ≤ 1.0 | 0.8 | ✅ | | | ||
| | Critical Defect Resolution Time | Avg. time to resolve P0/P1 defects | ≤ 4h | 3.2h | ✅ | | ||
| | Defect Reopen Rate | Reopened Defects / Total Resolved | ≤ 5% | 3.8% | ✅ | | ||
| | Test Escape Rate | Production Defects / Total Defects | ≤ 10% | 8.5% | ✅ | | ||
| | Defect Turnaround Time | Avg. time from report to resolution | ≤ 24h | 18.5h | ✅ | | ||
|
|
||
| ## Defect Trends (Last 8 Weeks) | ||
|
|
||
| ```mermaid | ||
| lineChart | ||
| title Defect Trends | ||
| x-axis Week | ||
| y-axis Count | ||
| series Reported, Resolved | ||
| Reported : 45, 38, 42, 36, 39, 33, 30, 28 | ||
| Resolved : 32, 41, 39, 40, 37, 35, 34, 31 | ||
| ``` | ||
|
|
||
| ## Defect Distribution | ||
|
|
||
| ```mermaid | ||
| pie | ||
| title Defect Priority Distribution | ||
| "P0: Critical" : 5 | ||
| "P1: High" : 12 | ||
| "P2: Medium" : 25 | ||
| "P3: Low" : 18 | ||
| ``` | ||
|
|
||
| ```mermaid | ||
| pie | ||
| title Defect Origin | ||
| "Frontend" : 24 | ||
| "Backend" : 28 | ||
| "Integration" : 12 | ||
| "Infrastructure" : 6 | ||
| "Documentation" : 10 | ||
| ``` | ||
|
|
||
| ## Action Items | ||
|
|
||
| - Investigate increase in P1 defects from integration points |
Comment on lines
+216
to
+217
| - [ ] README updated if the addition warrants it. | ||
| - [ ] QA-related additions reference appropriate agents, skills, and rules. |
|
|
||
| # Web Accessibility Rules | ||
|
|
||
| > This file extends [common/accessibility.md](../common/accessibility.md) with web-specific content. |
| services: { | ||
| sonarqube: 'https://sonarqube.example.com', | ||
| jenkins: 'https://jenkins.example.com', | ||
| datadog: 'https://api.datadoghq.com' |
|
|
||
| # Test Pyramid for Web Applications | ||
|
|
||
| > This file extends [common/test-pyramid.md](../common/test-pyramid.md) with web-specific guidelines. |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated 15 comments.
Comments suppressed due to low confidence (6)
Claude/skills/quality-metrics.md:303
- The metrics processor example references
CONFIG.services.jira, but theservicesconfig object does not define ajirabase URL. Add the missing config entry (or change the code to use an existing key) so the example is internally consistent.
const CONFIG = {
metricsPath: './metrics',
outputPath: './docs/dashboards',
services: {
sonarqube: 'https://sonarqube.example.com',
jenkins: 'https://jenkins.example.com',
datadog: 'https://api.datadoghq.com'
}
};
Claude/skills/quality-metrics.md:675
generateMarkdownDashboard()contains a malformed Mermaid code fence (e```mermaid). This will break rendering of the chart in Markdown; remove the stray character so the code fence starts with exactly ```mermaid.
- **Critical/High Defects**: ${this.metrics.defects.kpis.p0_p1_count}
- **Defect Trends**: See chart below
e```mermaid
lineChart
title Defect Trends
x-axis Week
y-axis Count
series Defects
Defects : ${this.metrics.defects.kpis.trends.map(t => t.count).join(', ')}
**Claude/skills/quality-metrics.md:760**
* The `getWeekString` implementation is not valid JavaScript (it contains backslash-escaped newlines and missing template literal delimiters). Replace this with a normal template literal or string concatenation so the example code is executable.
getWeekString(date) {
const year = date.getFullYear();
const week = this.getWeekNumber(date);
return
{year}-W
{week.toString().padStart(2, '0')}
`;
}
**Claude/skills/quality-metrics.md:854**
* The Vue example is corrupted around the chart section (there’s a trailing `\` and a stray backtick), which breaks the snippet’s validity. Clean up the extraneous characters so the `<template>` section is valid Vue SFC syntax.
<section class="charts">
<div class="chart-container">
<h2>Defect Trends</h2>
<line-chart :data="defectTrends" />\
`
**Claude/skills/quality-metrics.md:1225**
* The `getAge` method returns an invalid JavaScript string (backslash-escaped lines and an unmatched template literal). Use a proper template literal (or string concatenation) so the snippet is correct and copy/pasteable.
getAge(dateString) {
const date = new Date(dateString);
const now = new Date();
const diffTime = Math.abs(now - date);
const diffDays = Math.ceil(diffTime / (1000 * 60 * 60 * 24));
return \
{diffDays}d
`;
},
**Claude/skills/quality-metrics.md:1257**
* In the Vue `<style>` section, selectors like `dashboard-header`/`dashboard-layout`/`kpi-summary` are missing the leading `.` required for class selectors, so the styles won’t apply to elements with those classes. Change these to `.dashboard-header`, `.dashboard-layout`, `.kpi-summary`, etc.
dashboard-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 20px;
padding-bottom: 10px;
border-bottom: 1px solid #e0e0e0;
}
dashboard-header h1 {
margin: 0;
</details>
Comment on lines
+216
to
+217
| - [ ] README updated if the addition warrants it. | ||
| - [ ] QA-related additions reference appropriate agents, skills, and rules. |
Comment on lines
+1
to
+6
| --- | ||
| type: skill | ||
| name: test-environment-management | ||
| description: Framework for managing test environments, configuration, data provisioning, isolation, and lifecycle automation. | ||
| origin: ECC | ||
| --- |
Comment on lines
+50
to
+70
| Detailed examples, large YAML samples, CI workflows, cloud provisioning guidance, and monitoring runbooks should be stored in the reference files listed above rather than embedded in the primary skill document. | ||
| ssl_required: true | ||
| cors: | ||
| allowed_origins: ["${FRONTEND_URL}"] | ||
| allowed_methods: ["GET", "POST", "PUT", "DELETE"] | ||
|
|
||
| # Environment-specific override | ||
| # environments/staging/overrides.yaml | ||
| app: | ||
| environment: "staging" | ||
| feature_flags: | ||
| new_payment_gateway: true | ||
| enhanced_reporting: false | ||
|
|
||
| api: | ||
| base_url: "https://api.staging.example.com" | ||
| timeout: 45000 | ||
|
|
||
| logging: | ||
| level: "debug" | ||
| ``` |
Comment on lines
+86
to
+93
| db.create_collections() | ||
| db.load_fixtures('fixtures/common.yaml') | ||
| db.load_fixtures(f'fixtures/{test_env}.yaml') | ||
| return db | ||
|
|
||
| @ pytest.fixture(scope="session") | ||
| def database(): | ||
| return setup_clean_database() |
Comment on lines
+522
to
+533
| # Create RDS MongoDB instance | ||
| db_instance = self.rds.create_db_instance( | ||
| DBInstanceIdentifier=f"{env_name.lower()}-db", | ||
| DBInstanceClass='db.t3.small', | ||
| Engine='mongodb', | ||
| MasterUsername=self.config['database']['username'], | ||
| MasterUserPassword=self.config['database']['password'], | ||
| AllocatedStorage=20, | ||
| VpcSecurityGroupIds=[db_sg], | ||
| DBSubnetGroupName=self.create_db_subnet_group(vpc_id, private_subnet['Subnet']['SubnetId']) | ||
| ) | ||
|
|
Comment on lines
+1
to
+6
| --- | ||
| type: skill | ||
| name: quality-metrics | ||
| description: Comprehensive framework for measuring, tracking, and improving software quality through actionable metrics, KPIs, and continuous improvement practices. | ||
| origin: ECC | ||
| --- |
Comment on lines
+3
to
+4
| trigger: "use PROACTIVELY when conducting QA, debugging, or user testing" | ||
| description: Comprehensive guide to exploratory testing methodologies, charter development, session reporting, and best practices. |
| await driver.elementById('login-button').click() | ||
|
|
||
| // Verify login success | ||
| const welcomeText = await driver elementById('welcome-message').text() |
| | Unit Test Coverage | 80% | XX% | [Pass/Fail] | | ||
| | Integration Test Coverage | 80% | XX% | [Pass/Fail] | | ||
| | E2E Test Coverage | 100% of critical paths | XX% | [Pass/Fail] | | ||
| | Accessibility | WCAG 2.1 AA | [Score] | [Pass/Fail] | |
Comment on lines
+251
to
+261
| gh api | ||
| -X GET | ||
| -H "Accept: application/vnd.github.v3+json" | ||
| "/repos/${{ github.repository }}/actions/runs?per_page=100" | ||
| > metrics/pipeline_runs.json | ||
|
|
||
| # Get deployment data | ||
| gh api | ||
| -X GET | ||
| -H "Accept: application/vnd.github.v3+json" | ||
| "/repos/${{ github.repository }}/deployments?per_page=100" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Following changes are made,
Type of change
What was added / changed
Checklist
kebab-case)Related issues / references