QAira QRIP Component Test Generation Final

This runtime extends the existing QAira API + WebUI/QSG semantic compiler with a component-test pipeline.

Test types

QAIRA_TEST_TYPE=api        # API tests only
QAIRA_TEST_TYPE=webui      # WebUI pages / object repository only
QAIRA_TEST_TYPE=component  # Component Test Contracts + Playwright tests only
QAIRA_TEST_TYPE=both       # API + WebUI
QAIRA_TEST_TYPE=all        # API + WebUI + component tests
QAIRA_TEST_TYPE=auto       # API/WebUI auto-detect; use all to include component tests

Component pipeline

Existing API and WebUI agents are preserved. New agents run after graph/page discovery when QAIRA_TEST_TYPE=component or all:

1. ComponentDiscoveryAgent
2. PageComponentMappingAgent
3. ComponentApiMappingAgent
4. ComponentElementDiscoveryAgent
5. ComponentLocatorRankingAgent
6. ComponentStateDiscoveryAgent
7. ComponentEventDiscoveryAgent
8. ComponentValidationDiscoveryAgent
9. ComponentAssertionMiningAgent
10. ComponentFixtureMockDataAgent
11. ComponentTestContractAgent
12. ComponentPlaywrightGenerationAgent
13. ComponentExecutionPlanAgent
14. ComponentAutoRepairAgent
15. ComponentArtifactPackagingAgent

Key principle

Tests are not generated directly from source. The runtime first emits:

generated/components/component_test_contracts.json

Then Playwright specs are generated from those contracts.

Outputs

generated/components/component_inventory.json
generated/components/page_component_map.json
generated/components/component_api_dependencies.json
generated/components/component_elements.json
generated/components/component_ranked_locators.json
generated/components/component_states.json
generated/components/component_events.json
generated/components/component_validations.json
generated/components/component_assertions.json
generated/components/component_fixtures_and_mocks.json
generated/components/component_test_contracts.json
generated/components/component_playwright_tests.json
generated/components/component_tests.csv
generated/components/component_generation_summary.json
generated/component-tests/*.spec.ts
generated/component-tests/run-component-tests.sh
runtime/component_execution_plan.json
reports/component_auto_repair_plan.json

Playwright execution

Execution is intended to run from the repo workspace through the VS Code extension or locally:

PLAYWRIGHT_HTML_REPORT=qaira-output/component-html-report \
  npx playwright test qaira-output/generated/component-tests \
  --reporter=html,json \
  --output=qaira-output/component-test-results

Generated tests use:

test.use({ screenshot: 'only-on-failure', trace: 'retain-on-failure', video: 'retain-on-failure' })

Docker build

docker build -t jayarajumetta/semantic-compiler:latest .

Docker run all intelligence

docker run --rm \
  -e PYTHONUNBUFFERED=1 \
  -e QAIRA_TEST_TYPE=all \
  -e OPENAI_API_KEY="$OPENAI_API_KEY" \
  -v "$PWD:/workspace/source:ro" \
  -v "$PWD/qaira-output:/output" \
  -v "$PWD/qaira-learning:/learning" \
  jayarajumetta/semantic-compiler:latest

Playwright codegen

The runtime records codegen guidance in runtime/component_execution_plan.json. The VS Code extension provides QAira: Open Playwright Codegen, which runs:

npx playwright codegen <APP_URL>

Use this to enrich locator learning when static source cannot infer runtime-only names, portals, iframes, or shadow DOM.

July 8 Mature Agentic Robustness Upgrade

This build hardens the API, WebUI and Component pipelines with production-style agent controls:

• Preflight semantic prerequisite checks before API/WebUI/component phases.
• Per-agent outcomes in runtime/agent_outcomes.jsonl with fingerprints and artifact checks.
• Redacted OpenTelemetry-style trace records in verbose/trace.jsonl.
• Fail-open LLM review gateway with prompt/response audit files and call budgets.
• Self-learning snapshots in /learning/pipeline/self_learning_summary.latest.json.
• Component semantic quality gate for bad locators, code-expression labels and unstable selectors.
• Pipeline robustness gate after generation so untrusted artifacts are marked partial instead of silently accepted.

Recommended full command:

QAIRA_TEST_TYPE=all QAIRA_ENABLE_LLM=auto docker run --rm   -e PYTHONUNBUFFERED=1   -e QAIRA_TEST_TYPE=all   -e OPENAI_API_KEY="$OPENAI_API_KEY"   -v "$PWD:/workspace/source:ro"   -v "$PWD/qaira-output:/output"   -v "$PWD/qaira-learning:/learning"   jayarajumetta/semantic-compiler:latest

Important reports:

• reports/agent_outcome_review.json
• reports/pipeline_robustness_gate.json
• reports/component_semantic_quality_gate.json
• runtime/self_learning_summary.json
• runtime/agent_outcomes.jsonl
• verbose/trace.jsonl

QAira Pro Agentic Hardening — July 8 Final

This build keeps the existing API, WebUI/Object Repository, and Component Test Contract pipelines, then adds a pro-grade control plane around them. The goal is not to let any single agent invent outputs; each agent now has clearer inputs, outputs, fallback evidence, trace entries, quality signals, and downstream robustness checks.

New pro-grade agents

• FrameworkPatternIntelligenceAgent detects frontend/backend frameworks, routing patterns, API clients, dependency-injection signals, GraphQL/serverless/gateway hints, and organization-specific wrapper risks.
• PromptChainSynthesisAgent writes deterministic prompt chains for API, WebUI, and Component review. LLM calls remain fail-open and network-disabled unless explicitly enabled.
• CriticalPathReviewAgent checks consecutive failures, missing critical artifacts, low-confidence phases, and framework support gaps before assets are trusted.
• AgentOutcomeReviewAgent, PipelineRobustnessGateAgent, and SelfLearningMemoryAgent were hardened to summarize failure chains, confidence, and lessons across runs.

New reports

reports/framework_pattern_intelligence.json
prompts/api_prompt_chain.json
prompts/webui_prompt_chain.json
prompts/component_prompt_chain.json
prompts/prompt_chain_manifest.json
runtime/agent_outcomes.jsonl
runtime/quality_signals.jsonl
reports/critical_path_review.json
reports/pipeline_robustness_gate.json
runtime/self_learning_summary.json

Critical locator guardrails

The WebUI and Component pipelines now reject code-expression locators such as:

setSelectedCaseId(testCase.id)
event.stopPropagation()
`${projectLabel
void handleSave()
::
))
1

These are marked unstable or excluded from import/page-object generation instead of being promoted as stable locators.

Organization-specific pattern support

For enterprise projects using custom wrappers, add config/config.yml like:

qaira:
  custom_patterns:
    api_route_wrappers:
      - function: QairaRoute.define
        path_arg: 0
        handler_arg: 1
    page_wrappers:
      - component: AppScreen
        path_prop: path
        component_prop: component
    design_system:
      button_components: [QButton, IconButton]
      input_components: [QTextField, FormInput]

Recommended Docker run

docker run --rm \
  -e PYTHONUNBUFFERED=1 \
  -e QAIRA_TEST_TYPE=all \
  -e QAIRA_MATURE_GUARDS=true \
  -e QAIRA_FAIL_OPEN=true \
  -e OPENAI_API_KEY="$OPENAI_API_KEY" \
  -v "$PWD:/workspace/source:ro" \
  -v "$PWD/qaira-output:/output" \
  -v "$PWD/qaira-learning:/learning" \
  jayarajumetta/semantic-compiler:latest

Review these files after every pro run:

reports/framework_pattern_intelligence.json
reports/critical_path_review.json
reports/pipeline_robustness_gate.json
verbose/trace.jsonl