Skip to content

cli.md

Latest commit

 

History

History
90 lines (65 loc) · 2.36 KB

cli.md

File metadata and controls

90 lines (65 loc) · 2.36 KB
Raw

CLI

The local CLI is available through npm run cli -- <command>.

Discover

npm run cli -- list-benchmarks

Validate

npm run cli -- validate-manifest packages/ui-reconstruction/fixtures/mobile-login/eval.manifest.yaml
npm run cli -- validate-manifests

Validation checks manifest shape, duplicate IDs, evaluator references, report/card existence, and fixture references.

Run

npm run cli -- run ui-reconstruction ui.mobile-login --candidate bad
npm run cli -- run intent-vocabulary intent.running --track proposed

Run against your own outputs:

npm run cli -- run ui-reconstruction ui.mobile-login \
  --candidate-file examples/use-your-own-ui-output/candidate.screen.json \
  --candidate-label my-system-v1

npm run cli -- run intent-vocabulary intent.running \
  --track accepted \
  --actual-signals examples/use-your-own-intent-signals/actual.signals.yaml

npm run cli -- run intent-vocabulary intent.running \
  --track accepted \
  --resolver examples/use-your-own-intent-resolver/resolver.ts

Generate

npm run cli -- generate-report ui-reconstruction ui.mobile-login --candidate bad
npm run cli -- generate-report intent-vocabulary intent.running --track proposed
npm run cli -- generate-card ui.mobile-login
npm run cli -- generate-index

Calibration

Generate the canonical UI calibration report:

npm run cli -- calibrate ui.mobile-login

The command reads calibration/candidates/*.json, compares each run against calibration/adjudications/*.yaml, and writes:

  • packages/ui-reconstruction/fixtures/mobile-login/calibration/calibration.report.md
  • packages/ui-reconstruction/fixtures/mobile-login/calibration/calibration.report.json

Write custom Markdown and JSON outputs:

npm run cli -- generate-report ui-reconstruction ui.mobile-login \
  --candidate-file tmp/mobile-login.output.json \
  --candidate-label my-system-v1 \
  --out tmp/mobile-login.md \
  --json-out tmp/mobile-login.json

Batch

npm run cli -- run-batch ui-reconstruction --candidate-dir tmp/outputs/ui --out-dir tmp/reports/ui
npm run cli -- run-batch intent-vocabulary --actual-dir tmp/outputs/intents --track accepted --out-dir tmp/reports/intents

Root scripts wrap the same CLI:

npm run benchmarks:list
npm run manifests:validate
npm run reports:generate
npm run cards:generate
npm run benchmarks:index