feat(design-data): audit generated Figma Variable names against real baseline (11k.4) - #1321
Conversation
…baseline (11k.4) Adds `figma::audit::audit_names` and a `design-data figma audit` CLI subcommand that diffs the generator's output against the S2-Web baseline snapshot (11k.3), per collection, and commits the resulting audit artifact. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: 8f9f1c9 The changes in this PR will be included in the next version bump. This PR includes changesets to release 10 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
🎨 Token Changes ReportTokens Changed (0)Original Branch: This comment was automatically generated by the token diff tool. 🤖 |
🧩 Component Schema Changes ReportNo component schema changes detected.This comment was automatically generated by the component schema diff tool. 🤖 |
Run report for 8f9f1c99Total time: 2m 14s | Comparison time: 2m 27s | Estimated savings: 12.7s (8.6% faster)
Changed files |
GarthDB
left a comment
There was a problem hiding this comment.
Overview
Adds an offline design-data figma audit --snapshot <FILE> --token-dir <DIR> CLI command and a new sdk/core/src/figma/audit.rs module (audit_names). It runs the existing generator (build_export_payload) against a legacy token directory and diffs the resulting variable names, per collection, against a real captured Figma VariablesMeta snapshot — reporting matched / figma_only (coverage gaps) / generated_only (naming divergences) counts, plus a legacyKey → override scaffold seeded from every generated_only name, feeding the follow-on 11k.5 work. Includes a committed ~3200-line JSON artifact from running the audit against the 11k.3 baseline fixture.
Code quality
- Clean reuse of existing primitives (
build_export_payload,summarize_variables,VariableActionnaming convention) rather than reimplementing generation logic — no duplicated logic. - The
real_by_collectionunion-by-name handling (comment ataudit.rs:199-206) correctly guards against a real subtlety: two Figma collections can share a display name (local + remote-library stub), and HashMap iteration order is non-deterministic, so overwriting instead of unioning would make results flaky. Good catch, well-documented. overridesscaffold keys offlegacy_keyviasplit_once('/')on the{prefix}/{legacyKey}convention — correctly matchesmake_variable_action's naming inmapping.rs.- Copyright header, module doc comment, and CLI help text all follow repo conventions.
Issues / things to verify
- Test coverage gap: the single unit test (
uncovered_collection_and_coverage_gap_are_reported) never exercises theoverridesfield — the legacyKey scaffold that's the actual point of the 11k.5 handoff. Worth a small assertion that agenerated_onlyname populatesoverrideswith the correct stripped key. - Cross-collection key collision (minor, likely theoretical):
overridesis a singleBTreeMap<String, String>keyed only bylegacy_key, built across both.Color themeand.Platform scalecollections. If the same legacy key ever produced agenerated_onlyname in both collections, one would silently overwrite the other via.entry(...).or_default()(no-op on the second insert, but no signal either way). Not currently a bug given non-overlapping token namespaces, but worth a comment noting the assumption if it isn't already checked elsewhere. - Documentation:
sdk/README.mddocumentsfigma readandfigma export(lines ~157-158) but the newfigma auditsubcommand isn't added there. Minor, but the CLI surface is now inconsistent with the README's command list. - Fixture size: the ~3205-line
name-mapping.audit.jsonis a large committed artifact for a PR; that seems intentional per the changeset (captures the baseline audit for 11k.5 to consume), so likely fine, but worth confirming it isn't meant to be regenerated per-run instead of hand-committed.
Correctness
- Compared against
mapping.rsandtypes.rson disk —build_export_payload's error path (returnsFigmaError::Apiif.Color theme/.Platform scalearen't found) meansaudit_nameswill hard-fail rather than silently misreport if a snapshot lacks those collections — acceptable since a real snapshot always has them. - Changeset passes
changeset-linter --fail-on-warnings(verified locally). - Recommend confirming
cargo test -p design-data-core figma::auditis green in CI before merge (didn't finish compiling locally in review session; logic reads correct from inspection).
Risk
Low — additive, offline-only CLI command with no changes to existing export/read paths. No security concerns (no network calls, reads local files only).
- sdk/core/src/figma/audit.rs: cover the overrides scaffold in the existing unit test (a generated_only name now seeds its stripped legacyKey), and document the cross-collection key-collision assumption on the overrides map. - sdk/README.md: document the figma audit subcommand alongside read/export. Addresses review comments on PR #1321.
|
Pushed fixes for the review feedback (8f9f1c9):
Left issue 4 (fixture size) as-is per your own note — it's the intentional committed baseline artifact for 11k.5 to consume, not meant to be regenerated per-run. PR #1322 (11k.5) has been rebased on top of this fix. |
Description
Adds
figma::audit::audit_names(sdk/core/src/figma/audit.rs) and adesign-data figma audit --snapshot <FILE> --token-dir <DIR>CLI subcommand.It runs the existing generator (
build_export_payload) offline against alegacy token directory, then diffs the generated Figma Variable names against
a real Figma
VariablesMetasnapshot, per collection:matched— names that agree 1:1figma_only— real variables the generator never emits (coverage gapwithin a covered collection)
generated_only— names the generator emits that Figma doesn't have(naming divergence)
generator_covers: false)ExportSummaryskip buckets (composite/alias/unknown-schema/unparseable-value tokens)
overridesscaffold keyed by legacyKey, seeded fromgenerated_onlymismatches — the artifact the next step (11k.5) will consume
Regenerated the committed audit artifact
(
sdk/core/tests/fixtures/figma/name-mapping.audit.json) against the 11k.3baseline snapshot and verified two consecutive runs are byte-identical
(deterministic, no network).
Related Issue
spectrum-design-data-11k.4 (bd)
Motivation and Context
Feeds the July 31 director review pre-read with a measured status for the
Figma Variables authoring POC, replacing the previously qualitative "invert
the pipeline" framing with real coverage numbers:
.Color theme752/782(96.2%) matched,
.Platform scale807/865 (93.3%) matched, and 4 collections(
Typography,Layout,S2.Color-theme,Iconography) — 1,304 variables —entirely uncovered by the generator today.
How Has This Been Tested?
audit.rsagainst a hand-builtVariablesMetamock,covering: an uncovered whole collection, a matched name, and a
figma_onlycoverage gap within a covered collection.mapping.rstests remain green, unchanged.moon run sdk:codegen-check && moon run sdk:build && moon run sdk:test && moon run sdk:lint(clippy-D warnings) all pass.name-mapping.audit.jsontwice via the new CLI command andconfirmed the two runs are byte-identical.
node tools/changeset-linter/src/cli.js check --fail-on-warningspasses.Screenshots (if appropriate):
N/A — CLI/data change only.
Types of changes
Checklist: