Hoist artifacts to top level software factory workspace and incorporate tests#4192
Open
Hoist artifacts to top level software factory workspace and incorporate tests#4192
Conversation
….com/cardstack/boxel into ian-highly-experimental-dark-factory
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4734ea3eef
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Contributor
There was a problem hiding this comment.
Pull request overview
This PR standardizes the software-factory package layout and adds a Playwright-based CI loop to validate “adopt from public module” rendering using local realm fixtures.
Changes:
- Added Playwright configuration, global setup/teardown, and browser specs for the software-factory harness.
- Introduced CLI utilities + runtime metadata plumbing to share support-service context across commands/tests.
- Updated CI to detect
packages/software-factory/**changes and run the new Playwright job.
Reviewed changes
Copilot reviewed 139 out of 141 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| packages/software-factory/tsconfig.playwright.json | Adds a Playwright-specific TS config for test compilation. |
| packages/software-factory/tsconfig.json | Expands TS options to support decorators, TS extension imports, and Playwright config files. |
| packages/software-factory/tests/darkfactory.spec.ts | Adds Playwright coverage for adopting tracker cards from a public module URL. |
| packages/software-factory/test-fixtures/public-software-factory-source/index.json | Adds a public-source fixture realm entry card. |
| packages/software-factory/test-fixtures/public-software-factory-source/home.gts | Adds a minimal Home card for the public-source fixture realm. |
| packages/software-factory/test-fixtures/public-software-factory-source/darkfactory.gts | Adds a barrel export fixture for DarkFactory types. |
| packages/software-factory/test-fixtures/public-software-factory-source/darkfactory-schema.gts | Adds schema fixture for DarkFactory tracker card types. |
| packages/software-factory/test-fixtures/public-software-factory-source/.realm.json | Adds realm metadata for the public-source fixture realm. |
| packages/software-factory/test-fixtures/darkfactory-adopter/ticket-demo.json | Adds adopter fixture instance for Ticket adopted from public module. |
| packages/software-factory/test-fixtures/darkfactory-adopter/project-demo.json | Adds adopter fixture instance for Project adopted from public module. |
| packages/software-factory/test-fixtures/darkfactory-adopter/knowledge-article-demo.json | Adds adopter fixture instance for KnowledgeArticle adopted from public module. |
| packages/software-factory/test-fixtures/darkfactory-adopter/index.json | Adds adopter fixture realm entry card. |
| packages/software-factory/test-fixtures/darkfactory-adopter/home.gts | Adds adopter fixture realm Home card. |
| packages/software-factory/test-fixtures/darkfactory-adopter/factory-demo.json | Adds adopter fixture instance for DarkFactory adopted from public module. |
| packages/software-factory/test-fixtures/darkfactory-adopter/agent-demo.json | Adds adopter fixture instance for AgentProfile adopted from public module. |
| packages/software-factory/test-fixtures/darkfactory-adopter/Ticket/ticket-001.json | Adds adopter fixture instance under type folder layout. |
| packages/software-factory/test-fixtures/darkfactory-adopter/Project/demo-project.json | Adds adopter fixture instance under type folder layout. |
| packages/software-factory/test-fixtures/darkfactory-adopter/KnowledgeArticle/agent-onboarding.json | Adds adopter fixture knowledge article under type folder layout. |
| packages/software-factory/test-fixtures/darkfactory-adopter/DarkFactory/demo-factory.json | Adds adopter fixture factory under type folder layout. |
| packages/software-factory/test-fixtures/darkfactory-adopter/AgentProfile/demo-agent.json | Adds adopter fixture agent under type folder layout. |
| packages/software-factory/test-fixtures/darkfactory-adopter/.realm.json | Adds realm metadata for adopter fixture realm. |
| packages/software-factory/src/runtime-metadata.ts | Introduces support-metadata discovery + reading helper. |
| packages/software-factory/src/index.ts | Exposes harness API from the package entrypoint. |
| packages/software-factory/src/cli/smoke-realm.ts | Adds CLI command to fetch card JSON from an isolated realm server. |
| packages/software-factory/src/cli/serve-support.ts | Adds CLI command to start shared support services and persist context metadata. |
| packages/software-factory/src/cli/serve-realm.ts | Adds CLI command to start the isolated realm server and optionally emit metadata. |
| packages/software-factory/src/cli/serve-prerender.ts | Adds CLI command to start a prerender server and emit metadata. |
| packages/software-factory/src/cli/cache-realm.ts | Adds CLI command to prepare/reuse the cached template DB for a realm fixture. |
| packages/software-factory/scripts/pick-ticket.mjs | Adds Boxel helper script for ticket selection by query. |
| packages/software-factory/scripts/boxel-session.mjs | Adds Boxel helper script to build browser auth/session payloads. |
| packages/software-factory/scripts/boxel-search.mjs | Adds Boxel helper script to search a realm via CLI args. |
| packages/software-factory/realm/darkfactory.gts | Adds public barrel export for DarkFactory tracker types in the realm. |
| packages/software-factory/realm/darkfactory-schema.gts | Adds DarkFactory tracker schema in the realm. |
| packages/software-factory/prompts/.gitkeep | Removes placeholder file from prompts directory. |
| packages/software-factory/playwright.global-teardown.ts | Adds global teardown to stop support services and clean runtime dir. |
| packages/software-factory/playwright.global-setup.ts | Adds global setup to start support services + cache realm template. |
| packages/software-factory/playwright.config.ts | Adds Playwright config wired to global setup/teardown and fixed baseURL. |
| packages/software-factory/package.json | Adds scripts and devDependencies for the harness, Playwright, and CLI tooling. |
| packages/software-factory/experiment_1/tests/ticket-flow.spec.mjs | Adds experiment Playwright spec (currently outside main harness tests). |
| packages/software-factory/experiment_1/tests/helpers/realm-test.mjs | Adds experiment test helper for env-based realm config + auth seeding. |
| packages/software-factory/experiment_1/tests/helpers/boxel-auth.mjs | Adds experiment helper to seed Boxel auth into browser localStorage. |
| packages/software-factory/experiment_1/test-results/.last-run.json | Adds experiment Playwright last-run artifact file. |
| packages/software-factory/experiment_1/scripts/pick-ticket.mjs | Adds experiment Boxel helper script (duplicative with main scripts). |
| packages/software-factory/experiment_1/scripts/boxel-session.mjs | Adds experiment Boxel helper script (duplicative with main scripts). |
| packages/software-factory/experiment_1/scripts/boxel-search.mjs | Adds experiment Boxel helper script (duplicative with main scripts). |
| packages/software-factory/experiment_1/realms/guidance-tasks/index.json | Adds experiment realm index card for guidance-tasks. |
| packages/software-factory/experiment_1/realms/guidance-tasks/darkfactory.gts | Adds experiment DarkFactory barrel file with edit-tracking comments. |
| packages/software-factory/experiment_1/realms/guidance-tasks/Ticket/ticket-001.json | Adds experiment ticket instance fixture. |
| packages/software-factory/experiment_1/realms/guidance-tasks/Project/demo-project.json | Adds experiment project instance fixture. |
| packages/software-factory/experiment_1/realms/guidance-tasks/Project/02ecd21b-fd62-4e75-83d7-97da3af78b21.json | Adds experiment project instance fixture. |
| packages/software-factory/experiment_1/realms/guidance-tasks/KnowledgeArticle/agent-onboarding.json | Adds experiment knowledge article instance fixture. |
| packages/software-factory/experiment_1/realms/guidance-tasks/AgentProfile/adfdde24-d52d-4324-8f87-b46f42b2ea76.json | Adds experiment agent profile instance fixture. |
| packages/software-factory/experiment_1/playwright.realm.config.mjs | Adds experiment Playwright config. |
| packages/software-factory/experiment_1/package.json | Adds experiment package manifest and scripts. |
| packages/software-factory/experiment_1/.claude/skills | Links experiment Claude skills directory. |
| packages/software-factory/experiment_1/.agents/skills/software-factory-operations/SKILL.md | Adds experiment skill documentation for software-factory workflow. |
| packages/software-factory/experiment_1/.agents/skills/boxel-watch/SKILL.md | Adds experiment skill doc for boxel watch. |
| packages/software-factory/experiment_1/.agents/skills/boxel-track/SKILL.md | Adds experiment skill doc for boxel track. |
| packages/software-factory/experiment_1/.agents/skills/boxel-sync/SKILL.md | Adds experiment skill doc for boxel sync. |
| packages/software-factory/experiment_1/.agents/skills/boxel-setup/SKILL.md | Adds experiment skill doc for Boxel CLI setup. |
| packages/software-factory/experiment_1/.agents/skills/boxel-restore/SKILL.md | Adds experiment skill doc for restoring checkpoints. |
| packages/software-factory/experiment_1/.agents/skills/boxel-repair/SKILL.md | Adds experiment skill doc for repairing realms. |
| packages/software-factory/experiment_1/.agents/skills/boxel-development/references/dev-technical-rules.md | Adds experiment Boxel development reference docs. |
| packages/software-factory/experiment_1/.agents/skills/boxel-development/references/dev-styling-design.md | Adds experiment Boxel styling reference docs. |
| packages/software-factory/experiment_1/.agents/skills/boxel-development/references/dev-spec-usage.md | Adds experiment spec usage reference docs. |
| packages/software-factory/experiment_1/.agents/skills/boxel-development/references/dev-replicate-ai.md | Adds experiment Replicate reference docs. |
| packages/software-factory/experiment_1/.agents/skills/boxel-development/references/dev-quick-reference.md | Adds experiment quick reference docs. |
| packages/software-factory/experiment_1/.agents/skills/boxel-development/references/dev-query-systems.md | Adds experiment query reference docs. |
| packages/software-factory/experiment_1/.agents/skills/boxel-development/references/dev-fitted-formats.md | Adds experiment fitted-format reference docs. |
| packages/software-factory/experiment_1/.agents/skills/boxel-development/references/dev-file-editing.md | Adds experiment file editing reference docs. |
| packages/software-factory/experiment_1/.agents/skills/boxel-development/references/dev-external-libraries.md | Adds experiment external libraries reference docs. |
| packages/software-factory/experiment_1/.agents/skills/boxel-development/references/dev-enumerations.md | Adds experiment enumerations reference docs. |
| packages/software-factory/experiment_1/.agents/skills/boxel-development/references/dev-delegated-rendering.md | Adds experiment delegated rendering reference docs. |
| packages/software-factory/experiment_1/.agents/skills/boxel-development/references/dev-defensive-programming.md | Adds experiment defensive programming reference docs. |
| packages/software-factory/experiment_1/.agents/skills/boxel-development/references/dev-data-management.md | Adds experiment data management reference docs. |
| packages/software-factory/experiment_1/.agents/skills/boxel-development/references/dev-core-concept.md | Adds experiment core concept reference docs. |
| packages/software-factory/experiment_1/.agents/skills/boxel-development/references/dev-command-development.md | Adds experiment command development reference docs. |
| packages/software-factory/experiment_1/.agents/skills/boxel-development/SKILL.md | Adds experiment Boxel development skill doc. |
| packages/software-factory/README.md | Documents the harness, commands, and test/fixture layout. |
| packages/software-factory/.prettierignore | Excludes experiment and Playwright artifacts from formatting. |
| packages/software-factory/.gitignore | Ignores Playwright artifacts and cache directory. |
| packages/software-factory/.eslintignore | Excludes experiment and Playwright artifacts from linting. |
| packages/software-factory/.claude/skills | Links package Claude skills directory. |
| packages/software-factory/.agents/skills/software-factory-operations/SKILL.md | Adds package-level software-factory workflow skill doc. |
| packages/software-factory/.agents/skills/boxel-watch/SKILL.md | Adds package-level boxel watch skill doc. |
| packages/software-factory/.agents/skills/boxel-track/SKILL.md | Adds package-level boxel track skill doc. |
| packages/software-factory/.agents/skills/boxel-sync/SKILL.md | Adds package-level boxel sync skill doc. |
| packages/software-factory/.agents/skills/boxel-setup/SKILL.md | Adds package-level Boxel CLI setup skill doc. |
| packages/software-factory/.agents/skills/boxel-restore/SKILL.md | Adds package-level restore skill doc. |
| packages/software-factory/.agents/skills/boxel-repair/SKILL.md | Adds package-level repair skill doc. |
| packages/software-factory/.agents/skills/boxel-development/references/dev-technical-rules.md | Adds package-level Boxel development reference docs. |
| packages/software-factory/.agents/skills/boxel-development/references/dev-styling-design.md | Adds package-level styling reference docs. |
| packages/software-factory/.agents/skills/boxel-development/references/dev-spec-usage.md | Adds package-level spec usage reference docs. |
| packages/software-factory/.agents/skills/boxel-development/references/dev-replicate-ai.md | Adds package-level Replicate reference docs. |
| packages/software-factory/.agents/skills/boxel-development/references/dev-quick-reference.md | Adds package-level quick reference docs. |
| packages/software-factory/.agents/skills/boxel-development/references/dev-query-systems.md | Adds package-level query reference docs. |
| packages/software-factory/.agents/skills/boxel-development/references/dev-fitted-formats.md | Adds package-level fitted-format reference docs. |
| packages/software-factory/.agents/skills/boxel-development/references/dev-file-editing.md | Adds package-level file editing reference docs. |
| packages/software-factory/.agents/skills/boxel-development/references/dev-external-libraries.md | Adds package-level external libraries reference docs. |
| packages/software-factory/.agents/skills/boxel-development/references/dev-enumerations.md | Adds package-level enumerations reference docs. |
| packages/software-factory/.agents/skills/boxel-development/references/dev-delegated-rendering.md | Adds package-level delegated rendering reference docs. |
| packages/software-factory/.agents/skills/boxel-development/references/dev-defensive-programming.md | Adds package-level defensive programming reference docs. |
| packages/software-factory/.agents/skills/boxel-development/references/dev-data-management.md | Adds package-level data management reference docs. |
| packages/software-factory/.agents/skills/boxel-development/references/dev-core-concept.md | Adds package-level core concept reference docs. |
| packages/software-factory/.agents/skills/boxel-development/references/dev-command-development.md | Adds package-level command development reference docs. |
| packages/software-factory/.agents/skills/boxel-development/SKILL.md | Adds package-level Boxel development skill doc. |
| packages/realm-server/tests/helpers/index.ts | Loosens synapse registration secret sourcing for software-factory use. |
| packages/realm-server/main.ts | Adds env control to disable full indexing on startup. |
| packages/matrix/docker/synapse/index.ts | Formatting/structure changes and sync() call formatting. |
| packages/matrix/docker/index.ts | Minor type assertion cleanup when closing log files. |
| .github/workflows/ci.yaml | Adds software-factory change detection and a dedicated Playwright job. |
Files not reviewed (2)
- packages/software-factory/experiment_1/package-lock.json: Language not supported
- pnpm-lock.yaml: Language not supported
Comments suppressed due to low confidence (9)
packages/software-factory/src/index.ts:1
src/index.tsre-exports from./harness.js, but this package’stsconfig.jsonsetsnoEmit: trueand the README referencessrc/harness.ts. Ifsrc/harness.jsisn’t actually present in the repo, consumers will hit a runtime module-not-found. Prefer re-exporting from./harness.ts(sinceallowImportingTsExtensionsis enabled) or introduce an actual build step that emitsharness.jsand ensures entrypoints import the emitted extension.
packages/software-factory/src/cli/serve-support.ts:1runtime-metadata.tsreads the support context fromSOFTWARE_FACTORY_SUPPORT_METADATA_FILE, butserve-support.tswrites toSOFTWARE_FACTORY_METADATA_FILE. This mismatch makes it easy to configure one env var and have other commands not find the support context. Consider standardizing onSOFTWARE_FACTORY_SUPPORT_METADATA_FILEhere (or support both env vars consistently, with one as an alias) soreadSupportContext()and the writer agree.
packages/software-factory/src/cli/serve-realm.ts:1- Using
// @ts-nocheckon new CLI entrypoints defeats the newlint:typesstep for these files (they’re included bytsconfig.json). Prefer removing@ts-nocheckand addressing the specific typing issues (or use targeted@ts-expect-errorfor unavoidable cases) so the CI typecheck provides meaningful coverage.
packages/software-factory/tsconfig.json:1 - Several compiler options are contradictory or ineffective together (
noEmit: truemakesinlineSourceMap/inlineSourcesmoot, andnoEmitOnErroris irrelevant when nothing is emitted). This makes TS config harder to reason about. Consider removing the non-functional options, or, if emitting is intended later, moving source map settings into the build/emit config used for publishing.
packages/software-factory/tsconfig.json:1 - Several compiler options are contradictory or ineffective together (
noEmit: truemakesinlineSourceMap/inlineSourcesmoot, andnoEmitOnErroris irrelevant when nothing is emitted). This makes TS config harder to reason about. Consider removing the non-functional options, or, if emitting is intended later, moving source map settings into the build/emit config used for publishing.
packages/software-factory/src/runtime-metadata.ts:1 readSupportContext()will throw if the metadata file exists but contains partial/invalid JSON (e.g., writer crash, concurrent write, manual edit). Since this function is used to hydrate env for CLIs/tests, a hard crash can make failures harder to diagnose. Wrap the JSON parse in a try/catch and either returnundefinedwith a clear log/message includingmetadataFile, or throw a more actionable error that mentions the file path and expected shape.
packages/software-factory/playwright.global-setup.ts:1waitForMetadataFile()reads and parses the metadata file immediately once it exists. If the writer is still flushing (or writes non-atomically), this can produce flaky JSON parse errors. Consider either (a) writing the metadata atomically (write temp file + rename) in the producer, or (b) catching JSON parse errors here and continuing to retry until timeout.
packages/software-factory/playwright.global-teardown.ts:1process.kill(-pid, ...)relies on POSIX process groups and will fail on platforms that don’t support negative PIDs (notably Windows). If local contributors run these Playwright tests on Windows, teardown may fail to stop the background support process. Consider a platform-aware fallback (e.g., kill the direct PID when negative PIDs aren’t supported) or using a cross-platform process-tree termination strategy.
packages/software-factory/experiment_1/test-results/.last-run.json:1- This looks like a generated Playwright artifact (test runner output) and typically shouldn’t be committed. Consider removing
experiment_1/test-results/**from the repo and adding it to an applicable.gitignore(either at the repo root or withinpackages/software-factory) to avoid churn and accidental commits.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR normalizes a bunch of files in the software factory, adds some documentation and incorporates CI linting and testing for the software factory, and establishes our prototype factory that we'll begin iterating on. Shortly we'll remove the /experiment_1 folder (this is mostly duplicative), but i want to get a little farther before I do that.