Add reusable devenv perf workflow helper#648
Add reusable devenv perf workflow helper#648schickling-assistant wants to merge 4 commits intomainfrom
Conversation
schickling-assistant
force-pushed
the
schickling/2026-05-10-devenv-ci-perf
branch
from
dbd32e0 to
29cbe6b
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 29cbe6bb1c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| fi | ||
| } | ||
|
|
||
| measure 'shell_eval_traced' '$ARTIFACT_DIR/traces/shell_eval_traced.json' '$DEVENV_BIN' '--trace-output' '$trace_file' '--trace-format' 'json' 'shell' '--no-reload' '--' 'true' |
There was a problem hiding this comment.
The traced probe passes '$ARTIFACT_DIR/traces/shell_eval_traced.json' as a single-quoted literal, so measure receives $ARTIFACT_DIR verbatim instead of the env-expanded directory. In this job, that causes mkdir -p "$(dirname "$trace_file")" and --trace-output file:$trace_file to target a literal ./$ARTIFACT_DIR/... path, so the JSON trace is written outside tmp/devenv-perf-ci and is missing from the uploaded artifact bundle. This only affects traced probes but breaks the main regression-tracking output this job is meant to capture.
Useful? React with 👍 / 👎.
2 participants
Summary
devenvPerfJobhelper and benchmark/artifact steps for first-party repos to embed in their normal CI workflow.devenv-perfjob inside effect-utilsCIrather than as a separate workflow.PNPM_STORE_DIR/npm_config_store_dirper workspace, avoiding cross-workspace projection churn.Validation
bash nix/devenv-modules/tasks/shared/tests/pnpm-task-smoke.test.shpasses.devenv tasks run test:genie --mode before --no-tui --show-outputpasses after updating the stale standalone-workflow assertion.devenv tasks run check:quick --mode before --no-tui --show-outputpasses.devenv tasks run check:all --mode before --no-tui --show-outputpasses after the CI integration change.check:allpasses with29cbe6bb1c5a0e54ae66bbed301ef4841534ba63pinned.check:allpasses with29cbe6bb1c5a0e54ae66bbed301ef4841534ba63pinned.Benchmarks
Effect-utils local benchmark before downstream adoption:
devenv shell --no-reload -- truedevenv shell --no-reload -- truedevenv tasks listDotfiles final-state benchmark on
dev3after linked-store fixes:devenv shell --refresh-eval-cache --no-reload -- truewith JSON tracedevenv shell --no-reload -- truedevenv tasks listdevenv processes --helpStack
Posted on behalf of @schickling
agent_nameagent_session_idagent_toolagent_tool_versionagent_runtimeagent_modelworktreemachinetooling_profile