fix(ci): sync npm package versions to release tag + unmask publish failures (2.12.1)#86
Merged
Merged
Conversation
…ilures (2.12.1)
Releases 2.11.0/2.12.0 never reached npm: the pipeline bumps pyproject/__init__
but never the npm manifests, so publish re-tried surrealmemory@2.10.5 and the
registry E403 ("cannot publish over previously published") was masked by an
`|| echo ::warning` fallback — the job stayed green while nothing shipped.
- publish-npm / publish-sdk / publish-vscode now `npm version "${GITHUB_REF_NAME#v}"
--no-git-tag-version` after install, before build, so the published version
always matches the tag
- publish failures fail the job loudly (no more warning masks); graceful skip
only for a missing token or an idempotent re-run (exact version already live)
- bump 2.12.1 across pyproject, __init__, and the three npm manifests (+locks);
npm intentionally skips 2.11.0/2.12.0 and jumps 2.10.5 -> 2.12.1
6 tasks
acidkill
added a commit
that referenced
this pull request
Jul 18, 2026
…s (2.13.0) (#88) * fix(docker): mount host ~/.claude/projects into dashboard for reasoning mining The reasoning-training dashboard runs inside the surrealdb compose container (HOME=/home/appuser), but the miner scans $HOME/.claude/projects — which was empty in the container. Every dashboard Mine finished in <1s with 0 traces and no error, so it looked like nothing happened while the host's 3200+ transcripts sat unmounted. Add a read-only, projects-only bind mount (not the whole ~/.claude); overridable via HOST_CLAUDE_PROJECTS. Verified: a backfill mine now ingests 502 traces and distills 9 patterns. * feat(reasoning): deep transcript discovery via rglob (nested sessions + subagents) Replace the one-level `projects/*/*.jsonl` glob with a recursive `_discover_transcripts` that finds nested session directories and Task-tool subagent transcripts (`.../subagents/agent-*.jsonl`), which the old glob silently missed (~1000 files on the real corpus). Path-escape guard now applies at every depth, not just the top level. `_project_from_path` uses the first path segment under `projects/` instead of the immediate parent directory, so subagent files attribute to their real project instead of a literal "subagents" pseudo-project. A stray file placed directly in `projects/` (no project subdirectory) is still skipped. * fix(reasoning): remove per-scan trace cap + bump char-safety limit to 100k max_traces_per_scan silently truncated a scan once the running total hit 500, which loses full mineable history on the intended deep/backfill scans (u1). The recorded incremental state already always reflects a whole-file scan, so the cap only ever dropped later files in the same run for no safety benefit -- removed the field and its enforcement. max_trace_chars (per-trace content cap, the real safety valve now that there's no scan-wide cap) is bumped 20k->100k since real traces were being truncated with no evidence of runaway content. Exposed min/max_trace_chars on PUT /config for operator tuning. from_dict() already only reads known keys, so an old config.toml/dict still carrying max_traces_per_scan loads fine with the key silently ignored. * feat(reasoning): backfill = full re-scan bypass (not state deletion) scan_transcripts / ingest_reasoning_traces gain a backfill kwarg (default False). New _plan_file_scan helper: on a normal scan it applies today's incrementality exactly (size+mtime skip, start_line resume, lookback cutoff); on backfill it bypasses all three and reads every discovered file from line 0. Dedup-by-trace_hash at insert keeps the re-emission harmless, and scan state is always written per file (never deleted) so a later normal scan stays cheap. The route _run_mining, CLI _mine and MCP _reasoning_mine forward backfill; background consolidation never sets it (full rescan is explicit-only). * feat(reasoning): per-file ingest with asyncio.to_thread + progress callback ingest_reasoning_traces now discovers the transcript corpus, then scans and INSERTS one file at a time instead of gathering every trace into one list: each file's blocking read runs in asyncio.to_thread and its traces are staged immediately before the next file opens. Peak memory is one file's traces regardless of corpus size — what makes the un-capped full-corpus scan safe. New engine/reasoning_progress.py: MiningProgress snapshot (phase, file/trace counters, per-model distill counters) + ProgressCallback, emitted only from the event-loop thread as an immutable copy. ingest gains an optional progress kwarg (wired into the route/CLI/MCP in a later unit); ReasoningIngestResult gains files_total/files_scanned. Scan-state is saved every 50 files and in a finally block, and a file's state entry is recorded only after its traces are inserted, so a crash mid-ingest leaves the failed file un-recorded (re-scanned next run) while completed files persist. scan_transcripts stays a thin sync wrapper for the existing tests. * feat(reasoning): per-model pattern targets replace global run cap Distillation is now governed by per-model targets (reasoning_training.pattern_targets: model -> 0..100) instead of a single global max_patterns_per_run. For each detected source model the budget is max(0, target - existing_patterns_for_that_model); a model whose target is unset/0 or already met is skipped entirely and its traces stay unprocessed — so a preliminary Mine with no targets set only DETECTS models without distilling. A manual POST /mine (drain=True) drains each model to its budget or backlog exhaustion, marking consumed traces processed per batch and breaking on any no-forward-progress batch; background consolidation still does one batch per model per run. max_patterns_per_run is removed everywhere (a legacy TOML/dict key is now silently ignored on load). pattern_targets round-trips through config.toml (new subtable), to_dict/from_dict (keys validated, values clamped), and PUT /config (422 on a bad/thinking-less model). The reasoning-pattern fetch ceiling is raised 5000 -> 20_000 across distiller/route/CLI/MCP/injection for full-corpus coverage. * feat(reasoning): live mining progress in status, logs, CLI, MCP MiningJobState + _idle_mining_state gain additive progress fields (phase, files_total/scanned, traces_found/processed, current_model, models_done/total). _run_mining wires the MiningProgress callback from ingest and distill into the per-brain job state, merging per phase so distill snapshots don't zero the ingest file counters; phase advances scanning -> ingesting -> distilling -> done and GET /status reflects it live. logger.info at mining start, after ingest, and at end make the milestones visible in docker logs; the app lifespan now routes surreal_memory.* INFO to stdout (guarded so it never double-logs under an external log config) since uvicorn configures only its own loggers. CLI mine prints phase/file/model progress (human output only, --json stays clean) and returns files_scanned; MCP _reasoning_mine returns files_scanned/files_total. Tests: idle /status exposes the new fields; a _run_mining run with fake ingest/distill shows an intermediate ingest snapshot then a terminal phase=done, running=False state. * feat(reasoning): mine opus-4-8 (empty the thinking denylist) _MODELS_WITHOUT_THINKING is now an empty tuple: run-007 assumed claude-opus-4-8 emitted signature-only thinking, but the real corpus has ~933 opus-4-8 traces averaging ~1166 chars of genuine reasoning, so opus is mined like any other model. The denylist mechanism (_is_denylisted, the route _has_thinking / PUT-config 422 path) is kept for a future thinking-less model. Split test_synthetic_and_opus_are_skipped into test_synthetic_skipped + test_opus_4_8_is_mined; the route 422 / has-no-thinking / denylisted-target tests now pin a fictional denylisted model instead of relying on opus. * feat(dashboard): per-model target sliders + live mining progress Reasoning tab: new PatternTargetsCard renders a 0-100 (step 10) range slider per mineable model with its current pattern & trace counts; moving a slider PUTs the full pattern_targets map (optimistic via the shared config mutation) and shows a 'set a target and run mining' hint at 0. MiningConfigCard gains a live progress block while a run is active — phase label, files_scanned/total bar, '{found} found / {new} new' during ingest, '{model} ({done}/{total}) / {patterns} patterns' during distill. Types updated: MiningJobState +progress fields, ReasoningConfig -max_traces_per_scan -max_patterns_per_run +pattern_targets, ReasoningConfigUpdate +min/max_trace_chars +pattern_targets; status poll 3000->1500ms while running; en.json target/progress keys; e2e mock aligned to the new shape. Rebuilt + committed dist. * docs: changelog for run-008 reasoning mining (full corpus + targets + progress) Document under [Unreleased]: deep transcript discovery incl. subagents; per-model pattern targets (0-100 sliders) replacing max_patterns_per_run; live mining progress in status/dashboard/CLI/docker logs; backfill = true full re-scan; opus-4-8 now mined; per-scan cap removed + max_trace_chars raised to 100k. MCP/CLI generated docs verified up-to-date (no input-schema changes). * chore: release 2.13.0 — full-corpus reasoning mining Bump version 2.12.1 -> 2.13.0 across pyproject.toml, __init__.py, and the three npm manifests + lockfiles (surreal-memory-client, surrealmemory, vscode-extension), matching the sync convention established by #86. Rename CHANGELOG [Unreleased] -> [2.13.0]. Update the one hardcoded-version test. --------- Co-authored-by: Toni Nowak <toni@nowak.sh>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
npm packages have been silently stuck at 2.10.5 — releases 2.11.0 and 2.12.0 never shipped to npm.
Root cause (from the v2.12.0 release run logs):
The release pipeline bumps
pyproject.toml+__init__.pybut never the npm manifests, so every publish re-tried the stalepackage.jsonversion — and the|| echo "::warning::…"fallback masked the E403 as a green job.Fix
publish-npm/publish-sdk/publish-vscode: runnpm version "${GITHUB_REF_NAME#v}" --no-git-tag-version(after install, before build) so the published version always equals the tag — the drift class is eliminated structurally.pyproject.toml,__init__.py, the version test, and all three npm manifests (+lockfiles). npm intentionally skips 2.11.0/2.12.0 and jumps 2.10.5 → 2.12.1.Test plan
uv run make verifygreen locally (CI-equivalent): 6543 passed, 84 skipped, coverage 69.77%, lint/type/security clean.release.ymlparses as valid YAML;$GITHUB_REF_NAMEused as a runner env var (no${{ }}script interpolation).v2.12.1→ release run must show real publish results (green = actually published); registries verified post-release (PyPI + npm × 2 + marketplace).