Skip to content

v0.10: reliable memory — degraded transparency, reranking, usage observability, dedup, reactive auth#11

Merged
renezander030 merged 1 commit into
mainfrom
v0.10-reliable-memory
Jul 20, 2026
Merged

v0.10: reliable memory — degraded transparency, reranking, usage observability, dedup, reactive auth#11
renezander030 merged 1 commit into
mainfrom
v0.10-reliable-memory

Conversation

@renezander030

Copy link
Copy Markdown
Owner

Five focused improvements that make ATS's memory more trustworthy and observable. Each extends existing infrastructure (RRF fuse, corpus cache, usage log, typed links, adapter auth) rather than rewriting it. Full CI green — npm test: lint + pii + claims + 239 fast tests + all prove:* suites.

What's in it

1. Degraded-retrieval transparency

find now signals when a result is partial instead of silently serving a subset.

  • New top-level degraded boolean + warnings[] + corpus.sourcesFailed[].
  • A dropped corpus source (a composite child that errored, or a single project that failed to list) and a retrieval branch that errored/timed out roll up into that signal.
  • The composite adapter records which child it dropped (was: Promise.allSettled silently filtered failures away).
  • A known-partial corpus is no longer written to the cache as if it were complete.

2. Optional reranking

find gains a second-stage reranker over the RRF-fused pool.

  • rerank: true → built-in, dependency-free lexical scorer (query-term coverage + title/phrase weighting). Or pass a function to plug a cross-encoder/LLM.
  • Fuses a wider rerankDepth pool, then trims to limit. A failing reranker degrades to the fused order (surfaced via degraded).
  • CLI: ats find "q" --rerank [--rerank-depth N].

3. Usage observability

  • Retrieval calls now log durationMs.
  • New ats usage (--json, or markdown) reports per-tool volume, empty/error/degraded rates, latency (avg + p95), re-query pairs, and top queries.
  • The analysis lives in one core summarize() shared by the CLI and bench/analyze-usage, so both agree by construction.

4. Duplicate / contradiction detection

  • New ats dedup: scans the corpus for near-duplicate task clusters (dependency-free Jaccard over title + content + tags, transitively grouped) and flags field-level disagreements (status/priority/due) within a cluster as potential contradictions.
  • Detection only — it never edits. Pairs with the existing conflicts-with / supersedes link types (the action you take on a finding).

5. Reactive token refresh

  • The TickTick adapter refreshes its OAuth token and retries once on a 401 — the revoked / invalidated-early / wrong-stored-expiry cases the existing proactive clock-based refresh misses — then surfaces an actionable "run ats auth login".

Tests

+19 new tests (retrieval degraded/rerank, usage summarize, dedup, 401-retry). Full fast suite: 239 pass. npm test (incl. lint / pii / claims / prove:*) green.

Notes for review

  • Two originally-scoped items were dropped after reading the code: an MCP-server HTTP/OAuth transport (out of scope — ATS is CLI-first) and a corpus delta-sync (re-embedding is already content-hash cached in embedding.js, and a true fetch-delta is blocked by the TickTick Open API v1 having no modified-since endpoint).
  • Version not bumped and nothing published — review-only. CHANGELOG entry added under ## 0.10.0 - Unreleased.

…rvability, dedup, reactive auth

- find(): degraded/warnings rollup + corpus.sourcesFailed; composite records dropped children; a partial corpus is no longer cached as complete
- find(): optional rerank stage (built-in lexical scorer or custom fn) over a widened rerankDepth pool; degrades gracefully if the reranker fails
- observability: capture durationMs; new `ats usage` (JSON/markdown) via a shared usage-log summarize() — per-tool volume, empty/error/degraded rates, latency avg+p95, re-queries, top queries
- `ats dedup`: dependency-free corpus near-duplicate clustering (Jaccard, transitive) + status/priority/due conflict flags; detection only
- TickTick api: 401-reactive token refresh + retry-once, with an actionable re-auth error
@renezander030
renezander030 merged commit 5d71878 into main Jul 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant