feat(absence): disambiguate empty results (absent / external / refine / correct-empty)#405
Merged
Merged
Conversation
Design spec for disambiguating empty exploration results: an agent can't tell "refine my query" from "this symbol genuinely isn't in this project," so it loops on impossible searches. Proposes a 3-layer absence-diagnosis engine (verdict + cause + per-cause help) backed by a precomputed, versioned vocabulary index derived from the graph's Symbol nodes. Applies to all five exploration tools on both MCP and CLI. Proposed at propose/ABSENCE-DIAGNOSIS-PROPOSE.md. Co-Authored-By: Claude <noreply@anthropic.com>
PR-by-PR plan (PR-ABS-0..5) for the absence-diagnosis feature, grounded against source. Decomposes into shared types/config, the vocabulary index asset, the diagnosis module, MCP integration, CLI alignment, and docs. Corrects the spec's msgpack choice to stdlib JSON (msgpack is not a dependency; format is encapsulated behind absence_vocab load/save). Co-Authored-By: Claude <noreply@anthropic.com>
- Create absence_types.py with AbsenceDiagnosis DTOs (AbsenceProof, ExternalIdentity, VocabularyContext, FilterRelaxation) - Add optional absence: AbsenceDiagnosis | None = None field to 5 MCP output models (SearchOutput, FindOutput, DescribeOutput, NeighborsOutput, ResolveOutput) - Add config knobs: absence_close_threshold, absence_absent_floor, absence_candidate_count, absence_ngram_q, absence_diag_enabled - Add _pick_float and _pick_int helpers to config.py for numeric config parsing - Update pyproject.toml to include absence_types, absence_vocab, absence_diagnosis modules All changes are additive/no behavior change (absence stays None everywhere). Tests: test_absence_types.py (DTOs and output model fields), test_config.py (knobs with defaults). Co-Authored-By: Claude <noreply@anthropic.com>
- Add VocabularyIndex class with build/save/load/lookup/is_external methods - Build n-gram inverted index from LadybugGraph Symbol nodes - Persist as versioned JSON sidecar (vocab_index.json) - Hook build into write_ladybug after _write_meta, failure-isolated - Add jrag vocab-index subcommand for standalone rebuild - Add get_vocabulary_index() lazy-load helper with caching - Add test_absence_vocab.py with comprehensive coverage Co-Authored-By: Claude <noreply@anthropic.com>
PR-ABS-2: the stateless diagnose(...) classifier (Layer 2, the feature's core). Classifies an empty MCP result by cause and emits cause-specific help; pure function of its inputs incl. the VocabularyIndex. - absence_diagnosis.py: diagnose() with the exact kw-only signature; decision procedure: external-wins → neighbors → describe-by-node_id → find → search/resolve. Conservative two-band threshold policy defaults the middle band to refine_query; not_in_project only when best similarity < absence_absent_floor AND identifier-shaped. - did-you-mean uses difflib.SequenceMatcher (stdlib) on normalized_name (distance = 1 - similarity); no new deps (resolution 1). - False-absent guard: _exact_symbol_exists override forces refine_query when the query resolves to a real symbol — covers the FQN gap where vocab.lookup's simple_name match leaves a long FQN below the floor. closest_symbols/distances always populated regardless of verdict. - Filter relaxation ports _zero_result_guidance's tally-and-suggest logic into the structured FilterRelaxation payload (graph-backed, no argparse/mcp_v2 coupling). - Master toggle + exception guard: any failure degrades to a minimal refine_query (or None); diagnosis never fails the tool. - tests/test_absence_diagnosis.py: full unit matrix (16 tests) incl. the false-absent guard (middle-band + explicit exact + FQN), external prefix/phantom, neighbors correct_empty/refine_query, nl_miss, filter_miss (identifier + broad), master toggle, exception guard. Co-Authored-By: Claude <noreply@anthropic.com>
…mport; consistent role tally; unresolved-call test Co-Authored-By: Claude <noreply@anthropic.com>
- Add set_absence_config cfg holder in mcp_v2.py and resolve_service.py - Thread cfg from server.py to both modules - Wire search empty path: diagnose on hits=[] with tool=search, query - Wire find empty path: diagnose on refs=[] with tool=find, filt, filter_kind - Wire describe empty paths: diagnose on fqn miss (query=fqn) and node_id miss (query=None) - Wire neighbors empty path: diagnose on sliced=[] with tool=neighbors, root_node - Wire resolve none path: diagnose on status=none with tool=resolve, query, scope - Add integration tests for all 5 tools (13 tests, all passing) - All existing MCP v2, resolve, and hints tests still pass (180 passed total) PR-ABS-3
- Add Envelope.absence field with serialization (omit when None) - Thread out.absence from resolve-none into Envelope - Extend renderer: not_found shows verdict + did-you-mean - Extend traversal empty: verdict line for all 4 verdicts - Extend listing empty: verdict line instead of bare '0 <noun>' - Backward compatible: existing tests green Co-Authored-By: Claude <noreply@anthropic.com>
…ce; fix conservatism direction)
…k returns False for non-http
… fall through) + zero-edge test - Fixed indentation bug in _neighbors_meaningful_empty: moved 'return False' from inside 'if route_kind == "http_endpoint":' block to the route-block level - Added TestNeighborsMeaningfulEmptyPredicate with three tests: 1. test_zero_edge_kafka_topic_route_returns_false - isolates kafka route logic 2. test_http_endpoint_with_handlers_returns_true - verifies correct behavior 3. test_http_endpoint_without_handlers_returns_false - verifies handler-less routes - The new zero-edge-kafka test passes only with the dedent fix and would fail before Co-Authored-By: Claude <noreply@anthropic.com>
…; O(1) name lookup; docstring/kafka-rationale
HumanBean17
force-pushed
the
feat/not-part-of-project-feedback
branch
from
1581ab0 to
df3c592
Compare
HumanBean17
added a commit
that referenced
this pull request
Jul 8, 2026
Two test-only fixes; no production behavior change. Master went red on macOS Intel + Windows after #405 merged (those platforms were never exercised by these new tests before merge). macOS Intel (graph-only): the 4 search integration tests monkeypatch mcp_v2.run_search, which makes `lexical_mode = run_search is None` (mcp_v2.py:895) False, forcing the semantic path -> _get_sentence_transformer -> ModuleNotFoundError('sentence_transformers') on graph-only installs where the vector trio is gated off. Add the repo's needs_vectors skipif (mirrors test_mcp_v2.py:35-48) so they skip where the vector stack is absent. The absence classifier itself stays covered by test_absence_diagnosis.py unit tests on every platform. Windows: _find_symbol_with_edges ran `MATCH (s:Symbol)--() ... LIMIT 1` with no filter on s.resolved, so Kùzu's nondeterministic first row could be a phantom/external symbol -> diagnose classifies it external_dependency (external-wins precedence) instead of the expected refine_query. Add `WHERE s.resolved = true` (the codebase idiom, absence_diagnosis.py:526) to guarantee a genuine project symbol on every platform. Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Empty results from
search/find/neighbors/describe/resolvewere a bareresults=[]with no metadata — so an agent couldn't tell "refine my query" from "this symbol genuinely isn't in this project." Agents routinely looped on impossible searches. This PR gives every empty result anAbsenceDiagnosisthat disambiguates the two.Each empty result now carries a verdict (
refine_query|not_in_project|external_dependency|correct_empty), a cause, and cause-specific help (closest_symbols+distances+proof,vocabulary_context,filter_relaxation,external_identity).Architecture (3 layers)
absence_vocab.py: a precomputed vocabulary index (manifest + n-gram inverted index) derived from the graph'sSymbolnodes, persisted as a versioned sidecar JSON underJAVA_CODEBASE_RAG_INDEX_DIR, built at the end of graph build, lazily rebuilt if missing/stale.jrag vocab-indexrebuilds standalone.absence_diagnosis.py: a statelessdiagnose(...)— cause classifier,difflibdid-you-mean, a conservative two-band threshold, and per-cause help assembly.mcp_v2.py,resolve_service.py) and the CLI envelope/renderer (jrag_envelope.py,jrag_render.py); documented indocs/AGENT-GUIDE.md+docs/CONFIGURATION.md.PR breakdown
AbsenceDiagnosistypes,absencefield on 5 models, 5 config knobsjrag vocab-indexdiagnose(...)module — classifier + per-cause help + false-absent guardSpec:
propose/ABSENCE-DIAGNOSIS-PROPOSE.md· Plan:plans/active/PLAN-ABSENCE-DIAGNOSIS.md.Key invariants (verified)
not_in_project— 3-layer defense (conservative threshold + exact-match override + FQN scan), traced + tested.absenceis additive/optional; non-empty results areNone;absence_diag_enabled=falserestores pre-feature behavior.json+difflibonly. (The spec proposedmsgpack; the plan uses stdlib JSON since msgpack isn't a dependency. The format is encapsulated behindabsence_vocabload/save for a future swap.)diagnose(...)self-guards; build-hook failure is isolated from reprocess.Testing
Full suite: 1243 passed, 14 skipped. New tests:
test_absence_types.py,test_absence_vocab.py,test_absence_diagnosis.py,test_absence_mcp_integration.py,test_jrag_envelope_absence.py, + config-knob tests.🤖 Generated with Claude Code