refactor: move to src/ layout, organize package + tests into layer folders#409
Merged
Conversation
Phase 1 of a two-phase reorganization. The repo was a hybrid of one package
(java_codebase_rag/) and 23 loose top-level modules; everything now lives under
src/java_codebase_rag/ with the engine modules grouped into 7 layer subpackages.
Layout:
src/java_codebase_rag/{ast,graph,absence,search,index,mcp,analysis}/
App-shell modules (cli, jrag, config, pipeline, installer, ...) stay flat at
the package root, so their java_codebase_rag.X self-imports are unchanged.
Imports: every `import <loose>` / `from <loose> import` (including all deferred,
cycle-breaking ones) rewritten to absolute `java_codebase_rag.<layer>.<module>`
across 87 files.
cocoindex flow + builder (loaded by FILE PATH, not import): resolve paths via
__file__ arithmetic in pipeline.py and mcp/server.py (_FLOW_DIR/_FLOW_FILE/
_BUILDER_FILE) — kept import-free since pipeline.py must stay free of heavy ML
imports at load time. The flow/builder's own imports are now absolute package
paths, so they load under the editable install regardless of cwd. Kept the
relative `file:Class` cocoindex target with cwd = flow dir.
Config: pyproject -> [tool.setuptools.packages.find] where=["src"]; entry point
server:main -> java_codebase_rag.mcp.server:main. conftest editable-install
check + sys.path -> src/. pytest.ini +pythonpath = src.
Other rewrites: string-form monkeypatch/mock.patch targets (8 test files),
test_search_lexical regex + exec'd subprocess source, ~12 .py path-string
literals in tests, scripts/generate_edge_navigation.py (sys.path -> src/),
scripts/sync_agent_artifacts.py (install_data -> src/...).
Phase 2 (tests/ subfolder reorg to mirror src/) follows in a separate commit.
Verification: imports clean; jrag/java-codebase-rag/java-codebase-rag-mcp all
work; generate_edge_navigation.py --check passes. Full suite (CI-equivalent,
no heavy gate): 1293 passed, 14 skipped, 0 failed.
Co-Authored-By: Claude <noreply@anthropic.com>
Phase 2 of the src-layout reorganization.
- Move 81 test files into tests/{ast,graph,absence,search,index,mcp,analysis,
package,integration,_meta}/ mirroring the src/ layer structure. conftest.py,
_builders.py, pinned_ids.py stay at tests/ root (filenames unchanged).
- pytest.ini: pythonpath = src tests so _builders/_pinned_ids resolve from
subfolders under pytest's prepend import mode.
- Bump Path(__file__)-rooted path depth by +1 in 34 moved test files (moving a
file down one dir means every absolute target is one .parent further; .parents[N]
index bumped +1 too). Provably correct for __file__-rooted paths; conftest at
root left untouched. cwd-relative strings and tmp_path unchanged.
Full suite (CI-equivalent): 1293 passed, 14 skipped, 0 failed.
Co-Authored-By: Claude <noreply@anthropic.com>
HumanBean17
added a commit
that referenced
this pull request
Jul 12, 2026
#445) PR #443 added BM25 as a 3rd RRF list in _graph_expand_merge, but search_v2 never passed graph_expand=True to run_search — so the vector+graph+BM25 fusion was dormant for every user-facing search (jrag search, MCP search). Tests + eval stayed green only because they invoke run_search(graph_expand=True) directly, never through search_v2. The design spec's "Current" diagram claimed graph-expand was already on the live path; it wasn't (false since #409). - search_v2 (mcp_v2.py): pass graph_expand=(table == "java") at both run_search call sites — the primary path and the hybrid->vector FTS-missing retry. run_search already guards fusion to the java single-table path and degrades silently when the graph/FTS index is unavailable, so table=all/sql/yaml and airgapped installs are unaffected. search_payload routes through search_v2, so this covers jrag search + the MCP search tool. - tests: two contract tests through search_v2 (the wiring gap's blind spot). Negative control: forcing graph_expand=False makes them FAIL, proving the guard is non-vacuous and catches the dormancy. - docs: correct the design spec's false "Current" diagram with a post-merge note so the next implementer doesn't inherit the premise. ARCHITECTURE.md / DESIGN.md claims ("BM25 first-class on primary path") were false before and are now true — no edit needed. Full suite: 1535 passed, 38 skipped, 0 failed. Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Converts the repo from a hybrid layout (one
java_codebase_rag/package co-existing with 23 loose top-level modules) to a cleansrc/layout with the engine modules organized into 7 layer subpackages. Two commits, one per phase.Phase 1 —
src/move + fold (6f9850e)src/java_codebase_rag/{ast,graph,absence,search,index,mcp,analysis}/, each with an__init__.py. App-shell modules (cli, jrag, config, pipeline, installer, …) stay flat at the package root, so theirjava_codebase_rag.Xself-imports are unchanged.import <loose>/from <loose> import→java_codebase_rag.<layer>.<module>across 87 files (including all deferred, cycle-breaking imports).__file__arithmetic inpipeline.pyandmcp/server.py(_FLOW_DIR/_FLOW_FILE/_BUILDER_FILE), kept import-free sincepipeline.pymust stay free of heavy ML imports at load time. The flow/builder's own imports are now absolute package paths, so they load under the editable install regardless of cwd.pyproject.toml→[tool.setuptools.packages.find] where=["src"], entry pointserver:main→java_codebase_rag.mcp.server:main;conftest.pyeditable-install check + sys.path →src/;pytest.ini+pythonpath = src.monkeypatch/patchtargets,test_search_lexicalregex + exec'd subprocess source, ~12.pypath-string literals, andscripts/(import +install_datapath).Phase 2 —
tests/reorg (d7b4656)tests/{ast,graph,absence,search,index,mcp,analysis,package,integration,_meta}/mirroringsrc/.conftest.py,_builders.py,pinned_ids.pystay attests/root.pytest.ini:pythonpath = src testsso_builders/_pinned_idsresolve from subfolders.Path(__file__)-rooted path depth by +1 in 34 moved files (moving down one dir = every absolute target one.parentfurther);conftestat root left untouched.docs/,skills/,agents/untouched.scripts/got one import + one path fix each.Verification
jrag/java-codebase-rag/java-codebase-rag-mcpall work;scripts/generate_edge_navigation.py --checkpasses.Caveats
JAVA_CODEBASE_RAG_RUN_HEAVY=1, 2 e2e tests fail on agap=5file-count assertion. Verified the flow file's diff vsmasteris empty outside import lines (zero logic change) and these tests run cocoindex directly (bypassing the pipeline/server edits), while 34 other heavy tests pass — so the discrepancy is byte-identical tomasterand these tests skip in CI. Pre-existing, not caused by this refactor.docs/may still cite old paths (server.py,build_ast_graph.py) in prose — left untouched per scope; worth a docs pass if desired.🤖 Generated with Claude Code