Add agentic reuse tooling: injection telemetry, meta-analysis, fideli…#1
Merged
Conversation
…ty, provider-agnostic LLM New modules: - injection_scan.py: deterministic prompt-injection telemetry (scan_injection / scan_injection_batch) over untrusted article/supplemental text; pure and side-effect-free, returns per-category counts and a bounded risk score. - meta_analysis.py: fixed-effect + DerSimonian-Laird random-effects cross-study synthesis, exposed as a reusable MCP tool. - fidelity.py: quantifies how closely an ODDA-reproduced omics result matches a published result (correlation, DE concordance, set overlap). - llm.py: provider-agnostic, bring-your-own-key LLM abstraction (credentials read from files/env, never hardcoded). Also registers the new MCP tools in main.py, extends the database schema, and adds a .gitignore for Python build artifacts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…code Implements the synthesis sandbox specified in the threat model as the run_analysis MCP tool (odda_utils.sandbox): executes untrusted, agent- generated analysis code in a hardened Apptainer container (--containall, --no-home, --net --network none, read-only root, single writable /work bind, read-only dataset binds, ulimit CPU/mem/file caps, wall-clock and output-size limits), behind a tamper-evident code-hash review gate with provenance recording. Adds analysis.def + build_images.sh for the image and tests/test_sandbox.py. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Surfaced during a db=pmc microglia-proteomics test ingestion of 47 studies. - PMC search enablement (articles/pubmed.py, fetching/pmc.py, main.py): search_and_fetch accepts db="pmc" and resolves PMC UIDs to PMIDs via the NCBI ID Converter, so [body]/full-text field tags work. - Fix PMC OA downloader for NCBI's package relocation (fetching/pmc.py, #50): oa.fcgi still advertises dead ftp .../pub/pmc/oa_package/ URLs; try the new /pub/pmc/deprecated/oa_package/ (HTTPS+FTP) and original paths in order, and fall through to Europe PMC instead of aborting a batch on one bad archive. - Make LLM-metadata parser tolerant of key-name variants (metadata/llm_metadata.py, #51): parse_llm_response now normalizes keys (e.g. "dataset ID" -> dataset_id) so processed_data/raw_data entries are no longer silently dropped. - Fix validator DOI extraction and add NCBI backoff (article_validation.py, main.py, #52): extract article-scoped IDs instead of a descendant search that reached the cited-reference list (which returned unrelated, repeated DOIs); add 429/5xx exponential backoff honoring Retry-After and optional NCBI API key. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implements a reusable relevance gate so cross-study aggregation only
pools studies that directly measure the analyte of interest in the
correct biological system/compartment under the correct contrast.
- New MCP tool score_study_relevance (odda_utils.relevance): sends only a
bounded title+abstract+methods excerpt (or a cached measurement
descriptor) plus the research question to the configured chat model and
returns minimal JSON {score, directly_measures, reason} with output
tokens capped low. Runs the injection-telemetry scan on the untrusted
text first, escalates borderline cases to full text, applies the gating
policy (include >=0.7 & directly_measures; exclude <0.4; else flag), and
persists every judgement to study_relevance_scores so no study is
silently dropped.
- Ingestion-time measurement_descriptor captured on the existing LLM
extraction pass (biological system/cell type, measured compartment,
species, perturbations/contrasts, omics/assay), cached in
llm_measurement_descriptors for cheap reuse across questions.
- schema.sql: additive study_relevance_scores and
llm_measurement_descriptors tables + indexes; database.py insert/get
helpers; prompts and llm_metadata parsing/storage; main.py tool wiring.
- tests/test_relevance.py: offline unit tests (gating, excerpt, injection
capture, never-drop, escalation, descriptor context, DB persistence).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Enforces the cost/context invariant that whole omics matrices never enter model context. New odda_utils.table_summary.summarize_table (and MCP tool) reads a table with pandas and returns a hard-capped summary (shape, per-column dtype/null/uniqueness with numeric stats or top categorical values, and a few truncated example rows). Output is bounded along the row, column, and cell dimensions, so it can never reproduce the full matrix regardless of input size (e.g. a 4.9 MB / 20k-row matrix -> ~6.5 KB summary). Python remains the primary force for tables; heavy computation stays in the sandboxed run_analysis container and meta_analysis, which return only compact results. - src/odda_utils/table_summary.py: TableSummary/ColumnSummary + summarizer (CSV/TSV/delimited/Excel/Parquet/Feather; reports errors instead of raising at the tool boundary). - src/odda_utils/main.py: summarize_table MCP tool. - tests/test_table_summary.py: bounded-output, stats/top-values, column and row caps, TSV detection, missing-file, and JSON-serializability tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…ty, provider-agnostic LLM
New modules:
Also registers the new MCP tools in main.py, extends the database schema, and adds a .gitignore for Python build artifacts.