🌊 Ripple Consensus

Knowledge base consensus that evolves as your knowledge grows — not a snapshot frozen at ingestion time.

---

The Problem

Documents in a knowledge base are not islands. A document's "credibility" depends on how many other documents talk about the same thing.

Scenario	Your Judgment
One person says something	You might not believe it
Ten people say the same thing	You tend to believe it
Everyone says it	You trust it implicitly

But most KBs compute consensus once at ingestion, then never update it:

Doc A enters (the only doc about X) → NOVEL ← frozen forever
Docs B, C, D... follow, all about X → A is still NOVEL ← counterintuitive

The Solution

Ripple Consensus is designed for exactly this:

            💧 New Doc D
             ↓
    ○ ～ ～  ○ ～ ～  ○
   Doc A  ripple  Doc B  ripple  Doc C
  ↑0.30        ↑0.09      ↑0.027

When a new document enters the KB, its "influence" travels along FTS5 similarity chains via BFS, exponentially decaying with each hop until it converges (becomes negligible). Every document it reaches gets its consensus score updated in real time.

Consensus should not be a static snapshot captured at ingestion time — it should be a live state that evolves as the knowledge base grows.

---

Quick Start

# 1. Install dependencies (jieba only)
pip install jieba

# 2. Run the self-contained demo (no external DB needed)
python ripple_demo.py

The demo automatically creates an in-memory SQLite + FTS5 database, inserts 9 sample documents, and walks through the full ripple lifecycle:

1. Full backfill — compute initial consensus
2. Ripple propagation — new doc triggers neighbor consensus updates
3. Isolated doc test — novel concept stays NOVEL
4. Fusion ranking — consensus-weighted re-ranking of search results

---

Core Concepts

Ripple Propagation

Hop 0:   New doc D itself                weight w₀ = 1.0
Hop 1:   D's direct neighbors            weight w₁ = λ (0.3)
Hop 2:   Neighbors of neighbors          weight w₂ = λ² (0.09)
Hop k:                                    weight wₖ = λᵏ
When w < ε (0.05) → stop

Consensus → Interpretable Labels

peer_count ≥ 3 → CORROBORATED  # "Three people make a tiger"
peer_count ≥ 1 → EXTENDED      # At least one corroborating source
peer_count = 0 → NOVEL         # Solitary evidence

Fusion Ranking

score(i) = 0.35 × rel(i) + 0.40 × consensus(i) + 0.15 × quality(i) + 0.10 × freshness(i)

Consensus has the highest weight (0.40) — group corroboration is more reliable than single-match text relevance.

---

Repository Contents

ripple-consensus/
├── ripple_consensus.py          # 🌊 Ripple propagation engine (core)
├── ranking_fusion.py            # 🔗 Feature fusion ranking engine
├── ripple_demo.py               # 🎮 Self-contained demo (no external KB)
├── docs/
│   └── technical_report.md      # 📖 Full technical proposal
├── requirements.txt             # Dependencies
├── LICENSE                      # MIT
└── .gitignore

Key Files

File	Purpose
ripple_consensus.py	BFS ripple engine. ripple_from_doc() = entry point, backfill_all() = full KB backfill, consensus_boost() = search score amplification
ranking_fusion.py	4-feature fusion (relevance + consensus + quality + freshness). re_rank_results() plugs into search pipelines
ripple_demo.py	Self-contained demo. Creates in-memory DB + 9 sample docs, shows full lifecycle

---

KB Integration

Ripple Consensus is designed for incremental updates — it drops into existing knowledge pipelines:

Ingestion Pipeline
  │
  ├── Write new doc to okf_concepts + okf_fts (FTS5 index)
  │
  ├── Call ripple_from_doc(conn, new_id)
  │      ├── BFS along FTS5 similarity chains
  │      ├── Updates properties.consensus for all affected docs
  │      └── Commits transaction
  │
  └── On search: call consensus_boost() or re_rank_results()

DB Requirements

• Table okf_concepts: id TEXT PK, body TEXT, properties TEXT(JSON)
• Virtual table okf_fts: FTS5 index covering id, title, body
• SQLite json_extract for querying properties

See docs/technical_report.md sections 8 (storage format) and 13 (full SQL schema).

---

Tuning Parameters

# ripple_consensus.py
LAMBDA = 0.3            # Decay coefficient (0.1-0.5): higher = farther ripples
MAX_HOPS = 3            # Max propagation hops (2-5): 2 is enough for small KBs
EPSILON = 0.05          # Convergence threshold (0.01-0.1): lower = more sensitive
MIN_SCORE = 0.3         # Min neighbor similarity (0.1-0.5): higher = stricter
CORROBORATED_THRESHOLD = 3  # Peers needed for CORROBORATED (2-5)

# ranking_fusion.py
ETA = {
    "relevance": 0.35,   # Text relevance
    "consensus": 0.40,   # Consensus corroboration ← highest weight
    "quality": 0.15,     # Source quality
    "freshness": 0.10,   # Recency
}

---

Roadmap

Phase 1 ✅ Document-level consensus — in production
Phase 2 ✅ Feature fusion ranking — in production
Phase 3 🚧 Multi-factor edge weights + source independence
Phase 4 🚧 Document-entity bipartite graph
Phase 5 🔭 Heterogeneous knowledge graph expansion

See docs/technical_report.md section 10 for details.

---

Evaluation Metrics

Metric	Description
NDCG@K	Ranking quality
Consensus Lift	Improvement from adding consensus as a feature
Source Diversity	Diversity in top-K results
Graph Precision	Entity graph accuracy from high-consensus docs

---

License

MIT License. See LICENSE for details.

---

Ripple Consensus — Let your knowledge base grow wiser together.