Flush and reload

[giveen]

So I do this in my local agents and tools I've made.

In the config, I sent the context size.

CONTEXT=256K

then at 80%, I force the agent to summarize what has been done.

Then with llama.cpp you can order it to flush the context.

Then have the agent, reread where it was at in the summarization prompt and resume.

[giveen]

I also do this with a smaller model, a subagent if you will, that has its own context window.

[mlhher]

I think this is a good chance to think about what should happen in #33.
I have the token counter basically implemented and with llama.cpp it works fine showing percentages though I do agree that I'd like it to work gracefully. The issue I have with the summary is that it just degrades reasoning.

What do you think about e.g. simply prompting the user after some buffer (e.g. 1k tokens) "Hey your session is close to the limit please restart".

For the summarization; since I have seen it multiple times now I am looking into it but I really do not want to make it invisible as it degrades reasoning and speed. It should always be explicitly done by the user. we could say "Hey your session is close to the limit please restart. If you want to continue with this context please use "Summarize"" or something similar.

[giveen]

Context Lifecycle Management: A Tiered Architecture for Local LLMs

Goal: Maintain high-signal code context while preserving the ability to "zoom in" on specific symbols without exceeding the model's context window (n_ctx).

---

1. Structured Skeleton Pack

On each turn, construct a compact context pack with four immutable sections.

• Objective Pin: Current task, acceptance criteria, and non-negotiable constraints.
• State Pin: Current files touched, latest compiler/runtime errors, and pending edits.
• Code Skeletons: For relevant files, include:
  • Imports and type/class signatures.
  • Full body for the target function only.
  • Signatures only for sibling/helper functions.
• Risk Pin: Global mutable state, side-effect functions, IO boundaries, and concurrency-sensitive paths.

Why this works: The model retains the architectural "big picture" and interface definitions, but only consumes tokens for function bodies where reasoning is actively required.

---

2. Expansion as a First-Class Operation

Expansion must be an explicit tool call/request type, not an ad-hoc dumping of file contents.

Expansion API Shape

• Expand Symbol: Retrieve the full body for a specific function or method.
• Expand Neighborhood: Retrieve callers, callees, and adjacent helpers for a symbol.
• Expand Region: Retrieve a specific line range (e.g., for stack trace analysis).
• Expand Contract: Retrieve related type definitions, constants, or interfaces used by the target.

Requirement: The agent must provide a reason field for expansion (e.g., "Verifying side effects in helper function X before refactoring"). This makes expansion intentional and auditable.

---

3. Task-Local Expansion Cache

Treat expanded content as working memory that is cached per task.

• Cache Keys: task_id + file_hash + symbol_id + expansion_mode.
• Policy:
  • Keep recent expansions "hot" in the prompt.
  • Deduplicate repeated requests.
  • Invalidate cache entries immediately upon file content changes.
  • Prioritize symbol-level granularity over whole-file caching.

---

4. Utility-Based Re-Skeletonization

When the context budget crosses a threshold, compress older expansions back into skeleton form using a utility score rather than simple FIFO.

Utility Score Formula:
Utility = Recency + Dependency Distance + Error Relevance + Edit Likelihood

Eviction Order (Low Utility First):

1. Old expanded regions distant from the current cursor.
2. Stale diagnostic chatter or old error logs.
3. Explanatory prose or documentation not tied to current edits.

Never Evict: Objective Pin, Active TODO checklist, Current Error, or the rationale for the last accepted code change.

---

5. The Fact Ledger (Anti-Drift)

To prevent "semantic drift" during summarization, maintain a Fact Ledger.

• Entries:
  • What changed and why.
  • Invariants that must remain true.
  • Open risks and verification status.
• The Validation Gate: After any compression or summarization step, the model must verify it can still answer:
  1. What is the primary task?
  2. Which files are currently in play?
  3. What invariants must not be broken?

---

6. Expansion Heuristics

Expand Immediately When:

• The target function calls more than N local helpers.
• An error stack trace points into a non-expanded helper.
• A changed type signature affects multiple downstream functions.

Delay Expansion When:

• The agent is still in the "Planning" phase.
• Changes are localized to formatting, documentation, or trivial guard clauses.

---

7. Configuration Thresholds

Suggested defaults for managing the token budget:

Threshold	Mode	Action
< 60%	Standard	Skeleton-only mode for background files.
60% - 80%	Austerity	Controlled expansion; aggressive re-skeletonization of low-utility nodes.
80% - 90%	Panic	Re-skeletonize all non-active files; Fact Ledger + Active Function only.
> 90%	Hard Safe	Disable large tool calls; force summarization before next turn.

---

8. Success Metrics

Track these KPIs to validate the architecture:

• Tokens per Successful Edit: (Lower is better).
• JSON/Tool-Call Error Rate: Frequency of malformed payloads near context limits.
• Regression Rate: Frequency of bugs introduced due to "forgotten" invariants.
• Expansion Efficiency: Ratio of expansion requests to lines of code changed.

[giveen]

Proposal: Context-Resilient Coding Flow for Local Models

TL;DR

We can keep long coding sessions sharp by replacing raw file blobs with dynamic code skeletons, then allowing targeted expansion only when needed. This keeps token usage low, reduces malformed tool-call risk near context limits, and preserves reasoning quality with explicit safeguards.

Why This Matters

• Context Saturation: Full-history and full-file context causes brittle behavior near limits.
• Reasoning Degradation: Summarization alone can lose critical details.
• The Hybrid Solution: Deterministic context shaping followed by selective retrieval/expansion.

Design Overview

1. Start each turn with a compact skeleton context pack.
2. Enable explicit symbol expansion when uncertainty is high.
3. Cache expanded windows per task.
4. Re-skeletonize low-utility expansions as the budget nears its limit.
5. Preserve critical facts in a non-evictable pin lane.
6. Validate summaries to ensure essential details are never silently dropped.

Context Pack Structure

• Objective Pin: Task goal, acceptance criteria, constraints.
• State Pin: Active files, current errors, pending edits.
• Skeleton Lane: Imports, types, signatures, and full body only for the target symbol.
• Working Lane: Expanded symbols currently being edited/debugged.
• Safety Lane: Invariants, side effects, risky boundaries (IO, concurrency, migrations).

Expansion Contract

Expansion is an explicit tool call.

• Expand Symbol: Full body for one function/method.
• Expand Neighborhood: Callers, callees, related helpers.
• Expand Contract: Related types/constants/interfaces.
• Expand Range: Precise lines for local block analysis.

Requirements for each request:

• Reason for expansion.
• Expected outcome.
• Scope limit.

Caching Strategy

• Keying: task_id + file_hash + symbol_id + expansion_type.
• Lifecycle: Reuse across turns in the same task; invalidate on file hash change.
• Management: Cap total token footprint with utility-based eviction.

Re-Skeletonization Policy

• Threshold: Trigger at a soft limit (e.g., 80% of budget).
• Logic: Evict low-utility expanded windows first.
• Preservation: Keep non-evictable lanes (Pins) always present.
• Conversion: Turn stale windows back into signatures with concise notes.

Utility Scoring for Eviction

1. Recency of use.
2. Dependency distance to active target.
3. Relevance to current error/test failure.
4. Probability of near-term edits.

Summary Safety Guard

Post-compression continuity check:

• What is the exact objective now?
• Which files/symbols are active?
• What changed and why?
• What remains unresolved?
• Which invariants must hold?
• Action: Auto-expand missing context if any answer is weak.

Near-Limit Safe Mode

• Disable large inline payloads.
• Enforce compact subagent briefing formats.
• Prefer file references over embedded text.
• UX: Offer explicit choices: restart, summarize, or continue with reduced scope.

Suggested Rollout

• Phase 1: Request preflight budget checks and hard guardrails.
• Phase 2: Skeleton context pack + explicit expansion contract.
• Phase 3: Task-local expansion cache + utility eviction.
• Phase 4: Summary validation and safe mode UX.
• Phase 5: Optional embedding-backed semantic retrieval lane.

Success Metrics

• Reduced tokens per successful edit cycle.
• Lower near-limit request/tool-call failure rate.
• Stable or improved fix correctness.
• Fewer repeated context reads for the same symbols.

Scope Boundary

Included: Deterministic shaping, selective expansion/cache, safe-mode/validation.
Excluded (First Pass): Fully autonomous hidden summarization, large external vector infra requirements.

[giveen]

Based off this design idea, I am working a new branch and will test it out on some of my larger code projects.

[mlhher]

Feel free to report back I am not entirely sure I get the idea so far

[giveen]

I need to test it out now:

Context Lifecycle Management (CLM) Implementation Summary

What Changed

1. Structured Per-Turn Context Packing (agent.go)

• Tiered Assembly: Every turn now constructs a structured prompt with dedicated lanes: Objective Pin, State Pin, Context Skeletons, Working Lane, Risk Pin, and Fact Ledger.
• Skeletonization: File context is no longer a raw dump. We provide signatures, types, and imports by default, reserving full function bodies for explicitly expanded symbols only.

2. Explicit Expansion Contract (context_manager.go)

• Targeted Tools: Introduced expand_symbol, expand_neighborhood, expand_contract, and expand_range.
• Intent Tracking: Each request requires reason, expected_outcome, and scope_limit, ensuring expansion is an intentional pedagogical choice rather than a lazy reflex.
• Working Memory Integration: Outputs feed directly into the working lane rather than bloating the global context.

3. Working-Memory Cache & Utility Eviction (context_cache.go)

• Semantic Caching: Expanded symbols are cached per task using symbol and file-hash semantics to prevent redundant token spend.
• Intelligent Eviction: Under budget pressure, the system uses utility-based eviction to drop low-value expansions while anchoring pinned essentials.
• Logging: Re-skeletonization events are explicitly logged for auditability.

4. Safe Mode Guardrails (executor.go, agent.go)

• Threshold Policies: High-context mode (90%+) automatically restricts risky payload patterns and multi-file operations.
• Behavioral Shift: Steers the agent toward memory retrieval and scope narrowing rather than further expansion when near the ceiling.

5. Fact Ledger & Semantic Memory (fact_ledger.go, store.go, project_memory.go)

• Durable Continuity: Every code-changing turn records changes, rationales, and invariants.
• Vector-Backed Retrieval: Integration with chromem-go allows for embedded, persistent project memory.
• Semantic Search: Introduced search_project_memory to allow retrieval of old ledger entries and task summaries.

6. Telemetry & Metrics (metrics_collector.go, session_metrics.go)

• Performance Tracking: Wired instrumentation for tokens vs. edits, expansion efficiency, budget breach behavior, and cache hit rates.
• Pulse Logs: Real-time logging of "Efficiency" (tokens/LoC) and re-skeletonization triggers.

7. Dynamic Master Directives (instruction-coding.md)

• SSoT: System instructions are now loaded dynamically from instruction-coding.md.

[giveen]

Eh idea may be a bust. Trying to tell it how to trim and reload.

Maybe im just over stepping my knowledge.

Im going to reset my local branch and try some other ideas or work on whitelist stuff