Features and deep fixes

.gitignore

@@ -3,3 +3,5 @@
 
 ivaldi*
 CLAUDE.md
+.planning/*
+*/.planning/

.planning/codebase/ARCHITECTURE.md

@@ -0,0 +1,150 @@
+# Architecture
+
+**Analysis Date:** 2026-01-13
+
+## Pattern Overview
+
+**Overall:** Layered Monolithic CLI with Service-Oriented Design
+
+**Key Characteristics:**
+- Single executable with 20+ subcommands
+- Clean separation between CLI, services, and storage
+- Content-addressable storage with BLAKE3 hashing
+- Merkle Mountain Range for append-only history
+- HAMT directories for scalable tree structures
+
+## Layers
+
+**Command Layer (CLI):**
+- Purpose: Parse user input and route to handlers
+- Contains: Command definitions, argument parsing, user interaction
+- Location: `cli/*.go`
+- Depends on: Service layer packages in `internal/`
+- Used by: User via terminal
+
+**Service Layer (Core Logic):**
+- Purpose: Business logic for version control operations
+- Contains: Commit management, timeline handling, workspace operations
+- Location: `internal/commit/`, `internal/refs/`, `internal/workspace/`, `internal/butterfly/`
+- Depends on: Storage layer, utility packages
+- Used by: CLI commands
+
+**Storage Layer:**
+- Purpose: Persistent data management
+- Contains: CAS, BoltDB wrapper, workspace index
+- Location: `internal/cas/`, `internal/store/`, `internal/wsindex/`
+- Depends on: File system, BoltDB
+- Used by: Service layer
+
+**Data Structures Layer:**
+- Purpose: Core data types and algorithms
+- Contains: HAMT, MMR, file chunking, diff/merge
+- Location: `internal/hamtdir/`, `internal/history/`, `internal/filechunk/`, `internal/diffmerge/`
+- Depends on: Storage layer
+- Used by: Service layer
+
+## Data Flow
+
+**Commit Creation (ivaldi gather + seal):**
+
+1. User runs: `ivaldi gather` then `ivaldi seal`
+2. CLI parses command (`cli/management.go`)
+3. Workspace scanner detects file changes (`internal/workspace/`)
+4. Files chunked and hashed (`internal/filechunk/`)
+5. Content stored by BLAKE3 hash (`internal/cas/file_cas.go`)
+6. HAMT directory tree built (`internal/hamtdir/`)
+7. Commit object created with tree hash + parents (`internal/commit/`)
+8. Commit appended to MMR history (`internal/history/persistent_mmr.go`)
+9. Timeline reference updated (`internal/refs/`)
+
+**Timeline Switch (ivaldi timeline switch):**
+
+1. User runs: `ivaldi timeline switch feature`
+2. CLI routes to switch handler (`cli/timeline.go`)
+3. Current changes auto-shelved if dirty (`internal/shelf/`)
+4. Target commit loaded from refs (`internal/refs/`)
+5. Tree structure loaded from CAS (`internal/hamtdir/`)
+6. Workspace materialized from tree (`internal/workspace/`)
+7. Files reconstructed from CAS (`internal/cas/`)
+
+**State Management:**
+- File-based: All state in `.ivaldi/` directory
+- BoltDB for MMR and hash mappings
+- No in-memory persistent state between commands
+
+## Key Abstractions
+
+**CAS (Content-Addressable Storage):**
+- Purpose: Store and retrieve content by hash
+- Examples: `internal/cas/file_cas.go` (file-based), `internal/cas/cas.go` (in-memory)
+- Pattern: Interface with `Put()`, `Get()`, `Has()` methods
+
+**MMR (Merkle Mountain Range):**
+- Purpose: Append-only commit history with cryptographic proofs
+- Examples: `internal/history/mmr.go`, `internal/history/persistent_mmr.go`
+- Pattern: Persistent tree structure with efficient peak management
+
+**HAMT (Hash Array Mapped Trie):**
+- Purpose: Scalable directory representation
+- Examples: `internal/hamtdir/hamtdir.go`
+- Pattern: 32-way branching trie with canonical encoding
+
+**RefsManager:**
+- Purpose: Timeline/branch reference management
+- Examples: `internal/refs/refs.go`
+- Pattern: Singleton manager with file-based persistence
+
+**Butterfly:**
+- Purpose: Experimental sandbox branches with bidirectional sync
+- Examples: `internal/butterfly/manager.go`, `internal/butterfly/sync.go`
+- Pattern: State machine with metadata tracking
+
+## Entry Points
+
+**CLI Entry:**
+- Location: `main.go`
+- Triggers: User runs `ivaldi <command>`
+- Responsibilities: Initialize Cobra, call `cli.Execute()`
+
+**Command Dispatcher:**
+- Location: `cli/cli.go`
+- Triggers: Subcommand matched
+- Responsibilities: Register commands, parse flags, delegate to handlers
+
+## Error Handling
+
+**Strategy:** Return errors up the call stack, handle at CLI boundary
+
+**Patterns:**
+- Error wrapping: `fmt.Errorf("context: %w", err)`
+- Early return on error
+- Defer for cleanup (Close, RemoveAll)
+- User-facing errors printed to stderr
+
+## Cross-Cutting Concerns
+
+**Logging:**
+- Console output via fmt.Println/Printf
+- Colored output via `internal/colors/`
+- No structured logging framework
+
+**Validation:**
+- Input validation in CLI handlers
+- Path validation before file operations
+- Hash verification on content retrieval
+
+**Concurrency:**
+- Worker pools for parallel operations (8-16 workers)
+- sync.Mutex/RWMutex for shared state
+- WaitGroups + semaphores for coordination
+- Reference-counted database connections (`internal/store/manager.go`)
+
+**Authentication:**
+- OAuth device flow for GitHub/GitLab
+- Multiple fallback methods (env vars, git config, credential helpers)
+- Token storage with restricted permissions
+
+---
+
+*Architecture analysis: 2026-01-13*
+*Update when major patterns change*

.planning/codebase/CONCERNS.md

@@ -0,0 +1,163 @@
+# Codebase Concerns
+
+**Analysis Date:** 2026-01-13
+
+## Tech Debt
+
+**Incomplete TODO implementations:**
+- Issue: Multiple TODO comments indicate unfinished features
+- Files:
+  - `cli/diff.go:420` - "TODO: Convert tree to FileMetadata"
+  - `cli/diff.go:437` - "TODO: implement hash prefix resolution"
+  - `cli/fuse.go:189` - "TODO: Walk full parent chain"
+  - `cli/fuse.go:628` - "TODO: Implement interactive resolution using the ConflictResolver"
+  - `cli/management.go:789` - "TODO: Implement actual download/clone functionality for standard Ivaldi remotes"
+- Why: Features added incrementally during development
+- Impact: Some user workflows incomplete (hash prefix, interactive merge)
+- Fix approach: Prioritize and implement or remove if not needed
+
+**Large file complexity:**
+- Issue: Several files exceed 800 lines with complex logic
+- Files:
+  - `internal/github/sync.go` (1847 lines)
+  - `cli/management.go` (1459 lines)
+  - `internal/workspace/workspace.go` (827 lines)
+  - `cli/fuse.go` (758 lines)
+  - `internal/diffmerge/diffmerge.go` (757 lines)
+- Why: Features added incrementally without refactoring
+- Impact: Harder to maintain, test, and understand
+- Fix approach: Extract cohesive modules (e.g., split sync.go into tree fetching, object conversion, etc.)
+
+**Console output in internal packages:**
+- Issue: ~146 instances of fmt.Println/Printf in internal packages
+- Files: Throughout `internal/`
+- Why: Quick development without logging abstraction
+- Impact: Cannot suppress or redirect output, mixing concerns
+- Fix approach: Introduce proper logging package, migrate to structured logging
+
+## Known Bugs
+
+**No critical bugs identified through static analysis.**
+
+## Security Considerations
+
+**Hardcoded OAuth credentials:**
+- Risk: GitHub public app credentials embedded in code
+- Files: `internal/auth/oauth.go:30` - `GitHubClientID = "178c6fc778ccc68e1d6a"`
+- Current mitigation: Documented as intentional (public app), users can override via env vars
+- Recommendations: Consider documenting security implications in README
+
+**Missing input validation on downloaded paths:**
+- Risk: Path traversal attacks possible from malicious remote data
+- Files: `internal/github/sync.go`, `internal/gitlab/sync.go`
+- Current mitigation: None explicit
+- Recommendations: Validate paths from API responses before file operations
+
+**Token in memory:**
+- Risk: OAuth tokens held in memory during operations
+- Files: `internal/auth/oauth.go`
+- Current mitigation: Restricted file permissions (0600)
+- Recommendations: Consider secure memory handling for long-running sessions
+
+## Performance Bottlenecks
+
+**No measured performance issues identified.**
+
+The codebase uses efficient patterns:
+- Worker pools for concurrent operations (8-16 workers)
+- Content-addressable storage for deduplication
+- BoltDB for efficient key-value operations
+- HAMT for scalable directory structures
+
+## Fragile Areas
+
+**Error handling in cleanup paths:**
+- Why fragile: `os.Chdir(originalDir)` errors silently ignored
+- Files: `cli/management.go:148, 366, 493`
+- Common failures: Directory deleted during operation
+- Safe modification: Add error logging in cleanup
+- Test coverage: Not tested
+
+**Ignored error returns:**
+- Why fragile: Multiple places discard errors with `_`
+- Files:
+  - `cli/timeline.go:78, 82, 195` - Build() and NewManager() errors discarded
+  - `cli/fuse.go:259, 266, 407, 733` - Errors silently ignored
+  - `internal/keys/keys.go:24` - rand.Read() error ignored
+- Common failures: Silent failures lead to confusing behavior
+- Safe modification: Handle or log all errors
+- Test coverage: Not specifically tested
+
+**Panic in production code:**
+- Why fragile: Panic instead of error return
+- Files: `internal/fsmerkle/types.go:143` - Panic on content size mismatch
+- Common failures: Data corruption or API issues cause crash
+- Safe modification: Convert to error return
+- Test coverage: Has test for panic behavior
+
+## Scaling Limits
+
+**Not applicable for CLI tool.**
+
+The tool operates on local repositories with practical limits based on:
+- Filesystem capacity
+- Available memory for large operations
+- API rate limits (handled with backoff)
+
+## Dependencies at Risk
+
+**No critical dependency risks identified.**
+
+Dependencies are from reputable sources:
+- go.etcd.io/bbolt - CNCF project
+- github.com/spf13/cobra - Widely used CLI framework
+- github.com/go-git/go-git - Active Git implementation
+
+## Missing Critical Features
+
+**Interactive merge conflict resolution:**
+- Problem: CLI prompts for strategy but no interactive file-by-file resolution
+- Files: `cli/fuse.go:628` - TODO comment
+- Current workaround: Users must manually edit files, use strategy flags
+- Blocks: User-friendly merge workflow
+- Implementation complexity: Medium (UI + diffmerge integration)
+
+**Hash prefix resolution:**
+- Problem: Cannot use short commit hashes
+- Files: `cli/diff.go:437` - TODO comment
+- Current workaround: Use full hashes
+- Blocks: Convenient commit references
+- Implementation complexity: Low (prefix matching in refs)
+
+**GitLab OAuth registration:**
+- Problem: GitLab OAuth ClientID is empty
+- Files: `internal/auth/oauth.go:38` - Empty string
+- Current workaround: Use personal access tokens or git credentials
+- Blocks: Seamless GitLab authentication
+- Implementation complexity: Low (register app, add credentials)
+
+## Test Coverage Gaps
+
+**CLI commands:**
+- What's not tested: Most CLI command logic
+- Risk: Regressions in user-facing features
+- Priority: Medium
+- Difficulty to test: Requires integration testing setup
+
+**Sync operations:**
+- What's not tested: GitHub/GitLab sync logic (1800+ lines)
+- Files: `internal/github/sync.go`, `internal/gitlab/sync.go`
+- Risk: API changes could break sync silently
+- Priority: High
+- Difficulty to test: Requires API mocking
+
+**Error paths:**
+- What's not tested: Many error handling branches
+- Risk: Error recovery may not work as expected
+- Priority: Medium
+- Difficulty to test: Need to simulate failures
+
+---
+
+*Concerns audit: 2026-01-13*
+*Update as issues are fixed or new ones discovered*