Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 16 additions & 12 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
# Contributing
# Contributing to LexiconForge

Thanks for improving LexiconForge! This guide keeps changes safe and easy to review.
👋 **Welcome!** We're thrilled you want to help improve LexiconForge.

**🚀 New to the project?**
Check out our [**Newcomer Onboarding Guide**](docs/ONBOARDING.md) for a step-by-step walkthrough of the codebase and your first contribution.

---

## Setup

Expand All @@ -21,14 +26,14 @@ LexiconForge/
│ └── prompts.json # AI system prompts and translation instructions
├── components/ # 🎨 React UI components
│ ├── icons/ # SVG icon components (add custom toolbar emojis here!)
│ ├── icons/ # SVG icon components (used in selection/feedback controls)
│ │ ├── SettingsIcon.tsx
│ │ ├── TrashIcon.tsx
│ │ └── ... # Add your custom emoji icons here
│ ├── ChapterDisplay.tsx # Main translation display
│ ├── ChapterView.tsx # Main reader/translation view
│ ├── InputBar.tsx # URL input with website suggestions
│ ├── SettingsModal.tsx # Settings UI (model selection, API keys)
│ ├── Toolbar.tsx # Feedback toolbar with emoji buttons
│ ├── FeedbackPopover.tsx # Selection feedback controls (👍 👎 ? 🎨)
│ └── ...
├── services/ # 🔧 Business logic and external integrations
Expand All @@ -48,9 +53,8 @@ LexiconForge/
│ ├── chaptersSlice.ts # Chapter data & navigation
│ └── ...
├── adapters/ # 🔌 Data layer adapters
│ ├── providers/ # Translation provider adapters
│ └── repo/ # Repository pattern for database access
├── adapters/ # 🔌 Translation provider adapters
│ └── providers/ # Provider adapters + registration
├── types.ts # 📝 TypeScript type definitions
├── utils/ # 🛠️ Helper functions
Expand All @@ -67,15 +71,15 @@ LexiconForge/

**Want to add custom emojis to the toolbar?**
1. Add your SVG icon component to `components/icons/`
2. Import and use it in `components/Toolbar.tsx`
2. Import and use it in `components/FeedbackPopover.tsx`

**Want to change default models or AI parameters?**
- Edit `config/app.json` → `defaultModels` section

**Want to add a new translation provider?**
1. Create adapter in `adapters/providers/`
2. Follow the `TranslationProvider` interface
3. Register in the `Translator` (see `docs/META_ADAPTER.md`)
3. Register it in `adapters/providers/index.ts` (see `docs/META_ADAPTER.md`)

**Want to add support for a new website?**
1. Create adapter class in `services/adapters.ts`
Expand Down Expand Up @@ -106,13 +110,13 @@ LexiconForge/

## File Size Limits (Agent‑First)

- Services ≤ 200 LOC; Components ≤ 250 LOC (see ADR‑005)
- Services ≤ 200 LOC; Components ≤ 250 LOC (see [ADR‑005](docs/ADR-005-Agent-First-Code-Organization.md))
- Prefer extracting helpers and modules instead of growing files

## Adding Site Adapters / Providers

- Website adapters: follow `docs/META_ADAPTER.md`
- Translation providers: implement `TranslationProvider` and register with the `Translator`
- Translation providers: implement `TranslationProvider` and register in `adapters/providers/index.ts`

## Debugging

Expand Down
74 changes: 74 additions & 0 deletions docs/ONBOARDING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# 🚀 Newcomer Onboarding Guide

Welcome to LexiconForge! This guide is designed to get you from "zero" to "ready to contribute" as quickly as possible.

## 1. The Big Picture

LexiconForge is a React-based web application that helps users read and translate web novels. It's built with:
- **React 19** for the UI.
- **Zustand** for state management (think Redux, but simpler).
- **IndexedDB** for storing large amounts of novel data offline.
- **Vite** for fast development.

We have a strict "Agent-First" philosophy: keep files small and single-purpose so both humans and AI agents can understand them easily.

## 2. Setting Up Your Environment

1. **Node.js**: Ensure you have Node 18+ installed.
2. **Clone the Repo**:
```bash
git clone https://github.com/anantham/LexiconForge.git
cd LexiconForge
```
3. **Install Dependencies**:
```bash
npm install
```
4. **Environment Variables**:
Copy `.env.example` to `.env.local` and add at least one API key (e.g., Google Gemini is free/cheap to start).
```bash
cp .env.example .env.local
```

## 3. Your First Contribution: "The Walkthrough"

Let's walk through the codebase by following two user actions: **Load a chapter** and then **Translate it**.

### A) Load a chapter

1. **UI Trigger**: The URL/session loader lives in `components/InputBar.tsx`. On submit, it calls the store’s `handleFetch(url)`.
2. **State Logic**: `store/slices/chaptersSlice.ts` defines `handleFetch`, which delegates to `NavigationService.handleFetch`.
3. **Core Service**: `services/navigationService.ts` handles cache/hydration from IndexedDB and (if needed) fetching/parsing via `services/adapters.ts`.
4. **Persistence**: Reads/writes go through `services/db/operations/*` (e.g. `services/db/operations/translations.ts`).

### B) Translate a chapter

1. **UI Trigger**: In the reader, the retranslate button is rendered by `components/chapter/ChapterHeader.tsx` (wired up in `components/ChapterView.tsx`).
2. **State Logic**: `store/slices/translationsSlice.ts` orchestrates the translation workflow and calls `TranslationService.translateChapterSequential(...)`.
3. **Core Service**: `services/translationService.ts` builds history/context and calls `services/aiService.ts` (`translateChapter`).
4. **Routing + Adapters**: `services/ai/translatorRouter.ts` routes requests into `services/translate/Translator.ts`, using registered adapters from `adapters/providers/*`.

## 4. Key "Do's and Don'ts"

### ✅ DO
- **Check File Sizes**: Keep services under 200 lines and components under 250 lines.
- **Use "Ops" for DB**: Always use `services/db/operations/` to talk to the database. Never import `indexedDB` directly in components.
- **Run Tests**: `npm test` runs the unit tests. We value tests highly!

### ❌ DON'T
- **Create Monoliths**: If a file gets too big, split it.
- **Bypass the Store**: Components should read from Zustand stores, not fetch data directly from services (mostly).

## 5. Where to Start?

Check out `ISSUES.md` or look for "Good First Issue" tags on GitHub. Here are some safe areas to explore:

- **Icons**: Add a new icon to `components/icons/` and use it.
- **Prompts**: Tweak translation instructions in `config/prompts.json`.
- **CSS**: We use Tailwind. Adjust styling in `components/` to improve the look.

## 6. Need Help?

- Check `docs/PROJECT_STRUCTURE.md` for a map of the folders.
- Read `docs/ARCHITECTURE.md` for deep dives.
- Ask in our Telegram group or open a GitHub Discussion!
6 changes: 6 additions & 0 deletions docs/WORKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -1149,3 +1149,9 @@ Next: After running with reduced logs, gather traces for 'Chapter not found' and
- Why: Address dependency vulnerability chain (glob/path deps) and align tooling on supported versions.
- Details: Bumped `vitest`/`@vitest/*`, `vite`, `happy-dom`, and `@google/genai` (plus updated the browser importmap); adjusted OpenAI mocks in tests to use a constructible class under Vitest 4.
- Tests: `npm audit`; `npx tsc --noEmit`; `npm test -- --run`

2025-12-22 03:27 UTC - Contributor docs: onboarding + decomposition plans
- Files: CONTRIBUTING.md; docs/ONBOARDING.md; docs/plans/{EPUB-SERVICE-DECOMPOSITION.md,IMAGE-SLICE-DECOMPOSITION.md,NAVIGATION-SERVICE-DECOMPOSITION.md}; docs/WORKLOG.md
- Why: Make first-time contributors productive quickly and capture actionable decomposition plans for known monoliths.
- Details: Add onboarding walkthrough (load → translate flow), refresh CONTRIBUTING references to match current code locations, and add draft decomposition plans for `services/epubService.ts`, `store/slices/imageSlice.ts`, and `services/navigationService.ts`.
- Tests: N/A (docs only)
101 changes: 101 additions & 0 deletions docs/plans/EPUB-SERVICE-DECOMPOSITION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Decomposition Plan: `services/epubService.ts`

**Status:** Draft
**Target:** Decompose the 1,778-line `epubService.ts` monolith into focused, testable modules.

## 1. Problem Analysis
The current `services/epubService.ts` violates the Single Responsibility Principle and Agent-First file size limits (200 LOC). It mixes:
- Low-level XML/XHTML sanitization and DOM manipulation.
- Business logic for statistics calculation and cost tracking.
- Template string generation for EPUB pages.
- ZIP file binary packaging.
- Data collection from the Redux/Zustand store format.

## 2. Target Architecture

We will create a `services/epub/` directory with the following structure:

```
services/epub/
├── index.ts # Main entry point (orchestrator)
├── types.ts # Shared interfaces (ChapterForEpub, EpubExportOptions)
├── sanitizers/
│ └── xhtmlSanitizer.ts # XML namespaces, sanitization, strict XHTML conversion
├── data/
│ ├── collector.ts # collectActiveVersions (Data gathering)
│ └── stats.ts # calculateTranslationStats (Business logic)
├── templates/
│ ├── defaults.ts # Default templates and text
│ └── novelConfig.ts # Novel metadata inference logic
├── generators/
│ ├── titlePage.ts # Title page HTML generation
│ ├── toc.ts # Table of Contents HTML generation
│ ├── statsPage.ts # Statistics & Acknowledgments HTML generation
│ └── chapter.ts # Chapter content XHTML generation
└── packagers/
└── epubPackager.ts # JSZip logic, OEBPS structure, binary output
```

## 3. Interfaces & Boundaries

### `types.ts`
Will contain all exported interfaces currently in `epubService.ts`:
- `ChapterForEpub`
- `TranslationStats`
- `EpubExportOptions`
- `EpubTemplate`
- `NovelConfig`

### `sanitizers/xhtmlSanitizer.ts`
**Exports:**
- `sanitizeHtmlAllowlist(html: string): string`
- `htmlFragmentToXhtml(fragment: string): string`
- `escapeXml(text: string): string`

### `generators/chapter.ts`
**Exports:**
- `buildChapterXhtml(chapter: ChapterForEpub): string`

### `packagers/epubPackager.ts`
**Exports:**
- `generateEpub3WithJSZip(meta: EpubMeta, chapters: EpubChapter[]): Promise<ArrayBuffer>`

## 4. Execution Plan

### Step 1: Scaffold & Types (Safe)
1. Create `services/epub/types.ts` and move all interfaces there.
2. Update `epubService.ts` to import these types locally to ensure no breakage.

### Step 2: Extract Utilities (Low Risk)
1. Extract `sanitizers/xhtmlSanitizer.ts`. Move `sanitizeHtmlAllowlist`, `htmlFragmentToXhtml`, `cloneIntoXhtml`, and XML constants.
2. Extract `templates/defaults.ts` and `templates/novelConfig.ts`.
3. Update `epubService.ts` to use these new modules.

### Step 3: Extract Business Logic (Medium Risk)
1. Extract `data/stats.ts` (`calculateTranslationStats`).
2. Extract `data/collector.ts` (`collectActiveVersions`, `createChapterForEpub`).
3. Update `epubService.ts`.

### Step 4: Extract Generators (Medium Risk)
1. Extract `generators/titlePage.ts`, `generators/toc.ts`, `generators/statsPage.ts`.
2. Extract `generators/chapter.ts` (`buildChapterXhtml`). This is complex due to the dependency on `sanitizeHtmlAllowlist`. Ensure imports work.

### Step 5: Extract Packager (High Risk)
1. Extract `packagers/epubPackager.ts`. This contains the heavy `JSZip` logic and the `generateEpub3WithJSZip` function.
2. This module will need `escapeXml` from the sanitizer module.

### Step 6: Orchestrator & Cleanup
1. Rewrite `services/epub/index.ts` (formerly `epubService.ts`) to be a pure orchestrator that calls the above modules.
2. It should be < 200 lines, handling only the flow: Data -> Stats -> Generators -> Packager.

## 5. Verification Strategy

1. **Manual Export Test**:
- Before starting, generate an EPUB from the app and save it.
- After each step, generate a new EPUB and bitwise compare (or visually compare if timestamps differ) to ensure content is identical.
2. **Unit Tests**:
- Run `npm test services/epubService` if exists.
- If not, create a smoke test `tests/services/epub/smoke.test.ts` that imports the service and runs a mock export.

## 6. Rollback Plan
Since we are creating new files and importing them into the old one, we can revert by simply checking out the original `services/epubService.ts` and deleting the `services/epub/` directory.
70 changes: 70 additions & 0 deletions docs/plans/IMAGE-SLICE-DECOMPOSITION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Decomposition Plan: `store/slices/imageSlice.ts`

**Status:** Draft
**Target:** Decompose the 1,074-line `imageSlice.ts` into a modular feature slice.

## 1. Problem Analysis
The `imageSlice` has grown into a "God Slice" managing:
- **State Definition**: A large, complex state object.
- **Async Workflow**: Long-running generation/retry logic.
- **Persistence**: Complex snapshotting and IndexedDB writes.
- **Versioning**: Logic for managing multiple image versions.
- **UI Logic**: Getters for advanced controls.

## 2. Target Architecture

We will create a `store/slices/images/` directory:

```
store/slices/images/
├── index.ts # Main slice creator (composes parts)
├── types.ts # State interfaces
├── imageState.ts # Initial state & simple setters
├── imageActions.ts # Advanced controls setters
├── imageGeneration.ts # Async generation & retry logic
├── imageVersioning.ts # Version navigation & deletion
├── imagePersistence.ts # Persistence helpers & snapshotting
└── imageLegacy.ts # Migration logic for base64 images
```

## 3. Module Responsibilities

### `types.ts`
- Exports `ImageSliceState`, `ImageSliceActions`, `ImageSlice`.

### `imageState.ts`
- Exports `initialImageState`.
- Exports `createImageStateSlice`: Handles `setImageState`, `clearImageState`, `clearAllImages`.

### `imageActions.ts`
- Exports `createImageActionsSlice`: Handles setters for `steeringImages`, `negativePrompts`, `guidanceScales`, `loraModels`, etc.
- Also `resetAdvancedControls`.

### `imageGeneration.ts`
- Exports `createImageGenerationSlice`:
- `handleGenerateImages`
- `handleRetryImage`
- `loadExistingImages` (uses `imageLegacy` for migrations)

### `imageVersioning.ts`
- Exports `createImageVersioningSlice`:
- `navigateToNextVersion`
- `navigateToPreviousVersion`
- `deleteVersion`

### `imagePersistence.ts`
- Helper functions:
- `buildPersistenceSnapshot`
- `persistImageVersionState`
- Not a slice itself, but utilities imported by `generation` and `versioning`.

## 4. Execution Plan

1. **Scaffold**: Create folder and `types.ts`.
2. **Extract Helpers**: Move persistence and legacy migration logic to helper files.
3. **Split Slices**: Create the sub-slices (`State`, `Actions`, `Generation`, `Versioning`).
4. **Compose**: Re-assemble in `store/slices/imageSlice.ts` (or `store/slices/images/index.ts`) using Zustand's pattern.

## 5. Verification
- **Unit Tests**: Existing tests for `imageSlice` should pass without modification (if we keep the public API identical).
- **Manual Test**: Generate an image, switch versions, reload page (persistence check).
53 changes: 53 additions & 0 deletions docs/plans/NAVIGATION-SERVICE-DECOMPOSITION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Decomposition Plan: `services/navigationService.ts`

**Status:** Draft
**Target:** Decompose the 994-line `navigationService.ts` into a cohesive navigation module.

## 1. Problem Analysis
`NavigationService` combines:
- **Routing**: URL resolution and history management.
- **Data Access**: Complex hydration from IndexedDB (Chapter + Translation + Diffs).
- **Fetching**: Network fetching via adapters and Import/Export normalization.
- **Validation**: URL checking and error messaging.

## 2. Target Architecture

We will create a `services/navigation/` directory:

```
services/navigation/
├── index.ts # Main NavigationService class (Facade)
├── types.ts # Interfaces (NavigationContext, NavigationResult)
├── validation.ts # URL validation & supported site checks
├── converters.ts # DTO mapping (adaptTranslationRecordToResult)
├── hydration.ts # IDB loading logic (loadChapterFromIDB)
├── fetcher.ts # Network fetching (handleFetch)
└── history.ts # Browser history management
```

## 3. Module Responsibilities

### `converters.ts`
- `adaptTranslationRecordToResult`: Maps DB records to runtime objects.

### `hydration.ts`
- `loadChapterFromIDB`: The heavy lifter. Needs to import `ChapterOps`, `TranslationOps`, `DiffOps`.
- `tryServeChapterFromCache`: Short-circuit logic.

### `fetcher.ts`
- `handleFetch`: Orchestrates `fetchAndParseUrl`, `transformImportedChapters`, and `ImportOps`.

### `index.ts` (NavigationService)
- `handleNavigate`: The main entry point. Orchestrates the flow: Validation -> Cache Check -> Hydration -> (Fallback) -> Fetcher.

## 4. Execution Plan

1. **Scaffold**: Create directory and `types.ts`.
2. **Extract Utilities**: Move `converters.ts` and `validation.ts`.
3. **Extract Core Logic**: Move `hydration.ts` (biggest chunk) and `fetcher.ts`.
4. **Refactor Main Service**: `NavigationService` becomes a cleaner orchestrator importing these functions.

## 5. Verification
- **Critical Path**: Navigation is the core user loop.
- **Tests**: `tests/current-system/navigation.test.ts` must pass.
- **Manual**: Navigate to a new chapter, reload (hydration), navigate to a known chapter (cache).
Loading