From 43f25152bf7fa53edfe91cf194e5423b7cd25dc9 Mon Sep 17 00:00:00 2001 From: Aditya A P Date: Mon, 22 Dec 2025 08:58:53 +0530 Subject: [PATCH] docs: add onboarding and decomposition plans Context: salvage stash@{2} into a clean docs branch. Changes: update CONTRIBUTING intro + fix file references; add newcomer onboarding guide; add draft decomposition plans for epubService/imageSlice/navigationService; log in docs/WORKLOG.md. Tests: N/A (docs only) --- CONTRIBUTING.md | 28 ++--- docs/ONBOARDING.md | 74 +++++++++++++ docs/WORKLOG.md | 6 ++ docs/plans/EPUB-SERVICE-DECOMPOSITION.md | 101 ++++++++++++++++++ docs/plans/IMAGE-SLICE-DECOMPOSITION.md | 70 ++++++++++++ .../plans/NAVIGATION-SERVICE-DECOMPOSITION.md | 53 +++++++++ 6 files changed, 320 insertions(+), 12 deletions(-) create mode 100644 docs/ONBOARDING.md create mode 100644 docs/plans/EPUB-SERVICE-DECOMPOSITION.md create mode 100644 docs/plans/IMAGE-SLICE-DECOMPOSITION.md create mode 100644 docs/plans/NAVIGATION-SERVICE-DECOMPOSITION.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b11c304..5f8085c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,6 +1,11 @@ -# Contributing +# Contributing to LexiconForge -Thanks for improving LexiconForge! This guide keeps changes safe and easy to review. +πŸ‘‹ **Welcome!** We're thrilled you want to help improve LexiconForge. + +**πŸš€ New to the project?** +Check out our [**Newcomer Onboarding Guide**](docs/ONBOARDING.md) for a step-by-step walkthrough of the codebase and your first contribution. + +--- ## Setup @@ -21,14 +26,14 @@ LexiconForge/ β”‚ └── prompts.json # AI system prompts and translation instructions β”‚ β”œβ”€β”€ components/ # 🎨 React UI components -β”‚ β”œβ”€β”€ icons/ # SVG icon components (add custom toolbar emojis here!) +β”‚ β”œβ”€β”€ icons/ # SVG icon components (used in selection/feedback controls) β”‚ β”‚ β”œβ”€β”€ SettingsIcon.tsx β”‚ β”‚ β”œβ”€β”€ TrashIcon.tsx β”‚ β”‚ └── ... # Add your custom emoji icons here -β”‚ β”œβ”€β”€ ChapterDisplay.tsx # Main translation display +β”‚ β”œβ”€β”€ ChapterView.tsx # Main reader/translation view β”‚ β”œβ”€β”€ InputBar.tsx # URL input with website suggestions β”‚ β”œβ”€β”€ SettingsModal.tsx # Settings UI (model selection, API keys) -β”‚ β”œβ”€β”€ Toolbar.tsx # Feedback toolbar with emoji buttons +β”‚ β”œβ”€β”€ FeedbackPopover.tsx # Selection feedback controls (πŸ‘ πŸ‘Ž ? 🎨) β”‚ └── ... β”‚ β”œβ”€β”€ services/ # πŸ”§ Business logic and external integrations @@ -48,9 +53,8 @@ LexiconForge/ β”‚ β”œβ”€β”€ chaptersSlice.ts # Chapter data & navigation β”‚ └── ... β”‚ -β”œβ”€β”€ adapters/ # πŸ”Œ Data layer adapters -β”‚ β”œβ”€β”€ providers/ # Translation provider adapters -β”‚ └── repo/ # Repository pattern for database access +β”œβ”€β”€ adapters/ # πŸ”Œ Translation provider adapters +β”‚ └── providers/ # Provider adapters + registration β”‚ β”œβ”€β”€ types.ts # πŸ“ TypeScript type definitions β”œβ”€β”€ utils/ # πŸ› οΈ Helper functions @@ -67,7 +71,7 @@ LexiconForge/ **Want to add custom emojis to the toolbar?** 1. Add your SVG icon component to `components/icons/` -2. Import and use it in `components/Toolbar.tsx` +2. Import and use it in `components/FeedbackPopover.tsx` **Want to change default models or AI parameters?** - Edit `config/app.json` β†’ `defaultModels` section @@ -75,7 +79,7 @@ LexiconForge/ **Want to add a new translation provider?** 1. Create adapter in `adapters/providers/` 2. Follow the `TranslationProvider` interface -3. Register in the `Translator` (see `docs/META_ADAPTER.md`) +3. Register it in `adapters/providers/index.ts` (see `docs/META_ADAPTER.md`) **Want to add support for a new website?** 1. Create adapter class in `services/adapters.ts` @@ -106,13 +110,13 @@ LexiconForge/ ## File Size Limits (Agent‑First) -- Services ≀ 200 LOC; Components ≀ 250 LOC (see ADR‑005) +- Services ≀ 200 LOC; Components ≀ 250 LOC (see [ADR‑005](docs/ADR-005-Agent-First-Code-Organization.md)) - Prefer extracting helpers and modules instead of growing files ## Adding Site Adapters / Providers - Website adapters: follow `docs/META_ADAPTER.md` -- Translation providers: implement `TranslationProvider` and register with the `Translator` +- Translation providers: implement `TranslationProvider` and register in `adapters/providers/index.ts` ## Debugging diff --git a/docs/ONBOARDING.md b/docs/ONBOARDING.md new file mode 100644 index 0000000..35ab258 --- /dev/null +++ b/docs/ONBOARDING.md @@ -0,0 +1,74 @@ +# πŸš€ Newcomer Onboarding Guide + +Welcome to LexiconForge! This guide is designed to get you from "zero" to "ready to contribute" as quickly as possible. + +## 1. The Big Picture + +LexiconForge is a React-based web application that helps users read and translate web novels. It's built with: +- **React 19** for the UI. +- **Zustand** for state management (think Redux, but simpler). +- **IndexedDB** for storing large amounts of novel data offline. +- **Vite** for fast development. + +We have a strict "Agent-First" philosophy: keep files small and single-purpose so both humans and AI agents can understand them easily. + +## 2. Setting Up Your Environment + +1. **Node.js**: Ensure you have Node 18+ installed. +2. **Clone the Repo**: + ```bash + git clone https://github.com/anantham/LexiconForge.git + cd LexiconForge + ``` +3. **Install Dependencies**: + ```bash + npm install + ``` +4. **Environment Variables**: + Copy `.env.example` to `.env.local` and add at least one API key (e.g., Google Gemini is free/cheap to start). + ```bash + cp .env.example .env.local + ``` + +## 3. Your First Contribution: "The Walkthrough" + +Let's walk through the codebase by following two user actions: **Load a chapter** and then **Translate it**. + +### A) Load a chapter + +1. **UI Trigger**: The URL/session loader lives in `components/InputBar.tsx`. On submit, it calls the store’s `handleFetch(url)`. +2. **State Logic**: `store/slices/chaptersSlice.ts` defines `handleFetch`, which delegates to `NavigationService.handleFetch`. +3. **Core Service**: `services/navigationService.ts` handles cache/hydration from IndexedDB and (if needed) fetching/parsing via `services/adapters.ts`. +4. **Persistence**: Reads/writes go through `services/db/operations/*` (e.g. `services/db/operations/translations.ts`). + +### B) Translate a chapter + +1. **UI Trigger**: In the reader, the retranslate button is rendered by `components/chapter/ChapterHeader.tsx` (wired up in `components/ChapterView.tsx`). +2. **State Logic**: `store/slices/translationsSlice.ts` orchestrates the translation workflow and calls `TranslationService.translateChapterSequential(...)`. +3. **Core Service**: `services/translationService.ts` builds history/context and calls `services/aiService.ts` (`translateChapter`). +4. **Routing + Adapters**: `services/ai/translatorRouter.ts` routes requests into `services/translate/Translator.ts`, using registered adapters from `adapters/providers/*`. + +## 4. Key "Do's and Don'ts" + +### βœ… DO +- **Check File Sizes**: Keep services under 200 lines and components under 250 lines. +- **Use "Ops" for DB**: Always use `services/db/operations/` to talk to the database. Never import `indexedDB` directly in components. +- **Run Tests**: `npm test` runs the unit tests. We value tests highly! + +### ❌ DON'T +- **Create Monoliths**: If a file gets too big, split it. +- **Bypass the Store**: Components should read from Zustand stores, not fetch data directly from services (mostly). + +## 5. Where to Start? + +Check out `ISSUES.md` or look for "Good First Issue" tags on GitHub. Here are some safe areas to explore: + +- **Icons**: Add a new icon to `components/icons/` and use it. +- **Prompts**: Tweak translation instructions in `config/prompts.json`. +- **CSS**: We use Tailwind. Adjust styling in `components/` to improve the look. + +## 6. Need Help? + +- Check `docs/PROJECT_STRUCTURE.md` for a map of the folders. +- Read `docs/ARCHITECTURE.md` for deep dives. +- Ask in our Telegram group or open a GitHub Discussion! diff --git a/docs/WORKLOG.md b/docs/WORKLOG.md index 243ec74..813121c 100644 --- a/docs/WORKLOG.md +++ b/docs/WORKLOG.md @@ -1149,3 +1149,9 @@ Next: After running with reduced logs, gather traces for 'Chapter not found' and - Why: Address dependency vulnerability chain (glob/path deps) and align tooling on supported versions. - Details: Bumped `vitest`/`@vitest/*`, `vite`, `happy-dom`, and `@google/genai` (plus updated the browser importmap); adjusted OpenAI mocks in tests to use a constructible class under Vitest 4. - Tests: `npm audit`; `npx tsc --noEmit`; `npm test -- --run` + +2025-12-22 03:27 UTC - Contributor docs: onboarding + decomposition plans +- Files: CONTRIBUTING.md; docs/ONBOARDING.md; docs/plans/{EPUB-SERVICE-DECOMPOSITION.md,IMAGE-SLICE-DECOMPOSITION.md,NAVIGATION-SERVICE-DECOMPOSITION.md}; docs/WORKLOG.md +- Why: Make first-time contributors productive quickly and capture actionable decomposition plans for known monoliths. +- Details: Add onboarding walkthrough (load β†’ translate flow), refresh CONTRIBUTING references to match current code locations, and add draft decomposition plans for `services/epubService.ts`, `store/slices/imageSlice.ts`, and `services/navigationService.ts`. +- Tests: N/A (docs only) diff --git a/docs/plans/EPUB-SERVICE-DECOMPOSITION.md b/docs/plans/EPUB-SERVICE-DECOMPOSITION.md new file mode 100644 index 0000000..dcce446 --- /dev/null +++ b/docs/plans/EPUB-SERVICE-DECOMPOSITION.md @@ -0,0 +1,101 @@ +# Decomposition Plan: `services/epubService.ts` + +**Status:** Draft +**Target:** Decompose the 1,778-line `epubService.ts` monolith into focused, testable modules. + +## 1. Problem Analysis +The current `services/epubService.ts` violates the Single Responsibility Principle and Agent-First file size limits (200 LOC). It mixes: +- Low-level XML/XHTML sanitization and DOM manipulation. +- Business logic for statistics calculation and cost tracking. +- Template string generation for EPUB pages. +- ZIP file binary packaging. +- Data collection from the Redux/Zustand store format. + +## 2. Target Architecture + +We will create a `services/epub/` directory with the following structure: + +``` +services/epub/ +β”œβ”€β”€ index.ts # Main entry point (orchestrator) +β”œβ”€β”€ types.ts # Shared interfaces (ChapterForEpub, EpubExportOptions) +β”œβ”€β”€ sanitizers/ +β”‚ └── xhtmlSanitizer.ts # XML namespaces, sanitization, strict XHTML conversion +β”œβ”€β”€ data/ +β”‚ β”œβ”€β”€ collector.ts # collectActiveVersions (Data gathering) +β”‚ └── stats.ts # calculateTranslationStats (Business logic) +β”œβ”€β”€ templates/ +β”‚ β”œβ”€β”€ defaults.ts # Default templates and text +β”‚ └── novelConfig.ts # Novel metadata inference logic +β”œβ”€β”€ generators/ +β”‚ β”œβ”€β”€ titlePage.ts # Title page HTML generation +β”‚ β”œβ”€β”€ toc.ts # Table of Contents HTML generation +β”‚ β”œβ”€β”€ statsPage.ts # Statistics & Acknowledgments HTML generation +β”‚ └── chapter.ts # Chapter content XHTML generation +└── packagers/ + └── epubPackager.ts # JSZip logic, OEBPS structure, binary output +``` + +## 3. Interfaces & Boundaries + +### `types.ts` +Will contain all exported interfaces currently in `epubService.ts`: +- `ChapterForEpub` +- `TranslationStats` +- `EpubExportOptions` +- `EpubTemplate` +- `NovelConfig` + +### `sanitizers/xhtmlSanitizer.ts` +**Exports:** +- `sanitizeHtmlAllowlist(html: string): string` +- `htmlFragmentToXhtml(fragment: string): string` +- `escapeXml(text: string): string` + +### `generators/chapter.ts` +**Exports:** +- `buildChapterXhtml(chapter: ChapterForEpub): string` + +### `packagers/epubPackager.ts` +**Exports:** +- `generateEpub3WithJSZip(meta: EpubMeta, chapters: EpubChapter[]): Promise` + +## 4. Execution Plan + +### Step 1: Scaffold & Types (Safe) +1. Create `services/epub/types.ts` and move all interfaces there. +2. Update `epubService.ts` to import these types locally to ensure no breakage. + +### Step 2: Extract Utilities (Low Risk) +1. Extract `sanitizers/xhtmlSanitizer.ts`. Move `sanitizeHtmlAllowlist`, `htmlFragmentToXhtml`, `cloneIntoXhtml`, and XML constants. +2. Extract `templates/defaults.ts` and `templates/novelConfig.ts`. +3. Update `epubService.ts` to use these new modules. + +### Step 3: Extract Business Logic (Medium Risk) +1. Extract `data/stats.ts` (`calculateTranslationStats`). +2. Extract `data/collector.ts` (`collectActiveVersions`, `createChapterForEpub`). +3. Update `epubService.ts`. + +### Step 4: Extract Generators (Medium Risk) +1. Extract `generators/titlePage.ts`, `generators/toc.ts`, `generators/statsPage.ts`. +2. Extract `generators/chapter.ts` (`buildChapterXhtml`). This is complex due to the dependency on `sanitizeHtmlAllowlist`. Ensure imports work. + +### Step 5: Extract Packager (High Risk) +1. Extract `packagers/epubPackager.ts`. This contains the heavy `JSZip` logic and the `generateEpub3WithJSZip` function. +2. This module will need `escapeXml` from the sanitizer module. + +### Step 6: Orchestrator & Cleanup +1. Rewrite `services/epub/index.ts` (formerly `epubService.ts`) to be a pure orchestrator that calls the above modules. +2. It should be < 200 lines, handling only the flow: Data -> Stats -> Generators -> Packager. + +## 5. Verification Strategy + +1. **Manual Export Test**: + - Before starting, generate an EPUB from the app and save it. + - After each step, generate a new EPUB and bitwise compare (or visually compare if timestamps differ) to ensure content is identical. +2. **Unit Tests**: + - Run `npm test services/epubService` if exists. + - If not, create a smoke test `tests/services/epub/smoke.test.ts` that imports the service and runs a mock export. + +## 6. Rollback Plan +Since we are creating new files and importing them into the old one, we can revert by simply checking out the original `services/epubService.ts` and deleting the `services/epub/` directory. diff --git a/docs/plans/IMAGE-SLICE-DECOMPOSITION.md b/docs/plans/IMAGE-SLICE-DECOMPOSITION.md new file mode 100644 index 0000000..6f3aeb6 --- /dev/null +++ b/docs/plans/IMAGE-SLICE-DECOMPOSITION.md @@ -0,0 +1,70 @@ +# Decomposition Plan: `store/slices/imageSlice.ts` + +**Status:** Draft +**Target:** Decompose the 1,074-line `imageSlice.ts` into a modular feature slice. + +## 1. Problem Analysis +The `imageSlice` has grown into a "God Slice" managing: +- **State Definition**: A large, complex state object. +- **Async Workflow**: Long-running generation/retry logic. +- **Persistence**: Complex snapshotting and IndexedDB writes. +- **Versioning**: Logic for managing multiple image versions. +- **UI Logic**: Getters for advanced controls. + +## 2. Target Architecture + +We will create a `store/slices/images/` directory: + +``` +store/slices/images/ +β”œβ”€β”€ index.ts # Main slice creator (composes parts) +β”œβ”€β”€ types.ts # State interfaces +β”œβ”€β”€ imageState.ts # Initial state & simple setters +β”œβ”€β”€ imageActions.ts # Advanced controls setters +β”œβ”€β”€ imageGeneration.ts # Async generation & retry logic +β”œβ”€β”€ imageVersioning.ts # Version navigation & deletion +β”œβ”€β”€ imagePersistence.ts # Persistence helpers & snapshotting +└── imageLegacy.ts # Migration logic for base64 images +``` + +## 3. Module Responsibilities + +### `types.ts` +- Exports `ImageSliceState`, `ImageSliceActions`, `ImageSlice`. + +### `imageState.ts` +- Exports `initialImageState`. +- Exports `createImageStateSlice`: Handles `setImageState`, `clearImageState`, `clearAllImages`. + +### `imageActions.ts` +- Exports `createImageActionsSlice`: Handles setters for `steeringImages`, `negativePrompts`, `guidanceScales`, `loraModels`, etc. +- Also `resetAdvancedControls`. + +### `imageGeneration.ts` +- Exports `createImageGenerationSlice`: + - `handleGenerateImages` + - `handleRetryImage` + - `loadExistingImages` (uses `imageLegacy` for migrations) + +### `imageVersioning.ts` +- Exports `createImageVersioningSlice`: + - `navigateToNextVersion` + - `navigateToPreviousVersion` + - `deleteVersion` + +### `imagePersistence.ts` +- Helper functions: + - `buildPersistenceSnapshot` + - `persistImageVersionState` +- Not a slice itself, but utilities imported by `generation` and `versioning`. + +## 4. Execution Plan + +1. **Scaffold**: Create folder and `types.ts`. +2. **Extract Helpers**: Move persistence and legacy migration logic to helper files. +3. **Split Slices**: Create the sub-slices (`State`, `Actions`, `Generation`, `Versioning`). +4. **Compose**: Re-assemble in `store/slices/imageSlice.ts` (or `store/slices/images/index.ts`) using Zustand's pattern. + +## 5. Verification +- **Unit Tests**: Existing tests for `imageSlice` should pass without modification (if we keep the public API identical). +- **Manual Test**: Generate an image, switch versions, reload page (persistence check). diff --git a/docs/plans/NAVIGATION-SERVICE-DECOMPOSITION.md b/docs/plans/NAVIGATION-SERVICE-DECOMPOSITION.md new file mode 100644 index 0000000..a9c517f --- /dev/null +++ b/docs/plans/NAVIGATION-SERVICE-DECOMPOSITION.md @@ -0,0 +1,53 @@ +# Decomposition Plan: `services/navigationService.ts` + +**Status:** Draft +**Target:** Decompose the 994-line `navigationService.ts` into a cohesive navigation module. + +## 1. Problem Analysis +`NavigationService` combines: +- **Routing**: URL resolution and history management. +- **Data Access**: Complex hydration from IndexedDB (Chapter + Translation + Diffs). +- **Fetching**: Network fetching via adapters and Import/Export normalization. +- **Validation**: URL checking and error messaging. + +## 2. Target Architecture + +We will create a `services/navigation/` directory: + +``` +services/navigation/ +β”œβ”€β”€ index.ts # Main NavigationService class (Facade) +β”œβ”€β”€ types.ts # Interfaces (NavigationContext, NavigationResult) +β”œβ”€β”€ validation.ts # URL validation & supported site checks +β”œβ”€β”€ converters.ts # DTO mapping (adaptTranslationRecordToResult) +β”œβ”€β”€ hydration.ts # IDB loading logic (loadChapterFromIDB) +β”œβ”€β”€ fetcher.ts # Network fetching (handleFetch) +└── history.ts # Browser history management +``` + +## 3. Module Responsibilities + +### `converters.ts` +- `adaptTranslationRecordToResult`: Maps DB records to runtime objects. + +### `hydration.ts` +- `loadChapterFromIDB`: The heavy lifter. Needs to import `ChapterOps`, `TranslationOps`, `DiffOps`. +- `tryServeChapterFromCache`: Short-circuit logic. + +### `fetcher.ts` +- `handleFetch`: Orchestrates `fetchAndParseUrl`, `transformImportedChapters`, and `ImportOps`. + +### `index.ts` (NavigationService) +- `handleNavigate`: The main entry point. Orchestrates the flow: Validation -> Cache Check -> Hydration -> (Fallback) -> Fetcher. + +## 4. Execution Plan + +1. **Scaffold**: Create directory and `types.ts`. +2. **Extract Utilities**: Move `converters.ts` and `validation.ts`. +3. **Extract Core Logic**: Move `hydration.ts` (biggest chunk) and `fetcher.ts`. +4. **Refactor Main Service**: `NavigationService` becomes a cleaner orchestrator importing these functions. + +## 5. Verification +- **Critical Path**: Navigation is the core user loop. +- **Tests**: `tests/current-system/navigation.test.ts` must pass. +- **Manual**: Navigate to a new chapter, reload (hydration), navigate to a known chapter (cache).