From a8c9e18a21d7bfd30335b47bea2eb0e319d540e7 Mon Sep 17 00:00:00 2001 From: keypair34 <216233964+keypair34@users.noreply.github.com> Date: Sun, 12 Jul 2026 07:45:16 +0200 Subject: [PATCH 1/3] Add AGENTS.md with build, architecture, and convention guidance Co-Authored-By: siGit Code <297239231+sigitc@users.noreply.github.com> --- AGENTS.md | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..7e43f45 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,49 @@ +# AGENTS.md + +`onde-cli` is a Rust TUI (`onde` binary) for managing an Onde Inference account and running a local model pipeline: LoRA fine-tune → adapter merge → GGUF export → local chat test → Hugging Face upload → model assignment to an Onde app. npm, PyPI, NuGet, pub.dev, Homebrew, and crates.io are thin wrappers around this one binary; the Rust crate is the source of truth. + +## Build, test, lint + +```sh +cargo build # debug build +cargo run # launch the TUI +cargo test --locked # unit tests (heavy ones are #[ignore]) +cargo clippy --all-targets -- -D warnings +cargo fmt -- --check +``` + +The build **requires** these env vars (read from `.env` via `build.rs`): `ONDE_APP_ID`, `ONDE_APP_SECRET`, `GRESIQ_API_KEY`, `GRESIQ_API_SECRET`. `HF_TOKEN` is optional. They are baked in at compile time via `env!(...)` in `src/app.rs`. Changing `.env` triggers a full rebuild. + +On macOS, candle's `metal` + `accelerate` features are added automatically. **Fine-tuning always runs on CPU** — Metal's backward pass in candle 0.10.2 produces corrupt gradients. Use `ONDE_FINETUNE_METAL=1` only to test newer candle versions, and verify with `ONDE_FINETUNE_GRAD_DEBUG=1`. + +Linux CI requires `libclang-dev` (`sudo apt-get install -y libclang-dev`). + +## Architecture — `src/` + +| File(s) | Responsibility | +|---|---| +| `main.rs` | Entry point; redirects stdout/stderr to `~/.cache/onde/debug.log` before ratatui takes over | +| `app.rs` | `App` struct + `Screen` state machine + `AuthEvent` channel; `App::apply` folds background progress into state | +| `ui.rs` | Pure ratatui render of `App`; no SDK imports (re-exports types through `app.rs`) | +| `finetune.rs` | Hand-written LoRA trainer (candle, CPU). Writes `lora_adapter.safetensors` | +| `merge.rs` | Folds LoRA adapter into base weights; writes `model.safetensors` | +| `gguf.rs` | Hand-rolled GGUF writer (no llama.cpp). Tensor dims innermost-first; `head_dim` → `attention.key_length/value_length` | +| `chat.rs` | Local GGUF chat via `onde::mistralrs::GgufModelBuilder` | +| `hf*.rs` | Hugging Face Hub: upload, clone, search, cache resolution | +| `gresiq.rs` | `smbcloud-gresiq-sdk` wrapper; model assignment (`assign_model`) | +| `token.rs` | Auth token persistence | +| `project.rs` | Per-project fine-tune workspaces under `~/.onde` | + +**Background task rule:** network/IO → `tokio::spawn`; CPU-heavy tensor work (`finetune`, `merge`, `gguf`) → `std::thread::spawn` to avoid starving the async runtime. + +**Adding a feature pattern:** new `Screen` variant → key handler → `AuthEvent` variant → background task that streams progress events. + +## Conventions + +- **Git:** merge feature/release/hotfix branches with `--no-ff`. Tag the merge commit on `main`. Never tag a branch tip that hasn't been merged. See `.agents/skills/git/SKILL.md`. +- **Commits:** include `Co-Authored-By: siGit Code <297239231+sigitc@users.noreply.github.com>` trailer when committing with the agent. +- **`[patch.crates-io]` and `path =` deps:** commented out in `Cargo.toml` for local dev against `onde`/`smbcloud-*`/`candle`. Never commit with these uncommented. +- **Distribution:** before touching any wrapper package (npm, PyPI, NuGet, pub.dev, Homebrew), read `.agents/skills/distribution/SKILL.md`. Keep all channel versions aligned with the Rust crate version in `Cargo.toml`. +- **Wrapper packages** are in `npm/`, `pypi/`, `nuget/`, `pub/`. The npm base manifest is rewritten at release time by `npm/scripts/render-main-package.cjs`; do not hand-edit committed package.json versions as the source of truth for releases. +- **Clippy** runs with `-D warnings` in CI; all lints must pass. +- **Rust edition:** 2024; toolchain: stable. From 3218277ac5b82ae34e54c3cb1f54ec92ecd7a0b2 Mon Sep 17 00:00:00 2001 From: paydii <193906237+paydii@users.noreply.github.com> Date: Sun, 12 Jul 2026 08:00:08 +0200 Subject: [PATCH 2/3] Move AGENTS.md to .agents/AGENTS.md and symlink root Co-Authored-By: siGit Code <297239231+sigitc@users.noreply.github.com> --- .agents/AGENTS.md | 49 ++++++++++++++++++++++++++++++++++++++++++++++ AGENTS.md | 50 +---------------------------------------------- 2 files changed, 50 insertions(+), 49 deletions(-) create mode 100644 .agents/AGENTS.md mode change 100644 => 120000 AGENTS.md diff --git a/.agents/AGENTS.md b/.agents/AGENTS.md new file mode 100644 index 0000000..7e43f45 --- /dev/null +++ b/.agents/AGENTS.md @@ -0,0 +1,49 @@ +# AGENTS.md + +`onde-cli` is a Rust TUI (`onde` binary) for managing an Onde Inference account and running a local model pipeline: LoRA fine-tune → adapter merge → GGUF export → local chat test → Hugging Face upload → model assignment to an Onde app. npm, PyPI, NuGet, pub.dev, Homebrew, and crates.io are thin wrappers around this one binary; the Rust crate is the source of truth. + +## Build, test, lint + +```sh +cargo build # debug build +cargo run # launch the TUI +cargo test --locked # unit tests (heavy ones are #[ignore]) +cargo clippy --all-targets -- -D warnings +cargo fmt -- --check +``` + +The build **requires** these env vars (read from `.env` via `build.rs`): `ONDE_APP_ID`, `ONDE_APP_SECRET`, `GRESIQ_API_KEY`, `GRESIQ_API_SECRET`. `HF_TOKEN` is optional. They are baked in at compile time via `env!(...)` in `src/app.rs`. Changing `.env` triggers a full rebuild. + +On macOS, candle's `metal` + `accelerate` features are added automatically. **Fine-tuning always runs on CPU** — Metal's backward pass in candle 0.10.2 produces corrupt gradients. Use `ONDE_FINETUNE_METAL=1` only to test newer candle versions, and verify with `ONDE_FINETUNE_GRAD_DEBUG=1`. + +Linux CI requires `libclang-dev` (`sudo apt-get install -y libclang-dev`). + +## Architecture — `src/` + +| File(s) | Responsibility | +|---|---| +| `main.rs` | Entry point; redirects stdout/stderr to `~/.cache/onde/debug.log` before ratatui takes over | +| `app.rs` | `App` struct + `Screen` state machine + `AuthEvent` channel; `App::apply` folds background progress into state | +| `ui.rs` | Pure ratatui render of `App`; no SDK imports (re-exports types through `app.rs`) | +| `finetune.rs` | Hand-written LoRA trainer (candle, CPU). Writes `lora_adapter.safetensors` | +| `merge.rs` | Folds LoRA adapter into base weights; writes `model.safetensors` | +| `gguf.rs` | Hand-rolled GGUF writer (no llama.cpp). Tensor dims innermost-first; `head_dim` → `attention.key_length/value_length` | +| `chat.rs` | Local GGUF chat via `onde::mistralrs::GgufModelBuilder` | +| `hf*.rs` | Hugging Face Hub: upload, clone, search, cache resolution | +| `gresiq.rs` | `smbcloud-gresiq-sdk` wrapper; model assignment (`assign_model`) | +| `token.rs` | Auth token persistence | +| `project.rs` | Per-project fine-tune workspaces under `~/.onde` | + +**Background task rule:** network/IO → `tokio::spawn`; CPU-heavy tensor work (`finetune`, `merge`, `gguf`) → `std::thread::spawn` to avoid starving the async runtime. + +**Adding a feature pattern:** new `Screen` variant → key handler → `AuthEvent` variant → background task that streams progress events. + +## Conventions + +- **Git:** merge feature/release/hotfix branches with `--no-ff`. Tag the merge commit on `main`. Never tag a branch tip that hasn't been merged. See `.agents/skills/git/SKILL.md`. +- **Commits:** include `Co-Authored-By: siGit Code <297239231+sigitc@users.noreply.github.com>` trailer when committing with the agent. +- **`[patch.crates-io]` and `path =` deps:** commented out in `Cargo.toml` for local dev against `onde`/`smbcloud-*`/`candle`. Never commit with these uncommented. +- **Distribution:** before touching any wrapper package (npm, PyPI, NuGet, pub.dev, Homebrew), read `.agents/skills/distribution/SKILL.md`. Keep all channel versions aligned with the Rust crate version in `Cargo.toml`. +- **Wrapper packages** are in `npm/`, `pypi/`, `nuget/`, `pub/`. The npm base manifest is rewritten at release time by `npm/scripts/render-main-package.cjs`; do not hand-edit committed package.json versions as the source of truth for releases. +- **Clippy** runs with `-D warnings` in CI; all lints must pass. +- **Rust edition:** 2024; toolchain: stable. diff --git a/AGENTS.md b/AGENTS.md deleted file mode 100644 index 7e43f45..0000000 --- a/AGENTS.md +++ /dev/null @@ -1,49 +0,0 @@ -# AGENTS.md - -`onde-cli` is a Rust TUI (`onde` binary) for managing an Onde Inference account and running a local model pipeline: LoRA fine-tune → adapter merge → GGUF export → local chat test → Hugging Face upload → model assignment to an Onde app. npm, PyPI, NuGet, pub.dev, Homebrew, and crates.io are thin wrappers around this one binary; the Rust crate is the source of truth. - -## Build, test, lint - -```sh -cargo build # debug build -cargo run # launch the TUI -cargo test --locked # unit tests (heavy ones are #[ignore]) -cargo clippy --all-targets -- -D warnings -cargo fmt -- --check -``` - -The build **requires** these env vars (read from `.env` via `build.rs`): `ONDE_APP_ID`, `ONDE_APP_SECRET`, `GRESIQ_API_KEY`, `GRESIQ_API_SECRET`. `HF_TOKEN` is optional. They are baked in at compile time via `env!(...)` in `src/app.rs`. Changing `.env` triggers a full rebuild. - -On macOS, candle's `metal` + `accelerate` features are added automatically. **Fine-tuning always runs on CPU** — Metal's backward pass in candle 0.10.2 produces corrupt gradients. Use `ONDE_FINETUNE_METAL=1` only to test newer candle versions, and verify with `ONDE_FINETUNE_GRAD_DEBUG=1`. - -Linux CI requires `libclang-dev` (`sudo apt-get install -y libclang-dev`). - -## Architecture — `src/` - -| File(s) | Responsibility | -|---|---| -| `main.rs` | Entry point; redirects stdout/stderr to `~/.cache/onde/debug.log` before ratatui takes over | -| `app.rs` | `App` struct + `Screen` state machine + `AuthEvent` channel; `App::apply` folds background progress into state | -| `ui.rs` | Pure ratatui render of `App`; no SDK imports (re-exports types through `app.rs`) | -| `finetune.rs` | Hand-written LoRA trainer (candle, CPU). Writes `lora_adapter.safetensors` | -| `merge.rs` | Folds LoRA adapter into base weights; writes `model.safetensors` | -| `gguf.rs` | Hand-rolled GGUF writer (no llama.cpp). Tensor dims innermost-first; `head_dim` → `attention.key_length/value_length` | -| `chat.rs` | Local GGUF chat via `onde::mistralrs::GgufModelBuilder` | -| `hf*.rs` | Hugging Face Hub: upload, clone, search, cache resolution | -| `gresiq.rs` | `smbcloud-gresiq-sdk` wrapper; model assignment (`assign_model`) | -| `token.rs` | Auth token persistence | -| `project.rs` | Per-project fine-tune workspaces under `~/.onde` | - -**Background task rule:** network/IO → `tokio::spawn`; CPU-heavy tensor work (`finetune`, `merge`, `gguf`) → `std::thread::spawn` to avoid starving the async runtime. - -**Adding a feature pattern:** new `Screen` variant → key handler → `AuthEvent` variant → background task that streams progress events. - -## Conventions - -- **Git:** merge feature/release/hotfix branches with `--no-ff`. Tag the merge commit on `main`. Never tag a branch tip that hasn't been merged. See `.agents/skills/git/SKILL.md`. -- **Commits:** include `Co-Authored-By: siGit Code <297239231+sigitc@users.noreply.github.com>` trailer when committing with the agent. -- **`[patch.crates-io]` and `path =` deps:** commented out in `Cargo.toml` for local dev against `onde`/`smbcloud-*`/`candle`. Never commit with these uncommented. -- **Distribution:** before touching any wrapper package (npm, PyPI, NuGet, pub.dev, Homebrew), read `.agents/skills/distribution/SKILL.md`. Keep all channel versions aligned with the Rust crate version in `Cargo.toml`. -- **Wrapper packages** are in `npm/`, `pypi/`, `nuget/`, `pub/`. The npm base manifest is rewritten at release time by `npm/scripts/render-main-package.cjs`; do not hand-edit committed package.json versions as the source of truth for releases. -- **Clippy** runs with `-D warnings` in CI; all lints must pass. -- **Rust edition:** 2024; toolchain: stable. diff --git a/AGENTS.md b/AGENTS.md new file mode 120000 index 0000000..41280d2 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1 @@ +.agents/AGENTS.md \ No newline at end of file From 855e1ac295d1e229b92c5394d7b508c7b09ae6f8 Mon Sep 17 00:00:00 2001 From: keypair34 <216233964+keypair34@users.noreply.github.com> Date: Sun, 12 Jul 2026 08:11:22 +0200 Subject: [PATCH 3/3] Unify CLAUDE.md into AGENTS.md and symlink it - Merge the heavy-test env knobs, finetune gradient-safety warning, GGUF format details, and the onde dependency description from CLAUDE.md into .agents/AGENTS.md, since those were missing there - Replace the standalone CLAUDE.md with a symlink to .agents/AGENTS.md, matching the convention already used in smbcloud and smbcloud-cli Co-Authored-By: siGit Code <297239231+sigitc@users.noreply.github.com> --- .agents/AGENTS.md | 35 +++++++++++++++++++------ CLAUDE.md | 67 +---------------------------------------------- 2 files changed, 28 insertions(+), 74 deletions(-) mode change 100644 => 120000 CLAUDE.md diff --git a/.agents/AGENTS.md b/.agents/AGENTS.md index 7e43f45..2b3eef0 100644 --- a/.agents/AGENTS.md +++ b/.agents/AGENTS.md @@ -14,23 +14,38 @@ cargo fmt -- --check The build **requires** these env vars (read from `.env` via `build.rs`): `ONDE_APP_ID`, `ONDE_APP_SECRET`, `GRESIQ_API_KEY`, `GRESIQ_API_SECRET`. `HF_TOKEN` is optional. They are baked in at compile time via `env!(...)` in `src/app.rs`. Changing `.env` triggers a full rebuild. -On macOS, candle's `metal` + `accelerate` features are added automatically. **Fine-tuning always runs on CPU** — Metal's backward pass in candle 0.10.2 produces corrupt gradients. Use `ONDE_FINETUNE_METAL=1` only to test newer candle versions, and verify with `ONDE_FINETUNE_GRAD_DEBUG=1`. +On macOS, candle's `metal` + `accelerate` features are added automatically. **Fine-tuning always runs on CPU** — Metal's backward pass in candle 0.10.2 produces corrupt gradients (1e6–1e29 in magnitude, where CPU gives ~40–65 for identical losses). Use `ONDE_FINETUNE_METAL=1` only to test newer candle versions, and verify with `ONDE_FINETUNE_GRAD_DEBUG=1` (per-step loss/max_abs/norm). Linux CI requires `libclang-dev` (`sudo apt-get install -y libclang-dev`). +### Tests that load real models + +`src/gguf.rs` has two `#[ignore]` integration tests that export a ~1GB GGUF and run inference through `onde::mistralrs`. They need a Qwen3-0.6B model cached in the Onde App Group container and skip gracefully when it is absent. + +```sh +cargo test gguf::tests::exported_qwen3_gguf_is_runnable -- --ignored --nocapture +cargo test gguf::tests::finetune_merge_export_run -- --ignored --nocapture +``` + +Env knobs for these: `ONDE_TEST_MODEL_DIR`, `ONDE_TEST_DTYPE` (`f16`/`q8_0`), `ONDE_TEST_LR`, `ONDE_TEST_DATA_PATH`, `ONDE_TEST_EPOCHS`, `ONDE_TEST_MAX_SEQ_LEN`, `ONDE_TEST_EVAL_PROMPT` (`||`-separated prompts), `ONDE_TEST_SYSTEM_PROMPT`. This makes the full-pipeline test double as a headless fine-tune runner for a custom dataset. + +### Debugging the TUI + +`main.rs` redirects both stdout and stderr to `~/.cache/onde/debug.log` before ratatui takes the alternate screen, because `mistral.rs` writes to both fds and would otherwise tear up the display. To see what the app or the inference engine is doing, tail that log; `println!`/`eprintln!`/`log::*` all land there, not on screen. + ## Architecture — `src/` | File(s) | Responsibility | |---|---| | `main.rs` | Entry point; redirects stdout/stderr to `~/.cache/onde/debug.log` before ratatui takes over | -| `app.rs` | `App` struct + `Screen` state machine + `AuthEvent` channel; `App::apply` folds background progress into state | +| `app.rs` | `App` struct + `Screen` state machine + `AuthEvent` channel (`tokio::select!` over an `mpsc` channel and a `crossterm` `EventStream`); `App::apply` folds background progress into state | | `ui.rs` | Pure ratatui render of `App`; no SDK imports (re-exports types through `app.rs`) | -| `finetune.rs` | Hand-written LoRA trainer (candle, CPU). Writes `lora_adapter.safetensors` | -| `merge.rs` | Folds LoRA adapter into base weights; writes `model.safetensors` | -| `gguf.rs` | Hand-rolled GGUF writer (no llama.cpp). Tensor dims innermost-first; `head_dim` → `attention.key_length/value_length` | -| `chat.rs` | Local GGUF chat via `onde::mistralrs::GgufModelBuilder` | -| `hf*.rs` | Hugging Face Hub: upload, clone, search, cache resolution | -| `gresiq.rs` | `smbcloud-gresiq-sdk` wrapper; model assignment (`assign_model`) | +| `finetune.rs` | Hand-written LoRA trainer (candle, CPU): Qwen forward pass (RMSNorm, RoPE, GQA, optional Qwen3 QK-norm), LoRA A/B trained on q/v projections in F32, writes `lora_adapter.safetensors`. Gradients are sanitized (non-finite elements zeroed) and globally norm-clipped in a scaled space before each AdamW step; a step is skipped (loudly) only when the norm is zero or non-finite. **Skipping this guard corrupts every weight after the first bad step.** | +| `merge.rs` | Folds LoRA adapter into base weights (`W + scale·(B@A)`); writes `model.safetensors` plus copied config/tokenizer | +| `gguf.rs` | Hand-rolled GGUF writer (no llama.cpp). Tensor dims are written innermost-first — reverse of safetensors shape, because candle reverses on read; `head_dim` comes from config and is emitted as `attention.key_length`/`value_length` (Qwen3 decouples it from `hidden_size/num_heads`); `token_type` is an INT32 array; `general.architecture` is chosen from `model_type` so Qwen3 routes to candle's `quantized_qwen3` loader | +| `chat.rs` | Local GGUF chat via `onde::mistralrs::GgufModelBuilder`, the same engine the Onde SDK uses, so a model can be tested before publishing | +| `hf_upload.rs` / `hf_clone.rs` / `hf_search.rs` / `hf.rs` | Hugging Face Hub: upload the GGUF, check/create a repo, search models, and resolve/merge the local HF cache (incl. the macOS Onde App Group container) | +| `gresiq.rs` | `smbcloud-gresiq-sdk` wrapper for apps and the model catalog. "Deploying" a model means `assign_model(app_id, model_id)` against a catalog entry; the end app then fetches that assignment through the Onde SDK's `load_assigned_model` and downloads the GGUF. The CLI can only assign models that already exist in the GresIQ catalog | | `token.rs` | Auth token persistence | | `project.rs` | Per-project fine-tune workspaces under `~/.onde` | @@ -38,6 +53,10 @@ Linux CI requires `libclang-dev` (`sudo apt-get install -y libclang-dev`). **Adding a feature pattern:** new `Screen` variant → key handler → `AuthEvent` variant → background task that streams progress events. +### The `onde` dependency + +The `onde` crate provides the inference engine (`onde::mistralrs`, a vendored mistral.rs) and `onde::inference::models::SUPPORTED_MODEL_INFO` (the supported-model catalog the inference picker mirrors). It is normally the published crate; `Cargo.toml` has commented `path`/`[patch.crates-io]` blocks for developing against local checkouts of `onde`, the `smbcloud-*` crates, and `candle` — never commit with those uncommented. + ## Conventions - **Git:** merge feature/release/hotfix branches with `--no-ff`. Tag the merge commit on `main`. Never tag a branch tip that hasn't been merged. See `.agents/skills/git/SKILL.md`. diff --git a/CLAUDE.md b/CLAUDE.md deleted file mode 100644 index 4db8f75..0000000 --- a/CLAUDE.md +++ /dev/null @@ -1,66 +0,0 @@ -# CLAUDE.md - -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. - -## What this is - -`onde-cli` is a native Rust terminal UI (binary name `onde`) for managing an Onde Inference account and running a local model pipeline: fine-tune a safetensors small language model with LoRA, merge the adapter, export to GGUF, test it in a local chat, upload it to Hugging Face, and assign it to an Onde app. All packaging (npm, PyPI, NuGet, pub.dev, Homebrew, crates.io) is thin wrappers around this one binary; the Rust crate is the source of truth. - -## Build, run, test - -```sh -cargo build # debug build -cargo run # launches the TUI -cargo test # unit tests only (the heavy ones are #[ignore]) -cargo clippy --all-targets -``` - -The build **bakes credentials in at compile time** via `build.rs`, which reads `.env` (or the environment in CI). The build fails if any of these are missing: `ONDE_APP_ID`, `ONDE_APP_SECRET`, `GRESIQ_API_KEY`, `GRESIQ_API_SECRET`. `HF_TOKEN` is optional (defaults to empty). Changing `.env` triggers a rebuild. These are exposed in code as `env!(...)` constants in `src/app.rs`. - -On macOS the build pulls in candle's `metal` and `accelerate` features. Inference runs on Metal; **training deliberately runs on CPU everywhere** — candle 0.10.2's Metal backward pass produces garbage gradient magnitudes (1e6–1e29 where CPU gives ~40–65 for identical losses), and Accelerate is ~2x faster per training step anyway. `ONDE_FINETUNE_METAL=1` re-enables Metal training for testing newer candle versions; verify gradients with `ONDE_FINETUNE_GRAD_DEBUG=1` (per-step loss/max_abs/norm) before trusting it. - -### Tests that load real models - -`src/gguf.rs` has two `#[ignore]` integration tests that export a ~1GB GGUF and run inference through `onde::mistralrs`. They need a Qwen3-0.6B model cached in the Onde App Group container and skip gracefully when it is absent. - -```sh -cargo test gguf::tests::exported_qwen3_gguf_is_runnable -- --ignored --nocapture -cargo test gguf::tests::finetune_merge_export_run -- --ignored --nocapture -``` - -Env knobs for these: `ONDE_TEST_MODEL_DIR`, `ONDE_TEST_DTYPE` (`f16`/`q8_0`), `ONDE_TEST_LR`, `ONDE_TEST_DATA_PATH`, `ONDE_TEST_EPOCHS`, `ONDE_TEST_MAX_SEQ_LEN`, `ONDE_TEST_EVAL_PROMPT` (`||`-separated prompts), `ONDE_TEST_SYSTEM_PROMPT`. This makes the full-pipeline test double as a headless fine-tune runner for a custom dataset. - -### Debugging the TUI - -`main.rs` redirects both stdout and stderr to `~/.cache/onde/debug.log` before ratatui takes the alternate screen, because `mistral.rs` writes to both fds and would otherwise tear up the TUI. To see what the app or the inference engine is doing, tail that log; `println!`/`eprintln!`/`log::*` all land there, not on screen. - -## Architecture - -### TUI event loop (`app.rs`, `ui.rs`, `main.rs`) - -The whole app is one `App` struct plus a `Screen` enum acting as a state machine. `app::run` owns a single `tokio::sync::mpsc` channel of `AuthEvent`s and a `crossterm` `EventStream`, multiplexed with `tokio::select!`. Keystrokes mutate `App` synchronously; anything slow (network calls, downloads, fine-tune, merge, GGUF export, chat inference) is spawned as a background tokio task or OS thread that streams progress back as `AuthEvent`s, which `App::apply` folds into state. `ui.rs` is a pure render of `App` and intentionally does not depend on the SDK directly (it re-exports `OndeApp`/`OndeModel` through `app.rs`). When adding a feature, the pattern is: add a `Screen` variant, a key handler, an `AuthEvent` variant, and a background task that emits progress. - -Background work uses two task kinds deliberately: network/IO uses `tokio::spawn`; CPU-heavy tensor work (`finetune`, `merge`, `gguf`) uses `std::thread::spawn` so it does not starve the async runtime. - -### The local model pipeline - -These modules form the fine-tune-to-deploy chain, each running on a background thread and streaming a `*Progress` enum: - -- `finetune.rs` — hand-written LoRA trainer (candle). Builds a Qwen forward pass (RMSNorm, RoPE, GQA, optional Qwen3 QK-norm), trains LoRA A/B on q/v projections in F32, writes `lora_adapter.safetensors`. Gradients are sanitized (non-finite elements zeroed) and globally norm-clipped before each AdamW step; the norm is computed in a scaled space so huge-but-finite gradients clip instead of overflowing the f32 sum to `inf`. A step is skipped (loudly) only when the norm is zero or non-finite. Skipping this guard corrupts every weight after the first bad step. Training runs on CPU — see the Metal note above. -- `merge.rs` — folds the LoRA adapter back into base weights (`W + scale·(B@A)`), writes a merged `model.safetensors` plus copied config/tokenizer. -- `gguf.rs` — **hand-rolled GGUF writer** (no llama.cpp). Conventions that must hold for mistral.rs/candle to load and run the file correctly: tensor dims are written innermost-first (reverse of safetensors shape, because candle reverses on read); `head_dim` comes from config and is emitted as `attention.key_length`/`value_length` (Qwen3 decouples it from `hidden_size/num_heads`); `token_type` is an INT32 array; `general.architecture` is chosen from `model_type` so Qwen3 routes to candle's `quantized_qwen3` loader. -- `chat.rs` — loads a local GGUF via `onde::mistralrs::GgufModelBuilder` (the same engine the Onde SDK uses) and streams a multi-turn chat, so a model can be tested before publishing. -- `hf_upload.rs` / `hf_clone.rs` / `hf_search.rs` / `hf.rs` — Hugging Face Hub: upload the GGUF, check/create a repo, search models, and resolve/merge the local HF cache (incl. the macOS Onde App Group container). - -### Account / deploy side (`gresiq.rs`, `token.rs`, `project.rs`) - -`gresiq.rs` wraps `smbcloud-gresiq-sdk` for apps and the model catalog. "Deploying" a model means `assign_model(app_id, model_id)` against a catalog entry; the end app (e.g. `rumilearnpersian`) then fetches that assignment through the Onde SDK's `load_assigned_model` and downloads the GGUF. The CLI can only assign models that already exist in the GresIQ catalog. `token.rs` persists the auth token; `project.rs` manages per-project fine-tune workspaces under `~/.onde`. - -### The `onde` dependency - -The `onde` crate provides the inference engine (`onde::mistralrs`, a vendored mistral.rs) and `onde::inference::models::SUPPORTED_MODEL_INFO` (the supported-model catalog the inference picker mirrors). It is normally the published crate; `Cargo.toml` has commented `path`/`[patch.crates-io]` blocks for developing against local checkouts of `onde`, the `smbcloud-*` crates, and `candle`. Never commit with those uncommented. - -## Conventions - -- Git: merge feature/release/hotfix branches with `--no-ff` (explicit merge commits). Tag the merge commit that holds the final release state. See `.agents/skills/git/SKILL.md`. -- Distribution changes: see `.agents/skills/distribution/SKILL.md` before touching any wrapper package; keep all channel versions aligned with the Rust crate version. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 120000 index 0000000..41280d2 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +.agents/AGENTS.md \ No newline at end of file