Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 10 additions & 9 deletions ARCHITECTURE-ACCORDING-TO-CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

> This document was generated by Claude from reading the source code.

Pushwork is a CLI tool for bidirectional file synchronization using **Automerge CRDTs**. It maps a local filesystem directory tree to a mirror tree of Automerge documents and syncs them through either a WebSocket relay server (default) or the Subduction backend (opt-in via `--sub`). Multiple peers can edit the same files and changes merge automatically.
Pushwork is a CLI tool for bidirectional file synchronization using **Automerge CRDTs**. It maps a local filesystem directory tree to a mirror tree of Automerge documents and syncs them through either the Subduction backend (default) or a legacy WebSocket relay server (opt-in via `--legacy`). Multiple peers can edit the same files and changes merge automatically.

## Module Diagram

Expand All @@ -28,8 +28,8 @@ Pushwork is a CLI tool for bidirectional file synchronization using **Automerge
│ defaults < │ │ Automerge │ │ Orchestrates the │
│ global < │ │ Repo with │ │ entire sync cycle │
│ local │ │ storage + │ │ │
└──────────────┘ │ WebSocket or │ │ ┌────────────────┐ │
Subduction │ │ │ChangeDetector │ │
└──────────────┘ │ Subduction or│ │ ┌────────────────┐ │
WebSocket │ │ │ChangeDetector │ │
└──────────────┘ │ │ FS vs snapshot │ │
│ │ vs remote docs │ │
│ └────────────────┘ │
Expand Down Expand Up @@ -60,12 +60,13 @@ Pushwork is a CLI tool for bidirectional file synchronization using **Automerge
┌───────────────────────────────┐
│ Sync backend (one of) │
│ │
│ • WebSocket relay (default) │
│ sync3.automerge.org │
│ │
│ • Subduction (--sub opt-in) │
│ • Subduction (default) │
│ subduction.sync │
│ .inkandswitch.com │
│ │
│ • Legacy WebSocket relay │
│ (--legacy opt-in) │
│ sync3.automerge.org │
└───────────────────────────────┘
```

Expand Down Expand Up @@ -136,7 +137,7 @@ DirectoryDocument (root)

1. Verifies the `.pushwork/` directory exists
2. Loads and merges config (defaults < global `~/.pushwork/config.json` < local `.pushwork/config.json`)
3. Creates an Automerge `Repo` via `createRepo()` — sets up `NodeFSStorageAdapter` for local persistence, and either a `BrowserWebSocketClientAdapter` (default) or the Subduction backend (`subductionWebsocketEndpoints`, when `config.subduction === true`) for network sync
3. Creates an Automerge `Repo` via `createRepo()` — sets up `NodeFSStorageAdapter` for local persistence, and either the Subduction backend (default, via `subductionWebsocketEndpoints`) or a `BrowserWebSocketClientAdapter` (legacy, when `config.protocol === "legacy"`) for network sync
4. Instantiates a `SyncEngine` with the repo, working directory, and config

Every command (sync, commit, status, diff, ls, etc.) calls `setupCommandContext()`, uses the `SyncEngine`, then calls `safeRepoShutdown()`.
Expand Down Expand Up @@ -246,7 +247,7 @@ your-project/
| `@automerge/automerge` | Core CRDT engine: splice, changeAt, RawString |
| `@automerge/automerge-repo` | Repo, DocHandle, document lifecycle management |
| `@automerge/automerge-repo-network-websocket` | WebSocket transport to relay server (default backend) |
| `@automerge/automerge-subduction` | Subduction Wasm bindings (opt-in backend via `--sub`) |
| `@automerge/automerge-subduction` | Subduction Wasm bindings (default backend) |
| `@automerge/automerge-repo-storage-nodefs` | Local filesystem persistence for Automerge docs |
| `@commander-js/extra-typings` | CLI command framework |
| `diff` | Character-level diffing to feed `A.splice()` |
Expand Down
35 changes: 25 additions & 10 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,21 +139,23 @@ Used throughout sync-engine: if heads are available, calls `handle.changeAt(head
- **`waitForBidirectionalSync` on large trees.** Full tree traversal (`getAllDocumentHeads`) is expensive because it `repo.find()`s every document. For post-push stabilization, pass the `handles` option to only check documents that actually changed. The initial pre-pull call still needs the full scan to discover remote changes. The dynamic timeout adds the first scan's duration on top of the base timeout, since the first scan is just establishing baseline — its duration shouldn't count against stability-wait time.
- **Versioned URLs and `repo.find()`.** `repo.find(versionedUrl)` returns a view handle whose `.heads()` returns the VERSION heads, not the current document heads. Always use `getPlainUrl()` when you need the current/mutable state. The snapshot head update loop at the end of `sync()` must use `getPlainUrl(snapshotEntry.url)` — without this, artifact directories (which store versioned URLs) get stale heads written to the snapshot, causing `changeAt()` to fork from the wrong point on the next sync. This was the root cause of the artifact deletion resurrection bug: `batchUpdateDirectory` would `changeAt` from an empty directory state where the file entry didn't exist yet, so the splice found nothing to delete.

## Subduction sync backend (`--sub`)
## Sync backends (default: Subduction)

The `--sub` flag switches from the default WebSocket sync adapter to the Subduction backend built into `automerge-repo@2.6.0-subduction.14`. The Repo manages a `SubductionSource` internally — pushwork just passes `subductionWebsocketEndpoints` and the Repo handles connection management, sync, and retries.
Pushwork supports two sync backends. Subduction is the default; legacy WebSocket is opt-in via `--legacy` on `init`/`clone`/`track`.

The Repo manages a `SubductionSource` internally — pushwork just passes `subductionWebsocketEndpoints` (Subduction mode) or constructs a `BrowserWebSocketClientAdapter` (legacy mode), and the Repo handles connection management, sync, and retries.

### How it works

- `repo-factory.ts`: Initializes Subduction Wasm via ESM dynamic import, then creates Repo. When `sub: true`, passes `subductionWebsocketEndpoints: [syncServer]` and the Repo handles sync cadence internally. When `sub: false`, uses the traditional WebSocket network adapter instead.
- Default server: `wss://subduction.sync.inkandswitch.com` (vs `wss://sync3.automerge.org` for WebSocket)
- `repo-factory.ts`: Initializes Subduction Wasm via ESM dynamic import, then creates Repo. When `sub: true` (Subduction, default), passes `subductionWebsocketEndpoints: [syncServer]` and the Repo handles sync cadence internally. When `sub: false` (legacy), uses the traditional WebSocket network adapter instead.
- Default Subduction server: `wss://subduction.sync.inkandswitch.com`; legacy server: `wss://sync3.automerge.org`
- `network-sync.ts`: When no `StorageId` is provided (Subduction mode), `waitForSync` falls back to head-stability polling (3 consecutive stable checks at 100ms intervals) instead of `getSyncInfo`-based verification
- `sync-engine.ts`: In sub mode, skips `recreateFailedDocuments` — SubductionSource has its own heal-sync retry logic
- Everything else (push/pull phases, artifact handling, `nukeAndRebuildDocs`, change detection) is identical
- `sync-engine.ts`: In Subduction mode, skips `recreateFailedDocuments` — SubductionSource has its own heal-sync retry logic
- Everything else (push/pull phases, artifact handling, `nukeAndRebuildDocs`, change detection) is identical across backends

### Wasm initialization

As of `automerge-repo@2.6.0-subduction.14`, the Repo constructor _always_ creates a `SubductionSource` internally (even without Subduction endpoints), which imports `MemorySigner` and `set_subduction_logger` from `@automerge/automerge-subduction/slim`. The `/slim` entry does NOT auto-init the Wasm — so Wasm must be initialized before _any_ `new Repo()` call, including the default WebSocket path.
As of `automerge-repo@2.6.0-subduction.14`, the Repo constructor _always_ creates a `SubductionSource` internally (even without Subduction endpoints), which imports `MemorySigner` and `set_subduction_logger` from `@automerge/automerge-subduction/slim`. The `/slim` entry does NOT auto-init the Wasm — so Wasm must be initialized before _any_ `new Repo()` call, including the legacy WebSocket path.

`automerge-repo` exports `initSubduction()` which dynamically imports `@automerge/automerge-subduction` (the non-`/slim` entry that auto-inits Wasm). Pushwork calls this via `repoMod.initSubduction()` after loading the Repo module — no direct dependency on `@automerge/automerge-subduction` is needed.

Expand All @@ -168,14 +170,27 @@ The Repo class itself is also loaded via this ESM dynamic import (cached after f
- The `automerge-repo-network-websocket` adapter's `NetworkAdapter` types are slightly behind the repo's `NetworkAdapterInterface` (missing `state()` method in declarations). The adapter works at runtime; the type mismatch is worked around with `as unknown as NetworkAdapterInterface`.
- New `"heal-exhausted"` event on Repo fires when self-healing sync gives up after all retry attempts for a document. Not currently used by pushwork but available for better error reporting.

### Subduction mode persistence
### Backend persistence in config

`--sub` is only accepted on `init` and `clone`. It persists `subduction: true` in `.pushwork/config.json`. All subsequent commands (`sync`, `watch`, etc.) read it from config via `config.subduction ?? false`. The force-defaults path in `setupCommandContext` preserves `subduction` alongside `root_directory_url`.
`--legacy` is only accepted on `init`, `clone`, and `track`. It persists `"protocol": "legacy"` in `.pushwork/config.json`. Default (Subduction) installs persist `"protocol": "subduction"`. All subsequent commands (`sync`, `watch`, etc.) read it from config via `resolveProtocol(localConfig)`. The force-defaults path in `setupCommandContext` preserves the protocol alongside `root_directory_url` and any user-configured `sync_server`.

When Subduction mode is active, commands print a banner: "Using Subduction sync backend (from config)".
When legacy mode is active, commands print a banner: "Using legacy WebSocket sync backend (from config)". No banner is printed for default Subduction operation.

Every `sync` run prints the root Automerge URL at the end.

### Config schema version and migration

The `config_version` field in `.pushwork/config.json` tracks schema version. Current: `CONFIG_VERSION = 1` (exported from `src/types/config.ts`).

- **v0** (no `config_version` field): pre-flip configs. Had a `subduction?: boolean` opt-in flag. Absence of that flag ⇒ classic WebSocket install.
- **v1**: Subduction is the default. Uses `"protocol": "subduction" | "legacy"` instead of `subduction: boolean`. Always written explicitly.

Migration is in `ConfigManager.migrateIfNeeded()`:

- Write-ish commands (`init`, `clone`, `track`, `sync`, `watch`, `commit`) call `migrateConfigIfNeeded()` at the top. Read-only commands (`status`, `diff`, `log`, `ls`, `url`) do not — they read v0 configs transparently via `resolveProtocol` in memory without touching disk.
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section states that init, clone, and track call migrateConfigIfNeeded() at the top, but the current command implementations only invoke migration for sync, watch, and commit. Please update the command list here (or adjust the code) to avoid documenting behavior that doesn’t happen (especially around .bak backups).

Suggested change
- Write-ish commands (`init`, `clone`, `track`, `sync`, `watch`, `commit`) call `migrateConfigIfNeeded()` at the top. Read-only commands (`status`, `diff`, `log`, `ls`, `url`) do not — they read v0 configs transparently via `resolveProtocol` in memory without touching disk.
- Commands that currently call `migrateConfigIfNeeded()` at the top are `sync`, `watch`, and `commit`. Read-only commands (`status`, `diff`, `log`, `ls`, `url`) do not — they read v0 configs transparently via `resolveProtocol` in memory without touching disk.

Copilot uses AI. Check for mistakes.
- Migration reads the raw v0 config, infers protocol (`subduction: true` → `"subduction"`; `false` or absent → `"legacy"`), writes a backup to `config.json.bak` (or `.bak.1`, `.bak.2`, ... if prior backups exist), and rewrites the file in v1 shape. Prints a multi-line banner so the user sees what happened.
- `resolveProtocol(config)` is the single source of truth for backend selection across all paths. Given `null`/`undefined` it returns `"subduction"` (new-install default).

### Corrupt storage recovery

`repo-factory.ts` scans `.pushwork/automerge/` for 0-byte files before creating the Repo. These indicate incomplete writes from a previous run (process exited before storage flushed). If any are found, the entire automerge cache is wiped and recreated — data will re-download from the sync server. The snapshot (`.pushwork/snapshot.json`) is preserved so all document URLs are retained.
Expand Down
77 changes: 51 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,15 +45,15 @@ pushwork url

**`init [path]`** - Initialize sync in a directory

- `--sync-server <url> <storage-id>` - Custom sync server URL and storage ID
- `--sub` - Use the Subduction sync backend (opt-in, persisted in config)
- `--sync-server <url> <storage-id...>` - Custom sync server URL and storage ID
- `--legacy` - Use the legacy WebSocket sync backend (Subduction is default)
- `--debug` - Export performance flame graphs

**`clone <url> <path>`** - Clone an existing synced directory

- `--force` - Overwrite existing directory
- `--sync-server <url> <storage-id>` - Custom sync server URL and storage ID
- `--sub` - Use the Subduction sync backend (opt-in, persisted in config)
- `--sync-server <url> <storage-id...>` - Custom sync server URL and storage ID
- `--legacy` - Use the legacy WebSocket sync backend (Subduction is default)

**`sync [path]`** - Run bidirectional synchronization

Expand Down Expand Up @@ -105,38 +105,63 @@ Configuration is stored in `.pushwork/config.json`:

```json
{
"config_version": 1,
"protocol": "subduction",
"sync_server": "wss://subduction.sync.inkandswitch.com",
"sync_enabled": true,
"exclude_patterns": [".git", "node_modules", "*.tmp", ".pushwork"],
"artifact_directories": ["dist"],
"sync": {
"move_detection_threshold": 0.7
}
}
```

A legacy-backend config looks like:

```json
{
"config_version": 1,
"protocol": "legacy",
"sync_server": "wss://sync3.automerge.org",
"sync_server_storage_id": "3760df37-a4c6-4f66-9ecd-732039a9385d",
"sync_enabled": true,
"defaults": {
"exclude_patterns": [".git", "node_modules", "*.tmp", ".pushwork"],
"large_file_threshold": "100MB"
},
"diff": {
"show_binary": false
},
"exclude_patterns": [".git", "node_modules", "*.tmp", ".pushwork"],
"artifact_directories": ["dist"],
"sync": {
"move_detection_threshold": 0.8,
"prompt_threshold": 0.5,
"auto_sync": false,
"parallel_operations": 4
"move_detection_threshold": 0.7
}
}
```

### Sync Backends

Pushwork supports two sync backends:

- **WebSocket (default)** — talks to `wss://sync3.automerge.org` via the
standard Automerge sync protocol. Uses `sync_server_storage_id` to
verify delivery via `getSyncInfo`.
- **Subduction (opt-in)** — pass `--sub` on `init` or `clone` to select
the Subduction backend (default endpoint:
`wss://subduction.sync.inkandswitch.com`). The Subduction choice is
persisted in `.pushwork/config.json` as `"subduction": true`, so
subsequent `sync` / `watch` commands pick it up automatically.
`sync_server_storage_id` is not used in this mode.
Pushwork supports two sync backends. Subduction is the default.

- **Subduction (default)** — `wss://subduction.sync.inkandswitch.com`.
The backend is selected at `init` / `clone` time and persisted in
`.pushwork/config.json` as `"protocol": "subduction"`. Subsequent
`sync` / `watch` runs read the choice from config.
- **Legacy WebSocket** — opt in via `--legacy` on `init` or `clone` to
use `wss://sync3.automerge.org` with `sync_server_storage_id` for
delivery verification. Persisted as `"protocol": "legacy"`.

### Config schema version

Configs written by current pushwork include `"config_version": 1`.
Older configs (without this field) are automatically migrated on the
next write-ish command (`sync`, `watch`, `commit`, `init`, `clone`,
`track`). The original v0 file is saved as `config.json.bak` (or
`config.json.bak.1`, `.bak.2`, ... if earlier backups exist) and a
notice is printed.
Comment on lines +152 to +156
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs say v0 configs are migrated (with .bak backups) on init, clone, and track, but current behavior differs: init refuses when .pushwork/ already exists, and track --force overwrites config/snapshot without creating a .bak. Migration/backup currently happens when ConfigManager.migrateIfNeeded() is invoked (wired up in sync, watch, commit). Please adjust this list to match actual behavior (or update the code to match the doc).

Suggested change
Older configs (without this field) are automatically migrated on the
next write-ish command (`sync`, `watch`, `commit`, `init`, `clone`,
`track`). The original v0 file is saved as `config.json.bak` (or
`config.json.bak.1`, `.bak.2`, ... if earlier backups exist) and a
notice is printed.
Older configs (without this field) are automatically migrated when you
run `sync`, `watch`, or `commit`. The original v0 file is saved as
`config.json.bak` (or `config.json.bak.1`, `.bak.2`, ... if earlier
backups exist) and a notice is printed.

Copilot uses AI. Check for mistakes.

Migration inference:

- v0 config with `"subduction": true` → `"protocol": "subduction"`
- v0 config with `"subduction": false` → `"protocol": "legacy"`
- v0 config with no `subduction` key → `"protocol": "legacy"` (this
matches pre-Subduction installs that were already using the
WebSocket relay)

## How It Works

Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "pushwork",
"version": "1.2.2",
"version": "1.3.0",
"description": "Bidirectional directory synchronization using Automerge CRDTs",
"main": "dist/index.js",
"exports": {
Expand Down
18 changes: 9 additions & 9 deletions src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -77,21 +77,21 @@ program
"--sync-server <url> <storage-id...>",
"Custom sync server URL and storage ID"
)
.option("--sub", "Use Subduction sync backend", false)
.option("--legacy", "Use legacy WebSocket sync backend", false)
.action(async (path, opts) => {
const [syncServer, syncServerStorageId] = validateSyncServer(
opts.syncServer
);
await init(path, { syncServer, syncServerStorageId, sub: opts.sub });
await init(path, { syncServer, syncServerStorageId, legacy: opts.legacy });
});

// Track command (set root directory URL without full initialization)
const trackAction = async (
url: string,
path: string,
opts: { force: boolean; sub: boolean }
opts: { force: boolean; legacy: boolean }
) => {
await root(url, path, { force: opts.force, sub: opts.sub });
await root(url, path, { force: opts.force, legacy: opts.legacy });
};

program
Expand All @@ -107,7 +107,7 @@ program
"."
)
.option("-f, --force", "Overwrite existing pushwork setup", false)
.option("--sub", "Use Subduction sync backend", false)
.option("--legacy", "Use legacy WebSocket sync backend", false)
.action(async (url, path, opts) => {
await trackAction(url, path, opts);
});
Expand All @@ -118,8 +118,8 @@ program
.argument("<url>")
.argument("[path]", "", ".")
.option("-f, --force", "", false)
.option("--sub", "", false)
.action(async (url: string, path: string, opts: { force: boolean; sub: boolean }) => {
.option("--legacy", "", false)
.action(async (url: string, path: string, opts: { force: boolean; legacy: boolean }) => {
await trackAction(url, path, opts);
});

Expand All @@ -137,7 +137,7 @@ program
"--sync-server <url> <storage-id...>",
"Custom sync server URL and storage ID"
)
.option("--sub", "Use Subduction sync backend", false)
.option("--legacy", "Use legacy WebSocket sync backend", false)
.option("-v, --verbose", "Verbose output", false)
.action(async (url, path, opts) => {
const [syncServer, syncServerStorageId] = validateSyncServer(
Expand All @@ -148,7 +148,7 @@ program
verbose: opts.verbose,
syncServer,
syncServerStorageId,
sub: opts.sub,
legacy: opts.legacy,
});
});

Expand Down
Loading