Skip to content

Optimize wa-sqlite binary RPC reads#159

Merged
Antonov548 merged 1 commit into
mainfrom
wa-sqlite-rpc-transfer-efficiency
May 30, 2026
Merged

Optimize wa-sqlite binary RPC reads#159
Antonov548 merged 1 commit into
mainfrom
wa-sqlite-rpc-transfer-efficiency

Conversation

@marcus-pousette
Copy link
Copy Markdown
Collaborator

@marcus-pousette marcus-pousette commented May 21, 2026

Summary

  • return tree.parent and tree.getPayload worker RPC results as Uint8Array instead of number arrays
  • transfer those binary response buffers for dedicated-worker and shared-worker reads
  • chunk large browser appendMany calls at RPC boundaries so a single large sync/import batch does not monopolize the worker queue
  • extend the existing wa-sqlite responsiveness e2e to exercise one 5k-op append while sampling child and payload reads

Why

The stall shape we care about is a read getting stuck behind a large write batch in the worker. The test now runs one large append by default, then reports child-read and payload-read p50/p95/max during that write.

Local single-run comparison against origin/main with one 5k-op append:

Runtime main child read max PR child read max PR payload read max
dedicated-worker 720ms 236ms 236ms
shared-worker 802ms 537ms 493ms

The PR still keeps write throughput in the same rough range; the goal is to avoid second-scale read stalls, not to add a fragile microbenchmark threshold.

Testing

  • node packages/treecrdt-wa-sqlite/scripts/copy-opfs-vfs.mjs --sources-only && ./node_modules/.bin/tsc -p packages/treecrdt-wa-sqlite/tsconfig.json
  • ./node_modules/.bin/prettier --check packages/treecrdt-wa-sqlite/src/client.ts packages/treecrdt-wa-sqlite/e2e/tests/responsiveness.spec.ts packages/treecrdt-wa-sqlite/src/rpc.ts packages/treecrdt-wa-sqlite/src/common-worker.ts packages/treecrdt-wa-sqlite/src/worker.ts packages/treecrdt-wa-sqlite/src/shared-worker.ts packages/treecrdt-wa-sqlite/e2e/src/responsiveness.ts
  • git diff --check
  • ./node_modules/.bin/playwright test tests/responsiveness.spec.ts --project chromium-dev --reporter=line from packages/treecrdt-wa-sqlite/e2e using a local temporary pnpm shim because this shell has no pnpm binary

@marcus-pousette marcus-pousette force-pushed the wa-sqlite-rpc-transfer-efficiency branch from 6acca00 to 2ed710b Compare May 21, 2026 13:42
@Antonov548 Antonov548 merged commit 33adcb8 into main May 30, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants