From f93799e944aef824632fe00e191222c27d78bc2c Mon Sep 17 00:00:00 2001 From: lody Date: Tue, 6 Jan 2026 11:32:18 +0000 Subject: [PATCH 1/3] docs: add HTTP push + SSE transport profile Co-authored-by: lody --- README.md | 1 + http-push-sse-protocol.md | 156 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 157 insertions(+) create mode 100644 http-push-sse-protocol.md diff --git a/README.md b/README.md index 9f261ee..9bd1f00 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,7 @@ loro-protocol is a small, transport-agnostic syncing protocol for collaborative - Transports: WebSocket or any integrity-preserving transport (e.g., WebRTC) See `protocol.md` for the full wire spec. +See `http-push-sse-protocol.md` for an HTTP push + SSE transport profile (no wire format changes). ## Packages diff --git a/http-push-sse-protocol.md b/http-push-sse-protocol.md new file mode 100644 index 0000000..a024226 --- /dev/null +++ b/http-push-sse-protocol.md @@ -0,0 +1,156 @@ +# HTTP Push + SSE Transport Profile (Loro Syncing Protocol) + +Transport profile version: 0. Extends `protocol.md` (Protocol v1) without changing the binary wire format. + +This document describes a transport mapping that uses: + +- **HTTP push** for client → server frames (request body carries a single protocol frame). +- **SSE (Server‑Sent Events)** for server → client frames (each SSE event carries a single protocol frame). + +The goal is to reuse all message types, fragmentation rules, and semantics from `protocol.md`, while allowing two common exchanges to be handled as simple **HTTP request/response** pairs: + +- `JoinRequest` → `JoinResponseOk` / `JoinError` +- `DocUpdate` / `DocUpdateFragment*` → `Ack` + +## Non‑Goals + +- Defining specific HTTP routes, parameters, or auth schemes. These are application decisions. +- Providing reliable replay for SSE on reconnect (out of scope). Clients rejoin to recover. + +## Terminology + +- **Protocol frame**: the exact bytes produced by `encode(message)` from `loro-protocol` (and parsed by `decode(bytes)`). +- **Session key**: an application-defined opaque identifier that binds one client's push requests to its SSE stream. + - It can be carried via cookie, header, query parameter, etc. + - The transport must ensure the same session key is used for both directions. + +## Core Invariants (Unchanged from `protocol.md`) + +- The binary protocol frame format is unchanged. +- Message types and semantics are unchanged (`JoinRequest`, `DocUpdate`, fragments, `Ack`, `RoomError`, `Leave`, …). +- **Max frame size is still 256 KiB**. Payloads that would exceed the limit MUST use fragmentation. + +## Frame Encodings + +### HTTP push (client → server) + +- Request body: a single protocol frame (binary). +- Recommended headers: + - `Content-Type: application/octet-stream` + - `Content-Length: ` +- Push responses MAY return a protocol frame (binary) when the exchange is naturally request/response: + - `JoinRequest` → `JoinResponseOk` / `JoinError` + - `DocUpdate` / fragments completing a batch → `Ack` + +For other push messages (e.g., `Leave`), the response body can be empty. + +### SSE pull (server → client) + +SSE is text-based, so each binary protocol frame is encoded as base64url. + +Event format: + +``` +event: msg +data: + +``` + +Notes: + +- **Exactly one protocol frame per SSE event**. +- `data:` MAY be split across multiple lines; SSE concatenates them with `\n`. Implementations SHOULD either: + - emit a single `data:` line, or + - split into multiple `data:` lines and base64url‑decode after concatenation with `\n` removed. + +Base64url: + +- RFC 4648 "base64url" (`-` and `_` instead of `+` and `/`). +- Padding (`=`) is OPTIONAL; decoders SHOULD accept both forms. + +## Session Binding + +Because HTTP requests are stateless and SSE is a long-lived stream, implementations MUST bind them with a session key. + +The transport profile does not dictate how, but it MUST satisfy: + +- A push request can be associated with exactly one logical session. +- A server can route room broadcasts to all sessions that have joined that room. + +Security note: if the session key is sensitive, prefer cookie/header transport over query strings (URLs are often logged). + +## Request/Response Simplifications + +### Join handshake (`JoinRequest` → `JoinResponse*`) + +Recommended pattern: + +1. Client issues a push with a `JoinRequest` frame. +2. Server responds in the same HTTP response body with: + - `JoinResponseOk`, or + - `JoinError`. +3. After `JoinResponseOk`, server MAY send backfills (`DocUpdate` or fragments) over SSE. + +Rationale: SSE reconnections can drop in-flight frames; making join responses part of the push response avoids depending on SSE delivery guarantees. + +### Client-originated updates (`DocUpdate*` → `Ack`) + +Recommended pattern: + +- For `DocUpdate` (single frame): + - Client pushes `DocUpdate`. + - Server MUST respond with `Ack` (binary) in the HTTP response body. + +- For fragmented updates (`DocUpdateFragmentHeader` + `DocUpdateFragment`): + - Client pushes the header and fragments. + - Server MUST emit exactly one `Ack` per batch ID, referencing the batch ID. + - It is RECOMMENDED that the `Ack` is returned as the HTTP response to the push that completes the batch + (typically the final fragment). + - Server MAY return an early non‑OK `Ack` when it can reject immediately (not joined, permission denied, rate limited, etc.). + +After accepting and applying a client update, the server broadcasts it to other sessions joined to the room via SSE: + +- Broadcast is typically `DocUpdate` with the same `batchId`, or the original fragments if fragmentation was used. +- The sender does not need to receive its own update (implementation choice). + +## Client Handling of Server Frames (SSE) + +- Server-originated updates and backfills arrive on SSE as `event: msg` frames. +- The client processes them exactly as it would process WebSocket binary frames. + +Ack directionality: + +- Clients SHOULD NOT send `Ack(status=0x00)` for server-originated updates (same reasoning as `protocol.md` WebSocket directionality). +- Clients MAY send a non‑zero `Ack` via HTTP push if they fail to apply a server update (e.g., `invalid_update`, `fragment_timeout`). + +## Keepalive + +The `"ping"`/`"pong"` out-of-band keepalive in `protocol.md` is specific to WebSocket text frames. + +For SSE: + +- Implementations MAY send periodic SSE comments as heartbeats, e.g. `:keepalive\n\n`. +- Heartbeats MUST NOT be parsed as protocol frames and MUST NOT be forwarded to rooms. + +## Ordering and Concurrency + +HTTP push requests can arrive concurrently, which can break assumptions about fragment ordering. + +Recommendations: + +- Serialize push handling per session key. +- Enforce that `DocUpdateFragmentHeader` is observed before accepting fragments for that batch (or buffer until header arrives). +- Use existing batch IDs as the correlation key for both fragments and `Ack`. + +## Loss Recovery on SSE Reconnect + +This profile assumes SSE can disconnect without replay. To recover: + +- Clients SHOULD treat an SSE reconnect as a connection reconnect. +- Clients SHOULD re-issue `JoinRequest` for each active room with their current version so the server can backfill missing updates. + +## Compatibility + +- Works for all CRDT magic types defined in `protocol.md` (including `%ELO` from `protocol-e2ee.md`). +- `%ELO` payload semantics remain unchanged; only the transport encoding differs. + From a66137dcbbb609a7da546d214d9d5a83f240230a Mon Sep 17 00:00:00 2001 From: lody Date: Tue, 6 Jan 2026 11:41:20 +0000 Subject: [PATCH 2/3] docs: clarify HTTP push request/response rules Co-authored-by: lody --- http-push-sse-protocol.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/http-push-sse-protocol.md b/http-push-sse-protocol.md index a024226..22366f4 100644 --- a/http-push-sse-protocol.md +++ b/http-push-sse-protocol.md @@ -38,9 +38,16 @@ The goal is to reuse all message types, fragmentation rules, and semantics from - Recommended headers: - `Content-Type: application/octet-stream` - `Content-Length: ` -- Push responses MAY return a protocol frame (binary) when the exchange is naturally request/response: - - `JoinRequest` → `JoinResponseOk` / `JoinError` - - `DocUpdate` / fragments completing a batch → `Ack` +- Response body: either empty or a single protocol frame (binary). If present, servers SHOULD use: + - `Content-Type: application/octet-stream` + +In this transport profile, two flows are defined as request/response pairs: + +- **Join:** when the client pushes a `JoinRequest`, the server MUST respond in the same HTTP response body with exactly one protocol frame: + - `JoinResponseOk`, or + - `JoinError`. +- **Client-originated updates:** when the client pushes a `DocUpdate` (single frame), the server MUST respond with an `Ack` frame in the same HTTP response body. + - For fragmented updates, the server MUST emit exactly one `Ack` per batch ID; it is RECOMMENDED to return it as the HTTP response to the push that completes the batch (typically the final fragment), or earlier if it can reject immediately. For other push messages (e.g., `Leave`), the response body can be empty. @@ -83,7 +90,7 @@ Security note: if the session key is sensitive, prefer cookie/header transport o ### Join handshake (`JoinRequest` → `JoinResponse*`) -Recommended pattern: +This profile defines the following pattern: 1. Client issues a push with a `JoinRequest` frame. 2. Server responds in the same HTTP response body with: @@ -95,7 +102,7 @@ Rationale: SSE reconnections can drop in-flight frames; making join responses pa ### Client-originated updates (`DocUpdate*` → `Ack`) -Recommended pattern: +This profile defines the following pattern: - For `DocUpdate` (single frame): - Client pushes `DocUpdate`. @@ -153,4 +160,3 @@ This profile assumes SSE can disconnect without replay. To recover: - Works for all CRDT magic types defined in `protocol.md` (including `%ELO` from `protocol-e2ee.md`). - `%ELO` payload semantics remain unchanged; only the transport encoding differs. - From 60406c5bef976efa5c943f5dd58be2fb267f6b23 Mon Sep 17 00:00:00 2001 From: lody Date: Tue, 6 Jan 2026 11:43:55 +0000 Subject: [PATCH 3/3] docs: clarify fragment ordering assumptions for HTTP Co-authored-by: lody --- http-push-sse-protocol.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/http-push-sse-protocol.md b/http-push-sse-protocol.md index 22366f4..7a1d81d 100644 --- a/http-push-sse-protocol.md +++ b/http-push-sse-protocol.md @@ -141,12 +141,13 @@ For SSE: ## Ordering and Concurrency -HTTP push requests can arrive concurrently, which can break assumptions about fragment ordering. +HTTP push requests can arrive concurrently, so receivers may observe frames out of order (for example, a `DocUpdateFragment` arriving before its `DocUpdateFragmentHeader`). Recommendations: - Serialize push handling per session key. -- Enforce that `DocUpdateFragmentHeader` is observed before accepting fragments for that batch (or buffer until header arrives). +- Enforce that `DocUpdateFragmentHeader` is observed before accepting fragments for that batch (or buffer fragments until the header arrives). +- Fragments within a batch SHOULD be reassembled by `index` and MAY be accepted out of order once the header is known. - Use existing batch IDs as the correlation key for both fragments and `Ack`. ## Loss Recovery on SSE Reconnect