Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
fbf6cbf
feat(streams)!: the bearer key moves from a secret slot to a minted b…
wmadden-electric Jul 15, 2026
66a87e0
docs(streams): minted-key auth model in README/SCOPE; conformance pin…
wmadden-electric Jul 15, 2026
fc7e19d
chore(drive): streams-minted-key slice spec + plan
wmadden-electric Jul 15, 2026
11c15d8
chore(drive): record ADR-0030 outcome for streams consumer auth
wmadden-electric Jul 15, 2026
17c834e
chore(streams): reconcile with ADR-0031 (#93) — keep the provider-sco…
wmadden-electric Jul 16, 2026
9a34296
chore(drive): amend streams-minted-key spec — rework onto ADR-0031 (d…
wmadden-electric Jul 16, 2026
d0dd4bd
chore(drive): record consistency as the deciding rationale for the AD…
wmadden-electric Jul 16, 2026
bf6b612
refactor(streams)!: the bearer key is an ADR-0031 provisioning need, …
wmadden-electric Jul 16, 2026
b526045
fix(streams): the provider landing asserts its inbound keys agree
wmadden-electric Jul 16, 2026
d0f553d
chore(drive): amend spec — compute() must be brand-blind; landing bel…
wmadden-electric Jul 16, 2026
6b2a244
refactor(target)!: compute is brand-blind — a provisioned value lands…
wmadden-electric Jul 16, 2026
8f0ec23
test(target): unit net for the boot sweep
wmadden-electric Jul 16, 2026
b469910
fix(examples): the jobs app rides out a cold streams service
wmadden-electric Jul 16, 2026
024d9d5
docs(examples): correct the non-retried-append comment
wmadden-electric Jul 16, 2026
53bb085
fix(examples): size the cold-start retry to a measured cold start
wmadden-electric Jul 16, 2026
be0b5d2
fix(examples)!: the jobs app carries no Compute-specific retry
wmadden-electric Jul 17, 2026
7f26f51
docs(gotchas): describe the cold-start window as observed behavior, n…
wmadden-electric Jul 17, 2026
587030f
chore(drive): record the streams-client-lib direction (pending upstre…
wmadden-electric Jul 17, 2026
3604b61
chore(drive): amendment 3 — protocol logic ships in our client lib
wmadden-electric Jul 17, 2026
8b60b41
feat(streams)!: durableStreams() hydrates to a StreamsClient — protoc…
wmadden-electric Jul 17, 2026
97d6510
fix(streams): the client hints JSON mode — the deployed head answers …
wmadden-electric Jul 17, 2026
2ff7381
test(streams): pin the append contract where a 4xx test cannot reach
wmadden-electric Jul 17, 2026
e963514
ci: cold-start canary (PRO-217) — the fixed-yet? signal for the strea…
wmadden-electric Jul 17, 2026
21ee418
docs(gotchas): the PRO-217 workaround paragraph matches the client-li…
wmadden-electric Jul 17, 2026
077d295
refactor(streams): the stream-not-found predicate belongs to the clie…
wmadden-electric Jul 17, 2026
f8f9f5d
ci(canaries): requirable exits — fail only on the conclusive forcing …
wmadden-electric Jul 17, 2026
ee72faa
docs(gotchas): both canary removal-guards state the requirable exit c…
wmadden-electric Jul 17, 2026
0ca79a0
chore(drive): record the streams-binding redesign from Will's #92 review
wmadden-electric Jul 17, 2026
d787053
chore(drive): spec + dispatch plan for the streams-binding-redesign s…
wmadden-electric Jul 17, 2026
6823479
fix(prisma-cloud): provider-side minted values become declared, valid…
wmadden-electric Jul 17, 2026
94af9c9
refactor(prisma-cloud): move the reserved-provider-param list out of …
wmadden-electric Jul 17, 2026
22464c6
fix(prisma-cloud): close the D1c review findings on Part A (6823479, …
wmadden-electric Jul 17, 2026
9af56a5
docs(adr-0031): say where a provisioned value is stored, not "landing"
wmadden-electric Jul 17, 2026
5a032b4
fix(canaries): the cold-start canary now proves it hit a cold start
wmadden-electric Jul 17, 2026
bdcc2cf
fix(canaries): the cold-start canary stops producing false bug-gone v…
wmadden-electric Jul 17, 2026
ce7d27d
docs(gotchas): PRO-217 boot window, trigger, and removal guard match …
wmadden-electric Jul 17, 2026
df3701a
refactor(streams): the contract names its streams; handles own create…
wmadden-electric Jul 17, 2026
2d37778
refactor(streams): the example app loses all stream lifecycle code
wmadden-electric Jul 17, 2026
1d02fc4
docs(streams): the package README/SCOPE follow the handle-owned lifec…
wmadden-electric Jul 17, 2026
655bda1
merge: Part B — the streams contract names its streams, handles own t…
wmadden-electric Jul 17, 2026
f4b438a
chore(drive): as-built audit of the streams binding, on halt order
wmadden-electric Jul 17, 2026
7b07aa0
fix(streams): the provider contract is the postgres pattern, not a ne…
wmadden-electric Jul 17, 2026
71faaa9
docs: the design records the audit rulings; stale client-factory name…
wmadden-electric Jul 17, 2026
95f7dc9
merge: origin/main (Outputs/entities rename, typed phase handoffs, ap…
wmadden-electric Jul 17, 2026
c4101ea
merge: origin/main (teardown module, ADR-0034, containment model)
wmadden-electric Jul 17, 2026
e51b9d3
Merge remote-tracking branch 'origin/main' into claude/streams-minted…
wmadden-electric Jul 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 89 additions & 0 deletions .drive/projects/forcing-function-apps/design-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,3 +102,92 @@ cutover, streams integration shape, secrets backing, dev-loop architecture.
- [prisma/datahub](https://github.com/prisma/datahub),
[prisma/open-chat](https://github.com/prisma/open-chat),
[prisma/streams](https://github.com/prisma/streams)

## Streams consumer auth — settled by ADR-0030 (2026-07-15)

The streams slice shipped with a root-bound `secret()` bearer key and a
`{ url }`-only binding: consumers declare their own secret slot and the root
binds both to one platform variable. Flagged in review as the first
authenticated module contract — the two slots are connected only by
convention (nothing checks they name the same variable; a mismatch deploys
green and 401s at runtime).

PR #89 (ADR-0030, rpc-service-key project) settles the pattern framework-wide:
wired-peer auth uses a **framework-minted service key carried on the binding's
own config rail** (like the URL), kept in deploy state — explicitly not an
ADR-0029 secret, which is reserved for user-supplied external values.

**Follow-up (blocked on #89 slice 2 — the ServiceKey resource + generic
per-edge value channel):** re-shape streams to match — drop the module's
`apiKey` secret slot, mint the key at deploy, binding becomes
`{ url, apiKey }`. Constraint: `@prisma/streams-server` auth is single-key
(`API_KEY`), so v1 is one minted key per module instance delivered on every
binding; distinct per-edge keys need an upstream accepted-key-set change
(mirroring what #89 slice 1 added to rpc's serve()) — candidate minimal
upstream PR. Do this before S7 (open-chat port) consumes the module.

**ADR-0031 adopted (2026-07-16, corrected).** An earlier note here claimed
per-edge provisioning could not apply because `@prisma/streams-server`
authenticates a single `API_KEY`. That over-claimed: ADR-0031 leaves
cardinality to the provisioner, so a per-PROVIDER provisioner satisfies the
single-key server perfectly well. The design session settled on adopting
ADR-0031, and the deciding reason was **consistency**, not capability: a
module must build on the framework's general internals rather than stand a
second, module-shaped mechanism beside them — two ways to audit one concept
is one too many. Shipped: `durableStreams()`'s `apiKey` param declares a need
under a streams brand; the prisma-cloud target registers a per-provider
provisioner (same `ServiceKey` mint as RPC, keyed on the provider's address)
and lands the value on the streams service; the module owns no credential
resource, and `BearerKey` + the `streams` descriptor are deleted. Future
per-edge keys = flip the provisioner's cardinality and land an accepted set,
once upstream takes a key set (ADR-0031-provisioned-param-values-are-a-need-
resolved-through-a-target-registry.md).

## Streams client lib — the home for compensatory machinery (2026-07-17, direction from Will; upstream check pending)

The examples/streams walkthrough established that the example hand-rolls a
Durable Streams protocol client (URL layout, bearer scheme, JSON-array
appends, offset conventions, long-poll dance) — machinery every consumer
would re-write and that belongs in neither userspace nor the binding.
`@durable-streams/client` (npm, 0.2.x, matches server 0.1.11) is the
protocol's own client: supports per-poll `headers` (built for auth tokens),
`live: long-poll | sse`, pluggable fetch.

Direction: wrap it in a Composer-shipped streams client lib. That lib —
platform-aware by definition — is the legitimate home for the compensatory
machinery that was rightly deleted from the example (D12): auth from the
`{ url, apiKey }` binding, live tail defaulting to long-poll while PRO-218
stands, cold-start retry for idempotent calls only (never appends — no
idempotency key upstream; the D8-D12 evidence and reasoning carry over).
ADR-0015 intact: the binding stays config; the client lib is an app-side
dependency the app chooses, like aws-sdk for storage.

Open until Will's upstream check returns: what @durable-streams/client
already covers vs what we compensate (it may retry / it may grow auth
natively); what belongs upstream (prisma/streams or the client) vs in our
wrapper; where ours ships (likely a composer-prisma-cloud subpath) and its
name; whether the example swap lands with the lib slice. Blocked on that
check; examples/streams stays plain-fetch in #92 meanwhile.

## Streams binding redesign + provider params (2026-07-17, from Will's #92 review)

Recorded exhaustively in [streams-binding-design.md](streams-binding-design.md),
per Will's direction, before any slice spec/plan. Two parts: (A) provider-side
minted values (rpc accepted keys, streams API key) become target-owned reserved
params — declared, schema-validated, carried by the normal serialize/stash
pipeline — deleting `restashAddressFree` and the raw env scrapes it fed;
(B) the streams contract names its streams (optional per-stream event schema,
untyped retained as the `postgres()` parity), `durableStreams(contract)`
hydrates to per-stream handles that own ensure-create and the proven-safe
404 heal, so no stream lifecycle code remains in userspace.

## Halt + audit of the as-built streams binding (2026-07-17, Will's order)

Will halted execution on discovering the `never`-typed provider contract.
Full as-built audit in [streams-binding-audit.md](streams-binding-audit.md):
the checking model moment by moment, the `never` hole and a probe-proven
honest replacement (wide `required`, `__cmp: {}`), the behavior changes that
shipped inside Part B's refactor (implicit create, read-creates, contentType
dropped), the complete unenforced-invariant inventory (T1–T9), deviations
from the design doc, and decisions R1–R6 awaiting his ruling. No further
dispatches until he rules.
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Dispatch plan: streams-binding-redesign

Contract sources: [spec.md](spec.md) +
[streams-binding-design.md](../../streams-binding-design.md). Three
dispatches, sequential (one branch, PR #92). Reviewer rounds after D1 and
D2, per this branch's established loop; docs edits stay with the
orchestrator, not the implementers (the delegated-docs failure mode is on
record twice).

## D1 — Part A: provider params (target package)

**Outcome:** provider-side minted values flow through the declared-param
pipeline end to end. The registration type replaces `ProvisionLanding`
(name + schema + value-from-refs; brand-blind registry in `control.ts`);
`descriptors/compute.ts` writes the rows through the serializer's normal
encode; `deserialize` gains the reserved provider params as a second
enumeration source and the stash carries them (excluded from user `config()`
typing); `restashAddressFree` deleted; `serve()` and the streams entrypoint
read validated stash rows (absent = never provisioned, semantics unchanged);
`serviceKeyEnvName`/`streamsApiKeyEnvName` subsumed by `configKey`
derivation. No coined vocabulary in names or comments — "landing" goes away.

**Completed when:** the spec's invariant tests are green in their new homes;
the four restash tests are replaced by round-trip tests; grep shows no
`restashAddressFree` and no `Landing`; repo checks green; committed.

## D2 — Part B: contract + handles (streams package + example)

**Outcome:** `streamsContract({ jobs: streamDef(), … })` (untyped defs
only); `durableStreams(contract)` hydrates to per-stream handles;
bare `durableStreams()` hydrates to `stream(name)` dynamic handles;
`StreamsClient`/`StreamHandle` as classes; ensure-create + 404-heal
(retry-once) inside the handle; `isStreamNotFound` un-exported; append
no-retry/no-batch and `IDEMPOTENT_BACKOFF` untouched. The example app loses
`STREAM`/`ensureStream`/`withStream` and keeps routes + error mapping; the
heal test and wire-count tests move into the streams package.

**Completed when:** streams package + example tests green; both mutation
checks re-verified red in their new homes; the example has zero lifecycle
or wire-client knowledge; repo checks green; committed.

## D3 — Live re-proof, docs, thread closeout

**Outcome:** deploy/conformance/smoke/canary/destroy per the spec's bar;
gotchas + design-notes touched only where the code moved under them
(orchestrator writes docs); every open #92 thread replied-to (commit or
design-doc section) and resolved; PR body refreshed; Will's re-review
requested. No auto-merge armed; merge only on Will's word.
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Slice: streams binding redesign — provider params + contract-owned lifecycle

## At a glance

Close out Will's 2026-07-17 review of PR #92 by implementing the two settled
designs recorded in
[streams-binding-design.md](../../streams-binding-design.md) (the binding
contract for this slice — read it in full before implementing):

- **Part A**: provider-side minted values (rpc accepted keys, streams API
key) become target-owned **reserved params** — declared, schema-validated,
carried by the normal serialize → deserialize → stash pipeline.
`restashAddressFree` and the raw `process.env` scrapes it fed are deleted.
- **Part B (untyped defs only)**: the streams contract **names its
streams**; `durableStreams(contract)` hydrates to one handle per stream;
handles own ensure-create and the proven-safe 404 heal; the client
becomes a class. `streamDef({ event })` typed validation is the recorded
follow-up and does NOT ship here — no schema parameter that does nothing.

This lands on PR #92's branch (`claude/streams-minted-key`); the slice is
done when every open review thread is answered with either the fix or the
recorded rationale, and Will's re-review is requested.

## Scope

In: everything under "The design" in Parts A and B of the design doc, at the
#92 scope split; moving (not weakening) the existing invariant tests; the
live re-proof; replying on and resolving all 11 open review threads.

Out: `streamDef({ event })` / both-edge validation (follow-up slice); RPC
cold-start handling and its canary (separate task, operator direction);
required-checks branch protection (Will's manual step).

## Verification bar (inherited from this branch's history — do not lower)

- **Invariant tests move, not die**: zero-consumer deny-all `"[]"`;
no-expose-no-rows; a third brand touches only `control.ts`; the boot
round-trip of provider params through deserialize/stash (replaces the four
restash tests).
- **Mutation checks keep their teeth**: the wire-counted append tests (one
POST per append, zero retries — 503 + concurrency) and the heal test
(delete the stream out from under the handle; red without the heal) are
re-verified red after moving into the streams package.
- **Live re-proof** on real Prisma Cloud: deploy `examples/streams`,
conformance (215/239, the 24 = PRO-218 SSE), smoke (401 unauth, authed
append/read/tail), the PRO-217 canary still runs and classifies, destroy
clean.
- Repo checks green throughout: typecheck, biome, lint, depcruise, casts,
`pnpm test:scripts`.

## Review-thread closeout (the slice's exit)

Every open thread on #92 gets a reply naming the commit that resolves it or
the recorded rationale (design doc section), then resolved; the three stale
pre-2026-07-17 threads likewise. Re-request Will's review. No merge without
his word.
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Dispatch plan: streams-minted-key

Two dispatches, sequential. Contract source: [spec.md](spec.md).

## D1 — The key moves to the binding rail (framework + module + tests)

**Outcome:** on a fresh branch off origin/main, the streams binding is
`{ url, apiKey }` end to end locally: minted-credential resource + `streams`
descriptor in `@internal/prisma-cloud`, module/service/entrypoint rewired
(no secret slot anywhere in the package), contract + tests + example root
updated, everything green (package tests incl. the entrypoint integration
test, local conformance, repo lint/typecheck/depcruise/casts).

**Builds on:** merged streams-composed-module (main) + the templates named in
the spec.
**Hands to:** a locally-proven branch ready for the live re-proof.

**Completed when:** `bun test src` green with the integration test driving
`COMPOSER_CREDENTIALS_APIKEY`; `pnpm test:conformance:local` 239/239; grep
shows no `secret(` in packages/1-prisma-cloud/2-shared-modules/streams;
repo checks green; committed (DCO dual sign-off).

## D2 — Live re-proof + docs + PR

**Outcome:** the re-shaped module proven on real Prisma Cloud (deploy,
deployed conformance green modulo PRO-218 SSE failures, smoke: 401 unauth /
authed append + read from offset + long-poll — harness key read from deploy
state), deployment destroyed, README + SCOPE updated, PR open with the
ADR-0030 alignment narrative.

**Builds on:** D1's branch.
**Hands to:** review URL; slice enters review.

**Completed when:** deploy + proofs + destroy recorded with counts; docs
updated; PR open (no auto-merge armed).

## D4 — Rework onto ADR-0031 (added 2026-07-16, spec amendment)

**Outcome:** the key is provisioned through ADR-0031's registry — need on the
`apiKey` param, per-provider provisioner in the target, provider landing for
`API_KEY` — with BearerKey + the streams descriptor deleted, the example
carrying a consumer service, and all local verification green again.

**Builds on:** D1-D3's rebased branch; #93's service-keys machinery as
template.
**Hands to:** an ADR-0031-native branch for the final live re-proof + PR
refresh.

## D5 — Re-proof + PR refresh (after D4)

**Outcome:** live deploy re-proven (consumer service exercises the binding
in-deployment; conformance + smoke as before; destroy), PR body rewritten for
the new mechanism, reviewer round green.
Loading
Loading