pricesrpc+proxy: metered draw-down of prepaid L402 tokens, with a reference pricer daemon by Roasbeef · Pull Request #247 · lightninglabs/aperture

Roasbeef · 2026-07-06T22:30:02Z

In this PR, we extend the dynamic pricing hook so an external pricer can meter prepaid L402 tokens over their whole lifecycle, rather than only quoting a price at challenge time. The motivating shape is any metered, streaming service where the true cost of a request is only known after the response completes: LLM inference is the canonical example, where a client buys a bundle of tokens with one payment, draws it down across many streaming calls, and should be re-challenged exactly when the bundle runs dry. Today aperture only consults the pricer on unauthenticated requests, so once a token is minted it grants access until expiry and the seller has no per-request say and no view of actual usage.

The new pricesrpc surface

GetPrice stays as is. Three RPCs join it, all invoked by aperture against the configured price server:

ChallengeMinted: fired before a fresh 402 challenge goes out, carrying the minted macaroon's token ID and the quoted price. This is the association step: once the client pays, the pricer knows exactly which purchased balance that token anchors. If the notification fails, the challenge is aborted, since the client would otherwise pay for a bundle the pricer will not honor.
AuthorizeRequest: invoked on every authenticated request to a metered service. The pricer answers allow, or deny with the price for the next challenge, and aperture mints a fresh 402 on denial. This closes the token-reuse gap for metered services: one payment no longer means unlimited access until expiry.
ReportUsage: invoked once the response body completes (or the client disconnects), carrying a capped tail of the body. For SSE streams the trailing chunks include the final usage object, which is where the actual cost lives; the pricer does the cost analysis and debits the balance.

Metering is opt-in per service via dynamicprice.metered, with usagetailbytes bounding the captured tail. Requests authenticated through non-L402 schemes (e.g. MPP sessions) skip pricer metering, as session draw-down is already accounted for by the session authenticator, so the two models compose.

Proxy integration

The authorize check runs right after successful authentication; the response observer wraps the body in the ModifyResponse hook, captures a bounded tail as bytes stream through, and reports usage exactly once on EOF or early close, off the request path. The copy loop itself is untouched, so SSE pass-through behavior (immediate flushing) does not change.

While here, two small fixes the work surfaced: request serialization for the pricer now uses httputil.DumpRequest, which restores the body it reads (the old r.Write consumed it outright, so a zero-price dynamic response forwarded a bodyless request to the backend), and the freebie path now sends the dynamically fetched price into the challenge instead of the static service price.

meterd: a reference metering pricer

meterd (new cmd/meterd) is a small daemon implementing the full service: it quotes token bundles at a blended per-token msat rate from a per-model price table, books balances on ChallengeMinted, draws them down from the usage objects it extracts out of SSE or JSON response tails (front-truncation safe), and denies AuthorizeRequest with a re-quote once a bundle is spent. State persists as JSON across restarts. It doubles as the executable spec for what a metering price server must implement.

The full cycle has been exercised end to end on regtest against a real LND backend: 402 quoted and booked, invoice paid over a channel, paid retry authorized and streamed, exact usage debited, token reused without re-payment, and a drained bundle answered with a fresh challenge.

See each commit message for a detailed description w.r.t the incremental changes.

claude · 2026-07-06T22:39:12Z

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

In this commit, we extend the Prices service beyond the single up-front GetPrice query so an external pricer can meter prepaid L402 tokens over their whole lifecycle. ChallengeMinted tells the pricer which token ID a fresh challenge was minted under and at what price, so a paid token can be matched to the balance it purchased. AuthorizeRequest is consulted on every authenticated request, letting the pricer decide when a token's balance is exhausted and a fresh 402 is due. ReportUsage delivers a capped tail of the completed response body (e.g. the trailing usage object of an SSE stream), which is where the actual cost of a metered call becomes known. The motivating use case is metered LLM inference: a client buys a bundle of tokens with one Lightning payment, draws it down across many streaming requests, and re-purchases via a new challenge once the pricer says the bundle is spent.

In this commit, we add the MeteredPricer interface: a Pricer that can also be consulted per authenticated request (AuthorizeRequest), be told about freshly minted challenges (ChallengeMinted), and receive response usage reports (ReportUsage). GRPCPricer implements it over the new pricesrpc methods. The service-level dynamicprice config grows a 'metered' flag to opt a service into this flow, plus 'usagetailbytes' to bound the captured response tail. While here, we switch request serialization to httputil.DumpRequest, which buffers and restores the body on the request. The old r.Write call consumed the body outright, so a dynamic pricer returning a zero price would let the request through to the backend with an empty body. The restore also matters for AuthorizeRequest, which runs on requests that are about to be forwarded.

In this commit, we wire the metered pricer into the proxy's request path. For a service with dynamicprice.metered set, every authenticated request is first authorized against the pricer: if the token's prepaid balance is exhausted, the proxy answers with a fresh 402 challenge (priced by the pricer, falling back to GetPrice) instead of forwarding the request. Before any challenge for a metered service goes out, the pricer is notified of the minted token ID via ChallengeMinted; if that notification fails we abort the challenge, since the client would otherwise pay for a bundle the pricer will not honor. On the response side, the ModifyResponse hook wraps the body of an authorized metered request in an observer that captures a bounded tail of the bytes streaming through and reports the usage (status, content type, tail, completion flag) to the pricer exactly once, when the body hits EOF or is closed early by a disconnect. The copy loop itself is untouched, so SSE pass-through behavior does not change. Requests authenticated through non-L402 schemes (e.g. MPP sessions) skip pricer metering, as their draw-down is accounted for by the session authenticator itself. While here, we fix the freebie path to send the dynamically fetched price into handlePaymentRequired instead of the static service price it previously used.

In this commit, we add meterd, a reference implementation of the pricesrpc.Prices service that aperture's dynamicprice.metered mode talks to. The daemon sells prepaid bundles of LLM tokens over L402: a client pays a single Lightning invoice for a bundle of N tokens (1,000,000 by default), draws it down across requests, and receives a fresh 402 challenge once the bundle is spent. A bundle is priced at the model's blended per-token rate, the average of the configured input and output rates: price_sats = ceil(bundletokens * (inputmsat + outputmsat) / 2 / 1000) GetPrice resolves the model from the JSON body of the serialized HTTP request, falling back to a configurable default model. ChallengeMinted books a bundle for the minted token ID, idempotently. AuthorizeRequest allows requests while tokens remain and otherwise denies them with the quote for the next bundle attached, so aperture can mint the follow-up challenge without another GetPrice round trip. ReportUsage extracts the final usage object from the captured response tail, scanning SSE data lines for streamed responses and brace-matching the last "usage" object for plain (possibly front-truncated) JSON bodies, then debits the bundle: exact per-direction pricing when the prompt/completion split is known, the blended rate otherwise. Debits round up and remaining balances round down, so rounding never favors the client. Balances live in a mutex-guarded in-memory store that can persist to a JSON state file on every change, and a janitor expires bundles that were never used by an authorized request within 24 hours. Configuration comes from a YAML file plus flag overrides, with optional TLS; the server speaks plaintext when no certificate is configured, matching aperture's dynamicprice.insecure mode. A thin main lives at cmd/meterd.

Roasbeef · 2026-07-07T00:49:32Z

Addressed the review findings and force-pushed (branch inference-pricing, 4 commits, autosquashed). go build/go vet/gofmt clean; meterd, pricer, and the metering proxy tests all pass.

Metering bypasses closed (must-fix):

gzip evasion → no more free inference. Metered requests now strip the client's Accept-Encoding, so the upstream response comes back plaintext (Go's transport transparently decompresses) and the usage tail is always parseable. As defense-in-depth, Content-Encoding now rides the ReportUsage proto so a non-identity encoding is a flagged error rather than a silent zero-debit. New test: a request with Accept-Encoding: gzip still yields a correct debit.
Unauthenticated challenge spam. Bundle-state persistence is now coalesced (dirty flag + periodic flush + shutdown drain) instead of rewriting the whole state file per booking, and never-authorized bundles are capped with oldest-first eviction, so mint-spam can't grow memory unbounded or amplify disk I/O.

Correctness / accounting (should-fix):

TOCTOU overdraw. The store now reserves an estimated per-request cost at AuthorizeRequest (netted out of the available balance), so N concurrent requests on a near-empty bundle can't all authorize; the reservation is released on ReportUsage.
Model substitution. AuthorizeRequest now denies when the request's model differs from the bundle's booked model, quoting a fresh challenge for the model actually requested — a cheap-model bundle can't be spent on an expensive model.
Report-loss. ReportUsage now retries with exponential backoff before giving up, and a terminal failure logs loudly (a durable un-acked-report queue is noted as the real follow-up).
Hot-path body buffering. The pricer RPCs now serialize only a bounded 64 KiB prefix of the request body (enough for the model field), restoring the full body for the backend — no more full-body copy on every authenticated request or 4 MiB gRPC-limit 500s on large uploads.
Config validation. metered now requires enabled (startup error), UsageTailBytes is bounded, and the msat price arithmetic has an overflow guard.

Nits: a metered request whose L402 header is present-but-unparseable now returns 500 instead of free unmetered access; the usage brace-matcher requires "usage" in JSON-key position so an incidental substring can't latch a wrong object.

Note on CI vs local: the pre-existing TestProxyHTTP/TestProxyGRPC/TestProxyMPP tests bind fixed ports and do a gRPC DNS TXT lookup; in my sandbox they fail identically on clean master (502 / DNS hang), unrelated to this change. The metering-specific tests and the changed-logic packages (meterd, pricer) all pass.

claude · 2026-07-07T00:59:57Z

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

In this commit, we open up meterd so a downstream daemon can embed the metering logic as a library rather than forking the binary. The bundle store moves behind an exported Store interface (with the existing JSON file store renamed to JSONStore as its default implementation), and the static models map moves behind a RateSource interface that owns both model resolution and the per-model bundle sizing. NewServer grows functional options (WithStore, WithRateSource) so an embedder can swap in a database-backed store or a dynamically refreshed price table while reusing the booking, reservation, and draw-down logic unchanged. Existing call sites keep working: without options, NewServer builds the same JSON store and static rate table as before.

Roasbeef · 2026-07-07T01:37:36Z

Pushed one more commit, da3b973: meterd now exposes a Store interface (the JSON file store becomes JSONStore, its default implementation) and a RateSource interface owning model resolution plus per-model bundle sizing, with WithStore/WithRateSource options on NewServer. No behavior change without options; existing call sites are untouched. The motivation is letting a downstream daemon embed the metering logic as a library with a database-backed store and a dynamically refreshed price table, rather than forking the binary.

In this commit, we regenerate the pricesrpc gateway and message stubs through the repo's dockerized gen_protos_docker.sh, using the protoc-gen-go and grpc-gateway versions pinned in go.mod. The committed prices.pb.go and prices.pb.gw.go had drifted from the pinned toolchain versions, so CI's rpc-check job, which regenerates the stubs from scratch and diffs against the committed files, was failing. Regenerating locally with the same docker image CI uses produces a byte-identical diff to the one CI reports, so this brings the committed files back in sync with go.mod without hand editing any generated code.

…tine In this commit, we fix a data race between the detached usage-report goroutine spawned by usageObservingBody.report and the tests that exercise it. reportUsageWithRetry read the package-level reportInitialBackoff variable, and TestReportUsageWithRetry mutated that same variable to shrink the backoff for its own run. Because the report goroutine is detached from the request that spawned it, a goroutine left over from an earlier test (TestUsageObservingBody or one of the checkMeteredAccess tests) could still be reading the global while the next test wrote to it, tripping the race detector under unit-race and unit-race dbbackend=postgres. We replace the two package-level knobs, reportMaxAttempts and reportInitialBackoff, with a small reportSchedule struct carrying both the attempt count and the initial backoff. usageObservingBody now carries its own schedule, defaulting to defaultReportSchedule in production, and reportUsageWithRetry takes the schedule as a parameter instead of reading globals. Tests that need a fast schedule build their own reportSchedule value and pass it in, rather than mutating shared state that a leftover goroutine from a previous test might still be reading.

Roasbeef · 2026-07-07T01:54:14Z

Pushed two fixes for the CI failures on this branch.

The RPC check job was failing because the committed pricesrpc/prices.pb.go and prices.pb.gw.go had drifted from the protoc-gen-go and grpc-gateway versions pinned in go.mod. Regenerating locally through pricesrpc/gen_protos_docker.sh (same dockerized toolchain CI uses) produced a diff that matches CI's own reported diff line for line. Committed the regenerated stubs as is, no hand edits.

The unit-race failures (TestCheckMeteredAccess, TestReportUsageWithRetry, TestUsageObservingBody) were a real data race. usageObservingBody.report spawns a detached goroutine (reportUsageWithRetry) that read the package-level reportInitialBackoff variable, while TestReportUsageWithRetry mutated that same variable to shrink its backoff. Since the report goroutine outlives the request and the test that triggered it, a goroutine left over from an earlier test could still be reading the global while the next test wrote to it. Fixed by replacing the two package-level knobs with a reportSchedule struct that's passed explicitly into reportUsageWithRetry, defaulting to defaultReportSchedule in production. Tests now build their own schedule and pass it in rather than mutating shared state.

Verified proxy, meterd, and pricer with -race locally (bounded runs, not the full proxy suite since some of those tests are known to fail locally for environmental reasons unrelated to this PR).

In this commit, we close the loop between an authorization's reservation and the usage report that should release it. The proto had no way to correlate the two, so the price server could only release an approximate default estimate on every report; mismatched estimates accumulated as reservation residue until a bundle with real balance read as exhausted. AuthorizeRequestResponse gains a reserved_estimate field carrying the per-request token estimate the price server reserved, aperture threads it through the metering context into the usage observer, and ReportUsageRequest echoes it back verbatim. A zero value falls back to the old approximate behavior, so older pricers and proxies interoperate unchanged.

In this commit, we make the draw-down accounting exact on both sides of a usage report. The bundle is priced at the blended (input+output)/2 rate, but the buyer chooses the prompt/completion mix, so a raw token debit lets an output-heavy workload consume more msat value than it pays for. When the usage split is known, the debit is now weighted by direction, charging a completion token as outRate/blend bundle tokens, with the division rounding up so it never favors the client. The blended margin is then pure margin rather than partial insurance against an adversarial mix. On the release side, the authorization response now carries the reserved estimate and the report echoes it back, so the exact reservation is released instead of the configured default. Mismatched estimates previously accumulated as residue until a bundle with real balance read as exhausted; reports without an echo keep the old approximate release.

In this commit, we stop unknown models from riding another model's rates. Model resolution used to fall back to the default model for any identifier it did not recognize, which quietly defeated the authorize path's mismatch guard: a client holding a cheap-model bundle could name an upstream-hosted model the seller never priced, resolve into the booked default, and have aperture proxy the expensive model upstream while the debit ran at the cheap rate. Resolution now only falls back when the request names no model at all. A non-empty unknown model fails GetPrice and ChallengeMinted with InvalidArgument, and the authorize path denies it outright, so a model without a configured price is neither quoted nor served. The RateSource contract documents this as a requirement for embedders.

In this commit, we close the window in which a freshly paid bundle could be evicted as mint spam. Payment happens at the proxy, so the store cannot tell a just-paid bundle from an un-paid one until its first authorized request arrives; the count-based eviction treated both alike and a burst of unauthenticated challenge mints could push a just-paid bundle out before the client's retry landed. The store gains an eviction age floor: never-authorized bundles younger than it still count against the cap but are not candidates for eviction. The server wires the floor to the janitor's TTL, so within the window a paid bundle is safe and beyond it the janitor reaps un-paid bundles anyway. The worst-case state retained under sustained mint spam is the mint rate times the TTL, with aperture's rate limits as the front line.

Roasbeef · 2026-07-07T03:11:18Z

Four follow-up commits on top, tightening the metering accounting after another review pass:

109de2a adds a reserved_estimate field to the authorize response and usage report, so the price server releases the exact reservation an authorization took instead of an approximate default. Without the correlation, mismatched estimates accumulate as residue until a paid bundle reads as exhausted with balance left. Zero-valued echoes keep old proxies and pricers interoperating.
b7da329 weights debits by direction: bundles are priced at the blended rate but the buyer picks the prompt/completion mix, so raw token debits let an output-heavy workload consume more msat value than it pays for. The debit now charges tokens at their directional msat value against the blended rate, rounding up.
10394b9 makes model resolution fail closed: a request naming a model the rate source does not know is denied rather than falling back to the default model, which previously let an unpriced upstream model ride a cheap bundle through the mismatch guard.
a37c051 adds an eviction age floor so a mint-spam burst cannot evict a just-paid bundle in the window before its first authorized request.

In this commit, we appease the prealloc linter: the eviction scan now allocates its candidate slice with the bundle map's capacity up front instead of growing it per append.

In this commit, we teach the service header config to expand ${NAME} environment references at startup. The existing !file+hex and !file+base64 directives cover file-borne credentials like macaroons, but an upstream bearer token (e.g. a model API key) had to be pasted into the config file verbatim. With this change, the config can carry "Bearer ${UPSTREAM_KEY}" instead, keeping the secret out of the file and in the process environment. A reference to an unset or empty variable fails startup rather than substituting an empty string, so a missing secret can't silently turn into a malformed header upstream. Only the braced form is recognized, leaving literal dollar signs in header values untouched.

In this commit, we fix the director to replace a client-supplied header with the configured service value instead of appending after it. The comment always said overwrite, but the code used Header.Add, so a request's own Authorization (the L402 header itself) rode ahead of a configured upstream credential in the header list. Backends that read only the first value then rejected the request; Baseten's model API 403s on exactly this. Caught by the first live run against a real upstream: the L402 leg paid and settled fine, and the paid retry bounced off the backend with a bad-api-key error until the credential actually replaced the L402 header.

Roasbeef added 4 commits July 6, 2026 17:48

Roasbeef force-pushed the inference-pricing branch from fd347a2 to 67cdebe Compare July 7, 2026 00:49

Roasbeef added 2 commits July 6, 2026 18:53

Roasbeef added 4 commits July 6, 2026 20:07

Roasbeef added 3 commits July 6, 2026 20:17

meterd: pre-allocate the eviction candidate slice

ba946a3

In this commit, we appease the prealloc linter: the eviction scan now allocates its candidate slice with the bundle map's capacity up front instead of growing it per append.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

pricesrpc+proxy: metered draw-down of prepaid L402 tokens, with a reference pricer daemon#247

pricesrpc+proxy: metered draw-down of prepaid L402 tokens, with a reference pricer daemon#247
Roasbeef wants to merge 14 commits into
masterfrom
inference-pricing

Roasbeef commented Jul 6, 2026

Uh oh!

claude Bot commented Jul 6, 2026

Uh oh!

Roasbeef commented Jul 7, 2026

Uh oh!

claude Bot commented Jul 7, 2026

Uh oh!

Roasbeef commented Jul 7, 2026

Uh oh!

Roasbeef commented Jul 7, 2026

Uh oh!

Roasbeef commented Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

Roasbeef commented Jul 6, 2026

The new pricesrpc surface

Proxy integration

meterd: a reference metering pricer

Uh oh!

claude Bot commented Jul 6, 2026

Code review

Uh oh!

Roasbeef commented Jul 7, 2026

Uh oh!

claude Bot commented Jul 7, 2026

Code review

Uh oh!

Roasbeef commented Jul 7, 2026

Uh oh!

Roasbeef commented Jul 7, 2026

Uh oh!

Roasbeef commented Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant