Byte-verify MPMA batches mixing a hex memo with memo-less sends - #225
Merged
Conversation
The packer rejected any memos_are_hex list containing more than one distinct value, but composeMPMA filters that list to entries whose memo is non-empty before choosing the single flag it sends. The MPMA form emits one flag per row and a row with no memo emits false, so a batch with one hex memo and one blank memo arrived as "true,false": compose correctly sent memos_are_hex=true while the packer saw two values and declined. Byte verification silently dropped to field comparison for what is a perfectly ordinary CSV -- only some rows carrying memos. It failed safe, never passing a substitution, but the oracles could not catch it because every oracle case uses uniform flags. Only the flags belonging to an actual memo describe the encoding, so the packer now filters the same way composeMPMA does. A batch with two real memos of different encodings is still declined, since core's single flag cannot express it. The round-trip oracle also needed a fix. Its "nothing could be rebuilt" guard exists so a type cannot pass vacuously, but it cannot tell an unrepresentative sample from a broken packer, and real traffic is bursty: every one of the 100 most recent issuances was a locked subasset, which the borrow guard declines by design, so the issuance case failed on healthy code. Principled declines are now counted, and when nothing was compared the assertion is that every sample was one of them -- a sample that was attempted and still could not be rebuilt still fails. The window is also wider, which makes the situation rarer. Claude-Session: https://claude.ai/code/session_01CcjnCrgosSeshymXLBxdGj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PRIORITIES.md item 4b — a coverage gap I introduced in #216 and found reviewing my own work, plus an oracle fix it surfaced.
The gap
The packer rejected any
memos_are_hexlist containing more than one distinct value:But
composeMPMAfilters that list to entries whose memo is non-empty before choosing the single flag it sends. The MPMA form emits one flag per row, and a row with no memo emitsfalse— so a batch with one hex memo and one blank memo arrives as"true,false". Compose correctly sendsmemos_are_hex=true; the packer saw two distinct values and declined.Result: byte verification silently dropped to field comparison for a perfectly ordinary CSV — one where only some rows carry memos. It failed safe (never passing a substitution), but the oracles couldn't catch it because every oracle case uses uniform flags.
Only the flags belonging to an actual memo describe the encoding, so the packer now filters the same way
composeMPMAdoes. A batch with two real memos of different encodings is still declined, since core's single flag can't express it — both cases are tested, with the passing one pinned to bytes generated by core's own encoder.The oracle fix this surfaced
Running the live oracles afterward, the issuance round-trip case failed with "every sample was skipped". That turned out not to be a regression: all five samples were locked subasset issuances, which #219's borrow guard declines by design. Checking further, all 100 most recent on-chain issuances were locked subassets — one prolific minter had filled the entire window.
The guard exists so a type can't pass vacuously, which is right, but it couldn't distinguish an unrepresentative sample from a broken packer. Now:
SAMPLE_SIZEis wider (5 → 25), making the situation rarer.Verification
tsc --noEmitclean; 693 counterparty tests pass with both oracles live againstapi.counterparty.io(74 pack tests including the previously failing issuance case).compose/send/mpma.spec.ts6/6.https://claude.ai/code/session_01CcjnCrgosSeshymXLBxdGj