feat(mps): add save_mps / load_mps serialization primitive - #468
Merged
Conversation
Add a lossless, deterministic single-stream codec for the current Mps<St, L> type (Dense + BlockSparse over the built-in sectors), so an iterative MPS algorithm restarts by loading a saved state and feeding it back in — the reusable primitive is MPS serialization, not a DMRG-coupled restart mode. The format is a self-describing CBOR manifest plus an explicit little-endian numeric body: scalar / storage / sector type-identity tags are checked before any value is decoded, and NaN / signed-zero / infinity round-trip bit-exactly. Decode never panics on crafted input — a fallible BlockSparseLayout::try_new mirrors the panicking new with checked arithmetic, and every malformed descriptor maps to a typed MpsIoError. ariadnetor-tensor gains the per-tensor codec, the sealed SerializableSector capability, and the metadata DTOs; ariadnetor-mps gains the MpsCodec dispatch, the container framing, save/load plus atomic path wrappers, and MpsIoError. Closes #465
Loading rejected only versions above the supported maximum, so a stream declaring version 0 was decoded against the version-1 schema; enforce a minimum version too. save_mps_to_path used File::create for the temp file, which follows a symlink and truncates its target — a pre-planted symlink at the guessable temp name in a writable directory could redirect the write onto a victim file. Create the temp file with create_new (O_EXCL), which refuses an existing path, retrying a bounded number of times.
BlockSparseLayout::try_new enumerates the full Cartesian product of the per-leg block counts. A crafted many-legged descriptor (e.g. 64 two-block legs) could force astronomically many iterations from a compact, empty-data stream, hanging the loader or exhausting memory. Cap the candidate-coordinate count before enumerating; over-limit descriptors return TooManyBlocks.
The candidate-count cap bounded enumeration time but not memory: a compact descriptor could still enumerate millions of allowed blocks — each a BlockMeta with a rank-sized coordinate plus a hash-map entry — into gigabytes before the empty numeric body was ever checked. Pass the body-derived element budget into try_new and stop enumerating once the blocks would exceed it, so decode memory is bounded by the data actually supplied. A too-short body now fails fast with ExtentBudgetExceeded.
There was a problem hiding this comment.
Pull request overview
This PR introduces a deterministic, lossless serialization primitive for Mps<St, L> to support warm-restart workflows (e.g., restarting iterative algorithms by reloading a saved MPS). It implements a CBOR-based self-describing manifest plus an explicit little-endian numeric body, with typed error handling and decode-time validation intended to avoid panics on malformed input.
Changes:
- Add per-tensor encode/decode primitives in
ariadnetor-tensor(scalar/sector codecs, DTO metadata, and a panic-freeBlockSparseLayout::try_newpath for decode). - Add chain-level MPS container format and public
save_mps/load_mps(+ atomic path wrappers) inariadnetor-mps, includingMpsIoError. - Add comprehensive roundtrip/determinism and malformed-input tests for both tensor- and chain-level codecs, plus workspace dependency wiring for
serdeandciborium.
Reviewed changes
Copilot reviewed 21 out of 22 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| crates/ariadnetor-tensor/src/serialize/mod.rs | Introduces the tensor-level serialization module and re-exports codec/DTO surface. |
| crates/ariadnetor-tensor/src/serialize/codec.rs | Implements dense + block-sparse per-tensor encode/decode with checked arithmetic and typed errors. |
| crates/ariadnetor-tensor/src/serialize/meta.rs | Adds CBOR-serialized metadata DTOs (BodyMeta, QnIndexDto, etc.). |
| crates/ariadnetor-tensor/src/serialize/scalar.rs | Adds explicit little-endian scalar codec + scalar type tag. |
| crates/ariadnetor-tensor/src/serialize/sector.rs | Adds sealed SerializableSector, sector type tags, and raw sector value codec. |
| crates/ariadnetor-tensor/src/serialize/tests.rs | Adds tensor-level roundtrip and malformed-input tests. |
| crates/ariadnetor-tensor/src/block_sparse/layout.rs | Adds BlockLayoutError and panic-free BlockSparseLayout::try_new for untrusted decode. |
| crates/ariadnetor-tensor/src/block_sparse/mod.rs | Re-exports BlockLayoutError. |
| crates/ariadnetor-tensor/src/lib.rs | Wires in the new serialize module and re-exports serialization primitives from ariadnetor-tensor. |
| crates/ariadnetor-tensor/Cargo.toml | Adds serde + ciborium dependencies for tensor-level metadata encoding. |
| crates/ariadnetor-tensor/allowed-external-types.toml | Allows serde traits to appear in the mid-layer public API surface. |
| crates/ariadnetor-mps/src/serialize/mod.rs | Adds chain-level serialization module and re-exports public entry points. |
| crates/ariadnetor-mps/src/serialize/manifest.rs | Defines MpsManifest and SiteMeta (order + tensor body meta + data length). |
| crates/ariadnetor-mps/src/serialize/container.rs | Implements stream framing, save_mps/load_mps, and atomic path save/load wrappers. |
| crates/ariadnetor-mps/src/serialize/codec.rs | Implements sealed MpsCodec dispatch over supported storage/layout pairs. |
| crates/ariadnetor-mps/src/serialize/error.rs | Introduces MpsIoError and maps tensor decode errors into chain-level errors. |
| crates/ariadnetor-mps/src/serialize/tests.rs | Adds chain-level roundtrip/determinism and extensive malformed-input tests. |
| crates/ariadnetor-mps/src/types.rs | Adds serde derives for CanonicalForm to enable manifest roundtrip. |
| crates/ariadnetor-mps/src/lib.rs | Re-exports the new MPS serialization API from the crate root. |
| crates/ariadnetor-mps/Cargo.toml | Adds serde + ciborium dependencies for chain-level metadata encoding. |
| Cargo.toml | Adds workspace dependencies for serde and ciborium. |
| Cargo.lock | Records new dependency resolution for serde/ciborium usage. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a lossless, deterministic serialization primitive for the current
Mps<St, L>type, so an iterative MPS algorithm (starting with 2-site DMRG) can restart by loading a saved state and feeding it back in. Restart needs no bespoke mode: the MPS captures the full variational state, and the bra/ket environments are recomputed on re-entry, so the reusable primitive is MPS serialization rather than a DMRG-coupled feature.Closes #465
Format
A single stream:
[magic] [u64 manifest length] [CBOR manifest] [numeric data section]. Metadata travels as a self-describing CBOR manifest, robust to later struct evolution; numeric tensor bodies are explicit little-endian scalar bytes, so the complex representation stays layout-explicit and signed zero, infinities, and distinct NaN payloads round-trip bit-exactly. The manifest's scalar / storage / sector type-identity tags are checked before any value is decoded, so loading a file as the wrong type fails cleanly — including a U(1) file opened as Z2, whose stored values are otherwise byte-compatible.Changes
ariadnetor-tensor: the sealedSerializableSectorcapability (type tag, panic-freechecked_fuse/checked_dual, raw value codec), a fallibleBlockSparseLayout::try_newcoexisting with the unchangednew, theScalarCodeclittle-endian codec, the metadata DTOs, and the per-tensor encode / decode.ariadnetor-mps: theMpsCodecstorage-keyed dispatch, theMpsManifestframing, the single-stream container,save_mps/load_mpsplus atomic-visibility path wrappers, and the typedMpsIoError.CanonicalFormgainsserdederives, round-tripped verbatim.Decode safety
Load never panics on crafted input: every descriptor is validated and all extent arithmetic is checked before the panicking reconstruction constructors run, and block enumeration is bounded in both iteration count and memory (the numeric body caps the block table), so a compact descriptor cannot hang the loader or exhaust memory.
save_mps_to_pathcreates its temp file withO_EXCL(no symlink following), fsyncs it, and atomically renames into place, so a reader sees either the old file or the complete new one.Test plan
{f32, f64, Complex<f32>, Complex<f64>}and BlockSparse over{U1, Z2, tuple}paired with{f64, Complex<f64>}, both memory orders, single- and multi-site chains, everyCanonicalFormvariant.Notes
Scope is warm continuation from a saved state, not bit-exact replay of an interrupted run: run-state such as the energy baseline and sweep budget lives outside the MPS and resets on re-entry, and environments are recomputed rather than persisted. Also out of scope, deferred to a later per-algorithm checkpointing layer: persisting run-state and RNG state, a multi-file directory checkpoint format, serializing sector types defined outside this crate, and cross-version format migration. The primitive is designed to compose into such a layer — the per-tensor codec is reusable — and
save_mpo/load_mpois a near-free follow-on since the chain codec is rank-agnostic.@coderabbitai ignore