Skip to content

Polish bundle: file locking, sort-on-flush, RLE for kind#19

Merged
pbudzik merged 1 commit into
mainfrom
feat/polish-bundle-2
May 16, 2026
Merged

Polish bundle: file locking, sort-on-flush, RLE for kind#19
pbudzik merged 1 commit into
mainfrom
feat/polish-bundle-2

Conversation

@pbudzik

@pbudzik pbudzik commented May 16, 2026

Copy link
Copy Markdown
Owner

Summary

Three operational polish items bundled in one PR — none individually big enough for its own. Each tightens an existing path without changing public semantics.

1. File locking (DbLock)

New module src/runtime/lock.rs holds an exclusive flock(2) on db_root/LOCK. The server (run_server) and every admin command acquire it at startup; the OS releases it when the process exits. Prevents server + admin (or two servers) from racing on the same manifest.

API change: open_state_for_admin now returns (AppState, DbLock); callers destructure the tuple and hold the lock guard for the operation's lifetime.

2. Sort-on-flush

FlusherWorker::write_bucket sorts each per-bucket batch by (account, product, meter, model, ts) — the spec's canonical billing order — before writing. Saves a sort pass on every later compaction and improves dictionary-encoding locality. The sort algorithm is exposed as pub fn sort_events_canonical so tests verify it directly without driving the async flusher.

3. RLE for kind

Encoding::Rle = 4 added (the format reserved discriminants for this). The kind column has only 3 values with usually long runs of Usage. RLE collapses those to single (value, run_length) pairs, then zstd handles the rest. For a 1000-row all-Usage segment, the kind column drops from 1000 bytes pre-zstd to ~10 bytes.

Reader is permissive: accepts Plain or Rle for the kind column, so segments written before this PR still load.

Dep

fs4 = "0.13" for cross-platform flock.

Tests

8 new tests in tests/polish.rs:

  • db_lock_prevents_concurrent_acquisition + re-acquire after drop
  • open_state_for_admin_blocks_second_caller
  • sort_events_canonical_orders_by_billing_dimensions
  • sort_events_canonical_breaks_ties_by_meter_then_model
  • writer_round_trip_preserves_sorted_order
  • rle_kind_round_trips_all_usage
  • rle_kind_round_trips_all_corrections
  • rle_encoding_round_trips_mixed_kinds (Usage + Correction + Retraction interleaved)

Test plan

  • cargo build --all-targets clean with -D warnings
  • cargo test --all-targets — 114 tests pass (was 106; +8)
  • CI green

🤖 Generated with Claude Code

Four operational polish items in one PR — none individually big enough
for their own.

1. File locking (DbLock). New module src/runtime/lock.rs holds an
   exclusive flock on db_root/LOCK. Server (run_server) and every
   admin command acquire it for their full lifetime; OS releases it
   when the process exits. Prevents server + admin (or two servers)
   from racing on the same manifest.
   open_state_for_admin now returns (AppState, DbLock); existing
   tests destructured the tuple.

2. Sort-on-flush. FlusherWorker.write_bucket sorts the per-bucket
   batch by (account, product, meter, model, ts) — the spec's
   canonical billing order — before writing. Saves a sort pass on
   every later compaction and improves dictionary-encoding locality
   on the ID columns. The algorithm is exposed as
   `pub fn sort_events_canonical` so tests can verify it directly
   without driving the async flusher.

3. RLE for `kind`. Encoding::Rle = 4 added. The kind column has 3
   values with usually long runs of `Usage`; RLE collapses those to
   single (value, run_length) pairs (then zstd over that). For a
   1000-row all-Usage segment, the kind column goes from 1000 bytes
   pre-zstd to ~10 bytes. Reader is permissive (accepts Plain or Rle).

Tests (tests/polish.rs, 8 tests):
  - db_lock_prevents_concurrent_acquisition + re-acquire after drop
  - open_state_for_admin_blocks_second_caller
  - sort_events_canonical_orders_by_billing_dimensions
  - sort_events_canonical_breaks_ties_by_meter_then_model
  - writer_round_trip_preserves_sorted_order
  - rle_kind_round_trips_all_usage
  - rle_kind_round_trips_all_corrections
  - rle_encoding_round_trips_mixed_kinds (Usage + Correction +
    Retraction interleaved)

Dep: fs4 = "0.13" for cross-platform flock.

Total tests: 114 (was 106; +8). Clean under RUSTFLAGS=-D warnings.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@pbudzik pbudzik merged commit 88b6e5c into main May 16, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant