Cutover boundary: 3.6 MB setting re-parsed and linearly scanned per row, though the value is immutable

[TaprootFreak]

isCoveredByCutoverOpening reloads and re-parses a 3.6 MB setting on every row it checks, then scans a 469,111-element array linearly — for a value the code itself documents as immutable.

The value

setting key                                        holeIds    raw size
ledgerCutoverBoundary.trading_order                469,111    3,665 kB
ledgerCutoverBoundary.liquidity_order                7,409       51 kB
ledgerCutoverBoundary.liquidity_management_order     1,747       12 kB
ledgerCutoverBoundary.crypto_input                   1,604       11 kB
ledgerCutoverBoundary.exchange_tx                       31      250 B

All five were pinned on 16-07 and have not changed since — as intended: ledger-watermark.helper.ts states the boundary is "pinned ONCE … set-only-if-unset … a fail-loud retry reuses the pinned value verbatim and never overwrites it", and that unlike the watermark, "this copy does NOT [drift]".

The read path

src/subdomains/core/accounting/services/consumers/ledger-watermark.helper.ts:

export async function isCoveredByCutoverOpening(settingService, source, rowId) {
  const raw = await settingService.getObj<{ boundaryId: number; holeIds: number[] }>(...);
  if (!raw) return false;
  if (rowId > raw.boundaryId) return false;
  return !raw.holeIds.includes(rowId);
}

Two things per call:

1. getObj is the uncached variant — it goes through settingRepo.findOneBy(). SettingService also offers getObjCached (backed by findOneCachedBy), which is not used here. So the 3.6 MB is fetched and JSON.parsed on every call.
2. holeIds.includes(rowId) is a linear scan over 469,111 numbers.

It is called per row, inside the processing loop of five consumers — trading-order.consumer.ts:76, crypto-input.consumer.ts:76, exchange-tx.consumer.ts:141, liquidity-mgmt.consumer.ts:82, liquidity-order-dex.consumer.ts:93.

Volume

Rows created in the last 24 h, i.e. roughly the number of calls per source:

trading_order                4,036     ← the 3.6 MB entry
liquidity_order                111
exchange_tx                     66
liquidity_management_order      48
crypto_input                    27

The trading_order path alone therefore parses about 14 GB of JSON per day (4,036 × 3.6 MB) and performs roughly 1.9 billion array comparisons, to answer a yes/no question about a value that never changes. JSON.parse is synchronous, so this lands directly on the event loop.

Suggested fix

1. Use getObjCached. The value is immutable by design, so the cache cannot serve a stale answer that matters. This removes the fetch and the parse from all but the first call per cache period.
2. Hold holeIds as a Set. includes() is O(n) per row; a Set lookup is O(1). Worth doing together with (1), since the parsed array is then reused rather than rebuilt.

Both are local to the helper and do not change observable behaviour — the same rows are covered or not covered as before.

Not verified

The call counts above are derived from rows created per day, not instrumented directly; a backlog run would multiply them. The event-loop cost of a 3.6 MB JSON.parse was not measured in isolation on this host.