diff --git a/packages/kits/PersistenceKit/docs/AGENT_MAP.md b/packages/kits/PersistenceKit/docs/AGENT_MAP.md index 28dc85b..ef748b6 100644 --- a/packages/kits/PersistenceKit/docs/AGENT_MAP.md +++ b/packages/kits/PersistenceKit/docs/AGENT_MAP.md @@ -76,7 +76,7 @@ sources: - path: Sources/PersistenceKitPostgreSQL/PostgreSQLIdentifierValidator.swift blob: 120cf0c1576a7db45d79bf6865e2f15cff09a5ac - path: Sources/PersistenceKitPostgreSQL/PostgreSQLPool.swift - blob: eddc0c3af4135565e15d2d763b0d24240973dd6f + blob: 98dc0350228421361e43d1a363cb81989f790c91 - path: Sources/PersistenceKitPostgreSQL/PostgreSQLPredicateCompiler.swift blob: 6f0791e0372b09cf2605bae4cf149e48bbba2834 - path: Sources/PersistenceKitPostgreSQL/PostgreSQLSchema.swift @@ -94,7 +94,7 @@ sources: - path: Sources/PersistenceKitSQLite/KeychainKeyStore.swift blob: 0071732291a7cb6ce0777bd230a6188276fb4f32 - path: Sources/PersistenceKitSQLite/SQLiteConnection.swift - blob: ece56dc7e25e67656bb37f5222c18c1166c750cc + blob: d1e0aee8f4dceb815b24c3aac619ad71b4edcae9 - path: Sources/PersistenceKitSQLite/SQLiteIdentifierValidator.swift blob: 713339c137d6af1cfbba5a3584e05bdda70b42c5 - path: Sources/PersistenceKitSQLite/SQLiteObserver.swift @@ -104,7 +104,7 @@ sources: - path: Sources/PersistenceKitSQLite/SQLiteSchema.swift blob: 4ba6fc175fe9d486b17af1088a23da64041b12ae - path: Sources/PersistenceKitSQLite/SQLiteStorage.swift - blob: 417e63cc07b60295ac874187ebb796bf9f3b3c86 + blob: 2c73892fba5b5b4c6608dac070b1c2159f4538c6 - path: Sources/PersistenceKitSQLite/SQLiteStores.swift blob: 76499ffb70d979f0d13e8cf9e32bc38ff28ffdb5 --- @@ -283,14 +283,22 @@ ENTRY POINTS (most callers need only these): ### Backend: PersistenceKitSQLite - SQLiteStorage.swift:22 `final class SQLiteStorage: Storage, Sendable`: one connection per estate - SQLiteStorage.swift:110 `actor SQLiteBackend`: owns connection + inTransaction flag + pendingBlobNotifications (buffered during txn, SECFIX-WS2-PK F3) -- SQLiteStorage.swift:411 `insertRow` / :461 `upsertRow` / :495 `updateRows` / :535 `deleteRows` / :559 `queryRows`: ALL validate every table/column identifier via `validateSQLIdentifier` before interpolation (SECFIX-WS2-PK F9); insertRow+queryRows run RowCrypto encrypt/decrypt seam -- SQLiteStorage.swift:656 `queryRowsSkipCorrupt(...)`: cursor-level per-row skip+log on corruptStoredValue; other errors re-thrown (systemic) -- SQLiteStorage.swift:782 `storageStats(now:)`: PRAGMA page_size/page_count/freelist_count; WAL frame count derived from `-wal` FILE SIZE (NOT PRAGMA wal_checkpoint: can SQLITE_LOCKED even same-actor) -- SQLiteStorage.swift:917 `readColumn(...)`: type-tolerant decode (valid-but-coerced value passes through) vs. parse-failure (unparseable UUID/timestamp TEXT) → THROWS corruptStoredValue, never fabricates +- SQLiteStorage.swift:334 `runTransaction(isolation:_:)`: on `inTransaction==true`, WAITS (poll every 25ms, cap 60s) instead of throwing on sight, ordinary contention between two async callers suspended in `block`; wait-expired ⇒ transactionConflict; true self-reentrant nesting still fails this way (never hangs) +- SQLiteStorage.swift:396 `beginTransactionDirect()`: unchanged immediate-throw on `inTransaction==true` (synchronous call site, cannot await a wait) +- SQLiteStorage.swift:429 `insertRow` / :483 `upsertRow` / :521 `updateRows` / :565 `deleteRows` / :593 `queryRows`: ALL validate every table/column identifier via `validateSQLIdentifier` before interpolation (SECFIX-WS2-PK F9); insertRow+queryRows run RowCrypto encrypt/decrypt seam; ALL FIVE (+ queryRowsSkipCorrupt/fetchMatchingRowKeys/countRows/iterateAudit) now call `connection.prepareCached(_:)` not `connection.prepare(_:)` (see SQLiteConnection.swift statement cache below) +- SQLiteStorage.swift:694 `queryRowsSkipCorrupt(...)`: cursor-level per-row skip+log on corruptStoredValue; other errors re-thrown (systemic) +- SQLiteStorage.swift:824 `storageStats(now:)`: PRAGMA page_size/page_count/freelist_count; WAL frame count derived from `-wal` FILE SIZE (NOT PRAGMA wal_checkpoint: can SQLITE_LOCKED even same-actor) +- SQLiteStorage.swift:967 `readColumn(...)`: type-tolerant decode (valid-but-coerced value passes through) vs. parse-failure (unparseable UUID/timestamp TEXT) → THROWS corruptStoredValue, never fabricates - SQLiteConnection.swift:24 `final class SQLiteConnection: @unchecked Sendable`: thin sqlite3 C API wrapper - SQLiteConnection.swift:29 `init(url:busyTimeout:keyHex:)`: ORDER MATTERS: symlink-refusal check (CAND-052, lstat semantics) → sqlite3_open_v2 → PRAGMA key (Mode3, MUST be first stmt) → Apple Data Protection → WAL/synchronous/busy_timeout/foreign_keys pragmas -- SQLiteConnection.swift:162 `final class SQLiteStatement`: prepared-stmt wrapper; :180 `bind(_:at:)` TypedValue→sqlite3_bind_*; :239 `step()` -- SQLiteConnection.swift:318 `enum ISO8601`: :356 `string(from:)` clamps Date to RFC-3339 range [0001,9999] before formatting (logs warning on clamp, never emits unparseable-back string); :392 `date(from:)` tries :412 `fastParseCanonicalUTC` FIRST (allocation-free, handles the exact canonical shape this kit writes: Merkle rollup re-decode was ~80% of CPU via ICU formatter before this fast path), falls back to ISO8601DateFormatter for anything else +- SQLiteConnection.swift:102 `close()`: destroys every cached statement FIRST (loop over statementCache), THEN sqlite3_close_v2; sqlite3_close_v2 defers the real close while any prepared stmt survives, so skipping this order leaves a zombie connection +- SQLiteConnection.swift:156 `statementCache: [String: SQLiteStatement]` (:157 capacity 128): per-connection cache keyed by SQL text; at-capacity eviction destroys+rebuilds the WHOLE cache (crude, O(1) amortized); mirrors Rust rusqlite prepare_cached +- SQLiteConnection.swift:177 `prepareCached(_:)`: CHECKOUT semantics (mirrors rusqlite): statement REMOVED from cache slot while in use, returned by its own finalize(); a reentrant same-SQL call misses the now-empty slot and prepares fresh instead of clobbering the in-flight statement +- SQLiteConnection.swift:197 `returnToCache(_:)`: called from a cached statement's finalize(); if the slot is already occupied (reentrant duplicate), destroys the returner instead of clobbering the resident +- SQLiteConnection.swift:217 `final class SQLiteStatement`: prepared-stmt wrapper; :225 `isCached` / :228 `cacheKey` set by prepareCached, read by finalize(); :269 `bind(_:at:)` TypedValue→sqlite3_bind_*; :328 `step()` +- SQLiteConnection.swift:237 `finalize()`: on a cached stmt, RETURNS it to the cache reset-for-reuse rather than destroying it; existing `defer { stmt.finalize() }` call sites need no change whether the stmt came from prepare or prepareCached +- SQLiteConnection.swift:253 `resetForReuse()` (sqlite3_reset + sqlite3_clear_bindings, cache-reuse path) / :262 `destroy()` (unconditional sqlite3_finalize, used by cache eviction + close()) +- SQLiteConnection.swift:407 `enum ISO8601`: :445 `string(from:)` clamps Date to RFC-3339 range [0001,9999] before formatting (logs warning on clamp, never emits unparseable-back string); :481 `date(from:)` tries :501 `fastParseCanonicalUTC` FIRST (allocation-free, handles the exact canonical shape this kit writes: Merkle rollup re-decode was ~80% of CPU via ICU formatter before this fast path), falls back to ISO8601DateFormatter for anything else - SQLiteStores.swift:22 `final class SQLiteRowStore: RowStore, Sendable`: forwards to SQLiteBackend; :75 `querySkipCorrupt` overrides protocol default with real cursor-level skip - SQLiteStores.swift:90/102/119 `SQLiteBlobStore` / `SQLiteAuditLog` / `SQLiteTransaction`: thin forwards - SQLiteSchema.swift:9 `enum SQLiteSchema`: :12 `nativeType(_:)` ColumnType→SQLite storage class (.uuid/.timestamp→TEXT, .hlc→INTEGER packed, .fingerprint→BLOB); :29 `createTable(_:)`; :68 `appendOnlyTriggers(_:)` emits BEFORE UPDATE/DELETE RAISE(ABORT) trigger pair @@ -308,7 +316,9 @@ ENTRY POINTS (most callers need only these): - PostgreSQLStorage.swift:246 `storageStats(now:)`: pg_database_size / pg_stat_database (blks_hit,blks_read,xact_commit,xact_rollback,deadlocks) / pg_locks⋈pg_database (granted=false ⇒ lockContention) - PostgreSQLPool.swift:12 `actor PostgreSQLPool`: fixed-size; :38 `acquire()` reuse→open-new(if under size)→CheckedContinuation-wait w/ timeout→poolExhausted - PostgreSQLPool.swift:101 `openConnection()`: every new conn: CREATE SCHEMA IF NOT EXISTS + SET search_path, closes conn on setup failure (never hands back half-configured conn) -- PostgreSQLPool.swift:168 `parseTLSMode(host:)`: `ARIA_MCP_POSTGRES_TLS` env var: disable|require|(absent/unrecognized→prefer, incl. loopback: explicit opt-out required for plaintext) +- PostgreSQLPool.swift:178 `parseTLSMode(host:dsnSSLMode:)`: effective mode = MAX(DSN `sslmode=` via :219 `sslMode(from:)`, `ARIA_MCP_POSTGRES_TLS` env var) computed by :203 `effectiveTLSDecision(dsnSSLMode:envValue:)` (SECFIX-C-PG-SWIFT-TLS-SSLMODE, parity w/ Rust `postgres_tls::effective_sslmode`); env may RAISE never LOWER the DSN's requirement; absent+absent→prefer, incl. loopback (explicit opt-out required for plaintext) +- PostgreSQLPool.swift:197 `enum TLSDecision`: disable|prefer|require (the 3 PostgresNIO outcomes; extracted for unit-testability w/o a live server/NIOSSLContext) +- PostgreSQLPool.swift:231 `private enum TLSModeRank`: weakest→strongest ranking for libpq sslmode values: disable [(job: Job, sessionID: SessionID)]`: claim all available jobs -- QueueKit.swift:129 `reply(to:status:artifacts:) async throws`: mark one claimed job terminal -- QueueKit.swift:123 `watch(handler:) async throws`: subscribe a per-job handler +- QueueKit.swift:53 `QueueKit.init(root:hlcGenerator:) throws`: mount FilesystemBackend at root, create maildir, clean stale tmp/ +- QueueKit.swift:66 `QueueKit.init(backend:root:)`: mount explicit backend (PersistenceKitBackend, or a test double) +- QueueKit.swift:73 `send(_ job: Job) async throws`: enqueue one job +- QueueKit.swift:86 `drain() async throws -> [(job: Job, sessionID: SessionID)]`: claim all available jobs +- QueueKit.swift:130 `reply(to:status:artifacts:) async throws`: mark one claimed job terminal +- QueueKit.swift:124 `watch(handler:) async throws`: subscribe a per-job handler ## Symbol Table ### Facade: QueueKit.swift - :28 `staleTmpThreshold = 5 * 60`: tmp/ files older than this on init are deleted (crash between write and rename; never visible in new/, safe to remove) - :30 `final class QueueKit: Sendable` -- :39 `latencyWindow`: nonisolated(unsafe); caller (GLK scheduler actor / serial test) owns exclusivity -- :47 `estateTag: String = "unknown"`: nonisolated(unsafe); set ONCE at mount before any drain(), never again -- :52 `init(root:hlcGenerator:) throws`: see ENTRY POINTS -- :65 `init(backend:root:)`: see ENTRY POINTS -- :72 `send(_:)` / :81 `send(batch:) -> Int`: writeBatch twin; FS backend fsyncs new/ ONCE for the batch -- :85 `drain()` / :108 `drain(stream:)`: claim; wraps backend call with reportQueueStats timing -- :123 `watch(handler:)`: forwards to backend.watch -- :129 `reply(to:status:artifacts:)`: guards status.isTerminal, throws .invalidTerminalStatus otherwise -- :148 `reply(session:status:) -> Int`: fast path ONLY on PersistenceKitBackend (`as?` cast); returns 0 on FilesystemBackend (caller falls back to per-job reply) -- :171 `reply(batch:) -> Int`: routes to backend.completeBatch; FS backend's one-scan/one-fsync path -- :193 `reclaimInFlight(stream:) -> Int`: GATE: call ONLY immediately after DrainLease.tryAcquire SUCCEEDED for stream; fast path ONLY on PersistenceKitBackend, returns 0 otherwise -- :204 `inFlight()` / :214 `pendingCount()` / :224 `pendingCount(stream:)`: read-only depth probes; pendingCount()+inFlight().count = total outstanding work -- :261 `awaitDrain(pollInterval: .milliseconds(20), timeout: .seconds(30)) throws`: polls both frontiers; returns promptly if already empty; throws .drainTimeout, never hangs -- :292 `awaitDrain(stream:pollInterval:timeout:)`: stream-scoped twin; global awaitDrain would block forever on OTHER streams' jobs a scoped drainer never claims -- :314 `completed(streamID:)` -- :322 `maildirSubdirs = ["tmp","new","cur","done"]` -- :324 `ensureMaildir(root:) throws`: creates 4 subdirs if absent -- :341 `cleanStaleTmpFiles(root:) throws`: deletes tmp/ entries older than staleTmpThreshold +- :40 `latencyWindow: QueueLatencyWindowBox`: lock-backed, thread-safe; concurrent drain() calls (encode worker + import worker sharing an estate queue) both report through the same lock, never a raw var +- :48 `estateTag: String = "unknown"`: nonisolated(unsafe); set ONCE at mount before any drain(), never again +- :53 `init(root:hlcGenerator:) throws`: see ENTRY POINTS +- :66 `init(backend:root:)`: see ENTRY POINTS +- :73 `send(_:)` / :82 `send(batch:) -> Int`: writeBatch twin; FS backend fsyncs new/ ONCE for the batch +- :86 `drain()` / :109 `drain(stream:)`: claim; wraps backend call with reportQueueStats timing +- :124 `watch(handler:)`: forwards to backend.watch +- :130 `reply(to:status:artifacts:)`: guards status.isTerminal, throws .invalidTerminalStatus otherwise +- :149 `reply(session:status:) -> Int`: fast path ONLY on PersistenceKitBackend (`as?` cast); returns 0 on FilesystemBackend (caller falls back to per-job reply) +- :172 `reply(batch:) -> Int`: routes to backend.completeBatch; FS backend's one-scan/one-fsync path +- :194 `reclaimInFlight(stream:) -> Int`: GATE: call ONLY immediately after DrainLease.tryAcquire SUCCEEDED for stream; fast path ONLY on PersistenceKitBackend, returns 0 otherwise +- :205 `inFlight()` / :215 `pendingCount()` / :225 `pendingCount(stream:)`: read-only depth probes; pendingCount()+inFlight().count = total outstanding work +- :275 `awaitDrain(pollInterval: .milliseconds(20), timeout: .seconds(30)) throws`: polls both frontiers; returns promptly if already empty; PROGRESS-BASED deadline (resets whenever outstanding drops below its lowest seen value, not a total wall-clock cap); throws .drainTimeout only on a true stall, never hangs +- :319 `awaitDrain(stream:pollInterval:timeout:)`: stream-scoped twin; same progress-based deadline; global awaitDrain would block forever on OTHER streams' jobs a scoped drainer never claims +- :349 `completed(streamID:)` +- :357 `maildirSubdirs = ["tmp","new","cur","done"]` +- :359 `ensureMaildir(root:) throws`: creates 4 subdirs if absent +- :376 `cleanStaleTmpFiles(root:) throws`: deletes tmp/ entries older than staleTmpThreshold ### Backend contract: QueueBackend.swift - :9 `protocol QueueBackend: Sendable` @@ -154,9 +154,11 @@ ENTRY POINTS (most callers need only these): - :240 `final class ContinuationBox`: bridges DispatchSource cancel handler (DispatchQueue callback) to async wait() ### Telemetry: QueueKitTelemetry.swift (off-path cost ~1ns; metric namespace `queue.*`) -- :27 `struct QueueLatencyWindow`: capacity-100 rolling sample list -- :44 `percentile(_:) -> Double`: P7-secfix: guards non-finite/out-of-range p BEFORE index computation (NaN/inf could crash) -- :68 `reportQueueStats(backend:drained:drainStart:now:estateTag:window:) async`: gate: `Intellectus.isEnabled`; emits queue.depth (or queue.depth_unavailable on read failure: NEVER fabricates 0), queue.drain_count, queue.idle_nonempty (skipped if depth unknown), queue.latency_p50_ms, queue.latency_p95_ms, queue.head_of_line_age_s +- :31 `struct QueueLatencyWindow`: capacity-100 rolling sample list; NOT itself synchronized, concurrent access goes through the box below +- :48 `percentile(_:) -> Double`: P7-secfix: guards non-finite/out-of-range p BEFORE index computation (NaN/inf could crash) +- :70 `final class QueueLatencyWindowBox: Sendable`: Mutex-backed thread-safe holder; encode worker and import worker share one window across concurrent drain() calls +- :79 `sample(_:) -> (p50: Double, p95: Double)`: appends the sample and reads both percentiles under ONE lock acquisition, so concurrent drains cannot interleave append and read +- :103 `reportQueueStats(backend:drained:drainStart:now:estateTag:window: QueueLatencyWindowBox) async`: gate: `Intellectus.isEnabled`; emits queue.depth (or queue.depth_unavailable on read failure: NEVER fabricates 0), queue.drain_count, queue.idle_nonempty (skipped if depth unknown), queue.latency_p50_ms, queue.latency_p95_ms, queue.head_of_line_age_s; window param is now the box, not an inout raw window ## INVARIANTS / GOTCHAS @@ -170,8 +172,8 @@ ENTRY POINTS (most callers need only these): - PersistenceKitBackend.watch()'s observer event is a WAKE SIGNAL ONLY (invariant 2): never read job data from the TableChange event; always re-enter through drainAvailable(). drainUntilEmpty must propagate claim errors, never `try? ?? []`, or a live fault silently reads as "queue empty." - appendOnly MUST stay false on the queuekit_jobs table declaration (invariant 5): rows are mutated in place through the new→cur→done lifecycle. - FilesystemBackend.completeBatch's jobID lookup keys on the filename suffix after the LAST `-`. This is only unambiguous because JobIDs are 32 dashless hex characters (JobID.generate()); a job id containing a `-` would break this index. -- estateTag on QueueKit is `nonisolated(unsafe)`: set it ONCE at mount, before any drain() call, never again during concurrent use. Same discipline for the internal latencyWindow. -- awaitDrain / awaitDrain(stream:) are polling latches (20ms default interval, 30s default timeout), not push notifications: there is no native "queue just emptied" event on either backend. Always returns promptly on an already-empty queue; always throws QueueError.drainTimeout rather than hanging past the deadline. +- estateTag on QueueKit is `nonisolated(unsafe)`: set it ONCE at mount, before any drain() call, never again during concurrent use. The internal latencyWindow does NOT follow this discipline: it is a `QueueLatencyWindowBox`, lock-backed, and is written on every concurrent drain() call by design (encode worker + import worker sharing an estate queue). +- awaitDrain / awaitDrain(stream:) are polling latches (20ms default interval, 30s default timeout), not push notifications: there is no native "queue just emptied" event on either backend. Always returns promptly on an already-empty queue. The timeout is PROGRESS-BASED, not a total wall-clock cap: it resets every time outstanding (pending+inFlight) drops below its lowest observed value, so a slow-but-progressing drain never false-times-out. Only a true stall (no frontier movement for the full timeout) throws QueueError.drainTimeout. - Telemetry never fabricates queue.depth=0 on a read failure: it emits queue.depth_unavailable instead, and skips every metric that depends on a known depth. Do not "simplify" this to `(try? pendingCount()) ?? 0`. - Watcher wakes are ALWAYS advisory/spurious-tolerant. drainAvailable() (via atomic rename / serializable UPDATE) is the sole claim authority on every platform. - DrainLease is heartbeat-TTL, not PID-liveness: worst-case takeover latency is one full TTL (15s). Do not add OS-specific process-liveness checks; portability across macOS/Linux/Windows is a design constraint, not an oversight. diff --git a/packages/kits/QueueKit/docs/DETAILS.md b/packages/kits/QueueKit/docs/DETAILS.md index 4cc394b..eb01e3a 100644 --- a/packages/kits/QueueKit/docs/DETAILS.md +++ b/packages/kits/QueueKit/docs/DETAILS.md @@ -20,9 +20,9 @@ sources: - path: Sources/QueueKit/QueueError.swift blob: 2697c4b7404b9e04259267cd8f4008030ebb6754 - path: Sources/QueueKit/QueueKit.swift - blob: 60dfaa1e8f92ec051810b50d8b9cadc47388c02f + blob: 3878243f6da8ad1b55bba5271f7502e8d6b8e3d7 - path: Sources/QueueKit/QueueKitTelemetry.swift - blob: 29024f112bf133012283205175aa336b8d80d7c9 + blob: 0e656863d2c9f8e83dc13fc8b6b94ae2f25ecf06 - path: Sources/QueueKit/Watcher.swift blob: cf2b270b9c60da34f7a25c016f8c18b6ba6149e4 --- @@ -198,9 +198,11 @@ which backend is mounted underneath. `QueueKit` holds four properties. It holds a `backend`, anything conforming to `QueueBackend`. It holds an optional `root` URL, present -only when the mounted backend is a filesystem backend. It holds a rolling -latency window for telemetry. It holds an `estateTag` string, used to tag -telemetry metrics by estate. +only when the mounted backend is a filesystem backend. It holds a +latency window for telemetry, wrapped in `QueueLatencyWindowBox` for +thread safety. Two streams can drain the same queue at once. Their +reports must not corrupt the shared sample list. It holds an +`estateTag` string, used to tag telemetry metrics by estate. Two initializers exist. `init(root:hlcGenerator:)` is the common path. It builds a `FilesystemBackend` at the given root, and it cleans any stale @@ -279,11 +281,15 @@ iteration re-reads the live counts. Progress made by a concurrently running drain worker between polls is therefore observed on the very next tick. -Both methods return promptly when the queue is already empty. The first -check can succeed without the method ever sleeping. Both methods throw -`QueueError.drainTimeout` rather than blocking forever, if the deadline -passes with work still outstanding. A stuck or crashed drain worker -therefore surfaces as an error, instead of as a hang. +Both methods return promptly when the queue is already empty. The +first check can succeed without the method ever sleeping. The timeout +measures time without progress. It does not cap the total wait. Each +method tracks the lowest outstanding count seen so far. Every poll +that sees a lower count resets the deadline. A drain that keeps +shrinking the queue, even slowly, never times out. Only a stalled +drain, with no frontier movement for the full timeout, throws +`QueueError.drainTimeout`. A stuck or crashed drain worker therefore +surfaces as an error, instead of as a hang. `ensureMaildir(root:)` and `cleanStaleTmpFiles(root:)` are the maildir directory-management helpers that `FilesystemBackend`'s initializer calls. @@ -582,9 +588,10 @@ rather than inside Swift's structured concurrency. ## QueueKitTelemetry.swift -This file provides `QueueLatencyWindow` and `reportQueueStats`. These are -the self-report telemetry QueueKit emits through IntellectusLib, -MOOTx01's telemetry library, after every `drain()` call. +This file provides `QueueLatencyWindow`, `QueueLatencyWindowBox`, and +`reportQueueStats`. These are the self-report telemetry QueueKit emits +through IntellectusLib, MOOTx01's telemetry library, after every +`drain()` call. The reporting path is additive, and it is deliberately cheap when unused. The very first check inside `reportQueueStats` is a single @@ -604,6 +611,16 @@ caller's arithmetic error, returns `0` rather than computing an array index from it. Converting a non-finite floating-point value to an integer index can crash the program outright on some inputs. +`QueueLatencyWindowBox` wraps `QueueLatencyWindow` in a lock. Two +drainers can share one `QueueKit` instance. An encode worker and an +import worker can each drain the same per-estate queue at once. +Without a lock, two concurrent appends could corrupt the sample +array's storage. `reportQueueStats` now takes the box, not the raw +window. It calls `sample(_:)` once per drain call. `sample(_:)` +appends the new latency sample. It also reads both percentiles under +one lock acquisition. No two drains can interleave between the append +and the read this way. + `reportQueueStats` treats a failed `pendingCount()` read as its own signal, rather than as a zero. Reporting `queue.depth = 0` when the read actually failed would be indistinguishable from an honestly empty queue. diff --git a/packages/kits/QueueKit/docs/OVERVIEW.md b/packages/kits/QueueKit/docs/OVERVIEW.md index 25fa763..275315e 100644 --- a/packages/kits/QueueKit/docs/OVERVIEW.md +++ b/packages/kits/QueueKit/docs/OVERVIEW.md @@ -20,9 +20,9 @@ sources: - path: Sources/QueueKit/QueueError.swift blob: 2697c4b7404b9e04259267cd8f4008030ebb6754 - path: Sources/QueueKit/QueueKit.swift - blob: 60dfaa1e8f92ec051810b50d8b9cadc47388c02f + blob: 3878243f6da8ad1b55bba5271f7502e8d6b8e3d7 - path: Sources/QueueKit/QueueKitTelemetry.swift - blob: 29024f112bf133012283205175aa336b8d80d7c9 + blob: 0e656863d2c9f8e83dc13fc8b6b94ae2f25ecf06 - path: Sources/QueueKit/Watcher.swift blob: cf2b270b9c60da34f7a25c016f8c18b6ba6149e4 --- diff --git a/packages/kits/QueueKit/docs/topology.svg b/packages/kits/QueueKit/docs/topology.svg index a69eacb..9ef4a3e 100644 --- a/packages/kits/QueueKit/docs/topology.svg +++ b/packages/kits/QueueKit/docs/topology.svg @@ -7,7 +7,7 @@ authored_commit: 909513d0a8ecb1e9e903af9f4d25b3e4f2528242 authored_date: 2026-07-04 sources: - path: Sources/QueueKit/QueueKit.swift - blob: 60dfaa1e8f92ec051810b50d8b9cadc47388c02f + blob: 3878243f6da8ad1b55bba5271f7502e8d6b8e3d7 - path: Sources/QueueKit/QueueBackend.swift blob: eb67e0b821bc9bfdec8cfde621c5e70d4295d331 - path: Sources/QueueKit/FilesystemBackend.swift @@ -19,7 +19,7 @@ sources: - path: Sources/QueueKit/Watcher.swift blob: cf2b270b9c60da34f7a25c016f8c18b6ba6149e4 - path: Sources/QueueKit/QueueKitTelemetry.swift - blob: 29024f112bf133012283205175aa336b8d80d7c9 + blob: 0e656863d2c9f8e83dc13fc8b6b94ae2f25ecf06 -->