Skip to content

TOCTOU race in finalizeRunIfComplete #43

@techiejd

Description

@techiejd

Summary

finalizeRunIfComplete in src/tasks/bulkEmbedAll.ts has a time-of-check-to-time-of-use (TOCTOU) race condition. When two per-batch polling tasks complete simultaneously, both can read the run as non-terminal, both proceed to finalize.

Impact

  • Duplicate onError callbacks to user code
  • Double metadata deletion attempts (second is a no-op, no data loss)
  • Run record ends up correct either way (last-write-wins), so no data corruption

Root cause

The idempotency check at the top is a read-then-act, not an atomic compare-and-swap:

const currentRun = await payload.findByID({
  collection: BULK_EMBEDDINGS_RUNS_SLUG,
  id: runId,
})
if (TERMINAL_STATUSES.has((currentRun as any).status)) {
  return { finalized: true, status: (currentRun as any).status }
}

Two tasks can both pass this check before either writes the terminal status.

Suggested fix

Use a conditional update (e.g. WHERE status = 'running' via Payload's where on update, or a raw SQL compare-and-swap) so only the first task to reach the update actually finalizes. The second task's update would be a no-op, and it can detect this and return early.

Severity

Low — worst case is a duplicate onError call, not data loss. But worth fixing before a stable release.

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions