Skip to content

flaky: system-restart drain timer asserts an exact 60000 against an absolute deadline #1045

Description

@lidge-jun

What happens

tests/system-restart.test.ts:361 asserts the drain timer is scheduled for exactly MEMORY_DRAIN_RESTART_MS:

expect(timers[0]?.ms).toBe(MEMORY_DRAIN_RESTART_MS);

On a loaded runner it gets 59999.

Observed on run 30977993140, job 92216093297, on the dev push after #1022 merged. It passes when rerun in isolation.

Why it is the test, not the code

src/server/management/system-restart.ts:358 computes an absolute deadline:

const restartDeadlineMs = now() + MEMORY_DRAIN_RESTART_MS;

and the remaining budget is derived from a later clock read. One millisecond elapsing between the two is correct behavior — the deadline is absolute, so the scheduled delay shrinks to match it. The test asserts on the delay as if it were a constant.

Why it is worth fixing rather than rerunning

This is the same family as #1042: an assertion coupled to nondeterminism it does not intend to test, so it reddens CI without evidence of a defect, and it would not notice a real regression in the property it claims to guard. A drain that scheduled 30000 would still be caught, but one that drifted by a few hundred milliseconds under load would be indistinguishable from this noise.

Suggested fix

Either inject the clock so both reads return the same value, or assert the invariant the code actually maintains — that the scheduled delay is within a small tolerance of the remaining budget, and never exceeds it.

Found while verifying #1022 against current dev; not caused by it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions