Redis broker lacks reliable message delivery - jobs lost on consumer crash

[saravanasai]

Problem

The current Redis broker implementation uses BLPOP to consume messages, which has a critical reliability issue jobs are permanently lost if a consumer crashes after popping a message but before completing processing.

// brokers/redis/broker.go:195
res, err := b.conn.BLPop(ctx, b.opts.PollPeriod, queue).Result()
// Job is now REMOVED from queue
work <- []byte(msg)
// If consumer crashes here or during processing, job is LOST

There is no "processing" or "in-flight" queue to track jobs that have been dequeued but not yet acknowledged.

Proposed Solution

Implement the Reliable Queue Pattern using Redis BRPOPLPUSH/BLMOVE

// Atomically move from main queue to processing queue
msg := BRPOPLPUSH(sourceQueue, processingQueue, timeout)

// Process the job...

// On success: remove from processing queue
LREM(processingQueue, msg)

// On failure/retry: move back to source queue
RPOPLPUSH(processingQueue, sourceQueue)