Wheels 4.0 — background jobs without Redis

[bpamiri]

Wheels 4.0 ships a first-class job queue that lives in your existing database. This post is a guided tour: the Job CFC surface, the CLI daemon, what makes the implementation correct, the multi-tenancy adjacency, and where Redis still wins.

The Redis tax on small-to-medium apps

The pattern is well-trodden — delayed_job in Rails, Laravel's database queue driver. A wheels_jobs table, columns for payload / run-at / attempts / lock; workers poll, claim, execute. The trade is throughput for one less moving part: no separate process, no separate failure mode, no separate runbook. For the median Wheels app, the DB-queue ceiling is well above their floor.

Job CFC surface

// app/jobs/SendWelcomeEmailJob.cfc
component extends="wheels.Job" {
    function config() {
        super.config();
        this.queue = "mailers";
        this.maxRetries = 5;
        this.baseDelay = 2;
        this.maxDelay = 3600;
    }
    public void function perform(struct data = {}) {
        sendEmail(to=data.email, subject="Welcome!");
    }
}

Backoff: Min(baseDelay * 2^attempt, maxDelay). With the defaults above: 2, 4, 8, 16, 32 seconds, ceilings at 1 hour. Override $calculateBackoff() for linear / constant / custom.

Three enqueue methods on the instance:

job.enqueue(data={email: user.email});
job.enqueueIn(seconds=300, data={email: user.email});
job.enqueueAt(runAt=reminderDate, data={email: user.email});

CLI daemon

wheels jobs work --queue=mailers --interval=3
wheels jobs status
wheels jobs status --format=json
wheels jobs retry --queue=mailers
wheels jobs purge --completed --older-than=30
wheels jobs monitor          # live TUI dashboard

Exits cleanly on SIGTERM, so systemd / supervisord / Compose restart: unless-stopped configs are trivial. Stdout logs to your log stack — journald / Loki / Elasticsearch.

What makes the implementation correct

Claim is optimistic. Worker reads the next ready row, updates state to processing with a WHERE state = 'pending' guard, checks the affected-row count. If another worker grabbed it first, the count is zero and the loser moves on. No advisory locks, no external coordinator.

Timeout recovery handles crashes. Each claimed job records a worker heartbeat; if a job has been processing longer than its configured timeout without a heartbeat update, a sweeper requeues it. A killed-mid-job worker does not orphan its work.

Retries are exponential by default, per job class. When the budget is exhausted, the job is marked failed and sits there until you decide — retry manually, dig into the payload, or purge.

Auto-created wheels_jobs table. First enqueue or first worker run calls Job.cfc::$ensureJobTable(). No migration to ship.

Multi-tenancy without the payload ceremony

This is where it pays off. #1951 resolves the active tenant at the request layer. A request for tenant A enqueues a job → the row lands in tenant A's database (or schema). The worker, pointed at that datasource, resolves the tenant context the same way the request did.

You do not stuff tenant IDs into the payload. You do not write a tenant-aware job base class. The "did I remember to set tenant_id?" bug class disappears.

For pool-per-tenant deployments — one worker per tenant database — jobs route themselves naturally. For shared-pool with tenant-resolved datasources, enqueue and dequeue use the same resolution logic, so it composes by construction.

When to pick Redis anyway

• Hundreds of thousands of jobs per minute through one queue. Row-lock throughput becomes the bottleneck. Redis lists / streams do not.
• Pub/sub fan-out. One event lighting up N workers. Native in Redis, polling-approximate in a DB queue.
• Sub-50ms pickup latency. A 1-second poll is not it; pushing to 100ms puts pressure on the DB.

Most SaaS apps are nowhere near these limits. If your current Redis instance serves the queue and nothing else, and queue depth rarely exceeds a few thousand, you are paying Redis taxes for capacity you will not use this decade.

SSE adjacency

#1940 — pub/sub SSE channels. A completed job can publish; every connected browser gets the update. No websocket server, no Pusher account, no third-party dependency. DB-backed queue + SSE pub/sub covers a large slice of what you would previously have needed Redis + a websocket gateway for.

Production operations

• wheels jobs work under systemd / supervisord / Compose restart: unless-stopped.
• Stdout logs to your log stack.
• Horizontal scale = run more processes. Optimistic-lock claim handles contention; no coordinator.
• Monitoring: wheels jobs status --format=json from cron / Prometheus textfile, alert on failed > 0 or pending > N.

Links

• PR #1934 — Background jobs
• PR #1951 — Tenant-aware datasource switching
• PR #1940 — SSE pub/sub channels
• Background jobs guide
• Multi-tenancy guide

Question for the thread

If Redis is in your stack only for the job queue, what would it take to remove it? Specifically — what's the throughput peak, pub/sub need, or latency requirement keeping it pinned in your runbook? Useful input for 4.0.x tuning priorities.

Read the full post: https://blog.wheels.dev/posts/background-jobs-without-redis/