Queen MQ - Partitioned Message Queue backed by PostgreSQL

Unlimited ordered partitions that never block each other. Consumer groups, replay, transactional delivery — ACID-guaranteed.

Note that queen-mq is available also as a experimental PostgreSQL extension, see pg_qpubsub for more details.

📚 Complete Documentation • 🚀 Quick Start • ⚖️ Comparison

---

Queen MQ is a partitioned message queue backed by PostgreSQL, built with uWebSockets, libuv, and libpq async API. It features unlimited FIFO partitions that process independently, consumer groups with replay, transactional delivery, tracing, and ACID-guaranteed durability. With Queen you get Kafka semantics on PostgreSQL, using stateless clients that speak HTTP to a stateless server, easy to deploy and manage, but still powerful and flexible enough to sustain tens of thousands of requests per second across thousands of partitions.

For a experimental PostgreSQL extension version of Queen MQ, see pg_qpubsub.

Why Queen?

Born at Smartness to power Smartchat, Queen solves a unique problem: unlimited FIFO partitions where slow processing in one partition doesn't block others.

Perfect for:

• Processing messages in order, without losing them somewhere in the middle
• Avoid slow message processing blocking other partitions, solving Head of Line Blocking problem
• When you need tens of thousands of partitions to process messages in parallel, respecting the order of the messages
• Process the same messages in multiple pipelines
• Have a clear view of the message processing flow and traceability
• Build event-driven microservices with exactly-once delivery guarantees
• Critical systems that need to be highly available and reliable with zero message loss

Its push peaks for single request is around 45k req/s, with sustatined load (PUSH+POP) around 10k req/s, and with consumer groups around 60k req/s.

Quick Start

Create a Docker network and start PostgreSQL and Queen Server:

# Create a Docker network for Queen components
docker network create queen

# Start PostgreSQL
docker run --name qpg --network queen -e POSTGRES_PASSWORD=postgres -p 5433:5432 -d postgres

# Wait for PostgreSQL to start
sleep 2

# Start Queen Server
docker run -p 6632:6632 --network queen -e PG_HOST=qpg -e PG_PORT=5432 -e PG_PASSWORD=postgres -e NUM_WORKERS=2 -e DB_POOL_SIZE=5 -e SIDECAR_POOL_SIZE=30 smartnessai/queen-mq:0.12.2

Then in another terminal, use cURL (or the client libraries) to push and consume messages

Push message:

curl -X POST http://localhost:6632/api/v1/push \
  -H "Content-Type: application/json" \
  -d '{"items": [{"queue": "demo", "payload": {"hello": "world"}}]}'

that returns something like:

[{"message_id": "...", "status": "queued", ...}]

Consume message:

curl "http://localhost:6632/api/v1/pop/queue/demo?autoAck=true"

that returns something like:

{"messages": [{"data": {"hello": "world"}, ...}], "success": true}

Then go to the dashboard (http://localhost:6632) to see the messages and the status of the queue.

Documentation

📚 Complete Documentation

Getting Started

• Quick Start Guide
• Basic Concepts

Client Libraries

• JavaScript Client
• Python Client
• C++ Client
• HTTP API Reference

---

Structure of the repository

The repository is structured as follows:

• lib: C++ core queen library (libqueen), implementing libuv loops, sql schema and procedures
• server: Queen MQ server, implementing the HTTP API that talks to the libqueen library
• pg_qpubsub: PostgreSQL extension for using queen-mq semantics as a PostgreSQL extension
• client-js: JavaScript client library (browser and node.js)
• client-py: Python client library (python 3.8+)
• client-cpp: C++ client library (cpp 17)
• proxy: Proxy server (authentication)
• app: Vue.js dashboard (vue 3)
• website: Documentation website (vitepress)
• examples: JS client examples

---

Release History

JS clients from version 0.12.0 can be run inside a browser

Server Version	Description	Compatible Clients
0.12.3	Added JWT authentication	JS ≥0.7.4, Python ≥0.7.4, 0.12.0 if needs to use proxy auth
0.12.x	New frontend and docs	JS ≥0.7.4, Python ≥0.7.4, 0.12.0 if needs to use proxy auth
0.11.x	Libqueen 0.11.0; added stats tables and optimized analytics procedures, added DB statement timeout and stats reconcile interval	JS ≥0.7.4, Python ≥0.7.4
0.10.x	Total rewrite of the engine with libuv and stored procedures, removed streaming engine	JS ≥0.7.4, Python ≥0.7.4
0.8.0	Added Shared Cache with UDP sync for clustered deployment	JS ≥0.7.4, Python ≥0.7.4
0.7.5	First stable release	JS ≥0.7.4, Python ≥0.7.4

Full Release Notes →

---

Bug fixing and improvements

• Clients 0.12.1: Fixed bug in transaction with consumer groups

---

License

Queen MQ is released under the Apache 2.0 License.

---

Built with ❤️ by Smartness

Why "Queen"? Because years ago, when I first read "queue", I read it as "queen" in my mind.