Moving From Classic Mirrored Queues to Quorum Queues

Skip this intro, and take me to the how-to switch guide directly

When it comes to queueing messages in RabbitMQ, the first queue type most people think of is classic mirrored queues. That makes perfect sense, as they have been the go-to in messaging queueing for several years. All that has changed now with the introduction of quorum queues.

The Limitations of High Availability (Mirrored) Classic Queues

Using classic mirrored queues was the traditional route, but comes with several inefficiencies and limitations. Some of its known limitations include its performance and synchronization model.

Classic queues function with a single leader queue and one or more mirror queues. All reads and writes go through the leader queue, which replicates all the commands (write, read, ack, nack, etc.) to the mirrors. Once all the live mirrors have the message, the leader will send a confirmation to the publisher. At this point, if the leader failed, a mirror would get promoted to the leader and the queue would remain available, with no data loss. This uncovered two main design flaws.

A broker goes offline and comes back online

One of the critical flaws of the classic queues is that data might get discarded when a broker goes offline and comes back online. With the mirror queues back online, publishers are faced with either deciding to synchronize the mirror or not, which is another problem entirely.

Synchronization blocking

Synchronizing a mirrored queue means replicating the message from the leader to the mirror queues. Synchronization tampers with the entire queue by blocking it, which isn’t a big deal if the queues are short and the messages are small. However, it will take a long time to process. Another problem is that synchronization also causes memory issues within the cluster and, in some cases, requires a reboot.

The bottom line: The new face of message queueing is using quorum queues. Classic mirrored queues, and all their pitfalls, are outdated.

Learn about Quorum Queues

Quorum queues seek to address all the limitations of classic mirrored queues and add more benefits. Quorum queues are safer and achieve significantly higher throughput than classic mirrored queues and Quorum queues run through the Raft protocol, the widely known consensus algorithm.

Each quorum queue is a replicated queue; it has a leader and multiple followers. A quorum queue with a replication factor of five will consist of five replicated queues: the leader and four followers. Each replicated queue will be hosted on a different node (broker). Thus, clients (publishers and consumers) interact and send a command to the leader queue, it replicates such commands to the sub-queues. Clients do not interact with the sub-queues. In fact, they exist only for redundancy, allowing availability when a RabbitMQ broker fails, is shut down or gets rebooted. When a broker goes offline, a follower replica on another broker will be elected leader and service will continue.

Another reason why quorum queues are efficient compared to classic queues is that there is no ‘stop the world’ synchronization or availability vs. consistency choice to make. Quorum queues will only approve and confirm a command when it is replicated to a quorum of its nodes. Availability is only lost when the majority of the nodes are down.

Quorum queues are also simplistic compared to classic queues regarding network partition. That’s because they use a separate, quick failure detector that helps detect partitions faster to trigger a leader selection. This results in a quick restoration of availability or no impact at all.

Features of Quorum Queues

These are some of the unique features and the differences between quorum and classic queues.

No Exclusivity

Quorum queues replicate commands from the leader to the sub-queues. This makes it durable and not subject to exclusivity like classic queues. Quorum queues aren’t designed to be used as temporary queues.

Queue and message TTL

Quorum queues fully support Queue and message TTL (from version 3.10).

Queue Length Limit

Quorum queues fully support reject-publish and drop-head overflow behaviors in the system but do not support rejected messages for dead-letters, reject-publish-dlx, like classic queues.

For instance, if the quorum queue hits the max-length limit with the reject-publish configured, it will notify the publishing channels, which will reject all the incoming messages from the client.

Queue Leader Location

Quorum queues have a primary queue called the leader queue with replicated queues, often called followers or replica queues. All commands from the administrator go through the leader queue, which replicates the message to the others. This action guarantees ordered FIFO messages.

The leader queues are evenly distributed across cluster nodes to prevent some nodes from having most of the load.

Priority Processing

Quorum queues do not support priorities. Instead, use multiple queues and assign different queues to process different priorities; one for each priority.

How to Make Use of Quorum Queues

Declare a queue by setting the x-queue-type argument to quorum. This will declare a quorum queue with up to five replicas, which is the default. For example, a cluster of three nodes will have three replicas, one on each node. If you had a cluster of seven nodes, five out of the seven nodes would each host one replica while two particular nodes would not have any replicas.

After declaring a quorum queue, you can bind it to any exchange just as with other queue types.

Make the move from Classic Mirrored Queues to Quorum Queues

Directly changing queue types is not possible. Instead, there are three options when moving from classic mirrored queues to quorum queues:

Option 1: Create a new queue with the type quorum and move the publisher then the consumer.

Option 2: Move messages from the old queue to the new queue by using a shovel.

Option 3: Set up a new cluster and move messages by using a federation.

Option 1: Move the publisher, and then the consumer

1. Declare a Quorum Queue: Set the x-queue-type queue argument to quorum . The queue argument should be provided by the client and set when declaring the queue.
2. Move the publisher and empty the queue: Empty the queue by moving the publisher to the new queue and allowing the consumer to consume messages from the old queue until it's empty.
3. Move the consumer: Finally, switch the consumer to the new queue.

Option 2: Move messages using a shovel

1. Set up a shovel: Create a shovel from the RabbitMQ management interface. Click on the Admin tab and select Shovel Management from the dropdown menu.
2. Declare quorum queues: Set the x-queue-type queue argument to quorum . The queue argument should be provided by the client and set when declaring the queue.
3. Move message: Use the created shovel to move messages from one queue to the other.

Learn about Shovel Management here.

Option 3: Set up a new cluster and move messages by using a federation.

1. Set up a new cluster: Set up a new cluster with the same exchanges, bindings, and queues.
2. Declare quorum queues: Set the x-queue-type queue argument to quorum . The queue argument should be provided by the client and set when declaring the queue.
3. Set up queue federation: Set the old cluster as a federation upstream with empty federation queue settings (default is to use the same name as the federated queue).
4. Create a policy: Create a policy that uses the new federation upstream.
5. Move consumers: Move consumers to the new cluster.
6. Move the publisher: Once the old cluster is drained, move the producers.

A blog and example of cluster migration with queue federation can be found here.

The quorum queue is the new face of message-queueing with RabbitMQ. Whether you’re looking to improve your app performance and reliability or to integrate various microservices, using quorum queues is the answer.

Read our blog for more updates.