Collecting Unroutable Messages in a RabbitMQ Alternate Exchange

In this article, we describe how to set up and use alternate exchanges.

Issues with Unrouted Messages

Unroutable messages are an issue. They slow down processing times if applications make multiple attempts at delivery or are kept busy by continually logging them. Additionally, there is what to do with unrouted messages, which must eventually be handled or dropped.

Whether messages returns or not, unroutable messages can:

• Be returned to a broken application that constantly resends them.
• Be the result of malicious activity aimed at causing RabbitMQ to become unresponsive.
• Cause the loss of mission-critical data

As an example, think of a hospital that uses RabbitMQ to help store patient information in a database. Medical devices typically push vital data to a small server for processing, which then forwards the information to the cloud for storage. In this case, RabbitMQ can be used to push data to the cloud and be employed to facilitate streaming.

Should messages drop before they end up in RabbitMQ, doctors could lose vital information. This loss could impact diagnosis and treatment. The situation could become even worse if the system breaks and overloads the resources handling messages related to more than one patient.

What happens to unroutable messages in RabbitMQ?

It is possible to avoid the complete loss of a message. RabbitMQ handles unroutable messages in two ways based on the mandatory flag setting within the message header. The server either returns the message when the flag is set to true or silently drops the message when set to false.

Applications can log returned messages, but logging does not provide a mechanism for dealing with an unreachable exchange or queue.

Avoid Data Loss with a RabbitMQ Alternate Exchange

RabbitMQ also lets you define an alternate exchange to apply logic to unroutable messages. Set an alternate exchange using policies or within arguments when declaring an exchange.

The alternate exchange attaches to one or more primary exchanges. Set the exchange type to fanout to ensure that rejected messages always route to the alternate queue. Be advised that there is no way to catch invalid exchange names outside of the mandatory flag.

Creating the Alternate Exchange

Defining an alternate exchange is no different than defining any other part of your topology. The primary exchange forwards the message to the alternate exchange, which sends it to the alternate queues.

Create a fanout exchange and attach a queue:

conn_str = os.environ["AMQP_URI"]
params = pika.URLParameters(conn_str)
connection = pika.BlockingConnection(params)
channel = connection.channel()

channel.exchange_declare("alt_exchange", "fanout")
channel.queue_declare("alt_queue")
channel.queue_bind("alt_queue", "alt_exchange")

Full sample code can be found at GitHub.

Make sure that the alternate queue is attached to the exchange through the RabbitMQ API. You can use the API provided by RabbitMQ to list bindings:

http://localhost:15672/api/exchanges/test_vhost/primary_exchange

Change the URL to match your host and port. Additionally, enable the RabbitMQ management console.

Any message sent to the alt_exchange winds up in the alt_queue. Since we use a fanout exchange, the message ends up in all queues attached to alt_exchange.

Attaching the Alternate Exchange

Use policies, the UI, or message headers to set the alternate exchange. You must use the command line to attach an alternate exchange after declaring the primary exchange.

You can use message arguments to set the exchange:

args = {"alternate-exchange": "alt_exchange"}
channel.exchange_declare("primary_exchange", "direct", arguments=args)

Alternately, use the command line to specify the alternate exchange:

rabbitmqctl set_policy AE “primary_*” ‘{“alternate-exchange”: “alt_exchange”}’

RabbitMQ sends all messages from the primary_exchange to the alt_exchange when the routing key is invalid. Unrouted messages end up in the alt_queue.

What happens when the mandatory flag is set with an alternate exchange?

There is still a chance that messages won’t be routed if an alternate exchange is provided. The service may be unreachable, or the alternate queue may not be specified correctly. You might accidentally specify a non-existent exchange as well.

Setting the mandatory flag will catch these unrouted messages. Keep in mind that, if the message routes to the alternate exchange, RabbitMQ marks the message as delivered.

Happy coding!