Because this blog series is mostly going to be about connecting browser-based applications with RabbitMQ, we will begin by putting the conversation here in the context of the web.
We will start from the very beginning — the evolution of the web. You can skip to part 2
The World Wide Web has witnessed significant transformations over the years. Initially, web pages were static, then came the dynamic web. And now, the web has evolved — the need for immediate feedback and real-time interactions has grown. Whether it's for chat applications, live sports score updates, or stock market ticks, real-time interaction has become essential.
To achieve this real-time magic on the web, several techniques emerged.
Approaches to Real-time Interaction on the Web
- Polling: Here, the browser repeatedly asks (or "polls") the server if there's new data. This works, but it's not very efficient. If data changes infrequently, we end up making a lot of unnecessary requests.
- Server-Sent Events (SSEs): SSEs emerged after realizing that, instead of repeatedly polling the server, an open connection could be maintained that allows the server to send data to a client as it becomes available. This reduces the overhead of multiple unnecessary requests and responses. However, the main drawback of SSEs is its unidirectionality: while servers can push data, clients can't respond through the same connection, making SSEs ideal for one-sided server updates. For example, at CloudAMQP we use SSEs to show logs and updates when changing plans.
- WebSockets: This was a game-changer. Instead of repeatedly asking for data, why not keep a line of communication open between the browser and server? WebSockets do precisely that, enabling bi-directional communication.
But as the web matured even more, so did the complexity of applications. Today's apps are rarely monolithic; they're a symphony of microservices and third-party integrations. This new landscape demands not just dynamic and real-time communication on the web but robust, scalable and reliable communication between services as well. Enter …
RabbitMQ and WebSockets
WebSocket does indeed provide a way to establish bi-directional communication between the browser and the backend, however, it does not cater to the need for robust, scalable and reliable communication between services. And this is where RabbitMQ comes in.
RabbitMQ is a message broker with various features that offer advantages beyond just communication. When used as the medium of communication between the services in a distributed architecture, RabbitMQ offers benefits like decoupling of the services, ease of scalability, reliable message delivery and redundancy amongst others.
These are all benefits that are essential to the modern distributed software landscape.
While on the one hand, WebSockets simplifies real-time communication, RabbitMQ, on the other hand, ensures a more robust and scalable architecture. A real-time application that is built on a robust and scalable architecture is what we have when RabbitMQ and WebSockets come together. But…
How is Connecting RabbitMQ and WebSocket Achievable?
Before we get into the techniques of connecting to RabbitMQ from the browser, it's essential to comprehend the protocols that underlie these techniques: MQTT, AMQP, and STOMP.
MQTT (Message Queuing Telemetry Transport)
Sometime in the past, there was a challenge to monitor oil pipelines spanning vast, remote areas. These areas had unreliable, low-bandwidth networks. Plus, the monitoring devices ran on batteries, making energy conservation essential. Thus, constant communication or high network use wasn't practical.
There was a need for a wire protocol that caters to these needs and the answer was MQTT. MQTT was designed with specific features that are tailored to the challenges at hand:
Lightweight Protocol
MQTT was designed to be minimalistic, with concise message headers and a compact payload, reducing the amount of data that needed to be sent across the network. MQTT also focuses on just one messaging pattern — publish/subscribe.
Quality of Service (QoS) Levels
MQTT introduced different levels of message delivery guarantees, allowing for
trade-offs between guaranteed message delivery and bandwidth usage.
You can optionally specify one of three QoS levels when publishing a message:
0, 1, 2
client.publish('some/topic', 'Hello World', { qos: 1 }, function(err) {
if (!err) {
console.log("Message sent!");
}
});
QoS 0 has the lowest bandwidth usage, but also the lowest delivery guarantee. QoS 2 has the highest bandwidth usage and the highest delivery guarantee as well.
Last Will and Testament
Given the unreliable nature of the connections, devices could set a "last will" message that would be sent by the broker if they unexpectedly disconnected. This information allows subscribers to respond according to their requirements. The last will is usually specified by a client at connection time:
const options = {
will: {
topic: 'some/topic',
payload: 'Client has unexpectedly disconnected',
qos: 1,
retain: true
}
};
const client = mqtt.connect('mqtt://broker-url', options);
Retained Messages
MQTT brokers could retain the last known message on a topic, ensuring that any device that came online after a downtime would immediately get the latest message without waiting for a new update.
client.publish('some/topic', 'Hello World', { retain: true }, function(err) {
if (!err) {
console.log("Message sent and retained!");
}
});
All these features combine to make MQTT successful in this challenging scenario. Its success showcased its potential for other use cases as well. With the rise of the Internet of Things (IoT), where many devices face similar constraints of limited power, bandwidth, and unreliable connections, MQTT became an ideal choice. Its simplicity, lightweight nature, and features tailored for constrained environments made it a leading protocol for IoT deployments.
AMQP (Advanced Message Queuing Protocol)
Before AMQP, the financial industry, particularly sectors like investment banking, relied heavily on systems that needed to communicate with each other reliably, securely, and efficiently, often across different regions and among different institutions.
They used messaging middleware to integrate these different systems and facilitate communication between them. However, the landscape of messaging middleware was fragmented. Different vendors offered proprietary solutions, leading to significant issues around vendor lock-in, interoperability, integration complexity, and even concerns about cost.
There was a need for a new, open messaging protocol. This need would fuel the work that eventually became AMQP. AMQP was born out of the vision to have a protocol that any vendor could implement, ensuring interoperability between different systems and products.
The AMQP working group focused on ensuring that AMQP provided features crucial to enterprise messaging:
Open Standard
From the outset, AMQP was intended to be an open standard, meaning that its specifications would be publicly available. Any vendor or developer could use these specifications to implement AMQP in their products or systems.
Reliable Message Deliveries
One of the main mechanisms to ensure reliability in AMQP is acknowledgments. When a message is consumed, an acknowledgment is sent to the broker to confirm the message is processed.
const amqp = require('amqplib/callback_api');
amqp.connect('amqp://localhost', function(error, connection) {
connection.createChannel(function(error, channel) {
const queue = 'myQueue';
channel.assertQueue(queue, { durable: true });
channel.consume(queue, function(msg) {
console.log("Received message: %s", msg.content.toString());
// Manual acknowledgment after processing message
channel.ack(msg);
});
});
});
AMQP also provided mechanisms for reliable message deliveries via durable queues and message persistence amongst others.
Flexibility
In AMQP, messages go to an exchange first then to a queue. Queues are bound to exchanges with specific patterns or criteria. This binding mechanism, combined with various exchange types, allows for complex routing logic that, in turn, opens up doors for various messaging use cases like publish/subscribe, point-to-point, and request/reply patterns in AMQP.
Once AMQP reached a stable version, it started gaining traction. Several messaging products like RabbitMQ began supporting AMQP, leading to greater interoperability between different systems. While born from challenges in the financial sector, AMQP found relevance across multiple industries due to its reliability, flexibility, security, and open nature.
STOMP (Simple Text Oriented Messaging Protocol)
STOMP was born out of the desire to build something simple – a "bare-bones" protocol that could be easily implemented and understood.
It was designed with the following principles in mind:
Simplicity
STOMP was designed with a minimalist approach. The idea was to provide the basic features required for messaging without the overhead and complexity of more feature-rich protocols.
STOMP’s straightforwardness is reflected in the following:
Few Commands:
STOMP has a limited set of commands – basically verbs like
CONNECT, CONNECTED, SEND, SUBSCRIBE, UNSUBSCRIBE, BEGIN, COMMIT, ABORT, ACK, NACK, MESSAGE, CRECEIPT and ERROR.
This contrasts with AMQP, for example, which has a much richer set of methods.
For example, just the connection and channel classes expose the following methods.
Connection Class:
-
Start, Start-Ok, Secure, Secure-Ok, Tune, Tune-Ok, Open, Open-Ok, Close, Close-Ok
Channel Class:
-
Open, Open-Ok, Flow, Flow-Ok, Close, Close-Ok
Less State Management: STOMP is stateful but doesn't have the detailed state management of AMQP. For instance, in AMQP, you need to declare exchanges, bind queues to them, set up various properties for messages, etc. In STOMP, you simply send a message to a destination, and the broker handles the details.
Fewer Features by Design: STOMP generally aims to handle the basics of messaging quite well, many advanced features present in AMQP are either absent or implemented at a higher level in STOMP.
All these combine to make STOMP simple, not just from a usage perspective but from a client implementation perspective as well.
Text-Oriented
One of STOMP's defining features is its text-based nature. This makes it easy to understand and debug. When you look at raw STOMP frames, they're human-readable, almost like HTTP. This contrasts with many binary protocols, where the raw data is less intuitive to human observers. For example, publishing a message with the payload “Hello World” to a queue named “test”, would look like this under the hood:
SEND
destination:/queue/test
content-type:text/plain
Hello, World!
Conclusion
Now that we have an overview of MQTT, STOMP, and AMQP, in the next and final piece in this series, we will explore the different techniques for connecting browser-based applications with RabbitMQ.
Ready to start using RabbitMQ and WebSockets in your architecture? CloudAMQP is one of the world's largest RabbitMQ cloud hosting providers. In addition to RabbitMQ, we have also created our in-house message broker, LavinMQ - we benchmarked its throughput at around 1,000,000 messages/sec.
Easily create a free RabbitMQ or free LavinMQ instance on CloudAMQP. All are available after a quick and easy signup.
For any suggestions, questions, or feedback, get in touch with us at contact@cloudamqp.com
