Streaming with LavinMQ - an introduction

Choosing between queueing or streaming has often been a matter of choosing between a certain type of broker. However, this is no longer the case. For example. LavinMQ stream queues bring a Kafka-like experience to your AMQP broker.

In LavinMQ, messages in stream queues are immutable, allowing multiple consumers to read and re-read them. This hands-on tutorial introduces stream queues through a practical use case: streaming user activity data. It’s just one way to leverage streams in your applications.

Q&A LavinMQ Streams

What's the similarities and differences between LavinMQ Streams, RabbitMQ Streams and Apache Kafka?

LavinMQ Streams, Kafka, and RabbitMQ Streams share several core characteristics:

All three technologies treat messages as immutable.
Under the hood, they all use an append-only log to store messages, ensuring that multiple consumers can process the same data.
LavinMQ streams, Kafka and RabbitMQ streams, support offsets, allowing consumers to rewind messages or jump to and read messages from specific points in the stream.

Overall, If you rely heavily on log compaction, stream processing, or Kafka’s partitioning model, LavinMQ Streams may not be a direct replacement. However, if you’re looking for a simpler, AMQP-friendly and budget-friendly alternative with strong streaming capabilities, LavinMQ Streams is a solid choice.

How do I set up streaming with LavinMQ on CloudAMQP?

You first of all need to create a LavinMQ instance on CloudAMQP.
Declare streams just like queues using an AMQP client. Set x-queue-type to stream and ensure durable=true, as LavinMQ requires all streams to be durable. See code snippet below:

channel.queue_declare(
  queue='test_stream',
  durable=True,
  arguments={"x-queue-type": "stream"}
)

Can I change from queueing to streaming?

Yes, this is doable:

Declare one or more stream queue(s) — depending on your use-case.
Shovel messages from existing queues to corresponding stream queues, if need be.
Point your publishers and consumers to the stream queues.

Conclusion

Traditional queues in LavinMQ support typical message queuing use-cases where messages are deleted after being consumed. However, some messaging use-cases require some form of permanent data storage that could easily enable message replay and fan-out architectures, amongst others. Stream queues in LavinMQ were introduced to cater for such use-cases.

Do you want to try out message streaming with LavinMQ? Reach out to our support team