Scraping Prometheus metrics

CloudAMQP now supports scraping Prometheus server metrics on all dedicated plans. This enables access to server and broker metrics in Prometheus format on CloudAMQP servers with Prometheus support.

Prometheus metrics from a CloudAMQP server can be found on the endpoints /metrics, /metrics/detailed and /metrics/node.

/metrics provides aggregated broker metrics, meaning all metrics are aggregated on an object, e.g. connections or queues.
/metrics/detailed provides much more granular per-object broker metrics. It provides metrics per object, meaning you can get all queues per connection, all messages per queue, and other in-depth metrics. Scraping this endpoint may take a lot of server resources if you have a lot of objects like connections, channels, queues, or exchanges.
/metrics/node provides server metrics like CPU and memory usage.

Understand Prometheus metrics

Prometheus endpoints provide metrics per server, not per cluster. The server owning the queue will count only that queue. To get all queues, you need to scrape from all servers and aggregate the metric over all servers.

Metrics are mainly of two types: gauge and counter, (while some metrics are untyped). A gauge metric is a value that can increase and decrease, while a counter metric will only increase from the last value and is only reset when the system restarts.

rabbitmq_queues (gauge metric) tells how many queues exist at this particular moment.
rabbitmq_queues_declared_total (counter metric) continues to increase if more queues are declared even when queues are deleted. It will continue to increase until a broker restarts.

Note: Aggregated counter metrics, from /metrics, may decrease between two timestamps depending on the object they are aggregated on.
rabbitmq_queue_messages_delivered_total (counter metric) is aggregated across all queues, meaning it will decrease if a queue is deleted since it only aggregates queues that exist.
rabbitmq_channel_messages_delivered_total (counter metric) is aggregated by channel, meaning it will only count messages on channels that are still alive. If a channel is closed, the metric value will decrease.

global metrics are aggregated across the entire system and are not dependent on any object. This means they will only decrease and never increase, only on a broker restart.
rabbitmq_global_messages_delivered_total (counter metric) is aggregated across all messages, meaning it will continue to increase as messages are delivered.

Version support

LavinMQ supports Prometheus metrics natively and RabbitMQ versions 3.8.0 and later have the Prometheus plugin enabled by default. Due to limitations in Prometheus support for RabbitMQ, all features in RabbitMQ may not function as expected before version 3.10.0.