RabbitMQ 3.9.0

The most important changes are Streams, support for Erlang 24, and improved integration with our RabbitMQ Kubernetes Operator. Other changes worth mentioning are JSON logging, per-protocol and queue type message rates as Prometheus metrics and faster initial cluster formation on Kubernetes.

Find blog posts that expand on the previously mentioned topics: https://blog.rabbitmq.com/tags/v3.9.x/

Highlights

Streams

Streams are a new persistent and replicated data structure ("queue type") in RabbitMQ which models an append-only log with non-destructive consumer semantics.

They can be used as a regular AMQP 0.9.1 queue or through a new binary protocol plugin and associated client(s).

Streams enable messaging patterns that were not possible or practical before, for example, repeatable non-destructive consumption and mass fanouts to a very large number of consumers.

Erlang 24 Support

Erlang 24 provides 20%-50% higher throughput for many workloads tested by the RabbitMQ core team and community members. Supporting the latest runtime version also allowed RabbitMQ to replace a few external dependencies with libraries now provided by the runtime.

Erlang 24 is now used by default in the community RabbitMQ Docker image.

Kubernetes

RabbitMQ Cluster Operator automates provisioning, management, and operations of RabbitMQ clusters running on Kubernetes. It is open source and is developed by the RabbitMQ Core team at VMware, Inc. This Operator is not specific to 3.9 and can be used with latest 3.8.x release series.

Messaging Topology Operator makes it possible to define RabbitMQ resources (virtual hosts, users, permissions, topologies, policies, etc) as Kubernetes objects. This Operator is also open source and can be used with latest 3.8.x release series.

Logging in JSON

Switch from Lager to the new Erlang Logger API for logging introduces an option for structured logging in JSON. This is relevant for the Prometheus-based alerting feature that we have been investing in this year: Notify me when RabbitMQ has a problem.

Obtaining Packages

RabbitMQ releases are distributed via GitHub, Cloudsmith, and PackageCloud.

Erlang/OTP Compatibility Notes

This release requires Erlang 23.2 or later. Erlang 24 is recommended as it offers 20%-50% throughput improvements for many workloads.

Provisioning Latest Erlang Releases explains what package repositories and tools can be used to provision latest patch versions of Erlang 23.3.x and 24.x.

Upgrading to 3.9

See the Upgrading guide for documentation on upgrades and RabbitMQ change log for release notes of other releases.

RabbitMQ 3.9.0 nodes can run alongside 3.8.x nodes. However, 3.8.16+ as well as 3.9.x both require Erlang 23.2 or later (and support 24). 3.9.x-specific features can only be made available when all nodes in the cluster upgrade to 3.9.0 or any other patch release in the series.

While operating in mixed version mode, some aspects of the system may not behave as expected. The list of known behavior changes is covered below. Once all nodes are upgraded to 3.9.0, these irregularities will go away.

Mixed version clusters are a mechanism that allows rolling upgrade and are not meant to be run for extended periods of time (no more than a few hours).

Quorum Queue Declaration

In a mixed version cluster, declaration of new quorum queues will fail if executed on a node in the version minority. For example, in a five node cluster with two nodes running 3.8, the three 3.9 nodes are in the version majority and the remaining are in the version minority.

Existing quorum queues will operate just like before.

Classic Queue Metrics

HTTP API responses will provide a limited number of metrics or metrics for a limited number of queues (for operations such as GET /api/queues/{vhost}) when operating in mixed version mode.

Compatibility Notes

Client Library Compatibility

Client libraries that were compatible with RabbitMQ 3.8.x will be compatible with 3.9.0.

Upgrading to Erlang 24.x

RabbitMQ 3.8.16 and later require Erlang 23.2 and support Erlang 24 (recommended). Users of RabbitMQ versions that do not support Erlang 23.2 should first upgrade to the latest 3.8.x patch release on Erlang 23.2, then upgrade to 3.9.0, and ideally Erlang 24.

Management API

User tags are now exported as a list, not as a comma-separated list anymore. HTTP API client libraries need some updating to support both types. RabbitMQ 3.9 can import definitions in both formats.

Getting Help

Any questions about this release, upgrades or RabbitMQ in general are welcome on the RabbitMQ mailing list.

Changes Worth Mentioning

Release notes are kept under rabbitmq-server/release-notes.
Contributors are encouraged to update them together with their changes. This helps with release automation and more
consistent release schedule.

Core Server

Enhancements

• Streams, a new durable replicated data structure that allows for repeatable (non-destructive) consumption. RabbitMQ 3.9 introduces a new binary protocol for working with streams. While streams can be used by AMQP 0-9-1 clients, in most cases it will be used via a RabbitMQ stream binary protocol client. The new protocol exposes all the operations specific to this queue type and is heavily optimized for them, resulting in much higher read and write throughput.

  Java and Go clients

• Structural logging in JSON. Nodes can now log in JSON format.

  GitHub issue: #2861

• Peer discovery no longer uses randomized startup delay to ensure only one node starts as a seed one during initial (parallel) cluster formation. Instead it relies on an internal runtime locking library.

  GitHub issue: #3075

• Binding recovery on node startup is now more efficient. This means in clusters with a lot of queues and/or bindings, nodes start quicker.

  GitHub issue: #3137

• Classic mirrored queue replcas are now started synchronously.

  GitHub issue: #2751

• journald notifications are now performed using a library and not an external tool (socat)

  GitHub issue: #2940

• Startup banner now provides more information some of which can be customised.

  GitHub issues: #2247, #2777

• Per-protocol and queue type message rates are now available as Prometheus metrics, as well as Streams-specific errors. This also addresses a long-standing 3.8.x counters issue for most metrics of this type, as captured in #2783.

  GitHub issues: #3127, #3136, #3157

Internal API and Other Changes

• Standard runtime logger is now used for logging instead of Lager, a 3rd party dependency. In modern Erlang versions, the module offers at least a comparable feature set and efficiency of Lager.

  Users who relied on Lager-specific configuration, e.g. custom extensions or sinks, would have to rebuild them on top of the standard Erlang logger module.

  GitHub issue: #2861

• Different Raft-based features now can have different defaults and settings, with more configurability coming at a later point. For example, client ID tracking in the MQTT plugin can use settings for lower memory footprint and more frequent compaction while stream and quorum queues use settings more optimal for higher throughput on the same node.

  GitHub issue: #2909

• Continuous integration of open source RabbitMQ has switched to Bazel, GitHub Actions and BuildBuddy, resulting in much faster and incremental test runs.

  Bazel support for Erlang is a new project open sourced by the RabbitMQ Core team as a result.

• Process group membership now uses pg.

  GitHub issue: #2850

Bug Fixes

• powershell.exe now can run without profile

  GitHub issue: #2201

• Queue index recovery now can continue after a forced node shutdown during recovery (e.g. killed)

  GitHub issue: #3099

• Correct an issue where the num_acceptors.ssl configuration value was applied incorrectly

  GitHub issue: #3180

• Correct an issue where the --tags flag was ignored if --description was not specified in the rabbitmqctl add_vhost command

Kubernetes

Enhancements

• RabbitMQ Cluster Operator automates provisioning, management, and operations of RabbitMQ clusters running on Kubernetes.

• Messaging Topology Operator makes it possible to define RabbitMQ resources (virtual hosts, users, permissions, topologies, policies, etc) as Kubernetes objects.

• Kubernetes peer discovery mechanism no longer uses randomized delays to avoid the inherent race condition during initial cluster formation. Instead it relies on an internal distributed locking mechanism available in modern Erlang releases.

  While randomized startup delay was reasonably effective at ensuring only one cluster was formed even during parallel node startup, it by definitely assumes that with some low probability more than one node can still start as seed nodes, and be joined by different groups of cluster peers. The locking mechanism does not have this downside. Now that RabbitMQ requires Erlang 23.2, the internal locking library becomes an option for peer discovery.

  cluster_formation.randomized_startup_delay_range.min and cluster_formation.randomized_startup_delay_range.max configuration keys will no longer have any effect.

  GitHub issue: #3075

AWS Peer Discovery Plugin

Enhancements

• AWS peer discovery mechanism now supports Instance Metadata Service v2. In case it is not available, requests will fall back to the original metadata service endpoint.

  The v2 endpoint offers substantial security improvements and is one of the AWS best practices.

  Contributed by @thuandb (AWS).

  GitHub issue: #2952

Federation Plugin

Enhancements

• Quorum queues now can be federated.

  GitHub issue: #2804

Management Plugin

Bug Fixes

• Management UI now honors queue name length limits

  GitHub issue: #3202

• Management UI now sets a correct www-authenticate response header on authentication failures

  GitHub issue: #3205

Dependency Changes

• cowboy has been upgraded from 2.6.1 to 2.8.0
• cowlib has been upgraded from 2.7.0 to 2.9.1
• lager has been removed
• jsx has been upgraded from 2.9.0 to 3.1.0
• observer_cli has been upgraded from 1.5.2 to 1.6.2
• osiris had been introduced at 0.1.0
• prometheus has been upgraded from 4.6.0 to 4.8.1
• ra has been upgraded from 1.1.9 to 2.0.0
• ranch has been upgraded from 1.7.1 to 2.0.0
• recon has been upgraded from 2.5.0 to 2.5.1
• seshat had been introduced at 0.1.0
• socat has been removed
• syslog has been upgraded from 3.4.5 to 4.0.0
• systemd had been introduced at 0.6.1