3.9-rc updates

[yosifkit]

Adjust management images for 3.9: fixes #466
Also combine the management RUN lines into a single RUN

Edit: this also drops the ability to use the Docker-specific environment variable to generate a config in 3.9. Users should instead provide a config file. Here is the location that rabbitmq will look: /etc/rabbitmq/rabbitmq.conf (or .config for the erlang syntax file).

These are the specific variables that will no longer work:

RABBITMQ_DEFAULT_PASS
RABBITMQ_DEFAULT_PASS_FILE
RABBITMQ_DEFAULT_USER
RABBITMQ_DEFAULT_USER_FILE
RABBITMQ_DEFAULT_VHOST
RABBITMQ_MANAGEMENT_SSL_CACERTFILE
RABBITMQ_MANAGEMENT_SSL_CERTFILE
RABBITMQ_MANAGEMENT_SSL_DEPTH
RABBITMQ_MANAGEMENT_SSL_FAIL_IF_NO_PEER_CERT
RABBITMQ_MANAGEMENT_SSL_KEYFILE
RABBITMQ_MANAGEMENT_SSL_VERIFY
RABBITMQ_SSL_CACERTFILE
RABBITMQ_SSL_CERTFILE
RABBITMQ_SSL_DEPTH
RABBITMQ_SSL_FAIL_IF_NO_PEER_CERT
RABBITMQ_SSL_KEYFILE
RABBITMQ_SSL_VERIFY
RABBITMQ_VM_MEMORY_HIGH_WATERMARK

[yosifkit]

This PR should wait until we have a 3.9 (alpha/beta/rc) release to test and be updated to include it.

[tianon]

👀

[michaelklishin]

Thank you. We already have alphas of 3.9 but I'd wait until the first beta indeed. How plugins are loaded won't change before it but some other significant PRs are still to be finished and merged first.

[michaelklishin]

We will have an 3.9 beta very soon and filed #491 to make sure it can be built/tagged the same way RCs can be already.

[tianon]

I've just updated this with today's beta, but as mentioned over in #424, I'd love to resolve that and actually remove the deprecated entrypoint bits here if there's agreement to do so.

[yosifkit]

Rebased to pull in #424 and #492.

So, do we switch this new 3.9 series to drop most of the entrypoint or keep the warnings for 3.9.x and drop in 3.10 (or 4.0)?

[michaelklishin]

@yosifkit I'd say let's drop for 3.9.

[yosifkit]

Should we have it print and exit out if it gets any of the old variables, just print a warning, or silently ignore them?

Edit: added commit is the simplest of silently ignoring all environment variables (but still keeping the longname detection that wasn't deprecated: #424 (comment))

[michaelklishin]

@yosifkit assuming we have a changelog, we can document the behavior change and ignore them.

[yosifkit]

The only "changelog" options we currently have are the commits to this repo or a note to the Docker Hub page.

[tianon]

IMO our best option here is to hard-code a list of variables that if any are set, we error out with a deprecation notice -- the implementation of that should be pretty trivial (the harder bit is generating the list of variables we need to error out on, but that's not too bad). Then in 3.9 + 1 (3.10?) we can remove completely? (or even just after a suitable amount of time / number of patch releases)

[yosifkit]

Is docker-library/official-images#10425 similar to the expected setup users will need in order to use rabbitmq with TLS without our env vars?

Obviously, they should better manage the ssl keys via secrets/config maps and not build them into an image.

EDIT: Can our test be simplified or improved in some way?

[michaelklishin]

@edbyford @Zerpet this is the PR that should enable Docker image builds of 3.9 previews.

[michaelklishin]

@tianon I agree that hardcoding a list of known variables that are now deprecated would be the pragmatic thing to do.

[michaelklishin]

@yosifkit that test change looks reasonable. I'll leave it to @Zerpet @ansd and @gerhard to provide any container best practices but they will have to mount/generate/produce from secrets some key/value files, and specify their paths in rabbitmq.conf.

For inter-node communication, using a separate file in the classic config format is what we usually recommend but what the tests above do is acceptable.

[Zerpet]

Is docker-library/official-images#10425 similar to the expected setup users will need in order to use rabbitmq with TLS without our env vars?

Yes, that's right. Agreed that the best way to get the certificates inside the container would be via Docker/Kubernetes secrets. For local development, it should be possible to generate the certificates locally and mount the directory inside the container.

Can our test be simplified or improved in some way?

I can't think of anything to simplify the tests, they look good to me.

[yosifkit]

Latest commit adds a check to the 3.9-rc entrypoint to print errors and then exit if any old environment values are found. In a later release it will be adjusted to just print a warning and eventually not even check.

This does seem a bit backwards from a usual deprecation but I think this will be more likely to be noticed (a flapping container) versus having unapplied config and the symptom of being unable to connect.

[michaelklishin]

@yosifkit thank you, that is a firm guidance in the right direction but I think it's appropriate. I'd change the wording a bit to make it say something like "please use a configuration file instead, visit https://www.rabbitmq.com/configure.html to learn more". A doc guide link should greatly help folks learn about the recommended option.

I'd also be happy to produce an "equivalence table" of sorts for the env variables we deprecate and the rabbitmq.conf (or advanced.config) settings that do the same thing. We then can put it into the change log.

[tianon]

Makes sense (I've updated the PR accordingly)!

The conversion matrix is pretty straightforward:

• RABBITMQ_DEFAULT_PASS: default_pass
• RABBITMQ_DEFAULT_USER: default_user
• RABBITMQ_DEFAULT_VHOST: default_vhost
• RABBITMQ_MANAGEMENT_SSL_CACERTFILE: management.ssl.cacertfile
• RABBITMQ_MANAGEMENT_SSL_CERTFILE: management.ssl.certfile
• RABBITMQ_MANAGEMENT_SSL_DEPTH: management.ssl.depth
• RABBITMQ_MANAGEMENT_SSL_FAIL_IF_NO_PEER_CERT: management.ssl.fail_if_no_peer_cert
• RABBITMQ_MANAGEMENT_SSL_KEYFILE: management.ssl.keyfile
• RABBITMQ_MANAGEMENT_SSL_VERIFY: management.ssl.verify
• RABBITMQ_SSL_CACERTFILE: ssl_options.cacertfile
• RABBITMQ_SSL_CERTFILE: ssl_options.certfile
• RABBITMQ_SSL_DEPTH: ssl_options.depth
• RABBITMQ_SSL_FAIL_IF_NO_PEER_CERT: ssl_options.fail_if_no_peer_cert
• RABBITMQ_SSL_KEYFILE: ssl_options.keyfile
• RABBITMQ_SSL_VERIFY: ssl_options.verify
• RABBITMQ_VM_MEMORY_HIGH_WATERMARK: vm_memory_high_watermark.*

[tianon]

Thanks @michaelklishin 👍

I've rebased for hopefully the last time before merge! 😄

[gerhard]

3.9.0-rc.1 shipped on a Friday, as expected 😎

I'm wondering if we will be able to share the Docker version - rabbitmq:3.9-rc - at RabbitMQ Summit 2021

Happy weekend everyone 🙌🏻

[yosifkit]

Triggered the update bot and then opened the PR to official images: docker-library/official-images#10514. Should be able to merge it soon.

[gerhard]

Thank you @yosifkit for making rc.1 available in time for the RabbitMQ Summit 2021. Even though I can't see how many downloads this tag had, I'm sure it was easy for a few try it out. I know for a fact that @motmot80 gave it a quick check 👍🏻

3.9.0-rc.2 was made public early today. Any chance we could trigger an rc.2 image? Thanks!

By the way, this release calendar keeps track of what is expected to happen next. Here's a point-in-time snapshot:

[gerhard]

Thank you!