Add a default rabbitmq.conf file to the image

[HoloRin]

This allows the default user to connect from outside the running container, which has been the case for the 3.8 series, but is not currently the case with the 3.9 rc.

Intended to address #500

[tianon]

I'm not sure providing a default config file is going to be a great experience for users; for example, if they want to add additional configuration to the provided default, they now have to first somehow extract this file from the image/container in order to modify it, and then they become responsible for updating it over time if it ever needs more changes.

Additionally, the file provided is 1000 lines of comments for a single line of configuration, and I think it'd be much more effective (here, at least, where users aren't likely to have an interactive editor inside their container when they're inspecting this configuration) to instead just link to the official configuration documentation if users want to set more values or provide additional customization.

If this is the route we want to go, I'd love to see the stock default rabbitmq.conf.example this was created from become part of the generic-unix distribution somewhere so we can copy it and programmatically modify the one value and thus not have to maintain another copy of that file (and update it manually over time). Alternatively if it doesn't make sense to add to the distribution tarball we download, then we should download it inline from a URL like https://github.com/rabbitmq/rabbitmq-server/raw/v3.9.0-rc.1/deps/rabbit/docs/rabbitmq.conf.example so that it's always correct for the version we're embedding (although that creates an issue with validating that we downloaded the correct thing and that it downloaded completely and successfully, so we'd likely prefer to also have versions.sh and our template/automation embed an expected checksum).

However, I'd reiterate that my preference would be to not provide a file by default, since it gives a much simpler user experience, especially for users who need to provide additional configuration (which is going to be a much larger number of users now, thanks to #424). Given my own naiveté with respect to RabbitMQ (especially secure configurations of RabbitMQ), I'd also suggest that this is a good time to question / discuss whether this is a good / safe default, even in container environments (where users are very, very frequently foot-gunning themselves by exposing their services publicly and not understanding that they've done so / that -p is short for "publish" not "port"). 😅

cc @michaelklishin @gerhard

[HoloRin]

@tianon thanks for your feedback on this.

I could easily be persuaded that the contents of rabbitmq.conf be replaced simply with:

## allow access to the guest user from anywhere on the network
loopback_users.guest = false

## see https://www.rabbitmq.com/configure.html for further information on configuring RabbitMQ

I wouldn't advise this as a default anywhere else but docker, so I would not expect to embed it in the distribution tarball. It's true it is not the most secure choice, but without it I expect many confused users that have encountered documentation for the 3.8 images and cannot connect.

The RabbitMQ startup sequence logs the location of the file, so my original though in including the full example was that a user who notices this log line would easily discover it. Very rarely (if ever) do we change rabbitmq configuration options such that once working config will fail to start, so I'm not too worried about users needing to maintain their own version of the file if they customize. However the example will go out of date more easily, and for that reason I'm not at all opposed to the short version.

[gerhard]

This is a great discussion, thanks for kicking it off @pjk25!

The first goal is to address the long command that users need to run in the context of rabbitmq:3.9.0-rc to get the same behaviour as they had before. The focus is on good & consistent user experience, or POLA (Principle Of Least Astonishment) as I've learned from FreeBSD via @dumbbell.

My understanding of the issue captured by @pjk25 in #500:

# this is what users do today
docker run -it --rm rabbitmq:3.8

# this is what users need to do from 3.9 onwards for the same end-result
docker run -it --rm -e RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS='-rabbit loopback_users "none"' rabbitmq:3.9-rc

While #467 was the right move - thank you @tianon & @yosifkit for seeing it through - I feel it is important to preserve the same simple user experience by default, as mentioned above.

Assuming that we agree on this point, I also think that a single .conf file hits a number of issues, as you've very well captured @tianon. The middle-ground that we have settled on in the commercial image is to use config fragments and place them in /etc/rabbitmq/conf.d, such as /etc/rabbitmq/conf.d/11-default_user.conf. This is what we end up with when running under K8s:

find /etc/rabbitmq

/etc/rabbitmq
/etc/rabbitmq/rabbitmq.conf
/etc/rabbitmq/enabled_plugins
/etc/rabbitmq/conf.d
/etc/rabbitmq/conf.d/11-default_user.conf
/etc/rabbitmq/conf.d/10-operatorDefaults.conf
/etc/rabbitmq/conf.d/90-userDefinedConfiguration.conf

Do you see the above approach working better in this context too @tianon @yosifkit?

For completeness, this is the full manifest that sets up a 3-node RabbitMQ cluster on K8s with rabbitmq/cluster-operator v1.8.0:

apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
  name: rabbitmq-1-1
  namespace: lre-tanzu-rabbitmq-1-x
spec:
  replicas: 3
  image: vmware-tanzu-rabbitmq:1.1.1
  imagePullSecrets:
    - name: dev-tanzu-registry
  resources:
    requests:
      cpu: 2
      memory: 4Gi
    limits:
      cpu: 2
      memory: 4Gi
  persistence:
    storageClassName: premium-rwo
    storage: 100Gi
  rabbitmq:
    additionalConfig: |
      cluster_partition_handling = pause_minority
      vm_memory_high_watermark_paging_ratio = 0.99
      disk_free_limit.relative = 1.0
    additionalPlugins:
    - rabbitmq_mqtt
    - rabbitmq_stomp

[HoloRin]

I hadn't realized that the conf.d approach was supported out of the box by RabbitMQ. However, my preference would be for the shortened rabbitmq.conf now in the PR, since any user who want to further customize the config would probably not want guest access on anyway. If we want to later expand upon the default config, then splitting it up would make sense to me.

[michaelklishin]

I'm afraid this is a necessary evil. We should add two more links to the comment:

• https://www.rabbitmq.com/access-control.html#loopback-users
• https://www.rabbitmq.com/production-checklist.html#users

There will be plenty of users who would never read that so perhaps a README note for this image that says that
default settings are not meant to be taken straight into production would be great. I assume it will show up on
Docker Hub as well.

[gerhard]

Looks good to me.

I would add a commented line which links to the canonical rabbitmq.conf - https://github.com/rabbitmq/rabbitmq-server/blob/v3.9.x/deps/rabbit/docs/rabbitmq.conf.example - but it's not a blocker for merging.

[gerhard]

We intend to ship RabbitMQ 3.9.0 final on Friday, 23rd of July and announce it on Monday, 26th of July.

Would it be possible to get this change in before then? I'm happy for this to be merged as is if you're happy @tianon @yosifkit.

I'll ping tomorrow when the 3.9.0 artefacts are ready for the Docker image to pick them up.

Thanks!

[gerhard]

I'm not sure if I've mentioned this in the past, but if it would help to make myself & @michaelklishin maintainers of this repo, we would gladly accept it 🙂

[tianon]

A few minor nits and it needs to be applied to all the generated files.

[tianon]

Oh, and this shouldn't be added to 3.8 -- this should only be added in 3.9+.

[gerhard]

After this is merged, it would be good to trigger the 3.9.0 final image: https://github.com/rabbitmq/rabbitmq-server/releases/tag/v3.9.0

[tianon]

Opened #505 for 3.9.0 GA. 👍