Cant use as service in gitlab ci since 3.9

[kucix]

In gitlab-ci.yml we use rabbitmq as service that is configured via ENV props - gitlab does not allow volumes in services.

Example is like this

Acceptance tests:
  image: php
  stage: test
  variables:
    RABBITMQ_ERLANG_COOKIE: xxx""
    RABBITMQ_DEFAULT_USER: "guest"
    RABBITMQ_DEFAULT_PASS: "guest"
    RABBITMQ_DEFAULT_VHOST: "/xxx"
  services:
    - name: rabbitmq:3-management
      alias: rabbitmq
  script:
    - composer tests-acceptance

https://docs.gitlab.com/ee/ci/services/#available-settings-for-services

Now we fixed this by changing to rabbitmq:3.8-management, but what can we do for the future?
How can we configure default user/pass and default vhost?

[Diggsey]

I'm seeing a similar issue.

[wglambert]

That would be due to #467

Reasoning behind the change #506 (comment)

[kucix]

Yes I know, I have read it, but how can we use version 3.9 and later in gitlab ci as a service and configure it?
That is the question.

[michaelklishin]

This is a great question for the GitLab community. You have several options:

• Mount or download a rabbitmq.conf with the settings you need and the Erlang cookie file
• Alternatively the virtual host you set as pre-seeded (default) one can be created at any point using the HTTP API. Same goes for guest's permissions in said virtual host

Setting RABBITMQ_DEFAULT_USER and RABBITMQ_DEFAULT_PASS to guest is unnecessary since those are seed user defaults.

[michaelklishin]

Alternatively you can use definition import, both at boot time and after the node has started (with an HTTP API call), to set up any users, virtual hosts and permissions.

The Definitions doc guide uses rabbitmqadmin, there is nothing wrong
with using POST /api/definitions with a pre-defined definitions (JSON) file body.

[kucix]

• Mount or download a rabbitmq.conf with the settings you need and the Erlang cookie file

Yes, if I use rabbitmq as image in which scripts run.
But this is not that case.
In services section of gitlab-ci.yml is mounting not possible.

And if I get it correctly, guest can't connect "remotely" by default, thus can't set vhost via api etc.
Or can?

default:

# guest uses well known
# credentials and can only
# log in from localhost
# by default
loopback_users.guest = true

[joaomcarlos]

This change has broken hundreds of pipelines for us. Is there no easy setup with environment variables path anymore?

[michaelklishin]

@kucix I suggested using CLI tools or the HTTP API, neither of which has any restrictions for the default user.

You can use RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS to override rabbit.loopback_users:

docker run -it --rm --name rabbitmq -p 5672:5672 -p 15672:15672 -e RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS='-rabbit loopback_users "none"' rabbitmq:3.9-management

if really necessary.

[michaelklishin]

@joaomcarlos most environment variables that were specific to this image (not used by RabbitMQ itself) were intentionally removed.

Your options are outlined above in #508 (comment).

[michaelklishin]

One small feature in RabbitMQ itself that would simplify initial setup for tests would be introducing a load_definitions counterpart that uses a remote URL instead of a local file. That would instantly take care of users/virtual host/permission setup, or maybe even topologies if needed: rabbitmq/rabbitmq-server#3249

Assuming that an HTTP API call against the management plugin-enabled container is possible on GItLab,
I don't see why it would not be possible to import a basic definition file with users/virtual hosts/permissions without any additional features. Then simply use that user
in your suites, creating new users over remote guest access has been the recommended way for many years.

[gerhard]

Secrets as environment variables is a bad idea, contrary to what 12factor recommends.

RabbitMQ didn't really support these, which is why docker-entrypoint.sh used in this image became the most complex part to the point that most were afraid of changing it. More importantly, this was creating all sorts of weird issues between Docker and Kubernetes. The container startup command should not be generating any just-in-time config from environment variables, because these will clash with what the container scheduler may need to do, especially when it comes to rotating secrets, or declarative topologies (think GitOps).

The defaults work the same, meaning that user guest with pass guest and / vhost CAN login remotely. I just checked with rabbitmq:3-management. This default guest user config makes that possible:

rabbitmq/10-default-guest-user.conf

Line 8 in 56d547f

loopback_users.guest = false

If you need to change the default user or vhost, mount a config file.

If you can't mount a file, then declare config properties via RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS , as @michaelklishin describes. It's still possible, just harder, because it's not recommended. By the way, the Erlang cookie can be specified this way as well (again, it's bad idea, but it works).

Also, @michaelklishin proposal in rabbitmq/rabbitmq-server#3249 sounds like a sensible third solution for changing users, vhosts, permissions, etc. What do others think?

[eriksw]

Also, @michaelklishin proposal in rabbitmq/rabbitmq-server#3249 sounds like a sensible third solution for changing users, vhosts, permissions, etc. What do others think?

This is not useful unless it supports gs://, s3://, ... etc. (Using ambient creds to access those.)

ECS is a case where there's no way to inject a file, but where it is quite reasonable and well-supported to pass in secrets as environment variables.

Please put the brakes on these anti-environment-variable intentions.

Edit to add:

Please treat the removal of any particular environment variables as being just as breaking to end-users as the removal of a config key from the software itself.

[michaelklishin]

@eriksw why wouldn't downloads over HTTPS (rabbitmq/rabbitmq-server#3249) be sufficient for service container cases? What limits most users to S3 or similar?