Predetermined passwords for Cluster

[n3wscott]

Is your feature request related to a problem? Please describe.
It seems I am not able to make a password for rabbitMQ, I use what is generated by the operator?

Describe the solution you'd like
I know ahead of time what the auth url will be for a broker.

Describe alternatives you've considered
An additional operator and logic to deeply understand what this operator produces. Which is not ideal.

Additional context
I am unable to do late-bound config for a broker and a deployment, the secrets after reconciliation means I need to always do a two phase deployment and I need to wait for the secret to show up for the second apply.

[michaelklishin]

See Seeding Users and Permissions in RabbitMQ documentation.

[n3wscott]

/reopen

The linked docs do not talk about how to do this with the operator.

[n3wscott]

Plus, given this method: https://github.com/rabbitmq/cluster-operator/blob/main/internal/resource/admin_secret.go#L45-L67 it looks like there is no option to configure the username and password with the operator.

[michaelklishin]

Using RabbitMQ Cluster Operator explains how to retrieve generated credentials by inspecting a secret but not how to override it. This Federation plugin example covers the same thing.

@mkuratczyk how should those who want to override default seeded credentials approach this in this Operator's view, is the use of existing rabbitmq.conf settings discouraged? I assume definition import is not (but it's more friction for this particular problem). What about additional/per application users?

[ChunyiLyu]

@michaelklishin @n3wscott

I think this is a limitation of the current implementation. If users import definitions through statefulSet override, admin credentials will be set from the imported configuration. However, a generated secret object still exists for this cluster, and will not contain the updated credential from the definitions json, but an auto generated one. So yes you can configure admin credential using override, but you will get a secret object with wrong content, and that's confusing to users. We can consider adding better support for this use case @mkuratczyk

[michaelklishin]

We have seen this before on Cloud Foundry. The fundamental question is whether RabbitMQ itself is treated as the source of truth for users, virtual hosts, permissions or the platform it is deployed to should be considered the source. The latter option means that some RabbitMQ features effectively cannot do their job because the platform won't be aware of certain changes performed by RabbitMQ itself (mutli-DC schema sync plugin is one recent example, "no defaul seeding" if a definitions file is used also falls into this category).

For cases where RabbitMQ is deployed to Kubernetes but otherwise provisioned "the traditional way", the current feature set makes more sense.

For cases where RabbitMQ is deployed as a dependency of a Kubernetes-specific service such as Riff, it does not and there should be a way to define key RabbitMQ resources as Kubernetes objects. This part is currently missing.

@ChunyiLyu since we can't know the credentials imported from the definition file, maybe the Operator could skip creating the default user secret. Is this something that can be done or the Operator does not have enough information?

[n3wscott]

Or let me provide the secret.

[n3wscott]

The fundamental question is whether RabbitMQ itself is treated as the source of truth for users, virtual hosts, permissions or the platform it is deployed to should be considered the source.

For me, this is an easy question: Kubernetes is the source of truth. I do not want to learn or use RabbitMQ APIs. I want to automate the management of the infrastructure through custom resources and config. I would prefer disabling the direct broker management API in RabbitMQ. And if the password is changed, the operator should set it back to whatever is in the secret and hopefully the user account list (how ever that ends up working).

[michaelklishin]

This is a very Kubernetes-centric view. This means that potentially every single RabbitMQ feature or plugin has to be one way or another recreated as a Kubernetes resource. It also doesn't go very well with the philosophy of several messaging protocols that applications manage their topologies (and thus declare certain resources that Kubernetes will never be aware of).

We have plenty of evidence spanning years that some users want to deploy RabbitMQ to a PaaS but do not want to or cannot do everything via the PaaS features, in some cases because those features do not exist and won't be developed any time soon. RabbitMQ for PCF had the same philosophy and ended up scaling it back in several areas, or was abandoned by some users in favor of off-platform RabbitMQ installations. The same was mostly true for other services AFAIK. The philosophy was nice on slide decks but did not survive actual contact with real-world users and their imperfect realities and practices.

Speaking of user and permission management specifically, the PaaS can be the source of truth for some (in many cases, all) user credentials. "Sideloading" via definitions will still be possible but we can assume that if you use definition import for users and permissions, those users and permissions will be used by off-platform applications.

[gerhard]

@n3wscott is from Knative, we are all on different parts of the same page.

This is related to the RabbitMQ for Knative topic which is being actively discussed, expect more details to come forward in the coming weeks.

Please keep this issue open, the ask is valid.

cc @st3v @markfisher @bsnchan @edbyford

[SayakMukhopadhyay]

One way to solve this would be to allow to load the various config files as secrets. The kube-prometheus project uses Grafana configuration in a similar way. You can look up the code at https://github.com/brancz/kubernetes-grafana/blob/master/grafana/grafana.libsonnet#L195.

Essentially, this means one needs to create a secret base64 encoding the entire config file. The operator can load this secret, decode and mount it in the correct location.

The reason for this idea is that there are many other config properties that are secret apart from the user password, like enabling OAuth plugin needs client id and stuff which should be secret too.

[github-actions]

This issue has been marked as stale due to 60 days of inactivity. Stale issues will be closed after a further 30 days of inactivity; please remove the stale label in order to prevent this occurring.

[github-actions]

Closing stale issue due to further inactivity.

[SayakMukhopadhyay]

@gerhard Do you think that this issue should be reopened?

[gerhard]

@SayakMukhopadhyay I don't think so. All these concerns are starting to appear in https://github.com/rabbitmq/messaging-topology-operator. I believe rabbitmq/messaging-topology-operator#44 addresses this particular issue. Let me know if that's not what you need.

[mark-schaal]

@gerhard is the expected outcome that administrators will use cluster-operator to deploy an initial rmq cluster with autogenerated secrets, and then orchestrate messaging-topology-operator on top for user configs from custom K8s secrets?