No ability to specify a PDB?

[roy-work]

Describe the bug

The EMQX custom resource doesn't seem to have a means to specify a PDB, and none is otherwise created.

To Reproduce
Steps to reproduce the behavior:

1. Create an EMQX resource. (An EMQX "cluster"?)

Expected behavior

No PDB is created.

"No PDB" is Kubernetes means the pods are freely disruptable; i.e. nothing prevent Kubernetes from simply evicting all of the pods, together, at the same time. This is then an outage, of course.

A PDB specifies how the pods can be disrupted; the correct values usually depend on the nature of the pods, and how they interact. Some things (e.g., APIs) just need one pod up, some things (e.g., databases) need something like a quorum.

For EMQX, the "right" values probably depend on how it distributes data amongst its pods. I've not figured that out. But, e.g., a Raft system might require, say, maxAvailable: 51%, as it needs to maintain quorum.

Because disruption is a result of how the underlying application works, I'd lean towards having a reasonable default first; there shouldn't be a need to override this¹, since this is a property of how the application works, and not so much operator preference.

Environment details::

• Kubernetes version: v1.26.13-gke.1144000
• Cloud-provider/provisioner: GCP/GKE
• emqx-operator version: v2.1.1
• Install method: Helm

Anything else we need to know?:

Just an infra eng trying to learn this. The applications devs that deployed EMQX seem to believe that it goes down/causes outages, in particular, when the underlying machines are changing in some manner. Best I can tell, this is partly due to not have a PDB.

¹But for pragmatic practical reasons, it can sometimes be useful to do so.

[Rory-Z]

Great idea, I will finish it.

[Rory-Z]

Hi @roy-work the EMQX operator 2.2.18 has been released, could you please try it?

[roy-work]

@Rory-Z Thanks; I'll ask our devs if this is something they can do.

I see you added a PDB with minAvailable: 1; since EMQX is essentially a stateful service, it raises the question: how does EMQX replicate data between nodes? (There is some information here, but it doesn't really go into specifics.)

[Rory-Z]

how does EMQX replicate data between nodes? (There is some information here, but it doesn't really go into specifics.)

Sorry, I am not an expert in erlang. @id could you please take an answer