Use Cilium instead of Metalllb for service loadbalancing

[mrueg]

Since this project is now using Cilium, it might be good next step to remove one additional component needed and reduce complexity in the deployment.

See also: https://dariomader.io/post/how_i_moved_from_metallb_to_cilium/

[onedr0p]

I would love to go this route and will do at home when I get Cilium running there. However L2 is still the most generic method of bringing up a cluster, adding BGP into the mix brings into issues with people who can't run BGP on their routers.

I will recommend people to go this route over L2 if possible but I'm not sure supporting it by default would be a good idea. And to let it be an option, adding different types of config options to the configure script is not really maintainable :/

See: #686

[onedr0p]

@mrueg I converted the Cilium ingress request into a discussion and I am closing this in favor of the following discussion:

#686