Cannot exec into a privileged container - glusterfs

[zrahui]

Is this a request for help?: Yes

---

Is this an ISSUE or FEATURE REQUEST? (choose one): Issue

---

What version of acs-engine?: 0.12.5

---

Orchestrator and version (e.g. Kubernetes, DC/OS, Swarm)
Kubernetes 1.9.1

What happened:
Cannot exec into glusterfs pods is preventing the installation of GlusterFS into ACS-Engine enabled cluster. This used to work but i believe #1961 has killed this.

What you expected to happen:
Successfully install GlusterFS using the gluster-kubernetes repo.

How to reproduce it (as minimally and precisely as possible):
Run the ./gk-deploy scripts as part of the gluster-kubernetes repo to provision Gluster into the cluster.

Anything else we need to know:
Ideally, I'd like to know if this can be disabled for specific pods to enable GlusterFS to be deployed. Any help is much appreciated.
We've also got issues with using hostNetwork as part of that same repo which seems to kill DNS in an Azure CNI enabled cluster but that's another issue altogether....

[zrahui]

FYI: I've managed to remove the DenyEscalatingExec flag from the apiserver yaml to workaround this for now

[pidah]

@zrahui you can customize/override the admission-controller flags passed to the API server using the apiServerConfig option as follows:

  "orchestratorProfile": {
      "orchestratorType": "Kubernetes",
      "orchestratorRelease": "1.8",
      "kubernetesConfig": {
        "apiServerConfig": {
          "--admission-control":  "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota,AlwaysPullImages"
        }
      }
    } 

[zrahui]

Thanks @pidah that's a more elegant solution 👍

[jalberto]

@zrahui this requires a new deployment. How to fix it without creating a new cluster?

[huydinhle]

@jalberto you can only change it on the master nodes at /etc/kubenetes/manifest