.ambassador_ignore_crds files written to root and breaks non writeable setups

[n1koo]

Describe the bug
The files that are used for signaling existance of CRDs are written to the root device. This breaks non-rw containers

These files: https://github.com/datawire/ambassador/blob/c61e9c2f2e7a9f32a79095fbbbcd2babf5723d22/ambassador/kubewatch.py#L152-L173 in https://github.com/datawire/ambassador/blob/c61e9c2f2e7a9f32a79095fbbbcd2babf5723d22/ambassador/kubewatch.py#L196

2019-06-08 09:06:25 kubewatch [9 TMainThread] 0.71.0 DEBUG: CRD type definition not found for modules.getambassador.io
Traceback (most recent call last):
  File "/ambassador/kubewatch.py", line 212, in <module>
    main()
  File "/usr/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/ambassador/kubewatch.py", line 196, in main
    Path(touchfile).touch()
  File "/usr/lib/python3.6/pathlib.py", line 1238, in touch
    fd = self._raw_open(flags, mode)
  File "/usr/lib/python3.6/pathlib.py", line 1046, in _raw_open
    return self._accessor.open(self, flags, mode)
  File "/usr/lib/python3.6/pathlib.py", line 387, in wrapped
    return strfunc(str(pathobj), *args)
PermissionError: [Errno 13] Permission denied: '.ambassador_ignore_crds'

To Reproduce
Steps to reproduce the behavior:

1. Run the containers without write privs on root and without CRDs

Expected behavior
The files should be written to eg. AMBASSADOR_CONFIG_BASE_DIR which people already should have writable

Versions (please complete the following information):

• Ambassador: 0.71.0
• Kubernetes 1.34

Additional context
In general would be great if all these features would be tested with a read only FS setup as this is the Nth time a change has broken our setup and having more security on edge makes sense for most folks. I'll try to see if I have time to PR a change to the test suite.

[kflynn]

Ouch, sorry! Even if you don’t have time to do the PR, if you can share how you’re setting up your pods, I can make sure it’s included.

[n1koo]

We are running a PodSecurityPolicy with

spec:
  allowPrivilegeEscalation: false
  fsGroup:
    ranges:
    - max: 65535
      min: 1
    rule: MustRunAs
  readOnlyRootFilesystem: true

which basically means non-root, no writeable layer in the pod (except for outside volumes)

[richarddli]

Fixed in 0.72