feat: support CRI-O

[DelusionalOptimist]

Signed-off-by: Rudraksh Pareek [email protected]

Fixes #666

• This PR makes KubeArmor compatible with CRI-O container runtime.
• The approach used is similar to that of containerd with some changes here and there.

Tasks:

• Implement monitoring
• Tests
• Setup scripts

[codecov-commenter]

Codecov Report

Merging #697 (72993b5) into main (7e1810d) will decrease coverage by 1.61%.
The diff coverage is 9.56%.

@@            Coverage Diff             @@
##             main     #697      +/-   ##
==========================================
- Coverage   39.30%   37.69%   -1.62%     
==========================================
  Files          24       25       +1     
  Lines        8702     8997     +295     
==========================================
- Hits         3420     3391      -29     
- Misses       4828     5158     +330     
+ Partials      454      448       -6     

Impacted Files	Coverage Δ
KubeArmor/core/containerdHandler.go	3.35% <0.00%> (-12.47%)	⬇️
KubeArmor/core/crioHandler.go	0.00% <0.00%> (ø)
KubeArmor/core/kubeUpdate.go	48.30% <0.00%> (-0.16%)	⬇️
KubeArmor/core/kubeArmor.go	49.04% <20.83%> (-5.39%)	⬇️
KubeArmor/config/config.go	87.12% <75.00%> (-1.22%)	⬇️
KubeArmor/common/common.go	41.66% <83.33%> (+1.06%)	⬆️
KubeArmor/core/dockerHandler.go	47.95% <100.00%> (-4.09%)	⬇️
KubeArmor/enforcer/appArmorEnforcer.go	48.29% <100.00%> (ø)
... and 3 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7e1810d...72993b5. Read the comment docs.

[DelusionalOptimist]

@nam-jaehyun @nyrahul @daemon1024 Please give this a review. (I'm unable to request reviews 😅 ).
I've temporarily modified the gh workflow so that k3s runs with crio and tests can happen. Github tests 6, 8 & 13 are failing and I'll take a look into them in the meantime.
Also, please terminate https://github.com/kubearmor/KubeArmor/actions/runs/2360471345

[DelusionalOptimist]

Github test 06 and 08 passed eventually.
However, github test 13 didn't pass because crio by default doesn't permit the capability CAP_NET_RAW and it has to be first enabled by editing default capabilities in /etc/crio/crio.conf. What should we do here? 🤔

Also, in local tests, multiubuntu test 02 and 07 are failing. 07 fails due to the same reason as above but I'm not sure about 02 as audit action works fine when tested manually. Here are logs from karmor when the test conditions are created -

== Alert / 2022-05-22 15:42:57.985910 ==
Cluster Name: default
Host Name: kubearmor-dev
Namespace Name: multiubuntu
Pod Name: ubuntu-4-deployment-566bf47cd7-25dx6
Container ID: 72dbd9ab9ec9e2b997ac809087e228479f6f8e8df035b52e1ff251e8f5bb414f
Container Name: ubuntu-4-container
Policy Name: ksp-group-2-proc-path-audit
Severity: 4
Type: MatchedPolicy
Source: /bin/bash
Operation: Process
Resource: /bin/sleep 1
Data: syscall=SYS_EXECVE
Action: Audit
Result: Passed

For ref: the default capabilties applied by CRI-O:

default_capabilities = [
        "CHOWN",
        "DAC_OVERRIDE",
        "FSETID",
        "FOWNER",
        "SETGID",
        "SETUID",
        "SETPCAP",
        "NET_BIND_SERVICE",
        "KILL",
]

[DelusionalOptimist]

@nam-jaehyun @nyrahul as discussed, I've updated the github workflows to test for all the runtimes and skip the tests that have been specifically failing only for crio.

[nyrahul]

@nam-jaehyun @nyrahul as discussed, I've updated the github workflows to test for all the runtimes and skip the tests that have been specifically failing only for crio.

Hey, I see that tests are still failing! Also can you please squash the commits?

[DelusionalOptimist]

Hey, I see that tests are still failing! Also can you please squash the commits?

Oof yes. Taking a look into why the tests with containerd are failing 👀

[nam-jaehyun]

can you also update contribution/vagrant/Vagrantfile?

cri-o needs to work under the following commands.
KubeArmor/KubeArmor$ make vagrant-up RUNTIME=crio
KubeArmor/KubeArmor$ make vagrant-up RUNTIME=crio NETNEXT=1
if possible, KubeArmor/KubeArmor$ make vagrant-up RUNTIME=crio OS=centos

[DelusionalOptimist]

@nam-jaehyun @nyrahul

I and @daemon1024 were thinking about extending the use of cri-api and using it for docker and containerd as well. The current code for handling the two runtimes would be deprecated and the code used here would be used to communicate with the runtimes' socket.
It is possible as kubernetes mandates a cri friendly interface for every runtime. Containerd implements the cri plugin which it needs to communicate with kubelet and docker has dockershim (or cri-dockerd now). So since both of these are compatible with cri we can save the effort of maintaining code for their respective APIs.

Does this sound good?

[nyrahul]

@nam-jaehyun @nyrahul

I and @daemon1024 were thinking about extending the use of cri-api and using it for docker and containerd as well. The current code for handling the two runtimes would be deprecated and the code used here would be used to communicate with the runtimes' socket. It is possible as kubernetes mandates a cri friendly interface for every runtime. Containerd implements the cri plugin which it needs to communicate with kubelet and docker has dockershim (or cri-dockerd now). So since both of these are compatible with cri we can save the effort of maintaining code for their respective APIs.

Does this sound good?

Imo, it sounds very good. But my understanding was that we are using some containerd or docker specific apis which are not part of cri-api spec. If this understanding is wrong, what you saying is the best way to handle things i.e. just use cri-api for everything ... This will simplify lot of our program logic. But I am keen on what @nam-jaehyun has to say.

[daemon1024]

LGTM

[nyrahul]

Great PR!! Please check my comments inline.

[nyrahul]

@DelusionalOptimist , can you squash the commits?

[DelusionalOptimist]

@nyrahul done 👍

[nyrahul]

LGTM 👍🏼