Refactor KubeArmor Operator

[rksharma95]

Refactor KubeArmor Operator

• refactor kubearmorConfig keep it close to k8s manifest spec

  Current:

  kubearmorImage:
    image:
    imagePullPolicy
  ...  
  
  

  Proposed:

  kubearmor:
    image:
    args:
  
  kubearmorRelay:
    image:
    args:
  
  kubearmorController:
  ...
  

• avoid snitch's dependency for serviceaccount being created after KubeArmorConfig CR has been created, to avoid increasing time delay due to reconcilation.

• avoid seperate resource update for each configuration update

• optimize cert roatation logic

• discuss to decide if tight loop should be replaced with informer based reconcider. reconciler will require watch permission for managing k8s resources.

[carlosrodfern]

The change to the operator CRD is also desirable to be able to set the log level in the values.yaml in the helm chart for the controller: #1849

[carlosrodfern]

More details for the Proposal:

kubearmor:
  image:
     repository: ?
     tag: ?
     pullPolicy: ?
  args: []
  resources:
    limits:
      cpu: ?
      memory: ?
    requests:
      cpu: ?
      memory: ?

kubearmorRelay:
  image:
     repository: ?
     tag: ?
     pullPolicy: ?
  args: []
  resources:
    limits:
      cpu: ?
      memory: ?
    requests:
      cpu: ?
      memory: ?

kubearmorController:
  image:
     repository: ?
     tag: ?
     pullPolicy: ?
  args: []
  resources:
    limits:
      cpu: ?
      memory: ?
    requests:
      cpu: ?
      memory: ?

Perhaps even a global too?

global:
  image:
    pullPolicy: ?
    repository: ?

This is inspired on the typical pattern seen in helm values.

[bakito]

I'd also appreciate, if container resources (CPU/memory) of all operator managed resources could be defined in the CRD.
I tried to overwrite them manually, but the operator always remove them again.