Add support for all kubearmor command line configurations using KubeArmorConfig CR

[rksharma95]

Feature Request

Short Description

Add Support to the KubeArmorConfig to pass all the command line configurations supported by the kubearmor. KubeArmorConfig already support some of the configurations i.e. defaultPosture, defaultVisibility etc. that can be in-effect dynamically. the remaining configurations are mainly pre-install configurations and will require KubeArmor to reboot. Operator is reconciling to watch kubearmor daemonset and can handle any configuration updates.

Is your feature request related to a problem? Please describe the use case.

it will enable the user to provide kubearmor configurations using KubeArmorConfig CR.

Describe the solution you'd like

apiVersion: operator.kubearmor.com/v1
kind: KubeArmorConfig
Spec:
  cluster: "cluster-name"
  host: "host-name"
  grpc: "grpc-port"
  ...

Describe alternatives you've considered

• feel free to discuss/propose spec design to support the kubearmor configurations.

[rootxrishabh]

Hey @rksharma95, KubeArmorConfig already handles defaultPosture and defaultVisibility as shown here. The goal is to implement the remaing configurations from kubearmor cli to kubearmor config. Is my interpretation correct?
Kubearmor CLI options (some to be implemented)-

  -bpfFsPath string
        Path to the BPF filesystem to use for storing maps (default "/sys/fs/bpf")
  -cluster string
        cluster name (default "default")
  -coverageTest
        enabling CoverageTest
  -criSocket string
        path to CRI socket (format: unix:///path/to/file.sock)
  -debug
        Enable/Disable pushing KubeArmor debug logs over gRPC. NOTE: Set environment DEBUG=true to configure stdout debug logging
  -defaultCapabilitiesPosture string
        configuring default enforcement action in global capability context {allow|audit|block} (default "audit")
  -defaultFilePosture string
        configuring default enforcement action in global file context {allow|audit|block} (default "audit")
  -defaultNetworkPosture string
        configuring default enforcement action in global network context {allow|audit|block} (default "audit")
  -defaultPostureLogs
        Default Posture Alerts (for Apparmor only) (default true)
  -enableKubeArmorHostPolicy
        enabling KubeArmorHostPolicy
  -enableKubeArmorPolicy
        enabling KubeArmorPolicy (default true)
  -enableKubeArmorVm
        enabling KubeArmorVM
  -enforcerAlerts
        ebpf alerts (default true)
  -gRPC string
        gRPC port number (default "32767")
  -host string
        host name (default "ubuntu-jammy-1")
  -hostDefaultCapabilitiesPosture string
        configuring default enforcement action in global capability context {allow|audit|block} (default "audit")
  -hostDefaultFilePosture string
        configuring default enforcement action in global file context {allow|audit|block} (default "audit")
  -hostDefaultNetworkPosture string
        configuring default enforcement action in global network context {allow|audit|block} (default "audit")
  -hostVisibility string
        Host Visibility to use [process,file,network,capabilities,none] (default "none" for k8s, "process,file,network,capabilities" for VM) (default "default")
  -k8s
        is k8s env? (default true)
  -kubeconfig string
        Paths to a kubeconfig. Only required if out-of-cluster.
  -logPath string
        log file path, {path|stdout|none} (default "none")
  -lsm string
        lsm preference order to use, available lsms [bpf, apparmor, selinux] (default "bpf,apparmor,selinux")
  -seLinuxProfileDir string
        SELinux profile directory (default "/tmp/kubearmor.selinux")
  -untrackedNs string
        Namespaces which are not being tracked, default untracked:[kube-system, kubearmor] (default "kube-system,kubearmor")
  -visibility string
        Container Visibility to use [process,file,network,capabilities,none] (default "process,file,network,capabilities")

[rootxrishabh]

Can you explain 'Operator is reconciling to watch kubearmor daemonset' a bit in detail? Is it that the operator using the watch API of Kubernetes and it will implement any changes that require restart automatically and the user won't have to manually reboot kubearmor?

[rksharma95]

for example, we are using the image provided by the user using the CR KubeArmorConfig here, similarly we can define other configuration options and can use them to define the daemonset initially.

KubeArmor/pkg/KubeArmorOperator/internal/controller/resources.go

Line 77 in e64ccb7

daemonset.Spec.Template.Spec.Containers[0].Image = common.GetApplicationImage(common.KubeArmorName)

and to update the daemonset for each update event on KubeArmorConfig we can make use of the watcher defined here:

KubeArmor/pkg/KubeArmorOperator/internal/controller/cluster.go

Line 227 in e64ccb7

func (clusterWatcher *ClusterWatcher) WatchConfigCrd() {

let me know if you have any questions.

[rootxrishabh]

To my understanding, the daemonset configuration has to be extended to handle these CLI configurations at install time. Also, the operator CRD will have to be extended to make these options available. Once at least these steps are done(there could be more) the watcher will automatically take care of updates on the CR. Is my interpretation correct?

[rksharma95]

daemonset accepts these configurations as command-line arguments, however it would be better if we can handle these configurations with existing configmap kubearmor-config.

KubeArmor/KubeArmor/config/config.go

Line 19 in e64ccb7

type KubearmorConfig struct {

yes you're right these configurations need to be part of KubeArmorConfig CR spec and need to be handled as part of this issue.
the watcher will required related changes to handle the change in these configs at runtime.

[rootxrishabh]

Working on a PR : )

[rootxrishabh]

@rksharma95 Sorry for an array of doubts 😅. I understand that we are getting the config map from kubearmor-confg but how does a user-supplied value(such as image or init-image) in the CR is used to configure the deployments in the operator? I am unable to find that code piece.
Thank You

[rksharma95]

reconciler/watcher WatchRequiredResource() watches all the resources except daemonset and same can be used to initialize resources i.e.

KubeArmor/pkg/KubeArmorOperator/internal/controller/resources.go

Line 445 in e64ccb7

// update images

for dynamic updates, let's take a look here how we're updating the images

KubeArmor/pkg/KubeArmorOperator/internal/controller/cluster.go

Line 327 in e64ccb7

func (clusterWatcher *ClusterWatcher) UpdateKubeArmorImages(images []string) error {

let me know if you have any questions.

[DelusionalOptimist]

Folks how about instead of adding all fields to the CR we just add a cli_arguments field? Most of this is advanced configuration anyways and I'm not sure if most of the users will be needing it. It'll just make our CR more verbose.
Just a thought. Need more opinions. cc @rksharma95 @rootxrishabh

[rootxrishabh]

@DelusionalOptimist Agreed. Advanced configurations can be bundled into an array field. Haven't had the time to get on this, will start again soon!