fix(core): filter watched nodes and pods server-side #1676
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose of PR?:
Rather than each DaemonSet pod listing/watching every node and pod in the cluster, this tells Kubernetes which node and pods we're interested in for each Daemonset, reducing overhead significantly in larger clusters.
Does this PR introduce a breaking change? Yes
If the changes in this PR are manually verified, list down the scenarios covered: Tested changes in fork of KubeArmor, currently running in production.
Additional information for reviewer? :
This PR does change the hostname behavior in KubeArmor slightly. KubeArmor used to watch all nodes in the cluster and then filter them by which ones had the same hostname prefix, but that still requires listing all nodes, so the only way to filter nodes server-side is if we can tell Kubernetes exactly which node we're interested in.
KubeArmor's previous logic for this was to check the
KUBEARMOR_NODENAME
environment variable first (set by the Kubernetes configuration, pulling from afieldRef
), and if that's not set, default to the truncated OS hostname to determine the node to watch. In my experience, a node's OS hostname has always mapped to the node's name in Kubernetes, and so to continue not relying strictly onKUBEARMOR_NODENAME
being set, I've stopped truncating the OS hostname in KubeArmor.This has one other major effect which is alerting/monitoring where the node's hostname is logged. Now, rather than logging just the truncated hostname, it will log the full hostname, which in the cases where OS hostnames match up exactly with Kubernetes node names, will be very helpful with correlating unexpected behaviors to specific nodes.
One thing that I haven't done but that might make sense is to check
KUBEARMOR_NODENAME
as an override for the default hostname globally, while still allowing the end-user to override it with a CLI argument if necessary. This means we'd keep theKUBEARMOR_NODENAME
checks where they are in this PR too in case and end-user did override the CLI argument, sinceKUBEARMOR_NODENAME
is intended to be the definitive Kubernetes node name for the pod, whereas thecfg.GlobalCfg.Host
variable is used for lots of logging as well. This more depends on if, in situations where the Kubernetes node name differs from the OS hostname, the end-user would prefer a Kubernetes node name to be logged or an OS hostname.Checklist:
<type>(<scope>): <subject>