Explicitly set agent paths within --data-dir

[brandond]

Proposed Changes

Removing the cfg.DataDir mutation in 3e4fd7b did not break anything, but did change some paths in unwanted ways. Rather than mutating the user-supplied command-line flags (which we generally try to avoid doing), explicitly specify the agent subdirectory as needed.

Types of Changes

Regression fix

Verification

• Start k3s with k3s server --data-dir=/opt/k3s
• Confirm all items are correctly located: find /opt/k3s/agent/ -maxdepth 3:

/opt/k3s/agent/
/opt/k3s/agent/client-ca.crt
/opt/k3s/agent/server-ca.crt
/opt/k3s/agent/serving-kubelet.crt
/opt/k3s/agent/serving-kubelet.key
/opt/k3s/agent/client-kubelet.crt
/opt/k3s/agent/client-kubelet.key
/opt/k3s/agent/kubelet.kubeconfig
/opt/k3s/agent/client-kube-proxy.crt
/opt/k3s/agent/client-kube-proxy.key
/opt/k3s/agent/kubeproxy.kubeconfig
/opt/k3s/agent/client-k3s-controller.crt
/opt/k3s/agent/client-k3s-controller.key
/opt/k3s/agent/k3scontroller.kubeconfig
/opt/k3s/agent/etc
/opt/k3s/agent/etc/crictl.yaml
/opt/k3s/agent/etc/cni
/opt/k3s/agent/etc/cni/net.d
/opt/k3s/agent/etc/flannel
/opt/k3s/agent/etc/flannel/net-conf.json
/opt/k3s/agent/etc/containerd
/opt/k3s/agent/etc/containerd/config.toml
/opt/k3s/agent/containerd
/opt/k3s/agent/containerd/tmpmounts
/opt/k3s/agent/containerd/containerd.log
/opt/k3s/agent/containerd/io.containerd.content.v1.content
/opt/k3s/agent/containerd/io.containerd.content.v1.content/ingest
/opt/k3s/agent/containerd/io.containerd.content.v1.content/blobs
/opt/k3s/agent/containerd/io.containerd.snapshotter.v1.native
/opt/k3s/agent/containerd/io.containerd.snapshotter.v1.native/snapshots
/opt/k3s/agent/containerd/io.containerd.snapshotter.v1.overlayfs
/opt/k3s/agent/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots
/opt/k3s/agent/containerd/io.containerd.snapshotter.v1.overlayfs/metadata.db
/opt/k3s/agent/containerd/io.containerd.metadata.v1.bolt
/opt/k3s/agent/containerd/io.containerd.metadata.v1.bolt/meta.db
/opt/k3s/agent/containerd/io.containerd.runtime.v1.linux
/opt/k3s/agent/containerd/io.containerd.runtime.v2.task
/opt/k3s/agent/containerd/io.containerd.runtime.v2.task/k8s.io
/opt/k3s/agent/containerd/bin
/opt/k3s/agent/containerd/lib
/opt/k3s/agent/containerd/io.containerd.grpc.v1.cri
/opt/k3s/agent/containerd/io.containerd.grpc.v1.cri/sandboxes
/opt/k3s/agent/containerd/io.containerd.grpc.v1.cri/containers
/opt/k3s/agent/pod-manifests

Linked Issues

#2498
#2455 (comment)
rancher/rke2#474

Further Comments

Mea culpa @dweomer.

[dweomer]

@galal-hussein can you review this with an eye towards identifying possible other paths we may have missed here?

[galal-hussein]

LGTM

[galal-hussein]

@galal-hussein can you review this with an eye towards identifying possible other paths we may have missed here?

I dont think we have missed anything, but we should recheck etcd datadir stuff because its derived also from the data dir of the server

[brandond]

Confirmed that the etcd stuff is still in the right place:

/opt/k3s/server/db
/opt/k3s/server/db/etcd
/opt/k3s/server/db/etcd/name
/opt/k3s/server/db/etcd/config
/opt/k3s/server/db/etcd/member