How to configure Argo for container runtime containerd?

[emichaf]

Argo version : v2.7.1

I'm using K8S clusters with both container runtimes docker and containerd in worker nodes.

In K8S cluster with docker everything works fine, but having problem running the same workflows in K8S with containerd, but it's not working!

tried with configmap config: containerRuntimeExecutor: k8sapi
but it did not help, any more config required (ex MountPath or similar) ?

[alexec]

can you add your logs please?

[emichaf]

invalid spec: templates.ciworkflow.steps[0].gerrit-checkout templates.gerrit-checkout.outputs.artifacts.source: k8sapi executor does not support outputs from base image layer. must use emptyDir" namespace=argo-events workflow=ci-build-easy2use-helloworld-via-eiffel-event-kt7t2

##########################
workflow-controller-configmap
config:
containerRuntimeExecutor: k8sapi
artifactRepository:
s3:
bucket: argo-artifacts
endpoint: minio.argo-events:9000
insecure: true
accessKeySecret:
name: minio
key: accesskey
secretKeySecret:
name: minio
key: secretkey

########################
workflow-controller-xxx

time="2020-04-14T18:56:12Z" level=info msg="Alloc=4898 TotalAlloc=3027095 Sys=70848 NumGC=4789 Goroutines=101"
time="2020-04-14T18:59:10Z" level=info msg="Processing workflow" namespace=argo-events workflow=ci-build-easy2use-helloworld-via-eiffel-event-kt7t2
time="2020-04-14T18:59:10Z" level=info msg="Updated phase -> Running" namespace=argo-events workflow=ci-build-easy2use-helloworld-via-eiffel-event-kt7t2
time="2020-04-14T18:59:10Z" level=info msg="Updated phase Running -> Failed" namespace=argo-events workflow=ci-build-easy2use-helloworld-via-eiffel-event-kt7t2
time="2020-04-14T18:59:10Z" level=info msg="Updated message -> invalid spec: templates.ciworkflow.steps[0].gerrit-checkout templates.gerrit-checkout.outputs.artifacts.source: k8sapi executor does not support outputs from base image layer. must use emptyDir" namespace=argo-events workflow=ci-build-easy2use-helloworld-via-eiffel-event-kt7t2
time="2020-04-14T18:59:10Z" level=info msg="Marking workflow completed" namespace=argo-events workflow=ci-build-easy2use-helloworld-via-eiffel-event-kt7t2
time="2020-04-14T18:59:10Z" level=info msg="Checking daemoned children of " namespace=argo-events workflow=ci-build-easy2use-helloworld-via-eiffel-event-kt7t2
time="2020-04-14T18:59:10Z" level=info msg="Workflow update successful" namespace=argo-events phase=Failed resourceVersion=81143647 workflow=ci-build-easy2use-helloworld-via-eiffel-event

[emichaf]

#1256

K8s API

• secure. cannot escape privileges of pod's service account
• no extra configuration

• least scalable - log retrieval and container polling is done against k8s API server
• can only save params/artifacts in volumes (e.g. emptyDir), and not the base image layer (e.g. /tmp)

Trying pns.

• secure. cannot escape privileges of service account
• artifact collection can be collected from base image layer
• scalable - process polling is done over procfs and not kubelet/k8s API

• processes will no longer run with pid 1
• artifact collection from base image may fail for containers which complete too fast
• cannot capture artifact directories from base image layer which has a volume mounted under it
• immature

[emichaf]

containerRuntimeExecutor: pns was working with container runtime containerd without the need of updating the workflows, so now the workflows are working with both docker & containerd!