Support AutomountServiceAccountToken #1480

dtaniwaki · 2019-07-15T02:37:51Z

I may be going to use this feature in the future, so implemented it.

xianlubird · 2019-07-15T06:00:34Z

LGTM

xianlubird · 2019-07-15T07:06:43Z

If AutomountServiceAccountToken is set false, you will manual add a sa to the pod ?

dtaniwaki · 2019-07-15T07:22:16Z

When Argo workflow is used in a single namespace through another service in which a user provides only Docker image and command to execute, (we actually have this kind of internal system, but it doesn’t use Argo right now) , the service account should not be put in the main container because it’s so strong that the user can manipulate other user’s pods.

xianlubird · 2019-07-15T07:30:56Z

Yep, I know. This's very useful

dtaniwaki · 2019-07-15T08:08:56Z

I realized this setting doesn't work in the Resource template type. We may want to validate it and throw an error.

dtaniwaki · 2019-07-15T15:20:17Z

I added validation to check whether exec containers can consume kubernetes API even if automountServiceAccountToken is false.

dtaniwaki · 2019-07-15T16:33:25Z

Do we actually need KubeConfig if we can specify a service account token as I implemented in this PR?

xianlubird · 2019-07-16T06:09:45Z

Actual, in the pod , we don't need kubeConfig

jessesuen · 2019-07-16T20:09:58Z

When running pods with Service account, it's helpful if the container has service account token automatically mounted. Currently it's possible to specify only serviceAccountName.

This statement from the original issue description doesn't seem correct to me. Don't all containers (including the main container) automatically get the service account token mounted?

I understand that this can be actually undesirable to give main container any sort of API access, and that we should have a feature where the main container doesn't get the token mounted, but I think the default behavior is that it is behaving as expected. Am I missing something?

dtaniwaki · 2019-07-17T05:47:00Z

@jessesuen I read the original issue again. You're right. A service account token should be put in every container by default.

we should have a feature where the main container doesn't get the token mounted

This is what I wanted. As the default value of AutomountServiceAccountToken is nil and doesn't change the current behavior, my implementation should be fine. I'm thinking that ServiceAccountTokenName should be moved to Workflow manifests instead of the controller config and can be used if ServiceAccountName is not set. What do you think?

jessesuen · 2019-07-22T21:14:14Z

I'm thinking that ServiceAccountTokenName should be moved to Workflow manifests instead of the controller config and can be used if ServiceAccountName is not set. What do you think?

Yes, I agree. I think the less central configuration, the better. We're finding central config doesn't tend to work well in our multi-tenant clusters.

How about we provide the following controls:

spec:
  automountServiceAccountToken: false

and

spec:
  templates:
  - name: foo
    automountServiceAccountToken: false

The above would simply do a simple carry-forward of the value to the pod spec. NOTE that basic workflows that don't have outputs, should still work properly with automountServiceAccountToken: false (e.g. helloworld).

However, to handle the use case of not mounting the service account on the main container/sidecars, we would additionally introduce executorServiceAccountTokenName:

spec:
  automountServiceAccountToken: false
  executorServiceAccountTokenName: argoexec-sa

The above would result in the pod spec being created with spec.automountServiceAccountToken set to be false in the pod, but with a volumeMount of the service account token from argoexec-sa mounted under /var/run/secrets/kubernetes.io/serviceaccount/token of the init and wait sidecars.

If the user specifies the following, the result should be that the main container and sidecars get the default service account token mounted, but the init and wait containers get argoexec-sa mounted:

spec:
  automountServiceAccountToken: true # or omitted
  executorServiceAccountTokenName: argoexec-sa

Similarly, they could specify different service accounts for the main/sidecars vs. init/wait:

spec:
  serviceAccountTokenName: foo
  executorServiceAccountTokenName: argoexec-sa

dtaniwaki · 2019-07-23T05:52:57Z

Sounds good. I'll implement it like so. 👍

dtaniwaki · 2019-07-23T07:43:35Z

spec:
  automountServiceAccountToken: false

I found the first option doesn't work because clientset is initialized in the executor.
https://github.com/argoproj/argo/blob/270aabf1d8cabd69b9851209ad5d9c874348e21d/cmd/argoexec/commands/root.go#L75

Here's the error log of the executor.

panic: invalid configuration: no configuration has been provided

goroutine 1 [running]:
github.com/argoproj/argo/cmd/argoexec/commands.checkErr(0x17073c0, 0xc00000da20)
        /go/src/github.com/argoproj/argo/cmd/argoexec/commands/root.go:110 +0xb6
github.com/argoproj/argo/cmd/argoexec/commands.initExecutor(0x5d2425)
        /go/src/github.com/argoproj/argo/cmd/argoexec/commands/root.go:70 +0x6f
github.com/argoproj/argo/cmd/argoexec/commands.waitContainer(0x0, 0x0)
        /go/src/github.com/argoproj/argo/cmd/argoexec/commands/wait.go:26 +0x2e
github.com/argoproj/argo/cmd/argoexec/commands.NewWaitCommand.func1(0xc000355b80, 0x22ce090, 0x0, 0x0)
        /go/src/github.com/argoproj/argo/cmd/argoexec/commands/wait.go:16 +0x22
github.com/spf13/cobra.(*Command).execute(0xc000355b80, 0x22ce090, 0x0, 0x0, 0xc000355b80, 0x22ce090)
        /go/src/github.com/spf13/cobra/command.go:766 +0x2cc
github.com/spf13/cobra.(*Command).ExecuteC(0xc000355400, 0x0, 0x0, 0x0)
        /go/src/github.com/spf13/cobra/command.go:852 +0x2fd
github.com/spf13/cobra.(*Command).Execute(0xc000355400, 0x0, 0xc00041ee70)
        /go/src/github.com/spf13/cobra/command.go:800 +0x2b
main.main()
        /go/src/github.com/argoproj/argo/cmd/argoexec/main.go:17 +0x27

dtaniwaki · 2019-07-24T02:04:46Z

@jessesuen I updated the PR for ExecutorServiceAccountTokenName. Would you review it?

…-account

dtaniwaki · 2019-08-06T05:16:40Z

@jessesuen I resolved the conflicts with the master. Please review it again.

jessesuen

Please revert change to hello-world.yaml. Otherwise LGTM!

jessesuen · 2019-08-06T09:43:40Z

examples/hello-world.yaml

@@ -10,3 +10,5 @@ spec:
      image: docker/whalesay:latest
      command: [cowsay]
      args: ["hello world"]
+  automountServiceAccountToken: false
+  executorServiceAccountTokenName: argo-token-9rkvd


Seems like leftover from testing. Can we revert this please?

Oops, I'm removing it now...

jessesuen · 2019-08-06T10:54:59Z

pkg/apis/workflow/v1alpha1/workflow_types.go

+	AutomountServiceAccountToken *bool `json:"automountServiceAccountToken,omitempty"`
+
+	// ExecutorServiceAccountTokenName specifies a service account token name for executor containers.
+	ExecutorServiceAccountTokenName string `json:"executorServiceAccountTokenName,omitempty"`


@dtaniwaki - i was thinking, instead of executorServiceAccountTokenName, should this reference a service account name instead (i.e. executorServiceAccountName) and we would have the controller look up the the secret name from the service account for the volume mount? (similar to how serviceAccountName works)

This seems like it would be more user-friendly since users do not generally create tokens, they create service accounts. Note that this would require expanding the privileges of the controller to have read permissions on service accounts.

Agree. I will change it.

If we use ExecutorServiceAccountName instead of ExecutorServiceAccountTokenName, do you think AutomountServiceAccountToken still make sense to only disable automount service account token on the main container?

FYI
Found the corresponded code in Kubernetes

https://github.com/kubernetes/kubernetes/blob/b8eecd671d7d895ecd4f0fc6930b82e55298e257/plugin/pkg/admission/serviceaccount/admission.go#L327-L349

If we use ExecutorServiceAccountName instead of ExecutorServiceAccountTokenName, do you think AutomountServiceAccountToken still make sense to only disable automount service account token on the main container?

Yes. I think the following should disable mounting of service accounts entirely (in both main and wait):

spec: automountServiceAccountToken: false

and the following will disable it on main, but mount a different service account for the executor:

spec: automountServiceAccountToken: false executorServiceAccountName: foo

And the following would use separate service accounts for main and wait

spec: serviceAccountName: bar executorServiceAccountName: foo

Sounds good.
I realized we need to add read permission for service accounts to the cluster role.

For the following case:

spec: automountServiceAccountToken: false

We can't run a workflow because the current executor needs a service account token during the initialization, so I made it cause an error during validation.

jessesuen

Requesting change to move from executorServiceAccountTokenName to executorServiceAccountName

…okenName

sarabala1979 · 2019-08-07T18:26:32Z

I fixed style issue on your branch

sarabala1979

LGTM

dtaniwaki · 2019-08-08T05:49:06Z

I'm thinking if I should change ExecutorServiceAccountName to Executor.ServiceAccountName (creating Executor config to accommodate more settings for executors like SecurityContext.)... Is it too much for now?

dtaniwaki · 2019-08-08T06:06:03Z

workflow/common/serviceaccount.go

+)
+
+// GetServiceAccountTokenByAccountName returns the name of the first referenced secret which is a ServiceAccountToken for the service account
+func GetServiceAccountTokenByAccountName(clientset kubernetes.Interface, namespace, name string) (*corev1.Secret, error) {


Just FYI: I referenced the Kubernetes code to write the functions in this file.
https://github.com/kubernetes/kubernetes/blob/b8eecd671d7d895ecd4f0fc6930b82e55298e257/plugin/pkg/admission/serviceaccount/admission.go#L327-L349

sarabala1979 · 2019-08-08T07:07:25Z

it is a good idea. Let we keep simple for this release and create a refactor issue for this change and we can target for 2.5.

jessesuen · 2019-08-08T19:39:57Z

I'm thinking if I should change ExecutorServiceAccountName to Executor.ServiceAccountName (creating Executor config to accommodate more settings for executors like SecurityContext.)... Is it too much for now?

I agree this would be better structured.

it is a good idea. Let we keep simple for this release and create a refactor issue for this change and we can target for 2.5.

However I don't think we should introduce a spec change in v2.4, only to break that spec in v2.5, so I think this improved structure should go into v2.4

…-account

dtaniwaki · 2019-08-13T04:23:14Z

@sarabala1979
Sorry, I overwrote your commit of #1480 (comment)...

jessesuen

Please remove the Get/List privileges for secrets, and instead volume mount the secret name gathered from the service account's .secret list.

apiVersion: v1
kind: ServiceAccount
metadata:
  creationTimestamp: "2019-08-20T18:26:32Z"
  name: foobar
  namespace: default
  resourceVersion: "395208"
  selfLink: /api/v1/namespaces/default/serviceaccounts/foobar
  uid: 091f2460-c378-11e9-97ea-025000000001
secrets:
- name: foobar-token-kv2mt

jessesuen · 2019-08-20T18:40:35Z

manifests/cluster-install/workflow-controller-rbac/workflow-controller-clusterrole.yaml

+  resources:
+  - serviceaccounts
+  - serviceaccounts/finalizers
+  - secrets


We don't need get/list privileges for secrets. We only need to know the name of the secret for the purposes of volume mounting, which is already in the service account object:

$ k get sa foobar -o yaml apiVersion: v1 kind: ServiceAccount metadata: creationTimestamp: "2019-08-20T18:26:32Z" name: foobar namespace: default resourceVersion: "395208" selfLink: /api/v1/namespaces/default/serviceaccounts/foobar uid: 091f2460-c378-11e9-97ea-025000000001 secrets: - name: foobar-token-kv2mt <<<<<<<<<< here

I realized we still need secret for the workflow persistence... I think we should add the role separately in the manifests.

jessesuen · 2019-08-20T18:41:52Z

manifests/cluster-install/workflow-controller-rbac/workflow-controller-clusterrole.yaml

+  - ""
+  resources:
+  - serviceaccounts
+  - serviceaccounts/finalizers


I don't think you need to add /finalizers for the core types. /finalizers was originally added for workflow CRD to support openshift, but I don't think it is necessary for core types.

jessesuen · 2019-08-20T18:43:55Z

workflow/controller/workflowpod.go

+			Name: common.ServiceAccountTokenVolumeName,
+			VolumeSource: apiv1.VolumeSource{
+				Secret: &apiv1.SecretVolumeSource{
+					SecretName: token.Name,


As mentioned, we do not have any use for the actual token value. We only need to know the secret name containing the token so that we can volume mount it. The secret name is information already available from the service account object. By removing Get/List on secret, we improve the security of the controller.

dtaniwaki · 2019-08-21T04:24:15Z

@jessesuen I have updated the code. Please check it.

@shanesiebken

* Validate ArchiveLocation artifacts (argoproj#1167) * Update README and preview notice in CLA. * Update README. (argoproj#1173) (argoproj#1176) * Argo users: Equinor (argoproj#1175) * Do not mount unnecessary docker socket (argoproj#1178) * Issue argoproj#1113 - Wait for daemon pods completion to handle annotations (argoproj#1177) * Issue argoproj#1113 - Wait for daemon pods completion to handle annotations * Add output artifacts to influxdb-ci example * Increased S3 artifact retry time and added log (argoproj#1138) * Issue argoproj#1123 - Fix 'kubectl get' failure if resource namespace is different from workflow namespace (argoproj#1171) * Refactor Makefile/Dockerfile to remove volume binding in favor of build stages (argoproj#1189) * Add Docker Hub build hooks * Add documentation how to use parameter-file's (argoproj#1191) * Issue argoproj#988 - Submit should not print logs to stdout unless output is 'wide' (argoproj#1192) * Fix missing docker binary in argoexec image. Improve reuse of image layers * Fischerjulian adds ruby to rest docs (argoproj#1196) * Adds link to ruby kubernetes library. * Links to a ruby example on how to start a workflow * Updated OWNERS (argoproj#1198) * Update community/README (argoproj#1197) * Issue argoproj#1128 - Use polling instead of fs notify to get annotation changes (argoproj#1194) * Minor spelling, formatting, and style updates. (argoproj#1193) * Dockerfile: argoexec base image correction (fixes argoproj#1209) (argoproj#1213) * Set executor image pull policy for resource template (argoproj#1174) * Add schedulerName to workflow and template spec (argoproj#1184) * Issue argoproj#1190 - Fix incorrect retry node handling (argoproj#1208) * fix dag retries (argoproj#1221) * Executor can access the k8s apiserver with a out-of-cluster config file (argoproj#1134) Executor can access the k8s apiserver with a out-of-cluster config file * Update README with typo fixes (argoproj#1220) * Update README.md (argoproj#1236) * Remove extra quotes around output parameter value (argoproj#1232) Ensure we do not insert extra single quotes when using valueFrom: jsonPath to set the value of an output parameter for resource templates. Signed-off-by: Ilias Katsakioris <[email protected]> * Update README.md (argoproj#1224) * Include stderr when retrieving docker logs (argoproj#1225) * Add Gardener to "Who uses Argo" (argoproj#1228) * Add feature to continue workflow on failed/error steps/tasks (argoproj#1205) * Fix the Prometheus address references (argoproj#1237) * Fixed Issue#1223 Kubernetes Resource action: patch is not supported (argoproj#1245) * Fixed Issue#1223 Kubernetes Resource action: patch is not supported This PR is fixed the Issue#1223 reported by @shanesiebken . Argo kubernetes resource workflow failed on patch action. --patch or -p option is required for kubectl patch action. This PR is including the manifest yaml as patch argument for kubectl. This Fix will support the Patch action in Argo kubernetes resource workflow. This Fix will support only JSON merge strategic in patch action * udpated formating * typo, executo -> executor (argoproj#1243) * Issue#1165 fake outputs don't notify and task completes successfully (argoproj#1247) * Issue#1165 fake outputs don't notify and task completes successfully This PR is addressing the Issue#1165 reported by @alexfrieden. Issue/Bug: Argo is finishing the task successfully even artifact /file does exist. Fix: Validate the created gzip contains artifact or file. if file/artifact doesn't exist, Current step/stage/task will be failed with log message . Sample Log: ''' INFO[0029] Updating node artifact-passing-lkvj8[0].generate-artifact (artifact-passing-lkvj8-1949982165) status Running -> Error INFO[0029] Updating node artifact-passing-lkvj8[0].generate-artifact (artifact-passing-lkvj8-1949982165) message: failed to save outputs: File or Artifact does not exist. /tmp/hello_world.txt INFO[0029] Step group node artifact-passing-lkvj8[0] (artifact-passing-lkvj8-1067333159) deemed failed: child 'artifact-passing-lkvj8-1949982165' failed namespace=default workflow=artifact-passing-lkvj8 INFO[0029] node artifact-passing-lkvj8[0] (artifact-passing-lkvj8-1067333159) phase Running -> Failed namespace=default workflow=artifact-passing-lkvj8 ''' * fixed gometalinter errcheck issue * Git cloning via SSH was not verifying host public key (argoproj#1261) * Update versions (argoproj#1218) * Proxy Priority and PriorityClassName to pods (argoproj#1179) * Error running 1000s of tasks: "etcdserver: request is too large" argoproj#1186 (argoproj#1264) * Error running 1000s of tasks: "etcdserver: request is too large" argoproj#1186 This PR is addressing the feature request argoproj#1186. Issue: Nodestatus element keeps growing for big workflow. Workflow will fail once the workflow total size reachs 1 MB (maz size limit in ETCD) . Solution: Compressing the Nodestatus once size reachs the 1 MB which increasing 60% to 80% more steps to execute in compress mode. Latest: Argo cli and Argo UI will able to decode and print nodestatus from compressednoode. Limitation: Kubectl willl not decode the compressedNode element * added Operator.go * revert the testing yaml * Fixed the lint issue * fixed * fixed lint * Fixed Testcase * incorporated the review comments * Reverted the change * incorporated review comments * fixing gometalinter checks * incorporated review comments * Update pod-limits.yaml * updated few comments * updated error message format * reverted unwanted files * Reduce redundancy pod label action (argoproj#1271) * Add the `mergeStrategy` option to resource patching (argoproj#1269) * This adds the ability to pass a mergeStrategy to a patch resource. this is valuable because the default merge strategy for kubernetes is 'strategic', which does not work with Custom Resources. * This also updates the resource example to demonstrate how it is used * Fix bug with DockerExecutor's CopyFile (argoproj#1275) The check to see if the source path was in the tgz archive was wrong when source path was a folder, the arguments to strings.Contains were inverted. * Add workflow labels and annotations global vars (argoproj#1280) * Argo CI is current inactive (argoproj#1285) * Issue#896 Workflow steps with non-existant output artifact path will succeed (argoproj#1277) * Issue#896 Workflow steps with non-existant output artifact path will succeed Issue: argoproj#897 Solution: Added new element "optional" in Artifact. The default is false. This flag will make artifact as optional and existence check will be ignored if input/output artifact has optional=true. Output Artifact ( optional=true ): Artifact existence check will be ignored during the save artifact in destination and continued workflow Input Artifact ( optional=true ): Artifact exist check will be ignored during load artifact from source and continued workflow * added end of line * removed unwanted whitespace * Deleted test code * go formatted * added formatting directives * updated Codegen * Fixed format on merge conflict * format fix * updated comments * improved error case * Fix for Resource creation where template has same parameter templating (argoproj#1283) * Fix for Resource creation where template has same parameter templating This PR will enable to support the custom template variable reference. Soulltion: Workflow variable reference resolve will check the Workflow variable prefix. * added test * fixed gofmt issue * fixed format * fixed gofmt on common.go * fixed testcase * fixed gofmt * Added unit testcase and documented * fixed Gofmt format * updated comments * Admiralty: add link to blog post, add user (argoproj#1295) * Add dns config support (argoproj#1301) * Speed up podReconciliation using parallel goroutine (argoproj#1286) * Speed up podReconciliation using parallel goroutine * Fix make lint issue * put checkandcompress back * Add community meeting notes link (argoproj#1304) * Add Karius to users in README.md (argoproj#1305) * Added support for artifact path references (argoproj#1300) * Added support for artifact path references Adds new `{{inputs.artifacts.<NAME>.path}}` and `{{outputs.artifacts.<NAME>.path}}` placeholders. * Add support for init containers (argoproj#1183) * Secrets should be passed to pods using volumes instead of API calls (argoproj#1302) * Secrets should be passed to pods using downward API instead of API calls * Fixed Gogfmt format * fixed file close Gofmt * updated review comments * fixed gofmt * updated review comments * CheckandEstimate implementation to optimize podReconciliation (argoproj#1308) * CheckandEstimate implementation * fixed variable rename * fixed gofmt * fixed feedbacks * Update operator.go * Update operator.go * Add alibaba cloud to officially using argo list (argoproj#1313) * Refactor checkandEstimate to optimize podReconciliation (argoproj#1311) * Refactor checkandEstimate to optimize podReconciliation * Move compress function to persistUpdates * Fix formatting issues in examples documentation (argoproj#1310) * Fix nil pointer dereference with secret volumes (argoproj#1314) * Archive location should conditionally be added to template only when needed * Fix SIGSEGV in watch/CheckAndDecompress. Consolidate duplicate code (resolves argoproj#1315) * Implement support for PNS (Process Namespace Sharing) executor (argoproj#1214) * Implements PNS (Process Namespace Sharing) executor * Adds limited support for Kubelet/K8s API artifact collection by mirroring volume mounts to wait sidecar * Adds validation to detect when output artifacts are not supported by the executor * Adds ability to customize executor from workflow-controller-configmap (e.g. add environment variables, append command line args such as loglevel) * Fixes an issue where daemon steps were not getting terminated properly * Reorganize manifests to kustomize 2 and update version to v2.3.0-rc1 * Update v2.3.0 CHANGELOG.md * Export the methods of `KubernetesClientInterface` (argoproj#1294) All calls to these methods previously generated a panic at runtime because the calls resolved to the default, panic-always implementation, not to the overrides provided by `k8sAPIClient` and `kubeletClient`. Embedding an exported interface with unexported methods into a struct is the only way to implement that interface in another package. When doing this, the compiler generates default, panic-always implementations for all methods from the interface. Implementors can override exported methods, but it's not possible to override an unexported method from the interface. All invocations that go through the interface will come to the default implementation, even if the struct tries to provide an override. * Update README.md (argoproj#1321) * Issue1316 Pod creation with secret volumemount (argoproj#1318) * CheckandEstimate implementation * fixed variable rename * fixed gofmt * fixed feedbacks * Fixed the duplicate mountpath issue * Support parameter substitution in the volumes attribute (argoproj#1238) * `argo list` was not displaying non-zero priorities correctly * Fix regression where argoexec wait would not return when podname was too long * wait will conditionally become privileged if main/sidecar privileged (resolves argoproj#1323) * Update version to v2.3.0-rc2. Update changelog * Add documentation on releasing * Fix missing template local volumes, Handle volumes only used in init containers (argoproj#1342) * Fix argoproj#1340 parameter substitution bug (argoproj#1345) Also create podParams map in substitutePodParams Signed-off-by: Ilias Katsakioris <[email protected]> * add / test (argoproj#1240) * Fix input artifacts with multiple ssh keys (argoproj#1338) * Fixed : Validate the secret credentials name and key (argoproj#1358) * CheckandEstimate implementation * fixed variable rename * fixed gofmt * fixed feedbacks * Fixed Issue1355 * fixed style * Delete e2e_temp.tmp * Fix: # 1328 argo submit --wait and argo wait quits while workflow is running (argoproj#1347) * CheckandEstimate implementation * fixed variable rename * fixed gofmt * fixed feedbacks * Fixed argo submit --wait and argo wait quits while workflow is running * fixed Style * Update version to v2.3.0-rc3 * Update release instructions * Add --status filter for get command (argoproj#1325) * Support an easy way to set owner reference (argoproj#1333) * Use golangci-lint instead of deprecated gometalinter (argoproj#1335) * [Fix argoproj#1242] Failed DAG nodes are now kept and set to running on RetryWorkflow. (argoproj#1250) * Fixed : CLI Does Not Honor metadata.namespace argoproj#1288 (argoproj#1352) * Validate action for resource templates (argoproj#1346) * Fix issue where a DAG with exhausted retries would get stuck Running (argoproj#1364) * Update README.md (argoproj#1372) Add Adevinta. https://www.adevinta.com/ * Update docs for the v2.3.0 release and to use the stable tag * Add Max Kelsen to USERS in README.md (argoproj#1374) Max Kelsen us utilising Argo throughout the organisation to manage data processing and machine learning pipelines. Incredibly thankful to the great community! * Fixed: Support hostAliases in WorkflowSpec argoproj#1265 (argoproj#1365) * Fixed : Support hostAliases in WorkflowSpec argoproj#1265 * Fixed: failed to save outputs: verify serviceaccount default:default has necessary privileges (argoproj#1362) Fixed: failed to save outputs: verify serviceaccount default:default has necessary privileges (argoproj#1362) * Fixed: make verify-codegen is failing on the master branch (argoproj#1399) (argoproj#1400) * Fixed: withParam parsing of JSON/YAML lists argoproj#1389 (argoproj#1397) * Make locating kubeconfig in example os independent (argoproj#1393) * Added Argo Rollouts to README (argoproj#1388) * Add Mirantis as an official user (argoproj#1401) * Update README.md (argoproj#1402) * Update README.md (argoproj#1404) Includes SAP Fieldglass in users section. * Fiixed: persistentvolumeclaims already exists argoproj#1130 (argoproj#1363) * Fixed: persistentvolumeclaims already exists argoproj#1130 * chore: add IBM to official users section in README.md (argoproj#1409) * Orders uses alphabetically (argoproj#1411) * Update OWNERS (argoproj#1429) * Typo fix in ARTIFACT_REPO.md (argoproj#1425) In the non-default artifact repo section, when showing the gcs example the bucket name said 'my-aws-bucket-name'. I've updated this to say 'my-gcs-bucket-name'. Super minor change but I've been banging my head against artifact repo outputs all day and this was bothering me. * Add OVH as official user (argoproj#1417) Add OVH as official user * Update demo.md (argoproj#1396) Step 2 instructs the user to create the namespace `argo`, and the coin-flip (at least) uses the service account `argo`, so it makes sense to provide `--serviceaccount=argo:argo` so that the initial experience works, "out of the box". * Fix typo (argoproj#1431) * PNS executor intermitently failed to capture entire log of script templates (argoproj#1406) * Terminate all containers within pod after main container completes (argoproj#1423) Resolves argoproj#1422 * Ability to configure hostPath mount for `/var/run/docker.sock` (argoproj#1419) * CheckandEstimate implementation * fixed variable rename * fixed gofmt * fixed feedbacks * implement the configurable Docker sock path * Update workflowpod.go * Style updated * Fixed: Implemented Template level service account (argoproj#1354) Fixed: Implemented Template level service account (argoproj#1354) * Add paging function for list command (argoproj#1420) * Add paging function for list command (argoproj#1420) * Revert "Update demo.md (argoproj#1396)" (argoproj#1433) This reverts commit 5635c33. * Update documentation for workflow.outputs.artifacts (argoproj#1439) * Improve bash completion (argoproj#1437) * Add threekit to user list (argoproj#1444) * Fix demo's doc issue of install minio chart (argoproj#1450) Signed-off-by: Aisuko <[email protected]> * mention sidecar in failure message for sidecar containers (argoproj#1430) * Centralized Longterm workflow persistence storage (argoproj#1344) * Centralized Longterm workflow persistence storage implementaion * New Feature: provide failFast flag, allow a DAG to run all branches of the DAG (either success or failure) (argoproj#1443) * Fix bug: dag will missing some nodes when another branch node fails * Add test file * New Feature: provide failFast flag, allow a DAG to run all branches of the DAG (either success or failure) * Move failFast flag to DAG template spec * * Move test case file to test/e2e/expectedfailures since it is expected to fail * Remove unused check code * issue-1445: changing temp directory for output artifacts from root to tmp (argoproj#1458) * Support PodSecurityContext (argoproj#1463) * Add doc about failFast feature (argoproj#1453) * Added Codec to the Argo community list (argoproj#1477) * fix typo: symboloic > symbolic (argoproj#1478) * Add --no-color flag to logs (argoproj#1479) * Fix failFast bug: When a node in the middle fails, the entire workflow will hang (argoproj#1468) * Document the insecureIgnoreHostKey git flag (argoproj#1483) * Fix: 1008 `argo wait` and `argo submit --wait` should exit 1 if workflow fails (argoproj#1467) Fix: 1008 `argo wait` and `argo submit --wait` should exit 1 if workflow fails (argoproj#1467) * Update OWNERS (argoproj#1485) * Add Commodus Tech as official user (argoproj#1484) * Fix: Argo CLI should show warning if there is no workflow definition in file argoproj#1486 Fix: Argo CLI should show warning if there is no workflow definition in file argoproj#1486 * Exposed workflow priority as a variable (argoproj#1476) * Fix argoproj#1366 unpredictable global artifact behavior (argoproj#1461) * Fix: Support the List within List type in withParam argoproj#1471 (argoproj#1473) Fix: Support the List within List type in withParam argoproj#1471 (argoproj#1473) * Fix a compiler error (argoproj#1500) Fix a compiler error (argoproj#1500) * Readme update to add argo and airflow comparison (argoproj#1502) * Added argo vs airflow presentation * Update README.md * change 'continue-on-fail' example to better reflect its description (argoproj#1494) * Implemented Conditionally annotate outputs of script template only when consumed argoproj#1359 (argoproj#1462) * Fixed argoproj#1359 Implemented Conditionally annotate outputs of script template only when consumed * Allow output parameters with .value, not only .valueFrom (argoproj#1336) Fixed argoproj#1329 Allow output parameters with .value, not only .valueFrom (argoproj#1336) * Fix the lint target (argoproj#1505) Fix the lint target This fixes an issue with the `make lint` target, where if a developer has golangci-lint installed and also has linter errors, the linter fails with an error, causing the next case to fall through (and the old linter is run). This also fixes all of the linter errors that had somehow cropped up in the repo. * Fix a compiler error in a unit test (argoproj#1514) * Allow Makefile variables to be set from the command line (argoproj#1501) This changes the assignment operator of various Makefile variables from the recursive expansion operator (=) to the conditional assignment operator (?=), such that a developer can define their own values for those variables. This is highly valuable for a dev who wants to do local development with a local docker image * Fixed argoproj#1287 Executor kubectl version is obsolete (argoproj#1513) Fixed argoproj#1287 Executor kubectl version is obsolete (argoproj#1513) * Fix issue [Documentation] kubectl get service argo-artifacts -o wide (argoproj#1516) * Allow overriding workflow labels in 'argo submit' (argoproj#1475) * Support git shallow clones and additional ref fetches (argoproj#1521) Implemented a `depth` field for git artifact configuration that, when specified, will result in a shallow clone (and fetch) of the given number of commits from the branch tip. Implemented a `fetch` field for git artifact configuration that fetches the given refspecs prior to checkout. This is necessary when one wants to retrieve git revisions that exist in non-branch/-tag refs. The motivation for these features is to support retrieval of patchset refs from Gerrit code review (`refs/changes/[n]/[change]/[patch]`) but these new fields should provide more flexibility to anyone integrating with other git-based systems. * Add --dry-run option to `argo submit` (argoproj#1506) * Fix validation (argoproj#1508) * Implemented support for WorkflowSpec.ArtifactRepositoryRef (argoproj#1350) This change allows the workflow to specify the reference the configMap holding the artifact repository configuration. * Fix argo logs empty content when workflow run in virtual kubelet env (argoproj#1201) * Expose all input parameters to template as JSON (argoproj#1488) * WorkflowTemplate CRD (argoproj#1312) * Added Architecture doc (argoproj#1515) Fixed argoproj#894 Added Architecture doc (argoproj#1515) * Format sources and order imports with the help of goimports (argoproj#1504) * Update ISSUE_TEMPLATE.md (argoproj#1528) edit to follow to current README.md installation guides. * Introduce podGC strategy for deleting completed/successful pods (argoproj#1234) * Update CHANGELOG for v2.4 (argoproj#1531) * Update README.md (argoproj#1533) * Use cache to retrieve WorkflowTemplates (argoproj#1534) * Update argo dependencies to kubernetes v1.14 (argoproj#1530) * Update argo dependencies to kubernetes v1.14 * Update version to v2.4.0-rc1 * Update main.go (argoproj#1536) * Update main.go (argoproj#1536) * Remove GLog config from argo executor (argoproj#1537) * Remove GLog config from argo executor (argoproj#1537) * Initialize the wfClientset before using it (argoproj#1548) * docs(readme): fix workflow types link (argoproj#1560) * Optimize argo binary install documentation (argoproj#1563) * Document workflow controller dockerSockPath config (argoproj#1555) * Add coverage make target (argoproj#1557) * Fix issue saving outputs which overlap paths with inputs (argoproj#1567) * Support AutomountServiceAccountToken and executor specific service account(argoproj#1480) * added DataStax as an organization that uses Argo (argoproj#1576) * Fix inputs and arguments during template resolution (argoproj#1545) * Add entrypoint label to workflow default labels (argoproj#1550) * remove redundant codes (argoproj#1582) Signed-off-by: xiechengsheng <[email protected]> * Fix workflow template in namespaced controller (argoproj#1580) * Add workflow template permissions to namespaced deployment manifests * Use filtered shared informer factory for namespaced deployment * Regard resource templates as leaf nodes (argoproj#1593) This enables retryStrategy to be respected on resource templates. This closes argoproj#1370 * Update from github.com/ghodss/yaml to sigs.k8s.io/yaml (argoproj#1572) * Update Gopkg.toml and Gopkg.lock (argoproj#1596) * Issue1571 Support ability to assume IAM roles in S3 Artifacts (argoproj#1587) * Fixed: Ability to interface with S3 using assumed roles (session tokens) This PR fixes argoproj#1571 * Added retry around RuntimeExecutor.Wait call when waiting for main container completion (argoproj#1597) * Do not relocate the mounted docker.sock (argoproj#1607) The mount path of the docker.sock should not depend on the host path of the docker.sock * Fix DAG enable failFast will hang in some case (argoproj#1595) * Fix failFast will hang in some case * Increased Lint timeout (argoproj#1612) * Add merge keys to Workflow objects to allow for StrategicMergePatches (argoproj#1611) * Small code cleanup and add tests (argoproj#1562) * Added WorkflowStatus and NodeStatus types to the Open API Spec. (argoproj#1614) * Prevent controller from crashing due to glog writing to /tmp (argoproj#1613) * Updated the API Rule Violations list (argoproj#1618) * updated invite link (argoproj#1621) * Increase timeout of golangci-lint (argoproj#1623) * Store resolved templates (argoproj#1552) * Store resolved templates in node status * Update operator.go (argoproj#1630) * Update operator.go * update API * Fix retry workflow state (argoproj#1632) * Save stored template ID in nodes (argoproj#1631) * Grant get secret role to controller to support persistence (argoproj#1615) * Regenerate installation manifests (argoproj#1638) * Update CHANGELOG for v2.4.0 (argoproj#1636) * Update version to v2.4.0 * Add back SetGlogLevel calls * Fix regression where parallelism could cause workflow to fail (argoproj#1639) * Fix regression where global outputs were unresolveable in DAGs (argoproj#1640) * Fix global lint issue (argoproj#1641) * pin colinmarc/hdfs to the next commit, which no longer has vendored deps (argoproj#1622) * Delay killing sidecars until artifacts are saved (argoproj#1645) * fixed example wrong comment (argoproj#1643) * Fix missing merged changes in validate.go (argoproj#1647) * Fix DAG output aggregation (argoproj#1648) * Fix dag output aggregation correctly (argoproj#1649) * Use stored templates to raggregate step outputs (argoproj#1651) * Fix child node template handling (argoproj#1654) * Stop failing if artifact file exists, but empty (argoproj#1653) * Resolve WorkflowTemplate lazily (argoproj#1655) * Don't provision VM for empty artifacts (argoproj#1660) * Update version to v2.4.1 * Fix typo (argoproj#1679) * Handle sidecar killing properly (argoproj#1675) * Update README.md Argo Ansible role: Provisioning Argo Workflows on Kubernetes/OpenShift (argoproj#1673) * Handle retried node properly (argoproj#1669) * Store locally referenced template properly (argoproj#1670) * Update version to v2.4.2 * Fix issue that workflow.priority substitution didn't pass validation (argoproj#1690) * Added status of previous steps as variables (argoproj#1681) * Print multiple workflows in one command (argoproj#1650) * Fix retry node processing (argoproj#1694) * Apply Strategic merge patch against the pod spec (argoproj#1687) * fixed broke metrics endpoint per argoproj#1634 (argoproj#1695) * Fixed incorrect `pod.name` in retry pods (argoproj#1699) * Added ability to auto-resume from suspended state (argoproj#1715) * Filter workflows in list based on name prefix (argoproj#1721) * Support no-headers flag (argoproj#1760) * Refactoring Template Resolution Logic (argoproj#1744) * Fix retry node name issue on error (argoproj#1732) * Do not resolve remote templates in lint (argoproj#1787) * Handle operation level errors PVC in Retry (argoproj#1762) * Added hint when using certain tokens in when expressions (argoproj#1810) * Added hint when using certain tokens in when expressions * Minor * SSL enabled database connection for workflow repository (argoproj#1712) (argoproj#1756) * Error occurred on pod watch should result in an error on the wait container (argoproj#1776) * Update version to v2.4.3 * Update version to v2.4.3 * rename * fixing jenkins, committing extra changes * jenkins Co-authored-by: Daisuke Taniwaki <[email protected]> Co-authored-by: Ed Lee <[email protected]> Co-authored-by: Erik Parmann <[email protected]> Co-authored-by: Alexander Matyushentsev <[email protected]> Co-authored-by: kshamajain99 <[email protected]> Co-authored-by: Jesse Suen <[email protected]> Co-authored-by: Marcin Karkocha <[email protected]> Co-authored-by: Julian Fischer <[email protected]> Co-authored-by: Anna Winkler <[email protected]> Co-authored-by: Ilias Katsakioris <[email protected]> Co-authored-by: jdfalko <[email protected]> Co-authored-by: Greg Roodt <[email protected]> Co-authored-by: Naoto Migita <[email protected]> Co-authored-by: shahin <[email protected]> Co-authored-by: Tim Schrodi <[email protected]> Co-authored-by: Matthew Coleman <[email protected]> Co-authored-by: Saravanan Balasubramanian <[email protected]> Co-authored-by: Nick Stott <[email protected]> Co-authored-by: Ismail Alidzhikov <[email protected]> Co-authored-by: Xianlu Bird <[email protected]> Co-authored-by: Ian Howell <[email protected]> Co-authored-by: Fred Dubois <[email protected]> Co-authored-by: Johannes 'fish' Ziemke <[email protected]> Co-authored-by: Adrien Trouillaud <[email protected]> Co-authored-by: xubofei1983 <[email protected]> Co-authored-by: Alexey Volkov <[email protected]> Co-authored-by: Clemens Lange <[email protected]> Co-authored-by: Chris Chambers <[email protected]> Co-authored-by: Hideto Inamura <[email protected]> Co-authored-by: almariah <[email protected]> Co-authored-by: Cristian Pop <[email protected]> Co-authored-by: Jaime <[email protected]> Co-authored-by: Jacob O'Farrell <[email protected]> Co-authored-by: Ben Wells <[email protected]> Co-authored-by: Paul Brit <[email protected]> Co-authored-by: Brandon Steinman <[email protected]> Co-authored-by: alex weidner <[email protected]> Co-authored-by: Alex Collins <[email protected]> Co-authored-by: ianCambrio <[email protected]> Co-authored-by: Jean-Louis Queguiner <[email protected]> Co-authored-by: Stephen Steiner <[email protected]> Co-authored-by: Jonathon Belotti <[email protected]> Co-authored-by: Semjon Kopp <[email protected]> Co-authored-by: Orion Delwaterman <[email protected]> Co-authored-by: Edwin Jacques <[email protected]> Co-authored-by: Ziyang Wang <[email protected]> Co-authored-by: Aisuko <[email protected]> Co-authored-by: tralexa <[email protected]> Co-authored-by: Alex Capras <[email protected]> Co-authored-by: mark9white <[email protected]> Co-authored-by: Mostapha Sadeghipour Roudsari <[email protected]> Co-authored-by: commodus-sebastien <[email protected]> Co-authored-by: Mukulikak <[email protected]> Co-authored-by: Daniel Duvall <[email protected]> Co-authored-by: Anes Benmerzoug <[email protected]> Co-authored-by: Christian Muehlhaeuser <[email protected]> Co-authored-by: hidekuro <[email protected]> Co-authored-by: jacky <[email protected]> Co-authored-by: Brian Mericle <[email protected]> Co-authored-by: Takayuki Kasai <[email protected]> Co-authored-by: Xie.CS <[email protected]> Co-authored-by: John Wass <[email protected]> Co-authored-by: Premkumar Masilamani <[email protected]> Co-authored-by: Pablo Osinaga <[email protected]> Co-authored-by: David Seapy <[email protected]> Co-authored-by: Anastasia Satonina <[email protected]> Co-authored-by: Simon Behar <[email protected]> Co-authored-by: Tobias Bradtke <[email protected]> Co-authored-by: Marek Čermák <[email protected]> Co-authored-by: Rick Avendaño <[email protected]> Co-authored-by: sang <[email protected]> Co-authored-by: Antoine Dao <[email protected]> Co-authored-by: gerdos82 <[email protected]>

Support AutomountServiceAccountToken

fa96696

dtaniwaki added 4 commits July 15, 2019 23:09

Disallow automountServiceAccountToken in resource templates

94d2828

Support ServiceAccountTokenName for init/wait containers

9d12235

Add a test for ServiceAccountTokenName

3a87b5a

Fix wrong condition of AutomountServiceAccountToken check

2a47ec8

Fix lint error

7601f69

dtaniwaki added 2 commits July 23, 2019 22:15

Implement ExecutorServiceAccountTokenName

0de424f

Fix unexpected volume mounts overwritten

f32ffcd

dtaniwaki requested a review from jessesuen July 24, 2019 07:45

Merge remote-tracking branch 'upstream/master' into automount-service…

1498c3c

…-account

jessesuen approved these changes Aug 6, 2019

View reviewed changes

Remove test fields

2dc2728

jessesuen reviewed Aug 6, 2019

View reviewed changes

jessesuen requested changes Aug 6, 2019

View reviewed changes

jessesuen added this to the v2.4 milestone Aug 7, 2019

Support ExecutorServiceAccountName instead of ExecutorServiceAccountT…

ef19bb2

…okenName

sarabala1979 approved these changes Aug 7, 2019

View reviewed changes

dtaniwaki commented Aug 8, 2019

View reviewed changes

dtaniwaki added 3 commits August 13, 2019 11:54

Merge remote-tracking branch 'upstream/master' into automount-service…

8039fa5

…-account

Structure executor config

fce2316

Fix lint failure

75e5f0b

alexec assigned jessesuen Aug 15, 2019

jessesuen assigned jessesuen and unassigned jessesuen Aug 15, 2019

jessesuen requested changes Aug 20, 2019

View reviewed changes

Stop getting actual secrets to find a service account token name

2e734a6

jessesuen approved these changes Aug 21, 2019

View reviewed changes

jessesuen merged commit b5f2fde into argoproj:master Aug 21, 2019

Support AutomountServiceAccountToken #1480

Support AutomountServiceAccountToken #1480

Conversation

dtaniwaki commented Jul 15, 2019

xianlubird commented Jul 15, 2019

xianlubird commented Jul 15, 2019

dtaniwaki commented Jul 15, 2019

xianlubird commented Jul 15, 2019

dtaniwaki commented Jul 15, 2019

dtaniwaki commented Jul 15, 2019 • edited Loading

dtaniwaki commented Jul 15, 2019

xianlubird commented Jul 16, 2019

jessesuen commented Jul 16, 2019 • edited Loading

dtaniwaki commented Jul 17, 2019 • edited Loading

jessesuen commented Jul 22, 2019 • edited Loading

dtaniwaki commented Jul 23, 2019

dtaniwaki commented Jul 23, 2019

dtaniwaki commented Jul 24, 2019 • edited Loading

dtaniwaki commented Aug 6, 2019

jessesuen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jessesuen Aug 6, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dtaniwaki Aug 6, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jessesuen left a comment

Choose a reason for hiding this comment

sarabala1979 commented Aug 7, 2019

sarabala1979 left a comment

Choose a reason for hiding this comment

dtaniwaki commented Aug 8, 2019 • edited Loading

dtaniwaki Aug 8, 2019 • edited Loading

Choose a reason for hiding this comment

sarabala1979 commented Aug 8, 2019

jessesuen commented Aug 8, 2019

dtaniwaki commented Aug 13, 2019

jessesuen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dtaniwaki Aug 21, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dtaniwaki commented Aug 21, 2019

dtaniwaki commented Jul 15, 2019 •

edited

Loading

jessesuen commented Jul 16, 2019 •

edited

Loading

dtaniwaki commented Jul 17, 2019 •

edited

Loading

jessesuen commented Jul 22, 2019 •

edited

Loading

dtaniwaki commented Jul 24, 2019 •

edited

Loading

jessesuen Aug 6, 2019 •

edited

Loading

dtaniwaki Aug 6, 2019 •

edited

Loading

dtaniwaki commented Aug 8, 2019 •

edited

Loading

dtaniwaki Aug 8, 2019 •

edited

Loading

dtaniwaki Aug 21, 2019 •

edited

Loading