loops instantiate too many containers too soon

[remster]

Is this a BUG REPORT or FEATURE REQUEST?:
Not sure
What happened:
Submitting this workflow:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: parallelism-nested-workflow-
spec:
  entrypoint: A
  templates:
  - name: A
    parallelism: 1
    steps:
    - - name: seq-step
        template: B
        withParam: |
          ["a","b","c","d","e","f","g","h","i","j"]
  - name: B
    steps:
    - - name: jobs
        template: one-job
        withParam: "[1, 2]"

  - name: one-job
    container:
      image: alpine
      command: ['/bin/sh', '-c']
      args: ["echo foo; sleep 5"]

results in a container spawned for every item even though, as instructed by parallelism only one runs at a time

If there is (plausibly) a 1000 steps generate by the loop it ends up hogging my minikube so that this happens:

kubectl get pods
Unable to connect to the server: unexpected EOF

What you expected to happen:
That containers are created just-in-time

How to reproduce it (as minimally and precisely as possible):
see above spec
Anything else we need to know?:

Environment:

• Argo version:

$ argo version
argo: v2.2.1
  BuildDate: 2018-10-11T16:26:28Z
  GitCommit: 3b52b26190163d1f72f3aef1a39f9f291378dafb
  GitTreeState: clean
  GitTag: v2.2.1
  GoVersion: go1.10.3
  Compiler: gc
  Platform: linux/amd64

• Kubernetes version :

$ kubectl version -o yaml
clientVersion:
  buildDate: "2018-12-13T10:39:04Z"
  compiler: gc
  gitCommit: eec55b9ba98609a46fee712359c7b5b365bdd920
  gitTreeState: clean
  gitVersion: v1.13.1
  goVersion: go1.11.2
  major: "1"
  minor: "13"
  platform: linux/amd64
serverVersion:
  buildDate: "2018-08-07T23:08:19Z"
  compiler: gc
  gitCommit: bb9ffb1654d4a729bb4cec18ff088eacc153c239
  gitTreeState: clean
  gitVersion: v1.11.2
  goVersion: go1.10.3
  major: "1"
  minor: "11"
  platform: linux/amd64

[alexmt]

Seems parallelism is not applied to pods launched from sub-workflows.

from: https://github.com/alexmt/argo/blob/5d003d7d48085253e86dc2c47ed884aca50f5cdc/pkg/apis/workflow/v1alpha1/types.go#L188

// Parallelism limits the max total parallel pods that can execute at the same time within the
// boundaries of this template invocation. If additional steps/dag templates are invoked, the
// pods created by those templates will not be counted towards this total.

I think it make sense to improve it and start counting pods from sub-workflows . @jessesuen that do you think?

[jessesuen]

This was fixed in PR: #1046, but it's not yet available in a release. Will go out with v2.3

[jessesuen]

Closing as fixed.