Fix DAG enable failFast will hang in some case

[xianlubird]

In this case

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: dag-hang-failFast-
spec:
  entrypoint: retry-with-dags
  templates:
  - name: retry-with-dags
    dag:
      failFast: false
      tasks:
      - name: A
        template: success
      - name: B
        template: success-2
        dependencies:
        - A
      - name: C
        template: sub-dag2
        dependencies:
        - A
      - name: D
        dependencies:
        - A
        - C
        template: success

  - name: sub-dag
    dag:
      tasks:
      - name: fail
        template: fail
      - name: success1
        template: success

  - name: fail
    container:
      command: [sh, -c, exit 1]
      image: alpine
    retryStrategy:
      limit: 1

  - name: sub-dag2
    steps:
    - - name: sub-dag-a
        template: success
    - - name: sub-dag-b
        template: fail

  - name: success
    container:
      command: [sh, -c, exit 0]
      image: alpine
    retryStrategy:
      limit: 1

  - name: success-2
    container:
      command: [sh, -c, sleep 30; exit 0]
      image: alpine
    retryStrategy:
      limit: 1

It will hang forever like this:

Name:                dag-hang-failfast-qqm8f
Namespace:           default
ServiceAccount:      default
Status:              Running
Created:             Thu Sep 05 19:46:07 +0800 (7 minutes ago)
Started:             Thu Sep 05 19:46:07 +0800 (7 minutes ago)
Duration:            44 seconds

STEP                        PODNAME                             DURATION  MESSAGE
 ✖ dag-hang-failfast-qqm8f
 ├-✔ A(0)                   dag-hang-failfast-qqm8f-3458137155  8s
 ├-✖ C                                                                    child 'dag-hang-failfast-qqm8f-2310091035' failed
 | ├---✔ sub-dag-a(0)       dag-hang-failfast-qqm8f-2915312254  3s
 | └---✖ sub-dag-b                                                        No more retries left
 |     ├-✖ sub-dag-b(0)     dag-hang-failfast-qqm8f-3248014870  4s        failed with exit code 1
 |     └-✖ sub-dag-b(1)     dag-hang-failfast-qqm8f-2643873491  3s        failed with exit code 1
 └-✔ B(0)                   dag-hang-failfast-qqm8f-3348498240  33s

Pls @sarabala1979 @jessesuen take a look.

[xianlubird]

The ci seems has problem, we should fix it.

Unable to retrieve container logs for docker://a10b60cc3c44ecdd178a67d670f645c252e3912dbf5d92dad23a3ac02c6f96af

cc @sarabala1979

[xianlubird]

@sarabala1979 Can you help to review and merge this ?

[sarabala1979]

LGTM