Auto-generating local artifact paths

[Ark-kun]

FEATURE REQUEST

I propose auto-generating local artifact paths when the path is not specified.

I'm already working on this feature and will submit a PR soon.

Rationale:

Currently the template author needs to explicitly specify the local path for each artifact (and then possibly use that same path in the command-line).

Often, the containerized program does not care about the exact paths - it just works with the paths it's been given. What if the system would just generate the paths for the user?

User-provided paths might be problematic for Argo's executors.
Some executors cannot mount a volume to particular places (e.g. the root).
Some executors have troubles with files written to the base image folders.
There are move mount-related problems that Argo has to consider and handle.
Some scenarios become impossible if multiple artifacts are placed in the same directory.
Some optimizations only become impossible if all artifact files have the same names.

It would be easier to have a way to just generate the paths that are convenient to Argo. This also takes the path-generation burden from the user's shoulders.

Current:

  - name: whalesay
    outputs:
      artifacts:
      - name: hello-art
        path: /hello-art.txt
    container:
      image: docker/whalesay:latest
      command: [sh, -c]
      args: ["cowsay hello world > /hello-art.txt"]

Proposed:

  - name: whalesay
    outputs:
      artifacts:
      - name: hello-art
    container:
      image: docker/whalesay:latest
      command: [sh, -c]
      args: ["cowsay hello world > {{outputs.artifacts.hello-art.path}}"]

As you see, there is no explicit artifact path specified in the outputs section. The generated path for the hello-art artifact can be /argo/outputs/artifacts/hello-art/data or some other path which is convenient to the system.

Having support for auto-generated paths requires the ability to reference them in the command-line. The support for the artifact path references ({{outputs.artifacts.hello-art.path}}) has already been added in #1300 .