Artifacts are not copied when running garden test in interactive mode (--interactive=true)

[cwseger]

Bug

Current Behavior

garden test doesn't copy artifacts when in interactive mode (garden test <test-name> --interactive=true).

Expected behavior

garden test copies artifacts in both interactive and non-interactive modes.

Ideal Test action:

kind: Test
type: container
name: foo-app-unit
build: foo-app
description: The unit tests for foo-app
spec:
  command: # This command requires --interactive=true
    - /bin/sh
    - -c
    - cd /foo/bar
    - <run some test command here>
  artifacts:
    - source: /foo/bar/reports/*
       target: foo-app-unit

Reproducible example

Run garden test <test-name> --interactive=true in a project that stores artifacts and note that they're not copied to the .garden/artifacts directory.

Workaround

Add a script to the application repo that contains the desired commands to execute and point to that script for the command executed when running garden test <test-name> instead of defining the test commands in the garden config.

Workaround Test action:

kind: Test
type: container
name: foo-app-unit
build: foo-app
description: The unit tests for foo-app
spec:
  command: # This command allows interactive to be false
    - /foo/bar/garden-unit.sh # Path to script that's inside the docker container that wraps the commands that need to be executed
  artifacts:
    - source: /foo/bar/reports/*
       target: foo-app-unit

Additional context

Global context:
This is the same bug reported by @eysi09 here and @stumpc here for Acorn. I am re-reporting it for Bonsai per this comment.

Personal context:
The garden config files for all the applications deployed are stored in a central repo that is separate from each application's repo. Each Build action points to the git repository where the source code is stored. At the end of each test run the test reports need to be copied back to the local machine running garden. Ideally, the test commands to execute in the container can be placed inside the garden config (this requires interactive mode so it can pass the command appropriately). This currently doesn't work because when running in interactive mode garden doesn't copy the artifacts specified into the .garden/artifacts directory. The work around is to run in non-interactive mode (the default) but this requires a single script to execute (that must be present in the container) instead of being able to define a list of bash commands in the garden config. This is problematic since the goal for this use case was to leave the application repos completely untouched. It is also confusing to have 99% of garden config in the central repo but then have scripts added to each application repo that defines how to execute the tests.

Your environment

• OS: macOS
• How I'm running Kubernetes: EKS

garden version
0.13.14

[vvagaytsev]

Thanks for reporting this @cwseger!

@thsig it looks like we never copy artifacts in the interactive mode because of this condition. Was that done intentionally? What was the reason for doing that?

[thsig]

I'm not quite sure why we're skipping the copying there. I think we may have run into problems with the I/O and entrypoint/command handling when using artifacts and interactive mode together, but we'll have to check.

Let's remove interactive from the conditional there and write an integ test case to verify that interactivity and artifacts can play nicely together.