List artifacts doesn't work during workflow run

[wchen342]

I tried to retrieve the list of artifacts in a second job using GET /repos/:owner/:repo/actions/runs/:run_id/artifacts after uploading some artifacts by actions/upload-artifact@v1 in the first job, but the artifacts won't show up. The API call will always return { "total_count": 0, "artifacts": [ ] }.
I have the second job depends on the first job.

Is there a solution to this?

[KurtGokhan]

Same for me. I tried to create an action that would delete an artifact created during the run. But artifacts aren't listed until after the run is complete.

[konradpabjan]

Both listing an deleting an artifact with the artifacts API only works after a run is complete so the APIs defined in https://developer.github.com/v3/actions/artifacts/ won't work for your scenario. This is largely due to the fact that we don't mark an artifact as complete until the end of the run.

To accomplish this, we need some lower level support to list/delete artifacts before the run is complete, see:
actions/toolkit#380
actions/toolkit#379

[nealkruis]

This may not be relevant, but is this the same reason artifacts aren't posted on the UI until the entire workflow is complete?

What I don't understand is why artifacts can be downloaded by subsequent jobs, but not through other means. Is there a reason not to mark them complete after each job?

[beikov]

I have the same issue. I'm publishing test failure reports for matrix entries. It's pretty annoying that I can't view the test reports until the workflow is completed. What is the reason for not marking artifacts as complete?

[CreMindES]

It's also an issue when you want to launch another workflow (2) that would use an artifact created in the original workflow (1) run, but since (1) is still executing other steps or jobs, the artifact is not available in the API, although it's uploaded already.

[reuben]

One /can/ technically just call the same internal APIs used by download-artifact but it's hacky and fragile, and the returned URLs only work with the authorization headers that only exist in the Actions context. It also doesn't give you an artifact ID that will be valid later when the artifacts show up in the REST API. You can see some code that does that here: https://github.com/mozilla/DeepSpeech/blob/a6bdf0ae3c190cbaf39dc4598cc87a55047e38fa/.github/actions/update-cache-index/main.js#L8-L37

I ended up giving up going down that path because I needed stable URLs for the artifacts. It also means that there's a more fundamental change that needs to happen in the way GitHub Actions implements artifacts internally to allow for this issue to be fixed. The internal API being queried is not the GitHub REST API, but rather an Azure Pipelines API, so I'm guessing there's some syncing of state between the two systems that only happens at predetermined times, such as workflow run completion.

[JacksonKearl]

It'd be great if this could work better for using Environments to gate the publishing of a built artifact. We'd like for one stage to build the artifact, upload it, then a person manually tests the artifact, submits an approving review once they've tested, then the second job comes to publish the same artifact. If we can't access artifacts mid-job, this won't work at all.

[pauleustice]

I'm also running into this issue in a release context. We trigger a repository_dispatch which creates artifacts (reports etc) for a release and should then upload them in the final job to the release.

Short of uploading the artifacts somewhere for re-use, I don't currently see how we can do this as they must be accessed in a separate workflow as it stands.

[panayot-cankov]

@JacksonKearl I was trying to achieve the exact same setup. We are building VS Code extensions that run on MacOS, Windows and Linux. The release workflow is supposed to build a .vsix file in one job, pass it down to few other jobs running e2e tests, and a QA is supposed to manually verify the artefact, if all is ok the team would authorise a publishing environment that would upload the artefact to the store.

I think Actions have some room for improvement for Jobs that include manual labour, as well as the artefacts in those cases should be available somehow before the workflow fully completes.

[relsqui]

This is impacting me while writing and testing a new workflow -- I want to verify that a new artifact uploading step is doing the right thing but I'm not ready to run the whole long workflow yet (and have not yet written the step that downloads and uses the artifacts).

[tomjohnburton]

We would love to see this feature too.

We build binaries, store them as artifacts, and then want to download them directly through the API.

[invalidred]

+1 it makes a tonne of sense to view artifacts immediately in the workflow and not wait for it to complete. For example we have a rollback step post deploy pending approval and our workflow never really ends, however we could have an e2e test fail and would like to view the artifacts even when the workflow is not completed.

[mm-at-hydro]

+1. Also have the same requirement to do some manual work before approving to deploy an artifact to an environment. Coming from Azure DevOps, I believe that this functionality was on that platform.

[robinxb]

+1 for deploying some artifacts when using other deployment. Now I have to sleep for few seconds to make this work.

[BradleyBertrimEmpire]

Is there any word about this being on the roadmap? I just stumbled upon this limitation yesterday after some head scratching why the list size was 0 after pushing artifacts for a good part of the day. Without this ability we're going to have to scrap the attempt to automate this specific process as it won't be possible without storing the artifacts with a 3rd party.

[mm-at-hydro]

There's also this thread in the community:
https://github.community/t/logs-and-artifacts-not-available-in-api-for-in-progress-runs/132091/5

May be a better place to get eyes on this issue instead of the action "issue" list.

[kyurkchyan]

This really really super-duper annoying :( Coming from Azure DevOps this is a must and not acceptable at all.

[P9155522]

I also have the same issue . I have a requirement to use build artifact in release task , which is present currently in a separate job within same dispatch workflow. How do I achieve this , I am not sure. If anyone has any solution work around . pls Suggest

[panayot-cankov]

@P9155522 download and upload artefact actions work between jobs. It is just an UI issue, but the actions can exchange artefacts between jobs. We have a setup where a windows machine would build some VS Cod extension and attach them then Mac Win and Linux will download them and run tests.

[grasbergerm]

+1 to this being a huge inconvenience. Coming from CircleCI, this is a reason I'd switch back.

[rdelmont]

+1 How are we suppose to make some validation if we cannot access the artifacts before rejecting/approving deployment?

[panayot-cankov]

As a workaround, my deployments are making release drafts and attach the artifacts for validation there.

[marcorighini]

+1 Is somebody aware if github devs are working on this?

[deco-dev]

+1 This is very useful in CircleCI. But actions does not support.

[acevos01]

does someone have a solution for this? 😭

[gselsidi]

lulz gg

[cortexcompiler]

Disappointed that GitHub has not solved this yet. I think a workflow to deploy an artifact that can be used during a CI build to do an initial deploy and re-used to deploy to further environments makes total sense. To get around this I have had to use an IF statement to use the correct Action based upon the context. This works but it painful:

      # Have to use this when the workflow is still in progress due to this issue:
      # https://github.com/actions/upload-artifact/issues/53
      - name: Download artifact (sub-flow)
        uses: actions/download-artifact@v3
        if: github.event_name == 'push'
        with:
          name: ${{ env.ARTIFACT_NAME }}

      - name: Download artifact (dispatch)
        if: github.event_name == 'workflow_dispatch'
        uses: dawidd6/action-download-artifact@v2
        with:
          github_token: ${{ env.GITHUB_TOKEN }}
          workflow: main-build.yaml
          workflow_conclusion: ""
          name: ${{ env.ARTIFACT_NAME }}

[StuartBow]

This is a joke that its still not fixed, I'm going back to Azure DevOps

[DPatrickBoyd]

+1 to this, just spent the last week building out a pipeline, and the last thing I had to do was delete the artifact from a terraform plan and .terraform folder. Now that it's not working, I feel like it was a waste of my time.

[DPatrickBoyd]

I got mad that I this was still a problem, so I just made my own workflow to tackle this. I am using the gh CLI to do the dirty work, but you could use curl or something similar. In my case I am actually deleting the artifact once the workflow that creates it is complete, but it does require to list it first. This may not work for most people since they likely need it during the previous run, but at least this might help someone!

• secrets.GH_REPO_ACCESS_PAT is a PAT that I made and gave it the ability read repos and org stuff as well, not sure how important the scope is.

name: Test

on:
  workflow_run:
    branches: [main]
    workflows: [nonprod,prod,test]
    types:
      - completed
jobs:
  on-success:
    runs-on: ubuntu-latest
    name: Delete artifact
    if: github.event.workflow_run.head_branch == 'main' # feel free to take this out if you don't need it
    steps:
      - id: get-id
        run: | # grab the id of the artifact we just created
          echo "ARTIFACT_ID=$(gh api -H 'Accept: application/vnd.github+json'   ${{github.event.workflow_run.artifacts_url}} --jq .artifacts[].id)" >> $GITHUB_ENV
        env:
          GITHUB_TOKEN: ${{ secrets.GH_REPO_ACCESS_PAT }}
      - run: |
          echo "artifact id is ${{ env.ARTIFACT_ID}}"
          echo "run url is ${{github.event.workflow_run.artifacts_url}}"
      - id: delete-artifact
        run: | # https://docs.github.com/en/rest/actions/artifacts?apiVersion=2022-11-28#delete-an-artifact
          gh api --method DELETE -H 'Accept: application/vnd.github+json' /repos/${{github.repository}}/actions/artifacts/${{env.ARTIFACT_ID}}
        env:
          GITHUB_TOKEN: ${{ secrets.GH_REPO_ACCESS_PAT }}
  on-failure:
    runs-on: ubuntu-latest
    if: ${{ github.event.workflow_run.conclusion == 'failure' }}
    steps:
      - run: echo 'The triggering workflow failed'

[robinxb]

it is a shame for not fixing this in almost 3 years.

[zwass]

Is anyone at GitHub listening? This is a sorely needed feature.

[gselsidi]

@zwass Lul they're all working "remotely" now wink, probably push a feature every decade or so.

[kykrueger]

Might be barking up the wrong tree here, but @TingluoHuang can you help make this issue more visible?

[nmattia]

I just made my own workflow to tackle this.
DPatrickBoyd comment

@DPatrickBoyd I was going to do something similar, but then realized a workflow can only be triggered by workflow_run if said workflow is on the default branch. Did you work around this? It sounds like it would be hard to edit the workflows if you can't test it in branches & PRs!

[kykrueger]

@nmattia it is indeed a bit annoying to debug the workflows when they are only triggered in main. It is possible to test in a fork as a bit of a workaround.

Here is an example of how I implemented it for systemds:
https://github.com/apache/systemds/actions/runs/5309254477

[Akhilvs007]

+1

[nibanks]

@konradpabjan is there any update here? I am just encountering this problem for microsoft/netperf where one repo is trying to consume the artifacts of another, but it doesn't work if the run is still on going.

[kennethpgreen]

+1

[lucasal-vermiculus]

+1

[gogetdane]

This is insane. Of course you would want to pass artifacts between jobs. We have a sequence of jobs that run on different runners for perfectly valid reasons and the only realistic solution is to use S3 or similar. We can split the workflow and work around it but why is that still a solution when years have passed since this was first raised?

[JazwinskiMichal]

Workaround for this if someone does not want to use external cloud storage, would be to create temporary Gtihub Repo, to store the files, download those in another workflow and delete the repo.
Of course this requires PAT with delete_repo scope, to delete the repo.

So that could be Steps to create and push to new Private Repo in Workflow A

- name: Create New Private Repository
  run: |
    curl -X POST -H "Authorization: token ${{ secrets.GIT_TOKEN }}" \
         -H "Accept: application/vnd.github.v3+json" \
         -d '{"name": "name-of-the-repo", "private": true}' \
         https://api.github.com/user/repos

- name: Commit and Push to Temp Repository
  run: |
    git config --global user.name 'Bot'
    git config --global user.email '[email protected]'
    git init
    git add .   # Take all files
    git commit -m "Add temp files"
    git branch -M main
    git remote add origin https://x-access-token:${{ secrets.GIT_TOKEN }}@github.com/owner/name-of-the-repo.git
    git push -u origin main

and that could be the Steps to download from the Repo and delete it in workflow B

- name: Download ZIP from Repository
  id: download-artifact
  run: |
    # Construct and download artifact
    ARTIFACT_NAME="${ARTIFACT_FILE_NAME}_v${ARTIFACT_FILE_VERSION}"
    curl -L -o "${ARTIFACT_NAME}.zip" -H "Authorization: token ${{ secrets.GIT_TOKEN }}" "https://github.com/owner/name-of-the-repo/archive/refs/heads/main.zip"

- name: Delete Temp Repository
  run: |
    curl -X DELETE \
    -H "Authorization: token ${{ secrets.GIT_TOKEN }}" \
    -H "Accept: application/vnd.github.v3+json" \
    "https://api.github.com/repos/owner/name-of-the-repo"

[TheMrMilchmann]

Since v4 is now available and "artifacts are [now] immediately available in the UI and REST API" (ref), is this still an issue?

[konradpabjan]

Yup! Closing this out after ages 🙃 Long time coming but it's finally out!

v4 has officially released today which brings lots of improvements: https://github.blog/changelog/2023-12-14-github-actions-artifacts-v4-is-now-generally-available/

With it, the limitation that APIs don't work until a run is fully complete. Now after upload and artifact id is immediately available and you can use any of the APIs to GET, LIST, DELETE or DOWNLOAD artifacts.

Recommend switching to v4 as soon as possible! v1-v3 artifact still have these limitations