Post setup Python step fails (pipenv cache)

[mrlubos]

Description:
A clear and concise description of what the bug is.

Hey! So this used to work just a few days ago. We’ve been having some issues with cache in the last few days (see here for details), this is a brand new issue which I think might or might not be related to the previous problem. Simply put, the action fails in the post step with the following message.

Post job cleanup.
[2](https://github.com/<org>/<repo>/runs/6889538635?check_suite_focus=true#step:20:2)
Error: Cache folder path is retrieved for pipenv but doesn't exist on disk: /home/runner/.local/share/virtualenvs

Here’s the actual step:

Run actions/[email protected]
[2](https://github.com/<org>/<repo>/runs/6889538635?check_suite_focus=true#step:5:2)
  with:
[3](https://github.com/<org>/<repo>/runs/6889538635?check_suite_focus=true#step:5:3)
    cache: pipenv
[4](https://github.com/<org>/<repo>/runs/6889538635?check_suite_focus=true#step:5:4)
    python-version: 3.9.13
[5](https://github.com/<org>/<repo>/runs/6889538635?check_suite_focus=true#step:5:5)
    token: ***
[6](https://github.com/<org>/<repo>/runs/6889538635?check_suite_focus=true#step:5:7)
Successfully setup CPython (3.9.13)
[7](https://github.com/<org>/<repo>/runs/6889538635?check_suite_focus=true#step:5:8)
pipenv cache is not found

Action version:
Specify the action version

3.1.2

Platform:

• Ubuntu
• macOS
• Windows

Runner type:

• Hosted
• Self-hosted

Tools version:

3.9.13

Repro steps:
A description with steps to reproduce the issue. If your have a public example or repo to share, please provide the link.

Expected behavior:
A description of what you expected to happen.

No issue occurs, the same way it used to work a week ago.

Actual behavior:
A description of what is actually happening.

This error message appears on every run.

[dmitry-shibanov]

Hello @mrlubos. Thank you for your report. I've tried to reproduce the issue but everything works as expected. Could you please provide public repository to reproduce the issue ?

[mrlubos]

@dmitry-shibanov Hi Dmitry, thank you for your reply! I might look into trying to reproduce this in a public repository if the issue persists, wanted to provide at least some pointers in the meantime in case anyone else runs into the same problem. Further behaviour I observed today is that even if the post setup step completes, the next run will show this message:

Run actions/[email protected]
  with:
    cache: pipenv
    python-version: [3](https://github.com/<org>/<repo>/runs/6896643828?check_suite_focus=true#step:5:3).9.13
    token: ***
Successfully setup CPython (3.9.13)
pipenv cache is not found

What I think might get us closer to the resolution is the fact I realised we've started experiencing these issues after upgrading from v2 to v3. That being said, the post step cache looks the same for both versions:

v2

Post job cleanup.
Cache hit occurred on the primary key setup-python-Linux-python-3.9.13-pipenv-cbf9160[2](https://github.com/<org>/<repo>/runs/6766531090?check_suite_focus=true#step:18:2)ee5daeb2e69e4e65d32ca4c617d796670afbc226f170e9e4035d058e, not saving cache.

v3

Post job cleanup.
/usr/bin/tar --posix --use-compress-program zstd -T0 -cf cache.tzst -P -C /home/runner/work/<repo>/<repo> --files-from manifest.txt
Cache Size: ~[2](https://github.com/<org>/<repo>/runs/6896643828?check_suite_focus=true#step:18:2)02 MB (212215749 B)
Cache saved successfully
Cache saved with the key: setup-python-Linux-python-[3](https://github.com/<org>/<repo>/runs/6896643828?check_suite_focus=true#step:18:3).9.13-pipenv-0c70[4](https://github.com/<org>/<repo>/runs/6896643828?check_suite_focus=true#step:18:4)b[5](https://github.com/<org>/<repo>/runs/6896643828?check_suite_focus=true#step:18:5)9407d6fda6f2722fa4daa27bc67f9d74da6402ee4c95f412a71936434

I would therefore expect the cache to be picked up by v3. I am shooting blanks here, would adding fetch-depth: 0 to the checkout action help by any chance?

[dmitry-shibanov]

Thank you for your reply @mrlubos. I do not think that it's related to fetch depth. I'll try to reproduce the issue from my side one more time. If you have any updates feel free to post them here.

[mrlubos]

Hi @dmitry-shibanov no updates beyond that this is still an issue. I tried removing the cache key which obviously removes the error but after adding it back, the error came back too. So I am stuck it seems between not having caching or receiving this error. I don't have this confirmed yet, but I suspect this error happens only on the master branch and not in other branches. Also this is interesting to me, after I added back the cache key, the next run produced this:

Run actions/[email protected]
  with:
    cache: pipenv
    python-version: 3.9.13
    token: ***
Successfully setup CPython (3.9.13)
pipenv cache is not found

And then in post step

Post job cleanup.
Error: Cache folder path is retrieved for pipenv but doesn't exist on disk: /home/runner/.local/share/virtualenvs

[mrlubos]

@dmitry-shibanov So I am fairly confident this is happening only on the master branch. All runs on other branches complete without issues, it's happening every time on master though

[coldino]

I was also seeing this when attempting to update to v4 (from v1!).

After some head scratching, I found it was because of some remnants of my old manual caching system:

env:
  PIPENV_NOSPIN: 1
  PIPENV_CACHE_DIR: /tmp/.pip-cache
  WORKON_HOME: /tmp/pipenv-venvs

Removing both cache location vars enables the "new" caching system to work as expected.

[mrlubos]

Yeah @coldino it does sound like something similar is happening to me, except I don't have any variables set explicitly so it's not obvious whether this is something I can resolve on my end

[lsmith77]

I am also seeing this issue. I upgraded to @v4 a while ago but I am only seeing this issue now that I changed a pipenv dependency

This here is the relevant portion of my github actions setup

name: run tests

on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    env:
      PIPENV_VENV_IN_PROJECT: enabled

    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0

      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v4
        with:
          python-version: "3.9"
          cache: "pipenv"

      - name: Install pipenv
        run: |
          python -m pip install --upgrade pipenv wheel

      - name: Install dependencies
        run: |
          pipenv install --deploy --dev --verbose

[lsmith77]

removing PIPENV_VENV_IN_PROJECT: enabled seems to have fixed things for me.

[MemphisMeng]

I figured that this issue persisted even when the name is main instead of master.

[matteius]

FWIW -- about a month ago pipenv changed to be more strict on if PIPENV_VENV_IN_PROJECT is set to one of these truthy values: https://github.com/pypa/pipenv/blob/main/pipenv/utils/constants.py#L5

Interestingly though, we ran into this issue today as well at work and found we also had PIPENV_VENV_IN_PROJECT = enabled, but after changing it to PIPENV_VENV_IN_PROJECT = 1 the issue is still obtained. I help maintain pipenv in my spare time, and this is odd behavior as we generally expect that folks can use PIPENV_VENV_IN_PROJECT = 1 in their github CIs. I'll have to think more about what could be causing this issue.

[matteius]

It would appear the pipenv cache implementation assumes it will be at this location, making PIPENV_VENV_IN_PROJECT probably unworkable at present versions of setup-python actions: https://github.com/actions/setup-python/blob/main/src/cache-distributions/pipenv-cache.ts#L25

[rburema]

At the risk of sounding very obvious, but maybe this'll help someone in any case:

We ran into this issue recently. (For example; see here.)

What 'fixed' it for us was the realization that whenever this issue occurred, the workflow as a whole needed to skip all the steps that actually required python and/or pip -- and then to disable the setup-python step as well whenever that was the case.

(That might also be going on with the reproduction of others' but in a more subtle way: All of the actions taken that I can see here are for packages that are likely to be included and at latest version anyway, so the cache might never get written to. Again, sorry if that was already obvious.)

In any case, however inelegant it may be, setting up something, not using it, and then cleaning it up again should probably (also) not result in an error. (Just putting that out there, since that is a slightly different case than what OP is talking about, which may or may not require a different approach to solve.)

[tiangolo]

I'm seeing the same error in FastAPI, with pip, not pipenv, here's an example of one of the runs: https://github.com/tiangolo/fastapi/actions/runs/4777885541/jobs/8541931450

In the initial stage of the run I get:

Run actions/setup-python@v4
Installed versions
/opt/hostedtoolcache/Python/3.9.16/x64/bin/pip cache dir
/home/runner/.cache/pip
pip cache is not found

And then in the Post Setup I get:

Post job cleanup.
Error: Cache folder path is retrieved for pip but doesn't exist on disk: /home/runner/.cache/pip

[doc-sheet]

Same story for me with pip cache. For some reason it works in one repo and fails in another with no useful info even in debug log.
Configs are almost same.

Maybe it's time to update cache module?
I kinda can't understand why there are two different locks for same(?) module:

• 3.0.4

  setup-python/package-lock.json

  Line 12 in db9987b

  "@actions/cache": "^3.0.4",

• 3.1.4

  setup-python/package-lock.json

  Line 6799 in db9987b

  "version": "3.1.4",

[MSK61]

I have the same error with a pipenv cache setting. I have a .venv directory under my checkout root directory, so pipenv creates the virtual environment inside the project workspace(as opposed to a location in the home directory). Looks like the cache logic doesn't honor the project-local location though.

[smougenot]

Same trouble for me with pipenv here

Tried workaround without luck

• disable venv in project here
• create cache dir here

Good luck

I'll disable cache for the moment

[doc-sheet]

@smougenot, I guess in your case it's better to call mkdir -p /home/runner/.local/share/virtualenvs directly.
looks like pipenv cache location differs from pip

[woodruffw]

I've experienced another variant of this: if you add actions/setup-python to a workflow job and enable caching, but you job doesn't actually use Python or fill the cached directory in any way, then the post-step cleanup will fail with this message.

Example failing run:

https://github.com/C2SP/x509-limbo/actions/runs/9739572591/job/26897420759

Exact reproduction steps:

1. Have no cache to restore from
2. Use actions/setup-python with cache: pip
3. Pre-step succeeds (but does nothing, as there's no cache to restore from)
4. Post-step fails (the cache wasn't modified during the job at all, so it isn't present)

The failure in (4) seems unambiguously incorrect to me: a caching layer should never fail on an empty cache if no cache was loaded to begin with.

[aparnajyothi-y]

Hello Everyone, Thank you once again for creating this issue and The error related to pipenv or poetry package managers cannot be changed to a warning due to the necessity of having the Pipfile.lock or poetry.lock file for caching in the actions/Setup-python.

To resolve this issue, please consider one of the following options:

Add the Pipfile.lock / poetry.lock file: Ensure that the Pipfile.lock file or poetry.lock is present in your repository. You can generate this file by running pipenv lock.
Install dependencies: Make sure all required dependencies are installed and committed.
Do not use cache: pip: If there are no dependencies to cache, remove the cache: pip configuration from your GitHub Actions workflow.
Additionally, please note that changing this behaviour would require multiple code changes, which may be implemented as a feature request.

[aparnajyothi-y]

Hello @mrlubos, Please let us know if you have any further concerns on the above information.

[woodruffw]

Do not use cache: pip: If there are no dependencies to cache, remove the cache: pip configuration from your GitHub Actions workflow.

I want to push back slightly on this: caching should always be referentially transparent, meaning that the cache step itself should never fail, even if the caching operation fails or can't be completed (e.g. if there's nothing to cache).

With that being said, I can understand that this would require multiple changes to fix. As a stop-gap, my suggestion would be to improve the error messaging here: if the caching step fails because there's nothing to cache, the error message could say that explicitly and suggest that the maintainer remove the step entirely, rather than returning the current (confusing) message.

[insanitybit]

It would be nice to fail gracefully here. Sometimes an action will install deps, sometimes it won't - right now this bug means you have to completely forgo caching via pip.

For example, imagine a task that first checks the time so that it only ever runs once a day, or some such thing.

[aparnajyothi-y]

Hello Everyone, The PR #968 has been merged to update the error message when no dependencies are found to cache, improving its clarity.

Please validate the changes by using the actions/setup-python@main branch and confirm to proceed to close this issue.

[woodruffw]

That error message looks better, thank you! However, I'd like to reiterate that I think this shouldn't be an error at all, per caching semantics. As such I think this should remain open until the post-step actually ceases to fail.

[aparnajyothi-y]

Hello @woodruffw , Thank you for the confirmation. Throwing an error message at this point in the post-setup phase is crucial for handling intermittent failures during package installations. Dependencies can sometimes fail to install due to issues such as network instability, timeouts, or conflicts. By raising an error at this stage, we can explicitly identify and address these failures, rather than letting the workflow proceed with an incomplete or inconsistent environment.

Adding a clear error message when package installation fails or when there are no dependencies to cache will significantly improve the reliability and predictability of the setup process.

Please feel free to reach us if any concerns/clarifications needed.