reduxcache* folders are not deleted starting at gatsby 2.19.22 in Docker containers

[asgarli]

Description

I have a Docker setup for Gatsby with the following configuration:

# docker-compose.yml
version: '3'
services:
  web:
    tty: true
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "8000:8000"
      - "9929:9929"
      - "9230:9230"
    volumes:
      - /app/.cache
      - /app/node_modules
      - /app/public
      - .:/app
    environment:
      - NODE_ENV=development
      - GATSBY_WEBPACK_PUBLICPATH=/
      - CHOKIDAR_USEPOLLING=1

# Dockerfile
FROM node:12
EXPOSE 8000 9929 9230

RUN npm i -g gatsby-cli
RUN gatsby telemetry --disable

WORKDIR /app
COPY ./package.json .
RUN npm i

# Run with `gatsby develop -H 0.0.0.0`

# .dockerignore
./.cache
./node_modules
./public

When I run gatsby develop, I get the following error:

warn Error persisting state: ENOENT: no such file or directory, rename 'reduxcacheF9pfSE' -> '/app/.cache/redux'
warn Error persisting state: ENOENT: no such file or directory, rename 'reduxcache7D8wgJ' -> '/app/.cache/redux'
warn Error persisting state: ENOENT: no such file or directory, rename 'reduxcacheCPgs5R' -> '/app/.cache/redux'

This happens with version 2.19.22, and not with version 2.19.21. I was getting another error earlier, saying something along the lines of cannot move files between incompatible systems. My Docker is on Windows, using Linux VM. I believe the error comes from trying to use system calls for moving files instead of npm package, but I might be wrong, since I don't know JS well enough to debug this.

Steps to reproduce

Described above

Expected result

Described above

Actual result

Described above

Environment

Run gatsby info --clipboard in your project directory and paste the output here.

  System:
    OS: Linux 4.19 Debian GNU/Linux 9 (stretch) 9 (stretch)
    CPU: (4) x64 Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz
    Shell: 4.4.12 - /bin/bash
  Binaries:
    Node: 12.13.1 - /usr/local/bin/node
    Yarn: 1.19.1 - /usr/local/bin/yarn
    npm: 6.12.1 - /usr/local/bin/npm
  Languages:
    Python: 2.7.13 - /usr/bin/python
  npmPackages:
    gatsby: 2.19.22 => 2.19.22
    gatsby-image: ^2.3.1 => 2.3.1
    gatsby-plugin-google-analytics: ^2.2.1 => 2.2.1
    gatsby-plugin-manifest: ^2.3.2 => 2.3.2
    gatsby-plugin-material-ui: ^2.1.6 => 2.1.6
    gatsby-plugin-offline: ^3.1.1 => 3.1.1
    gatsby-plugin-react-helmet: ^3.2.1 => 3.2.1
    gatsby-plugin-sass: ^2.2.1 => 2.2.1
    gatsby-plugin-sharp: ^2.5.3 => 2.5.3
    gatsby-plugin-typescript: ^2.3.1 => 2.3.1
    gatsby-source-filesystem: ^2.2.2 => 2.2.2
    gatsby-transformer-sharp: ^2.4.2 => 2.4.2
  npmGlobalPackages:
    gatsby-cli: 2.11.1

[pvdz]

@asgarli thanks for the report. This sounds like a duplicate of #21982

Can you please confirm whether or not the steps there resolve your problem? Thanks!

[asgarli]

I did find that post and run the above without VSCode in the picture and that's when I got the error above. With VSCode, I get the following error:

warn Error persisting state: EXDEV: cross-device link not permitted, rename 'reduxcacheIx3gkr' -> '/app/.cache/redux'
warn Error persisting state: EXDEV: cross-device link not permitted, rename 'reduxcacheLmhpds' -> '/app/.cache/redux'
success Generating image thumbnails - 7.893s - 165/165 20.90/s
warn Error persisting state: EXDEV: cross-device link not permitted, rename 'reduxcachehpKLPD' -> '/app/.cache/redux'

Note that I'm running this in Docker container with a Linux VM, so maybe WSL options don't apply.

[liviu-marciu]

The steps in #21982 don't resolve the problem.

I've got the same problem with this container:
System:
OS: Linux 4.19 Ubuntu 18.04.4 LTS (Bionic Beaver)
CPU: (5) x64 Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz
Shell: 4.4.20 - /bin/bash
Binaries:
Node: 13.11.0 - /usr/local/bin/node
Yarn: 1.22.4 - /usr/bin/yarn
npm: 6.13.7 - /usr/local/bin/npm
Languages:
Python: 2.7.17 - /usr/bin/python
npmPackages:
gatsby: ^2.19.41 => 2.19.41
gatsby-image: ^2.2.43 => 2.2.43
gatsby-plugin-config: ^1.0.7 => 1.0.7
gatsby-plugin-create-client-paths: ^2.1.23 => 2.1.23
gatsby-plugin-emotion: ^4.1.24 => 4.1.24
gatsby-plugin-flow: ^1.2.0 => 1.2.0
gatsby-plugin-manifest: ^2.2.47 => 2.2.47
gatsby-plugin-offline: ^3.0.40 => 3.0.40
gatsby-plugin-react-helmet: ^3.1.23 => 3.1.23
gatsby-plugin-sharp: ^2.4.12 => 2.4.12
gatsby-plugin-stripe: ^1.2.4 => 1.2.4
gatsby-source-contentful: ^2.1.96 => 2.1.96
gatsby-source-filesystem: ^2.1.54 => 2.1.54
gatsby-transformer-sharp: ^2.3.18 => 2.3.18
npmGlobalPackages:
gatsby-cli: 2.8.22

---

but in this container is working fine:
System:
OS: Linux 4.9 Ubuntu 18.04.3 LTS (Bionic Beaver)
CPU: (4) x64 Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz
Shell: 4.4.20 - /bin/bash
Binaries:
Node: 13.8.0 - /usr/local/bin/node
Yarn: 1.22.4 - /usr/bin/yarn
npm: 6.13.6 - /usr/local/bin/npm
Languages:
Python: 2.7.17 - /usr/bin/python
npmPackages:
gatsby: ^2.19.41 => 2.19.41
gatsby-image: ^2.2.43 => 2.2.43
gatsby-plugin-config: ^1.0.7 => 1.0.7
gatsby-plugin-create-client-paths: ^2.1.23 => 2.1.23
gatsby-plugin-emotion: ^4.1.24 => 4.1.24
gatsby-plugin-flow: ^1.2.0 => 1.2.0
gatsby-plugin-manifest: ^2.2.47 => 2.2.47
gatsby-plugin-offline: ^3.0.40 => 3.0.40
gatsby-plugin-react-helmet: ^3.1.23 => 3.1.23
gatsby-plugin-sharp: ^2.4.12 => 2.4.12
gatsby-plugin-stripe: ^1.2.4 => 1.2.4
gatsby-source-contentful: ^2.1.96 => 2.1.96
gatsby-source-filesystem: ^2.1.54 => 2.1.54
gatsby-transformer-sharp: ^2.3.18 => 2.3.18
npmGlobalPackages:
gatsby-cli: 2.8.22
gatsby: 2.19.12

[pvdz]

@liviu-marciu I see two different gatsby version. I'm pretty sure this problem will bisect to the version that introduced the reduxcache folder (.19 I think?).

I don't know, this is weird. Perhaps the Docker binding is somehow causing weird permission problems. As you can see this issue-tracker is not flooded with people who are having permission trouble, so this does appear to be something very specific to this kind of setup. If removing the node_modules and ~/.npm folders does not offer a solution, you'd have to start debugging this yourself.

Please make sure there's nothing in this setup that somehow uses root permissions to generate these folders or whatever. Did you check who "owns" these folders, anyways? Is it your user or root? What about the process that runs gatsby build, is that your user or another?

The steps for debugging are not that hard, normally. So if somebody is willing to debug let me know so I can help you out there.

[liviu-marciu]

I am executing everything under root users, so all files have got root permission.
Removing node_modules and ~/.npm didn't do the trick.

Happy to debug if you can guide me through.

Thanks.

[pvdz]

Does #22959 happen to solve this problem?

[drmats]

Because of #22959 I discovered #22999 today, which essentially is the same as this issue, but without docker involved.

There is #23005 which may also fix this issue.

[asgarli]

Tried with gatsby 2.20.24, it works now! Thanks for the fix!