Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trailing slash in outputs array prevents correct caching #10549

Closed
niklasravnsborg opened this issue Jun 1, 2022 · 11 comments · Fixed by #10708
Closed

Trailing slash in outputs array prevents correct caching #10549

niklasravnsborg opened this issue Jun 1, 2022 · 11 comments · Fixed by #10708
Labels

Comments

@niklasravnsborg
Copy link
Contributor

Current Behavior

I created a target with the following configuration:

{
...
  "targets": {
    "build": {
    "executor": "nx:run-commands",
    "outputs": ["some-folder-to-cache/"],
    "options": {
      "command": "mkdir -p some-folder-to-cache/ && echo foo > some-folder-to-cache/hello.txt"
    }
  }
...
}

When I run the target the first time (nx build myapp), it properly generates hello.txt in some-folder-to-cache. When I run it the second time, there is a cache-hit (Nx read the output from the cache instead of running the command for 1 out of 1 tasks.), but after that some-folder-to-cache is empty.

Expected Behavior

I would expect the previously cached file to still be present.

Reproduction

I found that I can fix the error, when I omit the trailing slash in the outputs array:

"outputs": ["some-folder-to-cache"],

instead of

"outputs": ["some-folder-to-cache/"],

Environment

 >  NX   Report complete - copy this into the issue template

   Node : 16.13.2
   OS   : darwin x64
   npm  : 8.1.2

   nx : 14.1.9
   @nrwl/angular : Not Found
   @nrwl/cypress : Not Found
   @nrwl/detox : Not Found
   @nrwl/devkit : 14.1.9
   @nrwl/eslint-plugin-nx : Not Found
   @nrwl/express : Not Found
   @nrwl/jest : 14.1.9
   @nrwl/js : Not Found
   @nrwl/linter : 14.1.9
   @nrwl/nest : Not Found
   @nrwl/next : Not Found
   @nrwl/node : Not Found
   @nrwl/nx-cloud : 14.0.5
   @nrwl/nx-plugin : Not Found
   @nrwl/react : Not Found
   @nrwl/react-native : Not Found
   @nrwl/schematics : Not Found
   @nrwl/storybook : Not Found
   @nrwl/web : Not Found
   @nrwl/workspace : 14.1.9
   typescript : 4.7.2
   ---------------------------------------
   Community plugins:
@AgentEnder AgentEnder added the scope: core core nx functionality label Jun 6, 2022
danielsharvey added a commit to danielsharvey/nx that referenced this issue Jun 11, 2022
Fix by removing trailing slash when performing `cp -a` into cache.

Add test to confirm fix.

Resolves nrwl#10549
@FrozenPandaz
Copy link
Collaborator

I made these changes on my nx-examples clone and saw that the files were properly cached with the trailing slash:

https://github.com/FrozenPandaz/nx-examples/tree/10549

Can you retry with the latest version and/or provide a repo that exhibits the issue?

@FrozenPandaz FrozenPandaz added the blocked: retry with latest Retry with latest release or head. label Jul 29, 2022
@danielsharvey
Copy link
Contributor

Thanks for the work @FrozenPandaz, I've reproduced and tested my fix in your example. I cloned your example above and used latest nx (14.5.1).

Note that @niklasravnsborg and I are running on Mac. What platform are you testing?

Output reproducing error:

Daniels-MBP:nx-examples daniel$ nx run cart:build

> nx run cart:build


 ——————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  NX   Successfully ran target build for project cart (2s)
 
   See Nx Cloud run details at https://nx.app/runs/G7MabqMHdyW

Daniels-MBP:nx-examples daniel$ ls some-folder-to-cache/
hello.txt
Daniels-MBP:nx-examples daniel$ nx run cart:build

> nx run cart:build  [local cache]


 ——————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  NX   Successfully ran target build for project cart (112ms)
 
   Nx read the output from the cache instead of running the command for 1 out of 1 tasks.
 
   See Nx Cloud run details at https://nx.app/runs/4en6sZpCUnk

Daniels-MBP:nx-examples daniel$ ls some-folder-to-cache/
Daniels-MBP:nx-examples daniel$ 

Manually patching with my fix #10708 corrects the error:

Daniels-MBP:nx-examples daniel$ rm -rf node_modules/.cache/nx
Daniels-MBP:nx-examples daniel$ nx run cart:build

> nx run cart:build


 ——————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  NX   Successfully ran target build for project cart (1s)
 
   See Nx Cloud run details at https://nx.app/runs/QZ7fpNbV6IM

Daniels-MBP:nx-examples daniel$ ls some-folder-to-cache/
hello.txt
Daniels-MBP:nx-examples daniel$ nx run cart:build

> nx run cart:build  [existing outputs match the cache, left as is]


 ——————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  NX   Successfully ran target build for project cart (74ms)
 
   Nx read the output from the cache instead of running the command for 1 out of 1 tasks.
 
   See Nx Cloud run details at https://nx.app/runs/Kj17lAMf9Tm

Daniels-MBP:nx-examples daniel$ ls some-folder-to-cache/
hello.txt
Daniels-MBP:nx-examples daniel$ 

nx report:

 >  NX   Report complete - copy this into the issue template

   Node : 16.15.0
   OS   : darwin arm64
   yarn : 1.22.15
   
   nx : 14.5.1
   @nrwl/angular : 14.5.1
   @nrwl/cypress : 14.5.1
   @nrwl/detox : Not Found
   @nrwl/devkit : 14.5.1
   @nrwl/eslint-plugin-nx : 14.5.1
   @nrwl/express : Not Found
   @nrwl/jest : 14.5.1
   @nrwl/js : 14.5.1
   @nrwl/linter : 14.5.1
   @nrwl/nest : Not Found
   @nrwl/next : Not Found
   @nrwl/node : Not Found
   @nrwl/nx-cloud : 14.2.0
   @nrwl/nx-plugin : Not Found
   @nrwl/react : 14.5.1
   @nrwl/react-native : Not Found
   @nrwl/schematics : Not Found
   @nrwl/storybook : 14.5.1
   @nrwl/web : 14.5.1
   @nrwl/workspace : 14.5.1
   typescript : 4.7.2
   ---------------------------------------
   Community plugins:
         @ngrx/component-store: 14.0.2
         @ngrx/effects: 14.0.2
         @ngrx/entity: 14.0.2
         @ngrx/router-store: 14.0.2
         @ngrx/store: 14.0.2
         rxjs: 6.5.5
         @ngrx/store-devtools: 14.0.2

@github-actions
Copy link

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs.
If we missed this issue please reply to keep it active.
Thanks for being a part of the Nx community! 🙏

@github-actions github-actions bot added the stale label Aug 14, 2022
@danielsharvey
Copy link
Contributor

Please keep issue active. I have re-tried and reproduced with the latest nx. Awaiting review by Nx team.

@github-actions github-actions bot removed the stale label Aug 15, 2022
@github-actions
Copy link

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs.
If we missed this issue please reply to keep it active.
Thanks for being a part of the Nx community! 🙏

@github-actions github-actions bot added the stale label Aug 29, 2022
@niklasravnsborg
Copy link
Contributor Author

There is already a fix for this available. The PR just needs to be reviewed. Is there anyone in the team that can take a look at it?

@github-actions github-actions bot removed the stale label Aug 30, 2022
@github-actions
Copy link

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs.
If we missed this issue please reply to keep it active.
Thanks for being a part of the Nx community! 🙏

@github-actions github-actions bot added the stale label Sep 14, 2022
@danielsharvey
Copy link
Contributor

Please keep issue active. I have re-tried and reproduced with the latest nx. Awaiting review by Nx team.

@github-actions github-actions bot removed the stale label Sep 15, 2022
@github-actions
Copy link

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs.
If we missed this issue please reply to keep it active.
Thanks for being a part of the Nx community! 🙏

@github-actions github-actions bot added the stale label Sep 29, 2022
@danielsharvey
Copy link
Contributor

Please keep issue active. See comments above. Awaiting review by Nx team.

danielsharvey added a commit to danielsharvey/nx that referenced this issue Sep 29, 2022
Fix by removing trailing slash when performing `cp -a` into cache.

Add test to confirm fix.

Resolves nrwl#10549
@github-actions github-actions bot removed the stale label Sep 30, 2022
meeroslav pushed a commit that referenced this issue Sep 30, 2022
…hing (#10708)

Fix by removing trailing slash when performing `cp -a` into cache.

Add test to confirm fix.

Resolves #10549
FrozenPandaz pushed a commit that referenced this issue Oct 4, 2022
…hing (#10708)

Fix by removing trailing slash when performing `cp -a` into cache.

Add test to confirm fix.

Resolves #10549

(cherry picked from commit 2889372)
@github-actions
Copy link

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants