Yarn causing to fail to build Docker container (ENOENT lstat)

[icarito]

Issue appears to be intermittent...

Step 18/19 : RUN yarn --ignore-engines --ignore-scripts --modules-folder ./public/lib && yarn postinstall
 ---> Running in bc98ea39dc08
yarn install v1.10.1
[1/4] Resolving packages...
�[91mwarning image-sequencer > ndarray-gaussian-filter > cwise > static-module > through2 > xtend > [email protected]: 
�[0m[2/4] Fetching packages...
[3/4] Linking dependencies...
�[91merror An unexpected error occurred: "ENOENT: no such file or directory, lstat '/app/public/lib/acorn'".
�[0minfo If you think this is a bug, please open a bug report with the information provided in "/app/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
Service 'web' failed to build: The command '/bin/sh -c yarn --ignore-engines --ignore-scripts --modules-folder ./public/lib && yarn postinstall' returned a non-zero code: 1
Makefile:2: recipe for target 'build' failed
make: *** [build] Error 1
Build step 'Execute shell' marked build as failure

[jywarren]

Lot of similar looking errors in yarn repo yarnpkg/yarn#4563

[icarito]

Yes I read them! And also tried a couple workarounds found there at https://github.com/publiclab/plots2/pull/3728/files

[jywarren]

Sorry, yeah, just wanting to document what we've tried, thanks!

[icarito]

Perhaps we can improve our Yarn deployment strategy taking some ideas from https://hackernoon.com/using-yarn-with-docker-c116ad289d56 - not sure if some version locking could help prevent issues like this.

[jywarren]

Hm, like including a yarn.lock?

[jywarren]

Oh, like the section on Wait! Now Yarn is executed at each container build?

Is this an intermittent issue or does the process always fail?

[jywarren]

Wondering if @Souravirus would be able to help figure this out, as he originally helped complete the Yarn transition work in #2659 ?

[Souravirus]

Yeah sure @jywarren, I would try my best to help

[Souravirus]

@jywarren yarn.lock is already included in our code

[icarito]

Here's another report from a similar issue with npm. I guess this is a permissions issue, possibly coming from an npm package, and arising from mounting the /app directory as a volume. Perhaps it can be solved by adding a chmod or chown to Makefile.

[Radhikadua123]

While I was creating this fix: #3775, I realised this issue. I'm able to reproduce it locally very well. I believe it's because of old yarn.lock file. After the new yarn.lock file, even if I delete modules directory, it doesn't give me this error. May be we should commit that file with latest changes ?

[Souravirus]

How did you reproduce this error locally @Radhikadua123?

[Radhikadua123]

@Souravirus

rm -rf public (basically remove all the changes by yarn install)
git checkout public 
yarn parameters (it would give same error but there would be changes in the yarn.lock)
yarn parameters (Run it again and it won't give error. I think due to yarn.lock changes)

[jywarren]

Hi! Very cool, could you share a link to a branch on GitHub where youve reproduced this and sourav can push it to our unstable container to see if it gets solved there? Then we can try merging in your changes here.

…

On Thu, Oct 25, 2018, 3:01 PM Radhika Dua ***@***.*** wrote: @Souravirus <https://github.com/Souravirus> rm -rf public git checkout public yarn parameters (it would give same error but there would be changes in the yarn.lock) yarn parameters (Run it again and it won't give error. I think due to yarn.lock changes) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3727 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABfJ7FDTg4HITDqzFTax_DVhSNpG4-Sks5uogpagaJpZM4XyP_s> .

[Souravirus]

Yeah sure. @Radhikadua123 please update your changes in one of your branches and share it here, so that I can try pushing it in unstable branch.

[Radhikadua123]

@jywarren @Souravirus Let me push them to #3775

[Souravirus]

@jywarren updating the yarn.lock worked. Here is the link to the Jenkins result https://jenkins.laboratoriopublico.org/job/Plots-Unstable/322/console. Please see #3775.

[icarito]

Hi Radhika Please yes, make the push for that commit and we can test it! Currently we push to `unstable` branch and it automatically gets deployed to https://unstable.publiclab.org/ - we would push your commits to that branch and see whether it deploys correctly or not! Thanks!

…

On 25/10/18 12:14, Radhika Dua wrote: While I was creating this fix: #3775 <#3775>, I realised this issue. I'm able to reproduce it locally very well. I believe it's because of old yarn.lock file. May be we should commit that file with latest changes ? I can push the fix in that PR but I'm not sure how it will be tested in docker since I'm completely unaware of CI setup. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#3727 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMMS7VkKdndzNoAXJNIoTuz064EioZqks5uofGIgaJpZM4XyP_s>.

[icarito]

Oh, cool!

…

On 25/10/18 15:51, Sourav Sahoo wrote: @jywarren <https://github.com/jywarren> updating the yarn.lock worked. Here is the link to the Jenkins result https://jenkins.laboratoriopublico.org/job/Plots-Unstable/322/console. Please see #3775 <#3775>. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#3727 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMMSxu8Haa1iIN5NmfkBZ5d7lUbfxnEks5uoiQ3gaJpZM4XyP_s>.

[Radhikadua123]

Fixed with #3775