Gatsby doesn't always refresh changes #27609

tu4mo · 2020-10-22T20:49:35Z

Description

Gatsby sometimes fails to reload changed components.

Issue can be reproduced after version v2.24.9, so something changed between 2.24.9-2.24.10.

Steps to reproduce

https://github.com/tu4mo/gatsby-reload-issue

git clone [email protected]:tu4mo/gatsby-reload-issue.git
cd gatsby-reload-issue
npm install
npm start

Run (multiple times if needed) in a another terminal:

node update-fakelib.js

Expected result

The scripts rewrites 10 components with current timestamp. All changed components are hot reloaded.

Actual result

Some, all or none of the components are updated, inconsistently.

Environment

  System:
    OS: macOS 11.0.1
    CPU: (8) x64 Intel(R) Core(TM) i7-8559U CPU @ 2.70GHz
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 14.15.1 - ~/.nvm/versions/node/v14.15.1/bin/node
    Yarn: 1.22.10 - /usr/local/bin/yarn
    npm: 6.14.8 - ~/.nvm/versions/node/v14.15.1/bin/npm
  Languages:
    Python: 2.7.16 - /usr/bin/python
  Browsers:
    Chrome: 87.0.4280.88
    Edge: 87.0.664.57
    Firefox: 83.0
    Safari: 14.0.1

The text was updated successfully, but these errors were encountered:

vladar · 2020-10-23T11:27:47Z

Oh, I see you've already followed my advice: #26580 (comment) 😄

We need some way to reproduce it to move forward. Does it happen for you on any project, including our blog starter? If not, please provide a minimal reproduction and we'll be able to take another look.

tu4mo · 2020-10-25T18:22:43Z

Yes, I can reproduce it with blog starter. Typing something in index.js, saving quickly twice (cmd-S), making a change, saving quickly twice; doing this a few times and you'll see that the change isn't shown on the page.

tu4mo · 2020-10-25T18:29:31Z

tu4mo · 2020-11-04T14:36:59Z

I also reversed blog starter to a commit that had a version later than v2.24.0 (v2.23.12) and I can't reproduce the issue there. Clearly something changed in the refresh mechanism after v2.24.0 and made it unreliable.

@vladar Is there anything else I can do to help with this issue getting fixed? Or am I forever stuck with v2.24.0... 😞

vladar · 2020-11-20T12:10:27Z

I tried to reproduce it with the latest gatsby version and also on 2.24.0 but I don't see stale results. Which makes me think that it may have something to do with your environment.

Do you have Dropbox or other similar tools installed in the same folder where your gatsby site is located? Maybe some file watchers in IDE or other tools?

vladar · 2020-11-20T12:17:07Z

Also, can you bisect a specific version where this problem was introduced for you? The 2.23.12 - 2.24.0 range has too many changes. We need to narrow it down.

tu4mo · 2020-11-20T13:11:25Z

I don't have Dropbox or any other file sharing apps using code files. In my project I have watchers that built the library (Rollup and TypeScript in watch mode).

Sorry for not being clear about the version. 2.24.9 works, 2.24.10 does not.

vladar · 2020-11-20T13:35:12Z

Then it sounds very similar to #26192 Have you tried upgrading to 2.24.57 as suggested here: #26192 (comment) ?

tu4mo · 2020-11-20T14:08:26Z

Tried 2.24.57, same thing, does not reload.

tu4mo · 2020-11-20T14:23:09Z

It feels like there's something little different in 2.24.57. After I edited a component in my library, I did get it reload by editing a Gatsby page where the component is imported. Sometimes it works, sometimes it doesn't, sometimes the whole reloading seems to freeze, and I have to restart Gatsby.

vladar · 2020-11-20T15:23:09Z

After I edited a component in my library, I did get it reload by editing a Gatsby page where the component is imported. Sometimes it works, sometimes it doesn't, sometimes the whole reloading seems to freeze, and I have to restart Gatsby.

But can you reproduce the problem on the blog starter with [email protected]? If not - then it means we are still stuck without any reproduction that we could use to troubleshoot your specific issue.

tu4mo · 2020-11-20T15:34:31Z

Yes, I can reproduce it everytime typing rapidly aaa, cmd-S, cmd-S, bbb, cmd-S, cmd-S, ccc, cmd-S, cmd-S, etc.

tu4mo · 2020-11-20T16:06:55Z

https://github.com/tu4mo/gatsby-reload-issue

git clone [email protected]:tu4mo/gatsby-reload-issue.git
cd gatsby-reload-issue
npm install
npm start

In a another terminal:

node update-fakelib.js

With v2.24.57 I rarely get to last update. Running it a few times and the page stops responding.

With v2.24.0 it works.

You might need to increase the amount of components "created" and decrease the sleep.

vladar · 2020-11-20T17:16:34Z

Thank you so much for reproduction! 💜

I definitely see how develop is getting stuck eventually. Also in the given version range - #25815 looks like a primary suspect.

Investigating this.

vladar · 2020-11-20T20:09:52Z

So one potential problem is here:

gatsby/packages/gatsby/src/state-machines/develop/index.ts

Lines 189 to 195 in 441263e

    
           recompiling: { 
        
             invoke: { 
        
               src: `recompile`, 
        
               onDone: { 
        
                 actions: `markSourceFilesClean`, 
        
                 target: `waiting`, 
        
               },

We call markSourceFilesClean upon finishing recompiling JS. But all changes that occur during recompile are discarded. That doesn't explain why it doesn't recover on subsequent file changes when in waiting state.

One possible hint - somehow webpack stops calling invalid hook (even though chokidar actually emits change events for those):

success Building development bundle - 5.377s
[chokidar_change] fakelib\Component.js
[webpack.compiler.hooks.invalid] fakelib\Component.js
success onPreExtractQueries - 0.004s
success extract queries from components - 0.062s
success write out requires - 0.001s
[chokidar_change] fakelib\Component.js
success Re-building development bundle - 0.204s
[webpack.compiler.hooks.invalid] fakelib\Component.js
[chokidar_change] fakelib\Component.js
[chokidar_change] fakelib\Component.js
[chokidar_change] fakelib\Component.js
[chokidar_change] fakelib\Component.js
[chokidar_change] fakelib\Component.js
[chokidar_change] fakelib\Component.js
[chokidar_change] fakelib\Component.js
[chokidar_change] fakelib\Component.js

So the next step is to dive deep into suspend/resume of the webpack. Just leaving some notes here to keep the context around.

vladar · 2020-11-23T12:59:00Z

OK. I think I've found the issue. This PR should fix it (hopefully): #28237

vladar · 2020-11-24T17:26:58Z

Published in [email protected]. Can you please check if it fixes the issue for you?

tu4mo · 2020-11-24T18:42:13Z

Tested, and it's better as in it doesn't crash anymore. But it still fails refreshing most of the time :(

Tried editing a Badge component in my library 4 times. Here is the browser log.

edit:

[HMR] bundle rebuilding
[HMR] bundle rebuilt in 10138ms

Component didn't update.

edit:

[HMR] bundle rebuilding
[HMR] bundle rebuilt in 11960ms
[HMR] Checking for updates on the server...
[HMR] Updated modules:
[HMR]  - ../lib/dist/lib/components/Badge/Badge.js
[HMR] App is up to date.

Component updated.

edit:

[HMR] bundle rebuilding
[HMR] bundle rebuilt in 11997ms
[HMR] bundle rebuilding
[HMR] bundle rebuilt in 419ms

Component didn't update.

At that time I also noticed this in terminal, not sure if related:

warning Warning: Event "xstate.after(200)#waitingMachine.aggregatingFileChanges" was sent to stopped service "waitingMachine". This service has already reached its final state, and will not transition.
Event: {"type":"xstate.after(200)#waitingMachine.aggregatingFileChanges"}

edit:

[HMR] bundle rebuilding
[HMR] bundle rebuilt in 532ms

Component didn't update.

Gave up after that.

tu4mo · 2020-11-24T18:46:37Z

I also see this in the terminal 4 times:

info changed file at /Users/tu4mo/Code/styleguide/lib/components/Badge/Badge.tsx

So Gatsby seems to pick up the change, but the rebuild times are much shorter.

tu4mo · 2020-11-24T20:38:08Z

I also updated Gatsby in https://github.com/tu4mo/gatsby-reload-issue and I can still reproduce this emulating multiple component updates.

tu4mo · 2020-11-26T07:06:40Z

@vladar Maybe you could re-open this issue?

tu4mo · 2021-03-11T14:13:59Z

Sorry but it didn't. Still reproducible with gatsby-reload-issue and in my real project.

vladar · 2021-03-11T22:21:16Z

There was another issue with webpack v5 which is hopefully fixed by #30193. Can you try again with [email protected]+ef7378ea10?

tu4mo · 2021-03-12T06:52:23Z

With 3.1.0-alpha-hmr-fix.11+ef7378ea10:

npm start
node update-fakelib.js

Components do not update, and this appears in console:

After hard-reload I see the updated components.

Second attempt with node update-fakelib, components update 🎉
Third try with node update-fakelib, components don't update 💔

vladar · 2021-03-12T13:06:46Z

Yeah, there were actually several underlying issues. And one was a race condition that was not that easy to repro. Can you please try again with [email protected]+92e67e9f09

I've tried an alternative approach there: synchronizing with webpack events properly and not relying on webpack working correctly with any sequence of resume/suspend calls at random moments.

tu4mo · 2021-03-12T14:09:16Z

I couldn't reproduce anymore with gatsby-repro-issue. I'll have to figure out if I can find another way.

On a larger scale, in my real app, I'm still having the old behavior though 😕. Here's a log of making changes in a Gatsby page. It reloaded changes a couple of times, but most of the times it didn't (those <1000ms rebuilds).

vladar · 2021-03-12T15:46:51Z

I suspected that there are several distinct issues there. Can you try again with [email protected]+831f9a9d64. If it doesn't help we will need a new repro (as the existing repro likely hits a different issue than your main project).

tu4mo · 2021-03-12T17:08:07Z

Definitely not reliable as v2.24.9 but this version seems to work almost everytime on my real project. I did get the same error as in #27609 (comment) after a few edits thought.

vladar · 2021-03-12T18:29:28Z

Opened an issue for the first bug in the webpack repo: webpack/webpack#12882

Still need to find a good repro for the second bug (edit: second issue: webpack/webpack#12898).

github-actions · 2021-04-05T00:06:39Z

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here.
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

tu4mo · 2021-04-06T06:02:01Z

Not stale.

pirtlj · 2021-05-09T23:38:55Z

Yeah this is still an issue for me

dustinlacewell · 2021-05-10T07:10:32Z

Does this issue have anything to do with the behavior of Gatsby not knowing about any of the types it should infer from gatsby-node queries until I hit the __refresh endpoint?

tu4mo · 2021-05-19T07:18:28Z

@vladar webpack v5.37.1 was just released with watching fixes.

vladar · 2021-05-20T12:40:29Z

Have you tried it? Maybe remove yarn.lock and try on your project? Gatsby has this dependency definition for webpack "webpack": "^5.35.0", so it shouldn't block you from upgrading?

tu4mo · 2021-05-20T12:52:02Z

Yes, I tried it. I have another issue with Gatsby v3 so I haven't yet been able to completely switch to v3, but regarding this issue things are looking good 🎉

pbxr250 · 2021-06-03T11:50:55Z

In my case the naming of components have caused the issue

tu4mo · 2021-06-09T09:50:13Z

I suggest opening a new issue. This particular issue that I had has been resolved in webpack. Closing this.

tu4mo · 2021-06-09T14:01:40Z

Extra special thanks to @vladar for sticking with me through this!

runemadsen · 2021-07-20T08:22:10Z

I just want to echo @pbxr250 comment: I was running into a bunch of issues related to hot module reloading, and it was only in a single component. Basically, my first change to either the CSS or JS would show in the browser, but all following edits would not. There were no warnings or errors in the console.

I finally figured out it was because of a camel-case problem. My component was named WhatWeAreBuilding.js and it was imported as WhatWeArebuilding.js (with a lower-case b). In my humble opinion, this should have thrown an import error, as that file doesn't exist. Instead, it seems to work, but only for the first hot module reload.

USAIPOST · 2022-02-03T19:48:29Z

@runemadsen camel-case solution worked for me. When I switched branches, one of my file names had more than one uppercase characters in a row. I had to fix the file name.

jayantamadhav · 2023-03-21T12:46:55Z

In my case the naming of components have caused the issue

Thanks, this was the issue in my case. I named the file with capital first and then was importing that component using all lowercase.

Suggestion: Maybe make import checks case-sensitive?

tu4mo added the type: bug An issue or pull request relating to a bug in Gatsby label Oct 22, 2020

gatsbot bot added the status: triage needed Issue or pull request that need to be triaged and assigned to a reviewer label Oct 22, 2020

vladar added topic: hot reloading* and removed status: triage needed Issue or pull request that need to be triaged and assigned to a reviewer labels Oct 23, 2020

vladar mentioned this issue Nov 20, 2020

Hot reload broken since 2.24.0 #28202

Closed

vladar added the status: needs reproduction This issue needs a simplified reproduction of the bug for further troubleshooting. label Nov 20, 2020

vladar removed the status: needs reproduction This issue needs a simplified reproduction of the bug for further troubleshooting. label Nov 20, 2020

vladar mentioned this issue Nov 23, 2020

fix(gatsby): do not ignore source file changes during recompilation #28237

Merged

vladar closed this as completed in #28237 Nov 23, 2020

kelvindecosta mentioned this issue Mar 11, 2021

fix(gatsby): workaround some webpack issues causing first save after running gatsby develop to not have any effect #30193

Merged

github-actions bot added the stale? Issue that may be closed soon due to the original author not responding any more. label Apr 5, 2021

github-actions bot removed the stale? Issue that may be closed soon due to the original author not responding any more. label Apr 6, 2021

LekoArts added the type: upstream Issues outside of Gatsby's control, caused by dependencies label Apr 6, 2021

LekoArts removed this from the Next Major milestone Apr 21, 2021

tu4mo mentioned this issue May 10, 2021

fix: do not skip changes when compilation is suspended webpack/webpack#12897

Closed

LekoArts added topic: DX Developer Experience (e.g. Fast Refresh, i18n, SSR, page creation, starters) and removed topic: hot reloading labels May 28, 2021

tu4mo closed this as completed Jun 9, 2021

Gatsby doesn't always refresh changes #27609

Gatsby doesn't always refresh changes #27609

Comments

tu4mo commented Oct 22, 2020 • edited Loading

Description

Steps to reproduce

Expected result

Actual result

Environment

vladar commented Oct 23, 2020

tu4mo commented Oct 25, 2020

tu4mo commented Oct 25, 2020

tu4mo commented Nov 4, 2020

vladar commented Nov 20, 2020

vladar commented Nov 20, 2020

tu4mo commented Nov 20, 2020

vladar commented Nov 20, 2020

tu4mo commented Nov 20, 2020

tu4mo commented Nov 20, 2020

vladar commented Nov 20, 2020

tu4mo commented Nov 20, 2020

tu4mo commented Nov 20, 2020 • edited Loading

vladar commented Nov 20, 2020

vladar commented Nov 20, 2020 • edited Loading

vladar commented Nov 23, 2020

vladar commented Nov 24, 2020

tu4mo commented Nov 24, 2020

tu4mo commented Nov 24, 2020 • edited Loading

tu4mo commented Nov 24, 2020 • edited Loading

tu4mo commented Nov 26, 2020

tu4mo commented Mar 11, 2021

vladar commented Mar 11, 2021

tu4mo commented Mar 12, 2021

vladar commented Mar 12, 2021 • edited Loading

tu4mo commented Mar 12, 2021 • edited Loading

vladar commented Mar 12, 2021

tu4mo commented Mar 12, 2021

vladar commented Mar 12, 2021 • edited Loading

github-actions bot commented Apr 5, 2021

tu4mo commented Apr 6, 2021

pirtlj commented May 9, 2021

dustinlacewell commented May 10, 2021

tu4mo commented May 19, 2021

vladar commented May 20, 2021

tu4mo commented May 20, 2021

pbxr250 commented Jun 3, 2021

tu4mo commented Jun 9, 2021

tu4mo commented Jun 9, 2021

runemadsen commented Jul 20, 2021

USAIPOST commented Feb 3, 2022

jayantamadhav commented Mar 21, 2023

tu4mo commented Oct 22, 2020 •

edited

Loading

tu4mo commented Nov 20, 2020 •

edited

Loading

vladar commented Nov 20, 2020 •

edited

Loading

tu4mo commented Nov 24, 2020 •

edited

Loading

tu4mo commented Nov 24, 2020 •

edited

Loading

vladar commented Mar 12, 2021 •

edited

Loading

tu4mo commented Mar 12, 2021 •

edited

Loading

vladar commented Mar 12, 2021 •

edited

Loading