Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use content hash in pages chunk name #6303

Closed
wbr911 opened this issue Feb 15, 2019 · 27 comments · Fixed by #14279
Closed

use content hash in pages chunk name #6303

wbr911 opened this issue Feb 15, 2019 · 27 comments · Fixed by #14279
Milestone

Comments

@wbr911
Copy link

wbr911 commented Feb 15, 2019

Feature request

Is your feature request related to a problem? Please describe.

GitHub Logo
for entries under pages folder , nextJs will not add content_hash to entries' name. Instead the path will be __next/$nextVersion/page/$pageName.js.
As a result , each time when we make a new release , nextJs will generate a new version, and all the path of pages js will be changed and in turn the web cache is gone. For example, if we have 10 page , and we only modify one page in next release , but all the 10 page js need to be downloaded again.

Describe the solution you'd like

use content hash in page name
__next/$nextVersion/page/$pageName.js. => __next/page/$pageName_$contentHash.js

@ccfnever
Copy link

Aha~! I am planning to submit an issue like this. This is also a problem that has been bothering me for a long time.

As @wbr911 said, this wastes a lot of resources and greatly reduces development and user experience.

@timneutkens
I remember u said 「Adding hashes to page files is going to be up to us the future.」last year.#4255
Can you provide the next solution, it is very important.
Look forward to your favourable reply~~~~

@timneutkens
Copy link
Member

timneutkens commented Feb 15, 2019

It's slightly more complicated than saying "I want to add hashes", the problem space is bigger than that. For example how are you going to route? The client side is not aware of the hashes for the script tag, and it shouldn't have a list of all possible routes + hashes (that wouldn't scale).

I've just created a flow chart that (from the top of my mind) explains how the router currently works:

screenshot 2019-02-15 at 13 23 33

This issue actually comes really close to the dynamic routing one, because there needs to be a way to specify patterns to a certain page.

Adding hashes would be super simple if for example we were to do something like:

import Link, {page} from 'next/link'

export default function MyPage() {
   return <>
     <Link href={page`./blog/post.js`} as="/blog/hello-world">
       <a>To Hello World Blog Post</a>
	 </Link>
   </>
}

Because in that case we can compile the href into the hashed page route.

Another one that came up when thinking about this is:

import Link, {page} from 'next/link'

export default function MyPage() {
   return <>
     <Link href={() => import('./blog/post.js')} as="/blog/hello-world">
       <a>To Hello World Blog Post</a>
	 </Link>
   </>
}

Note that this is just me brainstorming on solutions, this is not something we're currently implementing (still thinking more about it).

@ccfnever
Copy link

Thank @timneutkens for your detailed answers, and I agree that this feature is difficult to achieve.

Maybe we have to alleviate this problem by dividing complex projects into multiple next.js apps.

@wbr911
Copy link
Author

wbr911 commented Feb 17, 2019

@timneutkens First of All , Thanks a lot for your patient explanation
I could understand your concerns about routes, while my architecture is MPA (multiple-pages) instead of SPA which means we don't have the routes problems.
Could NextJs provides an alternative to add hashes to page js for MPA cases?

@timneutkens
Copy link
Member

@wbr911 could expand on this more, do you mean you're not using next/link and doing full page transitions?

@wbr911
Copy link
Author

wbr911 commented Feb 20, 2019

@wbr911 could expand on this more, do you mean you're not using next/link and doing full page transitions?

yes @timneutkens

@timneutkens
Copy link
Member

There is no way for us to distinguish between that, so the solution has to be generic and work for all cases.

@wbr911
Copy link
Author

wbr911 commented Feb 21, 2019

could you let developer do it, for example to add a configuration in next.config.js to declare whether use contentHash and remove versionId in the path of page.js @timneutkens

@timneutkens
Copy link
Member

It's not a simple as changing a configuration. We can't maintain 2 versions of page loading and building just for this one use case.

@wbr911
Copy link
Author

wbr911 commented Feb 22, 2019

just to share my solution to people sharing same situation with us:

  1. use hash in filename !isDev && (config.output.filename = '[name]_[chunkhash:8].js');
  2. use assets-webpack-plugin to record the relationship between entry and hashed js name;
  3. create custom _document.js and build your own NextScript by extending nextJs's one to replace js src with the json file created at step2

@azizhk
Copy link
Contributor

azizhk commented Mar 23, 2019

So I got the custom NextScript part working.

But this is very much tied with implementation details of next.js and is very fragile and can break with next.js versions. I think if next.js would be open to a few changes, then just a plugin to modify the asset filename can become very easy:

  1. Save the asset path which is currently being ignored.
    https://github.com/zeit/next.js/blob/a6ddaefe226b7ecf8d984244f82114df7e12c909/packages/next/build/webpack/plugins/build-manifest-plugin.js#L65-L67
  2. Use the path in NextScript and Head components
    https://github.com/zeit/next.js/blob/eb28d4cdcff48fb9d2d1f3f7e8ab93e5f9d334ce/packages/next/pages/_document.js#L445-L462

@timneutkens if you would be ok with this I can create a PR.

Then next-plugin creators like me can easily modify the asset pathname from webpack.
Then there is the part of Link component. Which we can create a wrapper around the original next/link and pick the href and map it to the correct asset path and set the correct url using asPath (unless specified). This would be very easy.

@timneutkens
Copy link
Member

and map it to the correct asset path and set the correct url using asPath (unless specified). This would be very easy.

Actually it's not easy, because it implies you know all routes + it doesn't account for dynamic href or as. Either way we're working on something to address this and it will be very different from above solution, so I would not pursue a PR for it at this point in time 🙏

@azizhk
Copy link
Contributor

azizhk commented Mar 23, 2019

Can you share your solution that you are working on or if there is a PR or a task list, maybe we can contribute?

@timneutkens
Copy link
Member

I'm still discussing it internally before writing up something.

@azizhk
Copy link
Contributor

azizhk commented Mar 24, 2019

@timneutkens
Copy link
Member

Posted #7329

@zen0wu
Copy link

zen0wu commented Aug 14, 2019

Correct me if I'm wrong. What's the issue of storing the page->hash map on the client side and do a lookup during page transition when via Link? I'm sure I missed something.

A middle ground IMO is to generate a BUILD ID that's a total hash of all frontend source code. This helps in my case, to not invalidate frontend cache when there's only backend changes. Do you guys think it's reasonable to do this?

@azizhk
Copy link
Contributor

azizhk commented Aug 14, 2019

@ZenoZen yup it's possible. I've documented the process https://medium.com/@azizhk/next-js-netlify-file-naming-and-caching-strategies-8665b213963f
Though not recommended by Next.js developers and can break in future versions of Next.js.

@zen0wu
Copy link

zen0wu commented Aug 14, 2019

@azizhk Yeah, I've seen your blog post. It's really helpful!

The process sounds just right, with the asset hash mapping dictionary. But as you said, the hack is a bit too involved and likely to break when an official solution comes up. That's why I'm considering the single total hash solution and hopefully we don't have to wait long for an official solution :)

@timneutkens
Copy link
Member

I would highly recommend not using the outlined solution, it will break for every single release we'll do, especially the ones that will be out soon.

@superkoh
Copy link

@timneutkens Is there any update on this issue? A small change cause all js need to be downloaded again on user side seems not elegant, especially for big sites. I'm eager for the solution.

@codingcircus
Copy link

@timneutkens also asking for an update on this. Since we often have multiple deploys of the same nextjs applications per week or even per day, we are constantly invalidating caches, even from pages, that didn't change. A more user focused solution on this problem would be great.

@timneutkens
Copy link
Member

We're making some changes soon, but will take a while as it's non-trivial to make it work. Feel free to reach out to [email protected] if you want us working on it in a shorter timeframe.

@kodiakhq kodiakhq bot closed this as completed in #14279 Jun 20, 2020
kodiakhq bot pushed a commit that referenced this issue Jun 20, 2020
Updates the way filenames are generated for browser compilation.
Notably:
- All entry bundles now have hashes in production, this includes pages (previously pages used a buildId in the path)
- The AmpFiles no longer depends on hardcoded bundle names, it uses the buildManifest instead (internals)
- All cases where we match the page name from the chunk/entrypoint name now use the same function `getRouteFromEntrypoint` (internals)
- In development we no longer include the "faked" `buildId` set to `development` for page files, instead we just use the `/_next/static/pages` path (was `/_next/static/development/pages`). This was changed as it caused unneeded complexity and makes generating the bundles easier (internals)
- Updated tons of tests to be more resilient to these changes by relying on the buildManifest instead of hardcoded paths (internals)

Follow up of these PRs:
#13759
#13870
#13937
#14130
#14176
#14268


Fixes #6303
Fixes #12087 
Fixes #1948
Fixes #4368
Fixes #4255
Fixes #2548
@timneutkens timneutkens modified the milestones: backlog, iteration 3 Jun 20, 2020
@timneutkens timneutkens reopened this Jun 20, 2020
@timneutkens timneutkens modified the milestone: iteration 3 Jun 20, 2020
rokinsky pushed a commit to rokinsky/next.js that referenced this issue Jul 11, 2020
Updates the way filenames are generated for browser compilation.
Notably:
- All entry bundles now have hashes in production, this includes pages (previously pages used a buildId in the path)
- The AmpFiles no longer depends on hardcoded bundle names, it uses the buildManifest instead (internals)
- All cases where we match the page name from the chunk/entrypoint name now use the same function `getRouteFromEntrypoint` (internals)
- In development we no longer include the "faked" `buildId` set to `development` for page files, instead we just use the `/_next/static/pages` path (was `/_next/static/development/pages`). This was changed as it caused unneeded complexity and makes generating the bundles easier (internals)
- Updated tons of tests to be more resilient to these changes by relying on the buildManifest instead of hardcoded paths (internals)

Follow up of these PRs:
vercel#13759
vercel#13870
vercel#13937
vercel#14130
vercel#14176
vercel#14268


Fixes vercel#6303
Fixes vercel#12087 
Fixes vercel#1948
Fixes vercel#4368
Fixes vercel#4255
Fixes vercel#2548
@subodh-malgonde
Copy link

FYI - this feature was a part of v9.5 release.

@pakerhoi
Copy link

pakerhoi commented Feb 4, 2021

Are there any plans to add a content hash to files from the public folder, like images and fonts?

@Kavian77
Copy link

Are there any plans to add a content hash to files from the public folder, like images and fonts?

It's important to have something like content hash in their name, so we can use immutable directive no them as well.
No plans? @timneutkens
Thanks in advance.

@balazsorban44
Copy link
Member

This issue has been automatically locked due to no recent activity. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.

@vercel vercel locked as resolved and limited conversation to collaborators Jan 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.