Custom Cache-Control response header for /_next/image

[jerrygreen]

Bug report

Describe the bug

When requesting /_next/image(.*), I'm getting this Response header:

Cache-Control: public, max-age=60

And it's fine... What's not fine is that I'm getting the same exact Response headers even when I use custom headers in next.config.js, i.e.:

module.exports = {
  async headers() {
    return [
      {
        // This works, and returns appropriate Response headers:
        source: '/(.*).jpg',
        headers: [
          {
            key: 'Cache-Control',
            value:
              'public, max-age=180, s-maxage=180, stale-while-revalidate=180',
          },
        ],
      },
      {
        // This doesn't work for 'Cache-Control' key (works for others though):
        source: '/_next/image(.*)',
        headers: [
          {
            key: 'Cache-Control',
            // Instead of this value:
            value: 'public, max-age=180, s-maxage=180, stale-while-revalidate=180',
            // Cache-Control response header is `public, max-age=60` in production
            // and `public, max-age=0, must-revalidate` in development
          },
        ],
      },
    ]
  },
}

To Reproduce

Try this next.config.js above, with an image that uses common <img> tag like this:

<img src="/YOUR_IMAGE_IN_PUBLIC_FOLDER.jpg">

And another tag that uses the new next/image component, with the same or whatever image url, for example:

<Image src="/YOUR_IMAGE_IN_PUBLIC_FOLDER.jpg">

And try requesting those images in the browser, and look into Devtools -> Network tab, the Response headers, and see that for common <img> tag it's actually changed, whereas for new <Image> component it is not.

Expected behavior

For /_next/image(.*) urls, I expected to see Headers that I setup, i.e. Cache-Control: public, max-age=180, s-maxage=180, stale-while-revalidate=180.

Screenshots

Common <img>:

New <Image>:

System information

• OS: not applicable / any
• Browser (if applies): not applicable / any
• Version of Next.js: 10.0.3
• Version of Node.js: 12.18.0
• Deployment: both next start on development machine, and actual Vercel deployment (screenshots demonstrate the case with local machine, but I've confirmed that the same is happening on a Vercel deployment)

Additional context

Originally I created an issue using "Feature request" template, and it got translated into "Idea discussion": #19896 (how to delete/close it pls? doesn't seem to be possible), - initially I wasn't sure if it's even possible to setup custom headers, and how that supposed to work, though I just needed that functionality. But later, upon discovering that there's such a functionality already, I figured out that actually /_next/image overrides both custom headers in next.config.js and headers property of Vercel config, i.e. it's not just some idea of a feature that is "not implemented", but it's actual bug.

[mehmetpekcan]

Are there any update here?

[aresrioja10]

Any updates? Lighthouse is lowering my score because I must serve the images with a more efficient cache policy

[guilehm]

I had to rollback to next 10.0.1, since this problem does not happen in that version.
I cannot serve my images with max-age=0.

[alexbrazier]

Looks like it was broken here https://github.com/vercel/next.js/pull/18986/files#diff-0a3c67e4ce009de800755e5b9944391b6747a989e4e3ec31586cfbf42f203bb4R305 and here https://github.com/vercel/next.js/pull/18986/files#diff-0a3c67e4ce009de800755e5b9944391b6747a989e4e3ec31586cfbf42f203bb4R152 in #18986

[PaulSenon]

Hi, I think I just had the same problem... And I found a solution to it.

TL;DR: just set the cache-control header from your provided images, pointed by the url param in /_next/image?url=....

Context:

• I'm using Vercel hosting with default CDN, and [email protected]
• I'm hosting my images on a public S3 bucket
• I use the nextJs <Image/> component with url pointing on my S3 so it construct the /_next/image?url=... for me
• The cache-control max-age is always 60 whatever I write in my next.config.js or vercel.json files...

I tried:

• Setting cache-control header through next.config.js
• Setting cache-control header through vercel.json
• Rollbacking to next 10.0.1 as suggested above and testing again the two previous points.

=> But still have the issue. I cannot control the cache of /_next/image (works just well on any other routes)

After diving a bit more into that, I found somewhere in the Vercel documentation where they clearly explain why it behaves like this, and therefore I suppose it is not a bug.

It says:

If no max-age value is provided in a Cache-Control header returned by the image URL/path, a default max-age of 60 seconds is set. In turn, source images will be retrieved again and optimized on incoming requests if more than 60 seconds have passed since the last optimization of that image (in the same Edge Network node).

So I just gave it a try, by setting the cache-control header directly on my files hosted on my S3 bucket. I first seen no changes and I thought "oh well... It is definitively bugged..." but like 20min later, when I was into something else, I just found that it actually worked !!!! it just take time to get the CDN aware of your changes on the original resource. I don't understand why, because the default TTL must be 60sec, but anyway... It works.

So I hope it helps, tell me if your issue wasn't exactly this, but I'm sure it will help someone one day so I posted this here.
It took me half a day to get it to work but I've learned plenty of stuff on browser and CDN caching. Let's say it was worth the cost of the fight :)

[aresrioja10]

Hi @PaulSenon !

I'm gonna try your solution right now. I'm hosting my images in Wordpress so I've added this code into my .htaccess:

<IfModule mod_headers.c>
  <FilesMatch "\.(jpg|png|svg)">
    Header set Cache-Control "max-age=604800, public"
  </FilesMatch>
</IfModule>

[aresrioja10]

@PaulSenon It works! 🎉

[PaulSenon]

@aresrioja10 Cool! Glad it helped!

@jerrygreen as you opened this issue, please give a try to this solution and eventually close the issue if it solves your problem too :)

[alexbrazier]

I don't think this fixes the issue if you are hosting the images locally with next, e.g. the path is <Image src="/images/local.jpg" /> as you can't set the cache control on those. It still seems to be a bug if it is ignoring your next.config.js.

[TheThirdRace]

Not sure if it applies to your particular case, but here's what I could find out with my own tests.

>>> I don't set any Cache-Control header <<< and host images on Vercel directly

DEV

The cache-control header is always: Cache-Control: public, max-age=0, must-revalidate.

Don't forget to disable the Disable Cache in the Network tab when using Chrome Dev Tools (must be the same with Firefox)

When cache is active, the image is cached correctly (response 304 Not Modified).

When revalidating to know if the cache is ok, it checks the Etag with an If-none-match header. If the resource hasn't changed, it will use the cached version.

Given the cache mechanism used here, there will still be a network call to validate the image hasn't changed, but it's a few bytes instead of the normal KB.

Verdict => It works as expected

PROD

The cache-control header is always: Cache-Control: public, max-age=0, must-revalidate.

Don't forget to disable the Disable Cache in the Network tab when using Chrome Dev Tools (must be the same with Firefox)

When cache is active, the image is cached correctly (response 304 Not Modified).

When revalidating to know if the cache is ok, it checks the Last-Modified with an If-modified-since header. If the resource hasn't changed, it will use the cached version. You can also see the x-vercel-cache: HIT header that shows the Vercel cache was successful.

Given the cache mechanism used here, there will still be a network call to validate the image hasn't changed, but it's a few bytes instead of the normal KB.

Verdict => It works as expected

Branches deployed on commits

I would need to check, but I think the cache is always disabled in this case. Which is kind of expected I guess, you're not supposed to run much on these anyway.

[joshmanders]

I'm getting forced loadings every page load on Safari, Brave, Chrome and FireFox, multiple people have reported similar to me also.

Caching is not working.

[joshmanders]

So I was able to figure out that my problem was that .next/cache folder wasn't writeable by the user on the container, so fixing that now caches the image locally in next, but now the first load for an optimized 91.9 kB image that was originally 500+ kB is aCTUALLY slower than just using <img /> directly and letting the browser cache it.

[starburst997]

Not sure if this should be a separate issue but since I've seen it mentioned here I'll comment here.

Basically I'm getting cache-control: public, max-age=0, must-revalidate no matter what, which is curious since it's seems like the minimum is always at least 60. I tried debugging a bit and getMaxAge does return the correct value from my hosted image (which is 31536000). However the resulting header is still max-age=0 (and with must-revalidate that seems like a "dev" header that shouldn't be on a prod server right?).

This is on a private hosted server (not vercel) in prod (not dev). All other routes / pages return the correct header. Only /_next/image return max-age=0. I fixed it on my end by writing a custom rule in my nginx config.

If max-age=0 is indeed the correct behavior might I ask why? For me on ios safari this results with "flashes" since the image isn't grab from cache immediately. I guess when this is hosted on vercel it is a little bit different since there is additional headers sent that might counter this but on a private server this results with poor behaviors.

On vercel (taken from this, I haven't tried hosting my site on vercel yet):

access-control-allow-origin: *
age: 3
cache-control: public, max-age=0, must-revalidate
content-disposition: inline; filename="mountains.jpg"
date: Sat, 02 Jan 2021 18:20:08 GMT
last-modified: Sat, 02 Jan 2021 18:20:04 GMT
server: Vercel
strict-transport-security: max-age=63072000
x-vercel-cache: HIT
x-vercel-id: iad1::sg6c5-1609611608267-147be5c7c5c8

(Edit: After some new tests on Vercel, the source's max-age header is properly honored, still not the case on private server tho)

On my server (next 10.0.4):

cache-control: public, max-age=0, must-revalidate
content-type: image/webp
date: Sat, 02 Jan 2021 18:21:00 GMT
etag: kSVa7RXI9R1Tski31528i1nXHYD8emqPVBhY8exwtFE=
server: nginx

Not sure if I'm missing some config on my server that explains the missing headers but it's a pretty barebone default nginx proxy.

This seems like a bug (for private server at least) and being able to specify the cache-control in the next js config would be great, otherwise it's still possible to fix with a custom rule in nginx but the default behavior seems like a poor experience.

[joshmanders]

@starburst997 I think the cache isn't for the header and caching locally but for serving cached versions from .next/cache/images

[starburst997]

@starburst997 I think the cache isn't for the header and caching locally but for serving cached versions from .next/cache/images

I see! That would explains it then, still you would think that if they check the Cache-Control header from the source it would be passed down.

Here's an example of the "flashing" that occurs because the Cache-Control isn't set properly. I can only see it on Safari iOS, it seems fine on desktop.

https://mot9t.sse.codesandbox.io/

The first image sends

cache-control: public, max-age=31536000, immutable

The second is hosted on vercel and use the image optimizer

cache-control: public, max-age=0, must-revalidate

(Edit: After some new tests on Vercel, the source's max-age header is properly honored, still not the case on private server tho)

Here's a gif

RPReplay_Final1609621051.mp4

Notice how the second image hosted on vercel with max-age=0 flickers.

It is particularly annoying when going back / forward in the history or with an infinite scroller

Being able to at least override this Cache-Control would be great to prevent this behavior on ios

[joshmanders]

@starburst997 I agree, there's something wonky about <Image /> on non-vercel hosted apps. I'm probably gonna revert back to standard <img /> until it's resolved.

[TheThirdRace]

@starburst997 Just to be sure, is your example has a normal <img> + 1 <NextImage> or is it both <NextImage>?

I ask because <NextImage> is a fully responsive and lazy loaded image.

If your page is static or server rendered, the first paint doesn't include the image because <NextImage> uses lazy loading and it's only when rehydrating takes place that the IntersectionObserver realize the image is within the viewport and retrieve the cached image. This would explain the flash.

Have you tried the eager prop to load the image immediately instead of lazy-loaded?

Have you tried the priority prop to preload the image as fast as possible?

<NextImage> is FAR from perfect, there are plenty of issues with styling them right now and it's entirely possible it only supports Vercel hosting and the 3 official providers: cloudinary, imgix, akamai

[yayvery]

@TheThirdRace I can confirm that neither eager nor priority do not solve this issue. The problem is the max-age=0, must-revalidate header. The flash happens even if you open the direct link to the image.

[starburst997]

@TheThirdRace I posted the code source of the example and I though it was clear but I will try to re-explain because I think this is an important issue that needs to fixed:

Image 1 is loaded and has a response header with cache-control: public, max-age=31536000, immutable, image 2 is loaded from Vercel and use the image optimizer (/_next/image?url=...) but the response header has cache-control: public, max-age=0, must-revalidate (this is how the image optimizer and vercel works currently, we can't change that).

(Edit: After some new tests on Vercel, the source's max-age header is properly honored, still not the case on private server tho)

On Safari iOS, you can see how Vercel's image flicker but not the first one.

This has nothing to do with hydrating / lazy loading or the likes.

Edit: Yes in the example I use <img> for both, the issue is not with the next image component itself but the response headers from the image optimizer.

[TheThirdRace]

@starburst997 Unfortunately, I never found the source code, all I saw was the link to the codesandbox running (https://mot9t.sse.codesandbox.io/), but no code was available...

Given the "new" information from you and @danbeneventano, I can only stand with you guys.

What happens if you configure the must-revalidate on the first image (not on vercel)? Do you get the same behavior as with the image hosted on Vercel?

If you do, then <NextImage> might not be the component you need. NextJs tightly integrated their backend image optimization with the <NextImage> component. They force must-revalidate in cache-control header to allow downloading the new images when using layout='response' or layout='intrinsic'. Without must-revalidate, whenever you resize your browser window, you would get the old cached image instead of a new image according to the sizes prop.

Personally, I think NextJs went too far with their <NextImage> component:

• They forced their cache control header instead of having a default that you can override
• The DOM is also a mess, you shouldn't need 2 wrapper <div> tags for an <img> tag... especially when both <div> use style directly on the DOM node, which forces you to use !important to override them. The lack of handles on the wrappers also doesn't help...

They gave us a wonderful solution if used exactly the way they intended, but you're gonna be in trouble if you want to change anything in the tightly integrated process. Which is a big letdown in my opinion... Given there's some time before I release what I'm building with NextJs, I'll wait and see what happens, but I'm also thinking of simply switching back to a plain old <img> tag or use another package if no improvement is made on the numerous issues <NextImage> has.

[starburst997]

@TheThirdRace I should've posted the link to the editor on the first post but you can get the editor link easily in any codesandbox (https://codesandbox.io/s/editor-XXXXX, replace with "mot9t"), my bad.

" might not be the component you need", things is, it is exactly the component I was looking for, it works perfectly in my use-case, I just want a way to be able to set the cache-control header.

I must admit I didn't look too far into it, but why would the image component absolutely need the must-revalidate to do that? The browser and srcSet takes care of that, I tried it and it works fine even without it. I didn't try the responsive one but I don't have any need for it.

[TheThirdRace]

I must admit I didn't look too far into it, but why would the image component absolutely need the must-revalidate to do that? The browser and srcSet takes care of that, I tried it and it works fine even without it. I didn't try the responsive one but I don't have any need for it

You mean you put srcSet directly as a prop?

Because if I read the documentation, there's no mention of that prop except that it's supposedly not passed to the image... https://nextjs.org/docs/api-reference/next/image#other-props

I must rectify my statement, it's only layout='responsive' that seem to download new copies while resizing your browser window. If you got images that span large area, you really want to use responsive; it allows to download the smallest acceptable image for the viewport, which can mean a lot of KB savings. You could set your own srcset I guess, but then you have to build it yourself...

I've tested the default layout='intrinsic' and to me it looks completely broken. I got a resized window at 500px and the component still fetch the 960px image even though the sizes props contains many breakpoints between 500px and 960px. The generated srcset simply has the 1x, 2x and 3x versions... Fun times!!!

I'm also banging my head on the wall with the CSS part... Images don't have sizes when using layout='responsive', they're kinda floating (using position: absolute). Ever tried to use overflow: hidden on a container with an image and multiple other stuff? The image doesn't resize, it simply squish itself... Fun times again!!!

I'm currently fighting a lot with the NextImage component, getting pretty tired of it actually. :(

[TheThirdRace]

@stefanprobst
Actually, even the import optimization method is not really done at build time.

At build time, you can check the static images produced in your .next/static/image/public folder and you're gonna see there's only 1 version for each image. Furthermore, it's the original unoptimized image.

The image optimization really happens at run time:

• NextJs image optimization service receive a request for an image
• It checks that static folder to see if the required optimized image is already there
• If that's the case, then serve that image with cache-control = public, max-age: 1year, immutable
• If the image is missing, e.g. you needed a webp and it ain't produced yet:
  • NextJs will optimize the image for the required format and size
  • It will write that image to the static folder
  • It will hash the content of that optimized image and use that hash to rename the image appropriately
  • It will serve the image with cache-control = public, max-age: 1year, immutable

For truly static image optimization, NextJs would have to offer a configuration in next.config.js so we can specify the formats and sizes we want. Then at build time, it would to generate all the optimized image in the appropriate folder.

There are downsides to truly static image optimization, such as:

• Longer build times, especially when you have a lot of images
• The filename is static, so if you make a mistake...
  • you'll have to rename the original image to bust the cache
  • you'll need to change your code or image url to point to the new one

The current import method don't have these problems because of the hashing feature, which is only possible because it happens at run time.

So long story short, no there isn't any build time image optimization for markdown content. In fact there isn't any build time image optimization at all.

[Amir-Mirkamali]

@TheThirdRace Thank for reply and thank for precise detail about next.js image optimization system 🙏
But i think you missed the main point that I have mentioned,
Please please please do not modify the cache when it set before!

[TheThirdRace]

@Amir-Mirkamali
I 100% agree with you, NextJs shouldn't modify the cache-control header if you set it in next.config.js.

While really unintuitive, there is a way to set the cache-control header on your images and NextJs won't change it.

All you need to do is to configure your hosting service to serve images with the cache-control you want. NextJs will then act as a passthrough and let your cache-control alone.

For hosting services like AWS, I think you can set the cache-control on a folder directly. It takes some time for the settings to propagate, but it does work.

[MaximeBernard]

Thanks for this explanation @TheThirdRace 👍

• you'll have to rename the original image to bust the cache

Wouldn't etag be enough?

[TheThirdRace]

@MaximeBernard
My comment was made in the context of a truly static image, meaning your image has a cache-control of public, max-age: 31536000, immutable.

To be able to use an etag, the browser will need to make a request to the server, which is exactly what we want to avoid by using public, max-age: 31536000, immutable.

But because your image is cached up to a year, then you need to change the filename if you want the clients to update to the new image instantly otherwise the browser will simply serve the image in cache.

[Amir-Mirkamali]

Still, I think that etag and current cache-control system is useless for us.
It’s good for whom use CDN! Unfortunately, I am not using CDN and I have a media server myself.
This system not gonna work for me and not customizable, Its s very simple: Let me decide myself!

I want to serve my image with one week cache and I want to modify that cache in next.config.js file but it’s not possible now.
Completely disappointed from Next.js Image component so I bypassed the image loader.
I wish that someone hear our voice and correct this part of code.

[styfle]

Take a look at PR #26739 which will preserve the upstream max-age so the browser can utilize it too.

This will support the first use case (source: '/(.*).jpg') when you want internal jpg images to have a different max-age than other images.

However, we won't be supporting the use case for source: '/_next/image(.*)'. This could lead to de-optimization because it would override all images including static imported images (should remain immutable) and external images (should rely on upstream server cache-control header).

Furthermore, we plan to build stale-while-revalidate behavior into the image optimization server to avoid making any client wait for the server to re-optimize an image again, so this header will not be required.

[Faey2222]

@styfle So how can I prevent my application to always use the file on the server if all my images come from an headless system (directus9)? The headers sent by directus are ignored when I use the next/image component. Therefore when a user revisits my website the load time for images is always unacceptable due to that fact that the images are always reloaded from the server.

Maybe I just missed something?

[TheThirdRace]

@styfle

About the PR

Just to clarify, could you confirm the PR will have these effects:

1. We can now set the cache-control header in next.config.js
2. We don't need a vercel.json anymore for "local" images as it's taken care of by next.config.js now
3. External images will still need to set their own cache-control header since next.config.js only applies to "local" images
4. By "local" images, you're referring to images hosted in our own NextJs repository, i.e. our public folder

Etags

Before this PR, the etag would be omitted from the response headers if:

1. I used a vercel.json to set the cache-control header
2. I used static images (import method)

This behavior isn't ideal though as it prevents some fallback situations.

Example

1. Images are cached for 30 days
2. On day 31, the client's browser will do a network request to retrieve the image (or revalidate the cache)
3. Without etag, the browser will check last-modified and expire-at headers and determine the image is expired, which in turn force the browser to re-download the image even if it hasn't changed
4. Any subsequent request to that image will force the browser to re-download the image as it's "expired" according to the server

With an etag, the server would have returned a 304 Not Modified, which is only a few bytes in comparison.

Even with static images, which use immutable, not all browsers support that value. If by a low chance a client keeps an image in its cache for 1 year, then he would redownload the image on every single request past that. All because there's no etag to compare and the last-modified / expire-at headers combination would compute to be expired.

My point is, there's only benefits to include the etag in the response header.

Cloudinary behavior

I want to point out that Cloudinary will also one-up this etag situation by updating the last-modified or expire-at on the original image once it actually becomes expired.

This has the benefit to reset the last-modified / expire-at combination so any browser that would just want to revalidate its cache could know the cache can be renewed and receive a 304 Not Modified.

If a browser doesn't support etag, the cache is still working.

If a browser does support etag, the request to revalidate the etag will only be made once every 30 days (from the example) before renewing the cache. Thus, avoiding revalidating the cache multiple times within the month, no more unneeded network requests.

[styfle]

The fix is available in [email protected] 🎉

Please give it a try and let us know if it solves your problem with a 👍 or 👎

If you leave a 👎 , please explain why it doesn't work for you and provide steps to reproduce the issue. Thanks!

[styfle]

Looks like we have 4 thumbs up and 0 thumbs down on the latest canary release after 2 weeks so I'm going to close this issue.

See my previous comment for more details.

Also, I created #27208 to track the the stale-while-revalidate feature request. Thanks!

[balazsorban44]

This issue has been automatically locked due to no recent activity. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.