RFC: Image Optimization

[timneutkens]

Goals

• Integrate with the Image Component by default: RFC: Image Component for Next.js #16832
• Automatically serve multiple modern formats
  • webp
  • avif
  • Fallback optimize the original format
• Allow for resizing images / srcset (integrates with RFC: Image Component for Next.js #16832)
• Does not significantly slow down npm install next time

Non-Goals

• Provide every possible transformation — The goal of this RFC is to provide the minimal transformations needed
• Support for next export — The goals of the RFC and it's particular characteristics of on-demand optimizing over optimizing at build time which causes build time slowdowns

Background

#16832 is introducing the Image component that allows for a bunch of optimizations to be handled like lazy-loading and srcset generation. In order to fully leverage all optimizations you'd need to use specific image hosting services, that's not as ideal as having this built-in to Next.js itself so this proposal aims to cover building in a solution for optimizing images.

Proposal

The image optimization will seamlessly integrate with next/image (#16832). In order to do so, Next.js will expose an endpoint similar to what image optimization services have.

This endpoint will be under /_next/image and take a couple query parameters:

• url the url of the image, this could be an absolute url or relative path
• w the width of the image
• q the output quality of the image

The endpoint will automatically serve the best format (e.g. webp or avif) based on if the browser supports it through the Accept header. E.g. Google Chrome provides the following values:

accept: image/avif,image/webp,image/apng,image/*,*/*;q=0.8

Example: /_next/image?url=/logo.png&w=100&h=100&q=70 Will serve the /logo.png as 100x100 with 70 quality.

Note that you'd generally not construct the url in this way as you'd be using the next/image component shown in #16832

To avoid abuse the user has to provide a list of supported image widths to avoid enumeration attacks:

// next.config.js
module.exports = {
  images: {
    // This setting would be defaulted to commonly known device sizes to ensure the srcset can be generated for next/image. It can optionally be overwritten to a custom setting.
    // sizes: [1024, 2000]
  }
}

Support for external images

Images on a web page are generally a blend of local images as part of the project and external images that come from an external data source. The image optimization will support both local images and external images.

In case of external images you'll need to add the domains (or domain pattern) to an allow list. This is to ensure that malicious users can't hijack the image optimization for their own use:

// next.config.js
module.exports = {
  images: {
    domains: ['cms.example.com', 'example.com']
  }
}

Install Time

The npm install time of Next.js is important as every new application being created, every CI build and development install would be slowed down by installing large dependencies. Image optimization libraries are notoriously large and depend on native dependencies. For example sharp which is being used in some frameworks has an install size of 28MB (source). For comparison the next package right now has an install size of 50MB of which the majority is webpack (source).

When looking at https://squoosh.app/ we (@Timer and me) noticed that it ships WebAssembly files for the image optimization. Furthermore these files are really small (a couple hundred kB) compared to the existing image optimization libraries on npm and do not have native dependencies. Seemingly these files are a great candidate to solve the install time issue. They're located here: https://github.com/GoogleChromeLabs/squoosh/tree/dev/codecs

Upon further investigation we found that @cyrilwanner has ported these codes to npm libraries in this repository: https://github.com/cyrilwanner/wasm-codecs potentially we can use/reuse parts of this. The ideal outcome would probably be having these files published to npm from the squoosh repository so they're always up to date.

Upon even further investigation with help from @cyrilwanner who did a performance comparison of the webassembly version against sharp. The Sharp version is significantly faster (see results linked below)

Cyril's comparison of image solutions

Performance comparison

​

Hardware

​
Tests run on a MacBook Pro 2018, with an 2.6 GHz 6-Core Intel Core i7 and 16 GB 2400 MHz DDR4 RAM.
​

JPEG

​

Tiny image (6.3KB)

​

Package	Images per second	Resulting file size
@wasm-codecs/mozjpeg	17.4 per sec	1.8KB
imagemin-mozjpeg	83.3 per sec	5.1KB
sharp (default)	95.7 per sec	2.3KB
sharp (more optimizations)	83.6 per sec	2.1KB
​

Small image (166.0KB)

​

Package	Images per second	Resulting file size
@wasm-codecs/mozjpeg	4.9 per sec	94.4KB
imagemin-mozjpeg	8.9 per sec	104.0KB
sharp (default)	39.3 per sec	137.3KB
sharp (more optimizations)	33.2 per sec	104.6KB
​

Medium image (603.8KB)

​

Package	Images per second	Resulting file size
@wasm-codecs/mozjpeg	4.9 per sec	349.8KB
imagemin-mozjpeg	8.9 per sec	374.6KB
sharp (default)	14.0 per sec	500.0KB
sharp (more optimizations)	10.8 per sec	377.8KB
​

Large image (2211.2KB)

​

Package	Images per second	Resulting file size
@wasm-codecs/mozjpeg	0.41 per sec	1205.7KB
imagemin-mozjpeg	0.68 per sec	1363.1KB
sharp (default)	4.41 per sec	1839.0KB
sharp (more optimizations)	3.24 per sec	1381.0KB
​

PNG

​

Tiny image (17.0KB)

​

Package	Images per second	Resulting file size
@wasm-codecs/oxipng	22.3 per sec	14.0KB
imagemin-optipng	6.3 per sec	14.1KB
sharp (default)	68.5 per sec	15.0KB
​

Small image (139.4KB)

​

Package	Images per second	Resulting file size
@wasm-codecs/oxipng	1.57 per sec	108.0KB
imagemin-optipng	1.15 per sec	112.6KB
sharp (default)	31.74 per sec	170.4KB
​

Medium image (538.7KB)

​

Package	Images per second	Resulting file size
@wasm-codecs/oxipng	0.20 per sec	472.5KB
imagemin-optipng	0.14 per sec	517.4KB
sharp (default)	6.63 per sec	557.7KB
​

Large image (1949.6KB)

​

Package	Images per second	Resulting file size
@wasm-codecs/oxipng	0.12 per sec	1497.8KB
imagemin-optipng	0.11 per sec	1519.7KB
sharp (default)	2.23 per sec	1836.0KB

Currently the install size of sharp is mostly affected by libvips (around 20MB) coming from the libvips binary. libvips recently announced a webassembly version that is around 5MB to install, however it has relatively similar results to what @cyril Wanner showed. It also currently depends on experimental Node.js / v8 features making it not feasible for adoption at this point in time.

Overall having looked into all these options it seems that eventually using libvips with webassembly will be the way to go in the future (whether or not it will be an optional dependency will be decided then).

However, for the time being we can make sharp an optional dependency if users want to leverage the image optimization and we'll make it work automatically after sharp is installed. This is very similar to how we tell people to install typescript which is 54MB (source).

[chandan-reddy-k]

Finally 🚀.

Can’t wait to get that 100 lighthouse performance scores once again 😋

[todortotev]

yay

[flybayer]

Wow, this is fantastic!!!

[paambaati]

Would it also allow overriding the image format with a query parameter? There will be scenarios where I’d want to explicitly specify a format — for example, using PNGs when the image has more lines and using JPGs when the image is more photo-realistic. Of course AVIF does both of these kinds really well, but while we wait for more widespread browser support, the ability to specify the image format would be nice.

[neefrehman]

Provide every possible transformation — The goal of this RFC is to provide the minimal transformations needed

I assume ^this non-goal also covers generating image placeholders? (blurred images, used colours, etc.)

#16832 also doesn't include the use of placeholders so it looks like it isn't a goal for either RFC, but they are one of the best features of next-optimized-images and gastby-image from a UX perspective (I did see the comment about perf issues in #16832, but haven't seen any benchmarks and would like to understand that point more)

[lorenries]

Is there any consideration for signing image URLs like Thumbor to prevent URL tampering? Even though you can configure the allowed image domains, this proposal would leave the image endpoint open to denial of service attacks.

From the Thumbor documentation:

Now let’s say that some malicious user wants to overload your service. [They] can easily ask for other sizes in loops or worse, like:

http://some.server.com/unsafe/300x301/smart/path/to/image.jpg
http://some.server.com/unsafe/300x302/smart/path/to/image.jpg
http://some.server.com/unsafe/300x303/smart/path/to/image.jpg

Other than that, the user can ask for images that do not exist, thus forcing us to perform useless http GET operations or filesystem operations.

[balazsorban44]

Does this mean that for the first time a request is made an image is generated, and then every subsequent requests will serve the created image? Is it basically a smart image cdn? I guess it will be cleaned between builds though?

[mi-na-bot]

At least for the serverless use case, there is not necessarily a permanent filesystem, so resized versions might only be cached on the CDN edge.

[baboons]

I'd like to propose a few things to consider adding into this RFC which would be very useful.

• Option to use signed urls for external images instead of white list domains. It would also protect the arguments for hijacking the sizes, quality and so on.
• Possibility to select crop method and positioning (center, top, left, right, bottom, etc)

We wrote a similar solution using vips/sharp (not maintained at the moment) to be able to replace our SaaS for image optimization and manipulation, it explains a bit more what I ment by signed urls and more params to control the image - in case that would be helpful :-)

https://github.com/barnebys/bimp

[blainegarrett]

I would be interested in an x,y argument for cropping.

[beaussan]

How would this work for generated static sites?

[dazuaz]

Non-Goals

• ...
• Support for next export — The goals of the RFC and it's particular characteristics of on-demand optimizing over optimizing at build time which causes build time slowdowns

I think optimizing at build time its valid in a lot of use cases and it does not require to have serverless support to handle the requests, I do agree that it should have some sort way to only transform images once and skip for subsequent builds.

Maybe add a new RFC for this? It is one of the most used features in some frameworks.

[mi-na-bot]

This would be similar to Gatsby, which smashes resized images into the bundle and relies heavily on build caching. I always found it to be a cool feature on paper, but often unwieldy in practice.

[frederickfogerty]

@Sever1an can you elaborate a bit why it's not useful in practice? I'm working with Gatsby right now and would love to hear your thoughts 🙌

[mi-na-bot]

@frederickfogerty It is useful, just has some limits in practice. It has been a while since I last used Gatsby, some of this might be better now. Just brainstorming some frustrations:

• One of the biggest issues for me was building time, particularly when it got into the realm of tens of minutes. The image resizing step was an extra kind of punishment for running gatsby clean in a lot of cases for me.

• While offering support to other developers I found that the interplay between gatsby-img and the graphql requests for image sizing was a frequent source of confusion.

• There was no process for expiring sized images in the build cache, so our CI cache just kept getting larger.

• Offering fixed and fluid mode in <Img> forces developers to understand browser responsive image technology, which they are often desperately trying to avoid.

• More generally on build time, not totally related to image sizing, but while I am ranting. The paradigm of making a query to build lots page data in gatsby-node at once instead of page queries resulted in many saves in preview mode taking an extremely long time re-building unchanged data for many unrelated pages.

[frederickfogerty]

Thanks for the info @Sever1an! It's good to know a bit more about the pain points of gatsby-image (and gatsby-transformer-sharp)

[simonireilly]

This is a great RFC, I am doing so already with Sharp, Vercel and NextJS. It works well, and Vercel gives good Cache Control.

1. Are there any considerations for the user controlling the http cache headers for the images returned by _next/image?

Some truly unique remote URLs will probably not change and can be cached after they are resized. Others like /_next/image/?logo.webp may need to be expired.

Deploys cannot be relied upon to clear the cache either as this is online image processing.

2. Some S3 best practices are no public access by default and for CORS policies for websites fetching assets from S3 or through Cloudfront CDN.

Since this is server side fetching credentials would be required for the server to access non public assets. This prevents DDoS and therefore requires no next support for masked URLs.

This may mean a user must supply a function that can fetch the image, it is not available over HTTP, or is only available securely over HTTP with environment variables. Would this be supported?

Thanks 👍

[smhmd]

What would be the HTML output of such component and how would responsive imagery be achieved? Sometimes you want to serve images of with 500 width on medium screens, but only 300 on larger screens because of tiling or something. This is of course achieved through the sizes attribute on the img tag. I very much like the idea of query parameters. You can add a sizes s query that adds media queries.

[Tom910]

What about DDOS? For example somebody makes thousands request with random query on _next/image?url=/logo.png&w=100&h=100&q=70

[mi-na-bot]

With a source image that is 2000x2000 px, there are 4M possible requests that can filter through the cache to the origin, combined with say 50 reasonable quality settings you might have 200M. If each image resizing job takes 500ms on Lambda, an attacker could wrack up an AWS bill on the order of thousands of dollars.

It may not be more viable than other DoS attacks, but it is worth consideration.

[mi-na-bot]

It would be trivial to blow past the fair-use execution limits on Vercel as well without some mitigation.

[crash7]

Indeed - that's one of the reasons why I think this sort of feature shouldn't be inside Next.js. It brings more cases at the infra level and not entirely related to the framework. It's like wanting to hit everything nail-like-shape with a hammer.

Unless.... on build time next creates a list of the valid urls that the image service should process, like a sort of whitelist.

[Tom910]

Unless.... on build time next creates a list of the valid urls that the image service should process, like a sort of whitelist.

Yes, or alternative solution presets (like https://docs.imgproxy.net/#/presets ). I integrated at my company a similar service and this case created for me a problem. I chose presets

quality settings you might have 200M.

Or more - 2000(width) * 2000(height) * 100(quality) * 3(types) = 1 200 000 000

[mi-na-bot]

It seems unlikely that anyone actually would want this service to distort the images by stretching or adding blank bars. Limiting the specification to a single dimension constraint like max-width or max-side (with the other inferred by native image aspect ratio) would drastically decrease the URL space.

Modal quality settings like { low, medium, high, full } would help a lot too.

2,000(max-width) * 4(quality) * 2 [webp, jpeg] = 16,000 (maybe a few dollars on Lambda)

[huv1k]

I wonder if image optimization is going to support rewrites? This could be helpful for adaptation and incrementally adopting Next.js.

[ayrtongomes]

The dream come true in v10.
I just came here to say thank you for being the best react framework ever ❤️

[bits-by-brandon]

Would love to see alternative storage options. Right now I'm assuming images get stored on disk, but with a large enough quantity of images we might eat up all the available storage on whatever server we are using. It also means we will be discarding these image caches if using Blue/Green deployments.

If we could select an S3 bucket, for example, to store images we could mitigate this storage problem

[AmazingDevTeam]

Is Fallback optimize image to original format available at the moment?