Add support for AVIF

[joehoyle]

Sharp should soon be releasing AVIF support. This PR uses the alpha-3 version of libvips and upcoming version of Sharp to add support for avif images.

This should work the same as webp: there is an X-Avif header or avif=1 param, and the detection is expected to be handled by a lamnda function the same as WebP (check for accept:image/avif header).

The results in test are good and bad:

┌───────────────┬───────────────┬────────────────────┬──────────┬──────────┬──────────┬───────────────┬───────────────┐
│ Image         │ Original Size │ Tachyon Size       │ 100px    │ 300px    │ 700px    │ 700px webp    │ 700px avif    │
├───────────────┼───────────────┼────────────────────┼──────────┼──────────┼──────────┼───────────────┼───────────────┤
│ briefing-cop… │ 442.52 KB     │ 112.67 KB (25%)    │ 3.01 KB  │ 9.34 KB  │ 28.51 KB │ 15.31 KB (53… │ 14.57 KB (51… │
│ hdr.jpg       │ 222.58 KB     │ 145.56 KB (65%)    │ 10.4 KB  │ 23.53 KB │ 85.59 KB │ 80.86 KB (94… │ 111.09 KB (1… │
│ icons.png     │ 94.24 KB      │ 27.37 KB (29%)     │ 3.89 KB  │ 11.31 KB │ 26.94 KB │ 23.93 KB (88… │ 13.25 KB (49… │
│ Website.png   │ 32.38 KB      │ 33.78 KB (104%)    │ 3.26 KB  │ 13.27 KB │ 33.78 KB │ 19.81 KB (58… │ 22.21 KB (65… │
│ humans.png    │ 2.22 MB       │ 853.21 KB (37%)    │ 8.96 KB  │ 54.66 KB │ 273.89 … │ 139.19 KB (5… │ 174.48 KB (6… │
└───────────────┴───────────────┴────────────────────┴──────────┴──────────┴──────────┴───────────────┴───────────────┘

In some cases it's less than webp, and in some cases more. I think it might be to do with these comparisons being quality:100, where we might do better at slightly dropping quality

[roborourke]

Oh nice. Have the security issues upstream in libvips been fixed?

[joehoyle]

Apparently so! Release imminent

…

On Dec 19, 2020, at 15:05, roborourke ***@***.***> wrote:  Oh nice. Have the security issues upstream in libvips been fixed? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[joehoyle]

I think we might want to default webp over avif still, I included a script to compare real-world data in webp images, creating AVIF versions of the same, and comparing on average.

I compared 1000 popular image requests to Tachyon, which resulting in the following:

Based off request counts, WebP total data: 2.15GB vs Avif: 2.38GB
Avif files 11% larger in total
225 of 1000 avif files were smaller than webp.
86s in webp generation, 1782s in avif (avif 20x slower) (on fastest compression option).

[joehoyle]

FYI based off https://www.industrialempathy.com/posts/avif-webp-quality-settings/, I applied the quality normalization, as that was responsible for making AVIF looking perticuliry bad, once I did that, I'm now seeing much better file sizes, even with switching to speed: 9:

Based off request counts, WebP total data: 0.44GB vs Avif: 0.40GB
Avif files 9% smaller in total
854 of 996 avif files were smaller than webp.
81885ms in webp, 247999ms in avif.

[joehoyle]

Also with new test-filesize:

┌───────────────┬───────────────┬────────────────────┬──────────┬──────────┬──────────┬───────────────┬───────────────┐
│ Image         │ Original Size │ Tachyon Size       │ 100px    │ 300px    │ 700px    │ 700px webp    │ 700px avif    │
├───────────────┼───────────────┼────────────────────┼──────────┼──────────┼──────────┼───────────────┼───────────────┤
│ hdr.jpg       │ 223 KB        │ 146 KB (65%)       │ 10 KB    │ 23 KB    │ 86 KB    │ 81 KB (94%)   │ 90 KB (104%)  │
│ briefing-cop… │ 443 KB        │ 113 KB (25%)       │ 3 KB     │ 9 KB     │ 28 KB    │ 15 KB (53%)   │ 12 KB (40%)   │
│ icons.png     │ 94 KB         │ 27 KB (28%)        │ 4 KB     │ 12 KB    │ 27 KB    │ 24 KB (89%)   │ 12 KB (45%)   │
│ Website.png   │ 32 KB         │ 34 KB (104%)       │ 3 KB     │ 13 KB    │ 34 KB    │ 20 KB (58%)   │ 16 KB (46%)   │
│ humans.png    │ 2 MB          │ 857 KB (37%)       │ 7 KB     │ 53 KB    │ 274 KB   │ 139 KB (50%)  │ 135 KB (49%)  │
└───────────────┴───────────────┴────────────────────┴──────────┴──────────┴──────────┴───────────────┴───────────────┘

[johnbillion]

@joehoyle Is there a plan to continue this implementation?

[joehoyle]

@johnbillion it's not scheduled, but yes given an infinite time scale, we'll want to do this. The pressure to do it reduced somewhat from the findings though, as it's not a lot smaller than webp, and now iOS support webp too then the coverage and gains from webp are still substantial. I think it would still be good to add though as per the above 9% improvement in file sizes. The other side to that is avif is about 3x slower to generate. That might have changed if the avif encoder has been more optimized since last year though.

[johnbillion]

Sharp is at version 0.30.7 now (0.29.3 in this PR). Could be worth upgrading.

[joehoyle]

Cool, I think we could pick this up after #162 is merged. We could still add AVIF as opt-in.

[joehoyle]

I cleaned this up and atleast we have it ready to do with opt-in. @roborourke could I get a final review and then I can merge. Then I want to add jpegxl once this is in.