Image Format Issue Tracker.

[etemesi254]

Image formats tracker.

This issue tracks currently supported images and gives details on their status and tracking issues.

Formats currently able to decode

• Decoder
  • JPG
  • PPM
  • PSD - (Experimental)
  • QOI
  • Farbfeld

Priority of decoders follows the following criteria.

First class image formats.

Images present everywhere and used for everything, and by first class support, we want to have excellent decoding +encoding of almost all images for that format.
Support for such takes priority.

1. JPEG

Full support is about to be complete, what is left is some polishing and some decoder fixes,
works on a lot of images but some cause errors, looking into them.

2. PNG

Not yet working, but work is underway, mainly spending time translating Eric Bigger's libdeflate to Rust, but support should be soon.

3. GIF.

Limited by the fact that I haven't decided how frame API will look, so this will take a while

Second class image formats.

Image formats on the rise

4. WEBP + AVIF + JPEG-XL+HEIF

Complicated and complex beasts, support for these formats will take a while, probably a year or so , decoding support should be pure rust(there is no hurry to reach production at all), while encoding should be their C libraries (encoding is another complicated beast and man has 24 hours a day).

Other file formats.

Support is welcome.

[etemesi254]

cc @Shnatsel , PSD decoding landed(limited in scope, not fully working), thought you might be interested.

[Shnatsel]

Is there a way to toggle which decoders are compiled, or some way to toggle them at runtime?

In many cases it would be nice to disable PSD support either to reduce attack surface (e.g. an image format converter accepting untrusted input) or just not to advertise support for a format that is not complete (e.g. in a desktop image viewer). PSD is an immensely complex format, and full support for it is a multi-year effort.

[Shnatsel]

PNG also has an animated variant, APNG, which is now supported by all web browsers as well as the Rust png and image crates. It would be nice to support that as well eventually.

[etemesi254]

Yea that's on my radar Webp + avif also have animation. Will be interesting to see how the final api version looks

…

On Sun, 27 Nov 2022, 19:01 Shnatsel, ***@***.***> wrote: PNG also has an animated variant, APNG <https://en.wikipedia.org/wiki/APNG>, which is now supported by all web browsers <https://caniuse.com/apng> as well as the Rust png and image crates. It would be nice to support that as well eventually. — Reply to this email directly, view it on GitHub <#9 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFZRVE4I6MH3MJHE4KGLCULWKOAWBANCNFSM6AAAAAASMSO2QE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[etemesi254]

Is there a way to toggle which decoders are compiled, or some way to toggle them at runtime?

In many cases it would be nice to disable PSD support either to reduce attack surface (e.g. an image format converter accepting untrusted input) or just not to advertise support for a format that is not complete (e.g. in a desktop image viewer). PSD is an immensely complex format, and full support for it is a multi-year effort.

zune-image supports ability to disable , I'm yet to add that to the binary.

For PSD complexity you make an interesing case,

Just found out ffmpeg supports PSD files, i.e https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/psd.c that's their implementation.
ImageMagick's one is longer since it supports more features i.e

https://github.com/ImageMagick/ImageMagick/blob/31176f90be5bb143302d2a7a2746a601dc9e2ac0/coders/psd.c

The I was using stb as a basis for my PSD implementation. about 246 LOC at https://github.com/nothings/stb/blob/8b5f1f37b5b75829fc72d38e7b5d4bcbf8a26d55/stb_image.h#L6002-L6248.

Currently we throw out a lot of data we don't need, i.e layers and such to just get the image, the same with ffmpeg and stb. Imagemagick does not do this, and the features it implements aren't

I'd rather not disable it, for the command line and add it under a feature , say --unsafe is needed to decode PSD files since it's experimental at best than trying to chase perfection since that will mean this library will never be ready for any real world decoding.

[Shnatsel]

Is the BMP format in scope? It should be relatively trivial, but I don't see it mentioned explicitly.

[etemesi254]

Hi, BMP support is planned.

The plan is to support at least anything image-rs supports first.(Like farbfeld decode support landed but can't currently be encoded, working on that one over the weekend).

This issue should probably be restructured in the way it is(I.e tracking image status and planned times )

[etemesi254]

Is there a way to toggle which decoders are compiled, or some way to toggle them at runtime?

In many cases it would be nice to disable PSD support either to reduce attack surface (e.g. an image format converter accepting untrusted input) or just not to advertise support for a format that is not complete (e.g. in a desktop image viewer). PSD is an immensely complex format, and full support for it is a multi-year effort.

Added is_experimental method to DecoderTrait, (DecoderTrait allows unanimous image decoding from multiple formats into a single image, i.e. implemented for all image decoders).

And added an --experimental flag in the command line, this adds a guard on all decoders which are experimental(currently farbfeld and psd) to require --experimental flag in the command line before decoding.

[etemesi254]

@Shnatsel , Finished a simple PNG decoder.

Doesn't do a lot yet, only works with 8 bpp pixel images but I think you'll love the speeds at
https://github.com/etemesi254/zune-image/blob/main/zune-png/Benches.md

Uses a custom zlib decoder at zune-inflate which is as fast as zlib-ng (for the tests I've done, which is extracting delfate chunks from a png image of your AI upscaled image from before, running decode on it) and about 20Mb/s faster than miniz-oxide(using the same procedure).

The speedups actually don't even come from that.
They come from a carefully written filter algorithm which the compiler can optimize, and that's pretty sweet(will probably write a blog post about it soon) but for now that's all.

Will work on finishing this and the jpeg decoder for now

[Shnatsel]

That's excellent news, thanks for sharing!

I'm curious, how does zune-inflate compare to libdeflate performance-wise?

Also, I should have mentioned it earlier, but WUFFS has a fast PNG decoder and detailed explanation of why it's fast, and there's also libspng that beats libpng. You might be able to lift some tricks from those and push performance even further.

[etemesi254]

Perfomance wise with libdeflate, zune-inflate is probably a 1 to 1 translation of libdeflate with some small language differences, will put up performance comparisions soon, but in face value I expect libdeflate to be slightly faster because it can take advantage of some C properties like incrementing pointers,unbound checks,some union hackery and some wrapping subtractions on steroids, which I didn't implement as it made some asserts hard to do in some places.

Didn't know about libspng let me check it out.
Also looking at wuffs.

Will cc you when I think it's ready for fuzz-testing

[Shnatsel]

FWIW what I've been doing so far is testing on real-world images instead of fuzz testing. Fuzzing produces random/invalid data to trigger unexpected codepaths, like panics or memory safety bugs.

Fuzzing usually finds a large amount of rather boring panics, so I've put it off until the correctness on real-world images is established.