hello_mmal_jpeg fails with images from USB webcam

[maximumsomething]

Describe the bug
Sometimes, perfectly valid jpeg images fail to decode. They come from a USB webcam.

$ ./hello_mmal_jpeg.bin frame.jpeg
vc.ril.image_decode:out:0(I420)
 type: 3, fourcc: I420
 bitrate: 0, framed: 0
 extra data: 0, (nil)
 width: 736, height: 368, (0,0,724,362)
start decoding
stop decoding

For reference, a typical successful run of the program looks like this:

$ ./hello_mmal_jpeg.bin test-image.jpg 
vc.ril.image_decode:out:0(I420)
 type: 3, fourcc: I420
 bitrate: 0, framed: 0
 extra data: 0, (nil)
 width: 736, height: 368, (0,0,724,362)
start decoding
received event length 96, EFCH
----------Port format changed----------
vc.ril.image_decode:out:0type: video, fourcc: I420
 bitrate: 0, framed: 0
 extra data: 0, (nil)
 width: 736, height: 368, (0,0,724,362)
 pixel aspect ratio: 0/0, frame rate: 0/0
 buffers num: 1(opt 1, min 1), size: 406272(opt 406272, min: 406272), align: 0
-----------------to---------------------
type: video, fourcc: I420
 bitrate: 0, framed: 0
 extra data: 0, 0x173e330
 width: 1248, height: 1104, (0,0,1244,1096)
 pixel aspect ratio: 0/0, frame rate: 0/0
 buffers num (opt 1, min 1), size (opt 2066688, min: 2066688)
----------------------------------------
Retrieved buffer 0x177c1b8
decoded frame (flags 1, size 2066688) count 0
stop decoding

To reproduce

The above image isn't viewable with most programs. It can be fixed with this ffmpeg command, which turns a Motion JPEG frame into a full JFIF image file:
ffmpeg -i frame-og.jpg -c:v copy -bsf:v mjpeg2jpeg frame-fixed.jpg

Both of them fail when inputted to hello_mmal_jpeg.bin. Other images work fine.

System

System Information
------------------

Raspberry Pi 4 Model B Rev 1.1
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"

Raspberry Pi reference 2019-09-26
Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 80d486687ea77d31fc3fc13cf3a2f8b464e129be, stage4

Linux team5708pi 4.19.75-v7l+ #1270 SMP Tue Sep 24 18:51:41 BST 2019 armv7l GNU/Linux
Revision	: b03111
Serial		: 100000000e728cb2
Model		: Raspberry Pi 4 Model B Rev 1.1
Throttled flag  : throttled=0x0
Camera          : supported=0 detected=0

Videocore information
---------------------

Sep 24 2019 17:34:30 
Copyright (c) 2012 Broadcom
version cd3add54955f8fa065b414d8fc07c525e7ddffc8 (clean) (release) (start)

alloc failures:     0
compactions:        0
legacy block fails: 0

[linhuifj]

That's because mjpeg is not well supported by mmal. You can try to convert mjpeg into jpeg format.
try ./ffmpeg -i ~/raw.jpg -codec copy -bsf:v mjpeg2jpeg yy.jpg

You can refer to the code of mjpeg2jpeg to write a converter.

[6by9]

I'd not noticed this issue as it's been raised on my fork of this repo instead of the main raspberrypi/userland.

MJPEG is fine with MMAL, although generally via video_decode rather than image_decode.

MJPEG is very poorly defined (there is no official spec).

There was a fix for decoding some MJPEG files in June 2020 under RPi-Distro/vlc#8 to deal with multiple padding bytes between markers. That may fix your issue.

Closing. Please retest with latest firmware and open at raspberrypi/userland if you still see problems.

[linhuifj]

In fact, resetting to the latest firmware will not resolve the problem. I have been struggling with this for these days and finally come out with the solution.

Some USB camera outputs mjpegs that will be decoded by mmal, but some types of USB camera's output cannot be decoded.

[6by9]

In which case open a new issue in the correct place with a sample file.
If you provide an example that fails then I'm happy to analyse it.