After upgrade to the latest firmware, images from Pi Noir V2 now purple on Pi Zero W

[anfractuosity]

With the latest firmware in buster, the results under similar lighting conditions:

With the firmware in stretch, the results under similar lighting conditions:

I've tried using the same version of raspivid as in stretch, on buster, but still got purple results, which leads me to think this is a firmware issue.

I've also looked at images from during the day and at night time, all are now purple. Whereas the results from Stretch were markedly different.

Cheers

[6by9]

As per https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=241633, the AWB algorithm was updated a month ago.
Can you check whether this is an issue with the new algorithm by running sudo vcdbg set awb_mode 0 before running raspivid. AWB is never really expecting the IR component, therefore it could be doing odd things.

[anfractuosity]

Thanks a lot! So I just ran it now (night time here) with my IR light, the results are now B&W, rather than purple :)

I'll run it again tomorrow daytime, but it looks like that fixed it! Thanks a lot!

[6by9]

That's a workaround rather than the fix. I've reported it to the guys who wrote the new algorithm.

It's a curious one, and I suspect that even the old algorithm is giving up on it, but it then drops back to the simpler greyworld scheme which is likely to give you more sane mono pictures.

[anfractuosity]

Gotcha thanks a lot. During daytime, the images now look like the non-purple version above again with the workaround.

[Pi-DD]

Had the same problem on Pi Zero + Buster + MotionEye.

Running the suggested workaround fixes the problem, but seems to need running after every reboot, but after stopping the camera that MotionEye has auto started at reboot.

[anfractuosity]

@Pi-DD I created a systemd service that runs it for me:

[Unit]
Description=AWB
After=network.target

[Service]
Type=simple
User=root
WorkingDirectory=/usr/bin
ExecStart=vcdbg set awb_mode 0 
Restart=on-failure

[Install]
WantedBy=multi-user.target

[Pi-DD]

Thank you!

[popcornmix]

Latest rpi-update firmware has a potential fix to the new AWB scheme. Can you test it?

[anfractuosity]

I did 'sudo rpi-update' and disabled the systemd service I created, but the images still look purple unfortunately:

I just took that image from a stream, doing the following on the Pi:

raspivid -o tcp://0.0.0.0:10000 -l -t 0 -fps 30 -b 8000000 

[popcornmix]

@anfractuosity can you report output of vcgencmd version?

[anfractuosity]

Sure:

Jul  8 2019 21:30:17 
Copyright (c) 2012 Broadcom
version 6c3fe3f096a93de3b34252ad98cdccadeb534be2 (clean) (release) (start_x)

[naushir]

@anfractuosity, could you provide some more details please?

• Is the scene illuminated with IR light?
• Would you happen to know the lux level of the scene? Failing that, is it bright or dark?
• Any chance you can attach a capture with the --raw option?

[anfractuosity]

Sure, github didn't like me attaching the files (15MB each), but I've added a link below instead:

https://www.anfractuosity.com/files/hack.jpg - taken after the 'sudo vcdbg set awb_mode 0' trick
https://www.anfractuosity.com/files/nohack.jpg - taken using the latest firmware without vcdbg hack.

Both images were taken with --raw. Hope that helps.

Both images where taken minutes after each other. It's from daytime, under typical British weather ;) (rain and cloud). It's not IR illuminated at the moment.

[naushir]

Thank you for the images.

We think we have identified why this may be happening - summary is that the algorithm searches along the colour temperature curve measured for the standard camera and is (correctly) restricted from moving too far off from there.

We will need to discuss how to fix this correctly for the NoIR case. Until then, could you please run with the vcdbg set awb_mode 0 workaround.

[microadam]

I am having the same issue. I assume doing vcdbg set awb_mode 0 completely disables auto white balance? As the white balance on the camera is all over the place (mostly either very yellow or very red)

[microadam]

Have realised that commands just changes the way white balance is calculated. Must be something else going wrong with my setup! (running that command does remove the pink glow though). Only just got the PI camera, so had nothing to compare it against

[naushir]

vcdbg set awb_mode 0 reverts to the old AWB algorithm. If that shows any issues, we are unlikely to work on fixing them now. Instead, our efforts would be better spent getting the new algorithm working correctly for the Noir cases.

[2sec]

just adding my 2 cents..

• running vcdbg set awb_mode 0 while the camera is capturing can freeze the system, in that case it needs to be physically unplugged.

• I'm using a bunch of NoIR cameras as a surveillance system (using only the Y luminance data in YUV), the new algorithm makes the cameras unusable, when the sun shines through the windows, or in dark scenes, almost nothing can be seen.

[Pi-DD]

@2sec - thanks for that - I wondered what had happened when, following kernel updates yesterday I got the “purple image problem” on Stretch remote device (that had not had it before) and which then froze when I tried to fix it. Fortunately I can get to it and restart it and then add the system service suggested above to force the old AWB algorithm.

[Pi-DD]

@anfractuosity - I would appreciate some guidance on implementing your systemd service. I had not needed it up until the latest Stretch updates over the weekend have introduced the "purple haze" problem.

Not having done anything like that before I have tried to read up on this and make it work, but am obviously missing something!

I have placed the file in /etc/systemd/system as other material seemed to guide me and have enabled it.
systemctl list-unit-files shows it as enabled but
systemctl AWB.service shows "Unknown operation AWB.service"
and
systemctl start AWB.service shows
Failed to start AWB.service: Unit AWB.service is not loaded properly: Invalid argument. See system logs and 'systemctl status AWB.service' for details.
systemstl status AWB.service shows
`● AWB.service - AWB
Loaded: error (Reason: Invalid argument)
Active: inactive (dead)

Jul 15 19:54:46 DDhomepi5 systemd[1]: [/etc/systemd/system/AWB.service:11] Executable path is not absolute, ignoring: vcdb
Jul 15 19:54:46 DDhomepi5 systemd[1]: AWB.service: Service lacks both ExecStart= and ExecStop= setting. Refusing.

~

`

Inexperience showing through here, but help much appreciated.

Thanks

[Pi-DD]

@popcornmix - thank you, I can see you are trying to help, but with my (rather limited!) level of skill and knowledge I have no idea what to do about what you are pointing me to!

[anfractuosity]

@popcornmix I just tried the latest firmware

Jul 15 2019 17:36:28
Copyright (c) 2012 Broadcom
version ccada45e1764783c9876237218ef886d96b24fc7 (clean) (release) (start_x)

And attached two pictures taken just now under IR light:

• with new firmware

• With vcdbg set awb_mode 0

@Pi-DD Try this instead maybe, although the version I listed did work for me:

[Unit]
Description=AWB
After=network.target

[Service]
Type=simple
User=root
ExecStart=/usr/bin/vcdbg set awb_mode 0 
Restart=on-failure

[Install]
WantedBy=multi-user.target

sudo systemctl enable AWB
sudo systemctl start AWB

[Pi-DD]

@anfractuosity - many thanks for that, perfect! Worked first time. No idea why the original did not, but now working perfectly from boot to use the old algorithm.

[naushir]

A new firmware has been released (use rpi-update to get) that provides some updates to the AWB algorithm. In particular, there is a new "Grey World" mode of operation that should hopefully fix the issues with the NoIR cameras. This is sort of what the old algorithm was doing to get the "correct" results, although mostly by accident.

To use this new awb mode, you will need to update the raspicam binaries with a patch - see here Until this is fully merged you will have to build the binaries manually. Once you have done that, run raspivid -awb greyworld ... command line option to get it working.

[JessicaWoods03]

no problem, my tower melted down from the upgrade, so did I, I found a way to fix it, my apologies-

…

________________________________ From: Phil Elwell <[email protected]> Sent: Sunday, November 24, 2019 1:04 PM To: raspberrypi/firmware <[email protected]> Cc: Jessica Woods <[email protected]>; Comment <[email protected]> Subject: Re: [raspberrypi/firmware] After upgrade to the latest firmware, images from Pi Noir V2 now purple on Pi Zero W (#1167) @JessicaWoods03<https://github.com/JessicaWoods03> Abuse will not be tolerated. Your message has deleted. Further offences will result in blocking. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#1167?email_source=notifications&email_token=ALSSROGDOYAPNY3YGNB5NALQVLT7JA5CNFSM4H4M3WR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFAU3NI#issuecomment-557927861>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ALSSROA5DR2B3D7CWXQKTOTQVLT7JANCNFSM4H4M3WRQ>.

[JessicaWoods03]

So I did the fix- and it fixed- but somehow all of these are temp fixes- I did the awb_mode- IDK if its another upgrade messup- for 2020? Anyone else's camera took a crap this year? Need some help on this-

[JessicaWoods03]

I have a Pi 3B+ I had a hard time finding software support for the pi4- so I went with my old pi-

[naushir]

@JessicaWoods03, could you post a detailed explanation of your problem to the Raspberry Pi camera forums please? You can find it at https://www.raspberrypi.org/forums/viewforum.php?f=43.

As far as we know, there are no known issues with awb_mode=greyworld on Pi4 and this githhub issues will remain closed.

[Pi-DD]

Has any recent kernel update removed the ability to run the vcdbg “fix” to go back to the old AWB version?

I ask because everything indicates that my AWB.service runs successfully, but I am now getting the purple effect again.

[JessicaWoods03]

Yeah mine too- I am rerunning the vcdbg "fix" once before last year and it fixed it- and it went back to purple- Just this year recently with the latest updates-

[JamesH65]

You should not be using the vcdbg command to fix this, we introduced a new AWB mode parameter, greyworld, that should fix it correctly.

For example,

raspistill --awb greyworld -o fred.fpg

The MMAL parameter is MMAL_PARAM_AWBMODE_GREYWORLD.

[JamesH65]

Note, this should only be necessary for the NoIR camera.

[Pi-DD]

James - in my case the problem is that I don’t feel competent to install Motion from a tar file and the setup I am using needs the latest level of Motion (4.3.0) that supports “greyworld” to pass the the parameter to the camera. It’s not yet available as a deb file.

But just to be entirely clear - am I correct that the “fix” no longer works since grey world was introduced?

[JamesH65]

Quick look at the code and I cannot see any immediate reason why it would not work, would need to spend longer trying to figure it out and don't have time right now, especially since the vcdbg was very much a nasty hack for testing purposes whilst we implemented the correct fix, so should not be relied upon.

[JessicaWoods03]

James your awesome!! Thanks for helping us out :)

[naushir]

@JessicaWoods03, could you please post a log captured with:

sudo vcdb log msg

After running and stopping your camera application please? As @JamesH65 mentioned, this bit of code has not been touched in a while, so I am also a bit surprised that it has now broken.

[JessicaWoods03]

Yeah I can...give me a second...its a bit of a mess right now- I unplugged it...just called it a loss. This was a new issue I am sure no one expected to deal with-

[saumyakswain]

You should not be using the vcdbg command to fix this, we introduced a new AWB mode parameter, greyworld, that should fix it correctly.

For example,

raspistill --awb greyworld -o fred.fpg

The MMAL parameter is MMAL_PARAM_AWBMODE_GREYWORLD.

Thanks a lot. I used my Pi after 1year and found everything was pink. At first, Thought my cam had got some hardware issues. This answer solved my issue.

[6by9]

FYI I've merged a further firmware change for those who can't update their userspace apps. Once it is released, if you add awb_auto_is_greyworld=1 to config.txt then it'll do as it says and selecting auto will choose greyworld.
The preferred solution is to fix the app, but this is the easier fallback than the vcdbg line.

[johnb8]

Is the auto white balance greyworld preset supposed to work with the v4l2 module as well? I've tried setting it in qv4l2 but I still get purple images.

[6by9]

Is the auto white balance greyworld preset supposed to work with the v4l2 module as well? I've tried setting it in qv4l2 but I still get purple images.

Yes it should be available through V4L2 as long as you have updated the firmware appropriately.

[johnb8]

I've flashed a new SD card with the latest version of raspian but I still get purple images from my noir camera when using V4L2, even though I've set the white balance to greyworld. Images from raspistill look correct with the greyworld awb though. I'm using qv4l2 as my capture application. qv4l2 and v4l2-ctl --all both show the white balance preset as greyworld.

[dougputt]

Is there a logical fix in chronological order on how to fix the purple camera issue on Noir? There is rag tag information everywhere but ...
running PI4, noir v2, motioneye

Thanks

[6by9]

If you're really stuck, then add awb_auto_is_greyworld=1 to /boot/config.txt. On firmware since the end of March 2020 it means "auto" setting for awb mode uses the simple greyworld algorithm.

[Flo0]

My pics turned pink just yesterday. Im currently iterating over awb_gains(red, blue) in steps of range 0.05 in python to find out what works best. Every awb_mode setting is horribly pink. Is the greyworld setting accessible via the python api?

Also running Pi4 noir v2

[JamesH65]

@6by9 Any idea why this has just started happening? Has a patch been missed somewhere?

@Flo0 Just use the awb_auto_is_greyworld=1 option in config.txt

[satmandu]

Pi4 8Gb, HQ camera setup as NO-IR, RaspiOS 32 bit, kernel 5.4.49-v7l+, awb_auto_is_greyworld=1 is set in config.txt & firmware:

vcgencmd version
Jun 26 2020 14:49:28 
Copyright (c) 2012 Broadcom
version b078012603a6a094178f959805996ad225a0d83b (clean) (release) (start_x)

I was seeing a pink cast via qv4l2 but not when I ran raspivid --awb greyworld -t 0. Selecting greyworld in qv4l2 did not do anything.

However, if I run vcdbg set awb_mode 0 first, then qv4l2 doesn't have a pink cast.

So maybe there is an issue with awb_auto_is_greyworld?

Log if useful:

sudo vcdbg log msg
024422.355: arasan: arasan_emmc_open
024851.037: brfs: File read: /mfs/sd/config.txt
024851.864: brfs: File read: 1830 bytes
024905.707: brfs: File read: /mfs/sd/config.txt
026014.753: gpioman: gpioman_get_pin_num: pin DISPLAY_DSI_PORT not defined
026017.206: *** Restart logging
026017.227: brfs: File read: 1830 bytes
026058.430: hdmi: HDMI:hdmi_get_state is deprecated, use hdmi_get_display_state instead
026058.446: HDMI0: hdmi_pixel_encoding: 300000000

026058.458: HDMI1: hdmi_pixel_encoding: 300000000

026063.390: dtb_file 'bcm2711-rpi-4-b.dtb'
026068.320: brfs: File read: /mfs/sd/bcm2711-rpi-4-b.dtb
026068.334: Loading 'bcm2711-rpi-4-b.dtb' to 0x100 size 0xb577
026144.495: brfs: File read: 46455 bytes
026158.462: brfs: File read: /mfs/sd/overlays/overlay_map.dtb
026626.783: brfs: File read: 1371 bytes
026629.472: brfs: File read: /mfs/sd/config.txt
026630.349: dtparam: audio=on
026714.514: brfs: File read: 1830 bytes
026728.650: brfs: File read: /mfs/sd/overlays/vc4-fkms-v3d.dtbo
026868.653: Loaded overlay 'vc4-fkms-v3d'
026058.446: HDMI0: hdmi_pixel_encoding: 300000000

026058.458: HDMI1: hdmi_pixel_encoding: 300000000

026063.390: dtb_file 'bcm2711-rpi-4-b.dtb'
026068.320: brfs: File read: /mfs/sd/bcm2711-rpi-4-b.dtb
026068.334: Loading 'bcm2711-rpi-4-b.dtb' to 0x100 size 0xb577
026144.495: brfs: File read: 46455 bytes
026158.462: brfs: File read: /mfs/sd/overlays/overlay_map.dtb
026626.783: brfs: File read: 1371 bytes
026629.472: brfs: File read: /mfs/sd/config.txt
026630.349: dtparam: audio=on
026714.514: brfs: File read: 1830 bytes
026728.650: brfs: File read: /mfs/sd/overlays/vc4-fkms-v3d.dtbo
026868.653: Loaded overlay 'vc4-fkms-v3d'
027195.748: brfs: File read: 1238 bytes
027197.765: brfs: File read: /mfs/sd/cmdline.txt
027197.845: Read command line from file 'cmdline.txt':
027197.874: 'console=serial0,115200 console=tty1 root=PARTUUID=c303f23e-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet splash plymouth.ignore-serial-consoles'
029262.453: brfs: File read: 166 bytes
030433.658: brfs: File read: /mfs/sd/kernel7l.img
030433.695: Loading 'kernel7l.img' to 0x8000 size 0x5b6b70
030433.740: Device tree loaded to 0x2eff4500 (size 0xbafd)
030442.585: gpioman: gpioman_get_pin_num: pin SDCARD_CONTROL_POWER not defined
035323.536: vchiq_core: vchiq_init_state: slot_zero = 0xded80000, is_master = 1
035327.178: hdmi: HDMI:hdmi_get_state is deprecated, use hdmi_get_display_state instead
035334.750: TV service:host side not connected, dropping notification 0x00000002, 0x00000001, 0x00000010
038507.857: camsubs: Looking for camera 0: i2c_port = 0, led gpio = -1, power enable gpio = 133
038808.498: camsubs: Camera not found
038808.551: camsubs: Looking for camera 0: i2c_port = 0, led gpio = -1, power enable gpio = 133
039109.186: camsubs: Camera found OK
039164.203: TV service:host side not connected, dropping notification 0x00000008, 0x00000004, 0x00000000
081110.258: imx477_open: read id 1143 succes 0!
081298.347: Using RPI AWB
081486.091: brfs: File read: 5991280 bytes
081486.162: mmal: mmal_vll_load: could not load VLL 'videnc.vll': 
093594.377: imx477_open: read id 1143 succes 0!
093608.039: Using RPI AWB
093796.009: mmal: mmal_vll_load: could not load VLL 'videnc.vll': 
111812.872: imx477_open: read id 1143 succes 0!
111826.585: Using RPI AWB
161600.413: imx477_open: read id 1143 succes 0!
161610.011: Using RPI AWB
473202.867: imx477_open: read id 1143 succes 0!
473216.410: Using RPI AWB
473406.910: mmal: mmal_vll_load: could not load VLL 'videnc.vll': 
495734.122: imx477_open: read id 1143 succes 0!
495747.723: Using BRCM AWB

[michaelferwer]

If you're really stuck, then add awb_auto_is_greyworld=1 to /boot/config.txt. On firmware since the end of March 2020 it means "auto" setting for awb mode uses the simple greyworld algorithm.

Thanks @6by9 for this option. For people who use Motion, I have not found an other way to specify greyworld mode directly in Motion.
I have modified /boot/config.txt as mentioned and it works like a charm.