add bayer array support for IMX477 sensor

[AlecVercruysse]

also fix the bug metioned in #563

[AlecVercruysse]

Closes #633.

Not sure if my commit to the tests was implemented correctly, I was unable to run any tests on my rpi.
Thanks.

[jtc42]

Hey, just to let you know I've merged a copy of this PR into a picamera fork we're using for some projects, found at labthings#2

Thanks!

[xlla]

I think the logic inside _unpack_data is different to old 10-bit unpack code.

[AlecVercruysse]

@xlla could you elaborate, do you mean the logic for the 10-bit unpack code has changed in this commit to be incorrect?
Plugging in n=5, and b=10 I see no difference between the old and new unpack code other than the fix of #563. Thanks.

[xlla]

@AlecVercruysse At first glance, I think there are many different. after read fix #563 , the only different is line 437, unpacked_array will be create many times.
and I think line 436, " (B * * 2 - 1) " should be " 2 * * B - 1 ", though for B=2 or B=4 , it's same.
I don't know how to create a PR for another PR, then I commit a mend in my repo,
here

[AlecVercruysse]

Good catches, thank you! I changed _unpack_data in picamera/array.py to reflect the changes you have put in docs/examples/bayer_data.py in #660.

[Bra1nsen]

Hey guys. Do you know how the Bayer colorfilter is aligned? I looked up the doc of the IMX477 but still not sure.

If you know, please response!

[Bra1nsen]

I work with bayer pattern images. I need to know which color the first Pixel in the captured array has..

[AlecVercruysse]

I no longer have access to the sensor, but you can check using this library. find what PiBayerArray._header.bayer_order is for the sensor, and BAYER_OFFSETS on line 398 of picamera/array.py will give you the pattern in terms of (ry, rx), (gy, gx), (Gy, Gx), (by, bx) coordinates. The PiBayerArray._to_3d method might be helpful to interpret the bayer offsets!

[Bra1nsen]

thanks for your help Alec, what would you say if you read the code, what color is the first pixel (1,1)? &does this also apply to the imx477 (HQ Cam)?

[6by9]

Bayer order varies depending on horizontal and vertical flips, so there is no one guaranteed answer.

The header of the raw capture includes metadata which tells you the Bayer order that frame has. @AlecVercruysse appears to have already given you the runes required to extract this in Python (in C see https://github.com/6by9/dcraw/blob/master/dcraw.c#L6465)

[Bra1nsen]

Thanks for reply. I use the following:

picam2 = Picamera2()
config = picam2.create_still_configuration(raw={"format": "SRGGB12", "size" : (2032, 1520)}, buffer_count=2)
picam2.configure(config)

picam2.set_controls({"ExposureTime": 560, "AnalogueGain": 1, "ColourGains": (1.0,1.0)})
picam2.start()
array = picam2.capture_array("raw")
picam2.stop()

hdr = array.view(np.uint16)

bayer = np.zeros((2*760, 2*1016, 3), dtype=np.uint16)
bayer[::2, ::2, 2]=hdr[0::2, 0::2] #blue
bayer[1::2, ::2, 1]=hdr[1::2, 0::2] #green
bayer[::2, 1::2, 1]=hdr[0::2, 1::2] #green
bayer[1::2, 1::2, 0]=hdr[1::2, 1::2] #red

cv2.imwrite(f"{utc_timestamp}_rgbayer.png", bayer)

#Normalize then scale to 255 and convert to uint8 
equals_rgb = (bayer/(2**12-1)) * 255 #12bit dynamic range
equals_rgb = np.uint8(equals_rgb)
iio.imwrite(f'{utc_timestamp}_rgb.tga', equals_rgb)

#Normalize according to exposure profile
convert8bit_rgb = img_as_ubyte(exposure.rescale_intensity(bayer))
iio.imwrite(f'{utc_timestamp}_rgbexp.tga', convert8bit_rgb)

Could you please show me in code, how to extract the metadata/information for the colorfilter information?

[Bra1nsen]

Hey I took the time to convert the raw bayer array to BGGR GBRG GRBG and RGGB.

#BGGR---------------------------------------------------------
bggr = np.zeros((2*760, 2*1016, 3), dtype=np.uint16)
bggr[::2, ::2, 2]=bayer[0::2, 0::2] #blue
bggr[1::2, ::2, 1]=bayer[1::2, 0::2] #green
bggr[::2, 1::2, 1]=bayer[0::2, 1::2] #green
bggr[1::2, 1::2, 0]=bayer[1::2, 1::2] #red
#RGGB------------------------------------------------------------
rggb = np.zeros((2*760, 2*1016, 3), dtype=np.uint16)
rggb[::2, ::2, 2]=bayer[1::2, 1::2] #blue
rggb[1::2, ::2, 1]=bayer[0::2, 1::2] #green
rggb[::2, 1::2, 1]=bayer[1::2, 0::2] #green
rggb[1::2, 1::2, 0]=bayer[0::2, 0::2] #red
#GRBG----------------------------------------------------------
grbg = np.zeros((2*760, 2*1016, 3), dtype=np.uint16)
grbg[::2, ::2, 2] = bayer[1::2, 0::2]  # blue
grbg[1::2, ::2, 1] = bayer[0::2, 0::2]  # green
grbg[::2, 1::2, 1] = bayer[1::2, 1::2]  # green
grbg[1::2, 1::2, 0] = bayer[0::2, 1::2]  # red
#GBRG----------------------------------------------------------------------
gbrg = np.zeros((2*760, 2*1016, 3), dtype=np.uint16)
gbrg[::2, ::2, 2] = bayer[0::2, 1::2]  # blue
gbrg[1::2, ::2, 1] = bayer[1::2, 1::2]  # green
gbrg[::2, 1::2, 1] = bayer[0::2, 0::2]  # green
gbrg[1::2, 1::2, 0] = bayer[1::2, 0::2]  # red

These are the result images:

it looks like BGGR is the way to go. since iam only interested in radiometric characteristics of the images, I was wondering if it would even matter^^

printing config['raw']['format'] shows SBGGR12. I guess I can trust that.