channel_shift_range not affecting images?

[Mahi-Mai]

So, I've decided adding random channel shift would be an important image augmentation for my training set. But when I test the output, I'm not seeing any changes no matter if I set channel_shift_range to 0.1 or 0.9. Am I misunderstanding how this augmentation works? I thought it was supposed to shift the colors around so the model would be robust to variances in color?

Here's my code:

from keras.preprocessing.image import ImageDataGenerator

import cv2
import matplotlib.pyplot as plt
%matplotlib inline

path = '/mnt/Project/Imaging/samples'

datagen = ImageDataGenerator(brightness_range=(0.25,1.35), channel_shift_range=0.9)

genObject = datagen.flow_from_directory(path,
                                        batch_size=1)

augs = []
i = 0

for batch in genObject:
    augs.append(batch)
    i += 1
    if i > 10:
        break
        
for item in augs:
    plt.imshow(item[0][0].astype('uint8'))
    plt.show()

Environment:
Jupyter Lab
Python 3.6.6
Keras==2.2.4
Keras-Applications==1.0.7
Keras-Preprocessing==1.0.9
tensorboard==1.9.0
tensorflow-gpu==1.9.0

Thanks in advance for the help!

[jingprism]

I think there was a mistake with the code. If you input some large value like 200.0, it will produce real channel shifts like mentioned here under "Old Channel_Shift Images(cifar10)": #17

However, it should be using a float between 0 and 1 rather than a large absolute value.

[jingprism]

Actually it used to work with that workaround in a previous version of keras, but not anymore. I looked through the code:

keras-preprocessing/keras_preprocessing/image/image_data_generator.py

Line 872 in 095aea9

if transform_parameters.get('channel_shift_intensity') is not None:

keras-preprocessing/keras_preprocessing/image/affine_transformations.py

Line 159 in 095aea9

def apply_channel_shift(x, intensity, channel_axis=0):

And tried using the apply_channel_shift on two different simple arrays:

arr = np.array([[[50,100,200],[50,100,200]], [[50,100,200],[50,100,200]]])
arr = np.array([[[50,50], [50,50]], [[100,100], [100,100]], [[200,200], [200,200]]])
apply_channel_shift(arr, 50.0, 0)

which results in:

array([[[100., 150., 200.],
        [100., 150., 200.]],
       [[100., 150., 200.],
        [100., 150., 200.]]])

or

array([[[100., 100.],
        [100., 100.]],
       [[150., 150.],
        [150., 150.]],
       [[200., 200.],
        [200., 200.]]])

No matter what channel_axis I use as a parameter

[Dref360]

This will be added to the list of issues needed to be fixed when we will work on the refactoring. (See keras-team/governance#10 for updates)

[mazatov]

It looks like 'channel_shift_range` adds/subtracts the same intensity from all channels in the image. So because of that the image just becomes brighter or darker instead of changing color. I imagine the expected behavior would be to change color too, no? In that case, we need to add different intensity to different channels.

    x = np.rollaxis(x, channel_axis, 0)
    min_x, max_x = np.min(x), np.max(x)
    channel_images = [
        np.clip(x_channel + intensity,
                min_x,
                max_x)
        for x_channel in x]
    x = np.stack(channel_images, axis=0)
    x = np.rollaxis(x, 0, channel_axis + 1)
    return x

[BSVogler]

I am still observing the same behavior. keras-team/governance#10 is merged. Is someone looking into this?

For reference: The code mazatov cited is found here

keras-preprocessing/keras_preprocessing/image/affine_transformations.py

Lines 159 to 180 in 0494094

	def apply_channel_shift(x, intensity, channel_axis=0):
	"""Performs a channel shift.
	# Arguments
	x: Input tensor. Must be 3D.
	intensity: Transformation intensity.
	channel_axis: Index of axis for channels in the input tensor.
	# Returns
	Numpy image tensor.
	"""
	x = np.rollaxis(x, channel_axis, 0)
	min_x, max_x = np.min(x), np.max(x)
	channel_images = [
	np.clip(x_channel + intensity,
	min_x,
	max_x)
	for x_channel in x]
	x = np.stack(channel_images, axis=0)
	x = np.rollaxis(x, 0, channel_axis + 1)
	return x

I am not familiar with some of the numpy methods, but I could look into finding a fix.

[BSVogler]

A fix should probably also address issue #170.

[apatsekin]

It works in a [0.0,255.0] range but instead of random channel shift i.e. r+20, g-10, b+3, it just applies random brightness: r+5, g+5,b+5 (shifting all channels by the same value).

[BSVogler]

I put a fix on a fork. Still needs to be tested before I add a pull request.
There is also some clipping to the min and max values contained in the channel. I do not understand the point of clipping to these min and max values. As noted in #170 is should probably clip to maximum allowed range for that data type. Further discussion about that in #170.

[nina-weng]

Has this problem solved yet? I checked @BSVogler 's fork mentioned before, but there's error about the np.random.uniform() function in the code (Line 206 in affine_transformations.py). Seems there's no parameters named channel anymore now.

Have you re-wrote the np.random.uniform function? how you write it?

[BSVogler]

I am sorry, I kinda forgot about this one. My fork needs to resolve the conflicts and probably update the usage of numpy since it has been more than a year.