No convenient way to apply class weights for a segmentation model

[AngledLuffa]

• Have I written custom code (as opposed to using example directory):

The custom code is to build a segmentation model: U-Net model, go from (batch, x, y, channels) -> (batch, x, y, classes)

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows, using Keras via Tensorflow 2.2
• TensorFlow backend (yes / no): Yes
• TensorFlow version: 2.2
• Keras version: 2.3.0
• Python version: 3.8.3
• CUDA/cuDNN version: 10.1
• GPU model and memory: Quadro RTX 5000 (16G)

Describe the current behavior

In older versions of Keras, such as version 2.2.4 which shipped with TF 2.1, it was possible to use class_weights when calling model.fit on a model with parameters shaped as above. Now it is no longer possible. Calling model.fit with such class_weights and a Y of (batch, x, y, classes) raises a ValueError:

ValueError: `class_weight` not supported for 3+ dimensional targets.

Using sample_weight also does not work, as it throws the following exception:

ValueError: Found a sample_weight array with shape (128, 256, 256). ... If you just mean to use sample-wise weights, make sure your sample_weight array is 1D.

There is no convenient alternative to any of the cross_entropy loss functions which takes class weights.

Expected behavior:

Any update which makes it easier to apply class_weights when doing segmentation or similar models would be appreciated.

For example, this causes the class_weight problems:

        class_weight = {0:.8,1:.3,2:.1,3:.3}
        model.fit(x_train, y_train, batch_size=BATCH_SIZE,
                  validation_data=(x_val, y_val),
                  class_weight=class_weight)

[gadagashwini]

@AngledLuffa,
Please follow the workarounds mentioned on similar thread #3653. And also refer this issue. Thanks!

[google-ml-butler]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[AngledLuffa]

Thank you for the suggestions. They will come in handy, should the project ever be resurrected... as with many academic projects, it seems to have been thrown over the fence once the paper was written

…

On Thu, May 5, 2022 at 5:16 AM gadagashwini ***@***.***> wrote: @AngledLuffa <https://github.com/AngledLuffa>, Please follow the workarounds mentioned on similar thread #3653 <#3653>. And also refer this issue <tensorflow/tensorflow#48714>. Thanks! — Reply to this email directly, view it on GitHub <#14126 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2AYWOYIWJHGWDAY4HS5DTVIO32BANCNFSM4OBANAZA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[gadagashwini]

Hi @AngledLuffa, Could you try the workaround mentioned in the comment and let us know if that solves your problem, Thank you!

[AngledLuffa]

The project I needed it for is basically over anyway, but thank you for the response. If it starts up again, I will give those workarounds a try. I do think supporting class weights for this kind of task would be a much cleaner interface than making the user reverse the semantics of sample_weights, though.

[gowthamkpr]

@AngledLuffa Sample weight for class imbalances is quite wasteful. One may not feel that when dealing with small datasets but it would become quite clear when dealing with volume data.Rather than wasting all the space to define similar values for an entire class incorporated into sample weights, one can write customized loss functions and split the loss function into multiple sums each with a single value for the associated class. You can look at the comment here. Can you please close this issue. Thanks!

[AngledLuffa]

Why do I need to close the issue? Presumably the keras team could decide to implement one of the solutions described in that thread and then close the issue as completed, or they could decide they won't implement it and close it as will not fix.

[google-ml-butler]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.

[willh5]

Hey I'm dealing with this exact issue now, would be nice if there was a fix