[Bug] mask color can‘t be changed

[fellen88]

Checklist

• I have searched related issues but cannot get the expected help.
• 2. I have read the FAQ documentation but cannot get the expected help.
• 3. The bug has not been fixed in the latest version.

Describe the bug

##Code：
mmdeploy::Detector::Result dets = detector->Apply(image_scene);
output_mask_count = 0;
for (const mmdeploy_detection_t& det : dets)
{
if (det.score > score_threshold)
{
//fprintf(stdout, "mask %d, height=%d, width=%d\n", index, mask->height, mask->width);
auto x0 = (int)std::max(std::floor(det.bbox.left) - 1, 0.f);
auto y0 = (int)std::max(std::floor(det.bbox.top) - 1, 0.f);
utils::Visualize v;
auto output = v.get_session(image_scene);
output.add_instance_mask({ x0, y0 }, 250, det.mask->data, det.mask->height, det.mask->width, 1.0);
dets_mask.push_back(output.get());
dets_label.push_back(det.label_id);

		output_mask_count++;
	}

Reproduction

##Issue:
I try to change color of mask (show as below 250),but it doesn't work.
output.add_instance_mask({ x0, y0 }, 250, det.mask->data, det.mask->height, det.mask->width, 1.0);

Environment

##Version： windows prebuild package “mmdeploy-1.1.0-windows-amd64-cuda11.3”

Error traceback

No response

[irexyc]

I don't understand what you are trying to do

utils::Visualize v;
https://github.com/open-mmlab/mmdeploy/blob/main/demo/csrc/cpp/utils/visualize.h#L239

This line create a Palette with 32 random color.

output.add_instance_mask({ x0, y0 }, 250, det.mask->data, det.mask->height, det.mask->width, 1.0);
https://github.com/open-mmlab/mmdeploy/blob/main/demo/csrc/cpp/utils/visualize.h#L97

This line will use the 250 % 32 = 26 th color in Palette

[fellen88]

I don't understand what you are trying to do

utils::Visualize v;
https://github.com/open-mmlab/mmdeploy/blob/main/demo/csrc/cpp/utils/visualize.h#L239

This line create a Palette with 32 random color.

output.add_instance_mask({ x0, y0 }, 250, det.mask->data, det.mask->height, det.mask->width, 1.0);
https://github.com/open-mmlab/mmdeploy/blob/main/demo/csrc/cpp/utils/visualize.h#L97

This line will use the 250 % 32 = 26 th color in Palette

I try a lot of color ，but the mask shown in image is alway black. Is that normal？

[fellen88]

I don't understand what you are trying to do

utils::Visualize v;
https://github.com/open-mmlab/mmdeploy/blob/main/demo/csrc/cpp/utils/visualize.h#L239

This line create a Palette with 32 random color.

output.add_instance_mask({ x0, y0 }, 250, det.mask->data, det.mask->height, det.mask->width, 1.0);
https://github.com/open-mmlab/mmdeploy/blob/main/demo/csrc/cpp/utils/visualize.h#L97

This line will use the 250 % 32 = 26 th color in Palette

I try a lot of color ，but the mask shown in image is alway black. Is that normal？

I want to change color of mask to blue，however I don't know how to set the parameter？

[irexyc]

what the output image looks like?

[fellen88]

what the output image looks like?

[fellen88]

@irexyc Do you have any advice?

[irexyc]

@fellen88

Are you using a official model? If so, can you provide your convertion command?

After convertion, there should be two visualize images, are they look same?

[fellen88]

@fellen88

Are you using a official model? If so, can you provide your convertion command?

After convertion, there should be two visualize images, are they look same?

@irexyc

Yes， I use official model. After convertion, two visualize images are not look same, one is red and the other is pink.

[irexyc]

@fellen88

Yes， I use official model. After convertion, two visualize images are not look same, one is red and the other is pink.

You can paste the two visualize images.

For the use of SDK, if you don't change the code, what is the output looks like? Could you also paste the image?

[fellen88]

@irexyc For the use of SDK, if I don't change the code, the output is as below.

[irexyc]

@fellen88
It seems the sdk can't inferenece the right result. You have first to make the sdk inference right results before changing the color.

Is it convenient for you to share the model config、model pth and convertion command?

[fellen88]

@irexyc
I think the reference result is right. The image above is one reference result seceleted by score, the other result is as below.
**convertion command:
python3 ./tools/deploy.py configs/mmdet/instance-seg/instance-seg_tensorrt_dynamic-320x320-1344x1344.py ../mmdetection/data/config/dy_mask-rcnn_r50-caffe_fpn_ms-poly-1x_coco.py ../mmdetection/data/model/epoch_12.pth ../mmdetection/data/data_set/test/demo.jpg --work-dir ../mmdetection/data/converter_mmdeploy/test --show --device cuda:0 --dump-info

**model config:

新配置继承了基本配置，并做了必要的修改

base = '../../configs/mask_rcnn/mask-rcnn_r50-caffe_fpn_ms-poly-1x_coco.py'

我们还需要更改 head 中的 num_classes 以匹配数据集中的类别数

model = dict(
roi_head=dict(
bbox_head=dict(num_classes=1), mask_head=dict(num_classes=1)))

修改数据集相关配置

data_root = 'data/data_set/test/'
metainfo = {
'classes': ('1', ),
'palette': [
(220, 20, 60),
]
}
train_dataloader = dict(
batch_size=1,
dataset=dict(
data_root=data_root,
metainfo=metainfo,
ann_file='train/annotations.json',
data_prefix=dict(img='train/')))
val_dataloader = dict(
dataset=dict(
data_root=data_root,
metainfo=metainfo,
ann_file='val/annotations.json',
data_prefix=dict(img='val/')))
test_dataloader = val_dataloader

修改评价指标相关配置

val_evaluator = dict(ann_file=data_root + 'val/annotations.json')
test_evaluator = val_evaluator

使用预训练的 Mask R-CNN 模型权重来做初始化，可以提高模型性能

load_from = 'https://download.openmmlab.com/mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_caffe_fpn_mstrain-poly_3x_coco/mask_rcnn_r50_caffe_fpn_mstrain-poly_3x_coco_bbox_mAP-0.408__segm_mAP-0.37_20200504_163245-42aa3d00.pth'

可视化

log_config = dict(
interval=1,
hooks=[
dict(type='TextLoggerHook'),
dict(type='TensorboardLoggerHook')
])

work_dir = 'data/model/test'

[irexyc]

现在有点乱。

你的数据集转模型，看起来output_pytorch.jpg和output_tensorrt.jpg是一样的，但是你使用sdk，输出貌似只有一个框？

再上面，椅子那张图，sdk的输入没结果？

我觉得先把sdk的输出对齐再说颜色的问题。

[fellen88]

@irexyc 椅子那幅图输出结果是正确的，因为根据score做了筛选，输出mask只有一辆车（黑色mask）。
后面圆盘的图输出结果也是正确的，也根据score做了筛选，sdk分成两个图显示的结果（也都是黑色mask）。
现在的问题是我测试了很多不同的值，输出mask的颜色都是黑色，没有改变，想问一下如果能够改变mask颜色

[fellen88]

@irexyc 我测试过，不根据score筛选的话，输出结果跟转换完成的模型输出的结果是相同的，所以sdk输出没有问题，只是颜色不能修改

[irexyc]

@fellen88
我看了一下，Palette随机生成颜色的代码有一处bug。你按照这个地方改一下，颜色应该就不是黑色的了 #2356

如果想要自定义颜色的话，可以按照你的需求构造一个Palette，然后使用set_palette来替换session默认的palette_，之后再add_instance_mask就可以了

[fellen88]

@irexyc 修改后颜色显示正确了，谢谢！

[wang1528186571]

@fellen88 我看了一下，Palette随机生成颜色的代码有一处bug。你按照这个地方改一下，颜色应该就不是黑色的了 #2356

如果想要自定义颜色的话，可以按照你的需求构造一个Palette，然后使用set_palette来替换session默认的palette_，之后再add_instance_mask就可以了

你好，我在推理中想创建一个自己的颜色，但我不太理解这个set_palette 可以给个实例吗？