Additional Multitask head

[abhigoku10]

hi thanks for open-sourcing this wonderful , i have the following queries

1. can we additional head called object_features which classifies the features of the detected objects eg car: type of car, brand of car, color of car-like tat as an additional branch to obtain fine-grained details ?
2. In bdd100k dataset there are other labels of scene classification like day-night weather, can we have another brach that gives this following results

for the above task how feasible can the current work be extended? thanks in advance

[Riser6]

Thank you for your attention to our project. I think all tasks you mentioned worth a try. We can construct our framework based on the relationship between different tasks

[abhigoku10]

@Riser6 thanks for the response , which one is more feasilbe according to you point1 or point2 . and should we make modifications in the loss ??

[PigLogic-Cyber]

I also have a similar question, the YOLOP model right now seems to have the ability to detect only the car, if I would like to detect more classes of object, what parameters should I modified? I had already tried to modify model.nc in train.py to be 41, change the single_cls to be False in bdd.py, uncomment the bdd_labels dict in convert.py, but still I got the error said:
Traceback (most recent call last): File "tools/train.py", line 406, in <module> main() File "tools/train.py", line 333, in main train(cfg, train_loader, model, criterion, optimizer, scaler, File "/home/wrz/Github/YOLOP-Infrared/lib/core/function.py", line 77, in train total_loss, head_losses = criterion(outputs, target, shapes,model) File "/home/wrz/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "/home/wrz/Github/YOLOP-Infrared/lib/core/loss.py", line 50, in forward total_loss, head_losses = self._forward_impl(head_fields, head_targets, shapes, model) File "/home/wrz/Github/YOLOP-Infrared/lib/core/loss.py", line 96, in _forward_impl iou = bbox_iou(pbox.T, tbox[i], x1y1x2y2=False, CIoU=True) # iou(prediction, target) File "/home/wrz/Github/YOLOP-Infrared/lib/core/general.py", line 38, in bbox_iou print(box1[0] - box1[2] / 2) File "/home/wrz/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/_tensor.py", line 249, in __repr__ return torch._tensor_str._str(self) File "/home/wrz/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/_tensor_str.py", line 415, in _str return _str_intern(self) File "/home/wrz/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/_tensor_str.py", line 390, in _str_intern tensor_str = _tensor_str(self, indent) File "/home/wrz/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/_tensor_str.py", line 251, in _tensor_str formatter = _Formatter(get_summarized_data(self) if summarize else self) File "/home/wrz/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/_tensor_str.py", line 90, in __init__ nonzero_finite_vals = torch.masked_select(tensor_view, torch.isfinite(tensor_view) & tensor_view.ne(0)) RuntimeError: CUDA error: device-side assert triggered CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Please help me out!

[eversouls]

hallo，i did some work to classify the scene(added a head for classification)，but the result is not good, is the feature map from the backbone(encoder) suitable for scene classification? or i did something wrong about the network construct.