clrnet

zkyseu · Sep 13, 2023 · fe8c6c8 · fe8c6c8
1 parent 9c73f22
commit fe8c6c8
Show file tree

Hide file tree

Showing 216 changed files with 1,087 additions and 728 deletions.
diff --git a/README.md b/README.md
@@ -8,11 +8,8 @@
 ## 🆕新闻
 在这个部分中，我们展示PPLanedet中最新的改进，如果您想要了解更多关于PPLanedet的改动，您可以浏览[改动历史](https://github.com/zkyseu/PPlanedet/blob/v5/file/change_log.md)。
 <ul class="nobull">
+  <li>[2023-09-13] :fire: 完成CLRNet的复现，发布CLRNet最新代码以及预训练权重
   <li>[2023-05-01] : 我们基于DETR提出一个端到端的车道线检测模型<a href="https://github.com/zkyseu/O2SFormer">O2SFormer</a>, 欢迎大家使用！
-  <li>[2023-03-05] : 我们公开了UFLD模型在CULane上的预训练权重，并且增加了模型导出为预训练格式的功能。
-  <li>[2023-03-04] : 我们在V5中增加了Visualdl可视化功能，VisualDL功能类似tensorboard。后续我们会完善PPLanedet文档关于如何在PPLanedet中增加组件，如果想尝试可以参考<a href="https://github.com/open-mmlab/mmdetection/blob/master/docs/en/tutorials/customize_models.md">mmdetection</a>。
-  <li>[2023-03-01] : 我们修改了PPLanedet中的一些bug，目前CLRNet还在调试中，如果您想获得高性能的车道线检测模型，我们建议您可以使用我们改进的CondLaneNet。
-  <li>[2023-02-24] :fire: 我们发布了PPLanedet的第五个版本(version5)。在V5中，我们复现了更多实用的backbone和Neck等模块(例如YOLOv6中的CSPRepBiFPN、CSPSimSPPF)。依靠这些更加先进的模块，我们得到了一个性能更佳的CondLaneNet。改进的CondLaneNet在CULane数据集上达到79.92的F1 score并且参数量只有11M，更多的细节可以参考CondLaneNet的<a href="https://github.com/zkyseu/PPlanedet/tree/v5/configs/condlane">配置文件</a>。 
 
 </ul>
 
@@ -120,28 +117,35 @@ PPLanedet是一个基于PaddlePaddle的车道线检测工具包。PaddlePaddle
   </tbody>
 </table>
 
+
 ## 🛠️安装
-### 步骤1 安装 PaddlePaddle>=2.4.0(如果有疑问可以参考[官方文档](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/install/conda/linux-conda.html))
+<details>
+<summary>具体步骤</summary>
+
+ 步骤1 安装 PaddlePaddle==2.4.2(如果有疑问可以参考[官方文档](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/install/conda/linux-conda.html))
 ```Shell
 conda create -n pplanedet python=3.8 -y
 conda activate pplanedet
-conda install paddlepaddle-gpu==2.4.1 cudatoolkit=10.2 --channel https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/Paddle/
+conda install paddlepaddle-gpu==2.4.2 cudatoolkit=10.2 --channel https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/Paddle/
 ```
 
-### 步骤2 Git clone PPlanedet
+ 步骤2 Git clone PPlanedet
 ```Shell
 git clone https://github.com/zkyseu/PPlanedet
 ```
 
-### 步骤3 安装 PPlanedet
+ 步骤3 安装 PPlanedet
 ```Shell
 cd PPlanedet
 pip install -r requirements.txt
 python setup.py build develop
 ```
+</details>
 
 ## 📘数据集准备(CULane和Tusimple为例)
 ### CULane
+<details>
+<summary>CULane数据集准备步骤</summary>
 
 下载 [CULane](https://xingangpan.github.io/projects/CULane.html). 接着解压到 `$CULANEROOT`. 创建 `data` 目录.
 
@@ -157,8 +161,11 @@ $CULANEROOT/driver_xx_xxframe    # data folders x6
 $CULANEROOT/laneseg_label_w16    # lane segmentation labels
 $CULANEROOT/list                 # data lists
 ```
+</details>
 
 ### Tusimple
+<details>
+<summary>Tusimple数据集准备步骤</summary>
 下载 [Tusimple](https://github.com/TuSimple/tusimple-benchmark/issues/3). 然后解压到 `$TUSIMPLEROOT`. 创建 `data` 文件夹.
 
 ```Shell
@@ -182,11 +189,16 @@ $TUSIMPLEROOT/test_label.json # test label json file
 python tools/generate_seg_tusimple.py --root $TUSIMPLEROOT
 # python tools/generate_seg_tusimple.py --root /root/paddlejob/workspace/train_data/datasets --savedir /root/paddlejob/workspace/train_data/datasets/seg_label
 ```
+</details>
+
 ### 自制数据集
 如果你想在自己数据集上进行训练，我们在[issue #1](https://github.com/zkyseu/PPlanedet/issues/1)中对该问题进行了讨论，大家可以进行参考
 
 ## 💎开始快乐炼丹
 ### 1、训练的命令
+<details>
+<summary>开启训练</summary>
+
 对于训练, 运行以下命令(shell脚本在script文件夹下)。更多的训练命令可以参考[documentation](https://github.com/zkyseu/PPlanedet/blob/v3/DOC.md)
 ```Shell
 # training on single-GPU
@@ -200,16 +212,24 @@ python tools/train.py -c configs/scnn/resnet50_tusimple.py
 export CUDA_VISIBLE_DEVICES=0,1,2,3
 python -m paddle.distributed.launch tools/train.py -c configs/scnn/resnet50_tusimple.py
 ```
+</details>
 
 ### 2、测试
+<details>
+<summary>开启测试</summary>
+
 运行以下命令开启模型的测试
 ```Shell
 python tools/train.py -c configs/scnn/resnet50_tusimple.py \
                       --load /home/fyj/zky/tusimple/new/pplanedet/output_dir/resnet50_tusimple/latest.pd \
                       --evaluate-only 
 ```
+</details>
 
 ### 3、推理/Demo
+<details>
+<summary>开启推理</summary>
+
 想了解更多细节，请参考 `tools/detect.py`.
 ```
 python tools/detect.py --help
@@ -243,27 +263,41 @@ python tools/detect.py configs/scnn/resnet50_tusimple.py --img images\
 python tools/detect.py configs/scnn/resnet50_tusimple.py --img images\
           --load_from model.pd --savedir ./vis
 ```
+</details>
+
 
 ### 4、测试模型检测速度
+<details>
+<summary>开启速度测试</summary>
+
 如果你想要测试模型的速度，你可以运行以下的命令。但是需要注意的是测试脚本使用python进行编写并未采用常见的C++，因此测试得到的模型检测速度会低于论文报告的结果，但是也可以用来衡量不同模型间检测速度快慢
 ```
  python tools/test_speed.py configs/condlane/cspresnet_50_culane.py --model_path output_dir/cspresnet_50_culane/model.pd
 ```
+</details>
 
 ### 5、VisualDL可视化
+<details>
+<summary>开启可视化</summary>
+
 如果你想可视化中间过程的loss，请在终端运行以下命令，其中log为存放日志的文件夹，更多的命令以及功能请参考[VisualDL](https://github.com/PaddlePaddle/VisualDL)
 ```
 # 首先你需要在配置文件中加上use_visual = True，训练完后即可得到日志文件，将其放在log文件夹下
 visualdl --logdir ./log
 ```
+</details>
 
 ### 6、模型导出
+<details>
+<summary>开启模型导出</summary>
+
 如果你想将模型导出为预训练的格式(只保留模型权重去除优化器以及学习率的权重)，可以使用以下命令
 ```
 python tools/train.py -c configs/ufld/mobilenetv3_culane.py --export output_dir/mobilenetv3_culane/epoch_51.pd
 #如果模型权重中包含RepVGG模块，可以运行以下命令来将RepVGG中卷积进行重参数化。
 #python tools/train.py -c config path --export model path --export_repvgg
 ```
+</details>
 
 ## License
 PPLanedet使用[MIT license](LICENSE)。但是我们仅允许您将PPLanedet用于学术用途。

diff --git a/configs/clrnet/README.md b/configs/clrnet/README.md
@@ -0,0 +1,9 @@
+# CLRNet: Cross Layer Refinement Network for Lane Detection
+
+## Abstract
+Lane is critical in the vision navigation system of the intelligent vehicle. Naturally, lane is a traffic sign with high-level semantics, whereas it owns the specific local pattern which needs detailed low-level features to localize accurately. Using different feature levels is of great importance for accurate lane detection, but it is still under-explored. In this work, we present Cross Layer Refinement Network (CLRNet) aiming at fully utilizing both high-level and low-level features in lane detection. In particular, it first detects lanes with high-level semantic features then performs refinement based on low-level features. In this way, we can exploit more contextual information to detect lanes while leveraging local detailed lane features to improve localization accuracy. We present ROIGather to gather global context, which further enhances the feature representation of lanes. In addition to our novel network design, we introduce Line IoU loss which regresses the lane line as a whole unit to improve the localization accuracy. Experiments demonstrate that the proposed method greatly outperforms the state-of-the-art lane detection approaches.
+
+## Model List
+| Architecture| Backbone |Dataset | Metric | Config| Checkpoints  |
+|-------------|----------|--------|--------|-------|--------------|
+| CLRNet      | ResNet34 | CUlane |F1: 78.94| [config](resnet34_culane.py)  | [model](https://github.com/zkyseu/PPlanedet/releases/download/erfnet/erfnet.pd)|
diff --git a/configs/clrnet/resnet18_tusimple.py b/configs/clrnet/resnet18_tusimple.py
@@ -0,0 +1,146 @@
+model = dict(name='Detector', )
+
+backbone = dict(
+    name='ResNetWrapper',
+    resnet='resnet18',
+    pretrained=True,
+    replace_stride_with_dilation=[False, False, False],
+    out_conv=False,
+)
+
+num_points = 72
+max_lanes = 5
+sample_y = range(710, 150, -10)
+featuremap_out_channel = 192
+
+heads = dict(name='CLRHead',
+             num_priors=192,
+             refine_layers=3,
+             fc_hidden_dim=64,
+             sample_points=36,
+             seg_decoder=dict(name='PlainDecoder'),
+             cls_loss = dict(name = 'FocalLoss_cls',alpha=0.25),
+             liou_loss = dict(name = 'Liou_loss'),
+             ce_loss = dict(name = 'CrossEntropyLoss',
+                   weight = (0.4,1,1,1,1,),))
+
+iou_loss_weight = 2.
+cls_loss_weight = 6.
+xyt_loss_weight = 0.5
+seg_loss_weight = 1.0
+
+neck = dict(name='FPN',
+            in_channels=[128, 256, 512],
+            out_channels=64,
+            num_outs=3,
+            attention=False)
+
+test_parameters = dict(conf_threshold=0.40, nms_thres=50, nms_topk=max_lanes)
+
+epochs = 70
+batch_size = 24 
+
+total_iter = (3616 // batch_size + 1) * epochs
+lr_scheduler = dict(name='CosineAnnealingDecay',learning_rate = 0.6e-3, T_max=total_iter)
+optimizer = dict(name='AdamW')  # 3e-4 for batchsize 8
+
+eval_ep = 3
+save_ep = epochs
+
+img_norm = dict(mean=[103.939, 116.779, 123.68], std=[1., 1., 1.])
+ori_img_w = 1280
+ori_img_h = 720
+img_height = 320
+img_width = 800
+cut_height = 160 
+
+train_transform = [
+    dict(
+        name='GenerateCLRLine',
+        transforms=[
+            dict(name='Resize',
+                 parameters=dict(size=dict(height=img_height, width=img_width)),
+                 p=1.0),
+            dict(name='HorizontalFlip', parameters=dict(p=1.0), p=0.5),
+            dict(name='ChannelShuffle', parameters=dict(p=1.0), p=0.1),
+            dict(name='MultiplyAndAddToBrightness',
+                 parameters=dict(mul=(0.85, 1.15), add=(-10, 10)),
+                 p=0.6),
+            dict(name='AddToHueAndSaturation',
+                 parameters=dict(value=(-10, 10)),
+                 p=0.7),
+            dict(name='OneOf',
+                 transforms=[
+                     dict(name='MotionBlur', parameters=dict(k=(3, 5))),
+                     dict(name='MedianBlur', parameters=dict(k=(3, 5)))
+                 ],
+                 p=0.2),
+            dict(name='Affine',
+                 parameters=dict(translate_percent=dict(x=(-0.1, 0.1),
+                                                        y=(-0.1, 0.1)),
+                                 rotate=(-10, 10),
+                                 scale=(0.8, 1.2)),
+                 p=0.7),
+            dict(name='Resize',
+                 parameters=dict(size=dict(height=img_height, width=img_width)),
+                 p=1.0),
+        ],
+    ),
+    dict(name='ToTensor', keys=['img', 'lane_line', 'seg']),
+]
+
+val_transform = [
+    dict(name='GenerateCLRLine',
+         transforms=[
+             dict(name='Resize',
+                  parameters=dict(size=dict(height=img_height, width=img_width)),
+                  p=1.0),
+         ],
+         training=False),
+    dict(name='ToTensor', keys=['img']),
+]
+
+dataset_path = '/home/fyj/zky/tusimple'
+
+dataset = dict(
+    train=dict(
+        name='TuSimple',
+        data_root=dataset_path,
+        split='trainval',
+        processes=train_transform,
+    ),
+    val=dict(
+        name='TuSimple',
+        data_root=dataset_path,
+        split='test',
+        processes=val_transform,
+    ),
+    test=dict(
+        name='TuSimple',
+        data_root=dataset_path,
+        split='test',
+        processes=val_transform,
+    )
+)
+
+
+log_config = dict(
+    name = 'LogHook',
+    interval = 50
+    )
+
+custom_config = [dict(
+    name = 'EvaluateHook'
+    )]
+
+device = 'gpu'
+seed =  0
+save_inference_dir = './inference'
+output_dir = './output_dir'
+best_dir = './output_dir/best_dir'
+pred_save_dir = './pred_save'
+num_workers = 4
+num_classes = 6 + 1
+view = False
+ignore_label = 255
+seg=False
diff --git a/configs/clrnet/resnet34_culane.py b/configs/clrnet/resnet34_culane.py
@@ -35,16 +35,15 @@
             num_outs=3,
             attention=False)
 
-test_parameters = dict(conf_threshold=0.40, nms_thres=50, nms_topk=max_lanes)
+test_parameters = dict(conf_threshold=0.40, nms_thres=0.8, nms_topk=max_lanes)
 
 epochs = 15
 batch_size = 48 
 
-total_iter = (88880 // batch_size + 1) * epochs
-lr_scheduler = dict(name='CosineAnnealingDecay',learning_rate = 0.6e-3, T_max=total_iter)
+lr_scheduler = dict(name='CosineAnnealingDecay',learning_rate = 0.6e-3, T_max=epochs)
 optimizer = dict(name='AdamW')  # 3e-4 for batchsize 8
 
-eval_ep = 3
+eval_ep = 1
 save_ep = epochs
 
 img_norm = dict(mean=[103.939, 116.779, 123.68], std=[1., 1., 1.])
@@ -139,4 +138,5 @@
 num_workers = 4
 num_classes = 5
 view = False
-ignore_label = 255
+ignore_label = 255
+seg=False
diff --git a/configs/condlane/convnext_culane.py b/configs/condlane/convnext_culane.py
@@ -228,3 +228,4 @@
 num_workers = 4
 view = False
 ignore_label = 255
+seg=False
diff --git a/configs/condlane/cspresnet_50_culane.py b/configs/condlane/cspresnet_50_culane.py
@@ -250,3 +250,4 @@
 num_workers = 4
 view = False
 ignore_label = 255
+seg=False
diff --git a/configs/condlane/resnet50_culane.py b/configs/condlane/resnet50_culane.py
@@ -230,3 +230,4 @@
 num_workers = 4
 view = False
 ignore_label = 255
+seg=False
diff --git a/configs/deeplabv3p/resnet50_tusimple.py b/configs/deeplabv3p/resnet50_tusimple.py
@@ -113,3 +113,4 @@
 view = False
 ignore_label = 255
 test_json_file='/home/fyj/zky/tusimple/test_label.json'
+seg=True
diff --git a/configs/erfnet/erfnet_tusimple.py b/configs/erfnet/erfnet_tusimple.py
@@ -105,3 +105,5 @@
 view = False
 ignore_label = 255
 test_json_file='/home/fyj/zky/tusimple/test_label.json'
+seg=True
+
diff --git a/configs/resa/resa18_tusimple.py b/configs/resa/resa18_tusimple.py
@@ -117,3 +117,5 @@
 view = False
 ignore_label = 255
 test_json_file='/home/fyj/zky/tusimple/test_label.json'
+seg=True
+
diff --git a/configs/rtformer/rtformer_tusimple.py b/configs/rtformer/rtformer_tusimple.py
@@ -112,3 +112,4 @@
 view = False
 ignore_label = 255
 test_json_file='/home/fyj/zky/tusimple/test_label.json'
+seg=True
diff --git a/configs/scnn/resnet50_culane.py b/configs/scnn/resnet50_culane.py
@@ -126,6 +126,8 @@
 exist_num_class = num_classes
 view = False
 ignore_label = 255
+seg=True
+