Changhong Fu∗, Yiheng Wang, Liangliang Yao, Guangze Zheng, Haobo Zuo, and Jia Pan * Corresponding author.
- [2024/10] 💻 NAT2024-1 benchmark and TDA-Track checkpoint have been released.
- [2024/03] 💻 Code has been released.
Nighttime UAV tracking has achieved great progress by domain adaptation (DA) under low-illuminated scenarios. However, previous DA works are defcient in narrowing the discrepancy of temporal contexts for UAV trackers. To address the issue, this work proposes a prompt-driven temporal domain adaptation framework to fully utilize temporal contexts for challenging nighttime UAV tracking, i.e., TDA-Track. Specifically, the proposed framework aligns the distribution of temporal contexts from different domains by training the temporal feature generator against the discriminator. The temporal-consistent discriminator progressively extracts shared domain-specifc features to generate coherent domain discrimination results in the time series. Additionally, to obtain high-quality training samples, a prompt-driven object miner is employed to precisely locate objects in unannotated nighttime videos. Moreover, a new benchmark for nighttime UAV tracking is constructed. Exhaustive evaluations of TDA-Track demonstrate remarkable performance on both public and self-constructed benchmarks. Real-world tests also show its practicality. The code and demo videos are available here.
Please install related libraries:
pip install -r requirement.txtTo train TDA-Track, more libraries are needed to obtain training samples from nighttime raw videos. More details can be found on NetTrack.
The checkpoint is available here, password: djp8.
To test on NAT2024-1 and NUT_L benchmarks, you need to download them from the following links:
python test.py
You can find the tracking results of NAT2024-1, NUT_L in results.zip file.
If you want to evaluate the tracker, please put those results into results directory.
python eval.py \
--tracker_path ./results \ # result path
--dataset UAV10 \ # dataset_name
--tracker_prefix 'result' # tracker_name
Download the daytime tracking datasets：
- VID
- GOT-10K
Note: train_dataset/dataset_name/readme.md has listed detailed operations about how to generate training datasets.
Download the nighttime tracking datasets：
- NAT2021
Note: NAT2021-train set is unannotated, the training samples are obtained with the prompt-driven object mining approach, as presented in [Preprocessing](#Preprocessing phase)
Preprocessing please refer to ... to be completed...
To train the model, run train.py with the desired configs:
to be completed...
-
📊 Long-term Nighttime UAV Tracking (NUT2024-40L) Benchmark:
- 🎬40 videos shot by UAV in various scenarios at night
- 🎯collected for artifical intelligence research
-
📥 Download NUT2024-40L dataset
The code is based on NetTrack, UDAT, and TCTrack. The authors appreciate the great work and the contributions they made.