Dataset autodownload feature addition

[glenn-jocher]

Dataset autodownload branch initial commit.

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Refinement of dataset handling and download procedures for YOLOv5.

📊 Key Changes

• Updated coco.yaml, coco128.yaml, and voc.yaml dataset configurations to include optional download commands.
• Removed the get_coco2017.sh script and replaced it with get_coco.sh in a new directory scripts.
• Moved and updated the PASCAL VOC download script get_voc.sh to the scripts directory.
• Deleted specific download commands and added a general check to confirm dataset existence or trigger an autodownload if necessary.
• Minor refactoring in test.py and train.py to call the new dataset check function and handle download if needed.

🎯 Purpose & Impact

• Improves user experience by streamlining dataset acquisition; users can now easily download needed datasets directly from the setup commands.
• Reduces ambiguity by centralizing dataset downloading scripts in a specific scripts directory.
• Enhances code maintainability and reduces code duplication by using a single function to check for dataset presence and initiate download.
• Potential impact includes reduced setup errors and an automated approach to managing dataset availability, allowing users to get started with training models more efficiently. 🚀

[glenn-jocher]

HI Glenn, one thing I saw was that this branch didn’t incorporate my fix on “MissingAttribute” stride.

Ah! Ok it needs a rebase then. Maybe this will work: /rebase

[NanoCode012]

My default unit test passed! I didn’t use any auto download functionality though.

How should I use it to download coco128?

Would calling train auto download it if I don’t have the folder available?

[glenn-jocher]

@NanoCode012 yes, now you can start training without downloading a dataset first! In this screenshot you can see we clone the repo, install requirements.txt, and run train.py right away. Each of the 3 datasets now have download directions in the yaml (either a zipfile URL, or a bash command).

If it all works correctly you should see something like this:

[NanoCode012]

WARNING: Dataset not found, nonexistant paths: ['/coco128/images/train2017', '/coco128/images/train2017']
Attempting autodownload from: https://github.com/ultralytics/yolov5/releases/download/v1.0/coco128.zip
100%|███████████████████████████████████████| 21.1M/21.1M [01:56<00:00, 189kB/s]
Dataset autodownload success

It works! Very simple and nice! However, I see that you output “train2017” directory twice. Is there a reason? Is it meant to be “val2017”?

[glenn-jocher]

WARNING: Dataset not found, nonexistant paths: ['/coco128/images/train2017', '/coco128/images/train2017']
Attempting autodownload from: https://github.com/ultralytics/yolov5/releases/download/v1.0/coco128.zip
100%|███████████████████████████████████████| 21.1M/21.1M [01:56<00:00, 189kB/s]
Dataset autodownload success

It works! Very simple and nice! However, I see that you output “train2017” directory twice. Is there a reason? Is it meant to be “val2017”?

Ah, yes this is because coco128 trains and tests on the same 128 images. It's really meant as a sanity check to ensure your setup converges before trying larger datasets.