Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ERROR: Dataset 'custom-coco128.yaml' images not found ⚠️, missing paths #7

Open
Ritchie3 opened this issue Dec 6, 2024 · 1 comment

Comments

@Ritchie3
Copy link

Ritchie3 commented Dec 6, 2024

Hi,
I followed your instructions for the python sdk.

I created a compute, a data asset, an environment..
but during the actual job I get an error regarding the dataset path:

New https://pypi.org/project/ultralytics/8.3.43 available 😃 Update with 'pip install -U ultralytics'
Ultralytics YOLOv8.0.132 🚀 Python-3.10.9 torch-2.0.0 CUDA:0 (NVIDIA A100 80GB PCIe, 81051MiB)
�[34m�[1myolo/engine/trainer: �[0mtask=detect, mode=train, model=/mnt/azureml/cr/j/b208deb5042e467c890a4306131debe6/cap/data-capability/wd/INPUT_model_to_train/yolov8n.pt, data=custom-coco128.yaml, epochs=3, patience=50, batch=16, imgsz=640, save=True, save_period=-1, cache=False, device=None, workers=8, project=yolov8-experiment, name=experiment, exist_ok=False, pretrained=True, optimizer=auto, verbose=True, seed=0, deterministic=True, single_cls=False, rect=False, cos_lr=False, close_mosaic=10, resume=False, amp=True, fraction=1.0, profile=False, overlap_mask=True, mask_ratio=4, dropout=0.0, val=True, split=val, save_json=False, save_hybrid=False, conf=None, iou=0.7, max_det=300, half=False, dnn=False, plots=True, source=None, show=False, save_txt=False, save_conf=False, save_crop=False, show_labels=True, show_conf=True, vid_stride=1, line_width=None, visualize=False, augment=False, agnostic_nms=False, classes=None, retina_masks=False, boxes=True, format=torchscript, keras=False, optimize=False, int8=False, dynamic=False, simplify=False, opset=None, workspace=4, nms=False, lr0=0.01, lrf=0.01, momentum=0.937, weight_decay=0.0005, warmup_epochs=3.0, warmup_momentum=0.8, warmup_bias_lr=0.1, box=7.5, cls=0.5, dfl=1.5, pose=12.0, kobj=1.0, label_smoothing=0.0, nbs=64, hsv_h=0.015, hsv_s=0.7, hsv_v=0.4, degrees=0.0, translate=0.1, scale=0.5, shear=0.0, perspective=0.0, flipud=0.0, fliplr=0.5, mosaic=1.0, mixup=0.0, copy_paste=0.0, cfg=None, v5loader=False, tracker=botsort.yaml, save_dir=yolov8-experiment/experiment
Traceback (most recent call last):
  File "/opt/conda/lib/python3.10/site-packages/ultralytics/yolo/engine/trainer.py", line 123, in __init__
    self.data = check_det_dataset(self.args.data)
  File "/opt/conda/lib/python3.10/site-packages/ultralytics/yolo/data/utils.py", line 251, in check_det_dataset
    raise FileNotFoundError(m)
FileNotFoundError: 
Dataset 'custom-coco128.yaml' images not found ⚠️, missing paths ['/mnt/azureml/cr/j/b208deb5042e467c890a4306131debe6/cap/data-capability/wd/INPUT_training_data/images/train2017']
Note dataset download directory is '/mnt/azureml/cr/j/b208deb5042e467c890a4306131debe6/exe/wd/datasets'. You can update this in '/root/.config/Ultralytics/settings.yaml'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/bin/yolo", line 8, in <module>
    sys.exit(entrypoint())
  File "/opt/conda/lib/python3.10/site-packages/ultralytics/yolo/cfg/__init__.py", line 407, in entrypoint
    getattr(model, mode)(**overrides)  # default args from model
  File "/opt/conda/lib/python3.10/site-packages/ultralytics/yolo/engine/model.py", line 368, in train
    self.trainer = TASK_MAP[self.task][1](overrides=overrides, _callbacks=self.callbacks)
  File "/opt/conda/lib/python3.10/site-packages/ultralytics/yolo/engine/trainer.py", line 127, in __init__
    raise RuntimeError(emojis(f"Dataset '{clean_url(self.args.data)}' error ❌ {e}")) from e
RuntimeError: Dataset 'custom-coco128.yaml' error ❌ 
Dataset 'custom-coco128.yaml' images not found ⚠️, missing paths ['/mnt/azureml/cr/j/b208deb5042e467c890a4306131debe6/cap/data-capability/wd/INPUT_training_data/images/train2017']
Note dataset download directory is '/mnt/azureml/cr/j/b208deb5042e467c890a4306131debe6/exe/wd/datasets'. You can update this in '/root/.config/Ultralytics/settings.yaml'

I notice that the data asset consume example differs from your code.

from azure.ai.ml import MLClient, command, Input
from azure.ai.ml.constants import AssetTypes, InputOutputModes
from azure.identity import DefaultAzureCredential

ml_client = MLClient.from_config(credential=DefaultAzureCredential())

data_asset = ml_client.data.get("coco128", version="1")

# to successfully create a job, customize the parameters below based on your workspace resources
job = command(
        command='ls "${{inputs.data}}"',
        inputs={
            "data": Input(path=data_asset.id,
                type=AssetTypes.URI_FOLDER,
                mode=InputOutputModes.RO_MOUNT
            )
        },
        environment="azureml:AzureML-sklearn-1.0-ubuntu20.04-py38-cpu@latest"
      )
returned_job = ml_client.jobs.create_or_update(job)

What Am I doing wrong? Could you have a look?
Thanks,
Ritchie

@ouphi
Copy link
Owner

ouphi commented Dec 12, 2024

Hi @Ritchie3 ,
Looks like the link used to download the coco128.yaml does not work anymore because it has been moved in the ultralytics repo.
Can you try to update the command with the correct link?

The bash command was:

wget https://raw.githubusercontent.com/ultralytics/ultralytics/main/ultralytics/datasets/coco128.yaml -O training-code/custom-coco128.yaml

Now try to change it to:

wget https://raw.githubusercontent.com/ultralytics/ultralytics/7f37790134527e328c475ac292efdc39b1cd7b5e/ultralytics/cfg/datasets/coco128.yaml -O training-code/custom-coco128.yaml

You can verify that it worked by looking at the file training-code/custom-coco128.yaml
After running the command that delete the last line, the file should look like this:

# Ultralytics YOLO 🚀, AGPL-3.0 license
# COCO128 dataset https://www.kaggle.com/datasets/ultralytics/coco128 (first 128 images from COCO train2017) by Ultralytics
# Documentation: https://docs.ultralytics.com/datasets/detect/coco/
# Example usage: yolo train data=coco128.yaml
# parent
# ├── ultralytics
# └── datasets
#     └── coco128  ← downloads here (7 MB)

# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]
path: ../datasets/coco128 # dataset root dir
train: images/train2017 # train images (relative to 'path') 128 images
val: images/train2017 # val images (relative to 'path') 128 images
test: # test images (optional)

# Classes
names:
  0: person
  1: bicycle
  2: car
  3: motorcycle
  4: airplane
  5: bus
  6: train
  7: truck
  8: boat
  9: traffic light
  10: fire hydrant
  11: stop sign
  12: parking meter
  13: bench
  14: bird
  15: cat
  16: dog
  17: horse
  18: sheep
  19: cow
  20: elephant
  21: bear
  22: zebra
  23: giraffe
  24: backpack
  25: umbrella
  26: handbag
  27: tie
  28: suitcase
  29: frisbee
  30: skis
  31: snowboard
  32: sports ball
  33: kite
  34: baseball bat
  35: baseball glove
  36: skateboard
  37: surfboard
  38: tennis racket
  39: bottle
  40: wine glass
  41: cup
  42: fork
  43: knife
  44: spoon
  45: bowl
  46: banana
  47: apple
  48: sandwich
  49: orange
  50: broccoli
  51: carrot
  52: hot dog
  53: pizza
  54: donut
  55: cake
  56: chair
  57: couch
  58: potted plant
  59: bed
  60: dining table
  61: toilet
  62: tv
  63: laptop
  64: mouse
  65: remote
  66: keyboard
  67: cell phone
  68: microwave
  69: oven
  70: toaster
  71: sink
  72: refrigerator
  73: book
  74: clock
  75: vase
  76: scissors
  77: teddy bear
  78: hair drier
  79: toothbrush

# Download script/URL (optional)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants