Bounding Box Error when Retraining Model

[LJWsao]

Describe the bug
I used the documentation recommendations for training a new model. I was using this colab notebook: https://colab.research.google.com/drive/1gKUiocwfCvcvVfiKzAaf6voiUVL2KK_r?usp=sharing to retrain the DeepForest model using my own annotations I made using label-studio. I used the defaults settings and configuration and changed the paths to use my own annotations. About halfway through training, I go an error that said ValueError: y_max is less than or equal to y_min for bbox(0.81, 0.5075, 0.8175, 0.5705, 0). I did some digging and found it originated when image transformations are done in the prewritten dataset.py file line 119 that comes with the DeepForest package. Here is the full screenshot of the error:

To Reproduce
The xml file I used is on github here: https://github.com/LJWsao/tree_canopy_research/blob/main/6ddae292-northPast_PNG.xml
The image I used is here: https://drive.google.com/file/d/1qvygFrM8y_eI2JgbUvsIXJuhx3vNMfn_/view?usp=sharing

If you run the colab notebook using these files and the default code, you should be able to get the same results.
How can I can edit dataset.py getitem(self, idx) method to check if the bounding box is wider than one pixel before applying the transformations?

[bw4sz]

Thanks for your interest, you have a poorly formatted annotation matrix.

from deepforest.utilities import xml_to_annotations
annotations = xml_to_annotations(xml_path="/Users/benweinstein/Downloads/6ddae292-northPast_PNG.xml")
>>> annotations[annotations.ymin==annotations.ymax]
                   image_path  xmin  ymin  xmax  ymax label
7  6ddae292-northPast_PNG.png  2984  5143  2987  5143  Tree

Your bounding box has no width.

@henrykironde I assume this happens due to float rounding or in the original file

DeepForest/deepforest/utilities.py

Line 228 in 7de55c1

# set dtypes, check for floats and round

and we could add a warning there.

malformed = annotations[annotations.ymin==annotations.ymax]
if not malformed.empty:
    warnings.warn("{} annotations are invalid due to ymin == ymax, and are removed".format(malformed.shape[0])
annotations =  annotations[~(annotations.ymin==annotations.ymax)]

@LJWsao remove this annotation and close when ready.

[LJWsao]

I removed this annotation and was able to proceed with training successfully.

[AlexPapas]

I am getting the same error and my file does not have the same issue with @LJWsao

here is my dataset

Here the .fit

and below the full error
`

UserWarning: Got processor for bboxes, but no transform to process it.
ValueError: Expected x_max for bbox [0.93409175 0.85505736 1.0256677 0.950346 0. ] to be in the range [0.0, 1.0], got 1.025667667388916.

ValueError Traceback (most recent call last)
File , line 2
1 start_time = time.time()
----> 2 m.trainer.fit(m)
3 print(f"--- Training on CPU: {(time.time() - start_time):.2f} seconds ---")

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py:539, in Trainer.fit(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
537 self.state.status = TrainerStatus.RUNNING
538 self.training = True
--> 539 call._call_and_handle_interrupt(
540 self, self._fit_impl, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path
541 )

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py:47, in _call_and_handle_interrupt(trainer, trainer_fn, *args, **kwargs)
45 if trainer.strategy.launcher is not None:
46 return trainer.strategy.launcher.launch(trainer_fn, *args, trainer=trainer, **kwargs)
---> 47 return trainer_fn(*args, **kwargs)
49 except _TunerExitException:
50 _call_teardown_hook(trainer)

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py:575, in Trainer._fit_impl(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
568 assert self.state.fn is not None
569 ckpt_path = self._checkpoint_connector._select_ckpt_path(
570 self.state.fn,
571 ckpt_path,
572 model_provided=True,
573 model_connected=self.lightning_module is not None,
574 )
--> 575 self._run(model, ckpt_path=ckpt_path)
577 assert self.state.stopped
578 self.training = False

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py:982, in Trainer._run(self, model, ckpt_path)
977 self._signal_connector.register_signal_handlers()
979 # ----------------------------
980 # RUN THE TRAINER
981 # ----------------------------
--> 982 results = self._run_stage()
984 # ----------------------------
985 # POST-Training CLEAN UP
986 # ----------------------------
987 log.debug(f"{self.class.name}: trainer tearing down")

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py:1024, in Trainer._run_stage(self)
1022 if self.training:
1023 with isolate_rng():
-> 1024 self._run_sanity_check()
1025 with torch.autograd.set_detect_anomaly(self._detect_anomaly):
1026 self.fit_loop.run()

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py:1053, in Trainer._run_sanity_check(self)
1050 call._call_callback_hooks(self, "on_sanity_check_start")
1052 # run eval step
-> 1053 val_loop.run()
1055 call._call_callback_hooks(self, "on_sanity_check_end")
1057 # reset logger connector

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pytorch_lightning/loops/utilities.py:179, in _no_grad_context.._decorator(self, *args, **kwargs)
177 context_manager = torch.no_grad
178 with context_manager():
--> 179 return loop_run(self, *args, **kwargs)

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pytorch_lightning/loops/evaluation_loop.py:137, in _EvaluationLoop.run(self)
135 else:
136 dataloader_iter = None
--> 137 batch, batch_idx, dataloader_idx = next(data_fetcher)
138 if previous_dataloader_idx != dataloader_idx:
139 # the dataloader has changed, notify the logger connector
140 self._store_dataloader_outputs()

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pytorch_lightning/loops/fetchers.py:134, in _PrefetchDataFetcher.next(self)
131 self.done = not self.batches
132 elif not self.done:
133 # this will run only when no pre-fetching was done.
--> 134 batch = super().next()
135 else:
136 # the iterator is empty
137 raise StopIteration

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pytorch_lightning/loops/fetchers.py:61, in _DataFetcher.next(self)
59 self._start_profiler()
60 try:
---> 61 batch = next(self.iterator)
62 except StopIteration:
63 self.done = True

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pytorch_lightning/utilities/combined_loader.py:341, in CombinedLoader.next(self)
339 def next(self) -> _ITERATOR_RETURN:
340 assert self._iterator is not None
--> 341 out = next(self._iterator)
342 if isinstance(self._iterator, _Sequential):
343 return out

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pytorch_lightning/utilities/combined_loader.py:142, in _Sequential.next(self)
139 raise StopIteration
141 try:
--> 142 out = next(self.iterators[0])
143 except StopIteration:
144 # try the next iterator
145 self._use_next_iterator()

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/torch/utils/data/dataloader.py:630, in _BaseDataLoaderIter.next(self)
627 if self._sampler_iter is None:
628 # TODO(pytorch/pytorch#76750)
629 self._reset() # type: ignore[call-arg]
--> 630 data = self._next_data()
631 self._num_yielded += 1
632 if self._dataset_kind == _DatasetKind.Iterable and
633 self._IterableDataset_len_called is not None and
634 self._num_yielded > self._IterableDataset_len_called:

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/torch/utils/data/dataloader.py:673, in _SingleProcessDataLoaderIter._next_data(self)
671 def _next_data(self):
672 index = self._next_index() # may raise StopIteration
--> 673 data = self._dataset_fetcher.fetch(index) # may raise StopIteration
674 if self._pin_memory:
675 data = _utils.pin_memory.pin_memory(data, self._pin_memory_device)

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py:52, in _MapDatasetFetcher.fetch(self, possibly_batched_index)
50 data = self.dataset.getitems(possibly_batched_index)
51 else:
---> 52 data = [self.dataset[idx] for idx in possibly_batched_index]
53 else:
54 data = self.dataset[possibly_batched_index]

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py:52, in (.0)
50 data = self.dataset.getitems(possibly_batched_index)
51 else:
---> 52 data = [self.dataset[idx] for idx in possibly_batched_index]
53 else:
54 data = self.dataset[possibly_batched_index]

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/deepforest/dataset.py:126, in TreeDataset.getitem(self, idx)
122 targets = {"boxes": boxes, "labels": labels}
124 return self.image_names[idx], image, targets
--> 126 augmented = self.transform(image=image,
127 bboxes=targets["boxes"],
128 category_ids=targets["labels"].astype(np.int64))
129 image = augmented["image"]
131 boxes = np.array(augmented["bboxes"])

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/albumentations/core/composition.py:475, in Compose.call(self, force_apply, *args, **data)
472 if not need_to_run:
473 return data
--> 475 self.preprocess(data)
477 for t in self.transforms:
478 data = t(**data)

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/albumentations/core/composition.py:515, in Compose.preprocess(self, data)
512 if self.strict:
513 self._validate_data(data)
--> 515 self._preprocess_processors(data)
516 self._preprocess_arrays(data)

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/albumentations/core/composition.py:542, in Compose._preprocess_processors(self, data)
540 processor.ensure_data_valid(data)
541 for processor in self.processors.values():
--> 542 processor.preprocess(data)

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/albumentations/core/utils.py:155, in DataProcessor.preprocess(self, data)
153 data = self.add_label_fields_to_data(data)
154 for data_name in set(self.data_fields) & set(data.keys()):
--> 155 data[data_name] = self.check_and_convert(data[data_name], shape, direction="to")

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/albumentations/core/bbox_utils.py:306, in BboxProcessor.check_and_convert(self, data, shape, direction)
296 converted_data = filter_bboxes(
297 converted_data,
298 shape,
(...)
302 min_height=0,
303 )
305 # Finally check the remaining boxes
--> 306 self.check(converted_data, shape)
307 return converted_data
308 self.check(data, shape)

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/albumentations/core/bbox_utils.py:314, in BboxProcessor.check(self, data, shape)
313 def check(self, data: np.ndarray, shape: ShapeType) -> None:
--> 314 check_bboxes(data)

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/albumentations/augmentations/utils.py:189, in handle_empty_array..decorator..wrapper(*args, **kwargs)
187 if len(array) == 0:
188 return array
--> 189 return func(*args, **kwargs)

File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/albumentations/core/bbox_utils.py:552, in check_bboxes(bboxes)
550 invalid_coord = ["x_min", "y_min", "x_max", "y_max"][np.where(~valid_range[invalid_idx])[0][0]]
551 invalid_value = invalid_bbox[np.where(~valid_range[invalid_idx])[0][0]]
--> 552 raise ValueError(
553 f"Expected {invalid_coord} for bbox {invalid_bbox} to be in the range [0.0, 1.0], got {invalid_value}.",
554 )
556 # Check if x_max > x_min and y_max > y_min
557 valid_order = (bboxes[:, 2] > bboxes[:, 0]) & (bboxes[:, 3] > bboxes[:, 1])

ValueError: Expected x_max for bbox [0.93409175 0.85505736 1.0256677 0.950346 0. ] to be in the range [0.0, 1.0], got 1.025667667388916.`

[ethanwhite]

Hi @AlexPapas - thanks for reporting this issue. Since it's a different cause can you open a new issue with the same information you posted here so that we can dig into it. Also, if it's possible to share the fully annotations file that will help us debug things.