make val split ratio configurable #760
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
samet-akcay
requested changes
Dec 5, 2022
There was a problem hiding this comment.
Thanks. Only a minor comment in the code base.
More general comment: Are these changes in line with #713? Do we need to change anything to get this work properly for the new CLI?
These parameters are passed directly to the constructors of the datamodule classes, so I don't foresee any problems with the new CLI, though admittedly I'm not 100% clear on the requirements of the CLI design. |
samet-akcay
approved these changes
Dec 5, 2022
djdameln
added a commit
that referenced
this pull request
Jan 6, 2023
* New datamodules design (#572) * move sample generation to datamodule instead of dataset * move sample generation from init to setup * remove inference stage and add base classes * replace dataset classes with AnomalibDataset * move setup to base class, create samples as class method * update docstrings * refactor btech to new format * allow training with no anomalous data * remove MVTec name from comment * raise NotImplementedError in base class * allow both png and bmp images for btech * use label_index to check if dataset contains anomalous images * refactor getitem in dataset class * use iloc for indexing * move dataloader getters to base class * refactor to add validate stage in setup * implement alternative datamodules solution * small improvements * improve design * remove unused constructor arguments * adapt btech to new design * add prepare_data method for mvtec * implement more generic random splitting function * update docstrings for folder module * ensure type consistency when performing operations on dataset * change imports * change variable names * replace pass with NotImplementedError * allow training on folder without test images * use relative path for normal_test_dir * fix dataset tests * update validation set parameter in configs * change default argument * use setter for samples * hint options for val_split_mode * update assert message and docstring * revert name change dataset vs datamodule * typing and docstrings * remove samples argument from dataset constructor * val/test -> eval * remove Split.Full from enum * sort samples when setting * update warn message * formatting * use setter when creating samples in dataset classes * add tests for new dataset class * add test case for label aware random split * update parameter name in inferencers * move _setup implementation to base class * address codacy issues * fix pylint issues * codacy * update example dataset config in docs * fix test * move base classes to separate files (avoid circular import) * add base classes * update docstring * fix imports * validation_split_mode -> val_split_mode * update docs * Update anomalib/data/base/dataset.py Co-authored-by: Joao P C Bertoldo <[email protected]> * get length from self.samples * assert unique indices * check is_setup for individual datasets Co-authored-by: Joao P C Bertoldo <[email protected]> * remove assert in __getitem_\ Co-authored-by: Joao P C Bertoldo <[email protected]> * Update anomalib/data/btech.py Co-authored-by: Joao P C Bertoldo <[email protected]> * clearer assert message * clarify list inversion in comment * comments and typing * validate contents of samples dataframe before setting * add file paths check * add seed to random_split function * fix expected columns * fix typo * add seed parameter to datamodules * set global seed in test entrypoint * add NONE option to valsplitmode * clarify setup behaviour in docstring * fix typo Co-authored-by: Joao P C Bertoldo <[email protected]> Co-authored-by: Joao P C Bertoldo <[email protected]> * Video Datamodules (#676) * move sample generation to datamodule instead of dataset * move sample generation from init to setup * remove inference stage and add base classes * replace dataset classes with AnomalibDataset * move setup to base class, create samples as class method * update docstrings * refactor btech to new format * allow training with no anomalous data * remove MVTec name from comment * raise NotImplementedError in base class * allow both png and bmp images for btech * use label_index to check if dataset contains anomalous images * refactor getitem in dataset class * use iloc for indexing * move dataloader getters to base class * refactor to add validate stage in setup * implement alternative datamodules solution * small improvements * improve design * remove unused constructor arguments * adapt btech to new design * add prepare_data method for mvtec * implement more generic random splitting function * update docstrings for folder module * ensure type consistency when performing operations on dataset * change imports * change variable names * replace pass with NotImplementedError * allow training on folder without test images * use relative path for normal_test_dir * fix dataset tests * update validation set parameter in configs * change default argument * use setter for samples * hint options for val_split_mode * update assert message and docstring * revert name change dataset vs datamodule * typing and docstrings * remove samples argument from dataset constructor * val/test -> eval * remove Split.Full from enum * sort samples when setting * update warn message * formatting * use setter when creating samples in dataset classes * add tests for new dataset class * add test case for label aware random split * update parameter name in inferencers * move _setup implementation to base class * address codacy issues * fix pylint issues * codacy * update example dataset config in docs * fix test * move base classes to separate files (avoid circular import) * add base classes * update docstring * fix imports * validation_split_mode -> val_split_mode * update docs * Update anomalib/data/base/dataset.py Co-authored-by: Joao P C Bertoldo <[email protected]> * get length from self.samples * assert unique indices * check is_setup for individual datasets Co-authored-by: Joao P C Bertoldo <[email protected]> * remove assert in __getitem_\ Co-authored-by: Joao P C Bertoldo <[email protected]> * Update anomalib/data/btech.py Co-authored-by: Joao P C Bertoldo <[email protected]> * clearer assert message * clarify list inversion in comment * comments and typing * validate contents of samples dataframe before setting * add file paths check * add seed to random_split function * fix expected columns * fix typo * add pedestrian and avenue datasets and video utils * add seed parameter to datamodules * set global seed in test entrypoint * add NONE option to valsplitmode * clarify setup behaviour in docstring * add basic visualization for video datasets * simplify ucsdped implementation * add ucsd and avenue to __all__ * add default value for task * add tests for ucsd and avenue * add tests for video dataset and utils * add download info for avenue dataset * add download info for ucsd pedestrian dataset * more consistent naming * fix path to masks folder in gt dir * pass original image in batch to facilitate visualization * convert mask files for avenue * suppress warning due to torchvision bug * fix bug in avenue masks * store visualizations for each video in separate folder * rename parameters * add warning for clip_length > 1 * fix dataset tests * fix labels tensor shape bug * add pyav to requirements * add description for avenue dataset * use pathlib * Update anomalib/data/avenue.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/avenue.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/utils/video.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/base/video.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/base/video.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/ucsd_ped.py Co-authored-by: Samet Akcay <[email protected]> * import video dataset from base * fix bug when collecting ucsd samples * clean up datamodules tests * fix tests * remove redundant test cases * retrieve masks as numpy array * use pathlib * variable name * pathlib * use preprocesser from arguments * fix indexing bug Co-authored-by: Joao P C Bertoldo <[email protected]> Co-authored-by: Samet Akcay <[email protected]> * Update lightning_inference.py * Make `val split ratio` configurable (#760) * make val split ratio configurable * use DeprecationWarning, update config key * Add support for Detection task type (#732) * add basic support for detection task * use enum for task type * formatting * small bugfix * add unit tests for bounding box conversion * update error message * use as_tensor * typing and docstring * explicit keyword arguments * simplify bbox handling in video dataset * docstring consistency * add missing licenses * add whitespace for readability * add missing license * Update anomalib/data/utils/boxes.py Co-authored-by: Samet Akcay <[email protected]> * Revert "Update anomalib/data/utils/boxes.py" This reverts commit cec6138. * add test case for custom collate function * docstring * add integration tests for detection dataloading * extend and clean up datamodules tests * add detection task type to visualizer tests * only show pred_boxes during inference * add detection support for torch inference * add detection support for openvino inference * test inference for all task types * pylint Co-authored-by: Samet Akcay <[email protected]> * [Datamodules] Update deprecation messages (#764) * update deprecation messages * raise warnings as DeprecationWarning * Improve image source parsing for Folder dataset (#784) * mask -> mask_dir * properly handle absolute and relative paths * make root path parameter optional * formatting * path -> root * update docs * remove options hint for name parameter * refactor function * Update anomalib/config/config.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/config/config.py Co-authored-by: Samet Akcay <[email protected]> * make root and abnormal_dir optional * Update anomalib/data/folder.py Co-authored-by: Samet Akcay <[email protected]> Co-authored-by: Samet Akcay <[email protected]> * Synthetic anomaly for testing and validation (#634) * move sample generation to datamodule instead of dataset * move sample generation from init to setup * remove inference stage and add base classes * replace dataset classes with AnomalibDataset * move setup to base class, create samples as class method * update docstrings * refactor btech to new format * allow training with no anomalous data * remove MVTec name from comment * raise NotImplementedError in base class * allow both png and bmp images for btech * use label_index to check if dataset contains anomalous images * refactor getitem in dataset class * use iloc for indexing * move dataloader getters to base class * refactor to add validate stage in setup * implement alternative datamodules solution * small improvements * improve design * remove unused constructor arguments * adapt btech to new design * add prepare_data method for mvtec * implement more generic random splitting function * update docstrings for folder module * ensure type consistency when performing operations on dataset * change imports * change variable names * replace pass with NotImplementedError * allow training on folder without test images * use relative path for normal_test_dir * fix dataset tests * update validation set parameter in configs * change default argument * use setter for samples * hint options for val_split_mode * update assert message and docstring * revert name change dataset vs datamodule * typing and docstrings * remove samples argument from dataset constructor * val/test -> eval * remove Split.Full from enum * sort samples when setting * update warn message * formatting * use setter when creating samples in dataset classes * add tests for new dataset class * add test case for label aware random split * update parameter name in inferencers * move _setup implementation to base class * address codacy issues * fix pylint issues * codacy * update example dataset config in docs * fix test * move base classes to separate files (avoid circular import) * add synthetic dataset class * move augmenter to data directory * add base classes * update docstring * use synthetic dataset in base datamodule * fix imports * clean up synthetic anomaly dataset implementation * fix mistake in augmenter * change default split ratio * remove accidentally added file * validation_split_mode -> val_split_mode * update docs * Update anomalib/data/base/dataset.py Co-authored-by: Joao P C Bertoldo <[email protected]> * get length from self.samples * assert unique indices * check is_setup for individual datasets Co-authored-by: Joao P C Bertoldo <[email protected]> * remove assert in __getitem_\ Co-authored-by: Joao P C Bertoldo <[email protected]> * Update anomalib/data/btech.py Co-authored-by: Joao P C Bertoldo <[email protected]> * clearer assert message * clarify list inversion in comment * comments and typing * validate contents of samples dataframe before setting * add file paths check * add seed to random_split function * fix expected columns * fix typo * add seed parameter to datamodules * set global seed in test entrypoint * add NONE option to valsplitmode * clarify setup behaviour in docstring * add logging message * use val_split_ratio for synthetic validation set * pathlib * make synthetic anomaly available for test set * update configs * add tests * simplify test set splitting logic * update docstring * add missing licence * split_normal_and_anomalous -> split_by_label * VideoAnomalib -> AnomalibVideo Co-authored-by: Joao P C Bertoldo <[email protected]> * Bugfixes for Datamodules feature branch (#800) * properly handle NoneType mask_dir and add test case * fix wrong deprecation handling * Deprecate PreProcessor (#795) * deprecate PreProcessor * update configs * update deprecation messages * update video dataset * update inference dataset * move transforms to data module * update and extend transform tests * fix cyclic import * add validity checks for image size and center crop * pass image size as tuple * update path to get_transforms * update error message * fix center crop tuple conversion * update inferencers * remove draem transform config * update changelog * fix cyclic import * add crop size vs image size check * improve readability * mypy * use enum to configure input normalization * update lightning inference * update inference dataset * [Datamodules] Fix bug in bbox score to image score conversion (#803) handle empty box predictions * Improve handling of `test_split_mode='none'` and `val_split_mode='none'` (#801) * enable none as split mode * use get to retrieve config keys * update deprecation message and config key * fix to float transform * Detection improvements (#820) * apply pixel threshold to bbox detections * allow visualizing normal boxes * normalize box scores * fix bbox logic in base anomaly module * boxes_scores -> box_scores * fix inferencers * update changelog * update csflow config to new format * remove unused imports * line length * suppress bandit warnings * use torch rng in augmenter * use tuple instead of list * add missing params to dosctring * add missing licence information * COLS -> COLUMNS * typing and variable naming * remove duplicate parameter in docstring * im_dir -> image_dir * typing and docstring * typing * ValSplitMode -> ValidationSplitMode * add missing licence * rename variable * remove empty comment * remove unused class attribute * [Detection] Compute box score when generating boxes from masks (#828) * infer box scores from anomaly maps * discard single pixel boxes * revert discard single pixel boxes * add test case for bbox scores * update torch inferencer * minor refactor * revert val_split_mode -> validation_split_mode Co-authored-by: Joao P C Bertoldo <[email protected]> Co-authored-by: Samet Akcay <[email protected]>
djdameln
added a commit
that referenced
this pull request
Jan 6, 2023
* New datamodules design (#572) * move sample generation to datamodule instead of dataset * move sample generation from init to setup * remove inference stage and add base classes * replace dataset classes with AnomalibDataset * move setup to base class, create samples as class method * update docstrings * refactor btech to new format * allow training with no anomalous data * remove MVTec name from comment * raise NotImplementedError in base class * allow both png and bmp images for btech * use label_index to check if dataset contains anomalous images * refactor getitem in dataset class * use iloc for indexing * move dataloader getters to base class * refactor to add validate stage in setup * implement alternative datamodules solution * small improvements * improve design * remove unused constructor arguments * adapt btech to new design * add prepare_data method for mvtec * implement more generic random splitting function * update docstrings for folder module * ensure type consistency when performing operations on dataset * change imports * change variable names * replace pass with NotImplementedError * allow training on folder without test images * use relative path for normal_test_dir * fix dataset tests * update validation set parameter in configs * change default argument * use setter for samples * hint options for val_split_mode * update assert message and docstring * revert name change dataset vs datamodule * typing and docstrings * remove samples argument from dataset constructor * val/test -> eval * remove Split.Full from enum * sort samples when setting * update warn message * formatting * use setter when creating samples in dataset classes * add tests for new dataset class * add test case for label aware random split * update parameter name in inferencers * move _setup implementation to base class * address codacy issues * fix pylint issues * codacy * update example dataset config in docs * fix test * move base classes to separate files (avoid circular import) * add base classes * update docstring * fix imports * validation_split_mode -> val_split_mode * update docs * Update anomalib/data/base/dataset.py Co-authored-by: Joao P C Bertoldo <[email protected]> * get length from self.samples * assert unique indices * check is_setup for individual datasets Co-authored-by: Joao P C Bertoldo <[email protected]> * remove assert in __getitem_\ Co-authored-by: Joao P C Bertoldo <[email protected]> * Update anomalib/data/btech.py Co-authored-by: Joao P C Bertoldo <[email protected]> * clearer assert message * clarify list inversion in comment * comments and typing * validate contents of samples dataframe before setting * add file paths check * add seed to random_split function * fix expected columns * fix typo * add seed parameter to datamodules * set global seed in test entrypoint * add NONE option to valsplitmode * clarify setup behaviour in docstring * fix typo Co-authored-by: Joao P C Bertoldo <[email protected]> Co-authored-by: Joao P C Bertoldo <[email protected]> * Video Datamodules (#676) * move sample generation to datamodule instead of dataset * move sample generation from init to setup * remove inference stage and add base classes * replace dataset classes with AnomalibDataset * move setup to base class, create samples as class method * update docstrings * refactor btech to new format * allow training with no anomalous data * remove MVTec name from comment * raise NotImplementedError in base class * allow both png and bmp images for btech * use label_index to check if dataset contains anomalous images * refactor getitem in dataset class * use iloc for indexing * move dataloader getters to base class * refactor to add validate stage in setup * implement alternative datamodules solution * small improvements * improve design * remove unused constructor arguments * adapt btech to new design * add prepare_data method for mvtec * implement more generic random splitting function * update docstrings for folder module * ensure type consistency when performing operations on dataset * change imports * change variable names * replace pass with NotImplementedError * allow training on folder without test images * use relative path for normal_test_dir * fix dataset tests * update validation set parameter in configs * change default argument * use setter for samples * hint options for val_split_mode * update assert message and docstring * revert name change dataset vs datamodule * typing and docstrings * remove samples argument from dataset constructor * val/test -> eval * remove Split.Full from enum * sort samples when setting * update warn message * formatting * use setter when creating samples in dataset classes * add tests for new dataset class * add test case for label aware random split * update parameter name in inferencers * move _setup implementation to base class * address codacy issues * fix pylint issues * codacy * update example dataset config in docs * fix test * move base classes to separate files (avoid circular import) * add base classes * update docstring * fix imports * validation_split_mode -> val_split_mode * update docs * Update anomalib/data/base/dataset.py Co-authored-by: Joao P C Bertoldo <[email protected]> * get length from self.samples * assert unique indices * check is_setup for individual datasets Co-authored-by: Joao P C Bertoldo <[email protected]> * remove assert in __getitem_\ Co-authored-by: Joao P C Bertoldo <[email protected]> * Update anomalib/data/btech.py Co-authored-by: Joao P C Bertoldo <[email protected]> * clearer assert message * clarify list inversion in comment * comments and typing * validate contents of samples dataframe before setting * add file paths check * add seed to random_split function * fix expected columns * fix typo * add pedestrian and avenue datasets and video utils * add seed parameter to datamodules * set global seed in test entrypoint * add NONE option to valsplitmode * clarify setup behaviour in docstring * add basic visualization for video datasets * simplify ucsdped implementation * add ucsd and avenue to __all__ * add default value for task * add tests for ucsd and avenue * add tests for video dataset and utils * add download info for avenue dataset * add download info for ucsd pedestrian dataset * more consistent naming * fix path to masks folder in gt dir * pass original image in batch to facilitate visualization * convert mask files for avenue * suppress warning due to torchvision bug * fix bug in avenue masks * store visualizations for each video in separate folder * rename parameters * add warning for clip_length > 1 * fix dataset tests * fix labels tensor shape bug * add pyav to requirements * add description for avenue dataset * use pathlib * Update anomalib/data/avenue.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/avenue.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/utils/video.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/base/video.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/base/video.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/ucsd_ped.py Co-authored-by: Samet Akcay <[email protected]> * import video dataset from base * fix bug when collecting ucsd samples * clean up datamodules tests * fix tests * remove redundant test cases * retrieve masks as numpy array * use pathlib * variable name * pathlib * use preprocesser from arguments * fix indexing bug Co-authored-by: Joao P C Bertoldo <[email protected]> Co-authored-by: Samet Akcay <[email protected]> * Update lightning_inference.py * Make `val split ratio` configurable (#760) * make val split ratio configurable * use DeprecationWarning, update config key * Add support for Detection task type (#732) * add basic support for detection task * use enum for task type * formatting * small bugfix * add unit tests for bounding box conversion * update error message * use as_tensor * typing and docstring * explicit keyword arguments * simplify bbox handling in video dataset * docstring consistency * add missing licenses * add whitespace for readability * add missing license * Update anomalib/data/utils/boxes.py Co-authored-by: Samet Akcay <[email protected]> * Revert "Update anomalib/data/utils/boxes.py" This reverts commit cec6138. * add test case for custom collate function * docstring * add integration tests for detection dataloading * extend and clean up datamodules tests * add detection task type to visualizer tests * only show pred_boxes during inference * add detection support for torch inference * add detection support for openvino inference * test inference for all task types * pylint Co-authored-by: Samet Akcay <[email protected]> * [Datamodules] Update deprecation messages (#764) * update deprecation messages * raise warnings as DeprecationWarning * Improve image source parsing for Folder dataset (#784) * mask -> mask_dir * properly handle absolute and relative paths * make root path parameter optional * formatting * path -> root * update docs * remove options hint for name parameter * refactor function * Update anomalib/config/config.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/config/config.py Co-authored-by: Samet Akcay <[email protected]> * make root and abnormal_dir optional * Update anomalib/data/folder.py Co-authored-by: Samet Akcay <[email protected]> Co-authored-by: Samet Akcay <[email protected]> * Synthetic anomaly for testing and validation (#634) * move sample generation to datamodule instead of dataset * move sample generation from init to setup * remove inference stage and add base classes * replace dataset classes with AnomalibDataset * move setup to base class, create samples as class method * update docstrings * refactor btech to new format * allow training with no anomalous data * remove MVTec name from comment * raise NotImplementedError in base class * allow both png and bmp images for btech * use label_index to check if dataset contains anomalous images * refactor getitem in dataset class * use iloc for indexing * move dataloader getters to base class * refactor to add validate stage in setup * implement alternative datamodules solution * small improvements * improve design * remove unused constructor arguments * adapt btech to new design * add prepare_data method for mvtec * implement more generic random splitting function * update docstrings for folder module * ensure type consistency when performing operations on dataset * change imports * change variable names * replace pass with NotImplementedError * allow training on folder without test images * use relative path for normal_test_dir * fix dataset tests * update validation set parameter in configs * change default argument * use setter for samples * hint options for val_split_mode * update assert message and docstring * revert name change dataset vs datamodule * typing and docstrings * remove samples argument from dataset constructor * val/test -> eval * remove Split.Full from enum * sort samples when setting * update warn message * formatting * use setter when creating samples in dataset classes * add tests for new dataset class * add test case for label aware random split * update parameter name in inferencers * move _setup implementation to base class * address codacy issues * fix pylint issues * codacy * update example dataset config in docs * fix test * move base classes to separate files (avoid circular import) * add synthetic dataset class * move augmenter to data directory * add base classes * update docstring * use synthetic dataset in base datamodule * fix imports * clean up synthetic anomaly dataset implementation * fix mistake in augmenter * change default split ratio * remove accidentally added file * validation_split_mode -> val_split_mode * update docs * Update anomalib/data/base/dataset.py Co-authored-by: Joao P C Bertoldo <[email protected]> * get length from self.samples * assert unique indices * check is_setup for individual datasets Co-authored-by: Joao P C Bertoldo <[email protected]> * remove assert in __getitem_\ Co-authored-by: Joao P C Bertoldo <[email protected]> * Update anomalib/data/btech.py Co-authored-by: Joao P C Bertoldo <[email protected]> * clearer assert message * clarify list inversion in comment * comments and typing * validate contents of samples dataframe before setting * add file paths check * add seed to random_split function * fix expected columns * fix typo * add seed parameter to datamodules * set global seed in test entrypoint * add NONE option to valsplitmode * clarify setup behaviour in docstring * add logging message * use val_split_ratio for synthetic validation set * pathlib * make synthetic anomaly available for test set * update configs * add tests * simplify test set splitting logic * update docstring * add missing licence * split_normal_and_anomalous -> split_by_label * VideoAnomalib -> AnomalibVideo Co-authored-by: Joao P C Bertoldo <[email protected]> * Bugfixes for Datamodules feature branch (#800) * properly handle NoneType mask_dir and add test case * fix wrong deprecation handling * Deprecate PreProcessor (#795) * deprecate PreProcessor * update configs * update deprecation messages * update video dataset * update inference dataset * move transforms to data module * update and extend transform tests * fix cyclic import * add validity checks for image size and center crop * pass image size as tuple * update path to get_transforms * update error message * fix center crop tuple conversion * update inferencers * remove draem transform config * update changelog * fix cyclic import * add crop size vs image size check * improve readability * mypy * use enum to configure input normalization * update lightning inference * update inference dataset * [Datamodules] Fix bug in bbox score to image score conversion (#803) handle empty box predictions * Improve handling of `test_split_mode='none'` and `val_split_mode='none'` (#801) * enable none as split mode * use get to retrieve config keys * update deprecation message and config key * fix to float transform * Detection improvements (#820) * apply pixel threshold to bbox detections * allow visualizing normal boxes * normalize box scores * fix bbox logic in base anomaly module * boxes_scores -> box_scores * fix inferencers * update changelog * update csflow config to new format * remove unused imports * line length * refactor make_mvtec_dataset to improve flexibility * add visa dataset * move download and extract functionality to shared location * move visa subset splitting to separate method * update changelog * add tests for visa dataset * suppress bandit url warning * update test * address PR comments * suppress bandit warnings * use torch rng in augmenter * fix logic in prepare_data * add comments * cleaner zipfile import * address PR comments * use tuple instead of list * add missing params to dosctring * add missing licence information * COLS -> COLUMNS * typing and variable naming * remove duplicate parameter in docstring * im_dir -> image_dir * typing and docstring * typing * ValSplitMode -> ValidationSplitMode * add missing licence * rename variable * remove empty comment * remove unused class attribute * [Detection] Compute box score when generating boxes from masks (#828) * infer box scores from anomaly maps * discard single pixel boxes * revert discard single pixel boxes * add test case for bbox scores * update torch inferencer * minor refactor * revert val_split_mode -> validation_split_mode * use empty string instead of nan as empty mask path * typing Co-authored-by: Joao P C Bertoldo <[email protected]> Co-authored-by: Samet Akcay <[email protected]>
djdameln
added a commit
that referenced
this pull request
Jan 6, 2023
* fix pylint issues * codacy * update example dataset config in docs * fix test * move base classes to separate files (avoid circular import) * add base classes * update docstring * fix imports * validation_split_mode -> val_split_mode * update docs * Update anomalib/data/base/dataset.py Co-authored-by: Joao P C Bertoldo <[email protected]> * get length from self.samples * assert unique indices * check is_setup for individual datasets Co-authored-by: Joao P C Bertoldo <[email protected]> * remove assert in __getitem_\ Co-authored-by: Joao P C Bertoldo <[email protected]> * Update anomalib/data/btech.py Co-authored-by: Joao P C Bertoldo <[email protected]> * clearer assert message * clarify list inversion in comment * comments and typing * validate contents of samples dataframe before setting * add file paths check * add seed to random_split function * fix expected columns * fix typo * add pedestrian and avenue datasets and video utils * add seed parameter to datamodules * set global seed in test entrypoint * add NONE option to valsplitmode * clarify setup behaviour in docstring * Created rbad directory * Keep refactoring region-extractor * rename new_image_sizes to transformed_image_sizes * Renamed the variables in region extractor * post-process function in region extractor * Refactored tile-boxes function * Added feature extractor * Add main.py * Added feature extractor to tests * Update the jupyter notebook * Uncomment loa weights from region.py * Add feature and region extractors * Finished feature-extractor implementation * Rename the algo as rkde * New datamodules design (#572) * move sample generation to datamodule instead of dataset * move sample generation from init to setup * remove inference stage and add base classes * replace dataset classes with AnomalibDataset * move setup to base class, create samples as class method * update docstrings * refactor btech to new format * allow training with no anomalous data * remove MVTec name from comment * raise NotImplementedError in base class * allow both png and bmp images for btech * use label_index to check if dataset contains anomalous images * refactor getitem in dataset class * use iloc for indexing * move dataloader getters to base class * refactor to add validate stage in setup * implement alternative datamodules solution * small improvements * improve design * remove unused constructor arguments * adapt btech to new design * add prepare_data method for mvtec * implement more generic random splitting function * update docstrings for folder module * ensure type consistency when performing operations on dataset * change imports * change variable names * replace pass with NotImplementedError * allow training on folder without test images * use relative path for normal_test_dir * fix dataset tests * update validation set parameter in configs * change default argument * use setter for samples * hint options for val_split_mode * update assert message and docstring * revert name change dataset vs datamodule * typing and docstrings * remove samples argument from dataset constructor * val/test -> eval * remove Split.Full from enum * sort samples when setting * update warn message * formatting * use setter when creating samples in dataset classes * add tests for new dataset class * add test case for label aware random split * update parameter name in inferencers * move _setup implementation to base class * address codacy issues * fix pylint issues * codacy * update example dataset config in docs * fix test * move base classes to separate files (avoid circular import) * add base classes * update docstring * fix imports * validation_split_mode -> val_split_mode * update docs * Update anomalib/data/base/dataset.py Co-authored-by: Joao P C Bertoldo <[email protected]> * get length from self.samples * assert unique indices * check is_setup for individual datasets Co-authored-by: Joao P C Bertoldo <[email protected]> * remove assert in __getitem_\ Co-authored-by: Joao P C Bertoldo <[email protected]> * Update anomalib/data/btech.py Co-authored-by: Joao P C Bertoldo <[email protected]> * clearer assert message * clarify list inversion in comment * comments and typing * validate contents of samples dataframe before setting * add file paths check * add seed to random_split function * fix expected columns * fix typo * add seed parameter to datamodules * set global seed in test entrypoint * add NONE option to valsplitmode * clarify setup behaviour in docstring * fix typo Co-authored-by: Joao P C Bertoldo <[email protected]> Co-authored-by: Joao P C Bertoldo <[email protected]> * add basic visualization for video datasets * simplify ucsdped implementation * TODO: Investigate torch_model * add ucsd and avenue to __all__ * add default value for task * add tests for ucsd and avenue * add tests for video dataset and utils * add download info for avenue dataset * add download info for ucsd pedestrian dataset * more consistent naming * fix path to masks folder in gt dir * pass original image in batch to facilitate visualization * convert mask files for avenue * suppress warning due to torchvision bug * fix bug in avenue masks * store visualizations for each video in separate folder * rename parameters * add warning for clip_length > 1 * fix dataset tests * fix labels tensor shape bug * add pyav to requirements * Add TODO notes * add todo notes * add description for avenue dataset * use pathlib * Update anomalib/data/avenue.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/avenue.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/utils/video.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/base/video.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/base/video.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/ucsd_ped.py Co-authored-by: Samet Akcay <[email protected]> * import video dataset from base * fix bug when collecting ucsd samples * clean up datamodules tests * fix tests * remove redundant test cases * add test case for normality model * retrieve masks as numpy array * use pathlib * variable name * pathlib * use preprocesser from arguments * fix indexing bug * Video Datamodules (#676) * move sample generation to datamodule instead of dataset * move sample generation from init to setup * remove inference stage and add base classes * replace dataset classes with AnomalibDataset * move setup to base class, create samples as class method * update docstrings * refactor btech to new format * allow training with no anomalous data * remove MVTec name from comment * raise NotImplementedError in base class * allow both png and bmp images for btech * use label_index to check if dataset contains anomalous images * refactor getitem in dataset class * use iloc for indexing * move dataloader getters to base class * refactor to add validate stage in setup * implement alternative datamodules solution * small improvements * improve design * remove unused constructor arguments * adapt btech to new design * add prepare_data method for mvtec * implement more generic random splitting function * update docstrings for folder module * ensure type consistency when performing operations on dataset * change imports * change variable names * replace pass with NotImplementedError * allow training on folder without test images * use relative path for normal_test_dir * fix dataset tests * update validation set parameter in configs * change default argument * use setter for samples * hint options for val_split_mode * update assert message and docstring * revert name change dataset vs datamodule * typing and docstrings * remove samples argument from dataset constructor * val/test -> eval * remove Split.Full from enum * sort samples when setting * update warn message * formatting * use setter when creating samples in dataset classes * add tests for new dataset class * add test case for label aware random split * update parameter name in inferencers * move _setup implementation to base class * address codacy issues * fix pylint issues * codacy * update example dataset config in docs * fix test * move base classes to separate files (avoid circular import) * add base classes * update docstring * fix imports * validation_split_mode -> val_split_mode * update docs * Update anomalib/data/base/dataset.py Co-authored-by: Joao P C Bertoldo <[email protected]> * get length from self.samples * assert unique indices * check is_setup for individual datasets Co-authored-by: Joao P C Bertoldo <[email protected]> * remove assert in __getitem_\ Co-authored-by: Joao P C Bertoldo <[email protected]> * Update anomalib/data/btech.py Co-authored-by: Joao P C Bertoldo <[email protected]> * clearer assert message * clarify list inversion in comment * comments and typing * validate contents of samples dataframe before setting * add file paths check * add seed to random_split function * fix expected columns * fix typo * add pedestrian and avenue datasets and video utils * add seed parameter to datamodules * set global seed in test entrypoint * add NONE option to valsplitmode * clarify setup behaviour in docstring * add basic visualization for video datasets * simplify ucsdped implementation * add ucsd and avenue to __all__ * add default value for task * add tests for ucsd and avenue * add tests for video dataset and utils * add download info for avenue dataset * add download info for ucsd pedestrian dataset * more consistent naming * fix path to masks folder in gt dir * pass original image in batch to facilitate visualization * convert mask files for avenue * suppress warning due to torchvision bug * fix bug in avenue masks * store visualizations for each video in separate folder * rename parameters * add warning for clip_length > 1 * fix dataset tests * fix labels tensor shape bug * add pyav to requirements * add description for avenue dataset * use pathlib * Update anomalib/data/avenue.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/avenue.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/utils/video.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/base/video.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/base/video.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/data/ucsd_ped.py Co-authored-by: Samet Akcay <[email protected]> * import video dataset from base * fix bug when collecting ucsd samples * clean up datamodules tests * fix tests * remove redundant test cases * retrieve masks as numpy array * use pathlib * variable name * pathlib * use preprocesser from arguments * fix indexing bug Co-authored-by: Joao P C Bertoldo <[email protected]> Co-authored-by: Samet Akcay <[email protected]> * properly handle batch processing * include batch index in rois tensor * return rkde results as lists * update default rkde config * add basic support for detection task * use enum for task type * formatting * small bugfix * add unit tests for bounding box conversion * update error message * use as_tensor * typing and docstring * explicit keyword arguments * simplify bbox handling in video dataset * docstring consistency * add missing licenses * add whitespace for readability * add missing license * Update anomalib/data/utils/boxes.py Co-authored-by: Samet Akcay <[email protected]> * Revert "Update anomalib/data/utils/boxes.py" This reverts commit cec6138. * add test case for custom collate function * docstring * add integration tests for detection dataloading * extend and clean up datamodules tests * add detection task type to visualizer tests * Update lightning_inference.py * only show pred_boxes during inference * add detection support for torch inference * add detection support for openvino inference * test inference for all task types * pylint * Make `val split ratio` configurable (#760) * make val split ratio configurable * use DeprecationWarning, update config key * Add support for Detection task type (#732) * add basic support for detection task * use enum for task type * formatting * small bugfix * add unit tests for bounding box conversion * update error message * use as_tensor * typing and docstring * explicit keyword arguments * simplify bbox handling in video dataset * docstring consistency * add missing licenses * add whitespace for readability * add missing license * Update anomalib/data/utils/boxes.py Co-authored-by: Samet Akcay <[email protected]> * Revert "Update anomalib/data/utils/boxes.py" This reverts commit cec6138. * add test case for custom collate function * docstring * add integration tests for detection dataloading * extend and clean up datamodules tests * add detection task type to visualizer tests * only show pred_boxes during inference * add detection support for torch inference * add detection support for openvino inference * test inference for all task types * pylint Co-authored-by: Samet Akcay <[email protected]> * [Datamodules] Update deprecation messages (#764) * update deprecation messages * raise warnings as DeprecationWarning * update rkde * Improve image source parsing for Folder dataset (#784) * mask -> mask_dir * properly handle absolute and relative paths * make root path parameter optional * formatting * path -> root * update docs * remove options hint for name parameter * refactor function * Update anomalib/config/config.py Co-authored-by: Samet Akcay <[email protected]> * Update anomalib/config/config.py Co-authored-by: Samet Akcay <[email protected]> * make root and abnormal_dir optional * Update anomalib/data/folder.py Co-authored-by: Samet Akcay <[email protected]> Co-authored-by: Samet Akcay <[email protected]> * Synthetic anomaly for testing and validation (#634) * move sample generation to datamodule instead of dataset * move sample generation from init to setup * remove inference stage and add base classes * replace dataset classes with AnomalibDataset * move setup to base class, create samples as class method * update docstrings * refactor btech to new format * allow training with no anomalous data * remove MVTec name from comment * raise NotImplementedError in base class * allow both png and bmp images for btech * use label_index to check if dataset contains anomalous images * refactor getitem in dataset class * use iloc for indexing * move dataloader getters to base class * refactor to add validate stage in setup * implement alternative datamodules solution * small improvements * improve design * remove unused constructor arguments * adapt btech to new design * add prepare_data method for mvtec * implement more generic random splitting function * update docstrings for folder module * ensure type consistency when performing operations on dataset * change imports * change variable names * replace pass with NotImplementedError * allow training on folder without test images * use relative path for normal_test_dir * fix dataset tests * update validation set parameter in configs * change default argument * use setter for samples * hint options for val_split_mode * update assert message and docstring * revert name change dataset vs datamodule * typing and docstrings * remove samples argument from dataset constructor * val/test -> eval * remove Split.Full from enum * sort samples when setting * update warn message * formatting * use setter when creating samples in dataset classes * add tests for new dataset class * add test case for label aware random split * update parameter name in inferencers * move _setup implementation to base class * address codacy issues * fix pylint issues * codacy * update example dataset config in docs * fix test * move base classes to separate files (avoid circular import) * add synthetic dataset class * move augmenter to data directory * add base classes * update docstring * use synthetic dataset in base datamodule * fix imports * clean up synthetic anomaly dataset implementation * fix mistake in augmenter * change default split ratio * remove accidentally added file * validation_split_mode -> val_split_mode * update docs * Update anomalib/data/base/dataset.py Co-authored-by: Joao P C Bertoldo <[email protected]> * get length from self.samples * assert unique indices * check is_setup for individual datasets Co-authored-by: Joao P C Bertoldo <[email protected]> * remove assert in __getitem_\ Co-authored-by: Joao P C Bertoldo <[email protected]> * Update anomalib/data/btech.py Co-authored-by: Joao P C Bertoldo <[email protected]> * clearer assert message * clarify list inversion in comment * comments and typing * validate contents of samples dataframe before setting * add file paths check * add seed to random_split function * fix expected columns * fix typo * add seed parameter to datamodules * set global seed in test entrypoint * add NONE option to valsplitmode * clarify setup behaviour in docstring * add logging message * use val_split_ratio for synthetic validation set * pathlib * make synthetic anomaly available for test set * update configs * add tests * simplify test set splitting logic * update docstring * add missing licence * split_normal_and_anomalous -> split_by_label * VideoAnomalib -> AnomalibVideo Co-authored-by: Joao P C Bertoldo <[email protected]> * Bugfixes for Datamodules feature branch (#800) * properly handle NoneType mask_dir and add test case * fix wrong deprecation handling * Deprecate PreProcessor (#795) * deprecate PreProcessor * update configs * update deprecation messages * update video dataset * update inference dataset * move transforms to data module * update and extend transform tests * fix cyclic import * add validity checks for image size and center crop * pass image size as tuple * update path to get_transforms * update error message * fix center crop tuple conversion * update inferencers * remove draem transform config * update changelog * fix cyclic import * add crop size vs image size check * improve readability * mypy * use enum to configure input normalization * update lightning inference * update inference dataset * expose more parameters and fix wrong return format * fix tdd tests * update config * [Datamodules] Fix bug in bbox score to image score conversion (#803) handle empty box predictions * update config * apply pixel threshold to bbox detections * remove confidence threshold parameter from rkde * hardcode steepness and offset * rename variable * remove unused parameters from config * Improve handling of `test_split_mode='none'` and `val_split_mode='none'` (#801) * enable none as split mode * use get to retrieve config keys * update deprecation message and config key * update config with new keys * remove unused parameter * set device in rpn stage * move prediction format conversion to lightning model * clean up torch model * move region- and feature-extractor to separate files * allow visualizing normal boxes * refactor * WIP: simplify region extractor * simplify region extractor * cleanup and docstrings * typing * expose max detections per image parameter * explain configurable parameters * fix wrong config value * remove unnecessary squeeze * box_likelihood -> rcnn_box_threshold * update comments * remove unnecessary typing * separate density estimation stage from torch model * improve readability * change default transform settings * fix to float transform * simplify feature extractor * normalize box scores * further simplify region extractor * update comment * improve prn configurability * remove unnecessary check * use enum for roi stage options * use enum for feature scaling method * re-order parameters * clean up model dir * fix bbox logic in base anomaly module * update key in output dict * boxes_scores -> box_scores * remove notebook * add comments and todo * Detection improvements (#820) * apply pixel threshold to bbox detections * allow visualizing normal boxes * normalize box scores * fix bbox logic in base anomaly module * boxes_scores -> box_scores * fix inferencers * add readme * update changelog * update changelog * update csflow config to new format * initialize max_length as empty tensor * include RKDE in model tests * remove unused imports * line length * move kde classifier to shared location * fix import * re-use RKDE classifier in DFKDE * remove old imports * docstrings * fix codacy issues * load feature extractor weights from url * suppress bandit warnings * use torch rng in augmenter * typing * add fit method to torch model * fix typo * use enum when checking stage * use tuple instead of list * add missing params to dosctring * add missing licence information * COLS -> COLUMNS * typing and variable naming * remove duplicate parameter in docstring * im_dir -> image_dir * typing and docstring * typing * ValSplitMode -> ValidationSplitMode * add missing licence * rename variable * remove empty comment * remove unused class attribute * [Detection] Compute box score when generating boxes from masks (#828) * infer box scores from anomaly maps * discard single pixel boxes * revert discard single pixel boxes * add test case for bbox scores * update torch inferencer * minor refactor * revert val_split_mode -> validation_split_mode Co-authored-by: Joao P C Bertoldo <[email protected]> Co-authored-by: Samet <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR makes the val/test split ratio configurable from the
config.yaml(see Changing ratio for create_validation_set #731, previously it was hardcoded as 0.5).The parameter
val_split_ratiowas added to the base datamodules class and subclasses.To avoid confusion with the newly added parameter, the
split_ratioparameter that determines the fraction of normal images to be used for testing in the Folder dataset, was renamed tonormal_split_ratio.config.yamlwas updated for the different models.also adds
seedparameter to Avenue and UCSDped datasets, which was missing by mistake.Fixes Changing ratio for create_validation_set #731
Changes
Checklist