This repository has been archived by the owner on Mar 21, 2024. It is now read-only.
Added ability to run segmentation inference module in the test data without or partial ground truth files. #465
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 3 commits
May 21, 2021 13:22
Creates an 'InferencePipeline.Result' object using pre-defined volumes, stores results and evaluates metrics.
ant0nsc
reviewed
May 25, 2021
ant0nsc
reviewed
May 25, 2021
ant0nsc
reviewed
May 25, 2021
ant0nsc
reviewed
May 25, 2021
ant0nsc
reviewed
May 25, 2021
ant0nsc
reviewed
May 25, 2021
ant0nsc
reviewed
May 25, 2021
added 3 commits
May 25, 2021 15:22
…ultiple ground truth images missing
For inference, given in a test dataset with all, partial, and no segmentation channels, inference will be executed and metrics will be estimated in when segmentation files exist. In overall, the changes involve
1) Setting to true the Boolean flag: "allow_incomplete_labels".
2) Keeping track of missing channels in "convert_channels_to_file_paths".
* We need to resolve which ground truth file is missing to be able to map back it correctly to the segmentation result from the inference.
3) A label volume is initialize to NaN if the corresponding ground truth file is missing
* The order is preserved so can be mapped back to corresponding segmentation in the inference.
4) Only label images that are different that NaN are evaluated in the expected order as implemented in the method "metrics.calculate_metrics_per_class".
ant0nsc
reviewed
May 27, 2021
ant0nsc
reviewed
May 27, 2021
ant0nsc
reviewed
May 27, 2021
ant0nsc
reviewed
May 27, 2021
ant0nsc
reviewed
May 27, 2021
ant0nsc
reviewed
May 27, 2021
ant0nsc
reviewed
May 27, 2021
ant0nsc
reviewed
May 27, 2021
ant0nsc
reviewed
May 27, 2021
ant0nsc
reviewed
May 27, 2021
ant0nsc
reviewed
May 27, 2021
Summary:
* For inference, the following cases are implemented
* No channels associated with mask or ground_truth_ids
* Some channels associated with mask or ground_truth_ids
* All channels associated with mask or ground_truth_ids
* Changes above, do not apply for tor training and validation.
Description: Major changes involve
1) Adding an optional boolean 'allow_incomplete_labels' flag in class 'FullImageDataset'
1.1) For training and validation, default value of 'allow_incomplete_labels' is False
1.1.1) If ground truth channels are missing, pipelines exits and reports errors
1.2) For Inference 'allow_incomplete_labels' is set to true
1.1.2) If ground truth channels are missing,
Method 'get_paths_for_channel_ids' returns a list containing corresponding paths (if provided) and None if no ground truth class files files were provided
2) Class 'Sample', main changes:
2.1) 'labels' object is optional, to support full/partial number of provided ground class files
2.1.1) # (Batches if from data loader) x Classes x Z X Y x X, where the first class is background
labels: Optional[Union[np.ndarray, torch.Tensor]]
2.2) Added 'missing_labels' list of booleans to indicate if any channel is missing
2.2.1) missing_labels: List[bool]
2.3.2) 'missing_labels' keeps the order of corresponding provided or not provided ground truth classes so that they can be correctly mapped back when for report generation
2.2.2) The length of 'missing_labels' list must be the same as the number of groundtruth classes (list 'ground_truth_ids')
3) 'load_labels_from_dataset_source', changes:
3.1) A label sample object containing ground-truth information if channel is provided
If no ground-truth channels provided, label is None
If some ground-truth channels provided are provided, then they are loaded
Background is loaded if at least one ground-truth channel is provided and is the first element of the tensor
3.2 ) A list of booleans indicating if ground-truth channel is missing
4) 'evaluate_model_predictions'
4.1) If no ground-truth channels provided, then returns: sample.metadata, MetricsDict(hues=config.ground_truth_ids)
4.2) If some or all ground-truth channel provided, then returns correponsing metrics
4.3) If all ground-truth classes are provided, then runs 'plot_contours_for_all_classes'
5) 'calculate_metrics_per_class'
5.1) From boolean list 'missing_labels' resolves which ground-truth channels corresponds to the segmentation provided by the inference and estimates corresponding metrics
6) Integrated/fix testing in the following files:
6.1) test_augmentation.py
6.2) test_plotting.py
6.3) test_metrics.py
6.4) test_inference.py
6.5) test_dataset.py
7) Integrated changes in the following dependencies:
7.1) util.py
7.2) io_util.py
7.3) dataset_util.py
7.4) augmentation.py
7.5) plotting.py
8) Added files to account the number of gt classes for training set consists of patients 1, 2, 3, and files id3* are required in unit test test_csv_dataset_as_data_loader
8.1) id3_channel2.nii.gz"
8.2) id3_mask.nii.gz"
8.3) id3_region.nii.gz"
8.4) id3_channel1.nii.gz"
9) Test unit that still will be fix:
9.1) 'test_load_images_from_dataset_source' as currently enforces ground-truth channels
ant0nsc
reviewed
May 31, 2021
ant0nsc
reviewed
May 31, 2021
ant0nsc
reviewed
May 31, 2021
ant0nsc
reviewed
May 31, 2021
ant0nsc
reviewed
May 31, 2021
ant0nsc
reviewed
Jul 2, 2021
ant0nsc
suggested changes
Jul 5, 2021
ant0nsc
previously approved these changes
Jul 5, 2021
dumbledad
previously approved these changes
Jul 5, 2021
Shruthi42
previously approved these changes
Jul 5, 2021
dumbledad
dismissed stale reviews from Shruthi42, ant0nsc, and themself
via
96840e8
Shruthi42
approved these changes
Jul 5, 2021
dumbledad
approved these changes
Jul 5, 2021
6 tasks
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Addressing issue #459: ability to run segmentation model inference on datasets without or partial segmentation files.