Refactor anomalib to new annotation format, add `refurb` and `pyupgrade` #845

samet-akcay · 2023-01-17T17:48:15Z

Description

Refactor anomalib to new annotation format
Modify the code using refurb. The tool has not yet been added to the pre-commit configuration since it requires Python3.10 to execute.
Add pyupgrade to pre-commit settings and modify the code accordingly.

Changes

Bug fix (non-breaking change which fixes an issue)
Refactor (non-breaking change which refactors the code base)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

Checklist

My code follows the pre-commit style and check guidelines of this project.
I have performed a self-review of my code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing tests pass locally with my changes
I have added a summary of my changes to the CHANGELOG (not for minor changes, docs and tests).

…tor/add-pyupgrade-and-refurb

anomalib/data/avenue.py

djdameln

Thanks. Much more readable now. I have a few comments. My main concern is with the deletion of unused attributes with the del keyword, which feels a bit hacky. Maybe we could think of an alternative approach.

anomalib/data/btech.py

anomalib/models/cfa/lightning_model.py

djdameln · 2023-01-20T15:09:29Z

anomalib/models/components/base/anomaly_module.py

        super().__init__()
        logger.info("Initializing %s model.", self.__class__.__name__)

        self.save_hyperparameters()
        self.model: nn.Module
-        self.loss: Tensor
-        self.callbacks: List[Callback]
+        self.loss: nn.Module


I don't understand this change. As far as I know, the loss attribute holds the value of the loss between epochs, so it should be a Tensor.

in almost all of the implementations, we use self.loss to compute the loss. For example, here

anomalib/anomalib/models/draem/lightning_model.py

Line 41 in ce70716

self.loss = DraemLoss()

When we define this as Tensor, pylint and mypy complains, because we use this as Callable

If PL uses this under the hood, we could then rename this to self.loss_func

anomalib/models/components/base/anomaly_module.py

anomalib/models/components/feature_extractors/timm.py

anomalib/models/csflow/lightning_model.py

anomalib/models/reverse_distillation/components/bottleneck.py

anomalib/utils/callbacks/cdf_normalization.py

anomalib/utils/cli/cli.py

setup.py

Co-authored-by: Dick Ameln <[email protected]>

…notoolkit/anomalib into refactor/add-pyupgrade-and-refurb

djdameln

Thanks, just one minor comment, which could probably be ignored

djdameln · 2023-01-26T15:56:47Z

anomalib/models/csflow/torch_model.py

-                            "channels_hidden": self.cross_conv_hidden_channels,
-                            "kernel_size": self.kernel_sizes[coupling_block],
-                        },
+            nodes.extend([permute_node])


I feel this would best be left as append, because it just adds a single instance. But if refurb complains about this I'm fine with this change.

I feel so too, but just did it this way to silence refurb :)

djdameln · 2023-01-26T16:00:23Z

anomalib/utils/cli/cli.py

@@ -147,7 +149,7 @@ def __set_default_root_dir(self) -> None:
        # If `resume_from_checkpoint` is not specified, it means that the project has not been created before.
        # Therefore, we need to create the project directory first.
        if config.trainer.resume_from_checkpoint is None:
-            root_dir = config.trainer.default_root_dir if config.trainer.default_root_dir else "./results"
+            root_dir = config.trainer.default_root_dir or "./results"


cool, I was not aware of this syntax

me neither :)

samet-akcay added 30 commits January 16, 2023 06:21

Add missing type hints to export

d10f166

Add missing type hints to inferencer interfaces

201a335

Add missing type hints to post-processing modules

1b4f2a0

Add missing type hints to pre-processing modules

039da47

Add missing type hints to nncf callback

f7c8db5

Add missing type hints to visualizer callbacks

66bda3c

Add missing type hints to cdf normalization callbacks

536078b

Add missing type hints to export callback

cfd86fa

Add missing type hints to metrics configuration callback

495a9bf

Add missing type hints to min-max normalization configuration callback

6c394d0

Add missing type hints to init callbacks

075b2fb

Add missing type hints to model loader callback

61beeae

Add missing type hints to post processing callback

fc448dd

Add missing type hints to tiler configuration callback

96b2092

Add missing type hints to timer callback

8ee2613

Add missing type hints to utils

3ab7cdc

Merge branch 'main' of github.com:openvinotoolkit/anomalib into refac…

a670e4b

…tor/add-pyupgrade-and-refurb

Refactored datamodule

e4bdb46

Add missing type hints to dataset

415e76b

Add missing type hints to download

e7a620e

Add missing type hints to folder dataset

7e521da

Add missing type hints to mvtec dataset

2b22dab

Add missing type hints to mvtec dataset

a8a167d

Changed method signature of forward in AnomalyModule

c87f582

Changed method signature of validation step

b05d20f

Add type hints to cflow

1226ecc

Add type hints to csflow

92f4335

Add type hints to dfkde

b654982

Add type hints to dfm

0f62690

Add type hints to dfm

8cb435c

github-actions bot added Benchmarking Callbacks CLI Config Data HPO Inference Logger Metrics Metric Component. Notebooks Post-Processing The components that are related to post-processing Pre-Processing Setup Tools labels Jan 17, 2023

Fix pre-commit

b89da66

github-actions bot added the Tests label Jan 17, 2023

Fix ganomaly errors

16ece78

djdameln reviewed Jan 20, 2023

View reviewed changes

anomalib/data/avenue.py Show resolved Hide resolved

djdameln requested changes Jan 20, 2023

View reviewed changes

samet-akcay and others added 6 commits January 20, 2023 16:09

Update anomalib/models/reverse_distillation/components/bottleneck.py

b238fad

Co-authored-by: Dick Ameln <[email protected]>

Address the PR comments.

fcd2176

Merge branch 'refactor/add-pyupgrade-and-refurb' of github.com:openvi…

6353730

…notoolkit/anomalib into refactor/add-pyupgrade-and-refurb

Address refurb comments

abb9900

add exits_ok=True to address failed tests

d47e149

Fix tests

7be9eb5

djdameln approved these changes Jan 26, 2023

View reviewed changes

Update CHANGELOG.md

936e0ca

github-actions bot added the Docs label Jan 26, 2023

samet-akcay merged commit 01f3323 into main Jan 26, 2023

samet-akcay deleted the refactor/add-pyupgrade-and-refurb branch January 26, 2023 16:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor anomalib to new annotation format, add `refurb` and `pyupgrade` #845

Refactor anomalib to new annotation format, add `refurb` and `pyupgrade` #845

samet-akcay commented Jan 17, 2023

djdameln left a comment

djdameln Jan 20, 2023

samet-akcay Jan 20, 2023

samet-akcay Jan 20, 2023

djdameln left a comment

djdameln Jan 26, 2023

samet-akcay Jan 26, 2023

djdameln Jan 26, 2023

samet-akcay Jan 26, 2023

Refactor anomalib to new annotation format, add refurb and pyupgrade #845

Refactor anomalib to new annotation format, add refurb and pyupgrade #845

Conversation

samet-akcay commented Jan 17, 2023

Description

Changes

Checklist

djdameln left a comment

Choose a reason for hiding this comment

djdameln Jan 20, 2023

Choose a reason for hiding this comment

samet-akcay Jan 20, 2023

Choose a reason for hiding this comment

samet-akcay Jan 20, 2023

Choose a reason for hiding this comment

djdameln left a comment

Choose a reason for hiding this comment

djdameln Jan 26, 2023

Choose a reason for hiding this comment

samet-akcay Jan 26, 2023

Choose a reason for hiding this comment

djdameln Jan 26, 2023

Choose a reason for hiding this comment

samet-akcay Jan 26, 2023

Choose a reason for hiding this comment

Refactor anomalib to new annotation format, add `refurb` and `pyupgrade` #845

Refactor anomalib to new annotation format, add `refurb` and `pyupgrade` #845