EfficientAD

[alexriedel1]

Description

Hey,
this is the work in progress for implementing EfficientAD based on the implementations https://github.com/nelson1425/EfficientAD/blob/main/efficientad.py and https://github.com/rximg/EfficientAD/tree/main

Right now the performance doesn't meet the paper results but I'm on it :)

• Fixes # (issue)

Changes

• Bug fix (non-breaking change which fixes an issue)
• Refactor (non-breaking change which refactors the code base)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Checklist

• My code follows the pre-commit style and check guidelines of this project.
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing tests pass locally with my changes
• I have added a summary of my changes to the CHANGELOG (not for minor changes, docs and tests).

[alexriedel1]

I have tried this using Notebooks section and also OpenVINO model. I got an error with 001_getting_started notebook.

`--------------------------------------------------------------------------- ValueError Traceback (most recent call last) Cell In[14], line 5 2 config.optimization.export_mode = "openvino" 4 # Get the model and callbacks ----> 5 model = get_model(config) 6 callbacks = get_callbacks(config)

File ~\Intel\tests\anomalib_effad_env\lib\site-packages\anomalib\models__init__.py:106, in get_model(config) 103 model = getattr(module, f"{_snake_to_pascal_case(config.model.name)}Lightning")(config) 105 else: --> 106 raise ValueError(f"Unknown model {config.model.name}!") 108 if "init_weights" in config.keys() and config.init_weights: 109 model.load_state_dict(load(os.path.join(config.project.path, config.init_weights))["state_dict"], strict=False)

ValueError: Unknown model efficientad!`

Guess you didn't pull the right branch? I have no issues loading the model

[paularamo]

I have tried this using Notebooks section and also OpenVINO model. I got an error with 001_getting_started notebook.
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) Cell In[14], line 5 2 config.optimization.export_mode = "openvino" 4 # Get the model and callbacks ----> 5 model = get_model(config) 6 callbacks = get_callbacks(config) File ~\Intel\tests\anomalib_effad_env\lib\site-packages\anomalib\models__init__.py:106, in get_model(config) 103 model = getattr(module, f"{_snake_to_pascal_case(config.model.name)}Lightning")(config) 105 else: --> 106 raise ValueError(f"Unknown model {config.model.name}!") 108 if "init_weights" in config.keys() and config.init_weights: 109 model.load_state_dict(load(os.path.join(config.project.path, config.init_weights))["state_dict"], strict=False) ValueError: Unknown model efficientad!

Guess you didn't pull the right branch? I have no issues loading the model

Yes, It is working installing from your source files. Thanks

[alexriedel1]

I have trained a teacher model using albumentation transform and it's giving even better results than the previous model. I added the model weights and removed the old ones. Everything should be in place now and ready for merging. What about adding the model to tests etc.?

[nelson1425]

@alexriedel1 very well done thank you for your work! Can you enter the Mvtec AD results in the tables in https://github.com/openvinotoolkit/anomalib/blob/main/README.md ?

[alexriedel1]

@alexriedel1 very well done thank you for your work! Can you enter the Mvtec AD results in the tables in https://github.com/openvinotoolkit/anomalib/blob/main/README.md ?

Hey, I dont have the ressources to run the anomalib implementation on all mvtec AD classes so I cannot add the results for this implementation of EfficientAD. If there is someone here who could do a benchmark (https://github.com/openvinotoolkit/anomalib/tree/main/tools/benchmarking) with EfficientAD that would be great

[samet-akcay]

@alexriedel1 very well done thank you for your work! Can you enter the Mvtec AD results in the tables in https://github.com/openvinotoolkit/anomalib/blob/main/README.md ?

Hey, I dont have the ressources to run the anomalib implementation on all mvtec AD classes so I cannot add the results for this implementation of EfficientAD. If there is someone here who could do a benchmark (https://github.com/openvinotoolkit/anomalib/tree/main/tools/benchmarking) with EfficientAD that would be great

I'll try to run a benchmark this week

[samet-akcay]

What would be the best action to take here? Should we keep them here or release them as assets in the releases section? @alexriedel1, @djdameln, @ashwinvaidya17 ?

[ashwinvaidya17]

My main concern in that the git tree might become too big. I am in favour of releasing them as assets. We can add a readme with acknowledgement and license. Also, release will give us a permalink that we can refer to in the code.

[alexriedel1]

Sure, I will add the download function once the weights are released as assets

[samet-akcay]

Here is the link to the weights
https://github.com/openvinotoolkit/anomalib/releases/tag/efficientad_pretrained_weights

@alexriedel1, can you please double check if they work ok? Let me know if there is something wrong with the link. Thanks!

[alexriedel1]

Alright it's done! I have added support for the model weight download (and I also added support for arbitrary image sizes which was not the case before)

[samet-akcay]

Thanks again @alexriedel1. Looking good to me now

[ashwinvaidya17]

Thanks!

[alexriedel1]

Im having a hard time getting the precommit hooks prettier and markdownlint to run on my windows machine ☹️ I will give it another try though today

[samet-akcay]

Im having a hard time getting the precommit hooks prettier and markdownlint to run on my windows machine ☹️ I will give it another try though today

We had this feedback from someone else too. Wasn't aware that pre-commit wasn't working well on a windows machine. Or maybe our configuration is problematic. We might need to investigate.

Meanwhile I could run the checks on your PR

[djdameln]

Thanks a lot for this contribution!

[watertianyi]

@alexriedel1 @samet-akcay
I saw that you used the https://github.com/nelson1425/EfficientAD code to improve it
What I want to know is:

1. Did you use pretraining.py in nelson1425 to train the teacher weights?
2. Then the training code in anomalib is equivalent to the efficientad.py training code in nelson1425, right?
3. If I use the teacher weights trained by pretraining.py in nelson1425, can they be used in anomalib?
4. I only used 200 positive samples of my own data to train efficientad in anomalib. The classification effect was image_AUROC 70% and image_F1Score 60%.
   If segmentation training is used, pixel_AUROC is 70% and pixel_F1Score is 30%. Is there any way to improve the above?
5. Also, when the above classification model is tested, the defect intensity of the heat map of the abnormal data is not obvious or the performance is not obvious. Is there any way to improve it?
6. Although the segmentation model can reflect the heat map defects of abnormal data, it is often segmented incorrectly?