Unable to init a TorchInferencer for EfficientAD model

[Sanjit10]

Hello everyone!!

I recently came across this library and have currently trained an EfficientAD model by following the code snippets provided in the repository readme, but I am having trouble initializing a torch inferencer with the trained model.

# Path to the new .pt model file
model_path = "./results/EfficientAd/MVTec/pill/v1/weights/lightning/model_with_model_key.pt"
print(f"Loading model from: {model_path}")


# Initialize the inferencer
inferencer = TorchInferencer(
    path=model_path,
    device="cuda",  # or "cpu" if you don't have a GPU
)

Error:

    inferencer = TorchInferencer(
  File "/opt/conda/lib/python3.10/site-packages/anomalib/deploy/inferencers/torch_inferencer.py", line 70, in __init__
    self.checkpoint = self._load_checkpoint(path)
  File "/opt/conda/lib/python3.10/site-packages/anomalib/deploy/inferencers/torch_inferencer.py", line 108, in _load_checkpoint
    raise ValueError(msg)
ValueError: Unknown torch checkpoint file format .ckpt. Make sure you save the Torch model.

The Torch Inferencer requires a .pt or .pth model while the training produces a .ckpt file, so I loaded the .ckpt file using Torch and saved it to a .pt file.

class TorchInferencer(Inferencer):
    def __init__(
        self,
        path: str | Path,
        device: str = "auto",
    ) -> None:
        self.device = self._get_device(device)

        # Load the model weights and metadata
        self.checkpoint = self._load_checkpoint(path)
        self.model = self.load_model(path)
        self.metadata = self._load_metadata(path)

The code is looking for information under metadata and model key in the .pt files which are not present causing the error while loading the inferencer.

      self.model = self.load_model(path)
  File "/opt/conda/lib/python3.10/site-packages/anomalib/deploy/inferencers/torch_inferencer.py", line 156, in load_model
    raise KeyError(msg)
KeyError: '``model`` is not found in the checkpoint. Please check the checkpoint file.'

At first, I tried to save the state_dict with the name model and created a new checkpoint file, but when that didn't work,
I tried to address the problem with the model by tweaking the code a little.

    def load_model(self, path: str | Path) -> nn.Module:
        """Load the PyTorch model.

        Args:
            path (str | Path): Path to the Torch model.

        Returns:
            (nn.Module): Torch model.
        """
        checkpoint = self._load_checkpoint(path)
        if "model" not in checkpoint:
            msg = "``model`` is not found in the checkpoint. Please check the checkpoint file."
            raise KeyError(msg)
        model_class = EfficientAd()
        model_class.load_state_dict(checkpoint["model"])
        model = model_class.eval()
        return model.to(self.device)

But I am still getting the issue of metadata,

epoch: <class 'int'>
global_step: <class 'int'>
pytorch-lightning_version: <class 'str'>
state_dict: ['image_threshold_class', 'pixel_threshold_class', 'normalization_class', 'model.teacher.conv1.weight', 'model.teacher.conv1.bias', 'model.teacher.conv2.weight', 'model.teacher.conv2.bias', 'model.teacher.conv3.weight', 'model.teacher.conv3.bias', 'model.teacher.conv4.weight', 'model.teacher.conv4.bias', 'model.student.conv1.weight', 'model.student.conv1.bias', 'model.student.conv2.weight', 'model.student.conv2.bias', 'model.student.conv3.weight', 'model.student.conv3.bias', 'model.student.conv4.weight', 'model.student.conv4.bias', 'model.ae.encoder.enconv1.weight', 'model.ae.encoder.enconv1.bias', 'model.ae.encoder.enconv2.weight', 'model.ae.encoder.enconv2.bias', 'model.ae.encoder.enconv3.weight', 'model.ae.encoder.enconv3.bias', 'model.ae.encoder.enconv4.weight', 'model.ae.encoder.enconv4.bias', 'model.ae.encoder.enconv5.weight', 'model.ae.encoder.enconv5.bias', 'model.ae.encoder.enconv6.weight', 'model.ae.encoder.enconv6.bias', 'model.ae.decoder.deconv1.weight', 'model.ae.decoder.deconv1.bias', 'model.ae.decoder.deconv2.weight', 'model.ae.decoder.deconv2.bias', 'model.ae.decoder.deconv3.weight', 'model.ae.decoder.deconv3.bias', 'model.ae.decoder.deconv4.weight', 'model.ae.decoder.deconv4.bias', 'model.ae.decoder.deconv5.weight', 'model.ae.decoder.deconv5.bias', 'model.ae.decoder.deconv6.weight', 'model.ae.decoder.deconv6.bias', 'model.ae.decoder.deconv7.weight', 'model.ae.decoder.deconv7.bias', 'model.ae.decoder.deconv8.weight', 'model.ae.decoder.deconv8.bias', 'model.mean_std.mean', 'model.mean_std.std', 'model.quantiles.qa_ae', 'model.quantiles.qa_st', 'model.quantiles.qb_ae', 'model.quantiles.qb_st', 'normalization_metrics.min', 'normalization_metrics.max', 'image_threshold.value', 'pixel_threshold.value']
loops: ['fit_loop', 'validate_loop', 'test_loop', 'predict_loop']
callbacks: ["ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}"]
optimizer_states: <class 'list'>
lr_schedulers: <class 'list'>
hparams_name: <class 'str'>
hyper_parameters: ['imagenet_dir', 'teacher_out_channels', 'model_size', 'lr', 'weight_decay', 'padding', 'pad_maps']
transform: <class 'torchvision.transforms.v2._container.Compose'>

I did see a few comments discussing this issue but no solution provided.
These are the details present in the .ckpt file trained I would appreciate any help regarding the matter.

[alexriedel1]

You have to export the model after training like
engine.export(model=model, export_type=ExportType.TORCH, export_root=os.path.join(root_folder, "export"))

[Sanjit10]

Thank you !!!!!!!!

[mmkolbe]

I'm in the same situation and the export did not work for me. This is how I made just after the training. The train generates an okay ckpt file. But the exported file, from model.export(), named model.pt, cannot be used by the torch_inferencer.py:

model = EfficientAd()
engine = Engine(task="classification",  max_epochs=10)
engine.train(datamodule=datamodule, model=model)
engine.export(model=model, export_type=ExportType.TORCH, export_root=os.path.join(root_folder, "export"))

I get this result:

Traceback (most recent call last):
  File "/home/marcel/tm2/engine/inference.py", line 111, in <module>
    infer(args=args)
  File "/home/marcel/tm2/engine/inference.py", line 87, in infer
    inferencer = TorchInferencer(path=args.weights, device=args.device)
  File "/home/marcel/anaconda3/envs/tm2/lib/python3.10/site-packages/anomalib/deploy/inferencers/torch_inferencer.py", line 69, in __init__
    self.checkpoint = self._load_checkpoint(path)
  File "/home/marcel/anaconda3/envs/tm2/lib/python3.10/site-packages/anomalib/deploy/inferencers/torch_inferencer.py", line 107, in _load_checkpoint
    raise ValueError(msg)
ValueError: Unknown torch checkpoint file format .pt'. Make sure you save the Torch model.

[alexriedel1]

It looks like you are not using the exported model (should be a .pt file). make sure to pass the model path that was created by the export function to your torch inferencer. it should be a located in the "export" directory.

[mmkolbe]

@alexriedel1 Thank you very much for pointing out. The [silly] error was passing the path with single quotes. ¬_¬

[samet-akcay]

Since many users experience this issue, maybe we could have a clearer error message for this case to explain what is wrong and what to do