24GB graphics card memory is not enough

[Miraclerice]

Hello, I tried to use 3090 to run your network, and found that the video memory was insufficient. The picture size is the same. Is there any way to reduce the video memory usage? I see that your server is 4090, which is also 24GB. If you have time, please reply

[MungoMeng]

Please make sure that the "use_checkpoint" is set as "True".
Turning on the checkpoint function will reduce more than 50% GPU memory consumption.

[Miraclerice]

Please make sure that the "use_checkpoint" is set as "True". Turning on the checkpoint function will reduce more than 50% GPU memory consumption.

Thank you very much for your answer. I keep the parameter setting of the network, 'use_checkpoint' defaults to True. Is there any other setting

Registration-CorrMLP/CorrMLP/networks.py

Line 22 in da5ce37

use_checkpoint: bool = True):

[MungoMeng]

This might be because using the checkpointing function in 4090 can reduce more GPU memory than 3090.
Perhaps you can reduce the 'enc_channels' and 'dec_channels' a little bit.

[Miraclerice]

Ok, I'll try. Thank you

[ZhaiJiaKai]

好的，我试试。谢谢

Hello, have you solved this problem? I also encountered the same problem when training. I used 4090

[2250432015]

Hello, I am currently engaged in training the CorrMLP model for medical image registration （OASIS dataset） on an NVIDIA RTX 3090. After several adjustments, I have managed to run the model with the parameters set to enc_channels: int = 8, dec_channels: int = 12, which allows the model to operate without running out of memory. However, as shown in the attached figure, these adjustments have had a significant impact on the model's accuracy.

I am seeking your expert advice on how to fine-tune the model to achieve a better balance between memory usage and accuracy. Specifically, I would like to know if there are recommended areas within the network structure that could be modified to reduce memory consumption without drastically affecting the model's performance.

[MungoMeng]

The default channel number settings are enc_channels: int = 8, dec_channels: int = 16 (as shown in Figure 1 in the paper) when using 24GB 4090 GPU. In these settings, our memory comsumption is ~20GB:

I am not sure why your model with enc_channels: int = 8, dec_channels: int = 12 has used 24GB memory. My PyTorch Version is 1.13 with Cuda 11.7, which might incur some differences when using checkpoint technique? Or do you use the images with size (144,192,160)? -- Larger size will result in more memory consumption.