GPU RAM Requirements for Formula Detection (do_formula_enrichment)

[JPC612]

Hi,

I am using docling with an RTX 3090 and encountering a CUDA out-of-memory error when enabling do_formula_enrichment=True. Could you provide information on the expected GPU RAM usage for formula detection? How much memory is typically required to process documents with this setting enabled?

Thanks in advance!

[Matteo-Omenetti]

Hello,

The CodeFormula model can be quite heavy, and a default batch size of 16 may be too high for some hardware setups. We’re currently working on an update that will allow each model to use its own batch size, ensuring better adaptability for models of different sizes.

In the meantime, you can manually reduce the batch size by importing and modifying the settings object. Please note that this change applies to all models, not just CodeFormula:

from docling.datamodel.settings import settings

settings.perf.elements_batch_size = 2

I hope this helps!

[Matteo-Omenetti]

I will update this issue as soon as the more fine-grained batch size selection feature is released.

[JPC612]

Thank you !!