How can I inference trained model using c++ with gpu?

[YuqiangJia]

I am loading a pre-trained lightgbm model and try to inference using GPU:
LGBM_BoosterPredictForMat(_handle, _agg_features.data(), C_API_DTYPE_FLOAT32, 1, _input_channels * _agg_feature_dim, 1, C_API_PREDICT_NORMAL, 0, -1, "device_type=gpu", &out_len, out.data());
However, I found it still using cpu to inference, is it the right way using GPU?

Summary

Motivation

Description

References

[YuqiangJia]

Can anyone help me?

[Kaiyang-Chen]

Anyone has any idea on this? Do we support GPU inference or not?

[jameslamb]

Thanks for using LightGBM.

I believe that LightGBM does not currently have GPU-accelerated prediction, and that only training-related workloads run on the GPU. @shiyu1994 could confirm.

If you're finding that LightGBM's existing prediction routines on CPU are not fast enough for your application and are looking to improve performance, you could also explore these projects that allow serving LightGBM models:

• treelite (https://github.com/dmlc/treelite/blob/mainline/docs/tutorials/import.rst)
• fil ("forest inference library") backend for Triton (https://github.com/triton-inference-server/fil_backend)

[github-actions]

This issue has been automatically closed because it has been awaiting a response for too long. When you have time to to work with the maintainers to resolve this issue, please post a new comment and it will be re-opened. If the issue has been locked for editing by the time you return to it, please open a new issue and reference this one. Thank you for taking the time to improve LightGBM!