Support different data types (when load data from Python)

[etveritas]

Summary

Specify data type when load data from Python, like col1-->float16, col2-->int8

Motivation

Save memory.

Description

We usually use Python API of LightGBM, and use pandas DataFrame reserve data. Sometimes, we specify data in int8 or float16, but when convert into LightGBM Dataset, they all convert to float64 or float32, it can cause more memory usage. So how can we specify data type when load data from Python.

[guolinke]

this feature is not supported yet, LightGBM requires the same type for all features.

[aldanor]

On a side note, it would be nice if lightgbm supported feature types like int16 etc. E.g. you may have a huge matrix of int16 features but you have to convert it to float32/float64 just in order for lightgbm to be able to consume it...

[etveritas]

@guolinke hello, would this feature will be added in the future?

[guolinke]

Currently, the data loader in LightGBM is designed for row-wise data. For the column-wise, it needs more refactoring works.
But I think this is quite important, as it can save memories when loading from many column-wise objects, like pandas/datatables.

[etveritas]

I see, most of ml system use C++ as backend can meet this problem, maybe we should get some change.
Thanks~
And feel free to close this issue or not.

[StrikerRUS]

Closed in favor of being in #2302. We decided to keep all feature requests in one place.

Welcome to contribute this feature! Please re-open this issue (or post a comment if you are not a topic starter) if you are actively working on implementing this feature.