[R-package] add support for specifying training indices in lgb.cv()

[julioasotodv]

Summary

The addition of a train_folds argument in lgb.cv() would allow for more fine-grained folds generation that is useful in some scenarios, such as time series forecasting (just like xgboost R package does).

Motivation

The R function lgb.cv() currently has got an argument that allows you to specify manual folds through the folds argument. This argument expects a list of indices that should go to the test set for each fold, and all the other indices will go to the train set.

However, in some types of datasets and tasks (such as in time series), you may actually want to have different folds where some indices are just not used for that specific fold, neither in the train or test sets (just to avoid leaking information from the future).

Description

The Xgboost R package included this feature a while back, which essentially consists of adding one more argument to the cv() function called train_folds. This way, train_folds, if specified, makes sure that only those indices will go to the train set in each fold. If it is not specified, the train indices will just be the opposite of the ones in the folds argument, just like lgb.cv() works right now.

References

Please see the train_folds argument in xgb.cv() here, and the relevant code in xgb can be found here

Thank you!

[jameslamb]

Hi @julioasotodv . Thanks for using LightGBM, and for taking the time to write up this excellent feature request!

Adding a few more details for whenever this is picked up

• PRs where {xgboost} added this feature: Added new train_folds parameter dmlc/xgboost#5114, Added new train_folds parameter for xgb.cv() dmlc/xgboost#5064
• Stack Overflow question that prompted the train_folds feature in {xgboost}: https://stackoverflow.com/questions/32433458/how-to-specify-train-and-test-indices-for-xgb-cv-in-r-package-xgboost/51412073

I think this example from that Stack Overflow question describes the situation well

# assume i have 100 strips of time-series data, where each strip is X_i
# validate only on 10 points after training
fold1:  train on X_1-X_10, validate on X_11-X_20
fold2:  train on X_1-X_20, validate on X_21-X_30
fold3:  train on X_1-X_30, validate on X_31-X_40
...

I've added this feature to #2302, where we organize the list of feature requests. I've also edited the title so that it's understandable for people who don't have prior knowledge of the train_folds argument in {xgboost}.

@julioasotodv are you interested in contributing this feature?

[julioasotodv]

@jameslamb sure! I just did. I will create a PR soon!

[jameslamb]

At the moment, this feature is not being actively worked on (see #3989 (comment)).

Per the current policy in this repository, I'm going to close this GitHub issue but keep the feature marked as open in #2302. Anyone who is interested in contributing this feature or who wants to add something to this discussion is encouraged to comment here, and the issue can be re-opened.