data parallel training uses local init score for some objectives, but should use a global init score

[jameslamb]

Description

Created from #4332 (comment).

During distributed training, when a user-provided init_score is not given (the default), each worker determines an initial score to boost from based on its local data. These initial scores are then synced up by mean between all workers.

In situations where the distribution of the target is very different across different workers, this could lead to a lower-quality initial score, which might increase the number of boosting rounds it takes to fit to the training data.

#4332 addressed this for binary classification, but this problem is still present in other objectives. See #4332 (review).

Reproducible example

Environment info

LightGBM version or commit hash: 0701a32

Command(s) you used to install LightGBM: this affects any installation of LightGBM used for distributed training.

Additional Comments

I've given this the label "dask" because it will affect training with lightgbm.dask, but it is not specific to this project's Dask interface. Any other distributed training option (https://lightgbm.readthedocs.io/en/latest/Parallel-Learning-Guide.html#integrations) will be affected by this as well.

[jameslamb]

@shiyu1994 please feel free to edit the description to add more details or fix anything I've written that is misleading or incorrect.

copying @imatiach-msft , as I think this issue is relevant for mmlspark as well.