You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Now I've looked at documentation in more detail. I'd like to suggest two minor enhancements for validation in Task and Learner:
There are many cases where a validation dataset may be required by a learner in order to make use of internal processes such as early stopping (e.g. GBMs, NNs). As validation is a common use-case, it may be worthwhile adding a simple public method to Task that creates arbitrary train/validation splits that can be passed to learner hyper-parameters, e.g.
Whilst this is clearly a thin wrapper around a basic function that could be performed by a user, it still requires user knowledge about row_roles and where to find validation splits (which aren't too well documented).
Alternatively if you don't agree this is worthwhile then I'd suggest adding an example to mlr-org/mlr3book#201, e.g.
Add 'validation' to learner and task properties (similar to 'weights'). Doing so would allow a more efficient method of implementing validation datasets in learners that can handle them, as currently we assume the user will pass in the validation data in the correct format, which is inconsistent with the general task interface, i.e. it is inconsistent for a user to pass Task for training data but a data object for validation data. This would also highlight to users when they can/cannot set the validation role.
EDIT: Fixed examples.
The text was updated successfully, but these errors were encountered:
Now I've looked at documentation in more detail. I'd like to suggest two minor enhancements for
validation
inTask
andLearner
:Task
that creates arbitrary train/validation splits that can be passed to learner hyper-parameters, e.g.Whilst this is clearly a thin wrapper around a basic function that could be performed by a user, it still requires user knowledge about
row_roles
and where to find validation splits (which aren't too well documented).Alternatively if you don't agree this is worthwhile then I'd suggest adding an example to mlr-org/mlr3book#201, e.g.
Task
for training data but a data object for validation data. This would also highlight to users when they can/cannot set thevalidation
role.EDIT: Fixed examples.
The text was updated successfully, but these errors were encountered: