Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update RLearner_classif_h2odeeplearning.R #2668

Merged
merged 8 commits into from
Nov 10, 2019
Merged

Update RLearner_classif_h2odeeplearning.R #2668

merged 8 commits into from
Nov 10, 2019

Conversation

albersonmiranda
Copy link
Contributor

@albersonmiranda albersonmiranda commented Oct 27, 2019

Added params.

fixes #2658

@lintr-bot
Copy link

R/RLearner_classif_h2odeeplearning.R:223:57: style: Put spaces around all infix operators (except exponentiation).

​      makeDiscreteLearnerParam("stopping_metric", values=c("AUTO", "deviance", "logloss", "MSE", "RMSE", "MAE", "RMSLE", "AUC",
                                                       ~^~

@larskotthoff
Copy link
Member

Thanks, looks good. Could you also please add at least some of the new parameters to the tests for this learner?

Are these parameters relevant for the regression learner as well? If so, it would be great if you could add them to that as well please.

@jakob-r
Copy link
Member

jakob-r commented Oct 27, 2019

We should check, because some of these "new" ones were already in the regr learner.

@albersonmiranda
Copy link
Contributor Author

albersonmiranda commented Oct 29, 2019

@jakob-r , i just verified and all of them are already set in regr learner, except for stopping_rounds which is set as integer in regr and i've set as numeric with another default. It worked locally but i'll make it match regr just in case.

Also, there's a few more stopping_metric values in documentation (which i've added in classif.) that aren't defined for regr.

@larskotthoff , if you guys not sure i can add it to the tests by saturday.

set stopping rounds as integer and added requirement to stopping metric.
@jakob-r
Copy link
Member

jakob-r commented Oct 29, 2019

Also, there's a few more stopping_metric values in documentation (which i've added in classif.) that aren't defined for regr.

I would assume that "deviance", "logloss", "MSE", "RMSE", "MAE", "RMSLE" are stopping metrics for regr and not classif. Unfortunately the h2o documentation does not give hints here.

stopping_rounds which is set as integer in regr and i've set as numeric with another default. It worked locally but i'll make it match regr just in case.

It might work because h2o converts it but certainly e.g. 43.4 would not be a plausible value.

@berndbischl
Copy link
Member

Log loss is for classify... Come on... Deviance probably too

@jakob-r
Copy link
Member

jakob-r commented Oct 29, 2019

My bad, I wrongly assumed there are ordered in some sense without giving too much thought. But deviance seems to be for regression according to this page Although I do not understand the "formula".

@berndbischl
Copy link
Member

i was wrong. deviance is for regression. i think it should be possible to use it also for classif, but maybe not supported, maybe simply test it for 1 min.

also i had to stare 5 min at the formula to understand it

Loss = Quadratic -> MSE==Deviance For Absolute/Laplace or Huber -> MSE != Deviance

you have to read it like this

if loss=l2, then mse == dev
if loss=l1 or huber, then mse!=dev

@pat-s
Copy link
Member

pat-s commented Nov 6, 2019

Whats the status here?

@albersonmiranda Will you incorporate the changes from Jakob?

@albersonmiranda
Copy link
Contributor Author

albersonmiranda commented Nov 7, 2019

@pat-s sorry for the delay. Updated stopping_metric as Jakob pointed out and added distribution with values c("bernoulli", "multinomial"). I'm not sure if others available make sense for classification problems.

Also, c("bernoulli", "multinomial") distributions and "logloss", "AUC", "misclassification" stopping_metrics are defined for regression learner. It's ok if i open a PR to exclude them?

@jakob-r
Copy link
Member

jakob-r commented Nov 8, 2019

From my side it is related enough to do it in one PR. Thanks for the effort!

@pat-s pat-s changed the title Update RLearner_classif_h2odeeplearning.R Update RLearner_classif_h2odeeplearning.R Nov 8, 2019
changes in distributions and stopping metrics
@albersonmiranda
Copy link
Contributor Author

Done!

@pat-s
Copy link
Member

pat-s commented Nov 10, 2019

Thanks for contributing!

@pat-s pat-s merged commit 362f598 into mlr-org:master Nov 10, 2019
@albersonmiranda albersonmiranda deleted the patch-1 branch November 10, 2019 19:58
vrodriguezf pushed a commit to vrodriguezf/mlr that referenced this pull request Jan 16, 2021
* Update RLearner_classif_h2odeeplearning.R

Added params.

* style changes for travis

* updated type

set stopping rounds as integer and added requirement to stopping metric.

* Update metrics and added distribution

* Update RLearner_regr_h2odeeplearning.R

changes in distributions and stopping metrics

* add NEWS
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

h2o.deeplearning early stopping: stopping_rounds is missing?
6 participants