Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix] Remove SmoothL1Loss from Tutorial #1298

Merged
merged 5 commits into from
Apr 21, 2023
Merged

[fix] Remove SmoothL1Loss from Tutorial #1298

merged 5 commits into from
Apr 21, 2023

Conversation

judussoari
Copy link
Collaborator

🔬 Background

Closes #1234
SmoothL1Loss is not supported by Torchmetrics. With the lightning migration, we only use those, however.
Also, m.test() does not return error metrics specified by the user.

🔮 Key changes

  • Remove SmoothL1Loss in crossvalidation tutorial.
  • The return values of m.test() have been adjusted.

📋 Review Checklist

  • I have performed a self-review of my own code.
  • I have commented my code, added docstrings and data types to function definitions.
  • I have added pytests to check whether my feature / fix works.

@judussoari judussoari added type:bug Something isn't working status: needs review PR needs to be reviewed by Reviewer(s) labels Apr 21, 2023
@judussoari judussoari added this to the Release 0.6.0 milestone Apr 21, 2023
@judussoari judussoari self-assigned this Apr 21, 2023
@judussoari judussoari changed the title [fix] Remove SmoothL1Loss from Tutoria [fix] Remove SmoothL1Loss from Tutorial Apr 21, 2023
@github-actions
Copy link

github-actions bot commented Apr 21, 2023

Model Benchmark

Benchmark Metric main current diff
AirPassengers MAE_val 13.0626 13.0626 0.0%
AirPassengers RMSE_val 15.9453 15.9453 0.0%
AirPassengers Loss_val 0.00131 0.00131 0.0%
AirPassengers MAE 9.88156 9.88156 0.0%
AirPassengers RMSE 11.7354 11.7354 0.0%
AirPassengers Loss 0.00052 0.00052 0.0%
AirPassengers time 5.8234 6.08 4.41% ⚠️
YosemiteTemps MAE_val 1.3442 1.3442 0.0%
YosemiteTemps RMSE_val 2.00245 2.00245 0.0%
YosemiteTemps Loss_val 0.00077 0.00077 0.0%
YosemiteTemps MAE 1.3192 1.3192 0.0%
YosemiteTemps RMSE 2.13518 2.13518 0.0%
YosemiteTemps Loss 0.00064 0.00064 0.0%
YosemiteTemps time 73.8248 77.55 5.05% ⚠️
PeytonManning MAE_val 0.58159 0.58159 0.0%
PeytonManning RMSE_val 0.72216 0.72216 0.0%
PeytonManning Loss_val 0.01239 0.01239 0.0%
PeytonManning MAE 0.41671 0.41671 0.0%
PeytonManning RMSE 0.55961 0.55961 0.0%
PeytonManning Loss 0.00612 0.00612 0.0%
PeytonManning time 14.4692 14.31 -1.1%
Model training plots

Model Training

PeytonManning

YosemiteTemps

AirPassengers

@codecov-commenter
Copy link

codecov-commenter commented Apr 21, 2023

Codecov Report

Merging #1298 (fc6c463) into main (edf8bf5) will increase coverage by 0.00%.
The diff coverage is 100.00%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@           Coverage Diff           @@
##             main    #1298   +/-   ##
=======================================
  Coverage   89.85%   89.86%           
=======================================
  Files          38       38           
  Lines        5128     5131    +3     
=======================================
+ Hits         4608     4611    +3     
  Misses        520      520           
Impacted Files Coverage Δ
neuralprophet/time_net.py 90.86% <100.00%> (+0.06%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@judussoari judussoari requested a review from leoniewgnr April 21, 2023 02:04
@@ -793,6 +793,9 @@ def test_step(self, batch, batch_idx):
loss, reg_loss = self.loss_func(inputs, predicted, targets)
# Metrics
if self.metrics_enabled:
predicted_denorm = self.denormalize(predicted[:, :, 0])
target_denorm = self.denormalize(targets.squeeze(dim=2))
self.log_dict(self.metrics_val(predicted_denorm, target_denorm), **self.log_args)
self.log("Loss_test", loss, **self.log_args)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to log (and eventually return) the error metrics specified by the user. those 3 lines are in validation_step and training_step, but (apparently) have been forgottten here

Copy link
Collaborator

@leoniewgnr leoniewgnr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@leoniewgnr leoniewgnr merged commit 326c1d2 into main Apr 21, 2023
@leoniewgnr leoniewgnr deleted the crossval_tut branch April 21, 2023 04:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: needs review PR needs to be reviewed by Reviewer(s) type:bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

cross-validation demo error
3 participants