Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DOC] - Tutorial on long-horizon forecasting #249

Merged
merged 10 commits into from
Mar 14, 2024
Merged

[DOC] - Tutorial on long-horizon forecasting #249

merged 10 commits into from
Mar 14, 2024

Conversation

marcopeix
Copy link
Contributor

Tutorial notebook on forecasting over a long horizon with TimeGPT.

  • Use datasetsforecast to load Etth1
  • Specifiy the use of timegpt-1-long-horizon for that kind of tasks
  • Add datasetforecast to dev dependencies

@review-notebook-app Review Notebook App
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

@github-actions GitHub Actions
Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 5.4504 5.5985 0.0093 0.0054

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 16.0417 5.4112 0.0063 0.0054

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 4.9052 5.9784 0.0085 0.0073

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 4.641 5.617 0.0082 0.0077

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 8.1452 3.977 0.0083 0.0077

Plot:

@github-actions GitHub Actions
Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 4.2467 5.8721 0.1429 0.0063

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 11.536 12.9846 0.0067 0.0056

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 8.2228 9.5256 0.0087 0.0095

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966295 422332 656723 3.17316e+06
total_time 12.5605 6.1742 0.0083 0.0078

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 6.5188 6.4966 0.0091 0.0085

Plot:

@github-actions GitHub Actions
Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 17.5745 4.1964 0.0104 0.0058

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 6.7032 5.6531 0.0067 0.0057

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 3.984 4.3506 0.009 0.008

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 6.8321 8.9144 0.0092 0.0083

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805038 441118 1.61572e+06 6.04619e+06
total_time 13.5449 8.2906 0.0087 0.0086

Plot:

@github-actions GitHub Actions
Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 3.6923 5.5208 0.0088 0.0052

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 3.5655 3.5594 0.0064 0.0054

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 4.48 4.3472 0.0087 0.0076

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 5.5542 7.707 0.0085 0.0079

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 13.4953 13.077 0.0087 0.0081

Plot:

@github-actions GitHub Actions
Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 3.2647 3.3469 0.0124 0.007

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 5.4638 6.8178 0.0062 0.0053

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 14.4588 12.07 0.0085 0.0075

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 4.5423 4.2988 0.0081 0.0075

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 5.7376 5.4766 0.0082 0.0077

Plot:

@github-actions GitHub Actions
Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.935 199.132 2571.33 10604.2
total_time 3.9161 9.0803 0.0098 0.0059

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.22 4110.79 5928.17 18859.2
total_time 14.8511 12.0899 0.0063 0.0055

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.8 123119 213677 4.68961e+06
total_time 8.7012 11.8984 0.0086 0.0076

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966295 422332 656723 3.17316e+06
total_time 12.3019 5.0175 0.0084 0.0078

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805038 441118 1.61572e+06 6.04619e+06
total_time 8.3822 10.1325 0.0085 0.0083

Plot:

@marcopeix
Copy link
Contributor Author

Updates:

  • Include confidence level of TimeGPT forecasts
  • Comparison with actual values
  • Use fine-tuning
  • Modified paragraphs according to Max's comments

Let me know if I missed something!

@AzulGarza AzulGarza self-requested a review March 13, 2024 21:08
@AzulGarza AzulGarza merged commit 21d23aa into main Mar 14, 2024
12 checks passed
@AzulGarza AzulGarza deleted the feature/timegpt_longhorizon_tutorial branch March 14, 2024 15:49
marcopeix added a commit that referenced this pull request Mar 14, 2024
@marcopeix
[DOC] - Tutorial on long-horizon forecasting (#249)
71d70a7
MMenchero pushed a commit that referenced this pull request Apr 23, 2024
@marcopeix @MMenchero
[DOC] - Tutorial on long-horizon forecasting (#249)
294e8a8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mergenthaler mergenthaler mergenthaler left review comments

@AzulGarza AzulGarza AzulGarza approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@marcopeix @mergenthaler @AzulGarza