Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updating Azure and trial information #500

Merged
merged 3 commits into from
Oct 24, 2024
Merged

updating Azure and trial information #500

merged 3 commits into from
Oct 24, 2024

Conversation

tracykteal
Copy link
Contributor

Updating free trial links to dashboard.nixtla.io, rather than dashboard.nixtla.io/freetrial and related FAQs and some Azure getting-started updates.

Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@tracykteal tracykteal requested a review from elephaint October 18, 2024 00:00
Copy link
Contributor

github-actions bot commented Oct 18, 2024

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 7.7417 1.6845 0.0058 0.0043

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.22 4110.79 5928.17 18859.2
total_time 1.0316 1.5451 0.0046 0.0042

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 178.293 268.13 269.23 1331.02
mape 0.0234 0.0311 0.0304 0.1692
mse 121589 219485 213677 4.68961e+06
total_time 0.4423 1.7763 0.0056 0.0057

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 465.497 346.972 398.956 1119.26
mape 0.062 0.0436 0.0512 0.1583
mse 835021 403760 656723 3.17316e+06
total_time 0.4871 0.521 0.0057 0.005

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 558.673 459.757 602.926 1340.95
mape 0.0697 0.0565 0.0787 0.17
mse 1.22723e+06 739114 1.61572e+06 6.04619e+06
total_time 0.6559 1.8978 0.0059 0.0055

Plot:

@tracykteal tracykteal requested a review from carla-cr October 23, 2024 17:15
Copy link

@carla-cr carla-cr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@tracykteal tracykteal requested a review from cchallu October 24, 2024 13:43
@tracykteal
Copy link
Contributor Author

Fixed some links and reviewed live. Set to merge.

@tracykteal tracykteal merged commit 2a07049 into main Oct 24, 2024
6 of 12 checks passed
@tracykteal tracykteal deleted the tkt-docs branch October 24, 2024 20:31
@jmoralez
Copy link
Member

Who reviewed the azure quickstart? The only supported model should be azureai, which is automatically set if an azure endpoint is detected, so there's no need to set it. Also azureai is the same as timegpt-1-long-horizon, so the sections about shorter and longer forecasts should be removed.

@tracykteal
Copy link
Contributor Author

Thanks so much for flagging this @jmoralez. Sorry, things were working for me when I set the model as 'azureai' and wasn't working when I wasn't. I'll re-check my setup to make sure it's working without it, and take out the short and longer forecasts section.

@jmoralez
Copy link
Member

Were you using a real azure endpoint? I think the override doesn't warn, so behind the scenes we change anything you provided as model and set it to azureai, so you wouldn't see anything, we probably need to add a warning there.

Can you also add #| eval: false to the cells in that tutorial? We run the tests against our own deployment so the CI is failing with errors like this (sample logs):

ValueError: unsupported model: azureai. supported models: ['timegpt-1', 'timegpt-1-long-horizon']

@tracykteal
Copy link
Contributor Author

I thought I was using a real azure endpoint, but maybe that's the issue. I'll check with you on what I'm using, and see if I should be using something else.

Thanks, yes, I'll add in that #| eval: false. I'll put that in as its own PR, so the tests will start passing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants