You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We want to have tests that check different inference scenarios for models.
Proposal
Add tests in test_inference on:
Forecast on subset of segments:
Check that forecast on subset of segments is equal to subset of forecast on all segments.
Predict on subset of segments.
Check that predict on subset of segments is equal to subset of predict on all segments.
Forecast on new segments.
Check that forecast fills the target without fail.
Predict on new segments.
Check that predict fills the target without fail (target will be already filled, but we want to check that it doesn't fail).
Predict on future data with gap (we can try to do it like with forecast: TestForecastOutSamplePrefix, TestForecastOutSampleSuffix)
Check that prediction on prefix of future dataset is the same as prefix of prediction on bigger future dataset.
Check that prediction on suffix of future dataset is the same as suffix of prediction on bigger future dataset.
If some tests are failed for some models mark this tests as to_be_fixed if we want to fix them. If we understand that it isn't possible to fix this (model's properties) make a separate test case for such models.
We have already tested
How forecast works on data in future with gap: TestForecastOutSampleSuffix.
How predict works on data right after train: TestPredictOutSample.
Test cases
No response
Additional context
No response
The text was updated successfully, but these errors were encountered:
🚀 Feature Request
We want to have tests that check different inference scenarios for models.
Proposal
Add tests in
test_inference
on:forecast
:TestForecastOutSamplePrefix
,TestForecastOutSampleSuffix
)If some tests are failed for some models mark this tests as
to_be_fixed
if we want to fix them. If we understand that it isn't possible to fix this (model's properties) make a separate test case for such models.We have already tested
forecast
works on data in future with gap:TestForecastOutSampleSuffix
.predict
works on data right after train:TestPredictOutSample
.Test cases
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: