-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conformal inference documentation #23
Conversation
✅ Deploy Preview for tidymodels-org ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
Merge commit 'af9341d2c4f6496b99e60750f73bd7120711f610' #Conflicts: # docs/404.html # docs/about/davis/index.html # docs/about/index.html # docs/about/max/index.html # docs/books/fes/index.html # docs/books/index.html # docs/books/moderndive/index.html # docs/books/smltar/index.html # docs/books/tidytext/index.html # docs/books/tmwr/index.html # docs/contribute/index.html # docs/find/all/index.html # docs/find/broom/index.html # docs/find/index.html # docs/find/parsnip/index.html # docs/find/recipes/index.html # docs/help/index.html # docs/index.html # docs/learn/develop/broom/index.html # docs/learn/develop/metrics/index.html # docs/learn/develop/models/index.html # docs/learn/develop/parameters/index.html # docs/learn/develop/recipes/index.html # docs/learn/index.html # docs/learn/models/calibration/index.html # docs/learn/models/coefficients/index.html # docs/learn/models/parsnip-nnet/index.html # docs/learn/models/parsnip-ranger-glmnet/index.html # docs/learn/models/pls/index.html # docs/learn/models/sub-sampling/index.html # docs/learn/models/time-series/index.html # docs/learn/statistics/bootstrap/index.html # docs/learn/statistics/infer/index.html # docs/learn/statistics/k-means/index.html # docs/learn/statistics/tidy-analysis/index.html # docs/learn/statistics/xtabs/index.html # docs/learn/work/bayes-opt/index.html # docs/learn/work/case-weights/index.html # docs/learn/work/nested-resampling/index.html # docs/learn/work/tune-svm/index.html # docs/learn/work/tune-text/index.html # docs/packages/index.html # docs/search.json # docs/start/case-study/index.html # docs/start/index.html # docs/start/models/index.html # docs/start/recipes/index.html # docs/start/resampling/index.html # docs/start/tuning/index.html
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking good! a number of comments
Once we are happy with it, I can re-render it with the right quarto version so the diff is smaller |
Co-authored-by: Emil Hvitfeldt <[email protected]>
Co-authored-by: Emil Hvitfeldt <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One very stats-y question and a couple of comments. I like the visualizations, I think that makes it all pretty easy to follow!
geom_vline(xintercept = quants, col = "red", lty = 2) | ||
``` | ||
|
||
If we were to get a new sample beyond these data, we would be able to say that we are about 90% sure that the new sample does not conform to the original distribution. This works under the assumption that the data are exchangeable. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we were to get a new sample beyond these data, we would be able to say that we are about 90% sure that the new sample does not conform to the original distribution. This works under the assumption that the data are exchangeable. | |
If we were to get a new sample beyond these boundaries, we would be able to say that we are about 90% sure that the new sample does not conform to the original distribution. This works under the assumption that the data are exchangeable. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm getting stuck on this sentence.
If we are talking confidence intervals, I think that sentence is skirting between "the interval covers the parameter" and "the parameter lies in the interval". But since we are talking about prediction intervals, does this still apply?
What I'm trying to say: is this sentence statistically correct? I can't judge that immediately and my training on how to talk about p-values is kicking in - but it might be misplaced here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I rewrote this (emphasis added here to show the differences):
If we compute a prediction on a new unknown sample, we could center this reference distribution around its predicted value. For some significance level, we now know the range of sample values that "conform" to the variance seen in the reference distribution. That range can define our prediction interval.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The use of "predicted values" instead of "sample values" was incorrect.
Co-authored-by: Hannah Frick <[email protected]>
For regression models.