Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation #48

Merged
merged 2 commits into from
Dec 9, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 17 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,13 @@

`ConformalPrediction.jl` is a package for Uncertainty Quantification (UQ) through Conformal Prediction (CP) in Julia. It is designed to work with supervised models trained in [MLJ](https://alan-turing-institute.github.io/MLJ.jl/dev/) (Blaom et al. 2020). Conformal Prediction is distribution-free, easy-to-understand, easy-to-use and model-agnostic.

# 📖 Background
## 🏃 Quick Tour

> First time here? Take a quick interactive [tour](https://binder.plutojl.org/v0.19.12/open?url=https%253A%252F%252Fraw.githubusercontent.com%252Fpat-alt%252FConformalPrediction.jl%252Fmain%252Fdev%252Fquick_tour%252Fnotebook.jl) to see what this package can do.

The [link](https://binder.plutojl.org/v0.19.12/open?url=https%253A%252F%252Fraw.githubusercontent.com%252Fpat-alt%252FConformalPrediction.jl%252Fmain%252Fdev%252Fquick_tour%252Fnotebook.jl) takes you to a `Pluto.jl` notebook hosted on binder.org. In my own experience, this may take a while to load, certainly enough time to get yourself a hot beverage ☕. Alternatively, you can clone this repo and run the notebook contained in `dev/quick-tour` locally or skip the tour for now and read on here.

## 📖 Background

Conformal Prediction is a scalable frequentist approach to uncertainty quantification and coverage control. It promises to be an easy-to-understand, distribution-free and model-agnostic way to generate statistically rigorous uncertainty estimates. Interestingly, it can even be used to complement Bayesian methods.

Expand Down Expand Up @@ -82,13 +88,13 @@ ŷ[1:show_first]
```

5-element Vector{Tuple{Float64, Float64}}:
(0.325476135568554, 2.5420611849529986)
(-0.8093221495344456, 1.513229072277355)
(0.24246467414510378, 2.531848511672005)
(-0.37629465789570005, 1.9144457517084361)
(-0.5411423339519135, 1.712803571302072)
(0.5113539995719073, 2.7791173590180245)
(0.15501260477711076, 2.491986075800726)
(-0.32783606947941524, 1.9302674946467009)
(-0.13732511816023366, 2.141708832043786)
(0.5089900787456267, 2.7771571126470387)

For simple models like this one, we can call `Plots.plot` on our instance, fit result and data to generate the chart below:
For simple models like this one, we can call a custom `Plots` recipe on our instance, fit result and data to generate the chart below:

``` julia
using Plots
Expand Down Expand Up @@ -117,13 +123,13 @@ println("SSC: $(round(_eval.measurement[2], digits=3))")
┌───────────────────────────────────────────────────────────┬───────────┬───────
│ measure │ operation │ meas ⋯
├───────────────────────────────────────────────────────────┼───────────┼───────
│ emp_coverage (generic function with 1 method) │ predict │ 0.95
│ size_stratified_coverage (generic function with 1 method) │ predict │ 0.90
│ emp_coverage (generic function with 1 method) │ predict │ 0.94
│ size_stratified_coverage (generic function with 1 method) │ predict │ 0.76
└───────────────────────────────────────────────────────────┴───────────┴───────
3 columns omitted

Empirical coverage: 0.957
SSC: 0.907
Empirical coverage: 0.947
SSC: 0.762

## 🔁 Status

Expand Down
152 changes: 79 additions & 73 deletions README_files/figure-commonmark/cell-11-output-1.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
823 changes: 440 additions & 383 deletions README_files/figure-commonmark/cell-7-output-1.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion _freeze/docs/src/index/execute-results/md.json

Large diffs are not rendered by default.

152 changes: 73 additions & 79 deletions _freeze/docs/src/index/figure-commonmark/cell-11-output-1.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
823 changes: 440 additions & 383 deletions _freeze/docs/src/index/figure-commonmark/cell-7-output-1.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
8 changes: 7 additions & 1 deletion docs/src/_intro.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,13 @@ using ConformalPrediction

`ConformalPrediction.jl` is a package for Uncertainty Quantification (UQ) through Conformal Prediction (CP) in Julia. It is designed to work with supervised models trained in [MLJ](https://alan-turing-institute.github.io/MLJ.jl/dev/) [@blaom2020mlj]. Conformal Prediction is distribution-free, easy-to-understand, easy-to-use and model-agnostic.

# 📖 Background
## 🏃 Quick Tour

> First time here? Take a quick interactive [tour](https://binder.plutojl.org/v0.19.12/open?url=https%253A%252F%252Fraw.githubusercontent.com%252Fpat-alt%252FConformalPrediction.jl%252Fmain%252Fdev%252Fquick_tour%252Fnotebook.jl) to see what this package can do.

The [link](https://binder.plutojl.org/v0.19.12/open?url=https%253A%252F%252Fraw.githubusercontent.com%252Fpat-alt%252FConformalPrediction.jl%252Fmain%252Fdev%252Fquick_tour%252Fnotebook.jl) takes you to a `Pluto.jl` notebook hosted on binder.org. In my own experience, this may take a while to load, certainly enough time to get yourself a hot beverage ☕. Alternatively, you can clone this repo and run the notebook contained in `dev/quick-tour` locally or skip the tour for now and read on here.

## 📖 Background

Conformal Prediction is a scalable frequentist approach to uncertainty quantification and coverage control. It promises to be an easy-to-understand, distribution-free and model-agnostic way to generate statistically rigorous uncertainty estimates. Interestingly, it can even be used to complement Bayesian methods.

Expand Down
28 changes: 17 additions & 11 deletions docs/src/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,13 @@ Documentation for [ConformalPrediction.jl](https://github.com/pat-alt/ConformalP

`ConformalPrediction.jl` is a package for Uncertainty Quantification (UQ) through Conformal Prediction (CP) in Julia. It is designed to work with supervised models trained in [MLJ](https://alan-turing-institute.github.io/MLJ.jl/dev/) (Blaom et al. 2020). Conformal Prediction is distribution-free, easy-to-understand, easy-to-use and model-agnostic.

# 📖 Background
## 🏃 Quick Tour

> First time here? Take a quick interactive [tour](https://binder.plutojl.org/v0.19.12/open?url=https%253A%252F%252Fraw.githubusercontent.com%252Fpat-alt%252FConformalPrediction.jl%252Fmain%252Fdev%252Fquick_tour%252Fnotebook.jl) to see what this package can do.

The [link](https://binder.plutojl.org/v0.19.12/open?url=https%253A%252F%252Fraw.githubusercontent.com%252Fpat-alt%252FConformalPrediction.jl%252Fmain%252Fdev%252Fquick_tour%252Fnotebook.jl) takes you to a `Pluto.jl` notebook hosted on binder.org. In my own experience, this may take a while to load, certainly enough time to get yourself a hot beverage ☕. Alternatively, you can clone this repo and run the notebook contained in `dev/quick-tour` locally or skip the tour for now and read on here.

## 📖 Background

Conformal Prediction is a scalable frequentist approach to uncertainty quantification and coverage control. It promises to be an easy-to-understand, distribution-free and model-agnostic way to generate statistically rigorous uncertainty estimates. Interestingly, it can even be used to complement Bayesian methods.

Expand Down Expand Up @@ -86,13 +92,13 @@ ŷ[1:show_first]
```

5-element Vector{Tuple{Float64, Float64}}:
(0.5113539995719073, 2.7791173590180245)
(0.15501260477711076, 2.491986075800726)
(-0.32783606947941524, 1.9302674946467009)
(-0.13732511816023366, 2.141708832043786)
(0.5089900787456267, 2.7771571126470387)
(0.325476135568554, 2.5420611849529986)
(-0.8093221495344456, 1.513229072277355)
(0.24246467414510378, 2.531848511672005)
(-0.37629465789570005, 1.9144457517084361)
(-0.5411423339519135, 1.712803571302072)

For simple models like this one, we can call `Plots.plot` on our instance, fit result and data to generate the chart below:
For simple models like this one, we can call a custom `Plots` recipe on our instance, fit result and data to generate the chart below:

``` julia
using Plots
Expand Down Expand Up @@ -121,13 +127,13 @@ println("SSC: $(round(_eval.measurement[2], digits=3))")
┌───────────────────────────────────────────────────────────┬───────────┬───────
│ measure │ operation │ meas ⋯
├───────────────────────────────────────────────────────────┼───────────┼───────
│ emp_coverage (generic function with 1 method) │ predict │ 0.94
│ size_stratified_coverage (generic function with 1 method) │ predict │ 0.76
│ emp_coverage (generic function with 1 method) │ predict │ 0.95
│ size_stratified_coverage (generic function with 1 method) │ predict │ 0.90
└───────────────────────────────────────────────────────────┴───────────┴───────
3 columns omitted

Empirical coverage: 0.947
SSC: 0.762
Empirical coverage: 0.957
SSC: 0.907

## 🔁 Status

Expand Down
152 changes: 73 additions & 79 deletions docs/src/index_files/figure-commonmark/cell-11-output-1.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading