Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plot() recipe returns unexpected prediction intervals #173

Closed
azev77 opened this issue Jun 15, 2021 · 7 comments
Closed

plot() recipe returns unexpected prediction intervals #173

azev77 opened this issue Jun 15, 2021 · 7 comments

Comments

@azev77
Copy link

azev77 commented Jun 15, 2021

# Load dataset.
using HTTP, CSV, DataFrames, Dates
link = "https://bit.ly/3vtTaW5";
r = HTTP.get(link);
df = CSV.read(r.body, DataFrame; missingstring="NA");

using AbstractGPs, Plots
X = 0:1.0:(length(df.date)-1) 
Y = df.value;
XH = 0:1.0:(length(df.date)-1 +5) 

f = GP(Matern32Kernel()) # Define GP prior w Matern32 kernel
fx = f(X, 0.001)         # Finite projection at the inputs `X` 
logpdf(fx, Y)            # -512 # Data's log-likelihood w.r.t prior GP `f`. 
p_fx = posterior(fx, Y)  # Exact posterior given `Y`.
logpdf(p_fx(X), Y)       # 184 # Data's LL w.r.t prior posterior GP `p_fx`.

px = XH[end-4:end]
pm = mean.(marginals(p_fx(XH))[end-4:end])
ps = std.(marginals(p_fx(XH))[end-4:end])

# Plot posterior.
scatter(X, Y; label="Data")
plot!(XH, f(XH, 0.001); label="Prior")
plot!(p_fx(XH), label="Posterior" )
scatter!(px, pm, lab = "oos prediction μ")
scatter!(px, pm - 1.0*ps, lab="oos prediction μ - σ") 
scatter!(px, pm + 1.0*ps, lab="oos prediction μ + σ") 

image
It looks like the plot() recipe returns μ ± 1.0σ instead of (what I'd expect) μ ± 1.96σ.
Is this intended? Can this be customized?

@azev77
Copy link
Author

azev77 commented Jun 15, 2021

Looks like it might be set to 1.0 here:

scale::Float64 = pop!(plotattributes, :ribbon_scale, 1.0)

Not sure & idk how to modify it...

@theogf
Copy link
Member

theogf commented Jun 15, 2021

The 1.96sigma corresponds to the 95%quantile but there is no particular reason to prefer it to just 1sigma or 2 sigma.
What would you expect 1.96 in particular?

@azev77
Copy link
Author

azev77 commented Jun 15, 2021

What would you expect 1.96 in particular?

Not sure what you mean.

How can I make plot(p_fx(XH) ) return a plot with μ ± 1.96σ?

@theogf
Copy link
Member

theogf commented Jun 15, 2021

If I am not mistaking you can pass the kwarg scale=1.96

@theogf
Copy link
Member

theogf commented Jun 15, 2021

No sorry it should be ribbon_scale=1.96

@azev77
Copy link
Author

azev77 commented Jun 15, 2021

It works now:

    plot(p_fx(XH), lab="oos prediction μ - σ")
    plot!(p_fx(XH),ribbon_scale=1.96, lab="oos prediction μ - 1.96σ")

image

It would be great to make this clearer in the docs

@azev77
Copy link
Author

azev77 commented Jun 15, 2021

While we're here. I asked the following question in other repos:
A recent paper makes 1 year ahead predictions for the distribution of GDP growth, with current GDP growth & NFCI as conditioning variables. Their data & Matlab code is here.
Target variable y_{t+1} = GDP_Growth_{t+1}
predictors X_t =(y_t, NFCI_t) = (GDP_Growth_{t}, NFCI_t)
Goal: predict the conditional distribution P(y_{t+1} | y_{t}, NFCI_t )
image
The authors use quantile regression.
Is it possible to produce this kind of figure w/ AbstractGPs?
Are GPs good at this type of thing rel to NGBoost.py?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants