Base implementation of SVGP #9
Conversation
src/elbo.jl
Outdated
| fx::FiniteGP{<:AbstractGP, <:AbstractVector, <:Diagonal{<:Real, <:Fill}}, | ||
| y::AbstractVector{<:Real}, | ||
| fz::FiniteGP, | ||
| q::AbstractMvNormal; |
There was a problem hiding this comment.
does this count as type piracy?
maybe we should leave this for separate discussion in an issue / future refactoring, but I'm wondering about whether it'd be helpful to have an object that contains everything needed for p(f | u) [for u=f(Z) that might be fz already, but might need to be something different for interdomain inducing variables] as well as q(u) = q]
There was a problem hiding this comment.
I think refactoring this a little to put more information in a single object would be a good idea, but I'm happy for this to go in as-is for now.
src/elbo.jl
Outdated
| f_mean, f_var = mean_and_var(post, fx.x) | ||
| variational_exp = expected_loglik(method, y, f_mean, f_var, lik) |
There was a problem hiding this comment.
The only reason _elbo needs fx is to access x, right? what do you think of passing x as an explicit argument and writing
| f_mean, f_var = mean_and_var(post, fx.x) | |
| variational_exp = expected_loglik(method, y, f_mean, f_var, lik) | |
| variational_exp = expected_loglik(method, y, post(x), lik) |
- how would that work ?
Co-authored-by: st-- <[email protected]>
|
@st-- @willtebbutt @theogf |
src/elbo.jl
Outdated
| fx::FiniteGP{<:AbstractGP, <:AbstractVector, <:Diagonal{<:Real, <:Fill}}, | ||
| y::AbstractVector{<:Real}, | ||
| fz::FiniteGP, | ||
| q::AbstractMvNormal; |
There was a problem hiding this comment.
I think refactoring this a little to put more information in a single object would be a good idea, but I'm happy for this to go in as-is for now.
This PR has the core implementation of stochastic variational GPs.
The approximate posterior in
svgp.jlquite closely follows that in AbstractGPs.elbo.jlhas the implementations of both the closed form Gaussian likelihood ELBO and quadrature for non-conjugate - I'll add other closed form likelihoods and MC estimation in other PRs.The examples in this PR are extremely rough, and will be superseded soon, but give working examples of how to train the SVGP end to end.