make online 1d regression forecasting example

[murphyk]

We should try to reproduce the example shown in the screenshot below (see video at https://cbmm.mit.edu/video/scaling-inference-mission starting at 7:16 minutes in), where the data stream in over time, left to right, and we visualize posterior predictive distributions (forecasts) over the future.

Vikash uses SMC over a probabilistic program (unpublished work), which is quite slow and complicated.
(See sampled programs on RHS of his screen.) By contrast, our approach uses deterministic VI-like methods over a parametric model (DNN), which is much faster and simpler. However, to get good results, we need richer priors (eg encoding periodicity), so we can emulate this PPL behavior with our methods. Thus we first need to solve #33 , which does batch (offline) HMC for a suitable time series BNN. Once we have the offline case working, we can move to the harder online case.

Here is an older paper related to this PPL approach.

F. A. Saad, M. F. Cusumano-Towner, U. Schaechtle, M. C. Rinard, and V. K. Mansinghka, “Bayesian synthesis of probabilistic programs for automatic data modeling,” Proc. ACM Program. Lang., vol. 3, no. POPL, pp. 1–32, Jan. 2019, doi: 10.1145/3290350. [Online]. Available: https://doi.org/10.1145/3290350