Description of effective sample size in paper

[matt-graham]

Submitting as part of JOSS review openjournals/joss-reviews#3397

ecological-inference/paper/paper.md

Line 68 in 2d47262

First, the Bayesian hierarchical methods implemented in `PyEI` rest on modern probabilistic programming tooling [@salvatier2016probabilistic] and gradient-based MCMC methods such as the No U-Turn Sampler (NUTS) [@hoffman2014no]. Using NUTS where possible should allow for faster convergence than existing implementations that rest primarily on Metropolis-Hastings and Gibbs sampling steps. Consider effective sample size, which is a measure of how the variance of the mean of drawn samples compare to the variance of i.i.d. samples from the posterior distribution (or, very roughly, how “effective” the samples are for computing the posterior mean, compared to i.i.d. samples) [@BDA3]. In Metropolis-Hastings, the number of evaluations of the log-posterior required for a given effective sample size scales linearly with the dimensionality of the parameter space, while in Hamiltonian Monte Carlo approaches such as NUTS, the number of required evaluations of the gradient of the log-posterior scales only as the fourth root of the dimension [@neal2011mcmc]. Reasonable scaling with the dimensionality of the parameter space is important in ecological inference, as that dimensionality is large when there are many precincts.

In the description of effective sample size (lines 62--65 in proof PDF), I think it would be better to refer to estimates of posterior expectations in general rather than the posterior mean in particular, as effective sample sizes can and commonly are computed for other quantities such as the posterior variances. For example, something like: 'Consider effective sample size, which is a measure of how the variance of a Monte Carlo estimate of a posterior expectation computed from dependent samples compares to the variance of a corresponding estimate computed from independent samples (or, very roughly, how "effective" the samples are for estimating a posterior expectation compared to independent samples)'.

[matt-graham]

Changes to text made in 9430426 look good to me so closing.