Cannot plot prior on corner plot due to missing atribute nlive

[Jammy2211]

Describe the bug

If have adapted the following code from the Quickstart tutorial (https://anesthetic.readthedocs.io/en/latest/quickstart.html):

from anesthetic import read_chains, make_2d_axes
samples = read_chains("../../tests/example_data/pc_250")
prior = samples.prior()
params = ['x0', 'x1', 'x2', 'x3', 'x4']
fig, axes = make_2d_axes(params, figsize=(6, 6), facecolor='w')
prior.plot_2d(axes, alpha=0.9, label="prior")
samples.plot_2d(axes, alpha=0.9, label="posterior")
axes.iloc[-1, 0].legend(bbox_to_anchor=(len(axes)/2, len(axes)), loc='lower center', ncols=2)

The code reads (note the prior statements are commented out):

        samples = NestedSamples(
            np.asarray(self.samples.parameter_lists),
            weights=self.samples.weight_list,
            columns=self.model.parameter_labels_with_superscripts_latex
        )

        fig, axes = make_2d_axes(
            self.model.parameter_labels_with_superscripts_latex,
            figsize=(12, 12),
            facecolor="k"
        )

        # prior = samples.prior()
        # prior.plot_2d(axes, alpha=0.9, label="prior")
        samples.plot_2d(
            axes,
            alpha=0.9
            label="posterior",
        )
        axes.iloc[-1, 0].legend(
            bbox_to_anchor=(len(axes) / 2, len(axes)),
            loc='lower center',
            ncols=2
        )

Provided the prior statements are commented out, the code works and I get the expected posterior plot.

However, if I try to also plot the prior, I get the following error:

  File "/home/jammy/venvs/PyAuto/lib/python3.10/site-packages/pandas/core/generic.py", line 6236, in __setattr__
    object.__setattr__(self, name, value)
  File "/home/jammy/venvs/PyAuto/lib/python3.10/site-packages/anesthetic/samples.py", line 706, in beta
    logw = self.logw(beta=beta)
  File "/home/jammy/venvs/PyAuto/lib/python3.10/site-packages/anesthetic/samples.py", line 972, in logw
    logdX = self.logdX(nsamples)
  File "/home/jammy/venvs/PyAuto/lib/python3.10/site-packages/anesthetic/samples.py", line 903, in logdX
    logX = self.logX(nsamples)
  File "/home/jammy/venvs/PyAuto/lib/python3.10/site-packages/anesthetic/samples.py", line 868, in logX
    t = np.log(self.nlive/(self.nlive+1))
  File "/home/jammy/venvs/PyAuto/lib/python3.10/site-packages/pandas/core/generic.py", line 6204, in __getattr__
    return object.__getattribute__(self, name)
AttributeError: 'NestedSamples' object has no attribute 'nlive'

To Reproduce
Steps to reproduce the behaviour.

from anesthetic.samples import NestedSamples
from anesthetic import make_2d_axes

import numpy as np

parameter_lists = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
weight_list = [0.1, 0.2, 0.3]

samples = NestedSamples(
    np.asarray(parameter_lists),
    weights=weight_list,
#    columns=self.model.parameter_labels_with_superscripts_latex
)

fig, axes = make_2d_axes(
    ["x", "y", "z"],
    figsize=(12, 12),
    facecolor="k"
)

prior = samples.prior()
prior.plot_2d(axes, alpha=0.9, label="prior")
samples.plot_2d(
    axes,
    alpha=0.9,
    label="posterior",
)
axes.iloc[-1, 0].legend(
    bbox_to_anchor=(len(axes) / 2, len(axes)),
    loc='lower center',
    ncols=2
)

Expected behavior

Priors to appear in plot, as shown on readthedocs.

[williamjameshandley]

Hi @Jammy2211, many thanks for the detailed reproducible example!

The issue is arising because in order to use the .prior() function you need dead points from a nested sampling run. At the moment it looks like your samples would be better suited to a Samples or MCMCSamples object.

At minimum a nested sampling run should contain:

• dead points (the list of discarded live points)
• logL (the loglikelihood values associated with each dead point)
• nlive (the number of live points active at each iteration
  sorted in logL order.

The best way to achieve this is to initialise with a logL and a logL_birth argument to NestedSamples, but if logL_birth is not recorded by your sampler, you can provide logL_birth=nlive.

What nested sampling algorithm has produced your samples?

[Jammy2211]

The sampler was the nested sampler nautilus (https://nautilus-sampler.readthedocs.io/en/stable/), which as you say does not produce dead points (unlike most nested samplers).

I can easily supply log likelihood values and nlive, but it sounds like I should try an MCMCSamples object -- will this reliably get the priors out even without dead points?

[williamjameshandley]

You won't be able to get prior samples with MCMCSamples -- One of the unique selling points of nested sampling that you get every temperature for free between prior and posterior.

To get this to work we would need to get nautilus output to talk to anesthetic, which would mean implementing a new ImportanceNestedSamples class (and ideally a nautilus reader), which on the face of it should not be that tricky. If you would like to work on this together as a pull request, this would merit authorship on the next anesthetic 2.0 paper.

[williamjameshandley]

@johannesulf do you have any input here?

[johannesulf]

Sorry, I'm not very familiar with the method discussed here which, as far as I understand, is related to obtaining an estimate of the prior from a nested sampling run. However, I'd assume one can do that with nautilus by taking the posterior sample from the run and then adjusting the weights according to log_w -> log_w - log_l. That should convert the posterior sample into a prior sample. Does that make sense? Sorry, I haven't used anesthetic myself and may have misunderstood the objectives here.