Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX ignore masked voxels when averaging with nanmean=True #436

Merged
merged 3 commits into from
Apr 7, 2022

Conversation

TomDLT
Copy link
Contributor

@TomDLT TomDLT commented Apr 7, 2022

When averaging over voxels to produce a flatmap, NaNs can be ignored using nanmean=True.
However, if the data is masked (e.g. with a "thin" mask), masked voxels are not ignored like NaNs.

Example with data filled with 1s and NaNs:
"thick" mask: correct behavior, averaging 1s into 1s, and NaNs into NaNs.
thick

"thin" mask: incorrect behavior, averaging 1s into value in [0, 1].
thin

The fix is to use masked_data.filled(), which fills masked numpy arrays with the value in fill_vallue.

import cortex
import numpy as np
from matplotlib import pyplot as plt

subject = 'S1'
transform = 'fullhead'

for type_ in ["thin", "thick"]:
    # create dataset with volume from 0 to n_voxels
    mask = cortex.db.get_mask(subject, transform, type=type_)
    vol = cortex.Volume(np.ones(mask.sum()), subject, transform, cmap="viridis", vmin=0, vmax=1)

    # set 50% of the values in the dataset to NaN
    vol.data[np.random.rand(*vol.data.shape) > 0.5] = np.nan

    _ = cortex.quickshow(vol, nanmean=True, with_curvature=False)
    plt.show()

@TomDLT
Copy link
Contributor Author

TomDLT commented Apr 7, 2022

Actually, this fix is incomplete, because the same problem arises with nanmean=False:
thick mask:
thick
thin mask:
thin

@TomDLT
Copy link
Contributor Author

TomDLT commented Apr 7, 2022

Full fix
Before:
before
After:
after

@mvdoc
Copy link
Contributor

mvdoc commented Apr 7, 2022

Thanks for this! the docs/tests are failing, and It looks like it might be failing when a non-masked numpy array is passed

@mvdoc
Copy link
Contributor

mvdoc commented Apr 7, 2022

awesome, thank you!

@mvdoc mvdoc merged commit 1f23d57 into gallantlab:main Apr 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants