Tophat filter doesn't work with float16 output

[asarnow]

I have scipy 1.13 and scipy.ndimage.white_tophat throws RuntimeError: array type not supported at pytom_tm/extract.py:72. If I add score_volume.astype(float) in the call, it runs but no particles are extracted (star file is empty). My tomograms are float16 and so are the score volumes.

The white_tophat in cupy actually works with float16, maybe that is a solution It doesn't work with a structuring element. (Or casting to float for scipy, but maybe something has to be rescaled).

The command I tried (with or without .astype(float).
pytom_extract_candidates.py -j pytom/matching/23dec02b_ts10_10.96Apx_job.json -c 0 -n 5000 --number-of-false-positives 1 -r 6 --log info --tophat-filter

[McHaillet]

Hi @asarnow, thanks for reporting and a possible fix! Casting to float32 sounds like a good solution. If you want you can make a pull request to solve the issue, otherwise I will make one. Let me know!

[rdrighetto]

Just came here to say that I get the same error on float16 scoremaps, and the fix proposed indeed results in empty STAR files.
I do get a bunch of warnings:

WARNING:root:Voxel size annotation in MRC is slightly different between dimensions, namely (7.92, 7.92, 7.9199996). It might be a tiny numerical inaccuracy, but please ensure this is not problematic.
WARNING: Provided voxel size does not match voxel size annotated in tomogram/template mrc.
/scicore/projects/scicore-p-structsoft/ubuntu/software/Miniconda3/miniconda3_python3.12.1/envs/pytom-tm-ub/lib/python3.12/site-packages/numpy/core/_methods.py:152: RuntimeWarning: overflow encountered in reduce
  arrmean = umr_sum(arr, axis, dtype, keepdims=True, where=where)
/scicore/projects/scicore-p-structsoft/ubuntu/software/Miniconda3/miniconda3_python3.12.1/envs/pytom-tm-ub/lib/python3.12/site-packages/pytom_tm/extract.py:112: OptimizeWarning: Covariance of the parameters could not be estimated
  coeff = curve_fit(gauss, x_fit, y_fit, p0=guess)[
/scicore/projects/scicore-p-structsoft/ubuntu/software/Miniconda3/miniconda3_python3.12.1/envs/pytom-tm-ub/lib/python3.12/site-packages/pytom_tm/extract.py:107: RuntimeWarning: invalid value encountered in log
  return np.log(gauss(x, amp, mu, sigma))
/scicore/projects/scicore-p-structsoft/ubuntu/software/Miniconda3/miniconda3_python3.12.1/envs/pytom-tm-ub/lib/python3.12/site-packages/pytom_tm/extract.py:115: OptimizeWarning: Covariance of the parameters could not be estimated
  coeff_log = curve_fit(log_gauss, x_fit, np.log(y_fit), p0=coeff)[
/scicore/projects/scicore-p-structsoft/ubuntu/software/Miniconda3/miniconda3_python3.12.1/envs/pytom-tm-ub/lib/python3.12/site-packages/pytom_tm/extract.py:123: RuntimeWarning: divide by zero encountered in scalar divide
  erfcinv((2 * n_false_positives) / search_space) * np.sqrt(2) * coeff_log[2]
INFO:root:cut off for particle extraction: 0.06453510767349378
  0%|                                                                                                                                                                 | 0/100 [00:00<?, ?it/s]
/scicore/projects/scicore-p-structsoft/ubuntu/software/Miniconda3/miniconda3_python3.12.1/envs/pytom-tm-ub/lib/python3.12/site-packages/pytom_tm/extract.py:380: UserWarning: Attempting to set identical low and high ylims makes transformation singular; automatically expanding.
  ax.set_ylim(bottom=0, top=2 * max(y))

[asarnow]

That's why I said something might have to be rescaled. But what we should really do is add the float16 type to scipy, I think we only need to add an additional set of type specialization annotations to scipy.ndimage.white_tophat.