maxatac prepare erroring out at normalization

[codymowery]

Hello Miraldi Lab! Thanks for generating such a valuable tool, and thanks in advance for your help! I followed installation directions into a dedicated conda environment and began running maxatac prepare on some public ATAC data (i.e. SRR7650829) aligned to hg38 with bowtie2. Two things:

First, I think it'd be helpful to list specific package versions for the conda install. I had some issues with the default samtools version and ended up having to upgrade to 1.15.1 in order for the pre-normalization functions to work properly.

Second, I haven't been able to resolve this error during the normalization step. I appreciate any suggestions you can offer!

Package versions from conda list here:

Name Version Build Channel

_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
absl-py 1.0.0 pypi_0 pypi
astunparse 1.6.3 pypi_0 pypi
bedtools 2.30.0 h468198e_3 bioconda
biopython 1.79 pypi_0 pypi
bzip2 1.0.8 h7f98852_4 conda-forge
c-ares 1.18.1 h7f8727e_0
ca-certificates 2021.10.8 ha878542_0 conda-forge
cachetools 5.0.0 pypi_0 pypi
certifi 2021.10.8 pypi_0 pypi
charset-normalizer 2.0.12 pypi_0 pypi
curl 7.83.1 h7bff187_0 conda-forge
cycler 0.11.0 pypi_0 pypi
flatbuffers 2.0 pypi_0 pypi
fonttools 4.33.3 pypi_0 pypi
gast 0.5.3 pypi_0 pypi
google-auth 2.6.6 pypi_0 pypi
google-auth-oauthlib 0.4.6 pypi_0 pypi
google-pasta 0.2.0 pypi_0 pypi
grpcio 1.46.1 pypi_0 pypi
h5py 3.6.0 pypi_0 pypi
htslib 1.15.1 h9753748_0 bioconda
idna 3.3 pypi_0 pypi
importlib-metadata 4.11.3 pypi_0 pypi
joblib 1.1.0 pypi_0 pypi
keras 2.8.0 pypi_0 pypi
keras-preprocessing 1.1.2 pypi_0 pypi
keyutils 1.6.1 h166bdaf_0 conda-forge
kiwisolver 1.4.2 pypi_0 pypi
krb5 1.19.3 h3790be6_0 conda-forge
ld_impl_linux-64 2.36.1 hea4e1c9_2 conda-forge
libclang 14.0.1 pypi_0 pypi
libcurl 7.83.1 h7bff187_0 conda-forge
libdeflate 1.10 h7f98852_0 conda-forge
libedit 3.1.20210910 h7f8727e_0
libev 4.33 h7f8727e_1
libffi 3.4.2 h7f98852_5 conda-forge
libgcc 7.2.0 h69d50b8_2 conda-forge
libgcc-ng 12.1.0 h8d9b700_16 conda-forge
libgomp 12.1.0 h8d9b700_16 conda-forge
libidn2 2.3.2 h7f8727e_0
libnghttp2 1.47.0 h727a467_0 conda-forge
libnsl 2.0.0 h7f98852_0 conda-forge
libpng 1.6.37 h21135ba_2 conda-forge
libssh2 1.10.0 ha56f1ee_2 conda-forge
libstdcxx-ng 12.1.0 ha89aaad_16 conda-forge
libunistring 0.9.10 h7f98852_0 conda-forge
libuuid 2.32.1 h7f98852_1000 conda-forge
libzlib 1.2.11 h166bdaf_1014 conda-forge
markdown 3.3.7 pypi_0 pypi
matplotlib 3.5.2 pypi_0 pypi
maxatac 1.0.5 pypi_0 pypi
mysql-connector-c 6.1.11 h6eb9d5d_1007 conda-forge
ncurses 6.3 h7f8727e_2
numpy 1.22.3 pypi_0 pypi
oauthlib 3.2.0 pypi_0 pypi
openssl 1.1.1o h7f8727e_0
opt-einsum 3.3.0 pypi_0 pypi
packaging 21.3 pypi_0 pypi
pandas 1.4.2 pypi_0 pypi
pigz 2.6 h27cfd23_0
pillow 9.1.0 pypi_0 pypi
pip 22.1 pyhd8ed1ab_0 conda-forge
protobuf 3.20.1 pypi_0 pypi
py2bit 0.3.0 pypi_0 pypi
pyasn1 0.4.8 pypi_0 pypi
pyasn1-modules 0.2.8 pypi_0 pypi
pybedtools 0.9.0 pypi_0 pypi
pybigwig 0.3.18 pypi_0 pypi
pydot 1.4.2 pypi_0 pypi
pyfiglet 0.8.post1 pypi_0 pypi
pyparsing 3.0.9 pypi_0 pypi
pysam 0.19.0 pypi_0 pypi
python 3.9.12 h9a8a25e_1_cpython conda-forge
python-dateutil 2.8.2 pypi_0 pypi
python_abi 3.9 2_cp39 conda-forge
pytz 2022.1 pypi_0 pypi
pyyaml 6.0 pypi_0 pypi
readline 8.1.2 h7f8727e_1
requests 2.27.1 pypi_0 pypi
requests-oauthlib 1.3.1 pypi_0 pypi
rsa 4.8 pypi_0 pypi
samtools 1.15.1 h1170115_0 bioconda
scikit-learn 1.1.0 pypi_0 pypi
scipy 1.8.0 pypi_0 pypi
seaborn 0.11.2 pypi_0 pypi
setuptools 62.2.0 py39hf3d152e_0 conda-forge
six 1.16.0 pypi_0 pypi
sqlite 3.38.5 h4ff8645_0 conda-forge
tensorboard 2.8.0 pypi_0 pypi
tensorboard-data-server 0.6.1 pypi_0 pypi
tensorboard-plugin-wit 1.8.1 pypi_0 pypi
tensorflow 2.8.0 pypi_0 pypi
tensorflow-io-gcs-filesystem 0.25.0 pypi_0 pypi
termcolor 1.1.0 pypi_0 pypi
tf-estimator-nightly 2.8.0.dev2021122109 pypi_0 pypi
threadpoolctl 3.1.0 pypi_0 pypi
tk 8.6.12 h27826a3_0 conda-forge
typing-extensions 4.2.0 pypi_0 pypi
tzdata 2022a hda174b7_0
ucsc-bedgraphtobigwig 377 ha8a8165_3 bioconda
urllib3 1.26.9 pypi_0 pypi
werkzeug 2.1.2 pypi_0 pypi
wget 1.20.3 ha56f1ee_1 conda-forge
wheel 0.37.1 pyhd8ed1ab_0 conda-forge
wrapt 1.14.1 pypi_0 pypi
xz 5.2.5 h7f8727e_1
zipp 3.8.0 pypi_0 pypi
zlib 1.2.11 h166bdaf_1014 conda-forge

[tacazares]

Hey @codymowery ,
Thanks for posting. I have run into the pybigwig issue before: #87 (comment)

I think it has to do with pybigwig not recognizing or finding numpy when installing.

Test if this is the issue by activating your env. Then you need to open your python interpreter and test if pybigwig can find numpy:

import pyBigWig
pybigwig.numpy

If 0 is returned then you need to try to pip uninstall pybigwig then pip install pybigwig.

I have had this issue on HPC systems so far and a few version of Mac OS.

Let me know if this works!
Tareian

[codymowery]

Thanks @tacazares, and sorry I missed the other issue. That seems to have fixed it. I did have to specify pip install pybigwig --no-cache-dir to avoid re-installing from the problematic package.

Another suggestion -- It would be nice to update the README.md to specify the maxatac prepare output that is subsequently passed to maxatac predict. I was a little confused by maxatac predict --sequence hg38.2bit -tf CTCF --signal GM12878.bigwig -o outputdir/ in the README since maxatac prepare wasn't outputting a ${prefix}.bigwig file. Your schematic figure suggests it's the normalized ${prefix}_IS_slop20_RP20M_minmax01.bw output that goes into maxatac predict, but updating the examples would avoid potential confusion for others.

Thanks again for creating and maintaining the package! I'm super excited to see how the data look.