ImportError: cannot import name 'cached_property' from 'functools'

[AnyaKovalenko]

Dear Community,

please can you help with the following issue: <ImportError: cannot import name 'cached_property' from 'functools'>. Please see bellow:

$ homopolish polish -a barcodeXX.consensus.fasta -m R10.3.pkl -l reference_list_for_homopolish/virus.fasta -t 4 -o barcodeXX_homopolish 

Traceback (most recent call last):
  File "/home/annako/miniconda3/envs/homopolish/bin/homopolish", line 6, in <module>
    from homopolish.homopolish import main
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.7/site-packages/homopolish/homopolish.py", line 4, in <module>
    from modules.polish_interface import polish_genome
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.7/site-packages/modules/polish_interface.py", line 11, in <module>
    from modules import polish
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.7/site-packages/modules/polish.py", line 5, in <module>
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.7/site-packages/more_itertools/__init__.py", line 3, in <module>
    from .more import *  # noqa
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.7/site-packages/more_itertools/more.py", line 5, in <module>
    from functools import cached_property, partial, reduce, wraps
ImportError: cannot import name 'cached_property' from 'functools' (/home/annako/miniconda3/envs/homopolish/lib/python3.7/functools.py)

Has anyone faced with the same issue? please help me to find out how to resolve this. Many thanks and very looking forward to receiving the feedback! Best wishes!

[ythuang0522]

@AnyaKovalenko Can you upgrade your python to 3.8 or after? The cached_property seems introduced at 3.8. If still unresolved, can you provide the draft genome for us to debug?

[AnyaKovalenko]

Hi @ythuang0522 thank you. I updated python to 3.8 and this helped. But now another error <'ModuleNotFoundError: No module named 'sklearn.svm.classes'>, please see below.

[2023/11/22 05:33] INFO: RUN-ID: NC_039477.1
NC_039477.1
/mnt/d/university/Metagenomics/analysis/BC01_homopolish/debug

[M::mm_idx_gen::0.017*0.21] collected minimizers
[M::mm_idx_gen::0.019*0.30] sorted minimizers
[M::main::0.019*0.31] loaded/built the index for 1 target sequence(s)
[M::mm_mapopt_update::0.019*0.31] mid_occ = 50
[M::mm_idx_stat] kmer size: 19; skip: 19; is_hpc: 0; #seq: 1
[M::mm_idx_stat::0.019*0.32] distinct minimizers: 762 (100.00% are singletons); average occurrences: 1.000; average spacing: 9.920; total length: 7559
[M::worker_pipeline::4.761*2.66] mapped 2890 sequences
[M::main] Version: 2.26-r1175
[M::main] CMD: minimap2 -cx asm5 --cs=long -t 4 /mnt/d/university/Metagenomics/analysis/BC01_homopolish/debug/NC_039477.1/NC_039477.1.fasta /mnt/d/university/Metagenomics/analysis/BC01_homopolish/debug/NC_039477.1/All_homologous_sequences.fna.gz
[M::main] Real time: 4.763 sec; CPU: 12.689 sec; Peak RSS: 0.424 GB
4.811844825744629
[2023/11/22 05:33] INFO: Stage: Homologous retrieval
TIME Homologous retrieval: 0 MINS 3 SECS.
[2023/11/22 05:34] INFO: Stage: Prediction
Traceback (most recent call last):
  File "/home/annako/miniconda3/envs/homopolish/bin/homopolish", line 10, in <module>
    sys.exit(main())
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/site-packages/homopolish/homopolish.py", line 50, in main
    polish_genome(FLAGS.mash_screen, FLAGS.assembly, __pkg_path__, FLAGS.sketch_path, FLAGS.genus, FLAGS.threads, \
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/site-packages/modules/polish_interface.py", line 326, in polish_genome
    out = local_DB(out, assembly_name, output_dir_debug, mash_screen, assembly, model_path, sketch_path, genus_species, threads, output_dir, minimap_args, mash_threshold, download_contig_nums, debug, meta, local_DB_path)
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/site-packages/modules/polish_interface.py", line 307, in local_DB
    out.append(check_homopolish(paf, contig_name, contig_output_dir, contig, minimap_args, threads, db_path, model_path))
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/site-packages/modules/polish_interface.py", line 131, in check_homopolish
    finish = homopolish(contig_name, minimap_args, threads, db_path, model_path, contig_output_dir, dataframe)
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/site-packages/modules/polish_interface.py", line 91, in homopolish
    result = prediction.predict(dataframe, model_path, threads, contig_output_dir)
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/site-packages/modules/prediction.py", line 19, in predict
    model = joblib.load(model)
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 658, in load
    obj = _unpickle(fobj, filename, mmap_mode)
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 577, in _unpickle
    obj = unpickler.load()
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/pickle.py", line 1212, in load
    dispatch[key[0]](self)
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/pickle.py", line 1528, in load_global
    klass = self.find_class(module, name)
  File "/home/annako/miniconda3/envs/homopolish/lib/python3.8/pickle.py", line 1579, in find_class
    __import__(module, level=0)
ModuleNotFoundError: No module named 'sklearn.svm.classes'

Please can you help how to deal with this. Many thanks! Very looking forward to hearing from you.

[AnyaKovalenko]

@ythuang0522 what is the best way to send you the draft genome? can you share your email and I will send through it? thank you

[AnyaKovalenko]

I think the scikit-learn versions used in model and program loading it are different. Isn't it?
The scikit-learn was updated together with the python (the current scikit-learn version is 1.3.0)

[ythuang0522]

Yes. You should use the conda virtual env as the scikit-learn model cannot be used in some versions. Did you use the recommended conda env conda env create -f environment.yml? If so, please send your genome to [email protected]. Thanks.

[AnyaKovalenko]

Many thanks @ythuang0522 I sent the genomes.
I use conda env for homopolish, but installed it through the mamba create -n homopolish -c conda-forge -c bioconda homopolish Can it be the issue?

[Mossy-Frog]

Hi !
I know I am late, but if anyone still has this problem, just like I had this morning, a possible solution is to :
mamba remove python
mamba install "python==3.8"
Since it's a problem with the version of python, I uninstalled the previous python(s) and installed one that seemed to be compatible with every packages in homopolish

[AnyaKovalenko]

@Mossy-Frog thank you so much! It's never too late :)

[AnyaKovalenko]

Hi ! I know I am late, but if anyone still has this problem, just like I had this morning, a possible solution is to : mamba remove python mamba install "python==3.8" Since it's a problem with the version of python, I uninstalled the previous python(s) and installed one that seemed to be compatible with every packages in homopolish

Hi @Mossy-Frog I used these and still have an issue. So now the error: homopolish: command not found Did you have the same? thank you

[AnyaKovalenko]

okay, finally it was figured out! Just run :
python3 homopolish.py polish -a yourgenome.fasta -m <select-model> -o youroutput
instead of :
homopolish polish -a yourgenome.fasta -m <select-model> -o youroutput

[Mossy-Frog]

okay, finally it was figured out! Just run : python3 homopolish.py polish -a yourgenome.fasta -m <select-model> -o youroutput instead of : homopolish polish -a yourgenome.fasta -m <select-model> -o youroutput

Huh, weird, I do not have this issue at all, I wonder what causes this ?

[JonathanAbrahams1337]

#62 Similar to my experience too.

Can the README be updated?

[ythuang0522]

#62 Similar to my experience too.

Can the README be updated?

We found several issues associated with the Python version as well as outdated packages in the conda-forge channel.
e.g.,

1. The conda-forge only includes libedit upto 3.1.2017, while some packages required >=3.1.20210910.
2. The cached_property package requires python3.8 or later.

We have updated the installation instructions in README according to the issues we have found.
The following instructs work at our env.
conda create -n homopolish -c conda-forge -c bioconda -c defaults more-itertools=8.4.0 homopolish=0.4.1
or
mamba create -n homopolish_mamba_auto_0_4_1 -c conda-forge -c bioconda -c defaults python==3.8.16 homopolish==0.4.1

Thanks for all the feedback.

[JonathanAbrahams1337]

Great, thanks. Will give it a go.

Perhaps the inclusion of some test data can mean we can run the tool on the same data and test just the environment rather than the data+enviornment together. Will be very helpful for new users to ensure their install/conda env works.

[JonathanAbrahams1337]

I am still having troubles with this, many parts of the files contain errors in my env, due to wrong package versions. Can you post your environment? (conda env export)

name: homopolish_mamba_auto_0_4_1_26.03.2024
channels:
  - conda-forge
  - bioconda
  - defaults
dependencies:
  - _libgcc_mutex=0.1=conda_forge
  - _openmp_mutex=4.5=2_kmp_llvm
  - arrow-cpp=10.0.1=h3e2b116_4_cpu
  - aws-c-auth=0.6.21=hd93a3ba_3
  - aws-c-cal=0.5.20=hd3b2fe5_3
  - aws-c-common=0.8.5=h166bdaf_0
  - aws-c-compression=0.2.16=hf5f93bc_0
  - aws-c-event-stream=0.2.17=h57874a7_1
  - aws-c-http=0.7.0=h96ef541_0
  - aws-c-io=0.13.12=h57ca295_1
  - aws-c-mqtt=0.7.13=h0b5698f_12
  - aws-c-s3=0.2.1=h5d7e488_4
  - aws-c-sdkutils=0.1.7=hf5f93bc_0
  - aws-checksums=0.1.14=h6027aba_0
  - aws-crt-cpp=0.18.16=h26430d7_8
  - aws-sdk-cpp=1.9.379=h56135f1_7
  - biopython=1.83=py38h01eb140_0
  - blas=2.120=openblas
  - blas-devel=3.9.0=20_linux64_openblas
  - boost=1.70.0=py38h9de70de_1
  - boost-cpp=1.70.0=h8e57a91_2
  - brotli=1.0.9=h166bdaf_9
  - brotli-bin=1.0.9=h166bdaf_9
  - brotli-python=1.0.9=py38hfa26641_9
  - brotlipy=0.7.0=py38h01eb140_1006
  - bzip2=1.0.8=hd590300_5
  - c-ares=1.27.0=hd590300_0
  - ca-certificates=2024.2.2=hbcca054_0
  - capnproto=0.10.2=h6239696_0
  - certifi=2024.2.2=pyhd8ed1ab_0
  - cffi=1.16.0=py38h6d47a40_0
  - chardet=5.2.0=py38h578d9bd_1
  - charset-normalizer=3.3.2=pyhd8ed1ab_0
  - colorama=0.4.6=pyhd8ed1ab_0
  - cryptography=39.0.0=py38h1724139_0
  - curl=7.87.0=h6312ad2_0
  - double-conversion=3.3.0=h59595ed_0
  - fastani=1.34=h4dfc31f_1
  - feather-format=0.4.1=pyh9f0ad1d_0
  - gflags=2.2.2=he1b5a44_1004
  - glog=0.6.0=h6f12383_0
  - grpc-cpp=1.51.1=h3e2b116_0
  - gsl=2.7=he838d99_0
  - homopolish=0.4.1=pyhdfd78af_0
  - icu=64.2=he1b5a44_1
  - idna=3.6=pyhd8ed1ab_0
  - joblib=1.3.2=pyhd8ed1ab_0
  - k8=0.2.5=hdcf5f25_4
  - keyutils=1.6.1=h166bdaf_0
  - krb5=1.20.1=hf9c8cef_0
  - ld_impl_linux-64=2.40=h41732ed_0
  - libabseil=20220623.0=cxx17_h05df665_6
  - libarrow=10.0.1=hee49ebd_4_cpu
  - libblas=3.9.0=20_linux64_openblas
  - libbrotlicommon=1.0.9=h166bdaf_9
  - libbrotlidec=1.0.9=h166bdaf_9
  - libbrotlienc=1.0.9=h166bdaf_9
  - libcblas=3.9.0=20_linux64_openblas
  - libcrc32c=1.1.2=h9c3ff4c_0
  - libcurl=7.87.0=h6312ad2_0
  - libdeflate=1.13=h166bdaf_0
  - libedit=3.1.20230828=h5eee18b_0
  - libev=4.33=hd590300_2
  - libevent=2.1.10=h9b69904_4
  - libffi=3.4.2=h7f98852_5
  - libgcc-ng=13.2.0=h807b86a_5
  - libgfortran-ng=13.2.0=h69a702a_5
  - libgfortran5=13.2.0=ha4646dd_5
  - libgoogle-cloud=2.5.0=h5d50b59_1
  - libgrpc=1.51.1=h05bd8bd_0
  - liblapack=3.9.0=20_linux64_openblas
  - liblapacke=3.9.0=20_linux64_openblas
  - libnghttp2=1.51.0=hdcd2b5c_0
  - libopenblas=0.3.25=pthreads_h413a1c8_0
  - libprotobuf=3.21.12=hfc55251_2
  - libsqlite=3.45.2=h2797004_0
  - libssh2=1.10.0=haa6b8db_3
  - libstdcxx-ng=13.2.0=h7e041cc_5
  - libthrift=0.16.0=h491838f_2
  - libutf8proc=2.8.0=h166bdaf_0
  - libzlib=1.2.13=hd590300_5
  - llvm-openmp=18.1.2=h4dfa4b3_0
  - lz4-c=1.9.4=hcb278e6_0
  - mash=2.3=hd3113c8_6
  - minimap2=2.27=he4a0461_1
  - more-itertools=10.2.0=pyhd8ed1ab_0
  - ncbi-datasets-cli=16.10.1=ha770c72_2
  - ncurses=6.4.20240210=h59595ed_0
  - numpy=1.24.3=py38hf838250_0
  - numpy-base=1.24.3=py38h1e6e340_0
  - openblas=0.3.25=pthreads_h7a3da1a_0
  - openssl=1.1.1w=hd590300_0
  - orc=1.8.2=hfdbbad2_2
  - packaging=24.0=pyhd8ed1ab_0
  - pandas=2.0.3=py38h01efb38_1
  - parquet-cpp=1.5.1=2
  - pip=24.0=pyhd8ed1ab_0
  - platformdirs=4.2.0=pyhd8ed1ab_0
  - pooch=1.8.1=pyhd8ed1ab_0
  - pyarrow=10.0.1=py38hf54eea0_4_cpu
  - pycparser=2.21=pyhd8ed1ab_0
  - pycurl=7.45.1=py38h61f0cdf_3
  - pyopenssl=23.2.0=pyhd8ed1ab_1
  - pysam=0.21.0=py38h1c8baaf_0
  - pysocks=1.7.1=pyha2e5f31_6
  - python=3.8.16=h7a1cb2a_3
  - python-dateutil=2.9.0=pyhd8ed1ab_0
  - python-tzdata=2024.1=pyhd8ed1ab_0
  - python-wget=3.2=py_0
  - python_abi=3.8=2_cp38
  - pytz=2024.1=pyhd8ed1ab_0
  - re2=2022.06.01=h27087fc_1
  - readline=8.2=h8228510_1
  - requests=2.31.0=pyhd8ed1ab_0
  - s2n=1.3.31=hae46d1a_0
  - scikit-learn=1.3.2=py38ha25d942_2
  - scipy=1.10.1=py38h59b608b_3
  - setuptools=69.2.0=pyhd8ed1ab_0
  - six=1.16.0=pyh6c4a22f_0
  - snappy=1.1.10=h9fff704_0
  - sqlite=3.45.2=h2c6b66d_0
  - threadpoolctl=3.4.0=pyhc1e730c_0
  - thrift-compiler=0.16.0=h491838f_2
  - thrift-cpp=0.16.0=h3e2b116_2
  - tk=8.6.13=noxft_h4845f30_101
  - tqdm=4.66.2=pyhd8ed1ab_0
  - uriparser=0.9.7=h59595ed_1
  - urllib3=2.2.1=pyhd8ed1ab_0
  - wheel=0.43.0=pyhd8ed1ab_0
  - xz=5.2.10=h5eee18b_1
  - zlib=1.2.13=hd590300_5
  - zstd=1.5.5=hfc55251_0
prefix: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX