Phylophlan_metagenomic.py : Problem to generate output files

[vindarbot]

Hi,

First of all thank you for this very useful tool.

I have a little problem using phylophlan_metagenomic.py :
3 outputs are supposed to be generated. I have the first two ( output_sketches/ and output_dists) but the third isn't present (output.tsv)

Additionnaly, here's the error message when I try to re launch the script:

/home/vdarbot/.conda/envs/phylophlan/lib/python3.8/site-packages/numpy/core/fromnumeric.py:3334: RuntimeWarning: Mean of empty slice.
  return _methods._mean(a, axis=axis, dtype=dtype,
/home/vdarbot/.conda/envs/phylophlan/lib/python3.8/site-packages/numpy/core/_methods.py:161: RuntimeWarning: invalid value encountered in double_scalars
  ret = ret.dtype.type(ret / rcount)
Traceback (most recent call last):
  File "/home/vdarbot/.conda/envs/phylophlan/bin/phylophlan_metagenomic", line 10, in <module>
    sys.exit(phylophlan_metagenomic())
  File "/home/vdarbot/.conda/envs/phylophlan/lib/python3.8/site-packages/phylophlan/phylophlan_metagenomic.py", line 808, in phylophlan_metagenomic
    binn_2_dists[binn].append(float(rc[2]))
KeyError: ''

Thank's by advance for your help

[fasnicar]

Hi, can you please provide the command line and the log with the --verbose parameter (if you saved it)?

Many thanks,
Francesco

[skye-zhai]

I also met this trouble with ".tsv" file created.
This is my command line:
phylophlan_metagenomic -i ../checkm_bins -o phylophlan_metagenomic --nproc 10 -n 1 -d SGB.Jan19 --verbose 2>&1 | tee phylophlan_metagenomic.log
This is the --verbose output:
[e] Command '['mash', 'dist', '-p', '1', 'phylophlan_databases/SGB.Jan19/16015.msh', 'phylophlan_metagenomic_sketches/phylophlan_metagenomic_paste_1.msh']' died with <Signals.SIGSEGV: 11>.

[e] cannot execute command
mash dist -p 1 phylophlan_databases/SGB.Jan19/16015.msh phylophlan_metagenomic_sketches/phylophlan_metagenomic_paste_1.msh

[e] Command '['mash', 'dist', '-p', '1', 'phylophlan_databases/SGB.Jan19/16015.msh', 'phylophlan_metagenomic_sketches/phylophlan_metagenomic_paste_1.msh']' died with <Signals.SIGSEGV: 11>.

[e] error while disting
['phylophlan_metagenomic_sketches/phylophlan_metagenomic_paste_1.msh']
phylophlan_databases/SGB.Jan19/16015.msh
phylophlan_metagenomic_dists/16015.tsv
True
Dist for "phylophlan_databases/SGB.Jan19/10325.msh" computed in 0s
Dist for "phylophlan_databases/SGB.Jan19/16758.msh" computed in 0s
Dist for "phylophlan_databases/SGB.Jan19/15822.msh" computed in 0s
Dist for "phylophlan_databases/SGB.Jan19/9346.msh" computed in 0s
Dist for "phylophlan_databases/SGB.Jan19/3348.msh" computed in 0s
Dist for "phylophlan_databases/SGB.Jan19/10115.msh" computed in 6s
Dist for "phylophlan_databases/SGB.Jan19/6783.msh" computed in 0s

[e] Command '['mash', 'dist', '-p', '1', 'phylophlan_databases/SGB.Jan19/16015.msh', 'phylophlan_metagenomic_sketches/phylophlan_metagenomic_paste_1.msh']' died with <Signals.SIGSEGV: 11>.

[e] disting crashed
Dist for "phylophlan_databases/SGB.Jan19/5803.msh" computed in 0s
Dist for "phylophlan_databases/SGB.Jan19/3677.msh" computed in 0s

[fasnicar]

Hi @skye-zhai, thanks for reporting this.

Have you tried running the command in the error?

mash dist -p 1 phylophlan_databases/SGB.Jan19/16015.msh phylophlan_metagenomic_sketches/phylophlan_metagenomic_paste_1.msh

Many thanks,
Francesco

[Jimaz]

Hi!

First of all, this tools is great, congrats!

I have also faced this issue:

Loading mash dist files
Traceback (most recent call last):
  File "/home/jimaz/.conda/envs/phylophlan/bin/phylophlan_metagenomic", line 10, in <module>
    sys.exit(phylophlan_metagenomic())
  File "/home/jimaz/.conda/envs/phylophlan/lib/python3.9/site-packages/phylophlan/phylophlan_metagenomic.py", line 914, in phylophlan_metagenomic
    f.write('\t'.join([binn] + ["{}_{}:{}:{}:{}".format(sgb_2_info[i[0]][5],
  File "/home/jimaz/.conda/envs/phylophlan/lib/python3.9/site-packages/phylophlan/phylophlan_metagenomic.py", line 914, in <listcomp>
    f.write('\t'.join([binn] + ["{}_{}:{}:{}:{}".format(sgb_2_info[i[0]][5],
KeyError: '66244'

The command I have used:

$ phylophlan_metagenomic -i bins/ -o phylophlan_out -d SGB.Dec20 --nproc 12 --verbose 2>&1 | tee -a phylophlan.log

I have checked 66244.tsv from out_dists folder just in case it had something weird.

Still I cannot find why it crashes.

Many thanks,

Jaime

[fasnicar]

Hello Jaime,

Thanks for reporting this. I checked the SGB table of the Dec20 release and the 66244 is not present. So I'm guessing there is an error in the release. Sorry about that.
The person managing the releases is on vacation till the 20th of August. So, I won't be able to provide an updated database for that release with a fix before the end of the month.
In the meantime, can you try with a slightly older version?

Many thanks,
Francesco

[Jimaz]

Thank you for your prompt reply.

I tried with the previous release from Nov20 and got the same error.

Eventually, it worked with Jul20 released.

Thank you so much!

Jaime

[fasnicar]

Thank you Jamie, we'll investigate both releases and fix them. In the meantime, I'm going to remove them from phylophlan_metagenomic so others won't suffer from this issue.

Many thanks,
Francesco