Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*** Error in `../FastK': free(): corrupted unsorted chunks: 0x0000000001687480 *** when profiling sequences #8

Closed
chrisquince opened this issue Apr 9, 2021 · 7 comments

Comments

@chrisquince
Copy link

Thanks for creating yet another incredibly useful looking bit of a software :) I have been searching for a program with all this functionality for a while but I have run into this issue. I have created this as a new issue as I believe it is separate to the file write to /tmp problem raised previously. I am getting corrupted memory faults when try to profile the following sequences downloadable as:

wget https://ebitutorial.s3.climb.ac.uk/graph5.fasta

Running the command:

FastK -k51 graph5.fasta -v -T1 -p

I get:


Partitioning 1 .fasta file into 1 parts

Determining minimizer scheme & partition for graph5
Estimate 1.703M 51-mers
Handling data in a single block
Using 5-minimizers with 1024 core prefixes

Phase 1: Partitioning K-mers into 1 Super-mer Files

There are 38 reads totalling 1,705,389 bps

 Part:     51-mer  super-mers  ave. length
    0:  1,703,489      81,679         20.9
  Sum:  1,703,489      81,679         20.9

  Range 1,703,489 - 1,703,489 (0.00%)

Resources for phase: 0.076u 0.016s 0.108w 84.7%

Phase 2: Sorting & Counting K-mers in 1 blocks

  Part:  wgt'd k-mers  savings             
     0:       988,833      1.7
   All:       988,833      1.7

Resources for phase: 0.312u 0.028s 0.368w 92.1%

Phase 4 (-p option): Merging Profile Fragments

The profiles occupy 160.35 KB
Segmentation fault (core dumped)

That may be the /tmp dir problem. If I point it at a drive with a large amount (1 Tbp of free space) as temp dir with -P I get:

FastK -k51 graph5.fasta -v -P/mnt/chris-native/chris/tmp/ -T1 -p

Partitioning 1 .fasta file into 1 parts

Determining minimizer scheme & partition for graph5
Estimate 1.703M 51-mers
Handling data in a single block
Using 5-minimizers with 1024 core prefixes

Phase 1: Partitioning K-mers into 1 Super-mer Files

There are 38 reads totalling 1,705,389 bps

 Part:     51-mer  super-mers  ave. length
    0:  1,703,489      81,679         20.9
  Sum:  1,703,489      81,679         20.9

  Range 1,703,489 - 1,703,489 (0.00%)

Resources for phase: 0.088u 0.008s 0.356w 26.9%

Phase 2: Sorting & Counting K-mers in 1 blocks

  Part:  wgt'd k-mers  savings             
     0:       988,833      1.7
   All:       988,833      1.7

Resources for phase: 0.400u 0.076s 0.551w 86.4%

Phase 4 (-p option): Merging Profile Fragments

The profiles occupy 160.35 KB
*** Error in `../FastK': free(): corrupted unsorted chunks: 0x0000000001687480 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777f5)[0x7fb467d637f5]
/lib/x86_64-linux-gnu/libc.so.6(+0x8038a)[0x7fb467d6c38a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fb467d7058c]
../FastK[0x418bbb]
../FastK[0x40438a]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fb467d0c840]
../FastK[0x404819]
======= Memory map: ========
00400000-004b2000 r-xp 00000000 00:2b 474411334 /mnt/gpfs/chris/repos/FASTK/FastK
006b1000-006b2000 r--p 000b1000 00:2b 474411334 /mnt/gpfs/chris/repos/FASTK/FastK
006b2000-006b3000 rw-p 000b2000 00:2b 474411334 /mnt/gpfs/chris/repos/FASTK/FastK
006b3000-006b6000 rw-p 00000000 00:00 0
01664000-02eac000 rw-p 00000000 00:00 0 [heap]
7fb35c000000-7fb35c021000 rw-p 00000000 00:00 0
7fb35c021000-7fb360000000 ---p 00000000 00:00 0
7fb361e85000-7fb461e86000 rw-p 00000000 00:00 0
7fb461e86000-7fb461e87000 ---p 00000000 00:00 0
7fb461e87000-7fb462687000 rw-p 00000000 00:00 0
7fb462df9000-7fb462e10000 r-xp 00000000 08:02 656120 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fb462e10000-7fb46300f000 ---p 00017000 08:02 656120 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fb46300f000-7fb463010000 r--p 00016000 08:02 656120 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fb463010000-7fb463011000 rw-p 00017000 08:02 656120 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fb463011000-7fb46301a000 r-xp 00000000 08:02 654573 /lib/x86_64-linux-gnu/libcrypt-2.23.so
7fb46301a000-7fb463219000 ---p 00009000 08:02 654573 /lib/x86_64-linux-gnu/libcrypt-2.23.so
7fb463219000-7fb46321a000 r--p 00008000 08:02 654573 /lib/x86_64-linux-gnu/libcrypt-2.23.so
7fb46321a000-7fb46321b000 rw-p 00009000 08:02 654573 /lib/x86_64-linux-gnu/libcrypt-2.23.so
7fb46321b000-7fb463249000 rw-p 00000000 00:00 0
7fb463249000-7fb463319000 r-xp 00000000 08:02 134403 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6
7fb463319000-7fb463518000 ---p 000d0000 08:02 134403 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6
7fb463518000-7fb46351b000 r--p 000cf000 08:02 134403 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6
7fb46351b000-7fb46351d000 rw-p 000d2000 08:02 134403 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6
7fb46351d000-7fb46351e000 rw-p 00000000 00:00 0
7fb46351e000-7fb463565000 r-xp 00000000 08:02 137969 /usr/lib/x86_64-linux-gnu/libhx509.so.5.0.0
7fb463565000-7fb463764000 ---p 00047000 08:02 137969 /usr/lib/x86_64-linux-gnu/libhx509.so.5.0.0
7fb463764000-7fb463766000 r--p 00046000 08:02 137969 /usr/lib/x86_64-linux-gnu/libhx509.so.5.0.0
7fb463766000-7fb463768000 rw-p 00048000 08:02 137969 /usr/lib/x86_64-linux-gnu/libhx509.so.5.0.0
7fb463768000-7fb463769000 rw-p 00000000 00:00 0
7fb463769000-7fb463777000 r-xp 00000000 08:02 137959 /usr/lib/x86_64-linux-gnu/libheimbase.so.1.0.0
7fb463777000-7fb463976000 ---p 0000e000 08:02 137959 /usr/lib/x86_64-linux-gnu/libheimbase.so.1.0.0
7fb463976000-7fb463977000 r--p 0000d000 08:02 137959 /usr/lib/x86_64-linux-gnu/libheimbase.so.1.0.0
7fb463977000-7fb463978000 rw-p 0000e000 08:02 137959 /usr/lib/x86_64-linux-gnu/libheimbase.so.1.0.0
7fb463978000-7fb46399f000 r-xp 00000000 08:02 137965 /usr/lib/x86_64-linux-gnu/libwind.so.0.0.0
7fb46399f000-7fb463b9f000 ---p 00027000 08:02 137965 /usr/lib/x86_64-linux-gnu/libwind.so.0.0.0
7fb463b9f000-7fb463ba0000 r--p 00027000 08:02 137965 /usr/lib/x86_64-linux-gnu/libwind.so.0.0.0
7fb463ba0000-7fb463ba1000 rw-p 00028000 08:02 137965 /usr/lib/x86_64-linux-gnu/libwind.so.0.0.0
7fb463ba1000-7fb463ba8000 r-xp 00000000 08:02 133165 /usr/lib/x86_64-linux-gnu/libffi.so.6.0.4
7fb463ba8000-7fb463da7000 ---p 00007000 08:02 133165 /usr/lib/x86_64-linux-gnu/libffi.so.6.0.4
7fb463da7000-7fb463da8000 r--p 00006000 08:02 133165 /usr/lib/x86_64-linux-gnu/libffi.so.6.0.4
7fb463da8000-7fb463da9000 rw-p 00007000 08:02 133165 /usr/lib/x86_64-linux-gnu/libffi.so.6.0.4
7fb463da9000-7fb463dbe000 r-xp 00000000 08:02 145811 /usr/lib/x86_64-linux-gnu/libroken.so.18.1.0
7fb463dbe000-7fb463fbd000 ---p 00015000 08:02 145811 /usr/lib/x86_64-linux-gnu/libroken.so.18.1.0
7fb463fbd000-7fb463fbe000 r--p 00014000 08:02 145811 /usr/lib/x86_64-linux-gnu/libroken.so.18.1.0
7fb463fbe000-7fb463fbf000 rw-p 00015000 08:02 145811 /usr/lib/x86_64-linux-gnu/libroken.so.18.1.0
7fb463fbf000-7fb463fef000 r-xp 00000000 08:02 137955 /usr/lib/x86_64-linux-gnu/libhcrypto.so.4.1.0
7fb463fef000-7fb4641ef000 ---p 00030000 08:02 137955 /usr/lib/x86_64-linux-gnu/libhcrypto.so.4.1.0
7fb4641ef000-7fb4641f0000 r--p 00030000 08:02 137955 /usr/lib/x86_64-linux-gnu/libhcrypto.so.4.1.0
7fb4641f0000-7fb4641f1000 rw-p 00031000 08:02 137955 /usr/lib/x86_64-linux-gnu/libhcrypto.so.4.1.0
7fb4641f1000-7fb4641f2000 rw-p 00000000 00:00 0
7fb4641f2000-7fb464291000 r-xp 00000000 08:02 137952 /usr/lib/x86_64-linux-gnu/libasn1.so.8.0.0
7fb464291000-7fb464490000 ---p 0009f000 08:02 137952 /usr/lib/x86_64-linux-gnu/libasn1.so.8.0.0
7fb464490000-7fb464491000 r--p 0009e000 08:02 137952 /usr/lib/x86_64-linux-gnu/libasn1.so.8.0.0
7fb464491000-7fb464494000 rw-p 0009f000 08:02 137952 /usr/lib/x86_64-linux-gnu/libasn1.so.8.0.0
7fb464494000-7fb464518000 r-xp 00000000 08:02 137974 /usr/lib/x86_64-linux-gnu/libkrb5.so.26.0.0
7fb464518000-7fb464717000 ---p 00084000 08:02 137974 /usr/lib/x86_64-linux-gnu/libkrb5.so.26.0.0
7fb464717000-7fb46471a000 r--p 00083000 08:02 137974 /usr/lib/x86_64-linux-gnu/libkrb5.so.26.0.0
7fb46471a000-7fb46471d000 rw-p 00086000 08:02 137974 /usr/lib/x86_64-linux-gnu/libkrb5.so.26.0.0
7fb46471d000-7fb46471e000 rw-p 00000000 00:00 0
7fb46471e000-7fb464726000 r-xp 00000000 08:02 137979 /usr/lib/x86_64-linux-gnu/libheimntlm.so.0.1.0
7fb464726000-7fb464925000 ---p 00008000 08:02 137979 /usr/lib/x86_64-linux-gnu/libheimntlm.so.0.1.0
7fb464925000-7fb464926000 r--p 00007000 08:02 137979 /usr/lib/x86_64-linux-gnu/libheimntlm.so.0.1.0
7fb464926000-7fb464927000 rw-p 00008000 08:02 137979 /usr/lib/x86_64-linux-gnu/libheimntlm.so.0.1.0
7fb464927000-7fb46492a000 r-xp 00000000 08:02 660056 /lib/x86_64-linux-gnu/libkeyutils.so.1.5
7fb46492a000-7fb464b29000 ---p 00003000 08:02 660056 /lib/x86_64-linux-gnu/libkeyutils.so.1.5
7fb464b29000-7fb464b2a000 r--p 00002000 08:02 660056 /lib/x86_64-linux-gnu/libkeyutils.so.1.5
7fb464b2a000-7fb464b2b000 rw-p 00003000 08:02 660056 /lib/x86_64-linux-gnu/libkeyutils.so.1.5
7fb464b2b000-7fb464b3c000 r-xp 00000000 08:02 131781 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.5.1
7fb464b3c000-7fb464d3c000 ---p 00011000 08:02 131781 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.5.1
7fb464d3c000-7fb464d3d000 r--p 00011000 08:02 131781 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.5.1
7fb464d3d000-7fb464d3e000 rw-p 00012000 08:02 131781 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.5.1
7fb464d3e000-7fb464d97000 r-xp 00000000 08:02 137411 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.1.0
7fb464d97000-7fb464f96000 ---p 00059000 08:02 137411 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.1.0
7fb464f96000-7fb464fa0000 r--p 00058000 08:02 137411 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.1.0
7fb464fa0000-7fb464fa2000 rw-p 00062000 08:02 137411 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.1.0
7fb464fa2000-7fb464fdf000 r-xp 00000000 08:02 137984 /usr/lib/x86_64-linux-gnu/libgssapi.so.3.0.0
7fb464fdf000-7fb4651df000 ---p 0003d000 08:02 137984 /usr/lib/x86_64-linux-gnu/libgssapi.so.3.0.0
7fb4651df000-7fb4651e0000 r--p 0003d000 08:02 137984 /usr/lib/x86_64-linux-gnu/libgssapi.so.3.0.0
7fb4651e0000-7fb4651e2000 rw-p 0003e000 08:02 137984 /usr/lib/x86_64-linux-gnu/libgssapi.so.3.0.0
7fb4651e2000-7fb4651e3000 rw-p 00000000 00:00 0
7fb4651e3000-7fb4651fc000 r-xp 00000000 08:02 132347 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
7fb4651fc000-7fb4653fc000 ---p 00019000 08:02 132347 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
7fb4653fc000-7fb4653fd000 r--p 00019000 08:02 132347 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
7fb4653fd000-7fb4653fe000 rw-p 0001a000 08:02 132347 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
7fb4653fe000-7fb465415000 r-xp 00000000 08:02 655434 /lib/x86_64-linux-gnu/libresolv-2.23.so
7fb465415000-7fb465615000 ---p 00017000 08:02 655434 /lib/x86_64-linux-gnu/libresolv-2.23.so
7fb465615000-7fb465616000 r--p 00017000 08:02 655434 /lib/x86_64-linux-gnu/libresolv-2.23.so
7fb465616000-7fb465617000 rw-p 00018000 08:02 655434 /lib/x86_64-linux-gnu/libresolv-2.23.so
7fb465617000-7fb465619000 rw-p 00000000 00:00 0
7fb465619000-7fb465623000 r-xp 00000000 08:02 134938 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
7fb465623000-7fb465822000 ---p 0000a000 08:02 134938 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
7fb465822000-7fb465823000 r--p 00009000 08:02 134938 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
7fb465823000-7fb465824000 rw-p 0000a000 08:02 134938 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
7fb465824000-7fb465827000 r-xp 00000000 08:02 654290 /lib/x86_64-linux-gnu/libcom_err.so.2.1
7fb465827000-7fb465a26000 ---p 00003000 08:02 654290 /lib/x86_64-linux-gnu/libcom_err.so.2.1
7fb465a26000-7fb465a27000 r--p 00002000 08:02 654290 /lib/x86_64-linux-gnu/libcom_err.so.2.1
7fb465a27000-7fb465a28000 rw-p 00003000 08:02 654290 /lib/x86_64-linux-gnu/libcom_err.so.2.1
7fb465a28000-7fb465a54000 r-xp 00000000 08:02 132856 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
7fb465a54000-7fb465c53000 ---p 0002c000 08:02 132856 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
7fb465c53000-7fb465c55000 r--p 0002b000 08:02 132856 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
7fb465c55000-7fb465c56000 rw-p 0002d000 08:02 132856 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
7fb465c56000-7fb465c57000 rw-p 00000000 00:00 0
7fb465c57000-7fb465d1a000 r-xp 00000000 08:02 137943 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
7fb465d1a000-7fb465f1a000 ---p 000c3000 08:02 137943 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
7fb465f1a000-7fb465f27000 r--p 000c3000 08:02 137943 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
7fb465f27000-7fb465f29000 rw-p 000d0000 08:02 137943 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
7fb465f29000-7fb465fa8000 r-xp 00000000 08:02 133173 /usr/lib/x86_64-linux-gnu/libgmp.so.10.3.0
7fb465fa8000-7fb4661a7000 ---p 0007f000 08:02 133173 /usr/lib/x86_64-linux-gnu/libgmp.so.10.3.0
7fb4661a7000-7fb4661a8000 r--p 0007e000 08:02 133173 /usr/lib/x86_64-linux-gnu/libgmp.so.10.3.0
7fb4661a8000-7fb4661a9000 rw-p 0007f000 08:02 133173 /usr/lib/x86_64-linux-gnu/libgmp.so.10.3.0
7fb4661a9000-7fb4661dd000 r-xp 00000000 08:02 133193 /usr/lib/x86_64-linux-gnu/libnettle.so.6.2
7fb4661dd000-7fb4663dc000 ---p 00034000 08:02 133193 /usr/lib/x86_64-linux-gnu/libnettle.so.6.2
7fb4663dc000-7fb4663de000 r--p 00033000 08:02 133193 /usr/lib/x86_64-linux-gnu/libnettle.so.6.2
7fb4663de000-7fb4663df000 rw-p 00035000 08:02 133193 /usr/lib/x86_64-linux-gnu/libnettle.so.6.2
7fb4663df000-7fb466411000 r-xp 00000000 08:02 133179 /usr/lib/x86_64-linux-gnu/libhogweed.so.4.2
7fb466411000-7fb466610000 ---p 00032000 08:02 133179 /usr/lib/x86_64-linux-gnu/libhogweed.so.4.2
7fb466610000-7fb466611000 r--p 00031000 08:02 133179 /usr/lib/x86_64-linux-gnu/libhogweed.so.4.2
7fb466611000-7fb466612000 rw-p 00032000 08:02 133179 /usr/lib/x86_64-linux-gnu/libhogweed.so.4.2
7fb466612000-7fb466735000 r-xp 00000000 08:02 134875 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.6.2
7fb466735000-7fb466934000 ---p 00123000 08:02 134875 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.6.2
7fb466934000-7fb46693f000 r--p 00122000 08:02 134875 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.6.2
7fb46693f000-7fb466941000 rw-p 0012d000 08:02 134875 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.6.2
7fb466941000-7fb466942000 rw-p 00000000 00:00 0
7fb466942000-7fb46698f000 r-xp 00000000 08:02 131120 /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.10.5
7fb46698f000-7fb466b8e000 ---p 0004d000 08:02 131120 /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.10.5
7fb466b8e000-7fb466b90000 r--p 0004c000 08:02 131120 /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.10.5
7fb466b90000-7fb466b91000 rw-p 0004e000 08:02 131120 /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.10.5
7fb466b91000-7fb466b93000 rw-p 00000000 00:00 0
7fb466b93000-7fb466ba0000 r-xp 00000000 08:02 131171 /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.10.5
7fb466ba0000-7fb466da0000 ---p 0000d000 08:02 131171 /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.10.5
7fb466da0000-7fb466da1000 r--p 0000d000 08:02 131171 /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.10.5
7fb466da1000-7fb466da2000 rw-p 0000e000 08:02 131171 /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.10.5
7fb466da2000-7fb466de9000 r-xp 00000000 08:02 137935 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
7fb466de9000-7fb466fe8000 ---p 00047000 08:02 137935 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
7fb466fe8000-7fb466fea000 r--p 00046000 08:02 137935 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
7fb466fea000-7fb466fec000 rw-p 00048000 08:02 137935 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
7fb466fec000-7fb467207000 r-xp 00000000 08:02 654731 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7fb467207000-7fb467406000 ---p 0021b000 08:02 654731 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7fb467406000-7fb467422000 r--p 0021a000 08:02 654731 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7fb467422000-7fb46742e000 rw-p 00236000 08:02 654731 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7fb46742e000-7fb467431000 rw-p 00000000 00:00 0
7fb467431000-7fb46748f000 r-xp 00000000 08:02 654734 /lib/x86_64-linux-gnu/libssl.so.1.0.0
7fb46748f000-7fb46768f000 ---p 0005e000 08:02 654734 /lib/x86_64-linux-gnu/libssl.so.1.0.0
7fb46768f000-7fb467693000 r--p 0005e000 08:02 654734 /lib/x86_64-linux-gnu/libssl.so.1.0.0
7fb467693000-7fb467699000 rw-p 00062000 08:02 654734 /lib/x86_64-linux-gnu/libssl.so.1.0.0
7fb467699000-7fb4676b4000 r-xp 00000000 08:02 135071 /usr/lib/x86_64-linux-gnu/librtmp.so.1
7fb4676b4000-7fb4678b3000 ---p 0001b000 08:02 135071 /usr/lib/x86_64-linux-gnu/librtmp.so.1
7fb4678b3000-7fb4678b4000 r--p 0001a000 08:02 135071 /usr/lib/x86_64-linux-gnu/librtmp.so.1
7fb4678b4000-7fb4678b5000 rw-p 0001b000 08:02 135071 /usr/lib/x86_64-linux-gnu/librtmp.so.1
7fb4678b5000-7fb4678e6000 r-xp 00000000 08:02 161247 /usr/lib/x86_64-linux-gnu/libidn.so.11.6.15
7fb4678e6000-7fb467ae6000 ---p 00031000 08:02 161247 /usr/lib/x86_64-linux-gnu/libidn.so.11.6.15
7fb467ae6000-7fb467ae7000 r--p 00031000 08:02 161247 /usr/lib/x86_64-linux-gnu/libidn.so.11.6.15
7fb467ae7000-7fb467ae8000 rw-p 00032000 08:02 161247 /usr/lib/x86_64-linux-gnu/libidn.so.11.6.15
7fb467ae8000-7fb467aeb000 r-xp 00000000 08:02 654528 /lib/x86_64-linux-gnu/libdl-2.23.so
7fb467aeb000-7fb467cea000 ---p 00003000 08:02 654528 /lib/x86_64-linux-gnu/libdl-2.23.so
7fb467cea000-7fb467ceb000 r--p 00002000 08:02 654528 /lib/x86_64-linux-gnu/libdl-2.23.so
7fb467ceb000-7fb467cec000 rw-p 00003000 08:02 654528 /lib/x86_64-linux-gnu/libdl-2.23.so
7fb467cec000-7fb467eac000 r-xp 00000000 08:02 654530 /lib/x86_64-linux-gnu/libc-2.23.so
7fb467eac000-7fb4680ac000 ---p 001c0000 08:02 654530 /lib/x86_64-linux-gnu/libc-2.23.so
7fb4680ac000-7fb4680b0000 r--p 001c0000 08:02 654530 /lib/x86_64-linux-gnu/libc-2.23.so
7fb4680b0000-7fb4680b2000 rw-p 001c4000 08:02 654530 /lib/x86_64-linux-gnu/libc-2.23.so
7fb4680b2000-7fb4680b6000 rw-p 00000000 00:00 0
7fb4680b6000-7fb468124000 r-xp 00000000 08:02 137680 /usr/lib/x86_64-linux-gnu/libcurl.so.4.4.0
7fb468124000-7fb468324000 ---p 0006e000 08:02 137680 /usr/lib/x86_64-linux-gnu/libcurl.so.4.4.0
7fb468324000-7fb468327000 r--p 0006e000 08:02 137680 /usr/lib/x86_64-linux-gnu/libcurl.so.4.4.0
7fb468327000-7fb468328000 rw-p 00071000 08:02 137680 /usr/lib/x86_64-linux-gnu/libcurl.so.4.4.0
7fb468328000-7fb468349000 r-xp 00000000 08:02 654615 /lib/x86_64-linux-gnu/liblzma.so.5.0.0
7fb468349000-7fb468548000 ---p 00021000 08:02 654615 /lib/x86_64-linux-gnu/liblzma.so.5.0.0
7fb468548000-7fb468549000 r--p 00020000 08:02 654615 /lib/x86_64-linux-gnu/liblzma.so.5.0.0
7fb468549000-7fb46854a000 rw-p 00021000 08:02 654615 /lib/x86_64-linux-gnu/liblzma.so.5.0.0
7fb46854a000-7fb468559000 r-xp 00000000 08:02 654186 /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fb468559000-7fb468758000 ---p 0000f000 08:02 654186 /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fb468758000-7fb468759000 r--p 0000e000 08:02 654186 /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fb468759000-7fb46875a000 rw-p 0000f000 08:02 654186 /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fb46875a000-7fb468773000 r-xp 00000000 08:02 654834 /lib/x86_64-linux-gnu/libz.so.1.2.8
7fb468773000-7fb468972000 ---p 00019000 08:02 654834 /lib/x86_64-linux-gnu/libz.so.1.2.8
7fb468972000-7fb468973000 r--p 00018000 08:02 654834 /lib/x86_64-linux-gnu/libz.so.1.2.8
7fb468973000-7fb468974000 rw-p 00019000 08:02 654834 /lib/x86_64-linux-gnu/libz.so.1.2.8
7fb468974000-7fb46898c000 r-xp 00000000 08:02 654534 /lib/x86_64-linux-gnu/libpthread-2.23.so
7fb46898c000-7fb468b8b000 ---p 00018000 08:02 654534 /lib/x86_64-linux-gnu/libpthread-2.23.so
7fb468b8b000-7fb468b8c000 r--p 00017000 08:02 654534 /lib/x86_64-linux-gnu/libpthread-2.23.so
7fb468b8c000-7fb468b8d000 rw-p 00018000 08:02 654534 /lib/x86_64-linux-gnu/libpthread-2.23.so
7fb468b8d000-7fb468b91000 rw-p 00000000 00:00 0
7fb468b91000-7fb468bb7000 r-xp 00000000 08:02 655423 /lib/x86_64-linux-gnu/ld-2.23.so
7fb468d88000-7fb468d9b000 rw-p 00000000 00:00 0
7fb468db5000-7fb468db6000 rw-p 00000000 00:00 0
7fb468db6000-7fb468db7000 r--p 00025000 08:02 655423 /lib/x86_64-linux-gnu/ld-2.23.so
7fb468db7000-7fb468db8000 rw-p 00026000 08:02 655423 /lib/x86_64-linux-gnu/ld-2.23.so
7fb468db8000-7fb468db9000 rw-p 00000000 00:00 0
7ffd617cb000-7ffd6181f000 rw-p 00000000 00:00 0 [stack]
7ffd61932000-7ffd61934000 r--p 00000000 00:00 0 [vvar]
7ffd61934000-7ffd61936000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-fffffffff

Sorry for the long post but I have though the above may be helpful. This is compiled with gcc:

gcc (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609

@thegenemyers
Copy link
Owner

thegenemyers commented Apr 12, 2021 via email

@chrisquince
Copy link
Author

Thanks for the quick response. I was able to reproduce this on a clean VM. So I can give you an exact set-up. The VM is running Ubuntu 16.04 and I installed all required libraries through apt except HTSLIB that I compiled from source. gcc is:

gcc (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609

I will have a go at generating a stack trace this afternoon. I may be able to do that or it may be beyond my limited C debugging capability.

If you could e-mail me c.quince at warwick.ac.uk your public ssh key I could give you access to the VM?

I am in the UK so zoom is definitely possible. I will also try an Ubuntu 18.04 VM in case it is version specific.

Chris

@chrisquince
Copy link
Author

Here is the stack trace when running the command:

FastK -k51 graph5.fasta -v -P/home/ubuntu/tmp/ -T1 -p

On the Ubuntu 16.04 VM

#0 0x00007ffff6f67438 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1 0x00007ffff6f6903a in __GI_abort () at abort.c:89
#2 0x00007ffff6fa97fa in __libc_message (do_abort=2, fmt=fmt@entry=0x7ffff70c2f98 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:175
#3 0x00007ffff6fb2c81 in malloc_printerr (ar_ptr=0x7ffff72f6b20 <main_arena>, ptr=, str=0x7ffff70bfd52 "corrupted double-linked list", action=3) at malloc.c:5020
#4 _int_free (av=0x7ffff72f6b20 <main_arena>, p=, have_lock=0) at malloc.c:4021
#5 0x00007ffff6fb658c in __GI___libc_free (mem=) at malloc.c:2975
#6 0x000000000041c450 in Merge_Profiles (dpwd=0x6b95d0 ".", dbrt=0x6b9500 "graph5") at merge.c:832
#7 0x0000000000405af3 in main (argc=2, argv=0x7fffffffe558) at FastK.c:490

@thegenemyers
Copy link
Owner

thegenemyers commented Apr 13, 2021 via email

@chrisquince
Copy link
Author

Hi Gene,

I am sure this is a 64 bit VM and sizeof(long long int) returns 8.

I ran FastK through valgrind which is a program I used to use to find memory errors and it returned the following with the args above:

==2752== Warning: set address range perms: large range [0x395d8040, 0x1395d8030) (undefined)

124%==2752== Invalid write of size 1
==2752== at 0x4C35035: __GI_mempcpy (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2752== by 0x59906DD: _IO_default_xsputn (genops.c:438)
==2752== by 0x596351A: vfprintf (vfprintf.c:1632)
==2752== by 0x59851DA: vsprintf (iovsprintf.c:42)
==2752== by 0x596A9D6: sprintf (sprintf.c:32)
==2752== by 0x41B638: merge_profile_thread (merge.c:608)
==2752== by 0x41C122: Merge_Profiles (merge.c:789)
==2752== by 0x405AF2: main (FastK.c:490)
==2752== Address 0xd7e2e07 is 23 bytes inside a block of size 124 free'd
==2752== at 0x4C2EDEB: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2752== by 0x41B614: merge_profile_thread (merge.c:605)
==2752== by 0x41C122: Merge_Profiles (merge.c:789)
==2752== by 0x405AF2: main (FastK.c:490)
==2752== Block was alloc'd at
==2752== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2752== by 0x41E8E6: Malloc (gene_core.c:22)
==2752== by 0x41A42B: merge_profile_thread (merge.c:128)
==2752== by 0x41C122: Merge_Profiles (merge.c:789)
==2752== by 0x405AF2: main (FastK.c:490)
==2752==
==2752== Invalid write of size 1
==2752== at 0x599068B: _IO_default_xsputn (genops.c:450)
==2752== by 0x596293C: vfprintf (vfprintf.c:1663)
==2752== by 0x59851DA: vsprintf (iovsprintf.c:42)
==2752== by 0x596A9D6: sprintf (sprintf.c:32)
==2752== by 0x41B638: merge_profile_thread (merge.c:608)
==2752== by 0x41C122: Merge_Profiles (merge.c:789)
==2752== by 0x405AF2: main (FastK.c:490)
==2752== Address 0xd7e2e08 is 24 bytes inside a block of size 124 free'd
==2752== at 0x4C2EDEB: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2752== by 0x41B614: merge_profile_thread (merge.c:605)
==2752== by 0x41C122: Merge_Profiles (merge.c:789)
==2752== by 0x405AF2: main (FastK.c:490)
==2752== Block was alloc'd at
==2752== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2752== by 0x41E8E6: Malloc (gene_core.c:22)
==2752== by 0x41A42B: merge_profile_thread (merge.c:128)
==2752== by 0x41C122: Merge_Profiles (merge.c:789)
==2752== by 0x405AF2: main (FastK.c:490)
==2752==
==2752== Invalid write of size 1
==2752== at 0x599068B: _IO_default_xsputn (genops.c:450)
==2752== by 0x5962E0F: vfprintf (vfprintf.c:1631)
==2752== by 0x59851DA: vsprintf (iovsprintf.c:42)
==2752== by 0x596A9D6: sprintf (sprintf.c:32)
==2752== by 0x41B638: merge_profile_thread (merge.c:608)
==2752== by 0x41C122: Merge_Profiles (merge.c:789)
==2752== by 0x405AF2: main (FastK.c:490)
==2752== Address 0xd7e2e0d is 29 bytes inside a block of size 124 free'd
==2752== at 0x4C2EDEB: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2752== by 0x41B614: merge_profile_thread (merge.c:605)
==2752== by 0x41C122: Merge_Profiles (merge.c:789)
==2752== by 0x405AF2: main (FastK.c:490)
==2752== Block was alloc'd at
==2752== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2752== by 0x41E8E6: Malloc (gene_core.c:22)
==2752== by 0x41A42B: merge_profile_thread (merge.c:128)
==2752== by 0x41C122: Merge_Profiles (merge.c:789)
==2752== by 0x405AF2: main (FastK.c:490)
==2752==
==2752== Invalid write of size 1
==2752== at 0x59851E9: vsprintf (iovsprintf.c:43)
==2752== by 0x596A9D6: sprintf (sprintf.c:32)
==2752== by 0x41B638: merge_profile_thread (merge.c:608)
==2752== by 0x41C122: Merge_Profiles (merge.c:789)
==2752== by 0x405AF2: main (FastK.c:490)
==2752== Address 0xd7e2e0e is 30 bytes inside a block of size 124 free'd
==2752== at 0x4C2EDEB: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2752== by 0x41B614: merge_profile_thread (merge.c:605)
==2752== by 0x41C122: Merge_Profiles (merge.c:789)
==2752== by 0x405AF2: main (FastK.c:490)
==2752== Block was alloc'd at
==2752== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2752== by 0x41E8E6: Malloc (gene_core.c:22)
==2752== by 0x41A42B: merge_profile_thread (merge.c:128)
==2752== by 0x41C122: Merge_Profiles (merge.c:789)
==2752== by 0x405AF2: main (FastK.c:490)
==2752==
==2752== Syscall param unlink(pathname) points to unaddressable byte(s)
==2752== at 0x5A0DB57: unlink (syscall-template.S:84)
==2752== by 0x41B644: merge_profile_thread (merge.c:609)
==2752== by 0x41C122: Merge_Profiles (merge.c:789)
==2752== by 0x405AF2: main (FastK.c:490)
==2752== Address 0xd7e2df0 is 0 bytes inside a block of size 124 free'd
==2752== at 0x4C2EDEB: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2752== by 0x41B614: merge_profile_thread (merge.c:605)
==2752== by 0x41C122: Merge_Profiles (merge.c:789)
==2752== by 0x405AF2: main (FastK.c:490)
==2752== Block was alloc'd at
==2752== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2752== by 0x41E8E6: Malloc (gene_core.c:22)
==2752== by 0x41A42B: merge_profile_thread (merge.c:128)
==2752== by 0x41C122: Merge_Profiles (merge.c:789)
==2752== by 0x405AF2: main (FastK.c:490)
==2752==

Does that help at all? I can also run valgrind with more intensive options than the default?

Thanks,
Chris

@thegenemyers
Copy link
Owner

thegenemyers commented Apr 13, 2021 via email

@chrisquince
Copy link
Author

Hi Gene,

That is great! I pulled the latest version recompiled and it ran through without errors.

Thanks very much for your help. Looking forward to playing around with the profiles now.

Best,
Chris

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants