Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

samtools dependency #111

Open
igordot opened this issue Sep 12, 2018 · 15 comments · May be fixed by #412
Open

samtools dependency #111

igordot opened this issue Sep 12, 2018 · 15 comments · May be fixed by #412

Comments

@igordot
Copy link

igordot commented Sep 12, 2018

It might be worthwhile to add samtools to the installation requirements list. I used the suggested command:

conda install numpy scipy cython numba matplotlib scikit-learn h5py click

Then I found out about the samtools requirement after trying to run velocyto. Installing samtools separately led to numpy version conflicts. It was fixable, but it's probably best to just add it to the initial install command.

On a related note, pandas is also a requirement, but it is not listed.
@slinnarsson
Copy link

Literally the next line after the one you quoted reads:

Note: pysam is also a requirement but currently it is preferable to install it
through PyPI wheel (i.e. by pip install pysam) as we have experienced problems
with the version packaged by conda-forge.

But I agree pysam can be hard to install. On my Mac I had better luck with

conda install -c r -c bioconda pysam

@igordot
Copy link
Author

igordot commented Sep 12, 2018

I saw the next line about pysam. That installed fine with pip as suggested. However, I still needed to install samtools after that.

@slinnarsson
Copy link

Aha, ok I get it. That seems wrong, because pysam includes htslib (which is samtools), as far as I understand.

Or maybe I'm misunderstanding, and you're saying that samtools can be used e.g. to sort BAM files, and this might be required to run velocyto? That might be a good thing to point out generally, but it's not strictly a requirement for running velocyto.

@gioelelm
Copy link
Member

gioelelm commented Sep 12, 2018 via email

@slinnarsson
Copy link

Ok, I never noticed probably because I had samtools installed for some other reason. If the code is not too complicated and you want to drop the dependency on samtools instead, you can run any samtools command from pysam:

pysam.sort("-o", "output.bam", "ex1.bam")

@saeedfc
Copy link

saeedfc commented Oct 22, 2019 •
edited
Loading

Hi,

Thanks for discussing this issue.
I am running into issue of
FileNotFoundError: [Errno 2] No such file or directory: 'samtools': 'samtools'
So I tried,
$pip install pysam
Requirement already satisfied: pysam in ./anaconda3/lib/python3.7/site-packages (0.15.3)
@igordot @gioelelm
So should I install samtools separately now or can some one help me how they worked around this issue?

@saeedfc
Copy link

saeedfc commented Oct 22, 2019 •
edited
Loading

I installed samtools. However it still comes back with same error. Can someone advice, where I should place the samtools directory for it to be read properly by velocyto?

File "/home/luna.kuleuven.be/u0119129/anaconda3/lib/python3.7/subprocess.py", line 1522, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'samtools': 'samtools'

Here is a complete output
(base) u0119129@gbw-d-l0099:/$ velocyto run10x -m /mnt/DATA1/Velocyto/mm10_rmsk.gtf /mnt/DATA2/RAW_DATA/Re_Run_\ 2018_Data_with_updated_reference_genome/KUL-1-1000cells /mnt/DATA1/Velocyto/refdata-cellranger-mm10-3.0.0/genes/genes.gtf 2019-10-22 11:27:35,079 - DEBUG - Using logic: Default 2019-10-22 11:27:35,080 - INFO - Read 666 cell barcodes from /mnt/DATA2/RAW_DATA/Re_Run_ 2018_Data_with_updated_reference_genome/KUL-1-1000cells/outs/filtered_feature_bc_matrix/barcodes.tsv.gz 2019-10-22 11:27:35,080 - DEBUG - Example of barcode: AAACCTGAGATTACCC and cell_id: KUL-1-1000cells:AAACCTGAGATTACCC-1 2019-10-22 11:27:35,089 - DEBUG - Peeking into /mnt/DATA2/RAW_DATA/Re_Run_ 2018_Data_with_updated_reference_genome/KUL-1-1000cells/outs/possorted_genome_bam.bam 2019-10-22 11:27:35,099 - WARNING - Not found cell and umi barcode in entry 86 of the bam file 2019-10-22 11:27:35,100 - WARNING - Not found cell and umi barcode in entry 87 of the bam file 2019-10-22 11:27:35,100 - WARNING - Not found cell and umi barcode in entry 98 of the bam file 2019-10-22 11:27:35,100 - WARNING - Not found cell and umi barcode in entry 144 of the bam file 2019-10-22 11:27:35,100 - WARNING - Not found cell and umi barcode in entry 146 of the bam file 2019-10-22 11:27:35,100 - WARNING - Not found cell and umi barcode in entry 190 of the bam file 2019-10-22 11:27:35,100 - WARNING - Not found cell and umi barcode in entry 191 of the bam file 2019-10-22 11:27:35,100 - WARNING - Not found cell and umi barcode in entry 195 of the bam file 2019-10-22 11:27:35,100 - WARNING - Not found cell and umi barcode in entry 303 of the bam file 2019-10-22 11:27:35,100 - WARNING - Not found cell and umi barcode in entry 309 of the bam file 2019-10-22 11:27:35,100 - WARNING - Not found cell and umi barcode in entry 326 of the bam file 2019-10-22 11:27:35,100 - WARNING - Not found cell and umi barcode in entry 345 of the bam file 2019-10-22 11:27:35,100 - WARNING - Not found cell and umi barcode in entry 347 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 392 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 395 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 402 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 403 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 404 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 405 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 406 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 407 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 408 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 409 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 410 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 411 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 412 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 413 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 414 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 415 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 416 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 417 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 418 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 419 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 420 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 445 of the bam file 2019-10-22 11:27:35,101 - WARNING - Not found cell and umi barcode in entry 461 of the bam file 2019-10-22 11:27:35,102 - WARNING - Not found cell and umi barcode in entry 553 of the bam file 2019-10-22 11:27:35,102 - WARNING - Not found cell and umi barcode in entry 759 of the bam file 2019-10-22 11:27:35,102 - WARNING - Not found cell and umi barcode in entry 767 of the bam file 2019-10-22 11:27:35,102 - WARNING - Not found cell and umi barcode in entry 768 of the bam file 2019-10-22 11:27:35,102 - WARNING - Not found cell and umi barcode in entry 854 of the bam file Traceback (most recent call last): File "/home/luna.kuleuven.be/u0119129/anaconda3/bin/velocyto", line 11, in <module> load_entry_point('velocyto', 'console_scripts', 'velocyto')() File "/home/luna.kuleuven.be/u0119129/anaconda3/lib/python3.7/site-packages/click/core.py", line 764, in __call__ return self.main(*args, **kwargs) File "/home/luna.kuleuven.be/u0119129/anaconda3/lib/python3.7/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/home/luna.kuleuven.be/u0119129/anaconda3/lib/python3.7/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/luna.kuleuven.be/u0119129/anaconda3/lib/python3.7/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/luna.kuleuven.be/u0119129/anaconda3/lib/python3.7/site-packages/click/core.py", line 555, in invoke return callback(*args, **kwargs) File "/home/luna.kuleuven.be/u0119129/velocyto.py/velocyto/commands/run10x.py", line 115, in run10x samtools_memory=samtools_memory, dump=dump, loom_numeric_dtype=dtype, verbose=verbose, additional_ca=additional_ca) File "/home/luna.kuleuven.be/u0119129/velocyto.py/velocyto/commands/_run.py", line 178, in _run sorting_process[ni] = subprocess.Popen(command.split(), stdout=subprocess.PIPE) File "/home/luna.kuleuven.be/u0119129/anaconda3/lib/python3.7/subprocess.py", line 775, in __init__ restore_signals, start_new_session) File "/home/luna.kuleuven.be/u0119129/anaconda3/lib/python3.7/subprocess.py", line 1522, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: 'samtools': 'samtools'

@igordot
Copy link
Author

igordot commented Oct 22, 2019

@saeedfc It sounds like you are trying to install it manually ("where I should place the samtools directory"). The solution would be to add it to your PATH, but try installing samtools with conda. If you are using conda for other dependencies, you should use it for all dependencies, including samtools.

@saeedfc
Copy link

saeedfc commented Oct 23, 2019

Hi,
@igordot
Thanks for the suggestion. Indeed, installing samtools with conda worked. I am now hitting another strange error. I am allocating 10 threads with 24 gb each for the run and it doesn't work. I went with one thread and 240gb memory and still memory error comes. These are only 1000 cells with less than 50000 reads per cell in this sample. I think something else other than memory is going wrong her. Is this familiar to anyone?

(base) u0119129@gbw-d-l0099:~$ velocyto run10x -m /mnt/DATA1/Velocyto/alltracks_mask.gtf /mnt/DATA2/RAW_DATA/Re_Run_\ 2018_Data_with_updated_reference_genome/KUL-1-1000cells /mnt/DATA1/Velocyto/refdata-cellranger-mm10-3.0.0/genes/genes.gtf -@ 10 --samtools-memory 24000
2019-10-23 09:47:26,919 - DEBUG - Using logic: Default
2019-10-23 09:47:26,920 - INFO - Read 666 cell barcodes from /mnt/DATA2/RAW_DATA/Re_Run_ 2018_Data_with_updated_reference_genome/KUL-1-1000cells/outs/filtered_feature_bc_matrix/barcodes.tsv.gz
2019-10-23 09:47:26,920 - DEBUG - Example of barcode: AAACCTGAGATTACCC and cell_id: KUL-1-1000cells:AAACCTGAGATTACCC-1
2019-10-23 09:47:26,936 - DEBUG - Peeking into /mnt/DATA2/RAW_DATA/Re_Run_ 2018_Data_with_updated_reference_genome/KUL-1-1000cells/outs/possorted_genome_bam.bam
2019-10-23 09:47:26,947 - WARNING - Not found cell and umi barcode in entry 86 of the bam file
2019-10-23 09:47:26,948 - WARNING - Not found cell and umi barcode in entry 87 of the bam file
2019-10-23 09:47:26,948 - WARNING - Not found cell and umi barcode in entry 98 of the bam file
2019-10-23 09:47:26,948 - WARNING - Not found cell and umi barcode in entry 144 of the bam file
2019-10-23 09:47:26,948 - WARNING - Not found cell and umi barcode in entry 146 of the bam file
2019-10-23 09:47:26,948 - WARNING - Not found cell and umi barcode in entry 190 of the bam file
2019-10-23 09:47:26,948 - WARNING - Not found cell and umi barcode in entry 191 of the bam file
2019-10-23 09:47:26,948 - WARNING - Not found cell and umi barcode in entry 195 of the bam file
2019-10-23 09:47:26,948 - WARNING - Not found cell and umi barcode in entry 303 of the bam file
2019-10-23 09:47:26,948 - WARNING - Not found cell and umi barcode in entry 309 of the bam file
2019-10-23 09:47:26,948 - WARNING - Not found cell and umi barcode in entry 326 of the bam file
2019-10-23 09:47:26,948 - WARNING - Not found cell and umi barcode in entry 345 of the bam file
2019-10-23 09:47:26,948 - WARNING - Not found cell and umi barcode in entry 347 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 392 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 395 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 402 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 403 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 404 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 405 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 406 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 407 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 408 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 409 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 410 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 411 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 412 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 413 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 414 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 415 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 416 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 417 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 418 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 419 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 420 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 445 of the bam file
2019-10-23 09:47:26,949 - WARNING - Not found cell and umi barcode in entry 461 of the bam file
2019-10-23 09:47:26,950 - WARNING - Not found cell and umi barcode in entry 553 of the bam file
2019-10-23 09:47:26,950 - WARNING - Not found cell and umi barcode in entry 759 of the bam file
2019-10-23 09:47:26,950 - WARNING - Not found cell and umi barcode in entry 767 of the bam file
2019-10-23 09:47:26,950 - WARNING - Not found cell and umi barcode in entry 768 of the bam file
2019-10-23 09:47:26,950 - WARNING - Not found cell and umi barcode in entry 854 of the bam file
2019-10-23 09:47:26,968 - INFO - Starting the sorting process of /mnt/DATA2/RAW_DATA/Re_Run_ 2018_Data_with_updated_reference_genome/KUL-1-1000cells/outs/possorted_genome_bam.bam the output will be at: /mnt/DATA2/RAW_DATA/Re_Run_ 2018_Data_with_updated_reference_genome/KUL-1-1000cells/outs/cellsorted_possorted_genome_bam.bam
2019-10-23 09:47:26,968 - INFO - Command being run is: samtools sort -l 7 -m 24000M -t CB -O BAM -@ 10 -o /mnt/DATA2/RAW_DATA/Re_Run_ 2018_Data_with_updated_reference_genome/KUL-1-1000cells/outs/cellsorted_possorted_genome_bam.bam /mnt/DATA2/RAW_DATA/Re_Run_ 2018_Data_with_updated_reference_genome/KUL-1-1000cells/outs/possorted_genome_bam.bam
2019-10-23 09:47:26,968 - INFO - While the bam sorting happens do other things...
2019-10-23 09:47:26,968 - INFO - Load the annotation from /mnt/DATA1/Velocyto/refdata-cellranger-mm10-3.0.0/genes/genes.gtf
Usage: samtools sort [options...] [in.bam]
Options:
  -l INT     Set compression level, from 0 (uncompressed) to 9 (best)
  -m INT     Set maximum memory per thread; suffix K/M/G recognized [768M]
  -n         Sort by read name
  -t TAG     Sort by value of TAG. Uses position as secondary index (or read name if -n is set)
  -o FILE    Write final output to FILE rather than standard output
  -T PREFIX  Write temporary files to PREFIX.nnnn.bam
      --input-fmt-option OPT[=VAL]
               Specify a single input file format option in the form
               of OPTION or OPTION=VALUE
  -O, --output-fmt FORMAT[,OPT[=VAL]]...
               Specify output format (SAM, BAM, CRAM)
      --output-fmt-option OPT[=VAL]
               Specify a single output file format option in the form
               of OPTION or OPTION=VALUE
      --reference FILE
               Reference sequence FASTA FILE [null]
  -@, --threads INT
               Number of additional threads to use [0]
2019-10-23 09:47:30,072 - DEBUG - Parsing Chromosome 1 strand - [line 0]
2019-10-23 09:47:30,341 - DEBUG - Done with 1- [line 55263]
2019-10-23 09:47:30,341 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:30,344 - DEBUG - Seen 852 genes until now
2019-10-23 09:47:30,344 - DEBUG - Parsing Chromosome 1 strand + [line 55264]
2019-10-23 09:47:30,603 - DEBUG - Done with 1+ [line 108827]
2019-10-23 09:47:30,603 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:30,607 - DEBUG - Seen 1744 genes until now
2019-10-23 09:47:30,607 - DEBUG - Parsing Chromosome 10 strand - [line 108828]
2019-10-23 09:47:30,711 - DEBUG - Done with 10- [line 148287]
2019-10-23 09:47:30,711 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:30,713 - DEBUG - Seen 2492 genes until now
2019-10-23 09:47:30,713 - DEBUG - Parsing Chromosome 10 strand + [line 148288]
2019-10-23 09:47:30,822 - DEBUG - Done with 10+ [line 190661]
2019-10-23 09:47:30,822 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:30,825 - DEBUG - Seen 3285 genes until now
2019-10-23 09:47:30,825 - DEBUG - Parsing Chromosome 11 strand - [line 190662]
2019-10-23 09:47:31,123 - DEBUG - Done with 11- [line 252011]
2019-10-23 09:47:31,123 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:31,127 - DEBUG - Seen 4335 genes until now
2019-10-23 09:47:31,127 - DEBUG - Parsing Chromosome 11 strand + [line 252012]
2019-10-23 09:47:31,437 - DEBUG - Done with 11+ [line 314062]
2019-10-23 09:47:31,437 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:31,441 - DEBUG - Seen 5320 genes until now
2019-10-23 09:47:31,441 - DEBUG - Parsing Chromosome 12 strand - [line 314063]
2019-10-23 09:47:31,507 - DEBUG - Done with 12- [line 338958]
2019-10-23 09:47:31,507 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:31,509 - DEBUG - Seen 6056 genes until now
2019-10-23 09:47:31,509 - DEBUG - Parsing Chromosome 12 strand + [line 338959]
2019-10-23 09:47:31,596 - DEBUG - Done with 12+ [line 372787]
2019-10-23 09:47:31,596 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:31,598 - DEBUG - Seen 6615 genes until now
2019-10-23 09:47:31,598 - DEBUG - Parsing Chromosome 13 strand - [line 372788]
2019-10-23 09:47:31,685 - DEBUG - Done with 13- [line 406792]
2019-10-23 09:47:31,685 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:31,687 - DEBUG - Seen 7339 genes until now
2019-10-23 09:47:31,688 - DEBUG - Parsing Chromosome 13 strand + [line 406793]
2019-10-23 09:47:31,924 - DEBUG - Done with 13+ [line 433455]
2019-10-23 09:47:31,925 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:31,927 - DEBUG - Seen 8023 genes until now
2019-10-23 09:47:31,927 - DEBUG - Parsing Chromosome 14 strand - [line 433456]
2019-10-23 09:47:32,022 - DEBUG - Done with 14- [line 469680]
2019-10-23 09:47:32,022 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:32,024 - DEBUG - Seen 8704 genes until now
2019-10-23 09:47:32,024 - DEBUG - Parsing Chromosome 14 strand + [line 469681]
2019-10-23 09:47:32,109 - DEBUG - Done with 14+ [line 502857]
2019-10-23 09:47:32,109 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:32,111 - DEBUG - Seen 9518 genes until now
2019-10-23 09:47:32,111 - DEBUG - Parsing Chromosome 15 strand - [line 502858]
2019-10-23 09:47:32,207 - DEBUG - Done with 15- [line 539049]
2019-10-23 09:47:32,207 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:32,209 - DEBUG - Seen 10092 genes until now
2019-10-23 09:47:32,209 - DEBUG - Parsing Chromosome 15 strand + [line 539050]
2019-10-23 09:47:32,289 - DEBUG - Done with 15+ [line 569723]
2019-10-23 09:47:32,289 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:32,291 - DEBUG - Seen 10616 genes until now
2019-10-23 09:47:32,291 - DEBUG - Parsing Chromosome 16 strand - [line 569724]
2019-10-23 09:47:32,551 - DEBUG - Done with 16- [line 596286]
2019-10-23 09:47:32,552 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:32,554 - DEBUG - Seen 11093 genes until now
2019-10-23 09:47:32,554 - DEBUG - Parsing Chromosome 16 strand + [line 596287]
2019-10-23 09:47:32,633 - DEBUG - Done with 16+ [line 626571]
2019-10-23 09:47:32,633 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:32,634 - DEBUG - Seen 11545 genes until now
2019-10-23 09:47:32,634 - DEBUG - Parsing Chromosome 17 strand - [line 626572]
2019-10-23 09:47:32,720 - DEBUG - Done with 17- [line 659846]
2019-10-23 09:47:32,720 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:32,723 - DEBUG - Seen 12152 genes until now
2019-10-23 09:47:32,723 - DEBUG - Parsing Chromosome 17 strand + [line 659847]
2019-10-23 09:47:32,815 - DEBUG - Done with 17+ [line 696472]
2019-10-23 09:47:32,815 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:32,817 - DEBUG - Seen 12800 genes until now
2019-10-23 09:47:32,818 - DEBUG - Parsing Chromosome 18 strand - [line 696473]
2019-10-23 09:47:32,851 - DEBUG - Done with 18- [line 709486]
2019-10-23 09:47:32,851 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:32,852 - DEBUG - Seen 13059 genes until now
2019-10-23 09:47:32,852 - DEBUG - Parsing Chromosome 18 strand + [line 709487]
2019-10-23 09:47:32,898 - DEBUG - Done with 18+ [line 728027]
2019-10-23 09:47:32,898 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:32,899 - DEBUG - Seen 13409 genes until now
2019-10-23 09:47:32,899 - DEBUG - Parsing Chromosome 19 strand - [line 728028]
2019-10-23 09:47:32,962 - DEBUG - Done with 19- [line 752316]
2019-10-23 09:47:32,962 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:32,963 - DEBUG - Seen 13815 genes until now
2019-10-23 09:47:32,963 - DEBUG - Parsing Chromosome 19 strand + [line 752317]
2019-10-23 09:47:33,016 - DEBUG - Done with 19+ [line 773600]
2019-10-23 09:47:33,016 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:33,017 - DEBUG - Seen 14224 genes until now
2019-10-23 09:47:33,017 - DEBUG - Parsing Chromosome 2 strand - [line 773601]
2019-10-23 09:47:33,427 - DEBUG - Done with 2- [line 848821]
2019-10-23 09:47:33,427 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:33,432 - DEBUG - Seen 15398 genes until now
2019-10-23 09:47:33,432 - DEBUG - Parsing Chromosome 2 strand + [line 848822]
2019-10-23 09:47:33,611 - DEBUG - Done with 2+ [line 918704]
2019-10-23 09:47:33,611 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:33,615 - DEBUG - Seen 16582 genes until now
2019-10-23 09:47:33,616 - DEBUG - Parsing Chromosome 3 strand - [line 918705]
2019-10-23 09:47:33,714 - DEBUG - Done with 3- [line 957202]
2019-10-23 09:47:33,715 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:33,717 - DEBUG - Seen 17302 genes until now
2019-10-23 09:47:33,717 - DEBUG - Parsing Chromosome 3 strand + [line 957203]
2019-10-23 09:47:33,827 - DEBUG - Done with 3+ [line 1000138]
2019-10-23 09:47:33,827 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:33,830 - DEBUG - Seen 18021 genes until now
2019-10-23 09:47:33,830 - DEBUG - Parsing Chromosome 4 strand - [line 1000139]
2019-10-23 09:47:34,205 - DEBUG - Done with 4- [line 1050727]
2019-10-23 09:47:34,205 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:34,208 - DEBUG - Seen 18875 genes until now
2019-10-23 09:47:34,208 - DEBUG - Parsing Chromosome 4 strand + [line 1050728]
2019-10-23 09:47:34,334 - DEBUG - Done with 4+ [line 1099723]
2019-10-23 09:47:34,335 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:34,337 - DEBUG - Seen 19731 genes until now
2019-10-23 09:47:34,338 - DEBUG - Parsing Chromosome 5 strand - [line 1099724]
2019-10-23 09:47:34,488 - DEBUG - Done with 5- [line 1157394]
2019-10-23 09:47:34,488 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:34,492 - DEBUG - Seen 20622 genes until now
2019-10-23 09:47:34,492 - DEBUG - Parsing Chromosome 5 strand + [line 1157395]
2019-10-23 09:47:34,640 - DEBUG - Done with 5+ [line 1215626]
2019-10-23 09:47:34,640 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:34,644 - DEBUG - Seen 21539 genes until now
2019-10-23 09:47:34,644 - DEBUG - Parsing Chromosome 6 strand - [line 1215627]
2019-10-23 09:47:34,758 - DEBUG - Done with 6- [line 1260326]
2019-10-23 09:47:34,758 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:34,762 - DEBUG - Seen 22385 genes until now
2019-10-23 09:47:34,762 - DEBUG - Parsing Chromosome 6 strand + [line 1260327]
2019-10-23 09:47:34,884 - DEBUG - Done with 6+ [line 1307744]
2019-10-23 09:47:34,885 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:34,888 - DEBUG - Seen 23304 genes until now
2019-10-23 09:47:34,888 - DEBUG - Parsing Chromosome 7 strand - [line 1307745]
2019-10-23 09:47:35,358 - DEBUG - Done with 7- [line 1379880]
2019-10-23 09:47:35,358 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:35,364 - DEBUG - Seen 24656 genes until now
2019-10-23 09:47:35,364 - DEBUG - Parsing Chromosome 7 strand + [line 1379881]
2019-10-23 09:47:35,538 - DEBUG - Done with 7+ [line 1450335]
2019-10-23 09:47:35,538 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:35,542 - DEBUG - Seen 25873 genes until now
2019-10-23 09:47:35,542 - DEBUG - Parsing Chromosome 8 strand - [line 1450336]
2019-10-23 09:47:35,634 - DEBUG - Done with 8- [line 1485523]
2019-10-23 09:47:35,634 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:35,637 - DEBUG - Seen 26580 genes until now
2019-10-23 09:47:35,637 - DEBUG - Parsing Chromosome 8 strand + [line 1485524]
2019-10-23 09:47:35,755 - DEBUG - Done with 8+ [line 1531960]
2019-10-23 09:47:35,755 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:35,757 - DEBUG - Seen 27398 genes until now
2019-10-23 09:47:35,758 - DEBUG - Parsing Chromosome 9 strand - [line 1531961]
2019-10-23 09:47:35,893 - DEBUG - Done with 9- [line 1583207]
2019-10-23 09:47:35,893 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:35,896 - DEBUG - Seen 28230 genes until now
2019-10-23 09:47:35,896 - DEBUG - Parsing Chromosome 9 strand + [line 1583208]
2019-10-23 09:47:36,035 - DEBUG - Done with 9+ [line 1635954]
2019-10-23 09:47:36,035 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,039 - DEBUG - Seen 29142 genes until now
2019-10-23 09:47:36,039 - DEBUG - Parsing Chromosome GL456210.1 strand - [line 1635955]
2019-10-23 09:47:36,039 - DEBUG - Done with GL456210.1- [line 1635981]
2019-10-23 09:47:36,039 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,039 - DEBUG - Seen 29145 genes until now
2019-10-23 09:47:36,039 - DEBUG - Parsing Chromosome GL456210.1 strand + [line 1635982]
2019-10-23 09:47:36,039 - DEBUG - Done with GL456210.1+ [line 1635996]
2019-10-23 09:47:36,039 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,039 - DEBUG - Seen 29147 genes until now
2019-10-23 09:47:36,039 - DEBUG - Parsing Chromosome GL456211.1 strand - [line 1635997]
2019-10-23 09:47:36,039 - DEBUG - Done with GL456211.1- [line 1636024]
2019-10-23 09:47:36,039 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,039 - DEBUG - Seen 29151 genes until now
2019-10-23 09:47:36,039 - DEBUG - Parsing Chromosome GL456211.1 strand + [line 1636025]
2019-10-23 09:47:36,039 - DEBUG - Done with GL456211.1+ [line 1636050]
2019-10-23 09:47:36,039 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,039 - DEBUG - Seen 29153 genes until now
2019-10-23 09:47:36,039 - DEBUG - Parsing Chromosome GL456212.1 strand - [line 1636051]
2019-10-23 09:47:36,040 - DEBUG - Done with GL456212.1- [line 1636086]
2019-10-23 09:47:36,040 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,040 - DEBUG - Seen 29155 genes until now
2019-10-23 09:47:36,040 - DEBUG - Parsing Chromosome GL456216.1 strand + [line 1636087]
2019-10-23 09:47:36,040 - DEBUG - Done with GL456216.1+ [line 1636106]
2019-10-23 09:47:36,040 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,040 - DEBUG - Seen 29156 genes until now
2019-10-23 09:47:36,040 - DEBUG - Parsing Chromosome GL456219.1 strand - [line 1636107]
2019-10-23 09:47:36,040 - DEBUG - Done with GL456219.1- [line 1636118]
2019-10-23 09:47:36,040 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,040 - DEBUG - Seen 29158 genes until now
2019-10-23 09:47:36,040 - DEBUG - Parsing Chromosome GL456221.1 strand - [line 1636119]
2019-10-23 09:47:36,040 - DEBUG - Done with GL456221.1- [line 1636162]
2019-10-23 09:47:36,040 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,040 - DEBUG - Seen 29162 genes until now
2019-10-23 09:47:36,040 - DEBUG - Parsing Chromosome GL456221.1 strand + [line 1636163]
2019-10-23 09:47:36,040 - DEBUG - Done with GL456221.1+ [line 1636181]
2019-10-23 09:47:36,040 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,040 - DEBUG - Seen 29165 genes until now
2019-10-23 09:47:36,040 - DEBUG - Parsing Chromosome GL456233.1 strand - [line 1636182]
2019-10-23 09:47:36,040 - DEBUG - Done with GL456233.1- [line 1636213]
2019-10-23 09:47:36,040 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,040 - DEBUG - Seen 29167 genes until now
2019-10-23 09:47:36,040 - DEBUG - Parsing Chromosome GL456233.1 strand + [line 1636214]
2019-10-23 09:47:36,040 - DEBUG - Done with GL456233.1+ [line 1636244]
2019-10-23 09:47:36,040 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,040 - DEBUG - Seen 29169 genes until now
2019-10-23 09:47:36,040 - DEBUG - Parsing Chromosome GL456350.1 strand - [line 1636245]
2019-10-23 09:47:36,041 - DEBUG - Done with GL456350.1- [line 1636304]
2019-10-23 09:47:36,041 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,041 - DEBUG - Seen 29174 genes until now
2019-10-23 09:47:36,041 - DEBUG - Parsing Chromosome GL456354.1 strand - [line 1636305]
2019-10-23 09:47:36,041 - DEBUG - Done with GL456354.1- [line 1636337]
2019-10-23 09:47:36,041 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,041 - DEBUG - Seen 29177 genes until now
2019-10-23 09:47:36,041 - DEBUG - Parsing Chromosome GL456354.1 strand + [line 1636338]
2019-10-23 09:47:36,041 - DEBUG - Done with GL456354.1+ [line 1636357]
2019-10-23 09:47:36,041 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,041 - DEBUG - Seen 29179 genes until now
2019-10-23 09:47:36,041 - DEBUG - Parsing Chromosome JH584292.1 strand + [line 1636358]
2019-10-23 09:47:36,041 - DEBUG - Done with JH584292.1+ [line 1636433]
2019-10-23 09:47:36,041 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,041 - DEBUG - Seen 29180 genes until now
2019-10-23 09:47:36,041 - DEBUG - Parsing Chromosome JH584293.1 strand - [line 1636434]
2019-10-23 09:47:36,041 - DEBUG - Done with JH584293.1- [line 1636500]
2019-10-23 09:47:36,041 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,041 - DEBUG - Seen 29186 genes until now
2019-10-23 09:47:36,041 - DEBUG - Parsing Chromosome JH584293.1 strand + [line 1636501]
2019-10-23 09:47:36,042 - DEBUG - Done with JH584293.1+ [line 1636542]
2019-10-23 09:47:36,042 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,042 - DEBUG - Seen 29189 genes until now
2019-10-23 09:47:36,042 - DEBUG - Parsing Chromosome JH584294.1 strand - [line 1636543]
2019-10-23 09:47:36,042 - DEBUG - Done with JH584294.1- [line 1636607]
2019-10-23 09:47:36,042 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,042 - DEBUG - Seen 29193 genes until now
2019-10-23 09:47:36,042 - DEBUG - Parsing Chromosome JH584294.1 strand + [line 1636608]
2019-10-23 09:47:36,042 - DEBUG - Done with JH584294.1+ [line 1636631]
2019-10-23 09:47:36,042 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,042 - DEBUG - Seen 29195 genes until now
2019-10-23 09:47:36,042 - DEBUG - Parsing Chromosome JH584295.1 strand - [line 1636632]
2019-10-23 09:47:36,042 - DEBUG - Done with JH584295.1- [line 1636647]
2019-10-23 09:47:36,042 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,042 - DEBUG - Seen 29196 genes until now
2019-10-23 09:47:36,042 - DEBUG - Parsing Chromosome JH584296.1 strand - [line 1636648]
2019-10-23 09:47:36,042 - DEBUG - Done with JH584296.1- [line 1636675]
2019-10-23 09:47:36,042 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,042 - DEBUG - Seen 29199 genes until now
2019-10-23 09:47:36,042 - DEBUG - Parsing Chromosome JH584297.1 strand - [line 1636676]
2019-10-23 09:47:36,042 - DEBUG - Done with JH584297.1- [line 1636694]
2019-10-23 09:47:36,042 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,042 - DEBUG - Seen 29201 genes until now
2019-10-23 09:47:36,042 - DEBUG - Parsing Chromosome JH584297.1 strand + [line 1636695]
2019-10-23 09:47:36,042 - DEBUG - Done with JH584297.1+ [line 1636703]
2019-10-23 09:47:36,042 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,043 - DEBUG - Seen 29202 genes until now
2019-10-23 09:47:36,043 - DEBUG - Parsing Chromosome JH584298.1 strand - [line 1636704]
2019-10-23 09:47:36,043 - DEBUG - Done with JH584298.1- [line 1636714]
2019-10-23 09:47:36,043 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,043 - DEBUG - Seen 29203 genes until now
2019-10-23 09:47:36,043 - DEBUG - Parsing Chromosome JH584298.1 strand + [line 1636715]
2019-10-23 09:47:36,043 - DEBUG - Done with JH584298.1+ [line 1636735]
2019-10-23 09:47:36,043 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,043 - DEBUG - Seen 29205 genes until now
2019-10-23 09:47:36,043 - DEBUG - Parsing Chromosome JH584299.1 strand - [line 1636736]
2019-10-23 09:47:36,043 - DEBUG - Done with JH584299.1- [line 1636745]
2019-10-23 09:47:36,043 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,043 - DEBUG - Seen 29206 genes until now
2019-10-23 09:47:36,043 - DEBUG - Parsing Chromosome JH584299.1 strand + [line 1636746]
2019-10-23 09:47:36,043 - DEBUG - Done with JH584299.1+ [line 1636857]
2019-10-23 09:47:36,043 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,043 - DEBUG - Seen 29217 genes until now
2019-10-23 09:47:36,043 - DEBUG - Parsing Chromosome JH584303.1 strand + [line 1636858]
2019-10-23 09:47:36,043 - DEBUG - Done with JH584303.1+ [line 1636862]
2019-10-23 09:47:36,043 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,043 - DEBUG - Seen 29218 genes until now
2019-10-23 09:47:36,043 - DEBUG - Parsing Chromosome JH584304.1 strand - [line 1636863]
2019-10-23 09:47:36,043 - DEBUG - Done with JH584304.1- [line 1636895]
2019-10-23 09:47:36,043 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,043 - DEBUG - Seen 29219 genes until now
2019-10-23 09:47:36,043 - DEBUG - Parsing Chromosome MT strand - [line 1636896]
2019-10-23 09:47:36,043 - DEBUG - Done with MT- [line 1636901]
2019-10-23 09:47:36,043 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,043 - DEBUG - Seen 29220 genes until now
2019-10-23 09:47:36,044 - DEBUG - Parsing Chromosome MT strand + [line 1636902]
2019-10-23 09:47:36,044 - DEBUG - Done with MT+ [line 1636970]
2019-10-23 09:47:36,044 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,044 - DEBUG - Seen 29232 genes until now
2019-10-23 09:47:36,044 - DEBUG - Parsing Chromosome X strand - [line 1636971]
2019-10-23 09:47:36,115 - DEBUG - Done with X- [line 1663962]
2019-10-23 09:47:36,115 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,117 - DEBUG - Seen 29755 genes until now
2019-10-23 09:47:36,117 - DEBUG - Parsing Chromosome X strand + [line 1663963]
2019-10-23 09:47:36,193 - DEBUG - Done with X+ [line 1694333]
2019-10-23 09:47:36,193 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,195 - DEBUG - Seen 30320 genes until now
2019-10-23 09:47:36,195 - DEBUG - Parsing Chromosome Y strand - [line 1694334]
2019-10-23 09:47:36,207 - DEBUG - Done with Y- [line 1698842]
2019-10-23 09:47:36,207 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,208 - DEBUG - Seen 30662 genes until now
2019-10-23 09:47:36,208 - DEBUG - Parsing Chromosome Y strand + [line 1698843]
2019-10-23 09:47:36,223 - DEBUG - Assigning indexes to genes
2019-10-23 09:47:36,223 - DEBUG - Done with Y+ [line 1704063]
2019-10-23 09:47:36,224 - DEBUG - Fixing corner cases of transcript models containg intron longer than 1000Kbp
2019-10-23 09:47:36,896 - DEBUG - Generated 1423079 features corresponding to 111079 transcript models from /mnt/DATA1/Velocyto/refdata-cellranger-mm10-3.0.0/genes/genes.gtf
2019-10-23 09:47:36,912 - INFO - Load the repeat masking annotation from /mnt/DATA1/Velocyto/alltracks_mask.gtf
2019-10-23 09:47:36,912 - DEBUG - Reading /mnt/DATA1/Velocyto/alltracks_mask.gtf, the file will be sorted in memory
2019-10-23 09:47:55,053 - DEBUG - Processed masked annotation .gtf and generated 4252016 intervals to mask!
2019-10-23 09:47:55,237 - INFO - Scan /mnt/DATA2/RAW_DATA/Re_Run_ 2018_Data_with_updated_reference_genome/KUL-1-1000cells/outs/possorted_genome_bam.bam to validate intron intervals
2019-10-23 09:47:56,531 - DEBUG - Reading /mnt/DATA2/RAW_DATA/Re_Run_ 2018_Data_with_updated_reference_genome/KUL-1-1000cells/outs/possorted_genome_bam.bam
2019-10-23 09:47:56,538 - DEBUG - Read first 0 million reads
2019-10-23 09:47:56,539 - DEBUG - Marking up chromosome 1
2019-10-23 09:48:03,445 - DEBUG - Marking up chromosome 10
2019-10-23 09:48:08,626 - DEBUG - Marking up chromosome 11
2019-10-23 09:48:18,369 - DEBUG - Marking up chromosome 12
2019-10-23 09:48:22,738 - DEBUG - Marking up chromosome 13
2019-10-23 09:48:27,363 - DEBUG - Marking up chromosome 14
2019-10-23 09:48:32,116 - DEBUG - Marking up chromosome 15
2019-10-23 09:48:32,559 - DEBUG - Read first 10 million reads
2019-10-23 09:48:36,630 - DEBUG - Marking up chromosome 16
2019-10-23 09:48:39,858 - DEBUG - Marking up chromosome 17
2019-10-23 09:48:47,293 - DEBUG - Marking up chromosome 18
2019-10-23 09:48:51,713 - DEBUG - Marking up chromosome 19
2019-10-23 09:48:58,072 - DEBUG - Marking up chromosome 2
2019-10-23 09:49:05,819 - DEBUG - Marking up chromosome 3
2019-10-23 09:49:10,883 - DEBUG - Marking up chromosome 4
2019-10-23 09:49:13,937 - DEBUG - Read first 20 million reads
2019-10-23 09:49:17,630 - DEBUG - Marking up chromosome 5
2019-10-23 09:49:24,642 - DEBUG - Marking up chromosome 6
2019-10-23 09:49:31,974 - DEBUG - Marking up chromosome 7
2019-10-23 09:49:44,244 - DEBUG - Marking up chromosome 8
2019-10-23 09:49:49,368 - DEBUG - Marking up chromosome 9
2019-10-23 09:49:52,799 - DEBUG - Read first 30 million reads
2019-10-23 09:49:55,667 - DEBUG - Marking up chromosome MT
2019-10-23 09:49:57,463 - DEBUG - Marking up chromosome X
2019-10-23 09:50:02,825 - DEBUG - Marking up chromosome Y
2019-10-23 09:50:02,953 - DEBUG - Marking up chromosome JH584299.1
2019-10-23 09:50:02,954 - DEBUG - Marking up chromosome GL456233.1
2019-10-23 09:50:02,959 - DEBUG - Marking up chromosome GL456211.1
2019-10-23 09:50:02,971 - DEBUG - Marking up chromosome GL456221.1
2019-10-23 09:50:02,978 - DEBUG - Marking up chromosome JH584297.1
2019-10-23 09:50:02,979 - DEBUG - Marking up chromosome GL456354.1
2019-10-23 09:50:02,980 - DEBUG - Marking up chromosome JH584300.1
2019-10-23 09:50:02,980 - WARNING - The .bam file refers to a chromosome 'JH584300.1+' not present in the annotation (.gtf) file
2019-10-23 09:50:02,980 - WARNING - The .bam file refers to a chromosome 'JH584300.1-' not present in the annotation (.gtf) file
2019-10-23 09:50:02,980 - DEBUG - Marking up chromosome GL456210.1
2019-10-23 09:50:02,993 - DEBUG - Marking up chromosome GL456212.1
2019-10-23 09:50:02,993 - WARNING - The .bam file refers to a chromosome 'GL456212.1+' not present in the annotation (.gtf) file
2019-10-23 09:50:02,998 - DEBUG - Marking up chromosome JH584304.1
2019-10-23 09:50:02,998 - WARNING - The .bam file refers to a chromosome 'JH584304.1+' not present in the annotation (.gtf) file
2019-10-23 09:50:03,015 - DEBUG - Marking up chromosome GL456379.1
2019-10-23 09:50:03,015 - WARNING - The .bam file refers to a chromosome 'GL456379.1+' not present in the annotation (.gtf) file
2019-10-23 09:50:03,015 - WARNING - The .bam file refers to a chromosome 'GL456379.1-' not present in the annotation (.gtf) file
2019-10-23 09:50:03,015 - DEBUG - Marking up chromosome GL456216.1
2019-10-23 09:50:03,015 - WARNING - The .bam file refers to a chromosome 'GL456216.1-' not present in the annotation (.gtf) file
2019-10-23 09:50:03,020 - DEBUG - Marking up chromosome GL456393.1
2019-10-23 09:50:03,020 - WARNING - The .bam file refers to a chromosome 'GL456393.1+' not present in the annotation (.gtf) file
2019-10-23 09:50:03,020 - WARNING - The .bam file refers to a chromosome 'GL456393.1-' not present in the annotation (.gtf) file
2019-10-23 09:50:03,021 - DEBUG - Marking up chromosome GL456367.1
2019-10-23 09:50:03,021 - WARNING - The .bam file refers to a chromosome 'GL456367.1+' not present in the annotation (.gtf) file
2019-10-23 09:50:03,021 - WARNING - The .bam file refers to a chromosome 'GL456367.1-' not present in the annotation (.gtf) file
2019-10-23 09:50:03,021 - DEBUG - Marking up chromosome GL456239.1
2019-10-23 09:50:03,021 - WARNING - The .bam file refers to a chromosome 'GL456239.1+' not present in the annotation (.gtf) file
2019-10-23 09:50:03,021 - WARNING - The .bam file refers to a chromosome 'GL456239.1-' not present in the annotation (.gtf) file
2019-10-23 09:50:03,021 - DEBUG - Marking up chromosome GL456360.1
2019-10-23 09:50:03,021 - WARNING - The .bam file refers to a chromosome 'GL456360.1+' not present in the annotation (.gtf) file
2019-10-23 09:50:03,021 - WARNING - The .bam file refers to a chromosome 'GL456360.1-' not present in the annotation (.gtf) file
2019-10-23 09:50:03,021 - DEBUG - Marking up chromosome GL456372.1
2019-10-23 09:50:03,021 - WARNING - The .bam file refers to a chromosome 'GL456372.1+' not present in the annotation (.gtf) file
2019-10-23 09:50:03,021 - WARNING - The .bam file refers to a chromosome 'GL456372.1-' not present in the annotation (.gtf) file
2019-10-23 09:50:03,030 - DEBUG - Marking up chromosome GL456370.1
2019-10-23 09:50:03,030 - WARNING - The .bam file refers to a chromosome 'GL456370.1+' not present in the annotation (.gtf) file
2019-10-23 09:50:03,030 - WARNING - The .bam file refers to a chromosome 'GL456370.1-' not present in the annotation (.gtf) file
2019-10-23 09:50:03,031 - DEBUG - Marking up chromosome GL456390.1
2019-10-23 09:50:03,031 - WARNING - The .bam file refers to a chromosome 'GL456390.1+' not present in the annotation (.gtf) file
2019-10-23 09:50:03,031 - WARNING - The .bam file refers to a chromosome 'GL456390.1-' not present in the annotation (.gtf) file
2019-10-23 09:50:03,031 - DEBUG - Marking up chromosome GL456392.1
2019-10-23 09:50:03,031 - WARNING - The .bam file refers to a chromosome 'GL456392.1+' not present in the annotation (.gtf) file
2019-10-23 09:50:03,031 - WARNING - The .bam file refers to a chromosome 'GL456392.1-' not present in the annotation (.gtf) file
2019-10-23 09:50:03,031 - DEBUG - Marking up chromosome GL456382.1
2019-10-23 09:50:03,031 - WARNING - The .bam file refers to a chromosome 'GL456382.1+' not present in the annotation (.gtf) file
2019-10-23 09:50:03,031 - WARNING - The .bam file refers to a chromosome 'GL456382.1-' not present in the annotation (.gtf) file
2019-10-23 09:50:03,123 - DEBUG - Marking up chromosome GL456396.1
2019-10-23 09:50:03,123 - WARNING - The .bam file refers to a chromosome 'GL456396.1+' not present in the annotation (.gtf) file
2019-10-23 09:50:03,123 - WARNING - The .bam file refers to a chromosome 'GL456396.1-' not present in the annotation (.gtf) file
2019-10-23 09:50:03,123 - DEBUG - Marking up chromosome GL456368.1
2019-10-23 09:50:03,123 - WARNING - The .bam file refers to a chromosome 'GL456368.1+' not present in the annotation (.gtf) file
2019-10-23 09:50:03,123 - WARNING - The .bam file refers to a chromosome 'GL456368.1-' not present in the annotation (.gtf) file
2019-10-23 09:50:03,123 - DEBUG - Marking up chromosome JH584295.1
2019-10-23 09:50:03,123 - WARNING - The .bam file refers to a chromosome 'JH584295.1+' not present in the annotation (.gtf) file
2019-10-23 09:50:06,398 - DEBUG - End of file. Reset index: start scanning from initial position.
2019-10-23 09:50:06,398 - DEBUG - 234971 reads were skipped because no apropiate cell or umi barcode was found
2019-10-23 09:50:06,398 - INFO - Now just waiting that the bam sorting process terminates
Traceback (most recent call last):
  File "/home/luna.kuleuven.be/u0119129/anaconda3/bin/velocyto", line 11, in <module>
    load_entry_point('velocyto', 'console_scripts', 'velocyto')()
  File "/home/luna.kuleuven.be/u0119129/anaconda3/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/luna.kuleuven.be/u0119129/anaconda3/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/luna.kuleuven.be/u0119129/anaconda3/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/luna.kuleuven.be/u0119129/anaconda3/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/luna.kuleuven.be/u0119129/anaconda3/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/luna.kuleuven.be/u0119129/velocyto.py/velocyto/commands/run10x.py", line 115, in run10x
    samtools_memory=samtools_memory, dump=dump, loom_numeric_dtype=dtype, verbose=verbose, additional_ca=additional_ca)
  File "/home/luna.kuleuven.be/u0119129/velocyto.py/velocyto/commands/_run.py", line 225, in _run
    Otherwise sort manually by samtools ``sort -l [compression] -m [mb_to_use]M -t [tagname] -O BAM -@ [threads_to_use] -o cellsorted_[bamfile] [bamfile]``")
MemoryError: bam file #0 could not be sorted by cells.
                This is probably related to an old version of samtools, please install samtools >= 1.6.                In alternative this could be a memory error, try to set the --samtools_memory option to a value compatible with your system.                 Otherwise sort manually by samtools ``sort -l [compression] -m [mb_to_use]M -t [tagname] -O BAM -@ [threads_to_use] -o cellsorted_[bamfile] [bamfile]``

@saeedfc
Copy link

saeedfc commented Oct 23, 2019

Solved.
Ran samtools first to get the cellsorted file. And then ran velocyto
in the documentation, it says "If the file cellsorted_[ORIGINALBAMNAME] exists, the sorting procedure will be skipped and the file present will be used."

(base) u0119129@gbw-d-l0099:~$ samtools sort -t CB -O BAM -o/mnt/DATA2/RAW_DATA/Re_Run_\ 2018_Data_with_updated_reference_genome/KUL-1-1000cells/outs/cellsorted_possorted_genome_bam.bam /mnt/DATA2/RAW_DATA/Re_Run_\ 2018_Data_with_updated_reference_genome/KUL-1-1000cells/outs/possorted_genome_bam.bam

(base) u0119129@gbw-d-l0099:~$ velocyto run10x -m /mnt/DATA1/Velocyto/alltracks_mask.gtf /mnt/DATA2/RAW_DATA/Re_Run_\ 2018_Data_with_updated_reference_genome/KUL-1-1000cells /mnt/DATA1/Velocyto/refdata-cellranger-mm10-3.0.0/genes/genes.gtf

Thanks a lot @igordot and velocyto team

@gauravsinghrathore
Copy link

or rather install samtools using conda solves the problem automatically.

@sky1ove
Copy link

sky1ove commented Mar 24, 2022

for linux/ ubuntu, I solved this problem by 'sudo apt-install samtools'

@chrissymkcn
Copy link

@gauravsinghrathore that works like a charm! thanks a bunch

@megadesk
Copy link

@sky1ove thanks ! 'sudo yum install samtools' solved the problem for me on a centos7 machine

@IsaacUtah1379
Copy link

Igor is right, samtools is a requirement. Internally velocyto uses Popen to
call samtools sort.

I will change the instructions in the tutorial.

The tutorial's instructions still haven't been updated.

@IsaacUtah1379 IsaacUtah1379 linked a pull request Feb 5, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update installation instructions to include samtools IsaacUtah1379/velocyto.py
9 participants
@gioelelm @igordot @slinnarsson @gauravsinghrathore @sky1ove @saeedfc @chrissymkcn @megadesk @IsaacUtah1379