-
Notifications
You must be signed in to change notification settings - Fork 16
/
Copy pathHTS.pm
2394 lines (1829 loc) · 78.4 KB
/
HTS.pm
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
=head1 LICENSE
See the NOTICE file distributed with this work for additional information
regarding copyright ownership.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
=head1 NAME
Bio::DB::HTS -- Read files using HTSlib including BAM/CRAM, Tabix and BCF database files
=head1 SYNOPSIS
use Bio::DB::HTS;
# high level API
# Note that the high level API does not reset the CRAM file pointer to the start
# of the file as the method to do so is (at time or writing) not easily accessible.
# Therefore a new HTS object may be needed to repeat a query.
my $hts = Bio::DB::HTS->new(-bam =>"data/ex1.bam",
-fasta=>"data/ex1.fa",
);
my @targets = $hts->seq_ids;
my @alignments = $hts->get_features_by_location(-seq_id => 'seq2',
-start => 500,
-end => 800);
for my $a (@alignments) {
# where does the alignment start in the reference sequence
my $seqid = $a->seq_id;
my $start = $a->start;
my $end = $a->end;
my $strand = $a->strand;
my $cigar = $a->cigar_str;
my $paired = $a->get_tag_values('PAIRED');
# where does the alignment start in the query sequence
my $query_start = $a->query->start;
my $query_end = $a->query->end;
my $ref_dna = $a->dna; # reference sequence bases
my $query_dna = $a->query->dna; # query sequence bases
my @scores = $a->qscore; # per-base quality scores
my $match_qual= $a->qual; # quality of the match
}
my @pairs = $hts->get_features_by_location(-type => 'read_pair',
-seq_id => 'seq2',
-start => 500,
-end => 800);
for my $pair (@pairs)
{
my $length = $pair->length; # insert length
my ($first_mate,$second_mate) = $pair->get_SeqFeatures;
my $f_start = $first_mate->start;
my $s_start = $second_mate->start;
}
# low level API
my $hfile = Bio::DB::HTSfile->open('/path/to/alignment_file');
my $header = $hfile->header_read;
my $target_count = $header->n_targets;
my $target_names = $header->target_name;
while (my $align = $hfile->read1($header))
{
my $seqid = $target_names->[$align->tid];
my $start = $align->pos+1;
my $end = $align->calend;
my $cigar = $align->cigar_str;
}
Bio::DB::HTSfile->index_build($bamfile);
my $index = Bio::DB::HTSfile->index_load($hfile);
my $index = Bio::DB::HTSfile->index_open_in_safewd($hfile);
my $callback = sub {
my $alignment = shift;
my $start = $alignment->start;
my $end = $alignment->end;
my $seqid = $target_names->[$alignment->tid];
print $alignment->qname," aligns to $seqid:$start..$end\n";
}
my $header = $index->header;
$index->fetch($hfile,$header->parse_region('seq2'),$callback);
=head1 DESCRIPTION
This module provides a Perl interface to the HTSlib library for
indexed and unindexed SAM/BAM/CRAM sequence alignment databases.
It provides support for retrieving information on individual alignments,
read pairs, and alignment coverage information across large
regions. It also provides callback functionality for calling SNPs and
performing other base-by-base functions.
=head2 The high-level API
The high-level API provides a BioPerl-compatible interface to indexed
BAM and CRAM files. The alignment file database is treated as a collection of
Bio::SeqFeatureI features, and can be searched for features by name,
location, type and combinations of feature tags such as whether the
alignment is part of a mate-pair.
When opening a alignment database using the high-level API, you provide the
pathnames of two files: the FASTA file that contains the reference
genome sequence, and the BAM file that contains the query sequences
and their alignments. If either of the two files needs to be indexed,
the indexing will need to be built. You can then query the
database for alignment features by combinations of name, position,
type, and feature tag.
The high-level API provides access to up to four feature "types":
* "match": The "raw" unpaired alignment between a read and the
reference sequence.
* "read_pair": Paired alignments; a single composite
feature that contains two subfeatures for the alignments of each
of the mates in a mate pair.
* "coverage": A feature that spans a region of interest that contains
numeric information on the coverage of reads across the region.
* "region": A way of retrieving information about the reference
sequence. Searching for features of type "region" will return a
list of chromosomes or contigs in the reference sequence, rather
than read alignments.
* "chromosome": A synonym for "region".
B<Features> can be en masse in a single call, retrieved in a
memory-efficient streaming basis using an iterator, or interrogated
using a filehandle that return a series of SAM-format lines.
B<SAM alignment flags> can be retrieved using BioPerl's feature "tag"
mechanism. For example, to interrogate the FIRST_MATE flag, one
fetches the "FIRST_MATE" tag:
warn "aye aye captain!" if $alignment->get_tag_values('FIRST_MATE');
The Bio::SeqFeatureI interface has been extended to retrieve all flags
as a compact human-readable string, and to return the CIGAR alignment
in a variety of formats.
B<Split alignments>, such as reads that cover introns, are dealt with
in one of two ways. The default is to leave split alignments alone:
they can be detected by one or more "N" operations in the CIGAR
string. Optionally, you can choose to have the API split these
alignments across two or more subfeatures; the CIGAR strings of these
split alignments will be adjusted accordingly.
B<Interface to the pileup routines> The API provides you with access
to the samtools "pileup" API. This gives you the ability to write a
callback that will be invoked on every column of the alignment for the
purpose of calculating coverage, quality score metrics, or SNP
calling.
B<Access to the reference sequence> When you create the Bio::DB::HTS
object, you can pass the path to a FASTA file containing the reference
sequence. Alternatively, you may pass an object that knows how to
retrieve DNA sequences across a range via the seq() or fetch_seq()
methods, as described under new().
If the SAM/BAM file has MD tags, then these tags will be used to
reconstruct the reference sequence when necessary, in which case you
can completely omit the -fasta argument. Note that not all SAM/BAM
files have MD tags, and those that do may not use them correctly due
to the newness of this part of the SAM spec. You may wish to populate
these tags using samtools' "calmd" command.
If the -fasta argument is omitted and no MD tags are present, then the
reference sequence will be returned as 'N'.
The B<main object classes> that you will be dealing with in the
high-level API are as follows:
* Bio::DB::HTS -- A collection of alignments and reference sequences.
* Bio::DB::HTS::Alignment -- The alignment between a query and the reference.
* Bio::DB::HTS::Query -- An object corresponding to the query sequence in
which both (+) and (-) strand alignments are
shown in the reference (+) strand.
* Bio::DB::HTS::Target -- An interface to the query sequence in which
(-) strand alignments are shown in reverse
complement
You may encounter other classes as well. These include:
* Bio::DB::HTS::Segment -- This corresponds to a region on the reference
sequence.
* Bio::DB::HTS::Constants -- This defines CIGAR symbol constants and flags.
* Bio::DB::HTS::AlignWrapper -- An alignment helper object that adds split
alignment functionality. See Bio::DB::HTS::Alignment
for the documentation on using it.
* Bio::DB::HTS::ReadIterator -- An iterator that mediates the one-feature-at-a-time
retrieval mechanism.
* Bio::DB::HTS::FetchIterator -- Another iterator for feature-at-a-time retrieval.
=head2 The low-level API
The low-level API closely mirrors that of the HTSlib library. It
provides the ability to open and read SAM, BAM and CRAM files,
build indexes, and perform searches across them.
The classes you will be interacting with in the low-level API are as
follows:
* Bio::DB::HTS -- Methods that read and write SAM, BAM and CRAM files.
* Bio::DB::HTS::Header -- Methods for manipulating the BAM file header.
* Bio::DB::HTS::Alignment -- Methods for manipulating alignment data.
* Bio::DB::HTS::Pileup -- Methods for manipulating the pileup data structure.
* Bio::DB::HTS::Fai -- Methods for creating and reading from indexed Fasta
files.
=head1 METHODS
We cover the high-level API first. The high-level API code can be
found in the files Bio/DB/HTS.pm and Bio/DB/HTS/*.pm.
=head2 Bio::DB::HTS Constructor and basic accessors
=over 4
=item $sam = Bio::DB::HTS->new(%options)
The Bio::DB::HTS object combines a Fasta file of the reference
sequences with an SAM/BAM/CRAM alignment file to allow for convenient retrieval of
human-readable sequence IDs and reference sequences. The new()
constructor accepts a -name=>value style list of options as
follows:
Option Description
------ -------------
-bam Path to the SAM/BAM/CRAM alignment file that contains the
alignments (required). A http: or ftp: URL is accepted.
-fasta Path to the Fasta file that contains
the reference sequences (optional). Alternatively,
you may pass any object that supports a seq()
or fetch_seq() method and takes the three arguments
($seq_id,$start,$end).
-expand_flags A boolean value. If true then the standard
alignment flags will be broken out as
individual tags such as 'M_UNMAPPED' (default false).
-split_splices A boolean value. If true, then alignments that
are split across splices will be broken out
into a single alignment containing two sub-
alignments (default false).
-split The same as -split_splices.
-force_refseq Always use the reference sequence file to derive the
reference sequence, even when the sequence can be
derived from the MD tag. This is slower, but safer
when working with BAM files derived from buggy aligners
or when the reference contains non-canonical (modified)
bases.
-autoindex Create an alignment index file if one does not exist
or the current one has a modification date
earlier than the alignment file.
An example of a typical new() constructor invocation is:
$hts = Bio::DB::HTS->new(-fasta => '/home/projects/genomes/hu17.fa',
-bam => '/home/projects/alignments/ej88.bam',
-expand_flags => 1,
-split_splices => 1);
If the B<-fasta> argument is present, then you will be able to use the
interface to fetch the reference sequence's bases. Otherwise, calls
that return the reference sequence will return sequences consisting
entirely of "N".
B<-expand_flags> option, if true, has the effect of turning each of
the standard SAM flags into a separately retrievable B<tag> in the
Bio::SeqFeatureI interface. Otherwise, the standard flags will be
concatenated in easily parseable form as a tag named "FLAGS". See
get_all_tags() and get_tag_values() for more information.
Any two-letter extension flags, such as H0 or H1, will always appear
as separate tags regardless of the setting.
B<-split_splices> has the effect of breaking up alignments that
contain an "N" operation into subparts for more convenient
manipulation. For example, if you have both paired reads and spliced
alignments in the BAM file, the following code shows the subpart
relationships:
$pair = $hts->get_feature_by_name('E113:01:01:23');
@mates = $pair->get_SeqFeatures;
@mate1_parts = $mates[0]->get_SeqFeatures;
@mate2_parts = $mates[1]->get_SeqFeatures;
Because there is some overhead to splitting up the spliced alignments,
this option is false by default.
B<Remote access> to alignment files located on an HTTP or FTP server is
possible. Simply replace the path to the BAM file with the appropriate
URL. Note that incorrect URLs may lead to a core dump.
It is not currently possible to refer to a remote FASTA file. These
will have to be downloaded locally and indexed before using.
=item $flag = $hts->expand_flags([$new_value])
Get or set the expand_flags option. This can be done after object
creation and will have an immediate effect on all alignments fetched
from the alignment file.
=item $flag = $hts->split_splices([$new_value])
Get or set the split_splices option. This can be done after object
creation and will affect all alignments fetched from the alignment file
B<subsequently.>
=item $header = $hts->header
Return the Bio::DB::HTS::Header object associated with the alignment
file. You can manipulate the header using the low-level API.
=item $hts_path = $hts->hts_path
Return the path of the alignment file used to create the hts object. This
makes the object more portable.
=item $hts_file = $hts->$hts_file
Returns the low-level Bio::DB::HTSfile object associated with the opened
file.
=item $fai = $hts->fai
Returns the Bio::DB::HTS::Fai object associated with the Fasta
file. You can then manipulate this object with the low-level API.
B<The index can be built automatically for you if it does not already
exist.> If index building is necessarily, the process will need write
privileges to the same directory in which the Fasta file resides.> If
the process does not have write permission, then the call will fail.
=item $hts_idx = $hts->hts_index
Return the Bio::DB::HTS::Index object associated with the alignment file.
The index is not automatically built.
=item $hts->clone
Bio::DB::HTS objects are not stable across fork() operations. If you
fork, you must call clone() either in the parent or the child process
before attempting to call any methods.
=back
=head2 Getting information about reference sequences
The Bio::DB::HTS object provides the following methods for getting
information about the reference sequence(s) contained in the
associated Fasta file.
=over 4
=item @seq_ids = $hts->seq_ids
Returns an unsorted list of the IDs of the reference sequences (known
elsewhere in this document as seq_ids). This is the same as the
identifier following the ">" sign in the Fasta file (e.g. "chr1").
=item $num_targets = $hts->n_targets
Return the number of reference sequences.
=item $length = $hts->length('seqid')
Returns the length of the reference sequence named "seqid".
=item $seq_id = $hts->target_name($tid)
Translates a numeric target ID (TID) returned by the low-level API
into a seq_id used by the high-level API.
=item $length = $hts->target_len($tid)
Translates a numeric target ID (TID) from the low-level API to a
sequence length.
=item $dna = $hts->seq($seqid,$start,$end)
Returns the DNA across the region from start to end on reference
seqid. Note that this is a string, not a Bio::PrimarySeq object. If
no -fasta path was passed when the sam object was created, then you
will receive a series of N nucleotides of the requested length.
=back
=head2 Creating and querying segments
Bio::DB::HTS::Segment objects refer regions on the reference
sequence. They can be used to retrieve the sequence of the reference,
as well as alignments that overlap with the region.
=over 4
=item $segment = $hts->segment($seqid,$start,$end);
=item $segment = $hts->segment(-seq_id=>'chr1',-start=>5000,-end=>6000);
Segments are created using the Bio:DB::HTS->segment() method. It can
be called using one to three positional arguments corresponding to the
seq_id of the reference sequence, and optionally the start and end
positions of a subregion on the sequence. If the start and/or end are
undefined, they will be replaced with the beginning and end of the
sequence respectively.
Alternatively, you may call segment() with named -seq_id, -start and
-end arguments.
All coordinates are 1-based.
=item $seqid = $segment->seq_id
Return the segment's sequence ID.
=item $start = $segment->start
Return the segment's start position.
=item $end = $segment->end
Return the segment's end position.
=item $strand = $segment->strand
Return the strand of the segment (always 0).
=item $length = $segment->length
Return the length of the segment.
=item $dna = $segment->dna
Return the DNA string for the reference sequence under this segment.
=item $seq = $segment->seq
Return a Bio::PrimarySeq object corresponding to the sequence of the
reference under this segment. You can get the actual DNA string in
this redundant-looking way:
$dna = $segment->seq->seq
The advantage of working with a Bio::PrimarySeq object is that you can
perform operations on it, including taking its reverse complement and
subsequences.
=item @alignments = $segment->features(%args)
Return alignments that overlap the segment in the associated alignment
file. The optional %args list allows you to filter features by name,
tag or other attributes. See the documentation of the
Bio::DB::HTS->features() method for the full list of options. Here are
some typical examples:
# get all the overlapping alignments
@all_alignments = $segment->features;
# get an iterator across the alignments
my $iterator = $segment->features(-iterator=>1);
while (my $align = $iterator->next_seq) { do something }
# get a SAM filehandle across the alignments
my $fh = $segment->features(-fh=>1);
while (<$fh>) { print }
# get only the alignments with unmapped mates
my @unmapped = $segment->features(-flags=>{M_UNMAPPED=>1});
# get coverage across this region
my ($coverage) = $segment->features('coverage');
my @data_points = $coverage->coverage;
# grep through features using a coderef
my @reverse_alignments = $segment->features(
-filter => sub {
my $a = shift;
return $a->strand < 0;
});
=item $tag = $segment->primary_tag
=item $tag = $segment->source_tag
Return the strings "region" and "sam/bam" respectively. These methods
allow the segment to be passed to BioPerl methods that expect
Bio::SeqFeatureI objects.
=item $segment->name, $segment->display_name, $segment->get_SeqFeatures, $segment->get_tag_values
These methods are provided for Bio::SeqFeatureI compatibility and
don't do anything of interest.
=back
=head2 Retrieving alignments, mate pairs and coverage information
The features() method is an all-purpose tool for retrieving alignment
information from the SAM/BAM/CRAM alignment file database. In addition, the methods
get_features_by_name(), get_features_by_location() and others provide
convenient shortcuts to features().
These methods either return a list of features, an iterator across a
list of features, or a filehandle opened on a pseudo-SAM file.
=over 4
=item @features = $hts->features(%options)
=item $iterator = $hts->features(-iterator=>1,%more_options)
=item $filehandle = $hts->features(-fh=>1,%more_options)
=item @features = $hts->features('type1','type2'...)
This is the all-purpose interface for fetching alignments and other
types of features from the database. Arguments are a -name=>value
option list selected from the following list of options:
Option Description
------ -------------
-type Filter on features of a given type. You may provide
either a scalar typename, or a reference to an
array of desired feature types. Valid types are
"match", "read_pair", "coverage" and "chromosome."
See below for a full explanation of feature types.
-name Filter on reads with the designated name. Note that
this can be a slow operation unless accompanied by
the feature location as well.
-seq_id Filter on features that align to seq_id between start
-start and end. -start and -end must be used in conjunction
-end with -seq_id. If -start and/or -end are absent, they
will default to 1 and the end of the reference
sequence, respectively.
-flags Filter features that match a list of one or more
flags. See below for the format.
-attributes The same as -flags, for compatibility with other
-tags APIs.
-filter Filter on features with a coderef. The coderef will
receive a single argument consisting of the feature
and should return true to keep the feature, or false
to discard it.
-iterator Instead of returning a list of features, return an
iterator across the results. To retrieve the results,
call the iterator's next_seq() method repeatedly
until it returns undef to indicate that no more
matching features remain.
-fh Instead of returning a list of features, return a
filehandle. Read from the filehandle to retrieve
each of the results in TAM format, one alignment
per line read. This only works for features of type
"match."
The high-level API introduces the concept of a B<feature "type"> in order
to provide several convenience functions. You specify types by using
the optional B<-type> argument. The following types are currently
supported:
B<match>. The "match" type corresponds to the unprocessed SAM
alignment. It will retrieve single reads, either mapped or
unmapped. Each match feature's primary_tag() method will return the
string "match." The features returned by this call are of type
Bio::DB::HTS::AlignWrapper.
B<read_pair>. The "paired_end" type causes the sam interface to find
and merge together mate pairs. Fetching this type of feature will
yield a series of Bio::SeqFeatureI objects, each as long as the total
distance on the reference sequence spanned by the mate pairs. The
top-level feature is of type Bio::SeqFeature::Lite; it contains two
Bio::DB::HTS::AlignWrapper subparts.
Call get_SeqFeatures() to get the two individual reads. Example:
my @pairs = $hts->features(-type=>'read_pair');
my $p = $pairs[0];
my $i_length = $p->length;
my @ends = $p->get_SeqFeatures;
my $left = $ends[0]->start;
my $right = $ends[1]->end;
B<coverage>. The "coverage" type causes the sam interface to calculate
coverage across the designated region. It only works properly if
accompanied by the desired location of the coverage graph; -seq_id is
a mandatory argument for coverage calculation, and -start and -end are
optional. The call will return a single Bio::SeqFeatureI object whose
primary_tag() is "coverage." To recover the coverage data, call the
object's coverage() method to obtain an array (list context) or
arrayref (scalar context) of coverage counts across the region of
interest:
my ($coverage) = $hts->features(-type=>'coverage',-seq_id=>'seq1');
my @data = $coverage->coverage;
my $total;
for (@data) { $total += $_ }
my $average_coverage = $total/@data;
By default the coverage graph will be at the base pair level. So for a
region 5000 bp wide, coverage() will return an array or arrayref with
exactly 5000 elements. However, you also have the option of
calculating the coverage across larger bins. Simply append the number
of intervals you are interested to the "coverage" typename. For
example, fetching "coverage:500" will return a feature whose
coverage() method will return the coverage across 500 intervals.
B<chromosome> or B<region>. The "chromosome" or "region" type are
interchangeable. They ask the sam interface to construct
Bio::DB::HTS::Segment representing the reference sequences. These two
calls give similar results:
my $segment = $hts->segment('seq2',1=>500);
my ($seg) = $hts->features(-type=>'chromosome',
-seq_id=>'seq2',-start=>1,-end=>500);
Due to an unresolved bug, you cannot fetch chromosome features in the
same call with matches and other feature types call. Specifically,
this works as expected:
my @chromosomes = $hts->features (-type=>'chromosome');
But this doesn't (as of 18 June 2009):
my @chromosomes_and_matches = $hts->features(-type=>['match','chromosome']);
If no -type argument is provided, then features() defaults to finding
features of type "match."
You may call features() with a plain list of strings (positional
arguments, not -type=>value arguments). This will be interpreted as a
list of feature types to return:
my ($coverage) = $hts->features('coverage')
For a description of the methods available in the features returned
from this call, please see L<Bio::SeqfeatureI> and
L<Bio::DB::HTS::Alignment>.
You can B<filter> "match" and "read_pair" features by name, location
and/or flags. The name and flag filters are not very efficient. Unless
they are combined with a location filter, they will initiate an
exhaustive search of the BAM database.
Name filters are case-insensitive, and allow you to use shell-style
"*" and "?" wildcards. Flag filters created with the B<-flag>,
B<-attribute> or B<-tag> options have the following syntax:
-flag => { FLAG_NAME_1 => ['list','of','possible','values'],
FLAG_NAME_2 => ['list','of','possible','values'],
...
}
The value of B<-flag> is a hash reference in which the keys are flag
names and the values are array references containing lists of
acceptable values. The list of values are OR'd with each other, and
the flag names are AND'd with each other.
The B<-filter> option provides a completely generic filtering
interface. Provide a reference to a subroutine. It will be called
once for each potential feature. Return true to keep the feature, or
false to discard it. Here is an example of how to find all matches
whose alignment quality scores are greater than 80.
@features = $hts->features(-filter=>sub {shift->qual > 80} );
By default, features() returns a list of all matching features. You
may instead request an iterator across the results list by passing
-iterator=>1. This will give you an object that has a single method,
next_seq():
my $high_qual = $hts->features(-filter => sub {shift->qual > 80},
-iterator=> 1 );
while (my $feature = $high_qual->next_seq) {
# do something with the alignment
}
Similarly, by passing a true value to the argument B<-fh>, you can
obtain a filehandle to a virtual SAM file. This only works with the
"match" feature type:
my $high_qual = $hts->features(-filter => sub {shift->qual > 80},
-fh => 1 );
while (my $tam_line = <$high_qual>) {
chomp($tam_line);
# do something with it
}
=item @features = $hts->get_features_by_name($name)
Convenience method. The same as calling $hts->features(-name=>$name);
=item $feature = $hts->get_feature_by_name($name)
Convenience method. The same as ($hts->features(-name=>$name))[0].
=item @features = $hts->get_features_by_location($seqid,$start,$end)
Convenience method. The same as calling
$hts->features(-seq_id=>$seqid,-start=>$start,-end=>$end).
=item @features = $hts->get_features_by_flag(%flags)
Convenience method. The same as calling
$hts->features(-flags=>\%flags). This method is also called
get_features_by_attribute() and get_features_by_tag(). Example:
@features = $hts->get_features_by_flag(H0=>1)
=item $feature = $hts->get_feature_by_id($id)
The high-level API assigns each feature a unique ID composed of its
read name, position and strand and returns it when you call the
feature's primary_id() method. Given that ID, this method returns the
feature.
=item $iterator = $hts->get_seq_stream(%options)
Convenience method. This is the same as calling
$hts->features(%options,-iterator=>1).
=item $fh = $hts->get_seq_fh(%options)
Convenience method. This is the same as calling
$hts->features(%options,-fh=>1).
=item $fh = $hts->tam_fh
Convenience method. It is the same as calling $hts->features(-fh=>1).
=item @types = $hts->types
This method returns the list of feature types (e.g. "read_pair")
returned by the current version of the interface.
=back
=head2 The generic fetch() and pileup() methods
Lastly, the high-level API supports two methods for rapidly traversing
indexed BAM databases.
=over 4
=item $hts->fetch($region,$callback)
This method traverses the indicated region and invokes a callback
code reference on each match. Specify a region using the syntax
"seqid:start-end", or either of the alternative syntaxes
"seqid:start..end" and "seqid:start,end". If start and end are absent,
then the entire reference sequence is traversed. If end is absent,
then the end of the reference sequence is assumed.
The callback will be called repeatedly with a
Bio::DB::HTS::AlignWrapper on the argument list.
Example:
$hts->fetch('seq1:600-700',
sub {
my $a = shift;
print $a->display_name,' ',$a->cigar_str,"\n";
});
Note that the fetch() operation works on reads that B<overlap> the
indicated region. Therefore the callback may be called for reads that
align to the reference at positions that start before or end after the
indicated region.
=item $hts->pileup($region,$callback [,$keep_level])
This method, which is named after the native bam_lpileupfile()
function in the C interfaces, traverses the indicated region and
generates a "pileup" of all the mapped reads that cover it. The
user-provided callback function is then invoked on each position of
the alignment along with a data structure that provides access to the
individual aligned reads.
As with fetch(), the region is specified as a string in the format
"seqid:start-end", "seqid:start..end" or "seqid:start,end".
The callback is a coderef that will be invoked with three arguments:
the seq_id of the reference sequence, the current position on the
reference (in 1-based coordinates!), and a reference to an array of
Bio::DB::HTS::Pileup objects. Here is the typical call signature:
sub {
my ($seqid,$pos,$pileup) = @_;
# do something
}
For example, if you call pileup on the region "seq1:501-600", then the
callback will be invoked for all reads that overlap the indicated
region. The first invocation of the callback will typically have a
$pos argument somewhat to the left of the desired region and the last
call will be somewhat to the right. You may wish to ignore positions
that are outside of the requested region. Also be aware that the
reference sequence position uses 1-based coordinates, which is
different from the low-level interface, which use 0-based coordinates.
The size of the $pileup array reference indicates the read coverage
at that position. Here is a simple average coverage calculator:
my $depth = 0;
my $positions = 0;
my $callback = sub {
my ($seqid,$pos,$pileup) = @_;
next unless $pos >= 501 && $pos <= 600;
$positions++;
$depth += @$pileup;
}
$hts->pileup('seq1:501-600',$callback);
print "coverage = ",$depth/$positions;
Each Bio::DB::HTS::Pileup object describes the position of a read in
the alignment. Briefly, Bio::DB::HTS::Pileup has the following
methods:
$pileup->alignment The alignment at this level (a
Bio::DB::HTS::AlignWrapper object).
$pileup->qpos The position of the read base at the pileup site,
in 0-based coordinates.
$pileup->pos The position of the read base at the pileup site,
in 1-based coordinates;
$pileup->level The level of the read in the multiple alignment
view. Note that this field is only valid when
$keep_level is true, so it may not be relevant post
htslib move.
$pileup->indel Length of the indel at this position: 0 for no indel, positive
for an insertion (relative to the reference), negative for a
deletion (relative to the reference.)
$pileup->is_del True if the base on the padded read is a deletion.
$pileup->is_refskip True if the base on the padded read is a gap relative to the reference (denoted as < or > in the pileup)
$pileup->is_head True if this is the first base in the query sequence.
$pileup->is_tail True if this is the last base in the query sequence.
See L</Examples> for a very simple SNP caller.
=item $hts->fast_pileup($region,$callback [,$keep_level])
This is identical to pileup() except that the pileup object returns
low-level Bio::DB::HTS::Alignment objects rather than the higher-level
Bio::DB::HTS::AlignWrapper objects. This makes it roughly 50% faster,
but you lose the align objects' seq_id() and get_tag_values()
methods. As a compensation, the callback receives an additional
argument corresponding to the Bio::DB::HTS object. You can use this to
create AlignWrapper objects on an as needed basis:
my $callback = sub {
my($seqid,$pos,$pileup,$hts) = @_;
for my $p (@$pileup) {
my $alignment = $p->alignment;
my $wrapper = Bio::DB::HTS::AlignWrapper->new($alignment,$hts);
my $has_mate = $wrapper->get_tag_values('PAIRED');
}
};
=item Bio::DB::HTS->max_pileup_cnt([$new_cnt])
=item $hts->max_pileup_cnt([$new_cnt])
The HTSlib library caps pileups at a set level, defaulting to
8000. The callback will not be invoked on a single position more than
the level set by the cap, even if there are more reads. Called with no
arguments, this method returns the current cap value. Called with a
numeric argument, it changes the cap. There is currently no way to
specify an unlimited cap.
This method can be called as an instance method or a class method.
=item $hts->coverage2BedGraph([$fh])
This special-purpose method will compute a four-column BED graph of
the coverage across the entire alignment file and print it to STDOUT.
You may provide a filehandle to redirect output to a file or pipe.
=back
The next sections correspond to the low-level API, which let you
create and manipulate Perl objects that correspond directly to data
structures in the C interface. A major difference between the high and
low level APIs is that in the high-level API, the reference sequence
is identified using a human-readable seq_id. However, in the low-level
API, the reference is identified using a numeric target ID
("tid"). The target ID is established during the creation of the alignment
file and is a small 0-based integer index. The Bio::DB::HTS::Header
object provides methods for converting from seq_ids to tids.
=head2 Indexed Fasta Files
These methods relate to the indexed Fasta (".fai") files.
=over 4
=item $fai = Bio::DB::HTS::Fai->load('/path/to/file.fa')
Load an indexed Fasta file and return the object corresponding to
it. If the index does not exist, it will be created
automatically. Note that you pass the path to the Fasta file, not the
index.
For consistency with Bio::DB::HTS->open() this method is also called
open().
=item $dna_string = $fai->fetch("seqid:start-end")
Given a sequence ID contained in the Fasta file and optionally a
subrange in the form "start-end", finds the indicated subsequence and
returns it as a string.
=back
=head2 Alignment Files
These methods provide interfaces to alignment files in SAM/BAM/CRAM format.
=over 4
=item $hts_file = Bio::DB::HTSfile->open('/path/to/file.bam' [,$mode])
Open the alignment file at the indicated path. Mode, if present, must be
one of the file stream open flags ("r", "w", "wb", "wc", "a", "r+", etc.). If
absent, mode defaults to "r". [write formats: w = SAM, wb = BAM, wc = CRAM]
Note that Bio::DB::HTS objects are not stable across fork()
operations. If you fork, and intend to use the object in both parent
and child, you must reopen the Bio::DB::HTS in either the child or the
parent (but not both) before attempting to call any of the object's
methods.
The path may be an http: or ftp: URL, in which case a copy of the
index file will be downloaded to the current working directory (see
below) and all accesses will be performed on the remote BAM file.
Example:
$hfile = Bio::DB::HTSfile->open('http://some.site.com/nextgen/chr1_bowtie.bam');
=item $header = $hfile->header_read()
Given an open alignment file, return a Bio::DB::HTS::Header object
containing information about the reference sequence(s). Note that you
must invoke header_read() at least once before calling read1().
=item $status_code = $hfile->header_write($header, [$reference])
Given a Bio::DB::HTSfile::Header object and a BAM file opened in write mode, write the
header to the file. If the write fails the process will be terminated at the C layer.
If $hfile is CRAM formated a second argument $reference, which is the path to the
reference Fasta file, must be passed.
The result code is (currently) always zero.
=item $alignment = $hfile->read1($header)
Read one alignment from the alignment file and return it as a
Bio::DB::HTS::Alignment object. The $header parameter is returned by
invoking header().
=item $bytes = $hfile->write1($header, $alignment)
Given a BAM file that has been opened in write mode and a Bio::DB::HTS::Alignment object,