Add separate cache for bloomfilters

[jarifibrahim]

#1256 showed that a single cache might not be enough to store the
data blocks and the bloom filters.
This commit adds a separate cache for the bloom filters. This
commit also adds a new flag LoadBloomOnOpen which determines
if the bloom filters should be loaded when the table is opened on
or not.

The default value of MaxBfCacheSize is zero and
LoadBloomOnOpen is true.

This change has significant performance improvement on read speeds
because a single cache would lead to bloom filter eviction and we
would read the bloomfilter from the disk.

---

This change is 

[manishrjain]

Approved.

Reviewed 5 of 5 files at r1.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @ashish-goswami and @jarifibrahim)

---

options.go, line 131 at r1 (raw file):

		MaxCacheSize:            1 << 30, // 1 GB
		MaxBfCacheSize:          0,
		LoadBfLazily:            false,

LoadBloomsOnOpen

---

table/table.go, line 514 at r1 (raw file):

		if t.bf == nil {
			y.AssertTrue(t.opt.LoadBfLazily)
			t.bf, _ = t.readBloomFilter()

This might be introducing a race condition. Multiple goroutines could be reading and setting t.bf.

[jarifibrahim]

Reviewable status: 2 of 5 files reviewed, 2 unresolved discussions (waiting on @ashish-goswami and @manishrjain)

---

options.go, line 131 at r1 (raw file):

Previously, manishrjain (Manish R Jain) wrote…

LoadBloomsOnOpen

Done.

---

table/table.go, line 514 at r1 (raw file):

Previously, manishrjain (Manish R Jain) wrote…

This might be introducing a race condition. Multiple goroutines could be reading and setting t.bf.

The race condition could be possible but none of our tests picked up this race condition. I'll keep this lock-free for now.