Support for deleting keys

[judepereira]

No description provided.

[AkshayS96]

@judepereira
We can implement deletion using 2 steps:

1. remove the key from indexMap
2. add that key to an arraylist or write it to a file (for fault tolerance)
3. skip this key while compaction will be performed.

[judepereira]

What about during iteration? Will we have to check the indexMap at all times? Iteration performance might suffer.

[AkshayS96]

I think we can check whether a key is present in index map or not at the time of iteration itself here

stormdb/src/main/java/com/clevertap/stormdb/StormDB.java

Line 614 in 53edb09

final Consumer<ByteBuffer> entryConsumer = entry -> {

[judepereira]

The index map isn't checked. Only the bitset is checked.

[judepereira]

One good way to avoid checking the index map would be to "fake" reading them. by setting the keys directly to read in the keysRead bitset:

stormdb/src/main/java/com/clevertap/stormdb/StormDB.java

Line 612 in 53edb09

final BitSet keysRead = new BitSet(index.size());

[AkshayS96]

yeah, we can do this. first, iterate through the deleted keys array list and set them to this bitset.

[judepereira]

Sounds like a plan :)

[judepereira]

What do you think is the best method for persistence of the deleted keys?

[AkshayS96]

I was thinking of keeping both the in-memory data structure (Map or ArrayList) and a file(in case of sudden shutdown for persistence);

[judepereira]

Sounds good :)