PivotedStore

[rubanm]

Addresses #178

[rubanm]

This is a fun one.

@johnynek Can you take a peek? This change also adds an IterableStore since it looks like get would involve fetching all underlying keys that "contain" the outer key. Let me know if this makes sense. Thanks.

Tests TBD

[softprops]

In scala-land I like to steer clear of java-bean naming conventions. how about exposing something like IterableReadableStore#iterator ala Map#iterator. getAll also implies you are loading all members into memory. iterables/iterators can imply lazyiness which I like.

[rubanm]

Makes sense. Keeps the method name familiar and less scary.

[johnynek]

There is an object in util called a spool:

https://github.com/twitter/util/blob/master/util-core/src/main/scala/com/twitter/concurrent/Spool.scala

It may be that this returns a Future[Spool[(K1,V)]] which means that the we can make multiple future calls to page the results out.

[rubanm]

@softprops After reading your comment, I'm thinking Iterable might be a useful trait for reading from message queues or streams if we go down that path.

trait IterableSource[K, V] {
  def iterator : Future[Iterable[(K, V)]]
}

trait IterableReadableStore[-K, V] extends ReadableStore[K, V]
  with IterableSource[K, Option[V]]

trait IterableStore[-K, V] extends IterableReadableStore[K, V] with Store[K, V]

I'm not completely clear with the naming here. For sinks, we have kept the existing WriteableStore name, so IterableSource might be confusing.

Thoughts?

[johnynek]

1. This is cool, but I doubt many stores we want to use it with can actually be iterated like this.

2. Here is a second idea: make this ReadableStore[OuterK, ReadableStore[InnerK, V]]. I think that can be done without the Iterable for now.

[rubanm]

Yep, the point on not many stores being iterable is valid.

I'm not completely clear how making this extend ReadableStore[OuterK, ReadableStore[InnerK, V]] would help in avoiding Iterable though. Seems to me like we need some way to know which Ks in the underlying store constitute OuterK and InnerK when executing get(outerK).
Or what am I missing here?

[johnynek]

So, get(outerK) does nothing but adds a prefix. Then when someone calls .get on the inner store, you build the real keys and make the call.

I don't think you need to enumerate.

[rubanm]

Ah, I think I understand now :) Thanks.
So something like:

override def get(outerK: OuterK) : Future[Option[ReadableStore[InnerK, V]]] =
  Future.value(Some(ReadableStore[InnerK, V] {
    override def get(innerK: InnerK) {
    val k = inj((outerK, innerK))
    underlying.get(K)
  })
}

Also, this
http://thecodinglove.com/post/51069449556/senior-developer-explaining-how-to-use-his-library

[rubanm]

Posted implementation that uses ReadableStore[OuterK, ReadableStore[InnerK, V]].
A writeable version may still need some kind of enumeration, so I've reverted that for now.

Also removed IterableSource since that's probably going to be its own thing. Will send that out as a separate change.

[johnynek]

the usual pattern is that the injection be implicit. If you want to call explicitly, you can still do so. What do you think of that?

[rubanm]

Makes sense. Made it implicit.