How to Enable Cache to Set Changing Time to Lives

[rickilambert]

Hi There

I am playing around with Caffeine cache with some different use cases - and am not sure if this one in particular is achievable

I have an api that allows users to publish messages (essentially key value data) and I am trying to also allow them to specify how often this data becomes invalid on a per message basis - essentially they will pass me a time to live

I have seen that in Caffeine you can use the Policy.VarExpiration to achieve this but that means the cache must be initialised with a default expiry I believe - something like expireAfterWrite(ttl, TimeUnit.SECONDS)?

It seems as though when the cache is empty or the key does not currently exist then the Optional returned for cache.policy().expireVariably() is empty - so the only option is to do an insert in to the cache which will use the default TTL defined when the cache is instantiated?

Am i correct in thinking the above? Is it not possible to define an entirely empty cache with no default TTL, then for every put operation (whether the key is not present or whether it is an update) just provide the additional fields i need (the TTL)

Hope that makes sense - may be a bit of a ramble!

Thanks

[ben-manes]

You can use a default TTL that is essentially eternity, then use the VarExpiration api to put the entry in with an explicit duration.

Cache<Key, Graph> graphs = Caffeine.newBuilder()
    .expireAfter(Expiry.creating((Key key, Graph graph) -> Duration.ofNanos(Long.MAX_VALUE)))
    .build();
graphs.policy().expireVariably().orElseThrow()
    .put(key, value, Duration.ofMinutes(30));

wdyt?

[ben-manes]

Oh, I think your confusion was using expireAfterWrite and assuming it was a convenience method for variable expiration. It’s actually a different implementation entirely under the covers.

Caffeine only uses O(1) algorithms because we want to avoid surprising degradations. The fixed expiration does that using a simple doubly linked list.

Variable expiration was traditionally O(lg n) trees or lazily leaves dead entries a period sweep or LRU eviction. Neither are very efficient, are limited, and problematic. So it wasn’t initially supported as we explored ideas.

We much later added it using timing wheels, which are most often used by operating systems. That is more complex but O(1) thanks to hashing and amortization tricks.

Had we done that originally there would be just one algorithm and simpler policy api. Since it came later they coexist.

[rickilambert]

Exactly - Thanks so much for the explanation - that really helped