'Synchronous listener' isn't synchronous on expirations? #195
Comments
|
Yes, I think it is a disconnect of expectations. I am open to changing this in Java 9, if we can work out the details. Passive expirationCurrently expiration is not viewed as a scheduling service, where an event would be fired when the timer elapses. This would require a dedicated thread to actively perform the task, but the cache itself does not create any threads. Your use-case is reasonable, but falls into the gray area of what should be expected from expiration. As of now, expiration is performed passively. This is because the space was already taken and a time constraint is about freshness (not capacity). When an entry expires a miss is emulated if present, and it is discarded when the amortized maintenance cycle is triggered. Note that removing expired entries is O(1) for a fixed policy (LRU queue) and variable policy (timer wheel). The concept of a cache is to provide fast access to data within some policy constraints, so the passive behavior fits this purpose. The maintenance work can be triggered explicitly using InterceptorsThe Active expiration?Java 9 introduces a shared scheduler via The problem is active expiration assumes a strongly guarantee of event order. If the timer is not fired because order was imperfect then this would be viewed as a bug. That can occur for read-based expiration policies ( If active expiration was enabled, we would have to provide only best-effort for read-based but can guarantee it strongly for writes. That fits your expectations, but does add conceptual weight for anyone interested in this feature. AlternativesActive expiration might be outside of the scope of a cache. Its a fair debate. There are multiple ways you could perform the timing logic yourself. The simplest is if you are on Java 9 and can use Another possibility would be to schedule each entry in a Or, as mentioned above, if scheduled passively then you could have a periodic task that checks if there are any expired entries. Then you could still use the cache by calling |
|
Thanks for your elaborate response. I think I can follow most of it, and it makes sense that for this to be a general purpose (if you will) cache, there are bound to be some things falling outside the scope of a cache. I don't think I have a very strong idea on this definitely belonging in the scope of a cache considering I really don't oversee all of the drawbacks this might cause on the cache as a whole. Having said that I do think the documentation in that sense is a bit misleading:
I think especially that last sentence made me pretty sure that both I didn't really like the scheduled In the end I now schedule the task in a I'm kind of disliking the fact that I'm running this task for every request (and not only the ones that are actually timing out), but the intent is pretty clear at least and I don't expect thousands of requests per second yet. Thanks for the timer wheel read, interesting and I keep that in mind if I need the performance later on. |
In passive expiration, there is no write until the cache notices the entry expired and cleans up. The actual mutation on the hash table is communicated to the
I agree, 100ms is quite frequent and may seem wasteful when no work is needed. The overhead should be small due to using O(1) algorithms. This solution isn't great, but at least provides some support.
You could have the request future cancel the scheduler's using a |
|
Sorry for not getting onto this enhancement earlier. I have it implemented and half of the tests done, so now onto the hardest part - finding the time to wrap it up. I am curious if you think this would have been acceptable for your use-case, had it been available. The To avoid thrashing the scheduler there are a few minor caveats to pace the executions.
The maintenance work is cheap and fast, but pacing the executions seems appropriate. The tolerance to delay is necessary regardless, e.g. as For Java 9+ users, no additional threads are required by leveraging a system-wide scheduling thread hidden within The interface is below and is nothing special. The given task is run by the given executor, which is @FunctionalInterface
public interface Scheduler {
/**
* Returns a future that will submit the task to the given executor after the given delay.
*
* @param executor the executor to run the task
* @param command the runnable task to schedule
* @param delay how long to delay, in units of {@code unit}
* @param unit a {@code TimeUnit} determining how to interpret the {@code delay} parameter
*/
Future<?> schedule(Executor executor, Runnable command, long delay, TimeUnit unit);
/**
* Returns a scheduler that always returns a successfully completed future.
*
* @return a scheduler that always returns a successfully completed future
*/
static Scheduler disabledScheduler() {
return DisabledScheduler.INSTANCE;
}
/**
* Returns a scheduler that uses the system-when scheduling thread if available, or else returns
* {@link #disabledScheduler()} is not present. This scheduler is provided in Java 9 or above
* through {@link CompletableFuture#delayedExecutor}.
*
* @return a scheduler that uses the system-wide scheduling thread if available, or else a
* disabled scheduler
*/
static Scheduler systemScheduler() {
return SystemScheduler.isPresent() ? SystemScheduler.INSTANCE : disabledScheduler() ;
}
/**
* Returns a scheduler that delegates to the a {@link ScheduledExecutorService}.
*
* @param scheduledExecutorService the executor to schedule on
* @return a scheduler that delegates to the a {@link ScheduledExecutorService}
*/
static Scheduler forScheduledExecutorService(ScheduledExecutorService scheduledExecutorService) {
return new ExecutorServiceScheduler(scheduledExecutorService);
}
} |
Typically the cache using an amortized maintenance routine to periodically keep the eviction policies up to date. For size this is to replay hits or evict, for expiration this is to discover stale entries, and for reference caching this is to find GC'd keys or values. Each policy uses O(1) algorithms to be cheap, batches the work to avoid lock contention, and performs this routine maintenance as a side effect of normal cache operations. If there is no activity on the cache then the routine maintenance is not triggered, as that requires a background scheduling thread. Generally this is okay as the memory was already in consumed, no activity may indicate low system usage, and the cache is a transient data store. However, some do want to trigger business logic and leverage the cache as a timing subsystem, e.g. using the `RemovalListener` to notify another component. The cache itself does not create or manage threads, but can defer work to a configured `Executor`. Similarly now it can now schedule the maintenance routine on a configured `Scheduler`, default disabled. The cache will still perform maintenance periodically and this scheduler only augments it to add additional liveliness. The scheduling is based on the expiration time only; users may use `Cache.cleanUp()` for fixed periods to assist reference caching, etc as desired. The cache will rate limit the scheduler so that it runs at a reasonable pace and never thrashes. For example, if the next expiration times are tiny durations apart (e.g. 200ms), the cache will not run the maintenance task 5x/s. It will instead ensure a 1 second delay (2^30ns) between scheduled runs (not accounting for the lack of real-time guarantees by a Scheduler impl). Similarly an additional delay may occur due to the priority queues being held in best-effort order. The scheduling for variable expiration is based on the TimerWheel's bucket expiration, which cascades events. For example a timer of 1h 13m 5s would schedule to run in 1h, which would cascade the 1hr bucket into the minute buckets. The next schedule would be at 13m, followed by 5s. Java 9+ users should prefer using `Scheduler.systemScheduler()` rather than creating their own dedicated threads. This leverages the JVM-wide scheduling thread, primarily intended for `CompletableFuture.orTimeout(duration)`. In Java 8 this method returns `Scheduler.disabledScheduler` instead.
Typically the cache using an amortized maintenance routine to periodically keep the eviction policies up to date. For size this is to replay hits or evict, for expiration this is to discover stale entries, and for reference caching this is to find GC'd keys or values. Each policy uses O(1) algorithms to be cheap, batches the work to avoid lock contention, and performs this routine maintenance as a side effect of normal cache operations. If there is no activity on the cache then the routine maintenance is not triggered, as that requires a background scheduling thread. Generally this is okay as the memory was already in consumed, no activity may indicate low system usage, and the cache is a transient data store. However, some do want to trigger business logic and leverage the cache as a timing subsystem, e.g. using the `RemovalListener` to notify another component. The cache itself does not create or manage threads, but can defer work to a configured `Executor`. Similarly now it can now schedule the maintenance routine on a configured `Scheduler`, default disabled. The cache will still perform maintenance periodically and this scheduler only augments it to add additional liveliness. The scheduling is based on the expiration time only; users may use `Cache.cleanUp()` for fixed periods to assist reference caching, etc as desired. The cache will rate limit the scheduler so that it runs at a reasonable pace and never thrashes. For example, if the next expiration times are tiny durations apart (e.g. 200ms), the cache will not run the maintenance task 5x/s. It will instead ensure a 1 second delay (2^30ns) between scheduled runs (not accounting for the lack of real-time guarantees by a Scheduler impl). Similarly an additional delay may occur due to the priority queues being held in best-effort order. The scheduling for variable expiration is based on the TimerWheel's bucket expiration, which cascades events. For example a timer of 1h 13m 5s would schedule to run in 1h, which would cascade the 1hr bucket into the minute buckets. The next schedule would be at 13m, followed by 5s. Java 9+ users should prefer using `Scheduler.systemScheduler()` rather than creating their own dedicated threads. This leverages the JVM-wide scheduling thread, primarily intended for `CompletableFuture.orTimeout(duration)`. In Java 8 this method returns `Scheduler.disabledScheduler` instead.
Typically the cache using an amortized maintenance routine to periodically keep the eviction policies up to date. For size this is to replay hits or evict, for expiration this is to discover stale entries, and for reference caching this is to find GC'd keys or values. Each policy uses O(1) algorithms to be cheap, batches the work to avoid lock contention, and performs this routine maintenance as a side effect of normal cache operations. If there is no activity on the cache then the routine maintenance is not triggered, as that requires a background scheduling thread. Generally this is okay as the memory was already in consumed, no activity may indicate low system usage, and the cache is a transient data store. However, some do want to trigger business logic and leverage the cache as a timing subsystem, e.g. using the `RemovalListener` to notify another component. The cache itself does not create or manage threads, but can defer work to a configured `Executor`. Similarly now it can now schedule the maintenance routine on a configured `Scheduler`, default disabled. The cache will still perform maintenance periodically and this scheduler only augments it to add additional liveliness. The scheduling is based on the expiration time only; users may use `Cache.cleanUp()` for fixed periods to assist reference caching, etc as desired. The cache will rate limit the scheduler so that it runs at a reasonable pace and never thrashes. For example, if the next expiration times are tiny durations apart (e.g. 200ms), the cache will not run the maintenance task 5x/s. It will instead ensure a 1 second delay (2^30ns) between scheduled runs (not accounting for the lack of real-time guarantees by a Scheduler impl). Similarly an additional delay may occur due to the priority queues being held in best-effort order. The scheduling for variable expiration is based on the TimerWheel's bucket expiration, which cascades events. For example a timer of 1h 13m 5s would schedule to run in 1h, which would cascade the 1hr bucket into the minute buckets. The next schedule would be at 13m, followed by 5s. Java 9+ users should prefer using `Scheduler.systemScheduler()` rather than creating their own dedicated threads. This leverages the JVM-wide scheduling thread, primarily intended for `CompletableFuture.orTimeout(duration)`. In Java 8 this method returns `Scheduler.disabledScheduler` instead.
Typically the cache using an amortized maintenance routine to periodically keep the eviction policies up to date. For size this is to replay hits or evict, for expiration this is to discover stale entries, and for reference caching this is to find GC'd keys or values. Each policy uses O(1) algorithms to be cheap, batches the work to avoid lock contention, and performs this routine maintenance as a side effect of normal cache operations. If there is no activity on the cache then the routine maintenance is not triggered, as that requires a background scheduling thread. Generally this is okay as the memory was already in consumed, no activity may indicate low system usage, and the cache is a transient data store. However, some do want to trigger business logic and leverage the cache as a timing subsystem, e.g. using the `RemovalListener` to notify another component. The cache itself does not create or manage threads, but can defer work to a configured `Executor`. Similarly now it can now schedule the maintenance routine on a configured `Scheduler`, default disabled. The cache will still perform maintenance periodically and this scheduler only augments it to add additional liveliness. The scheduling is based on the expiration time only; users may use `Cache.cleanUp()` for fixed periods to assist reference caching, etc as desired. The cache will rate limit the scheduler so that it runs at a reasonable pace and never thrashes. For example, if the next expiration times are tiny durations apart (e.g. 200ms), the cache will not run the maintenance task 5x/s. It will instead ensure a 1 second delay (2^30ns) between scheduled runs (not accounting for the lack of real-time guarantees by a Scheduler impl). Similarly an additional delay may occur due to the priority queues being held in best-effort order. The scheduling for variable expiration is based on the TimerWheel's bucket expiration, which cascades events. For example a timer of 1h 13m 5s would schedule to run in 1h, which would cascade the 1hr bucket into the minute buckets. The next schedule would be at 13m, followed by 5s. Java 9+ users should prefer using `Scheduler.systemScheduler()` rather than creating their own dedicated threads. This leverages the JVM-wide scheduling thread, primarily intended for `CompletableFuture.orTimeout(duration)`. In Java 8 this method returns `Scheduler.disabledScheduler` instead.
Typically the cache using an amortized maintenance routine to periodically keep the eviction policies up to date. For size this is to replay hits or evict, for expiration this is to discover stale entries, and for reference caching this is to find GC'd keys or values. Each policy uses O(1) algorithms to be cheap, batches the work to avoid lock contention, and performs this routine maintenance as a side effect of normal cache operations. If there is no activity on the cache then the routine maintenance is not triggered, as that requires a background scheduling thread. Generally this is okay as the memory was already in consumed, no activity may indicate low system usage, and the cache is a transient data store. However, some do want to trigger business logic and leverage the cache as a timing subsystem, e.g. using the `RemovalListener` to notify another component. The cache itself does not create or manage threads, but can defer work to a configured `Executor`. Similarly now it can now schedule the maintenance routine on a configured `Scheduler`, default disabled. The cache will still perform maintenance periodically and this scheduler only augments it to add additional liveliness. The scheduling is based on the expiration time only; users may use `Cache.cleanUp()` for fixed periods to assist reference caching, etc as desired. The cache will rate limit the scheduler so that it runs at a reasonable pace and never thrashes. For example, if the next expiration times are tiny durations apart (e.g. 200ms), the cache will not run the maintenance task 5x/s. It will instead ensure a 1 second delay (2^30ns) between scheduled runs (not accounting for the lack of real-time guarantees by a Scheduler impl). Similarly an additional delay may occur due to the priority queues being held in best-effort order. The scheduling for variable expiration is based on the TimerWheel's bucket expiration, which cascades events. For example a timer of 1h 13m 5s would schedule to run in 1h, which would cascade the 1hr bucket into the minute buckets. The next schedule would be at 13m, followed by 5s. Java 9+ users should prefer using `Scheduler.systemScheduler()` rather than creating their own dedicated threads. This leverages the JVM-wide scheduling thread, primarily intended for `CompletableFuture.orTimeout(duration)`. In Java 8 this method returns `Scheduler.disabledScheduler` instead.
Typically the cache using an amortized maintenance routine to periodically keep the eviction policies up to date. For size this is to replay hits or evict, for expiration this is to discover stale entries, and for reference caching this is to find GC'd keys or values. Each policy uses O(1) algorithms to be cheap, batches the work to avoid lock contention, and performs this routine maintenance as a side effect of normal cache operations. If there is no activity on the cache then the routine maintenance is not triggered, as that requires a background scheduling thread. Generally this is okay as the memory was already in consumed, no activity may indicate low system usage, and the cache is a transient data store. However, some do want to trigger business logic and leverage the cache as a timing subsystem, e.g. using the `RemovalListener` to notify another component. The cache itself does not create or manage threads, but can defer work to a configured `Executor`. Similarly now it can now schedule the maintenance routine on a configured `Scheduler`, default disabled. The cache will still perform maintenance periodically and this scheduler only augments it to add additional liveliness. The scheduling is based on the expiration time only; users may use `Cache.cleanUp()` for fixed periods to assist reference caching, etc as desired. The cache will rate limit the scheduler so that it runs at a reasonable pace and never thrashes. For example, if the next expiration times are tiny durations apart (e.g. 200ms), the cache will not run the maintenance task 5x/s. It will instead ensure a 1 second delay (2^30ns) between scheduled runs (not accounting for the lack of real-time guarantees by a Scheduler impl). Similarly an additional delay may occur due to the priority queues being held in best-effort order. The scheduling for variable expiration is based on the TimerWheel's bucket expiration, which cascades events. For example a timer of 1h 13m 5s would schedule to run in 1h, which would cascade the 1hr bucket into the minute buckets. The next schedule would be at 13m, followed by 5s. Java 9+ users should prefer using `Scheduler.systemScheduler()` rather than creating their own dedicated threads. This leverages the JVM-wide scheduling thread, primarily intended for `CompletableFuture.orTimeout(duration)`. In Java 8 this method returns `Scheduler.disabledScheduler` instead.
Typically the cache using an amortized maintenance routine to periodically keep the eviction policies up to date. For size this is to replay hits or evict, for expiration this is to discover stale entries, and for reference caching this is to find GC'd keys or values. Each policy uses O(1) algorithms to be cheap, batches the work to avoid lock contention, and performs this routine maintenance as a side effect of normal cache operations. If there is no activity on the cache then the routine maintenance is not triggered, as that requires a background scheduling thread. Generally this is okay as the memory was already in consumed, no activity may indicate low system usage, and the cache is a transient data store. However, some do want to trigger business logic and leverage the cache as a timing subsystem, e.g. using the `RemovalListener` to notify another component. The cache itself does not create or manage threads, but can defer work to a configured `Executor`. Similarly now it can now schedule the maintenance routine on a configured `Scheduler`, default disabled. The cache will still perform maintenance periodically and this scheduler only augments it to add additional liveliness. The scheduling is based on the expiration time only; users may use `Cache.cleanUp()` for fixed periods to assist reference caching, etc as desired. The cache will rate limit the scheduler so that it runs at a reasonable pace and never thrashes. For example, if the next expiration times are tiny durations apart (e.g. 200ms), the cache will not run the maintenance task 5x/s. It will instead ensure a 1 second delay (2^30ns) between scheduled runs (not accounting for the lack of real-time guarantees by a Scheduler impl). Similarly an additional delay may occur due to the priority queues being held in best-effort order. The scheduling for variable expiration is based on the TimerWheel's bucket expiration, which cascades events. For example a timer of 1h 13m 5s would schedule to run in 1h, which would cascade the 1hr bucket into the minute buckets. The next schedule would be at 13m, followed by 5s. Java 9+ users should prefer using `Scheduler.systemScheduler()` rather than creating their own dedicated threads. This leverages the JVM-wide scheduling thread, primarily intended for `CompletableFuture.orTimeout(duration)`. In Java 8 this method returns `Scheduler.disabledScheduler` instead.
|
Released in v2.8 |
|
This is super handy! Probably not the right place for this, but is there a cleaner, one-liner-type solution for just triggering a cleanup at builder-construction-time for the scheduler? |
|
I'm not entirely sure what your intent is with that scheduler logic. The general usage would be to define it like, newOrderCache = Caffeine.newBuilder()
.scheduler(Scheduler.systemScheduler())
...
.build();and the cache will schedule based on the next expiration event. The CompletableFuture.runAsync(task,
CompletableFuture.delayedExecutor(delay, unit, executor));What are you trying to achieve with that custom scheduler statement? |
|
Perhaps I was overthinking it; I was merely wanting to ensure that expiry happened after the given interval, rather than wait for any operations that trigger its own cleanup. It's admittedly slightly against the purpose of a cache but in this case, I'm wanting things added to the cache to be evicted 30s later guaranteed. My cache in this instance isn't high throughput so something may go in 6 hours later, and I don't want to wait 6 hours for eviction. That may've been overly verbose for my use case, so apologies if so heh. If merely providing the systemScheduler actually triggers forced eviction after the given expiry interval then that's all I need. |
|
yeah, you probably over thought it :) If you provide a scheduler, without any fanciness, then the cache will schedule accordingly. If you are on Java 9+ then the There is some minor fuzziness so it is not strictly guaranteed to happen instantly, but within a small margin window. That should be good enough for most cases. You can try the feature in a test, e.g. public static void main(String[] args) {
Stopwatch stopwatch = Stopwatch.createUnstarted();
Cache<Integer, Integer> cache = Caffeine.newBuilder()
.expireAfterWrite(1, TimeUnit.SECONDS)
.scheduler(Scheduler.systemScheduler())
.removalListener((key, value, cause) -> {
System.out.printf("%s: %s at time %s%n", cause, key, stopwatch);
}).build();
stopwatch.start();
for (int i = 0; i < 3; i++) {
cache.put(i, i);
}
System.out.printf("Sleeping... %d entries%n", cache.estimatedSize());
Uninterruptibles.sleepUninterruptibly(2, TimeUnit.SECONDS);
System.out.println("Done... " + cache.asMap());
}Sleeping... 3 entries
EXPIRED: 0 at time 1.099 s
EXPIRED: 2 at time 1.119 s
EXPIRED: 1 at time 1.119 s
Done... {} |
Hello,
I wanted to leverage caffeine in a scenario as follows:
Component Aputs something on arequestqueue and stores the request in a (Caffeine) cache as well. WhenComponent Bis answering by means of putting something on aresponsequeue the original request is retrieved from the cache and processed further.However I also have the idea of a timeout on the amount of time waiting for a response. I've tried to implement that as follows:
Creating a cache in
Component Aas follows:I started out by implementing a
RemovalListener, but soon noticed that this was an asynchronous process but I also read that a synchronous 'listener' was possible using aCacheWriter.However this still doesn't seem to working as I expect. The behaviour that I see now is as follows:
CacheWriter.delete()of request 1 is triggered.Now I start out with a rather slow moving cache, so I can't rely on the fact that the cache is always busy enough to trigger something on the 'previous' request timeout.
Is this expected behaviour? It seems kind of odd to me, but I don't see anything wrong with what I'm doing.
Regards,
Auke
The text was updated successfully, but these errors were encountered: