(PROPOSAL) Method to update cache even if it's not empty

[tonidas]

Hello guys, first of all I would like to thank you for the work on this gem.

I have a use case that might be different from the usual: I need to update the cache daily with the new json value from from the OER API, without clearing the cache at all.
In other words, I need to call the "read_from_url" method ad-hoc and ensure that the cache stays daily updated with data for my rails instances to consume it.

From what I read in this gem, thats not possible, because the only way to trigger another request to the OER API to retrieve new data is to put a TTL on the cache data to trigger this logic when the cache is emptied:

def exchange_rates
  doc = JSON.parse(read_from_cache || read_from_url)
  @oer_rates = doc['rates']
end

My proposal is to somewhat leave a public method that resync the data with the API. (something like "read_from_url")
Does that makes sense?

[ilyacoding]

If you are using rails, cache engine will invalidate cache and turn it to nil.
Rails.cache.write(OXR_CACHE_KEY, text, expires_in: OXR_CACHE_TTL)

[tonidas]

Hi @ilyacoding.

The problem is that I can't afford the cache being nil, it must always have a value, or the clients will receive an error.

[spk]

Hi @tonidas, thanks for the detailed issue I see what you mean and will try to make a fix soon

[tonidas]

Awesome @spk, thanks :)

[spk]

Released in https://github.com/spk/money-open-exchange-rates/releases/tag/v1.1.0 @tonidas let me know if this work for you

[tonidas]

Hi @spk, I'm a little confused..
It seems that you've changed the in-memory cache to always request the OER API when the "ttl_in_seconds" is reached. What's the point of the external cache now?

Maybe I didn't got your change, can you explain it?

[ilyacoding]

Hi @tonidas,
As i understood, you want to never turn value in cache to nil and always have updated rates directly in cache?

[tonidas]

@ilyacoding Thats true, but I want that to be done daily, on a separate worker, and on my schedule. On this PR the external cache is just ignored, the values are updated on the memory itself. I think that the whole architecture was changed on that. It's not retro-compatible at all.

[spk]

Hi @tonidas, yes that was the point of #42 released in https://github.com/spk/money-open-exchange-rates/releases/tag/v1.0.0 (so not retro compatible)
On the issue #38 its expressed that people don't want to use separate worker/schedule anymore and invalidate fetch on get rates; so its difficult to make everyone happy
I'm going to keep the gem like this sorry; only thing I see is to add a public method like you said that resync the data with the API; is that ok for you ?

[spk]

I think I get the problem, if you use another store like database instead of default memory store this decrease the performance when ttl is expire so #42 and #38 was not a good idea ...

[spk]

Hi @tonidas everything should be fixed in v1.2.0 release; sorry for the confusion

[spk]

Oops wrong button I let you close this

[spk]

No feedback assuming its fixed feel free to open new issue

[tonidas]

Hi @spk I'm really sorry for the long delay to answer you. That should not happen again!

Let's recap a little bit, this is the architecture that I thought on the very first day that I saw this gem:

We have 2 layers of cache: The first one is in-memory, and the other one is the one configured on the "cache" configuration of this gem.

Every time the gem is called to return the exchange rate for a pair of currencies, it will check the exchange rate map stored on the first cache (in-memory). If the gem is called after X seconds of the last call (this X value is configured on ttl_in_seconds parameter), it will check the 2nd cache layer (the one set on the "cache" configuration) and update the first layer cache (in-memory).

Am I right about the gem flow in the first place?

I've created this issue because I found that the gem was updating the 2nd cache layer every time that the ttl_in_seconds was being reached, and I thought that this approach was not a fit for my application. So I suggested that the gem should have an option to update the 2nd layer cache on special moments, by different workers, on-clock.

Resuming my usecase: The application should never update the exchange rates on user requests, because that would generate a slow experience on those moments.

I've just downloaded the 1.2.0 gem and I've checked that the 2nd layer cache is being called on every gem call to retrieve an exchange rate, removing the 1st layer cache usage. Is that intended?