Cache Support at AMS 0.10.0 #693

joaomdmoura · 2014-10-23T17:33:34Z

This is a implementation of cache support to the new version of AMS.

It started originally at the mailing list:
AMS: Caching with Matthijs Langenberg.
New Cache Support on AMS. with me

This implementation proposes x different approaches of the old versions of AMS.

A different syntax. (no longer need the cache_key method)
An options argument that have the same arguments of ActiveSupport::Cache::Store, plus a key option that will be the prefix of the object cache on a pattern "#{key}-#{object.id}".
It cache the objects individually and not the whole Serializer return, re-using it in different requests (as a show and a index method for example.)

There are many benefits on this approach:

You can easily know which will be your final cache key, so you can expire it with any custom code.
You can use all options of ActiveSupport::Cache::Store (:expires_in for example)
All objects will be cached individually and re-used on any other return needed
It keeps you Serializer clean by not depending on old cache_key method.

hhff · 2014-10-23T17:45:35Z

👍 👍 👍

zlx · 2014-10-24T01:14:21Z

👍

zlx · 2014-10-24T02:31:05Z

It's awesome to bring cache back, Thank you.

I read your idea in New Cache Support on AMS, and found it resolve the problem to reuse cache between index and show actions.

I have use AMS on some projects and find two truth:

we always have AMS object with many associated objects
the AMS entire cache is often expired since associated object expired

In fact, I have try to design a new cache strategy, and benchmark result always >50x faster than no cache.

here is my demo app for benchmark and here is my Cache Strategy base AMS 0.9

I hope it's useful for you to design the new cache strategy for AMS 0.10

joaomdmoura · 2014-10-25T04:53:25Z

Hey @zlx, I have just checked your cache suggestion at 0.9.0. Btw, thank you for contributing!
I was also missing cache, it couldn't be outside 0.10.0.

Yes, this implementation focus on make the cached objects as reusable as possible, it cache the serializer output, each major object individually, and does not expire it unless you tell it to. (Using expires_in option or with any custom code you want). It seem to fit to the major needs and is pretty straightforward.

I'm always hungry for new ideas and suggestions so let's keep improving it even after merged 😄

zlx · 2014-10-30T07:31:07Z

Hi, @joaomdmoura I find in real world, sometimes in one AMS object, we want to not cache some attributes, eg: is_follow like twitter, so we can make cache live more long time.

consider add this support? thus the we can just use AMS for cache API.

joaomdmoura · 2014-10-30T13:24:40Z

Hi @zlx yeah, totally agree. I'll work on it :)
Thank you for the suggestion 👍

zlx · 2014-10-30T13:57:12Z

Awesome. look forward your great job

kurko · 2014-10-31T02:55:45Z

README.md

+on a pattern ```"#{key}-#{object.id}"```.
+
+**[NOTE] Every object is individually cached.**
+**[NOTE] The cache isn't automaticly expired after update an object.**


Typo, automaticly.

kurko · 2014-11-12T22:15:29Z

@joaomdmoura can you rebase?

Summoning @steveklabnik. Is this something you want for AMS right now?

steveklabnik · 2014-11-12T22:18:29Z

In general, caching is something we want in AMS, yes 😄

joaomdmoura · 2014-11-12T22:19:31Z

@kurko Of course, I might add some more commit too. I'll do this tonight 😄

BenMorganIO · 2015-01-15T18:28:54Z

Hey, has this been actively worked on lately? If caching is required for some apps, would 0.8 be a better fit?

kurko · 2015-01-21T21:46:15Z

README.md

+
+```ruby
+class PostSerializer < ActiveModel::Serializer
+  cache key: 'post', expires_in: 3.hours


@joaomdmoura could we make key optional? Then we could use object.cache_key by default. Makes sense?

@kurko I'm not sure if I have already done it 😕 But I'll check it.

kurko · 2015-01-21T22:01:38Z

Could you rebase this? We want to move forward with caching 😁

joaomdmoura · 2015-01-21T23:56:01Z

haha Actually I already started it 3 times 😞 but the tests always end up broken.
I'll try to really do this asap. 😄

kurko · 2015-01-22T00:05:01Z

I'm so sorry :( Getting this in is my number 1 priority now, I promise.

Could you also squash your commits into one with a nice title and description?

joaomdmoura · 2015-01-22T19:30:44Z

@kurko It's done! Rebased, working and squashed. 😄

kurko · 2015-01-23T16:33:00Z

lib/active_model/serializer/adapter.rb

+        klass = serializer.class
+        if klass._cache
+          _cache_key = klass._cache_key || serializer.object.class.name.downcase
+          klass._cache.fetch("#{_cache_key}-#{serializer.object.id}", klass._cache_options) do


I think I wasn't clear in my previous comment (https://github.com/rails-api/active_model_serializers/pull/693/files#diff-04c6e90faac2675aa89e2176d2eec7d8R252). Basically, I believe we should use model.cache_key here when key is not used (docs in http://api.rubyonrails.org/classes/ActiveRecord/Integration.html). If you don't specify a key, it'd use object.cache_key:

Person.find(5).cache_key # => "people/5-20071224150000"

The problem with using only an id here is that if we update the model, it'll reuse an old cache because update_at is disregarded. With cache_key, it'll regard update_at. With a custom key, I believe updated_at should also be included in this identifier.

What do you think?

joaomdmoura · 2015-01-25T20:02:54Z

@kurko totally agree. I'll work on this.

joaomdmoura · 2015-01-26T06:47:00Z

@kurko there is a down side on adding the updated_at to the key that I would like to discuss.
One of the reason I didn't added this before was:

On the later versions of cache on AMS there wasn't a way to expire an specific cache on an abstracted logic, because there was no way to know the cache key.
I added the key option and defined this simple "key-id" pattern in order to make it simple to expire this cache on any part of your application, like a background job for example.

Do you think there is no need to worry about it? Or do you see some workaround?

kurko · 2015-01-28T01:36:52Z

@joaomdmoura

when using updated_at, changed objects would automatically make caches obsolete. I believe this is what people would expect to be the convention.
expiring caches wouldn't be that hard. I'm pretty sure you could find objects with a Glob and wildcard.
people can still disregard this and set their own keys. Perhaps another option would be sense if you feel uncomfortable, e.g cache key: "post", identifier: :id vs cache key: "post", identifier: [:id, :updated_at]

joaomdmoura · 2015-01-28T03:19:23Z

@kurko yeah, u're probably right.
I may not be taking into account the most expected behaviour.
I'll move forward on this feature so that we can finally merge it.

nwjsmith · 2015-01-31T15:38:46Z

lib/active_model/serializer/adapter.rb

@@ -1,3 +1,4 @@
+require 'pry'


A wild pry appears

Observe, pry in its natural habitat.

hahahahahah sorry people, already removed it. 😄

nwjsmith · 2015-01-31T15:49:35Z

Is there a way for this cache-ing scheme to support something like Fragment/Russian Doll Caching? If not, this leaves AMS at a serious disadvantage to JBuilder for higher-performance apps.

One of the things that is nice about the old def cache_key-based caching scheme was that the key could be generated dynamically. You can, for example, generate a key based on the cache key for associated objects, or the current Git commit. Will there still be hooks in this new caching scheme to allow for that?

sandstrom · 2015-02-04T12:44:58Z

Awesome! Smooth ⛵

BenMorganIO · 2015-02-04T19:31:37Z

YAY

kenips · 2015-02-06T01:28:12Z

❤️ ❤️ ❤️

@kurko can we get a 0.10.0.alpha released soon?

kurko · 2015-02-06T01:30:10Z

@kenips we need to get fragment cache first :) Check #802.

joaomdmoura · 2015-02-06T11:52:08Z

@kenips It's already WIP. I'll try to get it done next week.
The first version is already working but there is some optimisations that I need to work on, and also think how I will deal with relationships.

It's an upgrade based on the new Cache implementation rails-api#693. It allows to use the Rails conventions to cache specific attributes or associations. It's based on the Cache Composition implementation.

ref: - rails-api#827 - rails-api#832 - rails-api#872 - rails-api#1045 - rails-api#912 - rails-api#911 - rails-api#904 - rails-api#810 - rails-api#693 - rails-api#849 - rails-api#1029 - rails-api#1300 - rails-api#1372

kurko reviewed Oct 31, 2014
View reviewed changes

joaomdmoura mentioned this pull request Nov 23, 2014

Expiring cache key with 0.8.0 #739

Closed

kurko added the V: 0.10.x label Jan 6, 2015

athal7 mentioned this pull request Jan 15, 2015

Switch from Rabl to Active Model Serializers spree/spree#4375

Closed

kurko reviewed Jan 21, 2015
View reviewed changes

joaomdmoura force-pushed the cache-support branch from be1516b to ed8353d Compare January 22, 2015 19:12

kurko reviewed Jan 23, 2015
View reviewed changes

kurko mentioned this pull request Jan 26, 2015

Is caching still removed? #787

Closed

nwjsmith reviewed Jan 31, 2015
View reviewed changes

joaomdmoura mentioned this pull request Feb 12, 2015

Adding Fragment Cache to AMS #810

Merged

joaomdmoura mentioned this pull request Mar 6, 2015

Adding fetch_multi to Cache Strategy #827

Open

joaomdmoura mentioned this pull request Mar 17, 2015

JSON adapter does not handle nested associations #835

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cache Support at AMS 0.10.0 #693

Cache Support at AMS 0.10.0 #693

joaomdmoura commented Oct 23, 2014

hhff commented Oct 23, 2014

zlx commented Oct 24, 2014

zlx commented Oct 24, 2014

joaomdmoura commented Oct 25, 2014

zlx commented Oct 30, 2014

joaomdmoura commented Oct 30, 2014

zlx commented Oct 30, 2014

kurko Oct 31, 2014

kurko commented Nov 12, 2014

steveklabnik commented Nov 12, 2014

joaomdmoura commented Nov 12, 2014

BenMorganIO commented Jan 15, 2015

kurko Jan 21, 2015

joaomdmoura Jan 21, 2015

kurko commented Jan 21, 2015

joaomdmoura commented Jan 21, 2015

kurko commented Jan 22, 2015

joaomdmoura commented Jan 22, 2015

kurko Jan 23, 2015

BenMorganIO Jan 23, 2015

joaomdmoura commented Jan 25, 2015

joaomdmoura commented Jan 26, 2015

kurko commented Jan 28, 2015

joaomdmoura commented Jan 28, 2015

nwjsmith Jan 31, 2015

BenMorganIO Feb 1, 2015

joaomdmoura Feb 2, 2015

nwjsmith commented Jan 31, 2015

sandstrom commented Feb 4, 2015

BenMorganIO commented Feb 4, 2015

kenips commented Feb 6, 2015

kurko commented Feb 6, 2015

joaomdmoura commented Feb 6, 2015

Cache Support at AMS 0.10.0 #693

Cache Support at AMS 0.10.0 #693

Conversation

joaomdmoura commented Oct 23, 2014

hhff commented Oct 23, 2014

zlx commented Oct 24, 2014

zlx commented Oct 24, 2014

joaomdmoura commented Oct 25, 2014

zlx commented Oct 30, 2014

joaomdmoura commented Oct 30, 2014

zlx commented Oct 30, 2014

Choose a reason for hiding this comment

kurko commented Nov 12, 2014

steveklabnik commented Nov 12, 2014

joaomdmoura commented Nov 12, 2014

BenMorganIO commented Jan 15, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kurko commented Jan 21, 2015

joaomdmoura commented Jan 21, 2015

kurko commented Jan 22, 2015

joaomdmoura commented Jan 22, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joaomdmoura commented Jan 25, 2015

joaomdmoura commented Jan 26, 2015

kurko commented Jan 28, 2015

joaomdmoura commented Jan 28, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nwjsmith commented Jan 31, 2015

sandstrom commented Feb 4, 2015

BenMorganIO commented Feb 4, 2015

kenips commented Feb 6, 2015

kurko commented Feb 6, 2015

joaomdmoura commented Feb 6, 2015