Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move impression counts to periodic background job #318

Closed
jace opened this issue Apr 4, 2016 · 0 comments
Closed

Move impression counts to periodic background job #318

jace opened this issue Apr 4, 2016 · 0 comments

Comments

@jace
Copy link
Member

jace commented Apr 4, 2016

Hasjob stores jobpost stickie impressions per session but displays statistics per unique user. This stat is cached in Redis but needs to be recalculated after every impression, ie after anyone visits the home page. This is obviously a huge drain on the database, and precise up-to-the-second counts are not necessary for anyone.

Hasjob should instead:

  1. When saving an impression, mark the jobpost's count as dirty and save this dirty flag to Redis. Do not recalculate.
  2. In JobPost.viewcounts, recalculate as usual only if the count is missing in Redis, and also remove from the dirty set.
  3. Add a new periodic job to manage.py (alongside sweep) that runs every five minutes, recalculates totals for all items in the dirty list, repopulates cache in redis, and removes each item from the dirty set.

In #309 we discuss the possibility of removing past JobImpression and EventSession data, requiring these totals to be stored somewhere more permanent than Redis. This exercise is left for a future issue.

@jace jace closed this as completed in 31f3b07 May 31, 2016
@jace jace changed the title Move impression counts to separate cache table Move impression counts to periodic background job Jun 1, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant