Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add scheduled rule to kick off the refresh weekly #3

Merged
merged 1 commit into from
May 25, 2021

Conversation

rtyley
Copy link
Member

@rtyley rtyley commented May 21, 2021

MaxMind say "Weekly updates are available every Tuesday" - so I guess we should grab a fresh copy of the GeoIP database once a week, on Wednesday!

This change adds a cloud-formed AWS Event Rule to trigger the lambda on a weekly basis.

ScheduledRule:
Type: AWS::Events::Rule
Properties:
ScheduleExpression: cron(20 11 ? * WED *) # MaxMind: "Weekly updates are available every Tuesday"
Copy link
Member Author

@rtyley rtyley May 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Truth be told, there is quite a lot of variation in implementations of cron expressions in general!

AWS have slightly unusual cron expressions, which we see here in that they have a 6th field which is the year of execution, and they can use ? in day-of-month & day-of-week to indicate you don't care what value that should be (rather than *, which means every value it could be)

If you get rid of those two idiosyncrasies, you can check this cron with https://crontab.guru/#20_11_*_*_WED :

image

So, the expression will execute at 11:20am on Wednesdays, the day after MaxMind release the weekly database update.

@rtyley
Copy link
Member Author

rtyley commented May 25, 2021

@lmath asks this good question: "How would you get the slab to know about updated geoIP data in the s3 bucket? Redeploy?" - that's right, a redeploy is necessary as the Slab only reads GeoIP data on startup.

We have a RiffRaff scheduled deploy for the Slab, and I've just updated it to occur 20 minutes after the ophan-geoip-db-refresher runs:

image

Note that the Schedule timezone in RiffRaff is UTC - this matches Amazon CloudWatch Schedule Expressions, which says "All scheduled events use UTC time zone" - so it's okay to schedule the lambda for 11.20am with this PR, and the redeploy of the Slab at 11.40am, as we know that they're both in the same sensible time zone of UTC.

@rtyley
Copy link
Member Author

rtyley commented May 26, 2021

The lambda executed successfully at the anticipated time of 11:20am UTC (12:20pm london time):

image
image
image

The scheduled deploy should occur in a minute or so...

@rtyley
Copy link
Member Author

rtyley commented May 26, 2021

Scheduled deploy of the Slab has completed, finishing at 12:42pm:

image

image

image

The country breakdown graph still looks good!
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants