Upgrade to latest Scanamo version (1.0.0-M28) #1143
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rtyley
force-pushed
the
upgrade-scanamo
branch
3 times, most recently
from
f4fe706 to
ea57fa7
Compare
rtyley
force-pushed
the
upgrade-scanamo
branch
5 times, most recently
from
86731d1 to
2bdc9d6
Compare
rtyley
commented
Feb 2, 2024
Comment on lines
-16
to
+18
| implicit val dateTimeFormat = DynamoFormat.coercedXmap[DateTime, String, IllegalArgumentException]( | ||
| DateTime.parse(_).withZone(DateTimeZone.UTC) | ||
| )(_.toString) | ||
| implicit val dateTimeFormat: DynamoFormat[DateTime] = DynamoFormat.coercedXmap[DateTime, String, IllegalArgumentException]( | ||
| DateTime.parse(_).withZone(DateTimeZone.UTC), | ||
| _.toString | ||
| ) |
There was a problem hiding this comment.
This change required by scanamo/scanamo#588 - apparently this gave better type inference, but I'm not sure how exactly...!
rtyley
force-pushed
the
upgrade-scanamo
branch
2 times, most recently
from
a333e9c to
4b85eb7
Compare
Merged
rhystmills
approved these changes
Feb 7, 2024
Here we're moving up from Scanamo 1.0.0-M9 to 1.0.0-M28, which involves
several changes to requirements:
* AWS SDK v1 → v2 - Scanamo 1.0-M13 and above require AWS SDK v2
* Scanamo API changes:
- When we `exec()` a Scanamo call, we now access an _instance_ of
Scanamo that _wraps_ an AWS DynamoDB client, rather than supplying
an AWS DynamoDB client to the `Scanamo` singleton. This means most
of the `DataStore` classes in `media-atom-maker` now accept a
`Scanamo` instance rather than a `AmazonDynamoDBClient`.
- Auto-generated `DynamoFormat` handlers are now created by
`org.scanamo.generic.auto._` rather than `org.scanamo.auto._`
- Query on entity keys by name as a String (`"id"`), rather than
using the old 'Symbol' apostrophe syntax (`'id`), and use `===`
rather than `->` to specify the required key value.
The AWS SDK v1 → v2 requirement of Scanamo is the tricky one to handle-
a lot of code `media-atom-maker` also makes AWS SDK calls, and I didn't
want to have to fix all of it to use AWS SDK v2 in this change. Instead,
this change introduces `com.gu.media.aws.CredentialsForBothSdkVersions`,
which makes a choice of AWS SDK v1 or v2 credentials available in the
already-existing `com.gu.media.aws.AwsCredentials` class. This allows
a partial move to AWS SDK v2, while other parts of the code remain on
AWS SDK v1 for now. Notably, we need both v1 _and_ v2 versions of the
DynamoDB client, as the atom-maker libraries in https://github.com/guardian/atom-maker
still use AWS SDK v1.
rtyley
force-pushed
the
upgrade-scanamo
branch
from
4b85eb7 to
5c541a0
Compare
Merged
rtyley
added a commit
to guardian/riff-raff
that referenced
this pull request
Jun 5, 2024
Here we're moving up from Scanamo [1.0.0-M11](https://github.com/scanamo/scanamo/releases/tag/v1.0.0-M11) to [1.1.1](https://github.com/scanamo/scanamo/releases/tag/v1.1.1), which involves several changes to requirements: * AWS SDK v1 → v2 - Scanamo [1.0-M13](https://github.com/scanamo/scanamo/releases/tag/v1.0-M13) and above require AWS SDK v2. AWS SDK v2 is actually used quite widely already in Riff Raff, so this wasn't hard - there does still seem to be _some_ other code in Riff Raff using AWS SDK v1 at the moment. * Scanamo API changes: - Auto-generated `DynamoFormat` handlers are now created with `org.scanamo.generic.auto._` rather than `org.scanamo.auto._` (scanamo/scanamo#538 - [1.0-M12](https://github.com/scanamo/scanamo/releases/tag/v1.0.0-M12)) - `DynamoFormat.coercedXmap()` now has a single set of two arguments, rather than curried arguments (scanamo/scanamo#588 - [1.0-M13](https://github.com/scanamo/scanamo/releases/tag/v1.0-M13)) - `Table.put()` now returns `Unit` rather than the old value- you can use `Table.putAndReturn()` if you want a value returned, which allows you to specify whether it's the prior value or the new one you want (scanamo/scanamo#486 - [1.0-M13](https://github.com/scanamo/scanamo/releases/tag/v1.0-M13)) See also: * guardian/media-atom-maker#1143 - upgrading to Scanamo v1.0.0-M28
This was referenced Jun 5, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a necessary precursor to the Scala 2.13 upgrade in PR #1147, as Scanamo 1.0.0-M9 only supports Scala 2.11 & 2.12.
Here we're moving up from Scanamo 1.0.0-M9 to 1.0.0-M28, which involves several changes to requirements:
exec()a Scanamo call, we now access an instance of Scanamo that wraps an AWS DynamoDB client, rather than supplying an AWS DynamoDB client to theScanamosingleton. This means most of theDataStoreclasses inmedia-atom-makernow accept aScanamoinstance rather than aAmazonDynamoDBClient(Breaking: Refactor Scanamo clients as classes scanamo/scanamo#406 - 1.0.0-M10).DynamoFormathandlers are now created withorg.scanamo.generic.auto._rather thanorg.scanamo.auto._(Generic derivation with Magnolia scanamo/scanamo#538 - 1.0-M12)"id"), rather than using the old 'Symbol' apostrophe syntax ('id- see 1.0.0-M11), and use===rather than->to specify the required key value (1.0-M13).The AWS SDK v1 → v2 requirement of Scanamo was the tricky one to handle- a lot of code
media-atom-makeralso makes AWS SDK calls to lots of other services, and I didn't want to have to fix all of it to use AWS SDK v2 in this change.Instead, this change introduces
com.gu.media.aws.CredentialsForBothSdkVersions, which makes a choice of AWS SDK v1 or v2 credentials available in the already-existingcom.gu.media.aws.AwsCredentialsclass. This allows a partial move to AWS SDK v2, while other parts of the code remain on AWS SDK v1 for now. Notably, we need both v1 and v2 versions of the DynamoDB client, as the atom-maker libraries in guardian/atom-maker still use AWS SDK v1.Testing
This deploys successfully to CODE, and https://video.code.dev-gutools.co.uk/videos is still accessible:
I was able to make a small metadata change without any obvious problems:
I'd need to pair with someone on the Content Production team to double check that all functionality, including the AWS lambdas, are working correctly!