Skip to content
This repository has been archived by the owner on Apr 17, 2021. It is now read-only.

Implement Eventbrite transformer #464

Merged
merged 1 commit into from
Oct 14, 2018

Conversation

alistairjcbrown
Copy link
Collaborator

What's Changed

Implements transformer for Eventbrite data, saving to the shared muxer-transformed-events bucket.
Examples of source data and transformed data: https://gist.github.com/alistairjcbrown/a93c15f65bc4b7fc52236c91506d1813

Brought up some questions to think about in the future:

  • The transformation structure is very similar. Once the meetupcom lambda is converted, it would be worth seeing if there's any transformation logic that can be pulled out.
  • Files are added to the transformed events bucket under a directory with time values for when they're created. There's nothing which ties this back to the original producer bucket files. Do we need some kind of "session" string to tie all of these together for easier debugging / auditing?

}
};

const transformEvent = function (defaults, event) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function transformEvent has 50 lines of code (exceeds 25 allowed). Consider refactoring.


const isValidEvent = (event) => validate(event, eventSchema).errors.length === 0;

const uploadData = function(bucketName, eventsPage) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar blocks of code found in 2 locations. Consider refactoring.

}));

const { eventsBucket } = buckets();
const filePaths = await Promise.all(await transformedFiles.map(async function (transformedFile) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar blocks of code found in 2 locations. Consider refactoring.

@codeclimate
Copy link

codeclimate bot commented Oct 14, 2018

Code Climate has analyzed commit 371cdd6 and detected 5 issues on this pull request.

Here's the issue category breakdown:

Category Count
Complexity 1
Duplication 4

View more on Code Climate.

@@ -110,7 +110,6 @@
"required": [
"name",
"address",
"city",
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer considering city a required field for a venue (what if it's not in a city?)

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

city as an optional field makes sense; we can always decorate this later from the address.

@@ -0,0 +1,260 @@
{
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carbon copy of the schema for the farset labs lambda - something to extract in the future.

Copy link
Owner

@apoclyps apoclyps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@alistairjcbrown alistairjcbrown merged commit 5df534c into master Oct 14, 2018
@alistairjcbrown alistairjcbrown deleted the implement-eventbrite-transformer branch October 14, 2018 17:18
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants