-
Notifications
You must be signed in to change notification settings - Fork 53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deployed feed version summaries #516
Merged
Merged
Changes from 7 commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
25e9704
refactor(Create end point to get deployed feed version): Provide the …
eaff169
refactor(Added new feed source end point to get deployed feed version):
77e0463
Merge branch 'dev' into deployed-feed-version-summaries
5e822fb
refactor(Update to include latest deployment feed version. Matching t…
9ea575b
refactor(FeedSource.java): Added missing comment
f4479ad
refactor(FeedSource.java): deployed feed version is now mutually excl…
63497d5
Merge branch 'dev' into deployed-feed-version-summaries
b55c40b
refactor(FeedSource.java): Updated comment and refactored if/else sta…
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
239 changes: 239 additions & 0 deletions
239
src/main/java/com/conveyal/datatools/manager/models/FeedVersionDeployed.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,239 @@ | ||
| package com.conveyal.datatools.manager.models; | ||
|
|
||
| import com.conveyal.datatools.editor.utils.JacksonSerializers; | ||
| import com.conveyal.datatools.manager.persistence.Persistence; | ||
| import com.fasterxml.jackson.databind.annotation.JsonDeserialize; | ||
| import com.fasterxml.jackson.databind.annotation.JsonSerialize; | ||
| import com.google.common.collect.Lists; | ||
| import com.mongodb.client.model.Sorts; | ||
| import org.bson.Document; | ||
| import org.bson.conversions.Bson; | ||
|
|
||
| import java.time.LocalDate; | ||
| import java.time.format.DateTimeFormatter; | ||
| import java.util.List; | ||
|
|
||
| import static com.mongodb.client.model.Aggregates.limit; | ||
| import static com.mongodb.client.model.Aggregates.lookup; | ||
| import static com.mongodb.client.model.Aggregates.match; | ||
| import static com.mongodb.client.model.Aggregates.replaceRoot; | ||
| import static com.mongodb.client.model.Aggregates.sort; | ||
| import static com.mongodb.client.model.Aggregates.unwind; | ||
| import static com.mongodb.client.model.Filters.in; | ||
|
|
||
| public class FeedVersionDeployed { | ||
| public String id; | ||
|
|
||
| @JsonSerialize(using = JacksonSerializers.LocalDateIsoSerializer.class) | ||
| @JsonDeserialize(using = JacksonSerializers.LocalDateIsoDeserializer.class) | ||
| public LocalDate startDate; | ||
|
|
||
| @JsonSerialize(using = JacksonSerializers.LocalDateIsoSerializer.class) | ||
| @JsonDeserialize(using = JacksonSerializers.LocalDateIsoDeserializer.class) | ||
| public LocalDate endDate; | ||
|
|
||
| public FeedVersionDeployed() { | ||
| } | ||
|
|
||
| public FeedVersionDeployed(Document feedVersionDocument) { | ||
| this.id = feedVersionDocument.getString("_id"); | ||
| DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyyMMdd"); | ||
| Document validationResult = (Document) feedVersionDocument.get("validationResult"); | ||
| if (validationResult != null) { | ||
| String first = validationResult.getString("firstCalendarDate"); | ||
| String last = validationResult.getString("lastCalendarDate"); | ||
| this.startDate = (first == null) ? null : LocalDate.parse(first, formatter); | ||
| this.endDate = (last == null) ? null : LocalDate.parse(last, formatter); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Get the deployed feed version from the pinned deployment for this feed source. | ||
| */ | ||
| public static FeedVersionDeployed getFeedVersionFromPinnedDeployment(String projectId, String feedSourceId) { | ||
| /* | ||
| Note: To test this script: | ||
| 1) Comment out the call to tearDownDeployedFeedVersion() in FeedSourceControllerTest -> tearDown(). | ||
| 2) Run FeedSourceControllerTest to created required objects referenced here. | ||
| 3) Once complete, delete documents via MongoDB. | ||
| 4) Uncomment the call to tearDownDeployedFeedVersion() in FeedSourceControllerTest -> tearDown(). | ||
| 5) Re-run FeedSourceControllerTest to confirm deletion of objects. | ||
|
|
||
| db.getCollection('Project').aggregate([ | ||
| { | ||
| $match: { | ||
| _id: "project-with-pinned-deployment" | ||
| } | ||
| }, | ||
| { | ||
| $lookup:{ | ||
| from:"Deployment", | ||
| localField:"pinnedDeploymentId", | ||
| foreignField:"_id", | ||
| as:"deployment" | ||
| } | ||
| }, | ||
| { | ||
| $lookup:{ | ||
| from:"FeedVersion", | ||
| localField:"deployment.feedVersionIds", | ||
| foreignField:"_id", | ||
| as:"feedVersions" | ||
| } | ||
| }, | ||
| { | ||
| $unwind: "$feedVersions" | ||
| }, | ||
| { | ||
| "$replaceRoot": { | ||
| "newRoot": "$feedVersions" | ||
| } | ||
| }, | ||
| { | ||
| $match: { | ||
| feedSourceId: "feed-source-with-pinned-deployment-feed-version" | ||
| } | ||
| }, | ||
| { | ||
| $sort: { | ||
| lastUpdated : -1 | ||
| } | ||
| }, | ||
| { | ||
| $limit: 1 | ||
| } | ||
| ]) | ||
| */ | ||
| List<Bson> stages = Lists.newArrayList( | ||
| match( | ||
| in("_id", projectId) | ||
| ), | ||
| lookup("Deployment", "pinnedDeploymentId", "_id", "deployment"), | ||
miles-grant-ibigroup marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| lookup("FeedVersion", "deployment.feedVersionIds", "_id", "feedVersions"), | ||
| unwind("$feedVersions"), | ||
| replaceRoot("$feedVersions"), | ||
| match( | ||
| in("feedSourceId", feedSourceId) | ||
| ), | ||
| // If more than one feed version for a feed source is held against a deployment the latest is used. | ||
| sort(Sorts.descending("lastUpdated")), | ||
| limit(1) | ||
| ); | ||
| return getFeedVersionDeployed(stages); | ||
| } | ||
|
|
||
| /** | ||
| * Get the deployed feed version from the latest deployment for this feed source. | ||
| */ | ||
| public static FeedVersionDeployed getFeedVersionFromLatestDeployment(String projectId, String feedSourceId) { | ||
| /* | ||
| Note: To test this script: | ||
| 1) Comment out the call to tearDownDeployedFeedVersion() in FeedSourceControllerTest -> tearDown(). | ||
| 2) Run FeedSourceControllerTest to created required objects referenced here. | ||
| 3) Once complete, delete documents via MongoDB. | ||
| 4) Uncomment the call to tearDownDeployedFeedVersion() in FeedSourceControllerTest -> tearDown(). | ||
| 5) Re-run FeedSourceControllerTest to confirm deletion of objects. | ||
|
|
||
| db.getCollection('Project').aggregate([ | ||
| { | ||
| // Match provided project id. | ||
| $match: { | ||
| _id: "project-with-latest-deployment" | ||
| } | ||
| }, | ||
| { | ||
| // Get all deployments for this project. | ||
| $lookup:{ | ||
| from:"Deployment", | ||
| localField:"_id", | ||
| foreignField:"projectId", | ||
| as:"deployments" | ||
| } | ||
| }, | ||
| { | ||
| // Deconstruct deployments array to a document for each element. | ||
| $unwind: "$deployments" | ||
| }, | ||
| { | ||
| // Make the deployment documents the input/root document. | ||
| "$replaceRoot": { | ||
| "newRoot": "$deployments" | ||
| } | ||
| }, | ||
| { | ||
| // Sort descending. | ||
| $sort: { | ||
| lastUpdated : -1 | ||
| } | ||
| }, | ||
| { | ||
| // At this point we will have the latest deployment for a project. | ||
| $limit: 1 | ||
| }, | ||
| { | ||
| // Get all feed versions that have been deployed as part of the latest deployment. | ||
| $lookup:{ | ||
| from:"FeedVersion", | ||
| localField:"feedVersionIds", | ||
| foreignField:"_id", | ||
| as:"feedVersions" | ||
| } | ||
| }, | ||
| { | ||
| // Deconstruct feedVersions array to a document for each element. | ||
| $unwind: "$feedVersions" | ||
| }, | ||
| { | ||
| // Make the feed version documents the input/root document. | ||
| "$replaceRoot": { | ||
| "newRoot": "$feedVersions" | ||
| } | ||
| }, | ||
| { | ||
| // Match the required feed source. | ||
| $match: { | ||
| feedSourceId: "feed-source-with-latest-deployment-feed-version" | ||
| } | ||
| }, | ||
| { | ||
| $sort: { | ||
| lastUpdated : -1 | ||
| } | ||
| }, | ||
| { | ||
| // At this point we will have the latest feed version from the latest deployment for a feed source. | ||
| $limit: 1 | ||
| } | ||
| ]) | ||
| */ | ||
| List<Bson> stages = Lists.newArrayList( | ||
| match( | ||
| in("_id", projectId) | ||
| ), | ||
| lookup("Deployment", "_id", "projectId", "deployments"), | ||
| unwind("$deployments"), | ||
| replaceRoot("$deployments"), | ||
| sort(Sorts.descending("lastUpdated")), | ||
| limit(1), | ||
| lookup("FeedVersion", "feedVersionIds", "_id", "feedVersions"), | ||
| unwind("$feedVersions"), | ||
| replaceRoot("$feedVersions"), | ||
| match( | ||
| in("feedSourceId", feedSourceId) | ||
| ), | ||
| // If more than one feed version for a feed source is held against a deployment the latest is used. | ||
| sort(Sorts.descending("lastUpdated")), | ||
| limit(1) | ||
| ); | ||
| return getFeedVersionDeployed(stages); | ||
| } | ||
|
|
||
| private static FeedVersionDeployed getFeedVersionDeployed(List<Bson> stages) { | ||
| Document feedVersionDocument = Persistence | ||
| .getMongoDatabase() | ||
| .getCollection("Project") | ||
| .aggregate(stages) | ||
| .first(); | ||
| return (feedVersionDocument == null) ? null : new FeedVersionDeployed(feedVersionDocument); | ||
| } | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we do this instead of a null check on deployedFeedVersion?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is to prevent calls to
retrieveDeployedFeedVersion()where there are no deployed feed versions. Using null resulted in the following scenerio: Call togetDeployedFeedVersionId()returned null, followed by a call togetDeployedFeedVersionStartDate()returning null and a call togetDeployedFeedVersionEndDate()also returning null.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this may just be issues with my java understanding how is this field better than those 3 methods returning null? Is it to avoid mongo queries?