Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(warehouse): syncs issues #2732

Merged
merged 7 commits into from
Nov 28, 2022
Merged

fix(warehouse): syncs issues #2732

merged 7 commits into from
Nov 28, 2022

Conversation

achettyiitr
Copy link
Member

@achettyiitr achettyiitr commented Nov 24, 2022

Description

  • Handle no syncs case when the warehouse tries to merge existing syncs with the new ones thereby users do not get the existing syncs and we are sending a 204 response code to handle it.
  • Added logs for warehouse Grpc for debugging purpose.

Notion Ticket

https://www.notion.so/rudderstacks/Syncs-dashboard-fixes-4d3c454856d844e09cc15a73012a37e1

Security

  • The code changed/added as part of this pull request won't create any security issues with how the software is being used.

warehouse/api.go Outdated Show resolved Hide resolved
@@ -73,6 +73,7 @@ message WHUploadResponse {
int32 duration = 14;
repeated WHTable tables = 15;
bool isArchivedUpload = 16;
int32 status_code = 17;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a bit concern with adding HTTP semantics into GRPC, is it because we are using an HTTP API at some part of the chain ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I followed the GRPC documentation, Currently, there is no way to send StatusNoContent (204).
grpc-ecosystem/grpc-gateway#240

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

List of HTTP status codes supported now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After a discussion with @abhimanyubabbar, From an API perspective, the status code should be 404 instead of 204.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although, in both of the scenarios the customer is looking for:

  1. Mergeable sync which now doesn't exist anymore.
  2. Unintended sync

We should probably return Resource Not Found.

@codecov
Copy link

codecov bot commented Nov 24, 2022

Codecov Report

Base: 46.71% // Head: 46.88% // Increases project coverage by +0.16% 🎉

Coverage data is based on head (588cd92) compared to base (cf80d67).
Patch coverage: 82.05% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2732      +/-   ##
==========================================
+ Coverage   46.71%   46.88%   +0.16%     
==========================================
  Files         298      298              
  Lines       48883    48934      +51     
==========================================
+ Hits        22837    22942     +105     
+ Misses      24597    24537      -60     
- Partials     1449     1455       +6     
Impacted Files Coverage Δ
warehouse/api.go 69.82% <71.42%> (-0.37%) ⬇️
warehouse/warehousegrpc.go 80.25% <100.00%> (+4.47%) ⬆️
config/backend-config/noop.go 0.00% <0.00%> (-42.11%) ⬇️
...rvices/streammanager/kafka/client/testutil/util.go 66.99% <0.00%> (-3.89%) ⬇️
testhelper/log/log.go 7.69% <0.00%> (-3.85%) ⬇️
services/rsources/handler.go 73.16% <0.00%> (-0.85%) ⬇️
jobsdb/jobsdb.go 72.96% <0.00%> (+0.12%) ⬆️
router/router.go 73.86% <0.00%> (+0.18%) ⬆️
jobsdb/backup.go 75.81% <0.00%> (+0.36%) ⬆️
warehouse/schema.go 51.42% <0.00%> (+1.14%) ⬆️
... and 3 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

Copy link
Contributor

@abhimanyubabbar abhimanyubabbar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nit, overall looks good. 👍🏼

warehouse/api.go Outdated
@@ -106,6 +110,7 @@ const (
TriggeredSuccessfully = "Triggered successfully"
NoPendingEvents = "No pending events to sync for this destination"
DownloadFileNamePattern = "downloadfile.*.tmp"
NoSuchSyncs = "No such sync exist"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: NoSuchSync

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants