Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[coordinator] Return HTTP 400 if all sent samples encounter too old/new or other bad request error #1692

Merged
merged 21 commits into from
Jun 7, 2019

Conversation

robskillington
Copy link
Collaborator

@robskillington robskillington commented Jun 1, 2019

What this PR does / why we need it:

This should fix issue #1709.

Special notes for your reviewer:

Does this PR introduce a user-facing and/or backwards incompatible change?:

NONE

Does this PR require updating code package or user-facing documentation?:

NONE

@robskillington
Copy link
Collaborator Author

Need to add tests still.

@robskillington robskillington force-pushed the r/return-400-on-bad-request-writes branch from 32bf1e0 to cf94bf8 Compare June 1, 2019 15:41
for _, err := range errs {
switch {
case client.IsBadRequestError(err):
fallthrough
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be worth having a different output error here if it's bad requests vs invalid params specifically?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need both to be HTTP 400 (since local aggregation can also cause a "too old" error, which will be invalidparams not a client bad request error).

The error actually returned to the client I haven't changed, it will always be the "last error".


Storage() storage.Storage
}

// BatchError allows for access to individual errors.
type BatchError interface {
error
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Considering we’re only returning the last error in this path, can we add a new implementation of multierr which doesn’t accumulate?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I need to actually look at each error and determine 100% of them are bad request like errors, otherwise if there was one real error, the request needs to be retried.

I use Errors() to first establish all real errors are bad request like errors, then I use LastError() just to grab the one for logging purposes. I could probably remote LastError() tbh from this interface.

Copy link
Collaborator

@prateek prateek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM w/nit

@@ -183,7 +186,7 @@ echo "Validating topology"
echo "Done validating topology"

echo "Waiting until shards are marked as available"
ATTEMPTS=10 TIMEOUT=2 retry_with_backoff \
ATTEMPTS=100 TIMEOUT=2 retry_with_backoff \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! Much nicer haha, can we do the same for integration tests?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True yeah, will update the common ones.

@robskillington robskillington changed the title [coordinator] Return 400 if all sent samples too old/new or bad request in some way [coordinator] Return HTTP 400 if all sent samples encounter too old/new or other bad request error Jun 7, 2019
@codecov
Copy link

codecov bot commented Jun 7, 2019

Codecov Report

Merging #1692 into master will decrease coverage by 14.7%.
The diff coverage is 66.6%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master   #1692      +/-   ##
=========================================
- Coverage    71.9%   57.1%   -14.8%     
=========================================
  Files         976     968       -8     
  Lines       81530   81092     -438     
=========================================
- Hits        58693   46378   -12315     
- Misses      18996   31113   +12117     
+ Partials     3841    3601     -240
Flag Coverage Δ
#aggregator 70.6% <ø> (-11.9%) ⬇️
#cluster 58.5% <ø> (-27.3%) ⬇️
#collector 63.9% <ø> (ø) ⬆️
#dbnode 72.2% <100%> (-7.9%) ⬇️
#m3em 52.3% <ø> (-21%) ⬇️
#m3ninx 61% <ø> (-13.2%) ⬇️
#m3nsch 78% <ø> (+26.8%) ⬆️
#metrics 17.6% <ø> (ø) ⬆️
#msg 74.7% <ø> (-0.2%) ⬇️
#query 32.2% <65%> (-34.2%) ⬇️
#x 68.3% <71.4%> (-17.2%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 294ce46...2a7382b. Read the comment docs.

@robskillington robskillington merged commit 78d138a into master Jun 7, 2019
@robskillington robskillington deleted the r/return-400-on-bad-request-writes branch June 7, 2019 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants