Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use airlift json codec in http event listener, fixing airlift/airlift#983 #10843

Merged
merged 3 commits into from
Feb 23, 2022

Conversation

mosiac1
Copy link
Contributor

@mosiac1 mosiac1 commented Jan 28, 2022

A bug in airlift described in this issue airlift/airlift#983 affects the event listener, causing it to leak threads and timeout all requests. This fixes the effect of the issue by not using the dangerous class but not the source of the issue.

I also changed the testServerDisconnectShouldRetry to cover the issue above. With the old setup, this would fail because the http-client's threads would go into deadlock.

Other changes: replace the name querylog with httpeventlistener or eventlistener in tests, cleanup logs.

@cla-bot cla-bot bot added the cla-signed label Jan 28, 2022
@hashhar
Copy link
Member

hashhar commented Jan 31, 2022

Supersedes #10781. cc: @losipiuk

@hashhar hashhar requested a review from electrum January 31, 2022 06:45
@mosiac1
Copy link
Contributor Author

mosiac1 commented Feb 3, 2022

@losipiuk @electrum hello I was wondering if I could get some 👀 on this, it would be nice to get some feedback and maybe have this ready for the next release. Thanks!

Comment on lines 79 to 81
this.queryCompletedEventJsonCodec = requireNonNull(queryCompletedEventJsonCodec, "query complete event json codec is null");
this.queryCreatedEventJsonCodec = requireNonNull(queryCreatedEventJsonCodec, "query create event json codec is null");
this.splitCompletedEventJsonCodec = requireNonNull(splitCompletedEventJsonCodec, "split complete json codec is null");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use variable name in error string (queryCompletedEventJsonCodec is null, ...)

Copy link
Member

@losipiuk losipiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mosiac1. Overally PR looks fine. Would you be so kind and split it to separate commits with independent changes.

@mosiac1
Copy link
Contributor Author

mosiac1 commented Feb 8, 2022

Thanks very much for the review?

How would like the commits be ordered? Would the following work?

@losipiuk
Copy link
Member

losipiuk commented Feb 8, 2022

Thanks very much for the review?

How would like the commits be ordered? Would the following work?

makes sense

  • Improve retry logic and logs for HTTP event listener

This one can be split into two

@mosiac1 mosiac1 force-pushed the 370_http_events_fix branch from a913c59 to 6a0c8b7 Compare February 14, 2022 17:13
@mosiac1
Copy link
Contributor Author

mosiac1 commented Feb 15, 2022

@losipiuk Hello, I reordered the commits as we discussed!

CI is failing because of iceberg, could we please rerun it?

@mosiac1 mosiac1 force-pushed the 370_http_events_fix branch from 6a0c8b7 to 36089ac Compare February 15, 2022 11:29
}
else {
log.error("QueryId = \"%s\", attempt = %d/%d, URL = %s | Ingest server responded with code %d, fatal error",
queryId, attempt + 1, config.getRetryCount() + 1, request.getUri().toString(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: toString() in uri.toString() not needed

queryId, attempt + 1, config.getRetryCount() + 1, request.getUri().toString());
else {
log.debug("QueryId = \"%s\", attempt = %d/%d, URL = %s | Query event delivered successfully",
queryId, attempt + 1, config.getRetryCount() + 1, request.getUri().toString());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: toString() in uri.toString() not needed

@losipiuk losipiuk merged commit 02fa8b4 into trinodb:master Feb 23, 2022
@losipiuk
Copy link
Member

Merged. Sorry for delay @mosiac1 (I forgot about this one).

@mosiac1
Copy link
Contributor Author

mosiac1 commented Feb 23, 2022

Thank you, no worries about the delay

@github-actions github-actions bot added this to the 372 milestone Feb 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

3 participants