Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle error conditions when simulating ingest pipelines with verbosity enabled #63327

Merged
merged 7 commits into from
Oct 7, 2020

Conversation

danhermann
Copy link
Contributor

Fixes #63199.

@danhermann danhermann added >bug :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP v8.0.0 v7.10.0 v7.9.3 labels Oct 6, 2020
@danhermann danhermann requested a review from jakelandis October 6, 2020 13:57
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features (:Core/Features/Ingest)

@elasticmachine elasticmachine added the Team:Data Management Meta label for data/management team label Oct 6, 2020
Comment on lines -77 to -80
if (e instanceof ElasticsearchException) {
ElasticsearchException elasticsearchException = (ElasticsearchException) e;
//else do nothing, let the tracking processors throw the exception while recording the path up to the failure
if (elasticsearchException.getCause() instanceof IllegalStateException) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This combination of if statements allowed cases of ElasticsearchException with causes other than those of type IllegalStateException to fall through with invoking the failure handler. I believe the original intention was to catch cyclical pipelines so I believe the correct behavior is to check for both conditions at once and allow any other exception scenarios to fall through to the "now that we know there are no cycles" condition.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally, it seems brittle to check for pipeline cycles based only on the class of the inner exception when there's nothing that would prevent other processors from throwing exceptions of type IllegalStateException that have nothing to do with pipeline cycles.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was briefly discussed a while back #34155 (comment) ..IIRC it had to do with amount of overhead to introduce a custom exception for an edge case.

@danhermann
Copy link
Contributor Author

@elasticmachine update branch

@danhermann
Copy link
Contributor Author

@elasticmachine run elasticsearch-ci/1

Copy link
Contributor

@jakelandis jakelandis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - thanks for fixing this. However a REST test would be a welcome addition.

Comment on lines -77 to -80
if (e instanceof ElasticsearchException) {
ElasticsearchException elasticsearchException = (ElasticsearchException) e;
//else do nothing, let the tracking processors throw the exception while recording the path up to the failure
if (elasticsearchException.getCause() instanceof IllegalStateException) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was briefly discussed a while back #34155 (comment) ..IIRC it had to do with amount of overhead to introduce a custom exception for an edge case.

@danhermann
Copy link
Contributor Author

@elasticmachine update branch

@danhermann
Copy link
Contributor Author

I added a REST test and narrowed the conditions used to identify the pipeline cycle error.

@andreidan andreidan added v7.11.0 and removed v7.10.0 labels Oct 7, 2020
@danhermann danhermann merged commit a644ba1 into elastic:master Oct 7, 2020
@danhermann danhermann deleted the 63199_simulate_pipeline_error branch October 7, 2020 19:11
danhermann added a commit to danhermann/elasticsearch that referenced this pull request Oct 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP Team:Data Management Meta label for data/management team v7.9.3 v7.10.0 v7.11.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Ingest node pipelines] 502 Bad Gateway error when simulating a pipeline
4 participants