Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add origin pipeline name to on_failure processor metadata #44920

Closed
peterpramb opened this issue Jul 26, 2019 · 1 comment · Fixed by #49076
Closed

Add origin pipeline name to on_failure processor metadata #44920

peterpramb opened this issue Jul 26, 2019 · 1 comment · Fixed by #49076
Assignees
@martijnvg
Labels
:Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >feature

Comments

@peterpramb
Copy link

Describe the feature:

When using multiple chained pipelines with a global failure handler in the parent pipeline (that is exceptions are propagated up) it would be useful to provide access to the failure originating pipeline's name in the on_failure processor.

This can be used in error messages to point to the pipeline where the failure actually occured, as processor names are ambiguous (unless unique per-processor tags are used).
@andyb-elastic andyb-elastic added :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >feature labels Jul 26, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features

@peterpramb peterpramb mentioned this issue Aug 6, 2019
Closed
@martijnvg martijnvg mentioned this issue Nov 12, 2019
Closed
12 tasks
@martijnvg martijnvg self-assigned this Nov 13, 2019
martijnvg added a commit to martijnvg/elasticsearch that referenced this issue Nov 14, 2019
@martijnvg
Introduce on_failure_pipeline ingest metadata
7929ec1 
In case an exception occurs inside a pipeline processor,
the pipeline stack is kept around as header in the exception.
Then in the on_failure processor the id of the pipeline the
exception occurred is made accessible via the `on_failure_pipeline`
ingest metadata.

Closes elastic#44920
@martijnvg martijnvg mentioned this issue Nov 14, 2019
Merged
martijnvg added a commit that referenced this issue Nov 26, 2019
@martijnvg
Introduce on_failure_pipeline ingest metadata inside on_failure block (…
2ba00c8 
…#49076)

In case an exception occurs inside a pipeline processor,
the pipeline stack is kept around as header in the exception.
Then in the on_failure processor the id of the pipeline the
exception occurred is made accessible via the `on_failure_pipeline`
ingest metadata.

Closes #44920
martijnvg added a commit to martijnvg/elasticsearch that referenced this issue Nov 26, 2019
@martijnvg
Introduce on_failure_pipeline ingest metadata inside on_failure block (…
6bbeaa5 
…elastic#49076)

In case an exception occurs inside a pipeline processor,
the pipeline stack is kept around as header in the exception.
Then in the on_failure processor the id of the pipeline the
exception occurred is made accessible via the `on_failure_pipeline`
ingest metadata.

Closes elastic#44920
@martijnvg martijnvg mentioned this issue Nov 26, 2019
Merged
martijnvg added a commit that referenced this issue Nov 27, 2019
@martijnvg
Backport: Introduce on_failure_pipeline ingest metadata inside on_fai…
90850f4 
…lure block (#49596)

Backport of #49076

In case an exception occurs inside a pipeline processor,
the pipeline stack is kept around as header in the exception.
Then in the on_failure processor the id of the pipeline the
exception occurred is made accessible via the `on_failure_pipeline`
ingest metadata.

Closes #44920
This was referenced Feb 3, 2020
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@martijnvg martijnvg

Labels
:Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Introduce on_failure_pipeline ingest metadata inside on_failure block martijnvg/elasticsearch
4 participants
@martijnvg @peterpramb @elasticmachine @andyb-elastic