Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BEAM-13981] Remove Spark Runner specific code for event logging #17073

Merged

Conversation

iemejia
Copy link
Member

@iemejia iemejia commented Mar 11, 2022

R: @aromanenko-dev
CC: @ibzib

Please add a meaningful description for your change here


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Choose reviewer(s) and mention them in a comment (R: @username).
  • Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests

See CI.md for more information about GitHub Actions CI.

@iemejia
Copy link
Member Author

iemejia commented Mar 11, 2022

Run Spark ValidatesRunner

@iemejia
Copy link
Member Author

iemejia commented Mar 11, 2022

Run Spark StructuredStreaming ValidatesRunner

@iemejia
Copy link
Member Author

iemejia commented Mar 11, 2022

Run Java Spark PortableValidatesRunner Batch

@iemejia
Copy link
Member Author

iemejia commented Mar 11, 2022

Run Python Spark ValidatesRunner

@iemejia
Copy link
Member Author

iemejia commented Mar 11, 2022

Run Go Spark ValidatesRunner

Copy link
Contributor

@aromanenko-dev aromanenko-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM since it fixes a problem with an empty event log. But what's about original issue stated in BEAM-11213 ?

@iemejia
Copy link
Member Author

iemejia commented Mar 11, 2022

I understood it more like a user friendliness issue but since this broke metrics on Spark History Server I preferred to go straight to remove the behavior, because history server is more important.

@iemejia iemejia merged commit eea81f4 into apache:master Mar 11, 2022
@iemejia iemejia deleted the BEAM-13981-spark-remove-event-logging-listener branch March 16, 2022 09:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants