Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BEAM-11948] Drop support for Flink 1.8 and 1.9 #14203

Merged
merged 1 commit into from
Apr 8, 2021
Merged

[BEAM-11948] Drop support for Flink 1.8 and 1.9 #14203

merged 1 commit into from
Apr 8, 2021

Conversation

iemejia
Copy link
Member

@iemejia iemejia commented Mar 11, 2021

@iemejia iemejia requested review from dmvk and ibzib March 11, 2021 18:17
@iemejia
Copy link
Member Author

iemejia commented Mar 11, 2021

Run Flink ValidatesRunner

@iemejia
Copy link
Member Author

iemejia commented Mar 11, 2021

Run Java Flink PortableValidatesRunner Batch

@iemejia
Copy link
Member Author

iemejia commented Mar 11, 2021

Run Java Flink PortableValidatesRunner Streaming

ibzib
ibzib reviewed Mar 11, 2021
View reviewed changes
Copy link
Contributor

@ibzib ibzib left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dont' forget to add a note to CHANGES.md.

website/www/site/content/en/documentation/runners/flink.md Outdated Show resolved Hide resolved
website/www/site/content/en/documentation/runners/flink.md Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Mar 11, 2021 •
edited
Loading

Codecov Report

Merging #14203 (05b3fd3) into master (ba4bb1f) will decrease coverage by 0.02%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #14203      +/-   ##
==========================================
- Coverage   83.41%   83.39%   -0.03%     
==========================================
  Files         469      469              
  Lines       58727    58727              
==========================================
- Hits        48989    48975      -14     
- Misses       9738     9752      +14
Impacted Files Coverage Δ
.../python/apache_beam/transforms/periodicsequence.py
.../python/apache_beam/runners/worker/statesampler.py
.../srcs/sdks/python/apache_beam/internal/gcp/auth.py
...sdks/python/apache_beam/portability/common_urns.py
...m/testing/benchmarks/nexmark/models/auction_bid.py
...amples/snippets/transforms/elementwise/__init__.py
...s/snippets/transforms/aggregation/combinevalues.py
.../sdks/python/apache_beam/options/value_provider.py
..._beam/testing/benchmarks/nexmark/queries/query0.py
...sdks/python/apache_beam/utils/subprocess_server.py
... and 928 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ba4bb1f...05b3fd3. Read the comment docs.

@iemejia iemejia force-pushed the BEAM-11948-flink-remove-1.8-1.9 branch from 9e3efdc to e42423f Compare March 11, 2021 19:47
@iemejia
Copy link
Member Author

iemejia commented Mar 11, 2021

Run Flink ValidatesRunner

@iemejia
Copy link
Member Author

iemejia commented Mar 11, 2021

Run Java Flink PortableValidatesRunner Batch

@iemejia
Copy link
Member Author

iemejia commented Mar 11, 2021

Run Java Flink PortableValidatesRunner Streaming

ibzib
ibzib approved these changes Mar 11, 2021
View reviewed changes
Copy link
Contributor

@ibzib ibzib left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@iemejia iemejia force-pushed the BEAM-11948-flink-remove-1.8-1.9 branch from e42423f to 4dacd2a Compare March 12, 2021 07:00
@iemejia
Copy link
Member Author

iemejia commented Mar 12, 2021

Rebased to catch the VR issue fix

@iemejia
Copy link
Member Author

iemejia commented Mar 12, 2021

Run Flink ValidatesRunner

@iemejia
Copy link
Member Author

iemejia commented Mar 12, 2021

Run Java Flink PortableValidatesRunner Batch

@iemejia
Copy link
Member Author

iemejia commented Mar 12, 2021

Run Java Flink PortableValidatesRunner Streaming

dmvk
dmvk reviewed Mar 12, 2021
View reviewed changes
runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkBatchTransformTranslators.java
outputDataSet =
new MapPartitionOperator<>(inputDataSet, typeInformation, doFnWrapper, fullName);
}
outputDataSet = new FlatMapOperator<>(inputDataSet, typeInformation, doFnWrapper, fullName);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can now remove MapPartition interface from https://github.com/apache/beam/blob/master/runners/flink/src/main/java/org/apache/beam/runners/flink/translation/functions/FlinkDoFnFunction.java#L65

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh good catch. I will fix, thx!

dmvk
dmvk approved these changes Mar 12, 2021
View reviewed changes
Copy link
Member

@dmvk dmvk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@iemejia iemejia force-pushed the BEAM-11948-flink-remove-1.8-1.9 branch from 4dacd2a to 58f2bf8 Compare March 12, 2021 11:31
@iemejia
Copy link
Member Author

iemejia commented Mar 12, 2021

Run Flink ValidatesRunner

@iemejia
Copy link
Member Author

iemejia commented Mar 12, 2021

Run Java Flink PortableValidatesRunner Batch

@iemejia
Copy link
Member Author

iemejia commented Mar 12, 2021

Run Java Flink PortableValidatesRunner Streaming

@iemejia iemejia mentioned this pull request Mar 25, 2021
Merged
@iemejia iemejia changed the title [BEAM-11948] Drop support for Flink 1.8 and 1.9 [DO NOT MERGE][BEAM-11948] Drop support for Flink 1.8 and 1.9 Mar 26, 2021
@iemejia
Copy link
Member Author

iemejia commented Mar 26, 2021

This one should not be merged until Beam 2.29.0 release is released

@iemejia iemejia force-pushed the BEAM-11948-flink-remove-1.8-1.9 branch from 58f2bf8 to 05b3fd3 Compare March 26, 2021 07:19
@iemejia
Copy link
Member Author

iemejia commented Mar 26, 2021

Rebased to remove the warning introduced in 14214

@iemejia
Copy link
Member Author

iemejia commented Mar 26, 2021

Run Flink ValidatesRunner

@iemejia
Copy link
Member Author

iemejia commented Mar 26, 2021

Run Java Flink PortableValidatesRunner Batch

@iemejia
Copy link
Member Author

iemejia commented Mar 26, 2021

Run Java Flink PortableValidatesRunner Streaming

@iemejia
Copy link
Member Author

iemejia commented Mar 26, 2021

Failing PVR Streaming tests are unrelated to this PR for ref #14313

@iemejia
Copy link
Member Author

iemejia commented Mar 26, 2021

Run Java Flink PortableValidatesRunner Streaming

@iemejia iemejia changed the title [DO NOT MERGE][BEAM-11948] Drop support for Flink 1.8 and 1.9 [BEAM-11948] Drop support for Flink 1.8 and 1.9 Apr 8, 2021
@iemejia
Copy link
Member Author

iemejia commented Apr 8, 2021

I was waiting in case there was a further need to recut the 2.29.0 branch so this did not get accidentally in but it seems the 2.29.0 branch is now stable, so we can go ahead and get this one ready for 2.30.0.

I will merge it and if any issue on 2.29.0 requires a branch recut I will backport this PR as a revert. Thanks everyone !

@iemejia iemejia merged commit 572a99b into apache:master Apr 8, 2021
@iemejia iemejia deleted the BEAM-11948-flink-remove-1.8-1.9 branch April 8, 2021 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ibzib ibzib ibzib approved these changes

@dmvk dmvk dmvk approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@iemejia @dmvk @ibzib