Skip to content

Commit

Permalink
[BEAM-12277] Make Flink 1.13 the default Flink version to run the tests
Browse files Browse the repository at this point in the history
  • Loading branch information
iemejia committed May 28, 2021
1 parent b342805 commit 8ad8965
Show file tree
Hide file tree
Showing 5 changed files with 31 additions and 6 deletions.
2 changes: 1 addition & 1 deletion .test-infra/jenkins/CommonTestProperties.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ class CommonTestProperties {
}

static String getFlinkVersion() {
return "1.12"
return "1.13"
}

enum Runner {
Expand Down
1 change: 1 addition & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@
## New Features / Improvements

* `CREATE FUNCTION` DDL statement added to Calcite SQL syntax. `JAR` and `AGGREGATE` are now reserved keywords. ([BEAM-12339](https://issues.apache.org/jira/browse/BEAM-12339)).
* Flink 1.13 is now supported by the Flink runner ([BEAM-12277](https://issues.apache.org/jira/browse/BEAM-12277)).
* X feature added (Java/Python) ([BEAM-X](https://issues.apache.org/jira/browse/BEAM-X)).

## Breaking Changes
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,13 @@ public void notifyOfRemovedMetric(Metric metric, String metricName, MetricGroup
final String name = group.getMetricIdentifier(metricName, this);
super.notifyOfRemovedMetric(metric, metricName, group);
synchronized (this) {
ps.printf("%s: %s%n", name, Metrics.toString(metric));
try {
ps.printf("%s: %s%n", name, Metrics.toString(metric));
} catch (NullPointerException e) {
// Workaround to avoid a NPE on Flink's DeclarativeSlotManager during unregister
// TODO Remove once FLINK-22646 is fixed on upstream Flink.
log.warn("unable to log details on metric {}", name);
}
}
}

Expand Down
2 changes: 1 addition & 1 deletion sdks/python/apache_beam/options/pipeline_options.py
Original file line number Diff line number Diff line change
Expand Up @@ -1274,7 +1274,7 @@ def _add_argparse_args(cls, parser):
class FlinkRunnerOptions(PipelineOptions):

# These should stay in sync with gradle.properties.
PUBLISHED_FLINK_VERSIONS = ['1.8', '1.9', '1.10', '1.11', '1.12']
PUBLISHED_FLINK_VERSIONS = ['1.10', '1.11', '1.12', '1.13']

@classmethod
def _add_argparse_args(cls, parser):
Expand Down
24 changes: 21 additions & 3 deletions website/www/site/content/en/documentation/runners/flink.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,7 @@ Starting with Beam 2.18.0, pre-built Flink Job Service Docker images are availab
[Flink 1.10](https://hub.docker.com/r/apache/beam_flink1.10_job_server),
[Flink 1.11](https://hub.docker.com/r/apache/beam_flink1.11_job_server),
[Flink 1.12](https://hub.docker.com/r/apache/beam_flink1.12_job_server).
[Flink 1.13](https://hub.docker.com/r/apache/beam_flink1.13_job_server).
{{< /paragraph >}}

<!-- TODO(BEAM-10214): Use actual lists here and below. -->
Expand Down Expand Up @@ -311,8 +312,8 @@ reference.
## Flink Version Compatibility

The Flink cluster version has to match the minor version used by the FlinkRunner.
The minor version is the first two numbers in the version string, e.g. in `1.12.0` the
minor version is `1.12`.
The minor version is the first two numbers in the version string, e.g. in `1.13.0` the
minor version is `1.13`.

We try to track the latest version of Apache Flink at the time of the Beam release.
A Flink version is supported by Beam for the time it is supported by the Flink community.
Expand All @@ -326,7 +327,24 @@ To find out which version of Flink is compatible with Beam please see the table
<th>Artifact Id</th>
</tr>
<tr>
<td rowspan="3">&ge; 2.30.0</td>
<td rowspan="4">&ge; 2.31.0</td>
<td>1.13.x <sup>*</sup></td>
<td>beam-runners-flink-1.13</td>
</tr>
<tr>
<td>1.12.x <sup>*</sup></td>
<td>beam-runners-flink-1.12</td>
</tr>
<tr>
<td>1.11.x <sup>*</sup></td>
<td>beam-runners-flink-1.11</td>
</tr>
<tr>
<td>1.10.x</td>
<td>beam-runners-flink-1.10</td>
</tr>
<tr>
<td rowspan="3">2.30.0</td>
<td>1.12.x <sup>*</sup></td>
<td>beam-runners-flink-1.12</td>
</tr>
Expand Down

0 comments on commit 8ad8965

Please sign in to comment.