[Bug]: Metrics not showing up in the FlinkRunner

[kennknowles]

What happened?

There have been a number of similar reports of limitations in metrics in the FlinkRunner. Here is one exmple from the mailing list:

I'm creating a beam pipeline to run on flink runner. I've created custom metrics in beam(JAVA SDK). But I'm unable to see it in flink dashboard or flink rest API response. Although I'm able to see under Accumulators section but not under metrics. Under accumulators I can see,

{ "metrics": { "attempted": [{ "urn": "beam:metric:user:sum_int64:v1", "type": "beam:metrics:sum_int64:v1", "payload": "FA==", "labels": { "NAMESPACE": "Bingo", "NAME": "bingotest", "PTRANSFORM": "log/ParMultiDo(LogOutput)" } }] } }

I'm also able to log the metrics in beam part.
I'm using Apache Beam 2.55.0 and Flink runner 1.16.

I also tried to export all the metrics using flink configuration to the prometheus metric reporter. There I'm able to see only the metrics generated by Flink.

This ticket is to generally track down the metrics plumbing problems. We will focus on Java on the non-portable runner at first, I think, since it is the most used.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[mravi]

@kennknowles I can confirm that this works on our internal infra. which runs on Beam 2.50, python sdk and Flink 1.17.

{host}/jobs/{job_id}/vertices/{vertex_id}/metrics/

Do you know if this is specific to a Beam/Flink version, sdk ?