[Task][Go SDK]: Dataflow Go PostCommits timing out

[lostluck]

What needs to happen?

Currently the postcommits are failing. We need to investigate where the time is being spent and if possible, either parallelized it better (WRT beam project Dataflow quotas), or consolidate some test pipelines into single jobs (better handling quota), or reduce the number of specific tests we're running against dataflow post commits (eg. remove low signal tests).

As a stop gap, double the 2.5h timeout to 5h.

Issue Priority

Priority: 1

Issue Component

Component: sdk-go

[lostluck]

Specifically, we could reduce the number of pipelines by consolidating them into a single pipeline for the specific area. The we have Dataflow filtering out the individual pipelines where the consolidated ones are used instead. This may require some test prefix renaming to simplify.

beam/sdks/go/test/integration/integration.go

Line 222 in a393efa

var dataflowFilters = []string{

eg. All the State Pipelines could be consolidated into a single pipeline:
TestConsolidated_StateAPIPipeline

And the State tests renamed and prefixed, eg. TestStateAPI_MapState

Allowing the other runners to filter out TestConsolidated.* and Dataflow to filter out TestStateAPI* (along with the indication that it's covered by the consolidated test)

And similarly across common areas in the integration tests.

[Abacn]

Seen this before and was transient #23311

[lostluck]

Yeah, transient or not, it's a flake, which means it interrupts velocity.

Since our apache-beam-testing project for Dataflow is a shared resource and we have only so many simultaneous job slots, I'd rather get the post commit back down under an hour.

[lostluck]

Looks like this has stabilized.