Larger CI Runners to Prevent MIRI OOMing and Improve CI Times

[tustvold]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

Since updating MIRI in #1828 it is periodically OOMing - https://github.com/apache/arrow-rs/actions/workflows/miri.yaml

https://github.com/apache/arrow-rs/actions/runs/2473012537

Describe the solution you'd like

I'm not entirely sure what the best course of action here is, rolling back to a 6 month old MIRI is not ideal and would require backing out changes in #1822, but then neither is having it randomly fail.

It has been a long-time annoyance of mine that the the CI currently takes ~40 minutes to chug through, despite significant caching. This is largely because the runners are rather piddly - https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources. This also precludes automatically running any meaningful benchmarks (#1274). Perhaps we should invest some time into a more powerful CI system such as buildkite which is used by other arrow projects...

Describe alternatives you've considered

None

[alamb]

other parts of the arrow project (e.g. C++) was talking about this -- there may be more beefy infrastructure we could take advantage of there

[jhorstmann]

I recently noticed some of the fuzzing tests for filters running for a long time. Watching htop also showed memory usage increasing while running fuzz_filter. Memory usage was already at ~8Gb before that test though.

Maybe some of those tests could be excluded from running in the miri cfg.

[tustvold]

Another dimension this may provide improvement is disk space, with the 14GB proving insufficient for some use-cases (#2004)

[viirya]

Recently I saw many times of running out of disk space in CI. Although it seems better now.

[alamb]

I got inspired this afternoon while waiting for some other PRs to finish up CI and started a few improvements using the github runners via #2149

[tustvold]

The improvements to split up the CI have largely addressed this, so closing