Preload page index for async ParquetObjectReader

[suremarc]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Currently the ParquetMetaData object has optional fields for the column & offset indexes which are unpopulated at first. When the ArrowReaderBuilder is created using ArrowReaderOptions::with_page_index(true) it loads the page index at query time. However, this is potentially suboptimal as it incurs additional latency making an extra request (typically to object storage which is high-latency) for each query.

Describe the solution you'd like
A new method for the ParquetObjectReader that toggles loading the page index at construction time, something like this:

impl ParquetObjectReader {
    pub fn preload_page_index(self, should_preload: bool) -> Self {
        self.preload_page_index = true
    }
}

which would trigger conditional logic in the get_metadata function to return metadata with the page index already loaded.

Describe alternatives you've considered
A public async API for deserializing the column & offset index, similar to index_reader but with async support and integrated with AsyncFileReader to enable coalescing of multiple fetches.

[tustvold]

The design of AsyncFileReader is already written in such a way as to allow this, in particular implementations may override get_metadata and return a Metadata that already has the page index loaded

[suremarc]

This is true, but there is no easy way to deserialize the page index asynchronously. Currently the easiest way I have found to do this is to fetch the relevant page index offsets, create a special implementation of ChunkReader that holds the serialized page index bytes in memory, and shove the page index data into that special structure, then use the synchronous functions in index_reader to deserialize it into the column & offset indexes.

I have found that the above approach works, although extremely hacky, but I'd ask that the maintainers of this library at least consider exposing a built-in way to deserialize the page index in async code. Again, something like index_reader but asyncified could work if making additional changes to the ParquetObjectReader is not desired.

[tustvold]

👍 I will spend some time working out an async API for reading metadata, #3851 is also related

[alamb]

PR #4216

[tustvold]

label_issue.py automatically added labels {'parquet'} from #4142