memoize s3 client

[trinity-1686a]

Description

memoize s3 client, keeping them until they seem unused. I don't think we ever use more than a single S3StorageConfig, so we could store a single value, but a map we gc seems less confusing if we start having multiple config at once
fix #4933
fix #5236

How was this PR tested?

ran a cluster and looked at searcher logs. Didn't look at janitor logs, though there is no reason for it to behave differently

[github-actions]

On SSD:

Average search latency is 1.01x that of the reference (lower is better).
Ref run id: 3235, ref commit: aac8b49
Link

On GCS:

Average search latency is 0.983x that of the reference (lower is better).
Ref run id: 3236, ref commit: aac8b49
Link

[rdettai]

Even without this caching we didn´t create a new client for every connection as long as the Storage object was reused.

[trinity-1686a]

it wasn't reused cross request

[rdettai]

This makes me wonder if the best approach wouln't be to cache the resolved storages in the storage resolver. This would solve the problem once and for all for all storage types. But I think this solution is already good as it does solve the linked issues.

[trinity-1686a]

the storages are related to a Uri (the S3Client isn't), which changes for each index. It would be interesting to be able to share the debouncer though, which isn't done here. That could make concurrent requests slightly more efficient.