[sourcemaps] support multiple fleet addresses

[stuartnelson3]

The current sourcemap fetching code for fleet assumes that we receive only a single fleet server.

Add support for round-robin requests between the fleet servers. Assume all fleet servers are in the same cluster, ie. an artifact is available on each individual server.

Original comment:
#5410 (comment)

[estolfo]

Is there any benefit to round-robining the requests vs just selecting a random address to send the request to? I'm guessing selecting a random address would be simpler to implement.
A basic round-robin implementation doesn't take into account load or anything else so I don't see the difference between that and random selection in this context.

[axw]

@estolfo random selection sounds good to me, and I don't think there's any benefit to round robin. @stuartnelson3 do you have any concerns with that?

[stuartnelson3]

No concerns from me

[estolfo]

@axw your comment here suggests that any artifact can be fetched from any fleet server. So why would we need a map of (service.name, service.version, bundle.path) combination to a fleet server address? It seems that we could simplify this code to just keep a list of fleet addresses and not bother with keeping a map.
The way the code currently is, we do already assume any artifact is available at any address, as we arbitrarily select the first fleet address in the slice fleetCfg.Hosts to use for all fetching requests. Let me know if you're aware of plans to keep certain artifacts on dedicated fleet servers or if I'm missing something else.

[axw]

@estolfo we don't need to map all of those things to every fleet server address, as they're indeed orthogonal. Does your question stem from the way the code is structured here?

apm-server/sourcemap/fleet_store.go

Lines 77 to 86 in aeafd43

	fleetURLs := make(map[key]string)
	for _, cfg := range cfgs {
	k := key{cfg.ServiceName, cfg.ServiceVersion, cfg.BundleFilepath}
	u, err := common.MakeURL(fleetCfg.Protocol, cfg.SourceMapURL, host, 8220)
	if err != nil {
	return fleetStore{}, err
	}
	fleetURLs[k] = u
	}

What we need is:

• a set of (service.name, service.version, bundle.filepath, sourcemap.url) tuples
• a set of Fleet Server addresses

The sourcemap.url bit is actually just the URL path, and should be combined with the Fleet Server protocol (URL scheme -- "http://" or "https://"), and the Fleet Server host. I think the way to go would be to defer the common.MakeURL call to fleetStore.fetch, after randomly selecting a host.

[estolfo]

Yes, I understood we had to keep the (service.name, service.version, bundle.filepath, sourcemap.url) tuples and I thought it didn't make sense to store every combination of tuple and fleet server address, as any fleet server could be used for any artifact. I'll change the implementation a bit and make the URL when selecting a host, as you've suggested.