Message size can exceed maximum limit in Teleport Connect when listing resources

[gzdunek]

When calling ListUnifiedResources, Connect returns an error if the response size is too large.
Web UI handles that by retrying the call with smaller limit:

teleport/api/client/client.go

Line 3746 in f198694

if trace.IsLimitExceeded(err) {

We should use that function in Connect too.

What's more, we can increase the default max message size from the default 4 MB (this should be done for both the gRPC client in tshd and in Electron).

Logs from tshd:

[05-06-24 09:27:56] ERRO [CONN:APIS] Request failed. error:[
[05-06-24 09:27:56] ERROR REPORT:
[05-06-24 09:27:56] Original Error: *interceptors.RemoteError grpc: received message larger than max (5752723 vs. 4194304)
[05-06-24 09:27:56] Stack Trace:
[05-06-24 09:27:56] 	github.com/gravitational/teleport/[email protected]/client/client.go:3891 github.com/gravitational/teleport/api/client.(*Client).ListUnifiedResources
[05-06-24 09:27:56] 	github.com/gravitational/teleport/lib/teleterm/services/unifiedresources/unifiedresources.go:52 github.com/gravitational/teleport/lib/teleterm/services/unifiedresources.List
[05-06-24 09:27:56] 	github.com/gravitational/teleport/lib/teleterm/daemon/daemon.go:1004 github.com/gravitational/teleport/lib/teleterm/daemon.(*Service).ListUnifiedResources.func1
[05-06-24 09:27:56] 	github.com/gravitational/teleport/lib/teleterm/clusters/cluster.go:311 github.com/gravitational/teleport/lib/teleterm/clusters.AddMetadataToRetryableError
[05-06-24 09:27:56] 	github.com/gravitational/teleport/lib/teleterm/daemon/daemon.go:1003 github.com/gravitational/teleport/lib/teleterm/daemon.(*Service).ListUnifiedResources
[05-06-24 09:27:56] 	github.com/gravitational/teleport/lib/teleterm/apiserver/handler/handler_unified_resources.go:45 github.com/gravitational/teleport/lib/teleterm/apiserver/handler.(*Handler).ListUnifiedResources
[05-06-24 09:27:56] 	github.com/gravitational/teleport/gen/proto/go/teleport/lib/teleterm/v1/service_grpc.pb.go:1564 github.com/gravitational/teleport/gen/proto/go/teleport/lib/teleterm/v1._TerminalService_ListUnifiedResources_Handler.func1
[05-06-24 09:27:56] 	github.com/gravitational/teleport/lib/teleterm/apiserver/middleware.go:37 github.com/gravitational/teleport/lib/teleterm/apiserver.New.withErrorHandling.func1
[05-06-24 09:27:56] 	github.com/gravitational/teleport/gen/proto/go/teleport/lib/teleterm/v1/service_grpc.pb.go:1566 github.com/gravitational/teleport/gen/proto/go/teleport/lib/teleterm/v1._TerminalService_ListUnifiedResources_Handler
[05-06-24 09:27:56] 	google.golang.org/[email protected]/server.go:1369 google.golang.org/grpc.(*Server).processUnaryRPC
[05-06-24 09:27:56] 	google.golang.org/[email protected]/server.go:1780 google.golang.org/grpc.(*Server).handleStream
[05-06-24 09:27:56] 	google.golang.org/[email protected]/server.go:1019 google.golang.org/grpc.(*Server).serveStreams.func2.1
[05-06-24 09:27:56] 	runtime/asm_amd64.s:1650 runtime.goexit
[05-06-24 09:27:56] User Message: grpc: received message larger than max (5752723 vs. 4194304)] apiserver\middleware.go:39

Slack discussion https://gravitational.slack.com/archives/C0DF0TPMY/p1717600956587019
Original issue https://gravitational.zendesk.com/agent/tickets/10781