feat(transport): add OpenTelemetry context propagation

[quartzmo]

OpenCensus sunsetting and OpenTelemetry support

The OpenCensus project is obsolete and was archived on July 31st, 2023. This means that any security vulnerabilities that are found will not be patched. We recommend that you begin migrating to OpenCensus tracing to OpenTelemetry, the successor project.

Using the OpenTelemetry-Go - OpenCensus Bridge, you can immediately begin exporting your traces with OpenTelemetry, even while dependencies such as the Google Cloud client libraries for Go remain instrumented with OpenCensus. If you do not use the bridge, you will need to migrate your entire application and all of its instrumented dependencies at once. For simple applications, this may be possible, but we expect the bridge to be helpful if multiple libraries with instrumentation are used.

One year after the Google Cloud Go client libraries release the experimental, opt-in OpenTelemetry tracing support in this PR (mid-Q4 2023), the prior experimental OpenCensus tracing will be removed.

Please refer to the following resources:

• Sunsetting OpenCensus
• OpenTelemetry-Go - OpenCensus Bridge
• What is OpenTelemetry?
• Cloud Trace - Go and OpenTelemetry
• On GCE, use Ops Agent and OpenTelemetry

OpenTelemetry trace context propagation

The context propagation support for OpenCensus tracing (based on the Cloud Trace header) must be supplemented with new and separate context propagation support for OpenTelemetry tracing (based on the W3C Trace Context header). The google.golang.org/api/transport package currently configures context propagation for the Cloud Trace header (X-Cloud-Trace-Context). This is used in all Go client libraries, both HTTP and gRPC, Discovery (Apiary) and GAPIC.

This PR adds support for by wrapping the underlying transport as follows:

func addOpenTelemetryTransport(trans http.RoundTripper, settings *internal.DialSettings) http.RoundTripper {
	if settings.TelemetryDisabled {
		return trans
	}
	return otelhttp.NewTransport(trans)
}

Passing options to otelhttp.NewTransport in HTTP-based clients

Users who want to pass options to otelhttp.NewTransport can disable the library support in the transport package and configure the transport and client themselves.

ctx := context.Background()
trans, err := htransport.NewTransport(ctx,
        http.DefaultTransport,
        option.WithScopes(storage.ScopeFullControl),
)
if err != nil {
        log.Fatal(err)
}
// An example of passing options to the otelhttp.Transport.
otelOpts := otelhttp.WithFilter(func(r *http.Request) bool {
	return r.URL.Path != "/ping"
})
hc := &http.Client{
	Transport: otelhttp.NewTransport(trans, otelOpts),
}
client, err := storage.NewClient(ctx, option.WithHTTPClient(hc))

Note that scopes must be set manually in the user-configured solution.

Passing options to otelgrpc.NewClientHandler in gRPC-based clients

The setup is pretty similar in gRPC using the otelgrpc.NewClientHandler.

projectID := "..."
ctx := context.Background()

// An example of passing options to grpc.WithStatsHandler.
otelOpts := otelgrpc.WithMessageEvents(otelgrpc.ReceivedEvents)
dialOpts := grpc.WithStatsHandler(otelgrpc.NewClientHandler(otelOpts))

ctx := context.Background()
c, err := datastore.NewClient(ctx, projectID, option.WithGRPCDialOption(dialOpts))
if err != nil {
	log.Fatal(err)
}
defer c.Close()

[tmc]

As most of the ecosystem has moved on to opentelemetry, it'd be interesting to hear how otel support here is going to evolve (and what the deprecation path of the opencensus instrumentation looks like). Any input there?

[shollyman]

LGTM, modulo a bit of feedback on the dialcontext testing.