API: Consistent tag/attribute/field/resource namespacing and structure

[jmacd]

The OpenTelemetry Java prototype has, in various places, five different kinds of key:value. Spans have attributes, context has tags, metrics have fields, events have fields, and everything has resources. The current Resource definition is a structure containing a type name as well as a set of key:values, whereas all of the other places where key:values appear, there is no "type" parameter.

As an implementor, having to implement separate code paths for structurally-equivalent types increases code size and complexity.

As an instrumentor, having distinct types adds confusion because, depending on circumstances, I may want to apply the same key in any one of these five key:value locations. For example, at LightStep we have two classes of satellite, those dedicated to specific customers and those belonging to our public service, which may accept traffic from all customers. Dedicated satellites will apply an "organization" tag at the resource level, because it applies to all spans from a tracer, whereas public satellites will apply the same key as a span attribute, since it is a dynamic property.

This raises a question as to why there is a structural difference between resources and the other kinds of key:value, and there are a couple of related issues that I want to bring up.

In #62, a discussion of how to merge resources deterministically, with the observation that namespaces could help.

In #67, a request for structured field values, for OpenTracing compatibility, which are somehow similar to resource definitions.

This leads me to look for similarities, based on the observation that resource key:values and span attribute key:values ought to be the same kind of thing. In the current Resource definition, there is a "Type" field that acts as both a key name ("thing being monitored") and a schema name ("semantics of the data").

Take for example the current form of a k8s resource:

{
  "Type": "k8s",
  "Labels": {
    "cluster.name": "xy",
    "namespace.name": "meta",
    "pod.name": "a-b-6d8f45bbc8-2smm4",
  }
}

As in the OpenCensus model, this is logically expanded into three key values, where "k8s" describes what is being monitored and the structure of the data.

"k8s.cluster.name": "xy"
"k8s.namespace.name": "meta"
"k8s.pod.name":"a-b-6d8f45bbc8-2smm4"

We could address the discrepency between resource key:values (which have namespaces and structure) and attribute key:values (which do not) by separating the name from the type information and allowing structured field values. Let's use the Google precedent for encoding protobuf Any types. The example would be rewritten as:

"resource.deployment": {
  "@type": "kubernetes.io/resource/k8s",
  "cluster.name": "xy",
  "namespace.name": "meta",
  "pod.name": "a-b-6d8f45bbc8-2smm4"
}

Here, we would have to adopt semantic conventions to interpret the data. Roughly, I'm imagining in this example:

1. "monitored.deployment" is adopted as the conventional attribute name for a deployment
2. "k8s" is a known type, referring to a fully-qualified type name
3. Keys use URL prefixes and "/" to encode hierarchical namespaces
4. Types use URL prefixes and "/" to encode hierarchical namespaces
5. Standard conventions (part of the OpenTelemtry spec) do not use a URL prefix, as they are shared conventions, so standard definitions (e.g., "db.type", "peer.ipv4") do not require a namespace.

These conventions, which support structured key:values and recognize the need for namespaces, helps address issue 62 and issue 67. To merge resources becomes no different than merging span attributes. Structured logging would be supported using structured values, as well.

In practice, it makes sense not to allow structured values everywhere. For context propagation (e.g., OpenTracing baggage), string values are enforced. For pre-aggregation fields (e.g., OpenCensus metrics), scalar values should be required. At the same time, it will be desireable to be able to refer to elements of structured values when defining aggregations. For example, to pre-aggregate by the deployment cluster name, use "resource.deployment#cluster.name" (i.e., "#" as a fragment separator).

[jmacd]

We have decided to use the "Named" Tracer and Meter as a namespace. We have decided not to include resources in the v1.0 release. This can be closed.