Skip to content

Commit

Permalink
Merge branch 'main' into add_ds_metrics
Browse files Browse the repository at this point in the history
  • Loading branch information
ChrsMark authored Dec 9, 2024
2 parents aefc91d + 373a695 commit e90cbaa
Show file tree
Hide file tree
Showing 39 changed files with 289 additions and 18 deletions.
22 changes: 22 additions & 0 deletions .chloggen/k8s-migration.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Use this changelog template to create an entry for release notes.
#
# If your change doesn't affect end users you should instead start
# your pull request title with [chore] or use the "Skip Changelog" label.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the area of concern in the attributes-registry, (e.g. http, cloud, db)
component: k8s

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Add migration guide for K8s semantic conventions

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
# The values here must be integers.
issues: [1597]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:
22 changes: 22 additions & 0 deletions .chloggen/rule_new.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Use this changelog template to create an entry for release notes.
#
# If your change doesn't affect end users you should instead start
# your pull request title with [chore] or use the "Skip Changelog" label.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: new_component

# The name of the area of concern in the attributes-registry, (e.g. http, cloud, db)
component: security-rule

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Introducing a new security rule namespace

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
# The values here must be integers.
issues: [903]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:
36 changes: 20 additions & 16 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -32,14 +32,15 @@
/model/jvm/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-jvm-approvers

# HTTP semantic conventions
/docs/http/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-http-approvers
/model/http/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-http-approvers
/model/error/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-http-approvers
/model/client/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-http-approvers
/model/network/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-http-approvers @open-telemetry/semconv-system-approvers @open-telemetry/semconv-security-approvers
/model/server/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-http-approvers
/model/url/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-http-approvers
/model/user-agent/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-http-approvers
/docs/http/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-http-approvers
/model/http/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-http-approvers
/model/error/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-http-approvers
/model/client/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-http-approvers
/model/network/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-http-approvers @open-telemetry/semconv-system-approvers @open-telemetry/semconv-security-approvers
/model/server/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-http-approvers
/model/url/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-http-approvers
/model/user-agent/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-http-approvers
/docs/non-normative/http-migration.md @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-http-approvers

# System semantic conventions
/docs/process/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-system-approvers @open-telemetry/semconv-security-approvers
Expand All @@ -58,8 +59,9 @@
/model/device/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-mobile-approvers

# K8s semantic conventions
/docs/resource/k8s.md @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-k8s-approvers
/model/k8s/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-k8s-approvers
/docs/resource/k8s.md @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-k8s-approvers
/model/k8s/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-k8s-approvers
/docs/non-normative/k8s-migration.md @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-k8s-approvers

# Container semantic conventions
/docs/resource/container.md @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-container-approvers
Expand Down Expand Up @@ -95,8 +97,9 @@
/model/vcs/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-cicd-approvers

# Database semantic conventions
/docs/database/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-db-approvers
/model/database/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-db-approvers
/docs/database/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-db-approvers
/model/database/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-db-approvers
/docs/non-normative/db-migration.md @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-db-approvers

# Messaging semantic conventions
/docs/messaging/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-messaging-approvers
Expand All @@ -107,7 +110,8 @@
/model/feature-flags/ @open-telemetry/specs-semconv-approvers @open-telemetry/semconv-feature-flag-approvers

# Tooling
/policies/ @open-telemetry/specs-semconv-approvers @open-telemetry/weaver-maintainers
/policies_test/ @open-telemetry/specs-semconv-approvers @open-telemetry/weaver-maintainers
/templates/ @open-telemetry/specs-semconv-approvers @open-telemetry/weaver-maintainers
/internal/ @open-telemetry/specs-semconv-approvers @open-telemetry/weaver-maintainers
/policies/ @open-telemetry/specs-semconv-approvers @open-telemetry/weaver-maintainers
/policies_test/ @open-telemetry/specs-semconv-approvers @open-telemetry/weaver-maintainers
/templates/ @open-telemetry/specs-semconv-approvers @open-telemetry/weaver-maintainers
/internal/ @open-telemetry/specs-semconv-approvers @open-telemetry/weaver-maintainers
/docs/non-normative/code-generation.md @open-telemetry/specs-semconv-approvers @open-telemetry/weaver-maintainers
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ body:
- area:process
- area:profile
- area:rpc
- area:security-rule
- area:server
- area:service
- area:session
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/change_proposal.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ body:
- area:process
- area:profile
- area:rpc
- area:security-rule
- area:server
- area:service
- area:session
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/new-conventions.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ body:
- area:process
- area:profile
- area:rpc
- area:security-rule
- area:server
- area:service
- area:session
Expand Down
2 changes: 1 addition & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
"MD040": false,
},
"yaml.schemas": {
"https://raw.githubusercontent.com/open-telemetry/weaver/v0.9.2/schemas/semconv.schema.json": [
"https://raw.githubusercontent.com/open-telemetry/weaver/v0.11.0/schemas/semconv.schema.json": [
"model/**/*.yaml"
]
},
Expand Down
2 changes: 1 addition & 1 deletion dependencies.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# Dependabot can keep this file up to date with latest containers.

# Weaver is used to generate markdown docs, and enforce policies on the model.
FROM otel/weaver:v0.10.0 AS weaver
FROM otel/weaver:v0.11.0 AS weaver

# OPA is used to test policies enforced by weaver.
FROM openpolicyagent/opa:0.70.0 AS opa
Expand Down
1 change: 1 addition & 0 deletions docs/attributes-registry/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ Currently, the following namespaces exist:
- [Process](process.md)
- [Profile](profile.md)
- [RPC](rpc.md)
- [Security Rule](security-rule.md)
- [Server](server.md)
- [Service](service.md)
- [Session](session.md)
Expand Down
24 changes: 24 additions & 0 deletions docs/attributes-registry/security-rule.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
<!--- Hugo front matter used to generate the website version of this page:
--->

<!-- NOTE: THIS FILE IS AUTOGENERATED. DO NOT EDIT BY HAND. -->
<!-- see templates/registry/markdown/attribute_namespace.md.j2 -->

# Security Rule

## Security Rule

Describes security rule attributes. Rule fields are used to capture the specifics of any observer or agent rules that generate alerts or other notable events.

| Attribute | Type | Description | Examples | Stability |
|---|---|---|---|---|
| <a id="security-rule-category" href="#security-rule-category">`security_rule.category`</a> | string | A categorization value keyword used by the entity using the rule for detection of this event | `Attempted Information Leak` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="security-rule-description" href="#security-rule-description">`security_rule.description`</a> | string | The description of the rule generating the event. | `Block requests to public DNS over HTTPS / TLS protocols` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="security-rule-license" href="#security-rule-license">`security_rule.license`</a> | string | Name of the license under which the rule used to generate this event is made available. | `Apache 2.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="security-rule-name" href="#security-rule-name">`security_rule.name`</a> | string | The name of the rule or signature generating the event. | `BLOCK_DNS_over_TLS` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="security-rule-reference" href="#security-rule-reference">`security_rule.reference`</a> | string | Reference URL to additional information about the rule used to generate this event. [1] | `https://en.wikipedia.org/wiki/DNS_over_TLS` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="security-rule-ruleset-name" href="#security-rule-ruleset-name">`security_rule.ruleset.name`</a> | string | Name of the ruleset, policy, group, or parent category in which the rule used to generate this event is a member. | `Standard_Protocol_Filters` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="security-rule-uuid" href="#security-rule-uuid">`security_rule.uuid`</a> | string | A rule ID that is unique within the scope of a set or group of agents, observers, or other entities using the rule for detection of this event. | `550e8400-e29b-41d4-a716-446655440000`; `1100110011` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="security-rule-version" href="#security-rule-version">`security_rule.version`</a> | string | The version / revision of the rule being used for analysis. | `1.0.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |

**[1] `security_rule.reference`:** The URL can point to the vendor’s documentation about the rule. If that’s not available, it can also be a link to a more general page describing this type of alert.
68 changes: 68 additions & 0 deletions docs/non-normative/k8s-migration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
<!--- Hugo front matter used to generate the website version of this page:
linkTitle: K8s migration
--->

# K8s semantic conventions stability migration

Due to the significant number of modifications and the extensive user base
affected by them, existing K8s instrumentations published by
OpenTelemetry are required to implement a migration plan that will assist users in
transitioning to the stable K8s semantic conventions.

When existing K8s instrumentations published by OpenTelemetry are
updated to the stable K8s semantic conventions, they:

- SHOULD introduce an environment variable `OTEL_SEMCONV_STABILITY_OPT_IN` in
their existing major version, which accepts:
- `k8s` - emit the stable k8s conventions, and stop emitting
the old k8s conventions that the instrumentation emitted previously.
- `k8s/dup` - emit both the old and the stable k8s conventions,
allowing for a phased rollout of the stable semantic conventions.
- The default behavior (in the absence of one of these values) is to continue
emitting whatever version of the old k8s conventions the
instrumentation was emitting previously.
- Need to maintain (security patching at a minimum) their existing major version
for at least six months after it starts emitting both sets of conventions.
- May drop the environment variable in their next major version and emit only
the stable k8s conventions.

Specifically for the Opentelemetry Collector:

The transition will happen through two different feature gates.
One for enabling the new schema called `semconv.k8s.enableStable`,
and one for disabling the old schema called `semconv.k8s.disableLegacy`. Then:

- On alpha the old schema is enabled by default (`semconv.k8s.disableLegacy` defaults to false),
while the new schema is disabled by default (`semconv.k8s.enableStable` defaults to false).
- On beta/stable the old schema is disabled by default (`semconv.k8s.disableLegacy` defaults to true),
while the new is enabled by default (`semconv.k8s.enableStable` defaults to true).
- It is an error to disable both schemas
- Both schemas can be enabled with `--feature-gates=-semconv.k8s.disableLegacy,+semconv.k8s.enableStable`.

<!-- toc -->

- [Summary of changes](#summary-of-changes)
- [K8s network metrics](#k8s-network-metrics)

<!-- tocstop -->

## Summary of changes

This section summarizes the changes made to the K8s semantic conventions
from a range of versions. Each starting version shows all the changes required
to bring the conventions to stable (TODO: link to specific version once it exists).

### K8s network metrics

The K8s network metrics implemented by the Collector and specifically the
[kubeletstats](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.112.0/receiver/kubeletstatsreceiver/documentation.md)
receiver were introduced as semantic conventions in [v1.29.0](https://github.com/open-telemetry/semantic-conventions/blob/v1.29.0/docs/system/k8s-metrics.md).

The changes in their attributes are the following:

<!-- prettier-ignore-start -->
| Old (Collector) ![changed](https://img.shields.io/badge/changed-orange?style=flat) | New |
|------------------------------------------------------------------------------------|---------------------------|
| `interface` | `network.interface.name` |
| `direction` | `network.io.direction` |
<!-- prettier-ignore-end -->
1 change: 1 addition & 0 deletions model/android/resources.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
groups:
- id: resource.android
type: resource
stability: experimental
name: android
brief: >
The Android platform on which the Android application is running.
Expand Down
1 change: 1 addition & 0 deletions model/aws/ecs-resources.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
groups:
- id: resource.aws.ecs
type: resource
stability: experimental
name: aws.ecs
brief: >
Resources used by AWS Elastic Container Service (ECS).
Expand Down
1 change: 1 addition & 0 deletions model/aws/eks-resources.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
groups:
- id: resource.aws.eks
type: resource
stability: experimental
name: aws.eks
brief: >
Resources used by AWS Elastic Kubernetes Service (EKS).
Expand Down
1 change: 1 addition & 0 deletions model/aws/lambda-spans.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
groups:
- id: span.aws.lambda
type: span
stability: experimental
brief: >
Span attributes used by AWS Lambda (in addition to general `faas` attributes).
attributes:
Expand Down
1 change: 1 addition & 0 deletions model/aws/logs-resources.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
groups:
- id: resource.aws.log
type: resource
stability: experimental
name: aws.log
brief: >
Resources specific to Amazon Web Services.
Expand Down
Loading

0 comments on commit e90cbaa

Please sign in to comment.