Skip to content

Commit

Permalink
feat: add github actions receiver temporarily (#542)
Browse files Browse the repository at this point in the history
* feat: add github actions receiver temporarily

* chore: fix up location of mdatagen
  • Loading branch information
adrielp authored Sep 18, 2024
1 parent 5b586b1 commit 915f0d7
Show file tree
Hide file tree
Showing 27 changed files with 2,877 additions and 0 deletions.
2 changes: 2 additions & 0 deletions config/manifest.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ receivers:
- gomod: go.opentelemetry.io/collector/receiver/otlpreceiver v0.109.0
- gomod: github.com/liatrio/liatrio-otel-collector/receiver/gitlabreceiver v0.1.0
- gomod: github.com/liatrio/liatrio-otel-collector/receiver/githubreceiver v0.1.0
- gomod: github.com/liatrio/liatrio-otel-collector/receiver/githubactionsreceiver v0.1.0
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.109.0
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/kubeletstatsreceiver v0.109.0
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/k8sclusterreceiver v0.109.0
Expand All @@ -69,4 +70,5 @@ providers:
replaces:
- github.com/liatrio/liatrio-otel-collector/receiver/gitlabreceiver => ../receiver/gitlabreceiver/
- github.com/liatrio/liatrio-otel-collector/receiver/githubreceiver => ../receiver/githubreceiver/
- github.com/liatrio/liatrio-otel-collector/receiver/githubactionsreceiver => ../receiver/githubactionsreceiver/
- github.com/liatrio/liatrio-otel-collector/extension/githubappauthextension => ../extension/githubappauthextension/
2 changes: 2 additions & 0 deletions receiver/githubactionsreceiver/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
include ../../Makefile.Common

234 changes: 234 additions & 0 deletions receiver/githubactionsreceiver/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,234 @@
# GitHub Actions Receiver

<!-- status autogenerated section -->
| Status | |
| ------------- |-----------|
| Stability | [development]: traces |
| Distributions | [] |
| Issues | [![Open issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aopen%20label%3Areceiver%2Fgithubactions%20&label=open&color=orange&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Areceiver%2Fgithubactions) [![Closed issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aclosed%20label%3Areceiver%2Fgithubactions%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Areceiver%2Fgithubactions) |
| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner) | |

[development]: https://github.com/open-telemetry/opentelemetry-collector#development
<!-- end autogenerated section -->

The GitHub Actions Receiver processes GitHub Actions webhook events to observe
workflows and jobs. It handles
[`workflow_job`](https://docs.github.com/en/webhooks/webhook-events-and-payloads#workflow_job)
and
[`workflow_run`](https://docs.github.com/en/webhooks/webhook-events-and-payloads#workflow_run)
event payloads, transforming them into `trace` telemetry.

Each GitHub Action workflow or job, along with its steps, are converted into
trace spans, allowing the observation of workflow execution times, success, and
failure rates.

If a secret is configured (recommended), it [validates the
payload](https://docs.github.com/en/webhooks/using-webhooks/validating-webhook-deliveries)
ensuring data integrity before processing.

The receiver supports linking spans to previous runs for `workflow_run` events,
enhancing traceability across workflow attempts. This feature utilises
deterministic Trace IDs generated based on the run ID and run attempt. When a
`workflow_run` event contains a `PreviousAttemptURL`, and the run attempt is
greater than `1`, the receiver automatically links the current run's root span
to the previous run's Trace ID, providing a direct connection between
sequential workflow attempts.

## Configuration

The following settings are required:

* `endpoint` (no default): The endpoint where you may point your webhook to emit events to

The following settings are optional:

* `path` (default: '/events'): Path where the receiver instance will accept events
* `secret`: GitHub webhook hash signature

Example:
```yaml
receivers:
githubactions:
endpoint: localhost:19418
path: /events
secret: It's a Secret to Everybody
```
The full list of settings exposed for this receiver are documented [here](./config.go) with a detailed sample configuration [here](./testdata/config.yaml)
## Advanced Configuration
Several helper files are leveraged to provide additional capabilities automatically:
- [HTTP server settings](https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/confighttp/README.md#server-configuration) including CORS
- [TLS and mTLS settings](https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configtls/README.md)
### Service Name Generation
The GitHub Actions Receiver dynamically generates the `service.name` attribute,
which is crucial for identifying the source of telemetry data in observability
platforms. This name helps in categorising and filtering traces originating
from different GitHub repositories or actions workflows.

#### Default Service Name

By default, the service name is derived from the **GitHub repository's full
name** (e.g., org/repo-name) provided in the payload. This name is then
formatted to be all lowercase and to have slashes (/) and underscores (_)
replaced with dashes (-), ensuring a clean, URL-friendly identifier.

#### Customisation Options

While the default naming convention should suffice for most use cases, there
might be scenarios where a more descriptive or specific service name is
required. For such cases, the receiver's configuration provides options for
customization:

* **Custom Service Name**: A completely custom service.name that overrides the
default naming convention. This is useful when you want to assign a specific,
recognisable name to the telemetry data from a particular GitHub repository
or workflow.

* **Service Name Prefix and Suffix**: These allow for the addition of prefixes
or suffixes to the automatically generated service name, offering a balance
between automatic naming and customization.

Here's an example snippet from a configuration file (config.yaml) demonstrating
how to set these options:

```yaml
receivers:
githubactionsreceiver:
custom_service_name: "github.aaakk.us.kg-myorg" # Completely overrides the default service name
service_name_prefix: "foo-" # Prepended to the default service name (ignored if custom_service_name is set)
service_name_suffix: "-bar" # Appended to the default service name (ignored if custom_service_name is set)
```

## GitHub repository webhooks

Webhooks provide a way for notifications to be delivered to an external web
server whenever certain events occur on GitHub.

### Delivery headers

HTTP POST payloads that are delivered to your webhook's configured URL endpoint
will contain several special headers:

* `X-Hub-Signature`: This header is sent if the webhook is configured with a
`secret`. This is the HMAC hex digest of the request body, and is generated
using the SHA-1 hash function and the `secret` as the HMAC `key`.
`X-Hub-Signature` is provided for compatibility with existing integrations.
It's recommended that you use the more secure `X-Hub-Signature-256` instead.
* `X-Hub-Signature-256`: This header is sent if the webhook is configured with
a `secret` (recommended). This is the HMAC hex digest of the request body,
and is generated using the SHA-256 hash function and the `secret` as the HMAC
`key`.
* `User-Agent`: This header will always have the prefix `GitHub-Hookshot/`.

### Creating webhooks

You can create webhooks to subscribe to specific events that occur on GitHub.

#### Creating a respositoty webhook

You can create a webhook to subscribe to events that occur in a specific
repository. You must be a repository owner or have admin access in the
repository to create webhooks in that repository.

1. On **GitHub.com**, navigate to the main page of the repository.
2. Under your repository name, click **Settings**. If you cannot see the
"Settings" tab, select the dropdown menu, then click Settings.
3. In the left sidebar, click **Webhooks**.
4. Click Add **webhook**.
5. Under "Payload URL", type the URL where you'd like to receive payloads.
6. Select the **Content type** drop-down menu, and click a data format to
receive the webhook payload in.
* Select `application/json`, which will deliver the JSON payload directly
as the body of the `POST` request.
7. Optionally (recomended), under "Secret", type a string to use as a secret
key. You should choose a random string of text with high entropy. You can use
the webhook secret to limit incoming requests to only those originating from
GitHub.
8. Under "Which events would you like to trigger this webhook?"
* Select **Let me select individual events**.
* Uncheck **Pushes**, which is selected by default.
* Select **Workflow jobs**.
* Select **Workflow runs**.
9. To make the webhook active immediately after adding the configuration,
select **Active**.
10. Click **Add webhook**.

You can use the GitHub web interface or the REST API to create a repository
webhook. For more information about using the REST API to create a repository
webhook, see [Repository
Webhooks](https://docs.github.com/en/rest/webhooks/repos?apiVersion=2022-11-28#create-a-repository-webhook).

Alternatively you can use a configuration management tool such as
[`terraform`](https://www.terraform.io/) with the
[`github_repository_webhook`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_webhook.html)
resource, to configure webhooks.


### Deterministic IDs

The GitHub Actions Receiver generates deterministic IDs to ensure traceability
and consistency across emitted spans. Here’s how the IDs are generated:

* **Trace ID**: Generated based on the run ID and run attempt, with a 't'
appended to ensure uniqueness across workflow runs and to distinguish it as a
trace ID.
* **Parent Span ID**: Derived from the workflow job ID and run attempt, with an
's' appended to distinguish it as a span ID and allow association of all
steps under a job.
* **Span ID**: Specifically generated for each step within a job, using the job
ID, run attempt, step name, and an optional step number, to ensure each step
within a job can be uniquely identified.

These IDs allow for the correlation of telemetry data within the observability
platform, enabling users to link their own spans to those emitted by the
receiver.

#### Generating IDs

Below are example functions in a couple of langues for generating each ID,
replicating the logic used by the receiver:

##### Generating IDs in `bash`

```bash
generate_trace_id() {
local run_id=$1
local run_attempt=$2
echo -n "${run_id}${run_attempt}t" | openssl dgst -sha256 | sed 's/^.* //' | cut -c-32
}
generate_job_span_id() {
local run_id=$1
local run_attempt=$2
local job_name=$3
echo -n "${run_id}${run_attempt}${job_name}" | openssl dgst -sha256 | sed 's/^.* //' | cut -c-16
}
generate_step_span_id() {
local run_id=$1
local run_attempt=$2
local job_name=$3
local step_name=$4
input="${run_id}${run_attempt}${job_name}${step_name}"
echo -n "$input" | openssl dgst -sha256 | sed 's/^.* //' | cut -c-16
}
# https://docs.github.com/en/actions/learn-github-actions/variables
trace_id=$(generate_trace_id ${GITHUB_RUN_ID} ${GITHUB_RUN_ATTEMPT})
job_span_id=$(generate_job_span_id ${GITHUB_RUN_ID} ${GITHUB_RUN_ATTEMPT} ${GITHUB_JOB})
step_span_id=$(generate_step_span_id ${GITHUB_RUN_ID} ${GITHUB_RUN_ATTEMPT} ${GITHUB_JOB} "your-step-name")
echo "Trace ID: ${trace_id}"
echo "Job Span ID: ${parent_span_id}"
echo "Step Span ID: ${span_id:0:16}"
```

##### Generating IDs in Other Languages

You can adapt the logic shown in the bash examples to any programming language
that supports SHA-256 hashing. The key is to ensure the input string format
matches between your code and the receiver for consistent ID generation.
38 changes: 38 additions & 0 deletions receiver/githubactionsreceiver/config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package githubactionsreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/githubactionsreceiver"

import (
"errors"

"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/config/confighttp"

"go.uber.org/multierr"
)

var errMissingEndpointFromConfig = errors.New("missing receiver server endpoint from config")

// Config defines configuration for GitHub Actions receiver
type Config struct {
confighttp.ServerConfig `mapstructure:",squash"` // squash ensures fields are correctly decoded in embedded struct
Path string `mapstructure:"path"` // path for data collection. Default is <host>:<port>/events
Secret string `mapstructure:"secret"` // github webhook hash signature. Default is empty
CustomServiceName string `mapstructure:"custom_service_name"` // custom service name. Default is empty
ServiceNamePrefix string `mapstructure:"service_name_prefix"` // service name prefix. Default is empty
ServiceNameSuffix string `mapstructure:"service_name_suffix"` // service name suffix. Default is empty
}

var _ component.Config = (*Config)(nil)

// Validate checks the receiver configuration is valid
func (cfg *Config) Validate() error {
var errs error

if cfg.Endpoint == "" {
errs = multierr.Append(errs, errMissingEndpointFromConfig)
}

return errs
}
55 changes: 55 additions & 0 deletions receiver/githubactionsreceiver/config_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package githubactionsreceiver

import (
"testing"

"github.com/stretchr/testify/require"
"go.opentelemetry.io/collector/config/confighttp"
)

// only one validate check so far
func TestValidateConfig(t *testing.T) {
t.Parallel()

tests := []struct {
desc string
expect error
conf Config
}{
{
desc: "Missing valid endpoint",
expect: errMissingEndpointFromConfig,
conf: Config{
ServerConfig: confighttp.ServerConfig{
Endpoint: "",
},
},
},
{
desc: "Valid Secret",
expect: nil,
conf: Config{
ServerConfig: confighttp.ServerConfig{
Endpoint: "localhost:8080",
},
Secret: "mysecret",
},
},
}

for _, test := range tests {
t.Run(test.desc, func(t *testing.T) {
err := test.conf.Validate()
if test.expect == nil {
require.NoError(t, err)
require.Equal(t, "mysecret", test.conf.Secret)
} else {
require.Error(t, err)
require.Contains(t, err.Error(), test.expect.Error())
}
})
}
}
6 changes: 6 additions & 0 deletions receiver/githubactionsreceiver/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

//go:generate ../../.tools/mdatagen metadata.yaml

package githubactionsreceiver // import "github.com/liatrio/liatrio-otel-collector/receiver/githubactionsreceiver"
Loading

0 comments on commit 915f0d7

Please sign in to comment.