Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Module components] Add provides receivers module #15

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

qclaogui
Copy link

@qclaogui qclaogui commented Apr 17, 2024

What this PR does

This PR comes from receivers provider, a module provides receivers components for collecting data(logs metrics traces profiles).

An simple use case config.alloy like so:

import.git "provider" {
  repository     = "https://github.com/qclaogui/codelab-monitoring.git"
  revision       = "main"
  path           = "alloy-modules/provider"
  pull_frequency = "24h"
}

// get the receivers from provider
provider.self_hosted_stack "compose" {
  metrics_endpoint_url  = "http://mimir:8080/api/v1/push"
}

// scrape metrics and write to metric receiver
prometheus.scrape "default" {
  targets = [
    {"__address__" = "127.0.0.1:12345"},
  ]

  forward_to = [
    provider.self_hosted_stack.compose.metrics_receiver,
  ]
}

Here is an example of my use of all-in-one.alloy in a docker compose scenario:

logging {
	level  = coalesce(env("ALLOY_LOG_LEVEL"), "info")
	format = "logfmt"
}

/********************************************
 * Grafana LGTMP Stack Receiver Provider
 ********************************************/
import.git "provider" {
	repository     = "https://github.com/qclaogui/codelab-monitoring.git"
	revision       = "main"
	path           = "alloy-modules/provider"
	pull_frequency = "24h"
}

+provider.self_hosted_stack "compose" {
+	metrics_endpoint_url  = coalesce(env("SELF_HOSTED_METRICS_ENDPOINT_URL"), "http://gateway:8080/api/v1/push")
+	logs_endpoint_url     = coalesce(env("SELF_HOSTED_LOGS_ENDPOINT_URL"), "http://gateway:3100/loki/api/v1/push")
+	traces_endpoint_url   = coalesce(env("SELF_HOSTED_TRACES_ENDPOINT_URL"), "http://gateway:4318")
+	profiles_endpoint_url = coalesce(env("SELF_HOSTED_PROFILES_ENDPOINT_URL"), "http://gateway:4040")
+}

/********************************************
 * Metrics
 ********************************************/
import.file "metrics" {
	filename = coalesce(env("ALLOY_MODULES_FOLDER"), "/etc/alloy/modules") + "/compose/metrics"
}

metrics.labels_scrape "compose" {
	label_prefix    = "metrics.grafana.com"
+	forward_to      = [provider.self_hosted_stack.compose.metrics_receiver]
	scrape_interval = "15s"
}

/********************************************
 * Logs
 ********************************************/
import.file "logs" {
	filename = coalesce(env("ALLOY_MODULES_FOLDER"), "/etc/alloy/modules") + "/compose/logs"
}

logs.labels_scrape "compose" {
	label_prefix = "logs.grafana.com"
	forward_to   = [logs.keep_labels.compose.receiver]
}

// logs Processing And Transformation
logs.keep_labels "compose" {
+	forward_to = [provider.self_hosted_stack.compose.logs_receiver]
}

/********************************************
 * Traces
 ********************************************/
import.file "traces" {
	filename = coalesce(env("ALLOY_MODULES_FOLDER"), "/etc/alloy/modules") + "/compose/traces"
}

// traces Processing And Transformation process_and_transform
traces.process_and_transform "compose" {
+	metrics_forward_to = [provider.self_hosted_stack.compose.metrics_receiver]
+	logs_forward_to    = [provider.self_hosted_stack.compose.logs_receiver]
+	traces_forward_to  = [provider.self_hosted_stack.compose.traces_receiver]
}

tracing {
	// Write all spans. Don't do this in production!
	sampling_fraction = 1

	// Forward Alloy internal spans to traces process.
	write_to = [traces.process_and_transform.compose.alloy_traces_input]
}

/********************************************
 * Profiles
 ********************************************/
import.file "profiles" {
	filename = coalesce(env("ALLOY_MODULES_FOLDER"), "/etc/alloy/modules") + "/compose/profiles"
}

profiles.labels_scrape "compose" {
	label_prefix = "profiles.grafana.com"
+	forward_to   = [provider.self_hosted_stack.compose.profiles_receiver]
}

Which issue(s) this PR fixes or relates to

#12
In addition, I also add the grafana_cloud component in the receivers provider moudle

@bentonam
Copy link
Collaborator

bentonam commented May 2, 2024

@qclaogui thank you for the contribution. Would you mind ensuring that the CLA is signed, all checks should pass, it is currently failing the markdownlint checks. Also please ensure the branch is up-to-date. Thanks!

@CLAassistant
Copy link

CLAassistant commented May 3, 2024

CLA assistant check
All committers have signed the CLA.

@qclaogui
Copy link
Author

qclaogui commented May 4, 2024

it is currently failing the markdownlint checks. Also please ensure the branch is up-to-date.

Hi @bentonam, Thank you for taking the time to reviews, I have run make lint-markdown and it has passed the markdownlint check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants