Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: order label matchers for multi-runners #3591

Merged
merged 7 commits into from
Nov 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ matcherConfig:
exactMatch: false
labelMatchers:
- [ self-hosted, linux, x64, amazon ]
priority: 1 # set ephemeral runner priority to 1
fifo: true
delay_webhook_event: 0
runner_config:
Expand Down
3 changes: 2 additions & 1 deletion modules/multi-runner/README.md

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions modules/multi-runner/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ variable "multi_runner_config" {
matcherConfig = object({
labelMatchers = list(list(string))
exactMatch = optional(bool, false)
priority = optional(number, 999)
})
fifo = optional(bool, false)
redrive_build_queue = optional(object({
Expand Down Expand Up @@ -182,6 +183,7 @@ variable "multi_runner_config" {
matcherConfig: {
labelMatchers: "The list of list of labels supported by the runner configuration. `[[self-hosted, linux, x64, example]]`"
exactMatch: "If set to true all labels in the workflow job must match the GitHub labels (os, architecture and `self-hosted`). When false if __any__ workflow label matches it will trigger the webhook."
priority: "If set it defines the priority of the matcher, the matcher with the lowest priority will be evaluated first. Default is 999, allowed values 0-999."
}
fifo: "Enable a FIFO queue to remain the order of events received by the webhook. Suggest to set to true for repo level runners."
redrive_build_queue: "Set options to attach (optional) a dead letter queue to the build queue, the queue between the webhook and the scale up lambda. You have the following options. 1. Disable by setting `enabled` to false. 2. Enable by setting `enabled` to `true`, `maxReceiveCount` to a number of max retries."
Expand Down
2 changes: 1 addition & 1 deletion modules/webhook/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ No modules.
| <a name="input_repository_white_list"></a> [repository\_white\_list](#input\_repository\_white\_list) | List of github repository full names (owner/repo\_name) that will be allowed to use the github app. Leave empty for no filtering. | `list(string)` | `[]` | no |
| <a name="input_role_path"></a> [role\_path](#input\_role\_path) | The path that will be added to the role; if not set, the environment name will be used. | `string` | `null` | no |
| <a name="input_role_permissions_boundary"></a> [role\_permissions\_boundary](#input\_role\_permissions\_boundary) | Permissions boundary that will be added to the created role for the lambda. | `string` | `null` | no |
| <a name="input_runner_config"></a> [runner\_config](#input\_runner\_config) | SQS queue to publish accepted build events based on the runner type. When exact match is disabled the webhook accecpts the event if one of the workflow job labels is part of the matcher. | <pre>map(object({<br> arn = string<br> id = string<br> fifo = bool<br> matcherConfig = object({<br> labelMatchers = list(list(string))<br> exactMatch = bool<br> })<br> }))</pre> | n/a | yes |
| <a name="input_runner_config"></a> [runner\_config](#input\_runner\_config) | SQS queue to publish accepted build events based on the runner type. When exact match is disabled the webhook accecpts the event if one of the workflow job labels is part of the matcher. The priority defines the order the matchers are applied. | <pre>map(object({<br> arn = string<br> id = string<br> fifo = bool<br> matcherConfig = object({<br> labelMatchers = list(list(string))<br> exactMatch = bool<br> priority = optional(number, 999)<br> })<br> }))</pre> | n/a | yes |
| <a name="input_sqs_workflow_job_queue"></a> [sqs\_workflow\_job\_queue](#input\_sqs\_workflow\_job\_queue) | SQS queue to monitor github events. | <pre>object({<br> id = string<br> arn = string<br> })</pre> | `null` | no |
| <a name="input_tags"></a> [tags](#input\_tags) | Map of tags that will be added to created resources. By default resources will be tagged with name and environment. | `map(string)` | `{}` | no |
| <a name="input_webhook_lambda_apigateway_access_log_settings"></a> [webhook\_lambda\_apigateway\_access\_log\_settings](#input\_webhook\_lambda\_apigateway\_access\_log\_settings) | Access log settings for webhook API gateway. | <pre>object({<br> destination_arn = string<br> format = string<br> })</pre> | `null` | no |
Expand Down
8 changes: 7 additions & 1 deletion modules/webhook/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -34,17 +34,23 @@ variable "tags" {
}

variable "runner_config" {
description = "SQS queue to publish accepted build events based on the runner type. When exact match is disabled the webhook accecpts the event if one of the workflow job labels is part of the matcher."
description = "SQS queue to publish accepted build events based on the runner type. When exact match is disabled the webhook accecpts the event if one of the workflow job labels is part of the matcher. The priority defines the order the matchers are applied."
type = map(object({
arn = string
id = string
fifo = bool
matcherConfig = object({
labelMatchers = list(list(string))
exactMatch = bool
priority = optional(number, 999)
})
}))
validation {
condition = try(var.runner_config.matcherConfig.priority, 999) >= 0 && try(var.runner_config.matcherConfig.priority, 999) < 1000
error_message = "The priority of the matcher must be between 0 and 999."
}
}

variable "sqs_workflow_job_queue" {
description = "SQS queue to monitor github events."
type = object({
Expand Down
10 changes: 9 additions & 1 deletion modules/webhook/webhook.tf
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
locals {
# config with combined key and order
runner_config = { for k, v in var.runner_config : format("%03d-%s", v.matcherConfig.priority, k) => merge(v, { key = k }) }

# sorted list
runner_config_sorted = [for k in sort(keys(local.runner_config)) : local.runner_config[k]]
}

resource "aws_lambda_function" "webhook" {
s3_bucket = var.lambda_s3_bucket != null ? var.lambda_s3_bucket : null
s3_key = var.webhook_lambda_s3_key != null ? var.webhook_lambda_s3_key : null
Expand All @@ -18,7 +26,7 @@ resource "aws_lambda_function" "webhook" {
POWERTOOLS_LOGGER_LOG_EVENT = var.log_level == "debug" ? "true" : "false"
PARAMETER_GITHUB_APP_WEBHOOK_SECRET = var.github_app_parameters.webhook_secret.name
REPOSITORY_WHITE_LIST = jsonencode(var.repository_white_list)
RUNNER_CONFIG = jsonencode([for k, v in var.runner_config : v])
RUNNER_CONFIG = jsonencode(local.runner_config_sorted)
SQS_WORKFLOW_JOB_QUEUE = try(var.sqs_workflow_job_queue, null) != null ? var.sqs_workflow_job_queue.id : ""
}
}
Expand Down
Loading