feat: add support for matcher config tiering options (#3953)

This feature is useful as a workaround when you have a larger number of runner configurations and run into problems where the parameter store limit is being reached (4KB). The limit of Standard is 4KB and Advanced is 8KB. ⚠️ **It should be noted that if you changed the parameter store tier for the matcher config manually, this change might override your changes!** Signed-off-by: Brend Smits <[email protected]> --------- Signed-off-by: Brend Smits <[email protected]> Co-authored-by: forest-pr|bot <forest-pr[bot]@users.noreply.github.com>
philips-labs · Jun 28, 2024 · 5f9d9eb · 5f9d9eb
1 parent ef25bd4
commit 5f9d9eb
Show file tree

Hide file tree

Showing 9 changed files with 37 additions and 2 deletions.
diff --git a/README.md b/README.md
@@ -179,6 +179,7 @@ Talk to the forestkeepers in the `runners-channel` on Slack.
 | <a name="input_log_level"></a> [log\_level](#input\_log\_level) | Logging level for lambda logging. Valid values are  'silly', 'trace', 'debug', 'info', 'warn', 'error', 'fatal'. | `string` | `"info"` | no |
 | <a name="input_logging_kms_key_id"></a> [logging\_kms\_key\_id](#input\_logging\_kms\_key\_id) | Specifies the kms key id to encrypt the logs with. | `string` | `null` | no |
 | <a name="input_logging_retention_in_days"></a> [logging\_retention\_in\_days](#input\_logging\_retention\_in\_days) | Specifies the number of days you want to retain log events for the lambda log group. Possible values are: 0, 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, and 3653. | `number` | `180` | no |
+| <a name="input_matcher_config_parameter_store_tier"></a> [matcher\_config\_parameter\_store\_tier](#input\_matcher\_config\_parameter\_store\_tier) | The tier of the parameter store for the matcher configuration. Valid values are `Standard`, and `Advanced`. | `string` | `"Standard"` | no |
 | <a name="input_metrics_namespace"></a> [metrics\_namespace](#input\_metrics\_namespace) | The namespace for the metrics created by the module. Merics will only be created if explicit enabled. | `string` | `"GitHub Runners"` | no |
 | <a name="input_minimum_running_time_in_minutes"></a> [minimum\_running\_time\_in\_minutes](#input\_minimum\_running\_time\_in\_minutes) | The time an ec2 action runner should be running at minimum before terminated, if not busy. | `number` | `null` | no |
 | <a name="input_pool_config"></a> [pool\_config](#input\_pool\_config) | The configuration for updating the pool. The `pool_size` to adjust to by the events triggered by the `schedule_expression`. For example you can configure a cron expression for weekdays to adjust the pool to 10 and another expression for the weekend to adjust the pool to 1. | <pre>list(object({<br>    schedule_expression = string<br>    size                = number<br>  }))</pre> | `[]` | no |

diff --git a/main.tf b/main.tf
@@ -142,7 +142,8 @@ module "webhook" {
       }
     }
   }
-  sqs_workflow_job_queue = length(aws_sqs_queue.webhook_events_workflow_job_queue) > 0 ? aws_sqs_queue.webhook_events_workflow_job_queue[0] : null
+  matcher_config_parameter_store_tier = var.matcher_config_parameter_store_tier
+  sqs_workflow_job_queue              = length(aws_sqs_queue.webhook_events_workflow_job_queue) > 0 ? aws_sqs_queue.webhook_events_workflow_job_queue[0] : null
 
   github_app_parameters = {
     webhook_secret = module.ssm.parameters.github_app_webhook_secret

diff --git a/modules/multi-runner/README.md b/modules/multi-runner/README.md
diff --git a/modules/multi-runner/variables.tf b/modules/multi-runner/variables.tf
@@ -644,3 +644,13 @@ variable "instance_termination_watcher" {
   })
   default = {}
 }
+
+variable "matcher_config_parameter_store_tier" {
+  description = "The tier of the parameter store for the matcher configuration. Valid values are `Standard`, and `Advanced`."
+  type        = string
+  default     = "Standard"
+  validation {
+    condition     = contains(["Standard", "Advanced"], var.matcher_config_parameter_store_tier)
+    error_message = "`matcher_config_parameter_store_tier` value is not valid, valid values are: `Standard`, and `Advanced`."
+  }
+}
diff --git a/modules/multi-runner/webhook.tf b/modules/multi-runner/webhook.tf
@@ -4,7 +4,8 @@ module "webhook" {
   tags        = local.tags
   kms_key_arn = var.kms_key_arn
 
-  runner_matcher_config = local.runner_config
+  runner_matcher_config               = local.runner_config
+  matcher_config_parameter_store_tier = var.matcher_config_parameter_store_tier
   ssm_paths = {
     root    = local.ssm_root_path
     webhook = var.ssm_paths.webhook

diff --git a/modules/webhook/README.md b/modules/webhook/README.md
@@ -91,6 +91,7 @@ No modules.
 | <a name="input_log_level"></a> [log\_level](#input\_log\_level) | Logging level for lambda logging. Valid values are  'silly', 'trace', 'debug', 'info', 'warn', 'error', 'fatal'. | `string` | `"info"` | no |
 | <a name="input_logging_kms_key_id"></a> [logging\_kms\_key\_id](#input\_logging\_kms\_key\_id) | Specifies the kms key id to encrypt the logs with | `string` | `null` | no |
 | <a name="input_logging_retention_in_days"></a> [logging\_retention\_in\_days](#input\_logging\_retention\_in\_days) | Specifies the number of days you want to retain log events for the lambda log group. Possible values are: 0, 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, and 3653. | `number` | `180` | no |
+| <a name="input_matcher_config_parameter_store_tier"></a> [matcher\_config\_parameter\_store\_tier](#input\_matcher\_config\_parameter\_store\_tier) | The tier of the parameter store for the matcher configuration. Valid values are `Standard`, and `Advanced`. | `string` | `"Standard"` | no |
 | <a name="input_prefix"></a> [prefix](#input\_prefix) | The prefix used for naming resources | `string` | `"github-actions"` | no |
 | <a name="input_repository_white_list"></a> [repository\_white\_list](#input\_repository\_white\_list) | List of github repository full names (owner/repo\_name) that will be allowed to use the github app. Leave empty for no filtering. | `list(string)` | `[]` | no |
 | <a name="input_role_path"></a> [role\_path](#input\_role\_path) | The path that will be added to the role; if not set, the environment name will be used. | `string` | `null` | no |

diff --git a/modules/webhook/main.tf b/modules/webhook/main.tf
@@ -61,4 +61,5 @@ resource "aws_ssm_parameter" "runner_matcher_config" {
   name  = "${var.ssm_paths.root}/${var.ssm_paths.webhook}/runner-matcher-config"
   type  = "String"
   value = jsonencode(local.runner_matcher_config_sorted)
+  tier  = var.matcher_config_parameter_store_tier
 }
diff --git a/modules/webhook/variables.tf b/modules/webhook/variables.tf
@@ -194,3 +194,13 @@ variable "ssm_paths" {
     webhook = string
   })
 }
+
+variable "matcher_config_parameter_store_tier" {
+  description = "The tier of the parameter store for the matcher configuration. Valid values are `Standard`, and `Advanced`."
+  type        = string
+  default     = "Standard"
+  validation {
+    condition     = contains(["Standard", "Advanced"], var.matcher_config_parameter_store_tier)
+    error_message = "`matcher_config_parameter_store_tier` value is not valid, valid values are: `Standard`, and `Advanced`."
+  }
+}
diff --git a/variables.tf b/variables.tf
@@ -565,6 +565,15 @@ variable "enable_runner_workflow_job_labels_check_all" {
   default     = true
 }
 
+variable "matcher_config_parameter_store_tier" {
+  description = "The tier of the parameter store for the matcher configuration. Valid values are `Standard`, and `Advanced`."
+  type        = string
+  default     = "Standard"
+  validation {
+    condition     = contains(["Standard", "Advanced"], var.matcher_config_parameter_store_tier)
+    error_message = "`matcher_config_parameter_store_tier` value is not valid, valid values are: `Standard`, and `Advanced`."
+  }
+}
 variable "runner_ec2_tags" {
   description = "Map of tags that will be added to the launch template instance tag specifications."
   type        = map(string)