feat: support AWS EventBridge

[npalm]

Description

This PR introduces the AWS EventBridge. The EventBridge can be enabled with the options webhook_mode, which can be set to either direct or eventbridge. In the direct mode the olds way of handling is still applied. When setting the mode to eventbridge events will publshed on the AWS EventBridge, which is not limited only to the event workflow_job with statues queued via a target rule events relevant for scaling a snet to the dispatcher lambda to distrute to a SQS queue for sacling.

Todo

• Refactor lambda and add EventBridge
• Refactor webhook module (TF) to support EventBridge
• Test example default
• Test example multi runner
• Adjust docs
• Reduce permissions on webhook and dispatcher lambda for eventbridge mode
• Add configuration for allowed events on the EventBridge
• Add support for CMK (encruption) to EventBridge Add support for CMK (encryption) for the EventBridge #4192

MIgration directions

The change is backwards compatible but will recreate resources managed by the internal module webhook. The only resource contianing data is the CloudWatch LogGroup. To retain the log geroup you can run a terraform state move. Or add a moved block to your deployemnt.

Migrating to this version

With module defaults or eventbridge is not eanavbled

# log group
moved {
   from = module.<runner-module-name>.module.webhook.aws_cloudwatch_log_group.webhook
   to = module.<runner-module-name>.module.webhook.module.direct[0].aws_cloudwatch_log_group.webhook
}

# lambda
moved {
   from = module.<runner-module-name>.module.webhook.aws_lambda_function.webhook
   to = module.<runner-module-name>.module.webhook.module.direct[0].aws_lambda_function.webhook
}

Or with webhook_mode = eventbridge

# log group
moved {
   from = module.<runner-module-name>.module.webhook.aws_cloudwatch_log_group.webhook
   to = module.<runner-module-name>.module.webhook.module.direct[0].aws_cloudwatch_log_group.webhook
}

# lambda
moved {
   from = module.<runner-module-name>.module.webhook.aws_lambda_function.webhook
   to = module.<runner-module-name>.module.webhook.module.direct[0].aws_lambda_function.webhook
}

When switching between direct and eventbridge

When enable mode eventbridge

# log group
moved {
  from = module.runners.module.webhook.module.direct[0].aws_cloudwatch_log_group.webhook
  to = module.runners.module.webhook.module.eventbridge[0].aws_cloudwatch_log_group.webhook
}

# lambda
moved {
  from = module.runners.module.webhook.module.direct[0].aws_lambda_function.webhook
  to = module.runners.module.webhook.module.eventbridge[0].aws_lambda_function.webhook
}

Or vice versa for moving from eventbride to webhook

[ShivaSDP]

@npalm

Test Results for 'default' and 'multi-runner' example deployment:

I have followed the steps below to verify the functionality of both the 'default' and 'multi-runner' examples.:

• Deployed both the default and multi-runner configurations using the latest code from the main branch (testing one at a time)

• Ran a workflow job to ensure the initial setup was working fine.

• Checked out the PR changes and proceeded with the following steps:
  * Rebuilt all the lambdas by running yarn dist.
  * Applied Terraform on the default example first and ran a test build successfully.
  * Enabled EventBridge mode and applied the Terraform changes again.
  * Verified that the sample build workflow executed successfully, confirming new instances were created and used correctly.

Repeated the above steps for the multi-runner configuration.

I also verified that the EC2 instances shut down automatically after job completion, in line with the expected behavior for ephemeral (direct) configurations.

github is van het padje