chore: Add support multi-runners #2517 #2567
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
npalm
force-pushed
the
next
branch
2 times, most recently
from
e03d903 to
c724f32
Compare
npalm
force-pushed
the
next
branch
3 times, most recently
from
083e881 to
8218c2a
Compare
npalm
force-pushed
the
next
branch
2 times, most recently
from
2c87ee9 to
24c19b7
Compare
Migration: In the past we used for scaling the check_run event, since GitHub had now support for a workflow_job event. Now the event is upport for quite some time. And also support exists on GHES. We will phase out check_run support. Ensure you configure the GitHub app to subscribe to workflow_job. In case you on an old GHES server version, the only. migration path is to update GHES or remain on old version of the module
Migration: Option 1, upgrade to 2.x could cause orphan instances which will not be terminated by the scale-down function, thoses instances needs to be removed manual. Option 2, upgrade first to a verions 1.x and keep this version running till all instances before the upgrade are terminated. Next upgrade to 2.x.
Migration: - Does the main module needs to be redeployed after this feature is released? There is no change in Terraform version for the main module, so it will not require TF upgrade. There is only one change in main module, which is a breaking change, the variable runner_enable_workflow_job_labels_check is not available under this module as we now only support the flagrunner_enable_workflow_job_labels_check_all to check all/partial workflow labels to be present in the runner labels. Running TF apply on existing deployment will apply some changes done in internal modules to existing infrastructure. - Impact on using internal (submodules). There are some changes in the webhook internal module as per this feature. So, If you have been using it directly, the changes will be towards accepting runner configuration (queue details and label matchers) as a single object instead of earlier configuration of scattered queue details and criteria to match the workflow labels against the runner labels. Co-authored-by: Niek Palm <[email protected]> Co-authored-by: Niek Palm <[email protected]> Co-authored-by: navdeepg2021 <[email protected]>
npalm
changed the title
* Bump min terraform version to 1.3.0 * Update terraform verify's per new min version
…odule. (#2570) * feat: Remove support check_run (#2521) * chore: Remove support check_run * format, lint * feat: Remove old scale down mechanism (< 0.19.0) (#2519) fix: Remove old cleanup mechanism (< 0.19.0) * chore: Enable pre releases for next branch * feat: Add multi-runner capability (#2472) * feat: Remove support check_run (#2521) * chore: Remove support check_run * format, lint * feat: Remove old scale down mechanism (< 0.19.0) (#2519) fix: Remove old cleanup mechanism (< 0.19.0) * feat: added changes for multi runner. * fix: region. * fix: more fixes. * tuple to list. * fixes. * fixes. * fixes. * fixes. * fixes. * fixes. * fix: formatting. * fix: formatting. * fix: formatting. * fix: moved some blocks outside runner config. * fix: few more updates * fix: liniting. * fix: updated example output * changed runner group name. * fix: updated the tests. * fix: addressed review comments. * fix: linting issues. * fix: formatting. * fix: updated tf version. * fix: Remove removed prerelease option * Add ubuntu runner to example * refactor: use each instead of count * fix: few small issues. * refactor: syncer to count for multi runner * fix: comments. * fix: added Readme. * fix: errors. * move variable to runner config * fix: updated the readme. * Add todos * feat: added windows runner configuration, completed todos and added the weight for runner config matchers. * chore: Update docs * fix: reverted tf versions. * fix: addressed comments. * fix: missed. * fix: formatting. * Update terraform versions in CI * Update terraform versions in CI * Update docs * fix: coverage. * Update docs * improve test coverage webhook * Apply suggestions from code review * fix: formatting. * fix: fixed merge issues. * fix: syntax. Co-authored-by: Niek Palm <[email protected]> Co-authored-by: Niek Palm <[email protected]> Co-authored-by: navdeepg2021 <[email protected]> * chore(ci): Enable ci for next branch BREAKING CHANGE: - Add option to deploy multiple runner at once, see for details: #2472 #2517 - Drop support check_run event, see for details: #2521 #2517 - Remove pre 0.19.0 scale down mechanism, see details: #2519 #2517 * chore(release): 2.0.0-next.1 [skip ci] * **ci:** * Add multi-runner capability ([#2472](https://github.com/philips-labs/terraform-aws-github-runner/issues/2472)) ([c08b335](philips-labs/terraform-aws-github-runner@c08b335)), closes [#2521](https://github.com/philips-labs/terraform-aws-github-runner/issues/2521) [#2519](https://github.com/philips-labs/terraform-aws-github-runner/issues/2519) * Experimental feature - Duplicate workflow job event to extra queue ([#2268](https://github.com/philips-labs/terraform-aws-github-runner/issues/2268)) ([ac046b8](philips-labs/terraform-aws-github-runner@ac046b8)) * Remove old scale down mechanism (< 0.19.0) ([#2519](https://github.com/philips-labs/terraform-aws-github-runner/issues/2519)) ([721d7c3](philips-labs/terraform-aws-github-runner@721d7c3)) * Remove support check_run ([#2521](https://github.com/philips-labs/terraform-aws-github-runner/issues/2521)) ([272a293](philips-labs/terraform-aws-github-runner@272a293)) * **ci:** Enable ci for next branch ([48769ca](philips-labs/terraform-aws-github-runner@48769ca)), closes [#2472](https://github.com/philips-labs/terraform-aws-github-runner/issues/2472) [#2517](https://github.com/philips-labs/terraform-aws-github-runner/issues/2517) [#2521](https://github.com/philips-labs/terraform-aws-github-runner/issues/2521) [#2517](https://github.com/philips-labs/terraform-aws-github-runner/issues/2517) [#2519](https://github.com/philips-labs/terraform-aws-github-runner/issues/2519) [#2517](https://github.com/philips-labs/terraform-aws-github-runner/issues/2517) * chore: fix release workflow * fix: added changes for publishing events to secondary queue. Co-authored-by: Niek Palm <[email protected]> Co-authored-by: Niek Palm <[email protected]> Co-authored-by: navdeepg2021 <[email protected]> Co-authored-by: semantic-release-bot <[email protected]> feat: added publishing to workflow_job event queue for multi runner module. (#2570) * feat: Remove support check_run (#2521) * chore: Remove support check_run * format, lint * feat: Remove old scale down mechanism (< 0.19.0) (#2519) fix: Remove old cleanup mechanism (< 0.19.0) * chore: Enable pre releases for next branch * feat: Add multi-runner capability (#2472) * feat: Remove support check_run (#2521) * chore: Remove support check_run * format, lint * feat: Remove old scale down mechanism (< 0.19.0) (#2519) fix: Remove old cleanup mechanism (< 0.19.0) * feat: added changes for multi runner. * fix: region. * fix: more fixes. * tuple to list. * fixes. * fixes. * fixes. * fixes. * fixes. * fixes. * fix: formatting. * fix: formatting. * fix: formatting. * fix: moved some blocks outside runner config. * fix: few more updates * fix: liniting. * fix: updated example output * changed runner group name. * fix: updated the tests. * fix: addressed review comments. * fix: linting issues. * fix: formatting. * fix: updated tf version. * fix: Remove removed prerelease option * Add ubuntu runner to example * refactor: use each instead of count * fix: few small issues. * refactor: syncer to count for multi runner * fix: comments. * fix: added Readme. * fix: errors. * move variable to runner config * fix: updated the readme. * Add todos * feat: added windows runner configuration, completed todos and added the weight for runner config matchers. * chore: Update docs * fix: reverted tf versions. * fix: addressed comments. * fix: missed. * fix: formatting. * Update terraform versions in CI * Update terraform versions in CI * Update docs * fix: coverage. * Update docs * improve test coverage webhook * Apply suggestions from code review * fix: formatting. * fix: fixed merge issues. * fix: syntax. Co-authored-by: Niek Palm <[email protected]> Co-authored-by: Niek Palm <[email protected]> Co-authored-by: navdeepg2021 <[email protected]> * chore(ci): Enable ci for next branch BREAKING CHANGE: - Add option to deploy multiple runner at once, see for details: #2472 #2517 - Drop support check_run event, see for details: #2521 #2517 - Remove pre 0.19.0 scale down mechanism, see details: #2519 #2517 * chore(release): 2.0.0-next.1 [skip ci] * **ci:** * Add multi-runner capability ([#2472](https://github.com/philips-labs/terraform-aws-github-runner/issues/2472)) ([c08b335](philips-labs/terraform-aws-github-runner@c08b335)), closes [#2521](https://github.com/philips-labs/terraform-aws-github-runner/issues/2521) [#2519](https://github.com/philips-labs/terraform-aws-github-runner/issues/2519) * Experimental feature - Duplicate workflow job event to extra queue ([#2268](https://github.com/philips-labs/terraform-aws-github-runner/issues/2268)) ([ac046b8](philips-labs/terraform-aws-github-runner@ac046b8)) * Remove old scale down mechanism (< 0.19.0) ([#2519](https://github.com/philips-labs/terraform-aws-github-runner/issues/2519)) ([721d7c3](philips-labs/terraform-aws-github-runner@721d7c3)) * Remove support check_run ([#2521](https://github.com/philips-labs/terraform-aws-github-runner/issues/2521)) ([272a293](philips-labs/terraform-aws-github-runner@272a293)) * **ci:** Enable ci for next branch ([48769ca](philips-labs/terraform-aws-github-runner@48769ca)), closes [#2472](https://github.com/philips-labs/terraform-aws-github-runner/issues/2472) [#2517](https://github.com/philips-labs/terraform-aws-github-runner/issues/2517) [#2521](https://github.com/philips-labs/terraform-aws-github-runner/issues/2521) [#2517](https://github.com/philips-labs/terraform-aws-github-runner/issues/2517) [#2519](https://github.com/philips-labs/terraform-aws-github-runner/issues/2519) [#2517](https://github.com/philips-labs/terraform-aws-github-runner/issues/2517) * chore: fix release workflow * fix: added changes for publishing events to secondary queue. Co-authored-by: Niek Palm <[email protected]> Co-authored-by: Niek Palm <[email protected]> Co-authored-by: navdeepg2021 <[email protected]> Co-authored-by: semantic-release-bot <[email protected]>
Co-authored-by: navdeepg2021 <[email protected]>
Migration is only required when using pre-build AMI with your tailored start-runner script. Before all parameters were based on convetion. We have update them to be predictable with still similar conventions as default. We set the tag ghr:ssm_config_path as EC2 instance tag to the path used for the configuration. In the config path we store the same values as before (run_as, enable_cloudwatch, agent_mode). The tokens are stored in a separate location which can be retrieved by looking up the value token_path in the config. For a more details example check-out the following diffs: Linux start script // TODO UPDATE AFTER MERGE Windows start script // TODO UPDATE AFTER MERGE
refactor: use optiona for variable block_device_mappings
[multi-runner] Default value validation error
Avoid validation error when `enable_runner_binaries_syncer=false`.
```
Error: Invalid value for input variable
│
│ on .terraform/modules/multi-runner/modules/multi-runner/runners.tf line 13, in module "runners":
│ 13: s3_runner_binaries = each.value.runner_config.enable_runner_binaries_syncer ? local.runner_binaries_by_os_and_arch_map["${each.value.runner_config.runner_os}_${each.value.runner_config.runner_architecture}"] : {}
│
│ The given value is not suitable for module.multi-runner.module.runners["linux-x64-default"].var.s3_runner_binaries declared at
│ .terraform/modules/multi-runner/modules/runners/variables.tf:49,1-30: attributes "arn", "id", and "key" are required.
```
* feat(multi-runner): Support multiple arrays of tags in matchers * Run ci for webhook * fix terraform type * fix formatting * fix terraform type * fix terraform type * fix: some fixes. * fix: readme. * fix example workflow labes * add test for multiple label match * add test for multiple label match Co-authored-by: navdeepg2021 <[email protected]>
- market_options - instance_type
* feat!: Set default lambda node runtime to 18x on arm64 - Update node runtime to node18.x (LTS) - Update default runtime to ARM * upgrade terrafrom versions
* Remove extraneous slash. * Remove more extraneous slashes.
* Support legacy `Application` tag key * Reverting test * fix(runners): Remove legacy Application tag. * fix formatter error Co-authored-by: Nathaniel McAuliffe <[email protected]>
#2784) * Rename variable enabled_userdata -> enable_userdata * Fix escaped occurrences of enabled_userdata * Reformat terraform code
fix: Main module broken after supporting multiple labels for multi-runnes
- generate terraform docs - update readme / annoucements
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
close #2517