Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Refactor user data handling in Oracle DB instance module and update scripts for SSH key setup #8950

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

georgepstaylor
Copy link
Member

@georgepstaylor georgepstaylor commented Dec 3, 2024

This pull request includes a change that modifies how user_data_raw is generated by incorporating a template file for better flexibility and management. The ssh key setup was missing and so this has also been fixed

Key change:

@georgepstaylor georgepstaylor requested review from a team as code owners December 3, 2024 15:32
@github-actions github-actions bot added the environments-repository Used to exclude PRs from this repo in our Slack PR update label Dec 3, 2024
Copy link
Contributor

github-actions bot commented Dec 3, 2024

Trivy Scan Success

Show Output ```hcl

Trivy will check the following folders:
terraform/environments/delius-core/modules/components/oracle_db_instance


Running Trivy in terraform/environments/delius-core/modules/components/oracle_db_instance
2024-12-03T15:34:28Z INFO [vulndb] Need to update DB
2024-12-03T15:34:28Z INFO [vulndb] Downloading vulnerability DB...
2024-12-03T15:34:28Z INFO [vulndb] Downloading artifact... repo="public.ecr.aws/aquasecurity/trivy-db:2"
2024-12-03T15:34:30Z INFO [vulndb] Artifact successfully downloaded repo="public.ecr.aws/aquasecurity/trivy-db:2"
2024-12-03T15:34:30Z INFO [vuln] Vulnerability scanning is enabled
2024-12-03T15:34:30Z INFO [misconfig] Misconfiguration scanning is enabled
2024-12-03T15:34:30Z INFO [misconfig] Need to update the built-in checks
2024-12-03T15:34:30Z INFO [misconfig] Downloading the built-in checks...
160.80 KiB / 160.80 KiB [------------------------------------------------------] 100.00% ? p/s 100ms2024-12-03T15:34:30Z INFO [secret] Secret scanning is enabled
2024-12-03T15:34:30Z INFO [secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-12-03T15:34:30Z INFO [secret] Please see also https://aquasecurity.github.io/trivy/v0.57/docs/scanner/secret#recommendation for faster secret detection
2024-12-03T15:34:32Z INFO [terraform scanner] Scanning root module file_path="."
2024-12-03T15:34:32Z WARN [terraform parser] Variable values was not found in the environment or variable files. Evaluating may not work correctly. module="root" variables="account_config, account_info, availability_zone, db_ami, ebs_volume_config, ebs_volumes, ec2_key_pair_name, env_name, environment_config, instance_profile_policies, security_group_ids, sns_topic_arn, ssh_keys_bucket_name, subnet_id, tags, user_data"
2024-12-03T15:34:32Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.instance.aws_ebs_volume.this" value="cty.NilVal"
2024-12-03T15:34:32Z ERROR [terraform evaluator] Failed to expand dynamic block. block="module.instance.aws_instance.this" err="2 errors occurred:\n\t* invalid for-each in aws_instance.this.dynamic.ephemeral_block_device block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\t* invalid for-each in aws_instance.this.dynamic.ebs_block_device block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"
2024-12-03T15:34:32Z ERROR [terraform evaluator] Failed to expand dynamic block. block="module.instance.aws_instance.this" err="2 errors occurred:\n\t* invalid for-each in aws_instance.this.dynamic.ephemeral_block_device block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\t* invalid for-each in aws_instance.this.dynamic.ebs_block_device block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"
2024-12-03T15:34:32Z INFO [terraform executor] Ignore finding rule="aws-ec2-enforce-http-token-imds" range="github.com/ministryofjustice/modernisation-platform-terraform-ec2-instance/main.tf:22"
2024-12-03T15:34:32Z INFO Number of language-specific files num=0
2024-12-03T15:34:32Z INFO Detected config files num=2
trivy_exitcode=0

</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>

```hcl

*****************************

Checkov will check the following folders:
terraform/environments/delius-core/modules/components/oracle_db_instance

*****************************

Running Checkov in terraform/environments/delius-core/modules/components/oracle_db_instance
Excluding the following checks: CKV_GIT_1,CKV_AWS_126,CKV2_AWS_38,CKV2_AWS_39
2024-12-03 15:34:35,282 [MainThread  ] [WARNI]  Failed to download module github.com/ministryofjustice/modernisation-platform-terraform-ec2-instance:None (for external modules, the --download-external-modules flag is required)
terraform scan results:

Passed checks: 0, Failed checks: 0, Skipped checks: 2


checkov_exitcode=0

CTFLint Scan Failed

Show Output
*****************************

Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.9.1)
tflint will check the following folders:
terraform/environments/delius-core/modules/components/oracle_db_instance

*****************************

Running tflint in terraform/environments/delius-core/modules/components/oracle_db_instance
Excluding the following checks: terraform_unused_declarations
5 issue(s) found:

Warning: Module source "github.com/ministryofjustice/modernisation-platform-terraform-ec2-instance" is not pinned (terraform_module_pinned_source)

  on terraform/environments/delius-core/modules/components/oracle_db_instance/instance.tf line 26:
  26:   source = "github.com/ministryofjustice/modernisation-platform-terraform-ec2-instance"

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.9.1/docs/rules/terraform_module_pinned_source.md

Warning: Duplicate key: "datapoints_to_alarm", first defined at terraform/environments/delius-core/modules/components/oracle_db_instance/locals.tf:49,9-28 (terraform_map_duplicate_keys)

  on terraform/environments/delius-core/modules/components/oracle_db_instance/locals.tf line 55:
  55:         datapoints_to_alarm = "10"

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.9.1/docs/rules/terraform_map_duplicate_keys.md

Warning: Duplicate key: "evaluation_periods", first defined at terraform/environments/delius-core/modules/components/oracle_db_instance/locals.tf:48,9-27 (terraform_map_duplicate_keys)

  on terraform/environments/delius-core/modules/components/oracle_db_instance/locals.tf line 56:
  56:         evaluation_periods  = "10"

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.9.1/docs/rules/terraform_map_duplicate_keys.md

Warning: `server_type_tag` variable has no type (terraform_typed_variables)

  on terraform/environments/delius-core/modules/components/oracle_db_instance/variables.tf line 36:
  36: variable "server_type_tag" {

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.9.1/docs/rules/terraform_typed_variables.md

Warning: `database_tag_prefix` variable has no type (terraform_typed_variables)

  on terraform/environments/delius-core/modules/components/oracle_db_instance/variables.tf line 41:
  41: variable "database_tag_prefix" {

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.9.1/docs/rules/terraform_typed_variables.md

tflint_exitcode=2

Trivy Scan Success

Show Output
*****************************

Trivy will check the following folders:
terraform/environments/delius-core/modules/components/oracle_db_instance

*****************************

Running Trivy in terraform/environments/delius-core/modules/components/oracle_db_instance
2024-12-03T15:34:28Z	INFO	[vulndb] Need to update DB
2024-12-03T15:34:28Z	INFO	[vulndb] Downloading vulnerability DB...
2024-12-03T15:34:28Z	INFO	[vulndb] Downloading artifact...	repo="public.ecr.aws/aquasecurity/trivy-db:2"
2024-12-03T15:34:30Z	INFO	[vulndb] Artifact successfully downloaded	repo="public.ecr.aws/aquasecurity/trivy-db:2"
2024-12-03T15:34:30Z	INFO	[vuln] Vulnerability scanning is enabled
2024-12-03T15:34:30Z	INFO	[misconfig] Misconfiguration scanning is enabled
2024-12-03T15:34:30Z	INFO	[misconfig] Need to update the built-in checks
2024-12-03T15:34:30Z	INFO	[misconfig] Downloading the built-in checks...
160.80 KiB / 160.80 KiB [------------------------------------------------------] 100.00% ? p/s 100ms2024-12-03T15:34:30Z	INFO	[secret] Secret scanning is enabled
2024-12-03T15:34:30Z	INFO	[secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-12-03T15:34:30Z	INFO	[secret] Please see also https://aquasecurity.github.io/trivy/v0.57/docs/scanner/secret#recommendation for faster secret detection
2024-12-03T15:34:32Z	INFO	[terraform scanner] Scanning root module	file_path="."
2024-12-03T15:34:32Z	WARN	[terraform parser] Variable values was not found in the environment or variable files. Evaluating may not work correctly.	module="root" variables="account_config, account_info, availability_zone, db_ami, ebs_volume_config, ebs_volumes, ec2_key_pair_name, env_name, environment_config, instance_profile_policies, security_group_ids, sns_topic_arn, ssh_keys_bucket_name, subnet_id, tags, user_data"
2024-12-03T15:34:32Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.instance.aws_ebs_volume.this" value="cty.NilVal"
2024-12-03T15:34:32Z	ERROR	[terraform evaluator] Failed to expand dynamic block.	block="module.instance.aws_instance.this" err="2 errors occurred:\n\t* invalid for-each in aws_instance.this.dynamic.ephemeral_block_device block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\t* invalid for-each in aws_instance.this.dynamic.ebs_block_device block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"
2024-12-03T15:34:32Z	ERROR	[terraform evaluator] Failed to expand dynamic block.	block="module.instance.aws_instance.this" err="2 errors occurred:\n\t* invalid for-each in aws_instance.this.dynamic.ephemeral_block_device block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\t* invalid for-each in aws_instance.this.dynamic.ebs_block_device block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"
2024-12-03T15:34:32Z	INFO	[terraform executor] Ignore finding	rule="aws-ec2-enforce-http-token-imds" range="github.com/ministryofjustice/modernisation-platform-terraform-ec2-instance/main.tf:22"
2024-12-03T15:34:32Z	INFO	Number of language-specific files	num=0
2024-12-03T15:34:32Z	INFO	Detected config files	num=2
trivy_exitcode=0

andrewmooreio
andrewmooreio previously approved these changes Dec 3, 2024
Copy link
Contributor

github-actions bot commented Dec 6, 2024

Trivy Scan Success

Show Output ```hcl

Trivy will check the following folders:
terraform/environments/delius-core/modules/components/oracle_db_instance


Running Trivy in terraform/environments/delius-core/modules/components/oracle_db_instance
2024-12-06T12:19:52Z INFO [vulndb] Need to update DB
2024-12-06T12:19:52Z INFO [vulndb] Downloading vulnerability DB...
2024-12-06T12:19:52Z INFO [vulndb] Downloading artifact... repo="public.ecr.aws/aquasecurity/trivy-db:2"
2024-12-06T12:19:54Z INFO [vulndb] Artifact successfully downloaded repo="public.ecr.aws/aquasecurity/trivy-db:2"
2024-12-06T12:19:54Z INFO [vuln] Vulnerability scanning is enabled
2024-12-06T12:19:54Z INFO [misconfig] Misconfiguration scanning is enabled
2024-12-06T12:19:54Z INFO [misconfig] Need to update the built-in checks
2024-12-06T12:19:54Z INFO [misconfig] Downloading the built-in checks...
160.80 KiB / 160.80 KiB [------------------------------------------------------] 100.00% ? p/s 100ms2024-12-06T12:19:55Z INFO [secret] Secret scanning is enabled
2024-12-06T12:19:55Z INFO [secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-12-06T12:19:55Z INFO [secret] Please see also https://aquasecurity.github.io/trivy/v0.58/docs/scanner/secret#recommendation for faster secret detection
2024-12-06T12:19:56Z INFO [terraform scanner] Scanning root module file_path="."
2024-12-06T12:19:56Z WARN [terraform parser] Variable values was not found in the environment or variable files. Evaluating may not work correctly. module="root" variables="account_config, account_info, availability_zone, db_ami, ebs_volume_config, ebs_volumes, ec2_key_pair_name, env_name, environment_config, instance_profile_policies, security_group_ids, sns_topic_arn, ssh_keys_bucket_name, subnet_id, tags, user_data"
2024-12-06T12:19:57Z INFO [terraform executor] Ignore finding rule="aws-ec2-enforce-http-token-imds" range="github.com/ministryofjustice/modernisation-platform-terraform-ec2-instance/main.tf:22"
2024-12-06T12:19:57Z INFO Number of language-specific files num=0
2024-12-06T12:19:57Z INFO Detected config files num=2
trivy_exitcode=0

</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>

```hcl

*****************************

Checkov will check the following folders:
terraform/environments/delius-core/modules/components/oracle_db_instance

*****************************

Running Checkov in terraform/environments/delius-core/modules/components/oracle_db_instance
Excluding the following checks: CKV_GIT_1,CKV_AWS_126,CKV2_AWS_38,CKV2_AWS_39
2024-12-06 12:19:59,420 [MainThread  ] [WARNI]  Failed to download module github.com/ministryofjustice/modernisation-platform-terraform-ec2-instance:None (for external modules, the --download-external-modules flag is required)
terraform scan results:

Passed checks: 0, Failed checks: 0, Skipped checks: 2


checkov_exitcode=0

CTFLint Scan Failed

Show Output
*****************************

Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.9.1)
tflint will check the following folders:
terraform/environments/delius-core/modules/components/oracle_db_instance

*****************************

Running tflint in terraform/environments/delius-core/modules/components/oracle_db_instance
Excluding the following checks: terraform_unused_declarations
5 issue(s) found:

Warning: Module source "github.com/ministryofjustice/modernisation-platform-terraform-ec2-instance" is not pinned (terraform_module_pinned_source)

  on terraform/environments/delius-core/modules/components/oracle_db_instance/instance.tf line 26:
  26:   source = "github.com/ministryofjustice/modernisation-platform-terraform-ec2-instance"

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.9.1/docs/rules/terraform_module_pinned_source.md

Warning: Duplicate key: "datapoints_to_alarm", first defined at terraform/environments/delius-core/modules/components/oracle_db_instance/locals.tf:49,9-28 (terraform_map_duplicate_keys)

  on terraform/environments/delius-core/modules/components/oracle_db_instance/locals.tf line 55:
  55:         datapoints_to_alarm = "10"

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.9.1/docs/rules/terraform_map_duplicate_keys.md

Warning: Duplicate key: "evaluation_periods", first defined at terraform/environments/delius-core/modules/components/oracle_db_instance/locals.tf:48,9-27 (terraform_map_duplicate_keys)

  on terraform/environments/delius-core/modules/components/oracle_db_instance/locals.tf line 56:
  56:         evaluation_periods  = "10"

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.9.1/docs/rules/terraform_map_duplicate_keys.md

Warning: `server_type_tag` variable has no type (terraform_typed_variables)

  on terraform/environments/delius-core/modules/components/oracle_db_instance/variables.tf line 36:
  36: variable "server_type_tag" {

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.9.1/docs/rules/terraform_typed_variables.md

Warning: `database_tag_prefix` variable has no type (terraform_typed_variables)

  on terraform/environments/delius-core/modules/components/oracle_db_instance/variables.tf line 41:
  41: variable "database_tag_prefix" {

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.9.1/docs/rules/terraform_typed_variables.md

tflint_exitcode=2

Trivy Scan Success

Show Output
*****************************

Trivy will check the following folders:
terraform/environments/delius-core/modules/components/oracle_db_instance

*****************************

Running Trivy in terraform/environments/delius-core/modules/components/oracle_db_instance
2024-12-06T12:19:52Z	INFO	[vulndb] Need to update DB
2024-12-06T12:19:52Z	INFO	[vulndb] Downloading vulnerability DB...
2024-12-06T12:19:52Z	INFO	[vulndb] Downloading artifact...	repo="public.ecr.aws/aquasecurity/trivy-db:2"
2024-12-06T12:19:54Z	INFO	[vulndb] Artifact successfully downloaded	repo="public.ecr.aws/aquasecurity/trivy-db:2"
2024-12-06T12:19:54Z	INFO	[vuln] Vulnerability scanning is enabled
2024-12-06T12:19:54Z	INFO	[misconfig] Misconfiguration scanning is enabled
2024-12-06T12:19:54Z	INFO	[misconfig] Need to update the built-in checks
2024-12-06T12:19:54Z	INFO	[misconfig] Downloading the built-in checks...
160.80 KiB / 160.80 KiB [------------------------------------------------------] 100.00% ? p/s 100ms2024-12-06T12:19:55Z	INFO	[secret] Secret scanning is enabled
2024-12-06T12:19:55Z	INFO	[secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-12-06T12:19:55Z	INFO	[secret] Please see also https://aquasecurity.github.io/trivy/v0.58/docs/scanner/secret#recommendation for faster secret detection
2024-12-06T12:19:56Z	INFO	[terraform scanner] Scanning root module	file_path="."
2024-12-06T12:19:56Z	WARN	[terraform parser] Variable values was not found in the environment or variable files. Evaluating may not work correctly.	module="root" variables="account_config, account_info, availability_zone, db_ami, ebs_volume_config, ebs_volumes, ec2_key_pair_name, env_name, environment_config, instance_profile_policies, security_group_ids, sns_topic_arn, ssh_keys_bucket_name, subnet_id, tags, user_data"
2024-12-06T12:19:57Z	INFO	[terraform executor] Ignore finding	rule="aws-ec2-enforce-http-token-imds" range="github.com/ministryofjustice/modernisation-platform-terraform-ec2-instance/main.tf:22"
2024-12-06T12:19:57Z	INFO	Number of language-specific files	num=0
2024-12-06T12:19:57Z	INFO	Detected config files	num=2
trivy_exitcode=0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
environments-repository Used to exclude PRs from this repo in our Slack PR update
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants