Skip to content

Commit

Permalink
Changes are related to AFM (IBM#558)
Browse files Browse the repository at this point in the history
* AFM changes

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* AFM ini.py file changes

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* added ,

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* BM Changes

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Modified vsi and BM parameters

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* commented output not in use for AFM

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Added option protocol subnet

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* fix 1

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* fix 2

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Added conditon for cos bucket creation

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* updated provider

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Modified AFM VSI and BM module

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* changed the condition for output

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Added change for multiple fileset and bucket

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Modified variable to be passed for afm config

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* added file system and afm ip in BM

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Hmac should be one for new cos instance

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* removed commented code

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* added resource group variable

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* removed commented code

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Added tuneable changes for AFM

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Fixed data block for tuneables

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Added function for afm

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Modified afm value function

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Modified afm function call

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* fix1

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Modified tuneable for AFM

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Fix for inventory file

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Changed the description

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Modified tuneable

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* removed afm node grp

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Added bucket type as variable

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Added changes for different bucket type

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* changed hmac key name

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* New changes for bucket types

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Changes for new bucket creation with different bucket type

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Modified mode and filesets

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* fixed classes

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* modfied rest of the function for mode and fileset

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Modifed tuneables for afm

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* modified afmHardMemThreshold

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Removed commented code

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Added logic for "" variabel for afm

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* added condition

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Fix1

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Fix2

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* fix3

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* fix4

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Removed outputs

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Fix for multi cos instance

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* Fix1 for bucket creation

Signed-off-by: Jayesh-Kumar3 <[email protected]>

* added condition

Signed-off-by: Jayesh-Kumar3 <[email protected]>

---------

Signed-off-by: Jayesh-Kumar3 <[email protected]>
Co-authored-by: Jayesh-Kumar3 <[email protected]>
Signed-off-by: Anand.Reddy7 <[email protected]>
  • Loading branch information
2 people authored and anandreddy71 committed Oct 29, 2024
1 parent 7a3df6d commit f9bb55e
Show file tree
Hide file tree
Showing 10 changed files with 1,160 additions and 29 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -54,12 +54,13 @@ Below steps will provision IBM Cloud resources (compute and storage instances in
| Name | Version |
|------|---------|
| <a name="requirement_github"></a> [github](#requirement_github) | 5.41.0 |
| <a name="requirement_ibm"></a> [ibm](#requirement_ibm) | 1.65.1 |
| <a name="requirement_ibm"></a> [ibm](#requirement_ibm) | 1.67.1 |
#### Inputs
| Name | Description | Type |
|------|-------------|------|
| <a name="input_afm_cos_config"></a> [afm_cos_config](#input_afm_cos_config) | Please add the details of COS instance, COS bucket, hmac key, AFM fileset and mode(Read-only (RO), Single writer (SW), Local updates (LU) and Independent writer (IW)). | <pre>list(object({<br> cos_instance = string,<br> bucket_name = string,<br> bucket_region = string,<br> cos_service_cred_key = string,<br> afm_fileset = string,<br> mode = string,<br> bucket_storage_class = string,<br> bucket_type = string<br> }))</pre> |
| <a name="input_resource_group_id"></a> [resource_group_id](#input_resource_group_id) | IBM Cloud resource group id. | `string` |
| <a name="input_storage_cluster_gui_password"></a> [storage_cluster_gui_password](#input_storage_cluster_gui_password) | Password for storage cluster GUI | `string` |
| <a name="input_storage_cluster_gui_username"></a> [storage_cluster_gui_username](#input_storage_cluster_gui_username) | GUI user to perform system management and monitoring tasks on storage cluster. | `string` |
Expand All @@ -79,6 +80,7 @@ Below steps will provision IBM Cloud resources (compute and storage instances in
| <a name="input_vpc_storage_cluster_dns_zone_id"></a> [vpc_storage_cluster_dns_zone_id](#input_vpc_storage_cluster_dns_zone_id) | IBM Cloud storage cluster DNS zone id. | `string` |
| <a name="input_vpc_storage_cluster_private_subnets"></a> [vpc_storage_cluster_private_subnets](#input_vpc_storage_cluster_private_subnets) | List of IDs of storage cluster private subnets. | `list(string)` |
| <a name="input_activity_tracker_plan_type"></a> [activity_tracker_plan_type](#input_activity_tracker_plan_type) | IBM Cloud activity tracker plan type (Valid: lite, 7-day, 14-day, 30-day, hipaa-30-day). | `string` |
| <a name="input_afm_vsi_profile"></a> [afm_vsi_profile](#input_afm_vsi_profile) | The virtual instance or bare metal server instance profile type name to be used to create the AFM gateway nodes. For more information, see [Instance Profiles](https://cloud.ibm.com/docs/vpc?topic=vpc-profiles&interface=ui) and [bare metal server profiles](https://cloud.ibm.com/docs/vpc?topic=vpc-bare-metal-servers-profile&interface=ui). | `string` |
| <a name="input_bastion_instance_id"></a> [bastion_instance_id](#input_bastion_instance_id) | Bastion instance id. | `string` |
| <a name="input_bastion_instance_public_ip"></a> [bastion_instance_public_ip](#input_bastion_instance_public_ip) | Bastion instance public ip address. | `string` |
| <a name="input_bastion_security_group_id"></a> [bastion_security_group_id](#input_bastion_security_group_id) | Bastion security group id. | `string` |
Expand Down Expand Up @@ -136,6 +138,7 @@ Below steps will provision IBM Cloud resources (compute and storage instances in
| <a name="input_storage_vsi_osimage_id"></a> [storage_vsi_osimage_id](#input_storage_vsi_osimage_id) | Image id to use for provisioning the storage cluster instances. | `string` |
| <a name="input_storage_vsi_osimage_name"></a> [storage_vsi_osimage_name](#input_storage_vsi_osimage_name) | Image name to use for provisioning the storage cluster instances. | `string` |
| <a name="input_storage_vsi_profile"></a> [storage_vsi_profile](#input_storage_vsi_profile) | Profile to be used for storage cluster virtual server instance. | `string` |
| <a name="input_total_afm_cluster_instances"></a> [total_afm_cluster_instances](#input_total_afm_cluster_instances) | Total number of instance count that you need to provision for afm nodes and enable AFM. | `number` |
| <a name="input_total_client_cluster_instances"></a> [total_client_cluster_instances](#input_total_client_cluster_instances) | Total number of client cluster instances that you need to provision. A minimum of 2 nodes and a maximum of 64 nodes are supported | `number` |
| <a name="input_total_compute_cluster_instances"></a> [total_compute_cluster_instances](#input_total_compute_cluster_instances) | Number of instances to be launched for compute cluster. | `number` |
| <a name="input_total_gklm_instances"></a> [total_gklm_instances](#input_total_gklm_instances) | Number of instances to be launched for GKLM. | `number` |
Expand Down
100 changes: 94 additions & 6 deletions ibmcloud_scale_templates/sub_modules/instance_template/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ locals {
enable_sec_interface_storage = local.scale_ces_enabled == false && var.storage_type != "persistent" && data.ibm_is_instance_profile.storage_profile.bandwidth[0].value >= 64000 ? true : false
enable_mrot_conf = local.enable_sec_interface_compute && local.enable_sec_interface_storage ? true : false
ldap_server = var.enable_ldap == true && var.ldap_server == "null" ? jsonencode(one(module.ldap_instance[*].vsi_private_ip)) : var.ldap_server
enable_afm = var.total_afm_cluster_instances > 0 ? true : false
afm_server_type = strcontains(var.afm_vsi_profile, "metal")
}

module "generate_compute_cluster_keys" {
Expand Down Expand Up @@ -612,6 +614,71 @@ module "gklm_instance" {
depends_on = [module.gklm_instance_ingress_security_rule, module.gklm_instance_ingress_security_rule_wt_bastion, module.gklm_instance_ingress_security_rule_wo_bastion, module.gklm_instance_egress_security_rule, var.vpc_custom_resolver_id]
}

data "ibm_is_bare_metal_server_profile" "afm_vsi_bm_server_profile" {
count = local.afm_server_type == true ? 1 : 0
name = var.afm_vsi_profile
}

data "ibm_is_instance_profile" "afm_vsi_server_profile" {
count = local.afm_server_type == false ? 1 : 0
name = var.afm_vsi_profile
}

module "afm_cluster_instances" {
source = "../../../resources/ibmcloud/compute/afm_vsi"
total_vsis = var.total_afm_cluster_instances
vsi_name_prefix = format("%s-afm", var.resource_prefix)
afm_server_type = local.afm_server_type
vpc_id = var.vpc_id
resource_group_id = var.resource_group_id
zones = [var.vpc_availability_zones[0]]
vsi_image_id = local.storage_instance_image_id
vsi_profile = var.afm_vsi_profile
dns_domain = var.vpc_storage_cluster_dns_domain
dns_service_id = var.vpc_storage_cluster_dns_service_id
dns_zone_id = var.vpc_storage_cluster_dns_zone_id
vsi_subnet_id = var.vpc_storage_cluster_private_subnets
vsi_security_group = [module.storage_cluster_security_group.sec_group_id]
vsi_user_public_key = data.ibm_is_ssh_key.storage_ssh_key[*].id
vsi_meta_private_key = module.generate_storage_cluster_keys.private_key_content
vsi_meta_public_key = module.generate_storage_cluster_keys.public_key_content
scale_firewall_rules_enabled = true
resource_tags = var.scale_cluster_resource_tags
depends_on = [module.storage_cluster_ingress_security_rule, module.storage_cluster_ingress_security_rule_wo_bastion, module.storage_cluster_ingress_security_rule_wt_bastion, module.storage_egress_security_rule, var.vpc_custom_resolver_id]
}

locals {
new_instance_bucket_hmac = [for details in var.afm_cos_config : details if(details.cos_instance == "" && details.bucket_name == "" && details.cos_service_cred_key == "")]
exstng_instance_new_bucket_hmac = [for details in var.afm_cos_config : details if(details.cos_instance != "" && details.bucket_name == "" && details.cos_service_cred_key == "")]
exstng_instance_bucket_new_hmac = [for details in var.afm_cos_config : details if(details.cos_instance != "" && details.bucket_name != "" && details.cos_service_cred_key == "")]
exstng_instance_hmac_new_bucket = [for details in var.afm_cos_config : details if(details.cos_instance != "" && details.bucket_name == "" && details.cos_service_cred_key != "")]
exstng_instance_bucket_hmac = [for details in var.afm_cos_config : details if(details.cos_instance != "" && details.bucket_name != "" && details.cos_service_cred_key != "")]

}

module "cos" {
count = local.enable_afm == true ? 1 : 0
source = "../../../resources/ibmcloud/compute/cos"
prefix = "${var.resource_prefix}-"
resource_group_id = var.resource_group_id
cos_instance_plan = "standard"
cos_instance_location = "global"
cos_instance_service = "cloud-object-storage"
cos_hmac_role = "Manager"
new_instance_bucket_hmac = local.new_instance_bucket_hmac
exstng_instance_new_bucket_hmac = local.exstng_instance_new_bucket_hmac
exstng_instance_bucket_new_hmac = local.exstng_instance_bucket_new_hmac
exstng_instance_hmac_new_bucket = local.exstng_instance_hmac_new_bucket
exstng_instance_bucket_hmac = local.exstng_instance_bucket_hmac
filesystem = var.storage_cluster_filesystem_mountpoint
depends_on = [module.afm_cluster_instances]
}

locals {
afm_cos_bucket_details = local.enable_afm == true ? flatten(module.cos[*].afm_cos_bucket_details) : []
afm_config_details = local.enable_afm == true ? flatten(module.cos[*].afm_config_details) : []
}

module "activity_tracker" {
source = "../../../resources/ibmcloud/resource_instance"
service_count = var.vpc_create_activity_tracker == true ? 1 : 0
Expand All @@ -637,19 +704,23 @@ data "ibm_is_subnet_reserved_ips" "protocol_subnet_reserved_ips" {
}

locals {
storage_instance_ids = var.storage_type != "persistent" ? values(one(module.storage_cluster_instances[*].instance_name_id_map)) : []
storage_instance_names = var.storage_type != "persistent" ? keys(one(module.storage_cluster_instances[*].instance_name_id_map)) : []
storage_instance_private_ips = var.storage_type != "persistent" ? values(one(module.storage_cluster_instances[*].instance_name_ip_map)) : []
afm_instance_ids = values(one(module.afm_cluster_instances[*].storage_cluster_instance_name_id_map_vsi_bm))
afm_instance_names = keys(one(module.afm_cluster_instances[*].storage_cluster_instance_name_id_map_vsi_bm))
afm_instance_private_ips = values(one(module.afm_cluster_instances[*].storage_cluster_instance_name_ip_map_vsi_bm))

storage_instance_ids = var.storage_type != "persistent" ? local.enable_afm == true ? concat(values(one(module.storage_cluster_instances[*].instance_name_id_map)), local.afm_instance_ids) : values(one(module.storage_cluster_instances[*].instance_name_id_map)) : []
storage_instance_names = var.storage_type != "persistent" ? local.enable_afm == true ? concat(keys(one(module.storage_cluster_instances[*].instance_name_id_map)), local.afm_instance_names) : keys(one(module.storage_cluster_instances[*].instance_name_id_map)) : []
storage_instance_private_ips = var.storage_type != "persistent" ? local.enable_afm == true ? concat(values(one(module.storage_cluster_instances[*].instance_name_ip_map)), local.afm_instance_private_ips) : values(one(module.storage_cluster_instances[*].instance_name_ip_map)) : []
storage_instance_private_dns_ip_map = var.storage_type != "persistent" ? one(module.storage_cluster_instances[*].instance_private_dns_ip_map) : {}

storage_cluster_instance_ids = local.scale_ces_enabled == false ? local.storage_instance_ids : concat(local.storage_instance_ids, values(one(module.protocol_cluster_instances[*].instance_name_id_map)))
storage_cluster_instance_names = local.scale_ces_enabled == false ? local.storage_instance_names : concat(local.storage_instance_names, keys(one(module.protocol_cluster_instances[*].instance_name_id_map)))
storage_cluster_instance_private_ips = local.scale_ces_enabled == false ? local.storage_instance_private_ips : concat(local.storage_instance_private_ips, values(one(module.protocol_cluster_instances[*].instance_name_ip_map)))
storage_cluster_instance_private_dns_ip_map = local.scale_ces_enabled == false ? local.storage_instance_private_dns_ip_map : merge(local.storage_instance_private_dns_ip_map, one(module.protocol_cluster_instances[*].instance_private_dns_ip_map))

baremetal_instance_ids = var.storage_type == "persistent" ? values(one(module.storage_cluster_bare_metal_server[*].storage_cluster_instance_name_id_map)) : []
baremetal_instance_names = var.storage_type == "persistent" ? keys(one(module.storage_cluster_bare_metal_server[*].storage_cluster_instance_name_id_map)) : []
baremetal_instance_private_ips = var.storage_type == "persistent" ? values(one(module.storage_cluster_bare_metal_server[*].storage_cluster_instance_name_ip_map)) : []
baremetal_instance_ids = var.storage_type == "persistent" ? local.enable_afm == true ? concat(values(one(module.storage_cluster_bare_metal_server[*].storage_cluster_instance_name_id_map)), local.afm_instance_ids) : values(one(module.storage_cluster_bare_metal_server[*].storage_cluster_instance_name_id_map)) : []
baremetal_instance_names = var.storage_type == "persistent" ? local.enable_afm == true ? concat(keys(one(module.storage_cluster_bare_metal_server[*].storage_cluster_instance_name_id_map)), local.afm_instance_names) : keys(one(module.storage_cluster_bare_metal_server[*].storage_cluster_instance_name_id_map)) : []
baremetal_instance_private_ips = var.storage_type == "persistent" ? local.enable_afm == true ? concat(values(one(module.storage_cluster_bare_metal_server[*].storage_cluster_instance_name_ip_map)), local.afm_instance_private_ips) : values(one(module.storage_cluster_bare_metal_server[*].storage_cluster_instance_name_ip_map)) : []
baremetal_instance_private_dns_ip_map = var.storage_type == "persistent" ? one(module.storage_cluster_bare_metal_server[*].instance_private_dns_ip_map) : {}

baremetal_cluster_instance_ids = var.storage_type == "persistent" && local.scale_ces_enabled == false ? local.baremetal_instance_ids : concat(local.baremetal_instance_ids, values(one(module.protocol_cluster_instances[*].instance_name_id_map)))
Expand Down Expand Up @@ -717,6 +788,9 @@ module "write_compute_cluster_inventory" {
mountpoint = jsonencode("")
protocol_gateway_ip = jsonencode("")
filesets = jsonencode({})
afm_cos_bucket_details = jsonencode([])
afm_config_details = jsonencode([])
afm_cluster_instance_names = jsonencode([])
}

module "write_storage_cluster_inventory" {
Expand Down Expand Up @@ -763,6 +837,9 @@ module "write_storage_cluster_inventory" {
mountpoint = local.scale_ces_enabled == true ? jsonencode(var.storage_cluster_filesystem_mountpoint) : jsonencode("")
protocol_gateway_ip = jsonencode(local.protocol_subnet_gateway_ip)
filesets = jsonencode(local.fileset_size_map)
afm_cos_bucket_details = local.enable_afm == true ? jsonencode(local.afm_cos_bucket_details) : jsonencode([])
afm_config_details = local.enable_afm == true ? jsonencode(local.afm_config_details) : jsonencode([])
afm_cluster_instance_names = jsonencode(local.afm_instance_names)
}

module "write_cluster_inventory" {
Expand Down Expand Up @@ -809,6 +886,9 @@ module "write_cluster_inventory" {
mountpoint = jsonencode("")
protocol_gateway_ip = jsonencode("")
filesets = jsonencode({})
afm_cos_bucket_details = jsonencode([])
afm_config_details = jsonencode([])
afm_cluster_instance_names = jsonencode([])
}

module "write_client_cluster_inventory" {
Expand Down Expand Up @@ -855,6 +935,9 @@ module "write_client_cluster_inventory" {
mountpoint = jsonencode("")
protocol_gateway_ip = jsonencode("")
filesets = local.scale_ces_enabled == true ? jsonencode(local.fileset_size_map) : jsonencode({})
afm_cos_bucket_details = jsonencode([])
afm_config_details = jsonencode([])
afm_cluster_instance_names = jsonencode([])
}

module "compute_cluster_configuration" {
Expand Down Expand Up @@ -882,6 +965,7 @@ module "compute_cluster_configuration" {
spectrumscale_rpms_path = var.spectrumscale_rpms_path
enable_mrot_conf = local.enable_mrot_conf ? "True" : "False"
enable_ces = "False"
enable_afm = "False"
scale_encryption_enabled = var.scale_encryption_enabled
scale_encryption_admin_password = var.scale_encryption_enabled ? var.scale_encryption_admin_password : null
scale_encryption_servers = var.scale_encryption_enabled ? jsonencode(one(module.gklm_instance[*].gklm_ip_addresses)) : null
Expand Down Expand Up @@ -924,6 +1008,9 @@ module "storage_cluster_configuration" {
strg_proto_memory = var.storage_type == "persistent" ? data.ibm_is_bare_metal_server_profile.storage_bare_metal_server_profile[0].memory[0].value : data.ibm_is_instance_profile.storage_profile.memory[0].value
strg_proto_vcpus_count = var.storage_type == "persistent" ? data.ibm_is_bare_metal_server_profile.storage_bare_metal_server_profile[0].cpu_core_count[0].value * data.ibm_is_bare_metal_server_profile.storage_bare_metal_server_profile[0].cpu_socket_count[0].value : data.ibm_is_instance_profile.storage_profile.vcpu_count[0].value
strg_proto_bandwidth = var.storage_type == "persistent" ? data.ibm_is_bare_metal_server_profile.storage_bare_metal_server_profile[0].bandwidth[0].value : data.ibm_is_instance_profile.storage_profile.bandwidth[0].value
afm_memory = local.afm_server_type == true ? data.ibm_is_bare_metal_server_profile.afm_vsi_bm_server_profile[0].memory[0].value : data.ibm_is_instance_profile.afm_vsi_server_profile[0].memory[0].value
afm_vcpus_count = local.afm_server_type == true ? data.ibm_is_bare_metal_server_profile.afm_vsi_bm_server_profile[0].cpu_core_count[0].value * data.ibm_is_bare_metal_server_profile.afm_vsi_bm_server_profile[0].cpu_socket_count[0].value : data.ibm_is_instance_profile.afm_vsi_server_profile[0].vcpu_count[0].value
afm_bandwidth = local.afm_server_type == true ? data.ibm_is_bare_metal_server_profile.afm_vsi_bm_server_profile[0].bandwidth[0].value : data.ibm_is_instance_profile.afm_vsi_server_profile[0].bandwidth[0].value
disk_type = "network-attached"
max_data_replicas = 3
max_metadata_replicas = 3
Expand All @@ -936,6 +1023,7 @@ module "storage_cluster_configuration" {
spectrumscale_rpms_path = var.spectrumscale_rpms_path
enable_mrot_conf = local.enable_mrot_conf ? "True" : "False"
enable_ces = local.scale_ces_enabled == true ? "True" : "False"
enable_afm = local.enable_afm == true ? "True" : "False"
scale_encryption_enabled = var.scale_encryption_enabled
scale_encryption_admin_password = var.scale_encryption_enabled ? var.scale_encryption_admin_password : null
scale_encryption_servers = var.scale_encryption_enabled ? jsonencode(one(module.gklm_instance[*].gklm_ip_addresses)) : null
Expand Down
Loading

0 comments on commit f9bb55e

Please sign in to comment.