Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

azurerm_container_app_custom_domain fails parsing the certificate ID for managed certificates #25788

Closed
1 task done
ghjklw opened this issue Apr 29, 2024 · 18 comments · Fixed by #25972
Closed
1 task done

Comments

@ghjklw
Copy link

ghjklw commented Apr 29, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.8.1

AzureRM Provider Version

3.101.0

Affected Resource(s)/Data Source(s)

azurerm_container_app_custom_domain

Terraform Configuration Files

resource "azurerm_container_app_custom_domain" "airflow" {
  container_app_id = module.container_app["webserver"].app.id
  name             = trimsuffix(azurerm_dns_cname_record.airflow.fqdn, ".")
  # The FQDN of the DNS CNAME Record which has a full-stop at the end is by design.
  # We must therefore remove it

  depends_on = [azurerm_dns_txt_record.airflow_verification]
  lifecycle {
    ignore_changes = [certificate_binding_type, container_app_environment_certificate_id]
  }
}

Debug Output/Panic Output

│ Error: parsing "/subscriptions/<subscription_id>/resourceGroups/<rg_name>/providers/Microsoft.App/managedEnvironments/<cae_name>/managedCertificates/<certificate_name>": parsing segment "staticCertificates": parsing the Certificate ID: the segment at position 8 didn't match

│ Expected a Certificate ID that matched:

│ > /subscriptions/12345678-1234-9876-4563-123456789012/resourceGroups/example-resource-group/providers/Microsoft.App/managedEnvironments/managedEnvironmentValue/certificates/certificateValue

│ However this value was provided:

│ > /subscriptions/<subscription_id>/resourceGroups/<rg_name>/providers/Microsoft.App/managedEnvironments/<cae_name>/managedCertificates/<certificate_name>

│ The parsed Resource ID was missing a value for the segment at position 8
│ (which should be the literal value "certificates").

Expected Behaviour

I would expect the managed certificate ID to be ignored, as defined in ignore_changes... or to be correctly parsed by accepting either certificates or managedCertificates. In that case, when running terraform apply I owuld have expected to get Apply complete! Resources: 0 added, 0 changed, 0 destroyed. as the container app was already in the expected state.

As a side note, when the domain was initially added, the certificate was not generated and the domain wasn't bound (certificate_binding_type remains set to Disabled). I don't know if that's expected behaviour, but it's slightly unfortunate as it doesn't enable fully automated deployment.

Actual Behaviour

When running terraform apply the first time, the custom domain name was deployed.

I then triggered the certificate generation and bound it manually.

When I ran terraform apply again, instead of getting Apply complete! Resources: 0 added, 0 changed, 0 destroyed. as expected, I got the error message above.

Steps to Reproduce

  1. Run terraform apply to create a new container app with a custom domain name
  2. Check that the domain name is created
  3. Manually trigger the creation of a managed certificate and bind it in Azure Portal
  4. Run terraform apply again

Important Factoids

No response

References

This is a newly added resource: #25356

@sinbai
Copy link
Contributor

sinbai commented Apr 30, 2024

Hi @ghjklw thanks for opening this issue. Per the information provided, I assume that the error is reported by resource azurerm_container_app_environment_certificate, correct? If so, I assume that this is by design because the Rest API corresponding to "azurerm_container_app_environment_certificate" is Certificates, and manually trigger the creation of a managed certificate and bind it in Azure Portal in step3, the created managed certificate corresponding Rest API is ManagedCertificates.As far as I know, currently, Terraform does not have resources to support the ManagedCertificates API.

@ghjklw
Copy link
Author

ghjklw commented Apr 30, 2024

Hi @sinbai,

Thanks for looking into it! No I am not using this resource at all, only azurerm_container_app_custom_domain. If I'm not mistaken, the whole point of #25356 was to be able to use Azure Managed Certificates, and I did apply ignore_changes = [certificate_binding_type, container_app_environment_certificate_id] as explicitly mentioned in the documentation.

@CRidge
Copy link

CRidge commented May 6, 2024

Hi @ghjklw and @sinbai,

I'm seeing the same issue. It would be great to know if the manual step of creating the managed certificate is by design, or if it is supposed to be created automatically on the first deployment - and if so, if it is normal for it to take some time?

@dhduvall
Copy link

dhduvall commented May 6, 2024

I'll note that the test added in #25356 only checks for the existence of the domain, not whether the certificate is set up properly.

@fabrideci
Copy link

I confirm the behaviour described above, in opposite to what's written in the documentation:

"Omit this value (container_app_environment_certificate_id) if you wish to use an Azure Managed certificate. You must create the relevant DNS verification steps before this process will be successful."

The managed certificate does never get created by Terraform, even though all DNS verification steps are run before.

It's useless having to generate the certificate through the portal afterwards in a fully automated infra-as-code environment as where Terraform is supposed to help with.

@brett-matson
Copy link

I'm getting this error too. Does anyone have a work around, other than not using managed certificates?

@fabrideci
Copy link

I'm getting this error too. Does anyone have a work around, other than not using managed certificates?

Our workaround is using the AzAPI provider to create both the Container App and Managed Certificate. Unfortunately, due to technical constraints on both the actual Azure API and the AzAPI provider itself, we must:

  1. Create the Container App first, with Custom Domain/s configured and bound to temporary self-signed certificate/s
  2. Create the Managed Certificate/s (requires Custom Domain/s to be there in advance - see point 1)
  3. Update the Container App to edit the Custom Domain/s and bind them to the Managed Certificate/s.

Note that azapi_update_resource requires to pass the entire JSON body of the Container App configuration again, so it won't definitely be nice as you'll have duplicated code to maintain.

@brett-matson
Copy link

@fabrideci Yikes, thanks for that. This isn't ideal.

@LynnAU
Copy link

LynnAU commented May 14, 2024

I've had success using a local-exec provisioner to run a bash script which calls out to azcli to provision the managed cert and bind it to the domain for me.

To bind the custom domain to the container app, use the following command:
az containerapp hostname add -g <resource_group> -n <container_app_name> --hostname <custom_domain>
This will bind the hostname to the app without requiring a certificate to do so.

Then to create a managed certificate:
az containerapp env certificate create -g <resource_group> -n <container_app_env_name> --hostname <custom_domain> --validation-method CNAME You can change the validation method to your preference.

And finally to bind the cert to the domain (providing the domain is already bound to the container app):
az containerapp hostname bind -g <resource_group> -n <container_app_name> --hostname <custom_domain> --environment <container_app_env_name> --certificate <certificate_name>

I've set my script up to work for adding/removing custom domains on already deployed container apps, here's a link to the gist if anyone is curious: https://gist.github.com/LynnAU/131426847d2793c76e36548f9937f966
The nice part about the script is, it will bind the custom domain to the container app if it's not already, import an existing managed certificate if one exists, and exit if the managed certificate is already bound.

@brett-matson
Copy link

@LynnAU Wow, thanks. That's very helpful.

@sinbai
Copy link
Contributor

sinbai commented May 15, 2024

Hi @ghjklw I have submitted PR to fix this issue. Could you please track it for more updates?

@rcskosir rcskosir added the bug label May 17, 2024
@rodyvansambeek
Copy link

rodyvansambeek commented May 24, 2024

This is a blocking issue for me too. I used a manual binding using the portal and now my Terraform scripts are failing. Is there a workaround available to be able to run the scripts, while using the manual binding to the custom domain?

Edited -> After trying out the manual "az containerapp" way described by @LynnAU to add the certificate and binding, it works. Somehow terraform does not recognizes that new certificate and binding as something to destroy on the next "terraform apply".

@1TT-Chris
Copy link

Thanks to @LynnAU for the awesome script. I had to make some tweaks to get it working. The final command to bind the hostname and certificate failed for me. Running it with --debug revealed that it fails because, when only specifying a certificate name, the Azure CLI only looks in the "certificates" resource namespace when managed certificates live in "managedCertificates".

The workaround was to use the managed certificate resource ID, rather than the name.

I've left a note on the gist, and raised a bug for the CLI issue here: Azure/azure-cli#29119

@thomasdewulf
Copy link

thomasdewulf commented Jun 11, 2024

I've been having the same issue, the script provided by @LynnAU works, but when the state gets refreshed on subsequent runs the error still occurs. Any ideas?

resource "azurerm_container_app_custom_domain" "app_domain" {
  container_app_id = azurerm_container_app.web_app.id
  name = "my.domain.com"
  provisioner "local-exec" {
    command = "chmod +x ./scripts/bind-custom-domain.sh; ./scripts/bind-custom-domain.sh" # Based on working dir set in github action
    environment = {
      RESOURCE_GROUP = data.azurerm_resource_group.resource_group.name
      CONTAINER_APP_ENV_NAME = azurerm_container_app_environment.container_app_environment.name
      CUSTOM_DOMAIN = "my.domain.com"
      CONTAINER_APP_NAME = azurerm_container_app.web_app.name
    }
  }
}

@bolsteryorick
Copy link

@thomasdewulf For this workaround you need to forgo using the azurerm_container_app_custom_domain resource entirely and use a null_resource resource to execute the local-exec in. If you want it to also destroy the domain on a terraform destroy you will have to add a script that does that with a when argument of 'destroy'

@LynnAU
Copy link

LynnAU commented Jun 12, 2024

Glad you liked the script guys, I'm back with some updates.

I've tweaked my create script to add some more checks prior to provisioning a certificate and included a destroy script for cleanup. The gist now also contains some terraform files to execute the scripts for you, just need to place the main.tf and variables.tf files in a folder and use it as a custom module in your deployments.

I've added a DNS check using dig to verify the asuid TXT record for the custom domain exists before main execution, this is because we use Cloudflare to handle our DNS and when adding custom domains after an already provisioned Container App, the execution of the script in our deployment pipeline happens within seconds of updating the DNS records (so I needed to delay my script for propagation). Feel free to comment out that section if it's not to your liking or needs, hell do what you want with the script lol...

@thomasdewulf bolsteryorick is correct, you need to use a null_resource terraform resource for the scripts to work correctly, check my gist linked above if you want to copy what I have. The files in the gist is what my module contains for provisioning custom domains.

@1TT-Chris Cheers for leaving feedback, weird that you had to change those lines. I wonder if it's to do with the AZCLI version I use, which is 2.53.0 currently. I've added this as a caveat to the gist's README incase anybody else encounters the problem.

@fabrideci
Copy link

Is there any update on this? Still it's not possible to use azurerm_container_app_custom_domain with managed certificates after months of being reported.

@github-actions github-actions bot added this to the v4.3.0 milestone Sep 20, 2024
Copy link

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 21, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet