Version 2 roadmap

[QuentinBrosse]

Dear Cloud Rider,

In order to improve our provider and support our new products we will make a significant refactoring on the whole codebase to finally release a version 2.

Naturally, we are taking the backward compatibility issues very seriously. Therefore we will follow a process designed to make it possible for you to execute a smooth transition from your current version to the v2.

Roadmap

Step 1: New minor versions [Non-Breaking Change]

This version will deprecate some elements of this provider but will not create any breaking change. It will allow you to prepare your configuration for the v2 without breaking your actual configuration.

Here is a list of actions we will take - all the details will be discussed in there dedicated sub-issue:

v1.10.0: milestone ✅

• Update the terraform SDK to 0.12 [#124]
• Fix all acceptance tests [Fix all acceptance tests #126]

v1.11.0: milestone ✅

• Refactor the internal meta interface to prepare the transition to our new Go SDK [Refactor the configuration interface #131]
• Update the provider schema and introduce the new Scaleway configuration (it will be consistent between all our developer tools) [Update the provider schema and introduce the new Scaleway configuration #140]
• Rename all current resources following this scheme scaleway_{product}_{resource}. The old resources will be kept but deprecated. This will allow the creation of resources for our new products [Recode all current resources following this scheme #143]
• Add baremetal resources [fix: baremetal doc #241, feat: add baremetal #228]
• Tests

v1.12.0: milestone ✅

• Add k8s beta resources [feat: k8s resource #258]
• Add lb resources (lb, backend, frontend) [feat(lb): add lb beta resource #278, WIP: feat(lb): add lb backend beta resource #284, feat(lb): add lb frontend beta resource #305 ]
• Bug fixes and minor features (more infos)

v1.13.0: milestone ✅

• Add lb backend healthcheck [Add support for scaleway_lb_backend healthcheck #309]
• Add database resource [feat(rdb): instance resource #331]
• Rework all scaleway data sources [Recode data sources and add missing ones #225]

v1.14.0: milestone ✅

• Add data-source/scaleway_registry_image_beta: [feat(registry): add image data source #409]
• Add data-source/scaleway_registry_namespace_beta: [feat(registry): add namespace data source #408]
• Add resource/scaleway_lb_certificate_beta [feat: added certificate to load-balancer #396]
• Add resource/scaleway_registry_namespace_beta: [feat(registry): add scaleway_registry_namespace_beta resource #330] [fix(registry): add registry back to list of resources #407]
• Add resource/scaleway_instance_security_group_rules [feat: add scaleway_instance_security_group_rules resource #391]

v1.15.0: milestone ✅

TBA

v1.16.0: milestone ✅

TBA

v1.17.0: milestone

TBA

Step 2: New major version v2.0.0 milestone

The version will clean all the deprecated elements introduced in the v1.10.X.

Here is a list of actions we will take - all the details will be discussed in there dedicated sub-issue:

• 1. Remove deprecated provider schema fields [#tba]
• 2. Remove deprecated configuration file and environment variables [#tba]
• 3. Remove deprecated resources (mainly for the instance product) [#tba]
• 4. Change the configuration precedence [#tba]
• [ x] 5. Remove support for dynamic ip from server resource [rejected]

[balupton]

Now that terraform 0.12 support has landed can a release be made for that, with the rest of the stuff above being in an additional release. As the 0.12 support is quite urgent to get out to users

[QuentinBrosse]

@balupton we are totally agree with you. We will release a new minor version next week :)

[QuentinBrosse]

@balupton Hi,

The v0.10.0 is now released! :)

[hazcod]

So I was trying to use terraform to setup VC1S instances, but those are not instrumentable by API according to support for stock reasons... (API always returns Quota reached for every instance)

Is there an ETA for this provider for the new instances, since it's unuseable right now?
Or would v0.10.0 provide a new instance type?

[jerome-quere]

Hello @hazcod when using v1.10.0 you can use type parameter to select a server type (https://www.terraform.io/docs/providers/scaleway/r/server.html#type). You can find availables server types in our web console (https://console.scaleway.com/instance/servers/create).

[hazcod]

Can confirm it works with DEV1-S instead of VC1S.

[QuentinBrosse]

Hi,

The v0.11.0 is now released! 🎉

[mdawar]

Hi, I have a question please about the new changes, the scaleway_instance_server resource doesn't seem to have a way to get the public IPv6 address, in version 0.10 we had public_ipv6 but starting version 0.11 we only have public_ip for the IPv4 only, any way to get the IPv6 address in the new version?

[QuentinBrosse]

Hi @mdawar,
Thanks for reporting it!

I just created an issue to expose a public_ipv6 attribute on the scaleway_instance_server resource. It will be released in the v1.12.0. 🙂

[mdawar]

Hi @QuentinBrosse, thanks for the quick response, I have another question please, why the scaleway_ssh_key resource will be removed in v2? I don't think that it's better to upload the keys in the console as suggested by the migration guide, being able to manage everything using Terraform is critical especially in cases where everything is automated, this is going to make a step backward in my opinion, all the other cloud providers provide a way to manager SSH keys using Terraform.

[QuentinBrosse]

Hi @mdawar,
The scaleway_ssh_key is deprecated and will be removed in v2.0. This resource is replaced by the new scaleway_account_ssh_key. You can find this info in the deprecated scaleway_ssh_key.

I just see that the migration guide is not up to date. I have just created #287 to fix it, sorry for the inconvenience.
cc @hazcod @gmile

[QuentinBrosse]

Hi,

The v0.12.0 is now released! 🎉

Changelog

[krzysztofantczak]

So, interesting case:

Terraform v0.12.16
+ provider.scaleway v1.12.0

Initial resource:

resource "scaleway_instance_security_group" "bastion-ams1" {
  inbound_default_policy = "drop"

  name        = "bastion-ams1"
  description = "bastion-ams1"

  inbound_rule {
    action = "accept"
    port   = 655
  }

  inbound_rule {
    action   = "accept"
    protocol = "UDP"
    port     = 655
  }

  inbound_rule {
    action   = "accept"
    protocol = "UDP"
    port     = 500
  }

  inbound_rule {
    action   = "accept"
    protocol = "UDP"
    port     = 4500
  }

  inbound_rule {
    action = "accept"
    port   = 59184
  }
}

It's all already applied and good, so after terraform plan:

No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.

Now, when I change my resource to:

resource "scaleway_instance_security_group" "bastion-ams1" {
  inbound_default_policy = "drop"

  name        = "bastion-ams1"
  description = "bastion-ams1"

  inbound_rule {
    action = "accept"
    port   = 333
  }

  inbound_rule {
    action = "accept"
    port   = 655
  }

  inbound_rule {
    action   = "accept"
    protocol = "UDP"
    port     = 655
  }

  inbound_rule {
    action   = "accept"
    protocol = "UDP"
    port     = 500
  }

  inbound_rule {
    action   = "accept"
    protocol = "UDP"
    port     = 4500
  }

  inbound_rule {
    action = "accept"
    port   = 59184
  }
}

Notice, I have only added one inbound_rule at the beginning with port 333, my terraform plan looks like this:

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # scaleway_instance_security_group.bastion-ams1 will be updated in-place
  ~ resource "scaleway_instance_security_group" "bastion-ams1" {
        description             = "bastion-ams1"
        id                      = "nl-ams-1/***** REMOVED *****"
        inbound_default_policy  = "drop"
        name                    = "bastion-ams1"
        organization_id         = "***** REMOVED *****"
        outbound_default_policy = "accept"
        zone                    = "nl-ams-1"

      ~ inbound_rule {
            action   = "accept"
          ~ port     = 655 -> 333
            protocol = "TCP"
        }
      ~ inbound_rule {
            action   = "accept"
            port     = 655
          ~ protocol = "UDP" -> "TCP"
        }
      ~ inbound_rule {
            action   = "accept"
          ~ port     = 500 -> 655
            protocol = "UDP"
        }
      ~ inbound_rule {
            action   = "accept"
          ~ port     = 4500 -> 500
            protocol = "UDP"
        }
      ~ inbound_rule {
            action   = "accept"
          ~ port     = 59184 -> 4500
          ~ protocol = "TCP" -> "UDP"
        }
      + inbound_rule {
          + action   = "accept"
          + port     = 59184
          + protocol = "TCP"
        }
    }

Plan: 0 to add, 1 to change, 0 to destroy.

I haven't put much effort to investigate this further but I'm a bit scared to touch my infra with it ;-D

[QuentinBrosse]

Hi,

The v0.13.0 is now released! 🎉

Changelog

[alekc]

We should remove beta tags from LB since they are stable now (at least according to scaleway dashboard).
Edit: Also, is there any particular reason vendor's folder has not been migrated to go.mod?

[jerome-quere]

Hello @aleck,

You are right the load-balancer product itself is stable but the terraform resources schemas are not. We are still gathering feedback from Terraform users such as yourself. Once we are confident enough that the resources schemas can be frozen we will remove the beta tag.

[alekc]

Fair enough, I am working on lb acl now, should it have the beta tag as well then?

[jerome-quere]

Thank you @alekc for trying to contribute to this project.

Adding _beta suffix would be better yes. Don't hesitate to open an MR with the resource schema first so we can discuss it before you work on the complete implementation.

[alekc]

@jerome-quere done. https://github.com/terraform-providers/terraform-provider-scaleway/pull/382

[lookfirst]

Naturally, we are taking the backward compatibility issues very seriously.

I was on terraform 0.11 and have migrated to 0.12. I have 3 servers running on scaleway that were originally deployed with TF. Everything changed!

After hours of going in circles with figuring out all the terraform and the configuration changes, now I'm stuck trying to get my tfstate updated with enough information that TF won't try to destroy all my servers.

Now that IP addresses are their own resource, I have to somehow figure out what the ID of the IP address is so that I can tf import it and I can't figure that out at all. Just some feedback... I really understand the need for progress and I appreciate all the hard work, but this is really really painful.

[george-angel]

@lookfirst you can find the ID of a "Flexible IP" by going to: https://console.scaleway.com/instance/ips "Toggle options" > "More info".

The things that have caught me off-guard were:

userdata

This now includes ssh fingerprints, which is easy enough to import and match with the remote.

image

This one is harder, since in my current state its "absent", but with the new provider, its "Required". Therefore any change requires instance to be recreated.

As a temporary solution I have ignored these two attributes:

  lifecycle {
    ignore_changes = [
      image,
      user_data,
    ]
  }

You still need to set image argument, but it can have any value.

[mdawar]

I was testing the latest version 1.13.0, I think I've found a bug that's forcing the instance to be replaced, what I'm doing:

1. Use the data source scaleway_instance_image to get the latest Debian image ID
2. Create a new instance using the Debian image ID
3. Run terraform apply to create the resources
4. Re-run terraform apply which shows me that the instance must be replaced because of a difference in the image ID which is prefixed by the zone name:

  # scaleway_instance_server.testing must be replaced
-/+ resource "scaleway_instance_server" "testing" {
      - additional_volume_ids            = [] -> null
        disable_dynamic_ip               = false
        disable_public_ip                = false
        enable_dynamic_ip                = false
        enable_ipv6                      = true
      ~ id                               = "fr-par-1/45b38255-1819-459d-a0ee-930d30e00838" -> (known after apply)
      ~ image                            = "cc9188b3-3938-47d7-b091-c9ecad1fe507" -> "fr-par-1/cc9188b3-3938-47d7-b091-c9ecad1fe507" # forces replacement
        ip_id                            = "fr-par-1/6c96b185-72eb-4433-853f-d358e6ce57f3"
      ~ ipv6_address                     = "2001:bc8:47ac:272a::1" -> (known after apply)
      ~ ipv6_gateway                     = "2001:bc8:47ac:272a::" -> (known after apply)
      ~ ipv6_prefix_length               = 64 -> (known after apply)
        name                             = "dev-testing"
      + organization_id                  = (known after apply)
      + placement_group_policy_respected = (known after apply)
      ~ private_ip                       = "10.65.62.85" -> (known after apply)
      ~ public_ip                        = "51.158.64.43" -> (known after apply)
      ~ security_group_id                = "1561d0ea-686e-4b4f-9850-05ac077df41b" -> "fr-par-1/1561d0ea-686e-4b4f-9850-05ac077df41b"
        state                            = "started"
        tags                             = [
            "terraform",
            "ytdl",
            "ytdl-dev",
        ]
        type                             = "DEV1-S"
        zone                             = "fr-par-1"

      ~ root_volume {
          ~ delete_on_termination = true -> (known after apply)
          ~ size_in_gb            = 20 -> (known after apply)
          ~ volume_id             = "950fc1a0-e9e3-4372-8702-9ea6d4627a5f" -> (known after apply)
        }
    }

Also as you can see the security_group_id also is missing the zone name.
Also there are 2 redundant attributes: disable_dynamic_ip and enable_dynamic_ip.

Update: I'm not able to create a new SSH key using scaleway_account_ssh_key, I'm always getting this error:

Error: scaleway-sdk-go: http error 403 Forbidden: Permission denied

But I can create other resources without any issues.

[QuentinBrosse]

Hi @mdawar !

The force recreation seems to be a bug, I can reproduce it. I just opened an issue to discuss and fix it: #411
Meanwhile, you can use debian image label in your server, without the data source:

resource "scaleway_instance_server" "web" {
  type = "DEV1-S"
  image = "debian"
}

Regarding your issue with scaleway_account_ssh_key resource, we have an issue opened in our Go SDK to discuss it: scaleway/scaleway-sdk-go#282. I suggest you subscribe to it.

Thanks for your report and sorry for the inconvenience. :/

[mdawar]

Hi @QuentinBrosse,

Thank you, I'll use image label for now.

What about the scaleway_instance_security_group resource? It seems to be also affected, as you can see the security_group_id is missing the zone name.

Also what about the disable_dynamic_ip and enable_dynamic_ip attributes, in the With a reserved IP section of the docs, disable_dynamic_ip is being used and down in the Arguments Reference we have only enable_dynamic_ip? The resource scaleway_instance_server seems to have both.

[QuentinBrosse]

@mdawar can you open issues for these subjects, please? :)

[alekc]

@mdawar if you are using only one region, you can also specify it in your provider configuration like this:

provider "scaleway" {
  access_key      = var.scw_access_key
  secret_key      = var.scw_secret_key
  organization_id = var.scw_organization_id
  zone            = var.scw_zone
  region          = "fr-par"
  version         = "~> 1.13"
}

In theory this should avoid the issue you are having.

[QuentinBrosse]

Hi,

This issue is dedicated to version 2 roadmap updates.
If you have a problem or a question, do not hesitate to open an issue.

Note that you can watch releases to be notified about new releases.

[QuentinBrosse]

Hi,

The v0.14.0 is now released! 🎉

Changelog

(Sorry for the delay..)

[remyleone]

I'm very happy to announce that the first release candidate of the v2 of terraform is released 🎉
This version includes new features (project id in resources, vpc, ...), massive refactoring, and more testing. To have a view of the changelog for this release go take a look here on the release page.
In particular, some of those changes are breaking changes, we wrote a guide to help you migrate as quickly as possible to the new version.

You can view this guide live on the terraform registry.