Local Alpine and Debian repositories adding project_environments to diff, causing changes

[zamanh]

Describe the bug
When running a plan against an already existing local Alpine and Debian repository, the diff is showing changes, where project_environments is going to be added. Even after applying, the diff doesn't change (ignore keypair_ref attribute, that is something I am trying to add)

Part of diff that shows Alpine

Part of diff that shows Debian

Requirements for and issue

• A description of the bug
• A fully functioning terraform snippet that can be copy&pasted (no outside files or ENV vars unless that's part of the issue). If this is not supplied, this issue will likely be closed without any effort expended.

resource "artifactory_keypair" "some-keypairGPG1" {
  pair_name         = "some-keypair${random_id.randid.id}"
  pair_type         = "GPG"
  alias             = "foo-alias1"
  private_key       = file("samples/gpg.priv")
  public_key        = file("samples/gpg.pub")
  lifecycle {
    ignore_changes  = [
      private_key,
      passphrase,
    ]
  }
}

resource "artifactory_local_debian_repository" "my-debian-repo" {
  key                       = "my-debian-repo"
  primary_keypair_ref       = artifactory_keypair.some-keypairGPG1.pair_name
  index_compression_formats = ["bz2", "lzma", "xz"]
  trivial_layout            = true
  depends_on                = [artifactory_keypair.some-keypairGPG1]
}

resource "artifactory_keypair" "some-keypairRSA" {
  pair_name         = "some-keypair"
  pair_type         = "RSA"
  alias             = "foo-alias"
  private_key       = file("samples/rsa.priv")
  public_key        = file("samples/rsa.pub")
  lifecycle {
    ignore_changes  = [
      private_key,
      passphrase,
    ]
  }
}

resource "artifactory_local_alpine_repository" "terraform-local-test-alpine-repo-basic" {
  key                 = "terraform-local-test-alpine-repo-basic"
  primary_keypair_ref = artifactory_keypair.some-keypairRSA.pair_name

  depends_on          = [artifactory_keypair.some-keypairRSA]
}

• Your version of artifactory (you can curl it at $host/artifactory/api/system/version: 7.98.13
• Your version of terraform: 1.98
• Your version of terraform provider: 12.8.3/12.8.4

Expected behavior

• project_environments shouldn't ever appear in the diff when not being set

[alexhung]

@zamanh Thanks for the report! I've added this to our plan to investigate and fix.

I suspect the project_environments diff is a red herring. There may be other attributes that are different but TF is not showing (known issue with TF core).

[alexhung]

@zamanh Using the following configuration, I was unable to reproduce your issue right now. Is this issue occurring consistently? Would you be able to execute this in debug mode and provide the log?

terraform {
  required_providers {
    artifactory = {
      source  = "jfrog/artifactory"
      version = "12.8.3"
    }
  }
}

provider "artifactory" {
  //  supply ARTIFACTORY_ACCESS_TOKEN / JFROG_ACCESS_TOKEN / ARTIFACTORY_API_KEY and ARTIFACTORY_URL / JFROG_URL as env vars
}

resource "random_id" "randid" {
  byte_length = 16
}

resource "artifactory_keypair" "some-keypairRSA" {
  pair_name   = "some-keypairfoo"
  pair_type   = "RSA"
  private_key = file("samples/rsa.priv")
  public_key  = file("samples/rsa.pub")
  alias       = "foo-aliasfoo"
  passphrase  = "some-passphrase"
}

resource "artifactory_keypair" "some-keypairGPG1" {
  pair_name   = "some-keypair${random_id.randid.id}"
  pair_type   = "GPG"
  alias       = "foo-alias1"
  private_key = file("samples/gpg.priv")
  public_key  = file("samples/gpg.pub")
  passphrase  = "some-passphrase"
}

resource "artifactory_keypair" "some-keypairGPG2" {
  pair_name   = "some-keypair4${random_id.randid.id}"
  pair_type   = "GPG"
  alias       = "foo-alias2"
  private_key = file("samples/gpg.priv")
  public_key  = file("samples/gpg.pub")
  passphrase  = "some-passphrase"
}

resource "artifactory_local_debian_repository" "my-debian-repo" {
  key                       = "my-debian-repo"
  primary_keypair_ref       = artifactory_keypair.some-keypairGPG1.pair_name
  secondary_keypair_ref     = artifactory_keypair.some-keypairGPG2.pair_name
  index_compression_formats = ["bz2", "lzma", "xz"]
  trivial_layout            = true
}

resource "artifactory_local_alpine_repository" "my-alpine-repo" {
  key                 = "my-alpine-repo"
  primary_keypair_ref = artifactory_keypair.some-keypairRSA.pair_name
}

[erlarson85]

I can confirm as well, a long standing config that involves many local repo's are attempting to add the project_environments attribution to repos that do not have them defined.

# artifactory_local_rpm_repository.tableau will be updated in-place
  ~ resource "artifactory_local_rpm_repository" "tableau" {
      ~ id                         = "tableau" -> (known after apply)
      + project_environments       = (known after apply)

This has been happening to me in ~ 12.8.1 on our Jfrog Cloud Instance, we Terraform our deployment and have not set a value for the attribs.

[alexhung]

@zamanh @erlarson85 If you call the API directly (e.g. GET /artifactory/api/repositories/tableau), does the environments field contain any values?

[alexhung]

@zamanh @erlarson85 Can you share the Terraform state for the resource using terraform state show? e.g. terraform state show 'artifactory_local_rpm_repository.tableau'.

Does the state contain project_environments attribute? If so, what is the value?

[erlarson85]

# artifactory_local_rpm_repository.tableau:
resource "artifactory_local_rpm_repository" "tableau" {
    archive_browsing_enabled   = false
    blacked_out                = false
    calculate_yum_metadata     = true
    download_direct            = false
    enable_file_lists_indexing = false
    id                         = "tableau"
    includes_pattern           = "**/*"
    key                        = "tableau"
    priority_resolution        = false
    repo_layout_ref            = "simple-default"
    xray_index                 = true
    yum_root_depth             = 0
}

Only 1 repo we have is using project_env which is in another Project. The above (and 29 other local repos) have no project or environment defined. 7 out of those 31 repos are trying to update the ID as well as adding the proj_env value for every plan/apply.

Artifactory 7.106.3 Cloud

• Installed jfrog/project v1.9.3
• Installed jfrog/artifactory v12.8.3
• Installed jfrog/platform v2.2.1

resource "artifactory_local_rpm_repository" "tableau" {
  key                    = "tableau"
  repo_layout_ref        = "simple-default"
  includes_pattern       = "**/*"
  calculate_yum_metadata = true
  xray_index             = true
}

[erlarson85]

Found the link to 6 out of the 7 repos being local_rpm repos. I have 1 local_debian repo that is also doing this.

tf state show 'artifactory_local_debian_repository.as-debian'

# artifactory_local_debian_repository.as-debian:
resource "artifactory_local_debian_repository" "as-debian" {
    archive_browsing_enabled  = false
    blacked_out               = false
    ddeb_supported            = false
    download_direct           = false
    id                        = "as-debian"
    includes_pattern          = "**/*"
    index_compression_formats = [
        "bz2",
    ]
    key                       = "as-debian"
    priority_resolution       = false
    repo_layout_ref           = "simple-default"
    trivial_layout            = false
    xray_index                = true
}

# artifactory_local_debian_repository.as-debian will be updated in-place
  ~ resource "artifactory_local_debian_repository" "as-debian" {
      ~ id                        = "as-debian" -> (known after apply)
      + project_environments      = (known after apply)
        # (11 unchanged attributes hidden)
    }

Other local repos, like local_maven, local_generic, etc are not an issue.

One of the local_rpm repos not affected:

tf state show 'artifactory_local_rpm_repository.as-amazon-linux'

# artifactory_local_rpm_repository.as-amazon-linux:
resource "artifactory_local_rpm_repository" "as-amazon-linux" {
    archive_browsing_enabled   = false
    blacked_out                = false
    calculate_yum_metadata     = true
    description                = "RPMs for Amazon Linux servers"
    download_direct            = false
    enable_file_lists_indexing = true
    id                         = "as-amazon-linux"
    includes_pattern           = "**/*"
    key                        = "as-amazon-linux"
    priority_resolution        = false
    repo_layout_ref            = "simple-default"
    xray_index                 = true
    yum_root_depth             = 2
}

[zamanh]

@zamanh @erlarson85 Can you share the Terraform state for the resource using terraform state show? e.g. terraform state show 'artifactory_local_rpm_repository.tableau'.

Does the state contain project_environments attribute? If so, what is the value?

@alexhung I see no project_environments attribute after running that command:

resource "artifactory_local_debian_repository" "debian" {
    archive_browsing_enabled  = false
    blacked_out               = false
    ddeb_supported            = false
    description               = null
    download_direct           = false
    excludes_pattern          = null
    id                        = "debian"
    includes_pattern          = "**/*"
    index_compression_formats = [
        "bz2",
    ]
    key                       = "debian"
    notes                     = null
    priority_resolution       = false
    project_key               = null
    property_sets             = [
        "artifactory",
    ]
    repo_layout_ref           = "simple-default"
    trivial_layout            = false
    xray_index                = false
}

[zamanh]

@zamanh @erlarson85 If you call the API directly (e.g. GET /artifactory/api/repositories/tableau), does the environments field contain any values?

@alexhung I see no environments field in the API response.

{
  "key" : "debian",
  "packageType" : "debian",
  "description" : "",
  "notes" : "",
  "includesPattern" : "**/*",
  "excludesPattern" : "",
  "repoLayoutRef" : "simple-default",
  "signedUrlTtl" : 90,
  "enableComposerSupport" : false,
  "enableNuGetSupport" : false,
  "enableGemsSupport" : false,
  "enableNpmSupport" : false,
  "enableBowerSupport" : false,
  "enableChefSupport" : false,
  "enableCocoaPodsSupport" : false,
  "enableConanSupport" : false,
  "enableDebianSupport" : true,
  "debianTrivialLayout" : false,
  "ddebSupported" : false,
  "enablePypiSupport" : false,
  "enablePuppetSupport" : false,
  "enableDockerSupport" : false,
  "dockerApiVersion" : "V2",
  "blockPushingSchema1" : true,
  "forceNugetAuthentication" : false,
  "enableNormalizedVersion" : false,
  "forceP2Authentication" : false,
  "forceConanAuthentication" : false,
  "enableVagrantSupport" : false,
  "enableGitLfsSupport" : false,
  "enableDistRepoSupport" : false,
  "dockerProjectId" : "",
  "priorityResolution" : false,
  "environments" : [ ],
  "checksumPolicyType" : "client-checksums",
  "handleReleases" : true,
  "handleSnapshots" : true,
  "maxUniqueSnapshots" : 0,
  "maxUniqueTags" : 0,
  "snapshotVersionBehavior" : "unique",
  "suppressPomConsistencyChecks" : false,
  "blackedOut" : false,
  "propertySets" : [ "artifactory" ],
  "optionalIndexCompressionFormats" : [ "bz2" ],
  "archiveBrowsingEnabled" : false,
  "calculateYumMetadata" : false,
  "enableFileListsIndexing" : false,
  "yumRootDepth" : 0,
  "dockerTagRetention" : 1,
  "enableComposerV1Indexing" : false,
  "terraformType" : "MODULE",
  "encryptStates" : true,
  "downloadRedirect" : false,
  "cdnRedirect" : false,
  "cargoInternalIndex" : false,
  "cargoAnonymousAccess" : false,
  "xrayIndex" : false,
  "xrayDataTtl" : 90,
  "rclass" : "local"
}

[alexhung]

@erlarson85 @zamanh Thanks for the payloads and states! Let me take a deeper look. I still think the project_environments state drift is a red herring. Another attribute may be triggering this issue.

[alexhung]

@erlarson85 @zamanh BTW, 12.8.4 has just been released with a fix for index_compression_formats state drift for Debian repo.

[zamanh]

@erlarson85 @zamanh BTW, 12.8.4 has just been released with a fix for index_compression_formats state drift for Debian repo.

Will take that upgrade, thank you so much for your swift actions @alexhung and ever continued development. Highly appreciated! :)

[erlarson85]

I've tried the plan, apply and plan again. (with 12.8.4) The trouble repos I have are still trouble and attempting to update with each plan, including the Debian repo, which in my env isn't trying to update the index_compression_formats

[zamanh]

@zamanh Using the following configuration, I was unable to reproduce your issue right now. Is this issue occurring consistently? Would you be able to execute this in debug mode and provide the log?

terraform {
required_providers {
artifactory = {
source = "jfrog/artifactory"
version = "12.8.3"
}
}
}

provider "artifactory" {
// supply ARTIFACTORY_ACCESS_TOKEN / JFROG_ACCESS_TOKEN / ARTIFACTORY_API_KEY and ARTIFACTORY_URL / JFROG_URL as env vars
}

resource "random_id" "randid" {
byte_length = 16
}

resource "artifactory_keypair" "some-keypairRSA" {
pair_name = "some-keypairfoo"
pair_type = "RSA"
private_key = file("samples/rsa.priv")
public_key = file("samples/rsa.pub")
alias = "foo-aliasfoo"
passphrase = "some-passphrase"
}

resource "artifactory_keypair" "some-keypairGPG1" {
pair_name = "some-keypair${random_id.randid.id}"
pair_type = "GPG"
alias = "foo-alias1"
private_key = file("samples/gpg.priv")
public_key = file("samples/gpg.pub")
passphrase = "some-passphrase"
}

resource "artifactory_keypair" "some-keypairGPG2" {
pair_name = "some-keypair4${random_id.randid.id}"
pair_type = "GPG"
alias = "foo-alias2"
private_key = file("samples/gpg.priv")
public_key = file("samples/gpg.pub")
passphrase = "some-passphrase"
}

resource "artifactory_local_debian_repository" "my-debian-repo" {
key = "my-debian-repo"
primary_keypair_ref = artifactory_keypair.some-keypairGPG1.pair_name
secondary_keypair_ref = artifactory_keypair.some-keypairGPG2.pair_name
index_compression_formats = ["bz2", "lzma", "xz"]
trivial_layout = true
}

resource "artifactory_local_alpine_repository" "my-alpine-repo" {
key = "my-alpine-repo"
primary_keypair_ref = artifactory_keypair.some-keypairRSA.pair_name
}

I have a slightly more involved example than this, which has some sensitive information inside it, if it's really required (regarding the debug log) I can try my best in depicting it and can upload it somewhere private where you can get it?

Yes it does happen consistently to confirm.

I am now also seeing it across other repos, in my setup which could be a state drifting issue that may be pertinent to TF itself?

# artifactory_remote_docker_repository.docker (docker) will be updated in-place
~ resource "artifactory_remote_docker_repository" "docker" {
    ~ id                                    = "docker" -> (known after apply)
    + project_environments                  = (known after apply)
      # (46 unchanged attributes hidden)
  }

# artifactory_remote_maven_repository.jenkins (jenkins) will be updated in-place
~ resource "artifactory_remote_maven_repository" "jenkins" {
  ~ id                                    = "jenkins" -> (known after apply)
  + project_environments                  = (known after apply)
    # (48 unchanged attributes hidden)

  # Not sure if below is related to this issue or another, or just my setup
  - - content_synchronisation {
      - enabled                         = true -> null
      - properties_enabled              = false -> null
      - source_origin_absence_detection = false -> null
      - statistics_enabled              = false -> null
    }
}

# artifactory_local_generic_repository.generic will be updated in-place
~ resource "artifactory_local_generic_repository" "generic" {
    + description              = "generic"
    ~ id                       = "generic" -> (known after apply)
    + project_environments     = (known after apply)
      # (13 unchanged attributes hidden)
  }

 # artifactory_remote_debian_repository.debian (debian) will be updated in-place
  ~ resource "artifactory_remote_debian_repository" "debian" {
      ~ description                           = " (local file cache)" -> "(local file cache)"
      ~ id                                    = "debian" -> (known after apply)
      + project_environments                  = (known after apply)
        # (38 unchanged attributes hidden)
    }

  # artifactory_remote_docker_repository.docker-bintray (docker-bintray) will be updated in-place
  ~ resource "artifactory_remote_docker_repository" "docker-bintray" {
      ~ description                           = " (local file cache)" -> "(local file cache)"
      ~ external_dependencies_enabled         = true -> false
      ~ external_dependencies_patterns        = [
          + "**",
        ]
      ~ id                                    = "docker-bintray" -> (known after apply)
      + project_environments                  = (known after apply)
        # (42 unchanged attributes hidden)
    }

  # artifactory_remote_gems_repository.gems (gems) will be updated in-place
  ~ resource "artifactory_remote_gems_repository" "gems" {
      ~ description                           = " (local file cache)" -> "(local file cache)"
      ~ id                                    = "gems" -> (known after apply)
      + project_environments                  = (known after apply)
        # (41 unchanged attributes hidden)
    }

  # artifactory_remote_generic_repository.remote-generic (remote-generic) will be updated in-place
  ~ resource "artifactory_remote_generic_repository" "remote-generic" {
      + description                           = "(local file cache)"
      ~ id                                    = "remote-generic" -> (known after apply)
      + project_environments                  = (known after apply)
        # (40 unchanged attributes hidden)
    }

  # artifactory_remote_pub_repository.dartlang (dartlang) will be updated in-place
  ~ resource "artifactory_remote_pub_repository" "dartlang" {
      ~ description                           = " (local file cache)" -> "(local file cache)"
      ~ id                                    = "dartlang" -> (known after apply)
      + project_environments                  = (known after apply)
        # (38 unchanged attributes hidden)
    }

  # artifactory_remote_pypi_repository.python (python) will be updated in-place
  ~ resource "artifactory_remote_pypi_repository" "esg-pypi-dirty-remote-pythonhosted" {
      ~ description                           = " (local file cache)" -> "(local file cache)"
      ~ id                                    = "python" -> (known after apply)
      + project_environments                  = (known after apply)
        # (41 unchanged attributes hidden)
    }

  # artifactory_remote_rpm_repository.centos (centos) will be updated in-place
  ~ resource "artifactory_remote_rpm_repository" "centos" {
      ~ description                           = " (local file cache)" -> "(local file cache)"
      ~ id                                    = "centos" -> (known after apply)
      + project_environments                  = (known after apply)
        # (38 unchanged attributes hidden)
    }

  # artifactory_local_cargo_repository.cargo (cargo) will be updated in-place
  ~ resource "artifactory_local_cargo_repository" "cargo" {
      ~ description              = " change" -> "Change"
      ~ id                       = "cargo" -> (known after apply)
      + project_environments     = (known after apply)
        # (14 unchanged attributes hidden)
    }

  # artifactory_local_docker_v2_repository.local-docker (local-docker) will be updated in-place
  ~ resource "artifactory_local_docker_v2_repository" "local-docker" {
      ~ api_version              = "V2" -> (known after apply)
      ~ block_pushing_schema1    = true -> false # Can be ignored => self inflicted
      ~ id                       = "local-docker" -> (known after apply)
      + project_environments     = (known after apply)
      ~ tag_retention            = 1 -> 0 # Can be ignored => self-inflicted
        # (14 unchanged attributes hidden)
    }

  # artifactory_local_maven_repository.local-maven (local-maven) will be updated in-place
  ~ resource "artifactory_local_maven_repository" "local-maven" {
      ~ id                              = "local-maven" -> (known after apply)
      + project_environments            = (known after apply)
      ~ snapshot_version_behavior       = "non-unique" -> "unique" # ignore, missed setting in my setup
        # (18 unchanged attributes hidden)
    }

  # artifactory_local_nuget_repository.local-nuget (local-nuget) will be updated in-place
  ~ resource "artifactory_local_nuget_repository" "local-nuget" {
      - description                = " change." -> "change"
      ~ id                         = "local-nuget" -> (known after apply)
      + project_environments       = (known after apply)
      ~ repo_layout_ref            = "simple-default" -> "nuget-default" # Ignore this - mistake of my own
        # (13 unchanged attributes hidden)
    }

  # artifactory_remote_alpine_repository.remote-alpine (remote-alpine) will be updated in-place
  ~ resource "artifactory_remote_alpine_repository" "remote-alpine" {
      ~ description                           = " (local file cache)" -> "(local file cache)"
      ~ id                                    = "remote-alpine" -> (known after apply)
      + project_environments                  = (known after apply)
        # (38 unchanged attributes hidden)
    }

[alexhung]

@zamanh Thanks for the detail message. I think some of them are new bugs

For example, " (local file cache)" -> "(local file cache)" this is either introduced when I migrated the local repo to Plugin Framework and forgot to include the leading space, or Artifactory introduced this new leading space recently. Either way, relatively easy to fix.

For repo_layout_ref with Nuget repo, that is likely a mapping bug from the migration process. Also simple to fix.

Similar with tag_retention with Docker repo.

[zamanh]

@zamanh Thanks for the detail message. I think some of them are new bugs

For example, " (local file cache)" -> "(local file cache)" this is either introduced when I migrated the local repo to Plugin Framework and forgot to include the leading space, or Artifactory introduced this new leading space recently. Either way, relatively easy to fix.

For repo_layout_ref with Nuget repo, that is likely a mapping bug from the migration process. Also simple to fix.

Similar with tag_retention with Docker repo.

@alexhung Ignore the local file cache comments, that is me and also the repo layout ref is from me too. Everything else is not.

[alexhung]

@zamanh We definitely had something to deal with this diff issue in the old SDKv2 code: https://github.com/jfrog/terraform-provider-artifactory/blob/master/pkg/artifactory/resource/repository/remote/remote.go#L844

[alexhung]

@zamanh For snapshot_version_behavior = "non-unique" -> "unique", this hasn't changed from old code to new code.

[alexhung]

@zamanh I dug into the remote repo description diff more and I think what happened is this:

• the REST API used to return the description (whether it is empty or some text) appended with (local file cache). Hence the legacy code to ignore the difference.
• Sometime in the last year(?), the API was fixed to not append that string. I have not been able to find any reference to this change in the release notes.
• web UI still show it though. See below.

[zamanmh1]

@zamanh I dug into the remote repo description diff more and I think what happened is this:

• the REST API used to return the description (whether it is empty or some text) appended with (local file cache). Hence the legacy code to ignore the difference.
• Sometime in the last year(?), the API was fixed to not append that string. I have not been able to find any reference to this change in the release notes.
• web UI still show it though. See below.

Hi @alexhung, I would back out any changes made to the docker repos, I've just looked at my code and I was trying to set the "correct" defaulted values on my Artifactory, I somehow didn't declare the local docker repo properly in my terraform so I apologise for the confusion there.

Just to confirm, the defaults should be as stated in the docs: https://registry.terraform.io/providers/jfrog/artifactory/latest/docs/resources/local_docker_v2_repository

[zamanh]

@zamanh For snapshot_version_behavior = "non-unique" -> "unique", this hasn't changed from old code to new code.

Again, something local to my setup, I didn't declare, now resolved.

[erlarson85]

Confirmed this resolves project_enviroments updates.

[erlarson85]

Spoke a bit too soon sadly:

artifactory_local_docker_v2_repository.this["ke-rtic-cubic"] will be updated in-place

~ resource "artifactory_local_docker_v2_repository" "this" {
~ api_version = "V2" -> (known after apply)
~ id = "ke-rtic-cubic" -> (known after apply)
~ project_environments = [
- "DEV",
]
# (13 unchanged attributes hidden)
}
These are the first Docker repo's I have created but all 4 are asking for the same changes using 12.9.0
It is resolved for RPM and DEB local repos though

[alexhung]

@erlarson85 Is this repo assigned to a project? One reason the project_environments attribute is set with DEV is because a repo assigned to a project by default also gets DEV environment.