Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Renovate does not understand Docker images with no versioning #6913

Closed
stefee opened this issue Aug 4, 2020 · 14 comments · Fixed by #6919
Closed

Renovate does not understand Docker images with no versioning #6913

stefee opened this issue Aug 4, 2020 · 14 comments · Fixed by #6919
Assignees
Labels
datasource:docker priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others type:bug Bug fix of existing functionality

Comments

@stefee
Copy link
Contributor

stefee commented Aug 4, 2020

What Renovate type are you using?

GitHub app

Describe the bug

We have found that Renovate has been downgrading the Docker image tag in our CircleCI configuration to a much older version of the image from a couple of years ago.

The image in question is hosted on quay.io and is a public image.

Renovate made the following change on 24 Apr 2020:

Package Type Update Change
quay.io/condenastinternational/cnid-build-base docker patch 01de4f4201a860947eb348cd85f28af590af8975 -> 1e916aa94eef3999bcfc907ef7a2319616b9827e

And this change on 29 Jul 2020:

Package Type Update Change
quay.io/condenastinternational/cnid-build-base docker patch 9d3a14ff8ed614fa720246b62243adb85b238a6b -> 9f7f934d7e55af2e6e4e53171be3c478771e918a

Relevant debug logs

Log output from reproduction:

Log output
INFO: Repository started
{
  "renovateVersion": "21.33.15"
}
DEBUG: Using localDir: /mnt/renovate/gh/CondeNast/renovate-issue-repro-6913
DEBUG: Repository cache is valid
DEBUG: initRepo("CondeNast/renovate-issue-repro-6913")
DEBUG: No dangling containers to remove
DEBUG: CondeNast/renovate-issue-repro-6913 default branch = master
DEBUG: Using app token for git init
DEBUG: Initializing git repository into /mnt/renovate/gh/CondeNast/renovate-issue-repro-6913
DEBUG: git clone completed
{
  "durationMs": 565
}
DEBUG: latest commit
{
  "latestCommitDate": "2020-08-04T15:45:01+01:00"
}
DEBUG: Setting git author name
{
  "gitAuthorName": "Renovate Bot"
}
DEBUG: Setting git author email
{
  "gitAuthorEmail": "[email protected]"
}
DEBUG: resetMemCache()
DEBUG: detectSemanticCommits()
DEBUG: getCommitMessages
DEBUG: Semantic commits detection: unknown
DEBUG: No semantic commits detected
DEBUG: Setting current branch to master
DEBUG: latest commit
{
  "branchName": "master",
  "latestCommitDate": "2020-08-04T15:45:01+01:00"
}
DEBUG: checkOnboarding()
DEBUG: isOnboarded()
DEBUG: findFile(renovate.json)
DEBUG: config file exists
DEBUG: ensureIssueClosing(Action required: Add a Renovate config)
DEBUG: Retrieving issueList
DEBUG: Retrieved 1 issues
DEBUG: Repo is onboarded
DEBUG: Found renovate.json config file
DEBUG: Repository config
{
  "configFile": "renovate.json",
  "config": {
    "extends": [
      "config:base",
      "github>CondeNast/renovate-config:groupMinorUpdates(dependencies)",
      "github>CondeNast/renovate-config:forceCondenastUpdates",
      "github>CondeNast/renovate-config:automergeMinor",
      "github>CondeNast/renovate-config:condenastStabilityDays",
      "github>CondeNast/renovate-config:dontAutomergeMajor(:heart: help wanted)",
      "github>CondeNast/renovate-config:dontAutomergeNode(:heart: help wanted)",
      "github>CondeNast/renovate-config:dontAutomergeTestFrameworks(:heart: help wanted)",
      "github>CondeNast/rocket:renovateignore"
    ],
    "ignorePresets": [
      ":ignoreModulesAndTests"
    ],
    "ignorePaths": [
      "**/node_modules/**"
    ],
    "timezone": "Europe/London",
    "schedule": [],
    "dependencyDashboard": true,
    "rebaseStalePrs": false,
    "separateMultipleMajor": true,
    "ignoreDeps": [
      "npm"
    ],
    "packageRules": [
      {
        "extends": [
          "github>CondeNast/renovate-config:testFrameworks"
        ],
        "minor": {
          "groupName": "test dependencies",
          "labels": [
            ":hand: requires manual validation"
          ]
        }
      },
      {
        "extends": [
          "github>CondeNast/renovate-config:node"
        ],
        "stabilityDays": 3,
        "prCreation": "not-pending"
      },
      {
        "packagePatterns": [
          "^@pollyjs"
        ],
        "groupName": "polly"
      },
      {
        "extends": [
          "packages:eslint",
          "packages:stylelint"
        ],
        "packageNames": [
          "prettier",
          "lint-staged"
        ],
        "minor": {
          "groupName": "linter packages"
        }
      },
      {
        "packagePatterns": [
          "^babel",
          "^@babel"
        ],
        "minor": {
          "groupName": "babel"
        }
      },
      {
        "packagePatterns": [
          "^@storybook"
        ],
        "minor": {
          "groupName": "storybook"
        }
      },
      {
        "packagePatterns": [
          "^@atjson",
          "^@condenast/atjson"
        ],
        "labels": [
          ":green_apple: gwa"
        ],
        "minor": {
          "groupName": "atjson"
        },
        "major": {
          "groupName": "atjson major releases"
        }
      }
    ]
  }
}
DEBUG: migrateAndValidate()
DEBUG: Config migration necessary
{
  "oldConfig": {
    "extends": [
      "config:base",
      "github>CondeNast/renovate-config:groupMinorUpdates(dependencies)",
      "github>CondeNast/renovate-config:forceCondenastUpdates",
      "github>CondeNast/renovate-config:automergeMinor",
      "github>CondeNast/renovate-config:condenastStabilityDays",
      "github>CondeNast/renovate-config:dontAutomergeMajor(:heart: help wanted)",
      "github>CondeNast/renovate-config:dontAutomergeNode(:heart: help wanted)",
      "github>CondeNast/renovate-config:dontAutomergeTestFrameworks(:heart: help wanted)",
      "github>CondeNast/rocket:renovateignore"
    ],
    "ignorePresets": [
      ":ignoreModulesAndTests"
    ],
    "ignorePaths": [
      "**/node_modules/**"
    ],
    "timezone": "Europe/London",
    "schedule": [],
    "dependencyDashboard": true,
    "rebaseStalePrs": false,
    "separateMultipleMajor": true,
    "ignoreDeps": [
      "npm"
    ],
    "packageRules": [
      {
        "extends": [
          "github>CondeNast/renovate-config:testFrameworks"
        ],
        "minor": {
          "groupName": "test dependencies",
          "labels": [
            ":hand: requires manual validation"
          ]
        }
      },
      {
        "extends": [
          "github>CondeNast/renovate-config:node"
        ],
        "stabilityDays": 3,
        "prCreation": "not-pending"
      },
      {
        "packagePatterns": [
          "^@pollyjs"
        ],
        "groupName": "polly"
      },
      {
        "extends": [
          "packages:eslint",
          "packages:stylelint"
        ],
        "packageNames": [
          "prettier",
          "lint-staged"
        ],
        "minor": {
          "groupName": "linter packages"
        }
      },
      {
        "packagePatterns": [
          "^babel",
          "^@babel"
        ],
        "minor": {
          "groupName": "babel"
        }
      },
      {
        "packagePatterns": [
          "^@storybook"
        ],
        "minor": {
          "groupName": "storybook"
        }
      },
      {
        "packagePatterns": [
          "^@atjson",
          "^@condenast/atjson"
        ],
        "labels": [
          ":green_apple: gwa"
        ],
        "minor": {
          "groupName": "atjson"
        },
        "major": {
          "groupName": "atjson major releases"
        }
      }
    ]
  },
  "newConfig": {
    "extends": [
      "config:base",
      "github>CondeNast/renovate-config:groupMinorUpdates(dependencies)",
      "github>CondeNast/renovate-config:forceCondenastUpdates",
      "github>CondeNast/renovate-config:automergeMinor",
      "github>CondeNast/renovate-config:condenastStabilityDays",
      "github>CondeNast/renovate-config:dontAutomergeMajor(:heart: help wanted)",
      "github>CondeNast/renovate-config:dontAutomergeNode(:heart: help wanted)",
      "github>CondeNast/renovate-config:dontAutomergeTestFrameworks(:heart: help wanted)",
      "github>CondeNast/rocket:renovateignore"
    ],
    "ignorePresets": [
      ":ignoreModulesAndTests"
    ],
    "ignorePaths": [
      "**/node_modules/**"
    ],
    "timezone": "Europe/London",
    "schedule": [],
    "dependencyDashboard": true,
    "separateMultipleMajor": true,
    "ignoreDeps": [
      "npm"
    ],
    "packageRules": [
      {
        "extends": [
          "github>CondeNast/renovate-config:testFrameworks"
        ],
        "minor": {
          "groupName": "test dependencies",
          "labels": [
            ":hand: requires manual validation"
          ]
        }
      },
      {
        "extends": [
          "github>CondeNast/renovate-config:node"
        ],
        "stabilityDays": 3,
        "prCreation": "not-pending"
      },
      {
        "packagePatterns": [
          "^@pollyjs"
        ],
        "groupName": "polly"
      },
      {
        "extends": [
          "packages:eslint",
          "packages:stylelint"
        ],
        "packageNames": [
          "prettier",
          "lint-staged"
        ],
        "minor": {
          "groupName": "linter packages"
        }
      },
      {
        "packagePatterns": [
          "^babel",
          "^@babel"
        ],
        "minor": {
          "groupName": "babel"
        }
      },
      {
        "packagePatterns": [
          "^@storybook"
        ],
        "minor": {
          "groupName": "storybook"
        }
      },
      {
        "packagePatterns": [
          "^@atjson",
          "^@condenast/atjson"
        ],
        "labels": [
          ":green_apple: gwa"
        ],
        "minor": {
          "groupName": "atjson"
        },
        "major": {
          "groupName": "atjson major releases"
        }
      }
    ],
    "rebaseWhen": "conflicted"
  }
}
DEBUG: massaged config
{
  "config": {
    "extends": [
      "config:base",
      "github>CondeNast/renovate-config:groupMinorUpdates(dependencies)",
      "github>CondeNast/renovate-config:forceCondenastUpdates",
      "github>CondeNast/renovate-config:automergeMinor",
      "github>CondeNast/renovate-config:condenastStabilityDays",
      "github>CondeNast/renovate-config:dontAutomergeMajor(:heart: help wanted)",
      "github>CondeNast/renovate-config:dontAutomergeNode(:heart: help wanted)",
      "github>CondeNast/renovate-config:dontAutomergeTestFrameworks(:heart: help wanted)",
      "github>CondeNast/rocket:renovateignore"
    ],
    "ignorePresets": [
      ":ignoreModulesAndTests"
    ],
    "ignorePaths": [
      "**/node_modules/**"
    ],
    "timezone": "Europe/London",
    "schedule": [],
    "dependencyDashboard": true,
    "separateMultipleMajor": true,
    "ignoreDeps": [
      "npm"
    ],
    "packageRules": [
      {
        "extends": [
          "github>CondeNast/renovate-config:testFrameworks"
        ]
      },
      {
        "extends": [
          "github>CondeNast/renovate-config:testFrameworks"
        ],
        "updateTypes": [
          "minor"
        ],
        "groupName": "test dependencies",
        "labels": [
          ":hand: requires manual validation"
        ]
      },
      {
        "extends": [
          "github>CondeNast/renovate-config:node"
        ],
        "stabilityDays": 3,
        "prCreation": "not-pending"
      },
      {
        "packagePatterns": [
          "^@pollyjs"
        ],
        "groupName": "polly"
      },
      {
        "extends": [
          "packages:eslint",
          "packages:stylelint"
        ],
        "packageNames": [
          "prettier",
          "lint-staged"
        ]
      },
      {
        "extends": [
          "packages:eslint",
          "packages:stylelint"
        ],
        "packageNames": [
          "prettier",
          "lint-staged"
        ],
        "updateTypes": [
          "minor"
        ],
        "groupName": "linter packages"
      },
      {
        "packagePatterns": [
          "^babel",
          "^@babel"
        ]
      },
      {
        "packagePatterns": [
          "^babel",
          "^@babel"
        ],
        "updateTypes": [
          "minor"
        ],
        "groupName": "babel"
      },
      {
        "packagePatterns": [
          "^@storybook"
        ]
      },
      {
        "packagePatterns": [
          "^@storybook"
        ],
        "updateTypes": [
          "minor"
        ],
        "groupName": "storybook"
      },
      {
        "packagePatterns": [
          "^@atjson",
          "^@condenast/atjson"
        ],
        "labels": [
          ":green_apple: gwa"
        ]
      },
      {
        "packagePatterns": [
          "^@atjson",
          "^@condenast/atjson"
        ],
        "labels": [
          ":green_apple: gwa"
        ],
        "updateTypes": [
          "minor"
        ],
        "groupName": "atjson"
      },
      {
        "packagePatterns": [
          "^@atjson",
          "^@condenast/atjson"
        ],
        "labels": [
          ":green_apple: gwa"
        ],
        "updateTypes": [
          "major"
        ],
        "groupName": "atjson major releases"
      }
    ],
    "rebaseWhen": "conflicted"
  }
}
DEBUG: migrated config
{
  "config": {
    "extends": [
      "config:base",
      "github>CondeNast/renovate-config:groupMinorUpdates(dependencies)",
      "github>CondeNast/renovate-config:forceCondenastUpdates",
      "github>CondeNast/renovate-config:automergeMinor",
      "github>CondeNast/renovate-config:condenastStabilityDays",
      "github>CondeNast/renovate-config:dontAutomergeMajor(:heart: help wanted)",
      "github>CondeNast/renovate-config:dontAutomergeNode(:heart: help wanted)",
      "github>CondeNast/renovate-config:dontAutomergeTestFrameworks(:heart: help wanted)",
      "github>CondeNast/rocket:renovateignore"
    ],
    "ignorePresets": [
      ":ignoreModulesAndTests"
    ],
    "ignorePaths": [
      "**/node_modules/**"
    ],
    "timezone": "Europe/London",
    "schedule": [],
    "dependencyDashboard": true,
    "separateMultipleMajor": true,
    "ignoreDeps": [
      "npm"
    ],
    "packageRules": [
      {
        "extends": [
          "github>CondeNast/renovate-config:testFrameworks"
        ]
      },
      {
        "extends": [
          "github>CondeNast/renovate-config:testFrameworks"
        ],
        "updateTypes": [
          "minor"
        ],
        "groupName": "test dependencies",
        "labels": [
          ":hand: requires manual validation"
        ]
      },
      {
        "extends": [
          "github>CondeNast/renovate-config:node"
        ],
        "stabilityDays": 3,
        "prCreation": "not-pending"
      },
      {
        "packagePatterns": [
          "^@pollyjs"
        ],
        "groupName": "polly"
      },
      {
        "extends": [
          "packages:eslint",
          "packages:stylelint"
        ],
        "packageNames": [
          "prettier",
          "lint-staged"
        ]
      },
      {
        "extends": [
          "packages:eslint",
          "packages:stylelint"
        ],
        "packageNames": [
          "prettier",
          "lint-staged"
        ],
        "updateTypes": [
          "minor"
        ],
        "groupName": "linter packages"
      },
      {
        "packagePatterns": [
          "^babel",
          "^@babel"
        ]
      },
      {
        "packagePatterns": [
          "^babel",
          "^@babel"
        ],
        "updateTypes": [
          "minor"
        ],
        "groupName": "babel"
      },
      {
        "packagePatterns": [
          "^@storybook"
        ]
      },
      {
        "packagePatterns": [
          "^@storybook"
        ],
        "updateTypes": [
          "minor"
        ],
        "groupName": "storybook"
      },
      {
        "packagePatterns": [
          "^@atjson",
          "^@condenast/atjson"
        ],
        "labels": [
          ":green_apple: gwa"
        ]
      },
      {
        "packagePatterns": [
          "^@atjson",
          "^@condenast/atjson"
        ],
        "labels": [
          ":green_apple: gwa"
        ],
        "updateTypes": [
          "minor"
        ],
        "groupName": "atjson"
      },
      {
        "packagePatterns": [
          "^@atjson",
          "^@condenast/atjson"
        ],
        "labels": [
          ":green_apple: gwa"
        ],
        "updateTypes": [
          "major"
        ],
        "groupName": "atjson major releases"
      }
    ],
    "rebaseWhen": "conflicted"
  }
}
DEBUG: Ignoring preset :ignoreModulesAndTests in config:base
DEBUG: Found repo ignorePaths
{
  "ignorePaths": [
    "**/node_modules/**"
  ]
}
DEBUG: checkBaseBranch()
DEBUG: config.repoIsOnboarded=true
DEBUG: Setting current branch to master
DEBUG: latest commit
{
  "branchName": "master",
  "latestCommitDate": "2020-08-04T15:45:01+01:00"
}
DEBUG: Setting branchPrefix: renovate/
DEBUG: Cannot read vulnerability alerts
DEBUG: No vulnerability alerts found
DEBUG: processRepo()
DEBUG: findIssue(Dependency Dashboard)
DEBUG: Found issue 2
DEBUG: No baseBranches
DEBUG: extract()
DEBUG: Found cached extract
{
  "baseBranch": "master",
  "baseBranchSha": "642d6e9953526daea6c256a4567078cfa85f4e97"
}
INFO: Dependency extraction complete
{
  "baseBranch": "master",
  "stats": {
    "managers": {
      "circleci": {
        "fileCount": 1,
        "depCount": 1
      }
    },
    "total": {
      "fileCount": 1,
      "depCount": 1
    }
  }
}
DEBUG: getLabels(https://quay.io, condenastinternational/cnid-build-base, docker19.03.7-git-docker)
DEBUG: getManifestResponse(https://quay.io, condenastinternational/cnid-build-base, docker19.03.7-git-docker)
DEBUG: Unauthorized docker lookup
{
  "registry": "https://quay.io",
  "dockerRepository": "condenastinternational/cnid-build-base",
  "err": {
    "name": "HTTPError",
    "host": "quay.io",
    "hostname": "quay.io",
    "method": "GET",
    "path": "/v2/condenastinternational/cnid-build-base/blobs/sha256:35c47bb164759700b310a8a4f2c8912ae6b22bb4df6556c1983a68415268162c",
    "protocol": "https:",
    "url": "https://quay.io/v2/condenastinternational/cnid-build-base/blobs/sha256:35c47bb164759700b310a8a4f2c8912ae6b22bb4df6556c1983a68415268162c",
    "gotOptions": {
      "path": "/v2/condenastinternational/cnid-build-base/blobs/sha256:35c47bb164759700b310a8a4f2c8912ae6b22bb4df6556c1983a68415268162c",
      "protocol": "https:",
      "slashes": true,
      "auth": null,
      "host": "quay.io",
      "port": null,
      "hostname": "quay.io",
      "hash": null,
      "search": null,
      "query": null,
      "pathname": "/v2/condenastinternational/cnid-build-base/blobs/sha256:35c47bb164759700b310a8a4f2c8912ae6b22bb4df6556c1983a68415268162c",
      "href": "https://quay.io/v2/condenastinternational/cnid-build-base/blobs/sha256:35c47bb164759700b310a8a4f2c8912ae6b22bb4df6556c1983a68415268162c",
      "retry": {
        "methods": {},
        "statusCodes": {},
        "errorCodes": {}
      },
      "headers": {
        "user-agent": "Renovate Bot (GitHub App 2740)",
        "authorization": "***********",
        "accept-encoding": "gzip, deflate"
      },
      "hooks": {
        "beforeRequest": [],
        "beforeRedirect": [
          null
        ],
        "beforeRetry": [],
        "afterResponse": [],
        "beforeError": [],
        "init": []
      },
      "decompress": true,
      "throwHttpErrors": true,
      "followRedirect": true,
      "stream": false,
      "form": false,
      "json": false,
      "cache": false,
      "useElectronNet": false,
      "method": "GET",
      "hostType": "docker"
    },
    "statusCode": 400,
    "statusMessage": "Bad Request",
    "headers": {
      "x-amz-request-id": "92BB219B228EB56D",
      "x-amz-id-2": "RePs5bBiDvTmVrUaOrHS6x4cu1uT1TbMwOZjrke1vfdc9WLDFAIJdb34vvwDttlBicUYDILHsvE=",
      "content-type": "application/xml",
      "transfer-encoding": "chunked",
      "date": "Tue, 04 Aug 2020 14:48:29 GMT",
      "connection": "close",
      "server": "AmazonS3"
    },
    "body": "\nInvalidArgumentOnly one auth mechanism allowed; only the X-Amz-Algorithm query parameter, Signature query string parameter or the Authorization header should be specifiedAuthorizationBearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6ImIwZDNkNDlhMjI3ZWFhNzNjMmI3ZTM3ZTkxMTAyY2ZiNTYwNjc1YTgxYzExNjMzMDQzNGY2MjdkNDczZDQwODgifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImNvbmRlbmFzdGludGVybmF0aW9uYWwvY25pZC1idWlsZC1iYXNlIiwiYWN0aW9ucyI6WyJwdWxsIl19XSwiY29udGV4dCI6eyJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImNvbmRlbmFzdGludGVybmF0aW9uYWwvY25pZC1idWlsZC1iYXNlIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9LCJhdWQiOiJxdWF5LmlvIiwiZXhwIjoxNTk2NTU2MTA4LCJpc3MiOiJxdWF5IiwiaWF0IjoxNTk2NTUyNTA4LCJuYmYiOjE1OTY1NTI1MDgsInN1YiI6Iihhbm9ueW1vdXMpIn0.G0REeS76c9iMXnhjfQt_zFIgu_LUX6dUb5dgQO749VX8_QE2QF5eT4tOUyAUCIuVwKuGs5IO0JsE85GM0E37olkr2MBC-Y42bKruTQEPrg0R_mnqslwqCLa-DOv9suxDOSzjexf1qGVmYDi0WA3rY1sCkyarW-VwVfSfzeLRLUS-KPysJi9gLMbl2OWs2sqE25IbFflDb6BMYpubpJ8_c5oVhysdFwwCopIjnOqwMumIJ5-9wwnuIRYPHk8U4FLV_-tPeeHMwbiRY3c7yCzOYVBNvtX8fNYmFdvhhsesugabIEvro7JqQOL-zjoppnOjx8zZlNuwDlzBbeJh5UKJ_Q92BB219B228EB56DRePs5bBiDvTmVrUaOrHS6x4cu1uT1TbMwOZjrke1vfdc9WLDFAIJdb34vvwDttlBicUYDILHsvE=",
    "message": "Response code 400 (Bad Request)",
    "stack": "HTTPError: Response code 400 (Bad Request)\n    at EventEmitter. (/home/ubuntu/renovateapp/node_modules/got/source/as-promise.js:74:19)\n    at runMicrotasks ()\n    at processTicksAndRejections (internal/process/task_queues.js:97:5)"
  }
}
DEBUG: getDigest(https://quay.io, condenastinternational/cnid-build-base, 1e916aa94eef3999bcfc907ef7a2319616b9827e)
DEBUG: getDigest(https://quay.io, condenastinternational/cnid-build-base, 01de4f4201a860947eb348cd85f28af590af8975)
DEBUG: Package releases lookups complete
{
  "baseBranch": "master"
}
DEBUG: packageFiles with updates
{
  "config": {
    "circleci": [
      {
        "deps": [
          {
            "depName": "quay.io/condenastinternational/cnid-build-base",
            "depType": "docker",
            "updates": [
              {
                "fromVersion": "01de4f4201a860947eb348cd85f28af590af8975",
                "toVersion": "1e916aa94eef3999bcfc907ef7a2319616b9827e",
                "newValue": "1e916aa94eef3999bcfc907ef7a2319616b9827e",
                "newMajor": 1,
                "newMinor": null,
                "updateType": "minor",
                "isSingleVersion": true,
                "newVersion": "1e916aa94eef3999bcfc907ef7a2319616b9827e",
                "newDigest": "sha256:34ac3665cc3849097f6b95ba4073d3f8059aeaefd43a144108c8d69c93f02540",
                "newDigestShort": "34ac366"
              }
            ],
            "depIndex": 0,
            "warnings": [],
            "sourceUrl": null,
            "datasource": "docker",
            "versioning": "docker",
            "currentValue": "01de4f4201a860947eb348cd85f28af590af8975",
            "currentDigest": "sha256:95a414e53fca401fa4d1e5c02d494e8a9210c6981a2f744bbc2d432dd1965362",
            "replaceString": "quay.io/condenastinternational/cnid-build-base:01de4f4201a860947eb348cd85f28af590af8975@sha256:95a414e53fca401fa4d1e5c02d494e8a9210c6981a2f744bbc2d432dd1965362",
            "dockerRegistry": "https://quay.io",
            "dockerRepository": "condenastinternational/cnid-build-base",
            "autoReplaceStringTemplate": "{{depName}}{{#if newValue}}:{{newValue}}{{/if}}{{#if newDigest}}@{{newDigest}}{{/if}}"
          }
        ],
        "manager": "circleci",
        "packageFile": ".circleci/config.yml"
      }
    ]
  }
}
DEBUG: branchifyUpgrades
DEBUG: 1 flattened updates found: quay.io/condenastinternational/cnid-build-base
DEBUG: Using group branchName template
DEBUG: Dependency quay.io/condenastinternational/cnid-build-base is part of group dependencies
DEBUG: Returning 1 branch(es)
DEBUG: config.repoIsOnboarded=true
DEBUG: processRepo()
DEBUG: Processing 1 branch: renovate/docker-dependencies
DEBUG: Calculating hourly PRs remaining
DEBUG: Retrieving PR list
DEBUG: Retrieved 1 Pull Requests
DEBUG: currentHourStart=1596549600000
DEBUG: PR hourly limit remaining: 1
DEBUG: Enforcing prConcurrentLimit (20)
DEBUG: 1 PRs are currently open
DEBUG: PR concurrent limit remaining: 19
DEBUG: Calculated maximum PRs remaining this run
{
  "prsRemaining": 1
}
DEBUG: processBranch with 1 upgrades(branch="renovate/docker-dependencies")
DEBUG: Setting current branch to master(branch="renovate/docker-dependencies")
DEBUG: latest commit(branch="renovate/docker-dependencies")
{
  "branchName": "master",
  "latestCommitDate": "2020-08-04T15:45:01+01:00"
}
DEBUG: getBranchPr(renovate/docker-dependencies)(branch="renovate/docker-dependencies")
DEBUG: findPr(renovate/docker-dependencies, undefined, open)(branch="renovate/docker-dependencies")
DEBUG: Found PR #1(branch="renovate/docker-dependencies")
DEBUG: Returning from graphql open PR list(branch="renovate/docker-dependencies")
DEBUG: branchExists=true(branch="renovate/docker-dependencies")
DEBUG: Branch pr rebase requested: true(branch="renovate/docker-dependencies")
DEBUG: Branch has 1 upgrade(s)(branch="renovate/docker-dependencies")
DEBUG: Checking if PR has been edited(branch="renovate/docker-dependencies")
DEBUG: Found existing branch PR(branch="renovate/docker-dependencies")
DEBUG: Checking schedule(, Europe/London)(branch="renovate/docker-dependencies")
DEBUG: No schedule defined(branch="renovate/docker-dependencies")
DEBUG: Branch already exists(branch="renovate/docker-dependencies")
DEBUG: getBranchPr(renovate/docker-dependencies)(branch="renovate/docker-dependencies")
DEBUG: findPr(renovate/docker-dependencies, undefined, open)(branch="renovate/docker-dependencies")
DEBUG: Found PR #1(branch="renovate/docker-dependencies")
DEBUG: Returning from graphql open PR list(branch="renovate/docker-dependencies")
DEBUG: Manual rebase requested via PR checkbox for #1(branch="renovate/docker-dependencies")
DEBUG: Using reuseExistingBranch: false(branch="renovate/docker-dependencies")
DEBUG: manager.getUpdatedPackageFiles()(branch="renovate/docker-dependencies")
{
  "reuseExistingBranch": false,
  "branchName": "renovate/docker-dependencies"
}
DEBUG: Starting search at index 223(packageFile=".circleci/config.yml", branch="renovate/docker-dependencies")
{
  "depName": "quay.io/condenastinternational/cnid-build-base"
}
DEBUG: Found match at index 223(packageFile=".circleci/config.yml", branch="renovate/docker-dependencies")
{
  "depName": "quay.io/condenastinternational/cnid-build-base"
}
DEBUG: CircleCI docker image(branch="renovate/docker-dependencies")
{
  "depName": "quay.io/condenastinternational/cnid-build-base",
  "currentValue": "1e916aa94eef3999bcfc907ef7a2319616b9827e",
  "currentDigest": "sha256:34ac3665cc3849097f6b95ba4073d3f8059aeaefd43a144108c8d69c93f02540"
}
DEBUG: Contents updated(packageFile=".circleci/config.yml", branch="renovate/docker-dependencies")
{
  "depName": "quay.io/condenastinternational/cnid-build-base"
}
DEBUG: Updated 1 package files(branch="renovate/docker-dependencies")
DEBUG: No updated lock files in branch(branch="renovate/docker-dependencies")
DEBUG: 1 file(s) to commit(branch="renovate/docker-dependencies")
DEBUG: Committing files to branch renovate/docker-dependencies(branch="renovate/docker-dependencies")
INFO: Branch updated(branch="renovate/docker-dependencies")
{
  "commitHash": "d45a508"
}
DEBUG: Checking if we can automerge branch(branch="renovate/docker-dependencies")
DEBUG: mergeStatus=no automerge(branch="renovate/docker-dependencies")
DEBUG: Ensuring PR(branch="renovate/docker-dependencies")
DEBUG: There are 0 errors and 0 warnings(branch="renovate/docker-dependencies")
DEBUG: getBranchPr(renovate/docker-dependencies)(branch="renovate/docker-dependencies")
DEBUG: findPr(renovate/docker-dependencies, undefined, open)(branch="renovate/docker-dependencies")
DEBUG: Found PR #1(branch="renovate/docker-dependencies")
DEBUG: Returning from graphql open PR list(branch="renovate/docker-dependencies")
DEBUG: Found existing PR(branch="renovate/docker-dependencies")
DEBUG: getBranchStatus(renovate/docker-dependencies)(branch="renovate/docker-dependencies")
DEBUG: branch status check result(branch="renovate/docker-dependencies")
{
  "state": "pending",
  "statuses": []
}
DEBUG: No check runs found(branch="renovate/docker-dependencies")
{
  "result": {
    "total_count": 0,
    "check_runs": []
  }
}
DEBUG: Processing existing PR(branch="renovate/docker-dependencies")
DEBUG: getBranchStatus(renovate/docker-dependencies)(branch="renovate/docker-dependencies")
DEBUG: branch status check result(branch="renovate/docker-dependencies")
{
  "state": "pending",
  "statuses": []
}
DEBUG: No check runs found(branch="renovate/docker-dependencies")
{
  "result": {
    "total_count": 0,
    "check_runs": []
  }
}
DEBUG: getBranchStatus() result(branch="renovate/docker-dependencies")
{
  "branchStatus": "yellow",
  "branchName": "renovate/docker-dependencies"
}
DEBUG: updatePr(1, Update quay.io/condenastinternational/cnid-build-base Docker tag to v1e916aa94eef3999bcfc907ef7a2319616b9827e, body)(branch="renovate/docker-dependencies")
DEBUG: PR updated(branch="renovate/docker-dependencies")
{
  "pr": 1
}
INFO: PR updated(branch="renovate/docker-dependencies")
{
  "pr": 1,
  "prTitle": "Update quay.io/condenastinternational/cnid-build-base Docker tag to v1e916aa94eef3999bcfc907ef7a2319616b9827e"
}
DEBUG: Checking #1 for automerge(branch="renovate/docker-dependencies")
{
  "automerge": true,
  "automergeType": "pr",
  "automergeComment": "automergeComment"
}
DEBUG: PR is configured for automerge(branch="renovate/docker-dependencies")
DEBUG: getBranchStatus(renovate/docker-dependencies)(branch="renovate/docker-dependencies")
DEBUG: branch status check result(branch="renovate/docker-dependencies")
{
  "state": "pending",
  "statuses": []
}
DEBUG: No check runs found(branch="renovate/docker-dependencies")
{
  "result": {
    "total_count": 0,
    "check_runs": []
  }
}
DEBUG: PR is not ready for merge (branch status is yellow)(branch="renovate/docker-dependencies")
DEBUG: Ensuring Dependency Dashboard
DEBUG: getBranchPr(renovate/docker-dependencies)
DEBUG: findPr(renovate/docker-dependencies, undefined, open)
DEBUG: Found PR #1
DEBUG: Returning from graphql open PR list
DEBUG: ensureIssue(Dependency Dashboard)
DEBUG: Patching issue
DEBUG: Issue updated
DEBUG: Removing any stale branches
DEBUG: config.repoIsOnboarded=true
DEBUG: Branch lists
{
  "branchList": [
    "renovate/docker-dependencies"
  ],
  "renovateBranches": [
    "renovate/docker-dependencies"
  ]
}
DEBUG: remainingBranches=
DEBUG: No branches to clean up
DEBUG: ensureIssueClosing(Action Required: Fix Renovate Configuration)
DEBUG: Repository timing splits (milliseconds)
{
  "splits": {
    "init": 5716,
    "extract": 240,
    "lookup": 1618,
    "update": 6850
  },
  "total": 15149
}
DEBUG: http statistics
{
  "hostStats": [
    "api.github.com, 22 requests, 253ms average",
    "quay.io, 3 requests, 310ms average"
  ],
  "totalRequests": 25
}
INFO: Repository finished
{
  "durationMs": 15149
}

To Reproduce

See reproduction here: CondeNast/renovate-issue-repro-6913#1

The Docker image is specified in the Docker executor config in the CircleCI configuration.

e.g.

docker:
  - image: quay.io/{user}/{image}:{tag}@sha256:{sha256}

We are using the config:base preset with automerge enabled for minor/patch, and some other rules which are not relevant to the CircleCI deps.

Additional context

Downgrading a Docker image tag in the CI configuration can pose a significant security risk if there are vulnerabilities in the older version of the image e.g. if the image was being used as a base image for a server app in production. In our case this should not be quite as risky since this was an image only for use in an internal pipeline.

@rarkins rarkins added the auto:reproduction A minimal reproduction is necessary to proceed label Aug 4, 2020
@rarkins
Copy link
Collaborator

rarkins commented Aug 4, 2020

Please provide a public repo that reproduces the error

@rarkins
Copy link
Collaborator

rarkins commented Aug 4, 2020

BTW I think this is because you are using digests as your tag values and Renovate is attempting to sort these like versions.

@stefee
Copy link
Contributor Author

stefee commented Aug 4, 2020

I think this is because you are using digests as your tag values

That makes sense. We use the Git commit SHA for our tagging rather than semver in quite a few of our base images.

As a user, I wouldn't expect Renovate to assume that Docker tags are written in semver format and I think this is what is leading to unexpected results?

@stefee
Copy link
Contributor Author

stefee commented Aug 4, 2020

Repro: CondeNast/renovate-issue-repro-6913#1

See log output in issue description.

@rarkins rarkins changed the title Renovate downgrades Docker image tags in CircleCI configuration Renovate does not understand Docker images with no versioning Aug 4, 2020
@rarkins
Copy link
Collaborator

rarkins commented Aug 4, 2020

We use the Git commit SHA for our tagging rather than semver in quite a few of our base images.

If Renovate can't sort the tags then it can't be used to update them. I suggest you disable Renovate for this image using a package rule.

As a user, I wouldn't expect Renovate to assume that Docker tags are written in semver format

You're in a pretty small minority with that expectation. Nearly all Docker images use some form of sortable versioning, for good reasons. Is it your expectation that Renovate always treats the latest published tag as the desired update?

image

e.g. update from commit SHA tags to kinda-versioned tags and vice versa?

@rarkins rarkins added reproduced: yes type:feature Feature (new functionality) priority-4-low Low priority, unlikely to be done unless it becomes important to more people datasource:docker and removed auto:reproduction A minimal reproduction is necessary to proceed labels Aug 4, 2020
@stefee
Copy link
Contributor Author

stefee commented Aug 4, 2020

Is it your expectation that Renovate always treats the latest published tag as the desired update?

My expectation would be that if it's not a semver tag (e.g. the tag contains letters as well as numbers or exceeds a certain string length) renovate would most-likely ignore it completely, or possibly use the latest tag.

Using the latest tag seems problematic for some scenarios where the tags are not properly versioned and could contain breaking changes, so I would recommend ignoring non-semver tags unless you have a better idea. That is of course assuming it is possible to invalidate the tag as being non-semver. Perhaps this is not possible if any string could technically be interpreted as valid semver string? I'm not sure whether or not that is the case.

My feeling here is that renovate should avoid this behaviour out of the box if at all possible and we should not have to tell it specifically to ignore this dependency. I appreciate that this might be poor practice on the part of the maintainers of the Docker image but as a user (I personally am not the maintainer of this Docker image) this behaviour is quite difficult to identify and mitigate.

@stefee
Copy link
Contributor Author

stefee commented Aug 4, 2020

For now we have disabled renovate automerge for "managers": ["circleci"].

@stefee
Copy link
Contributor Author

stefee commented Aug 4, 2020

I'm looking at some best practices docs on image tagging and there seems to be more of an emphasis on uniqueness and immutability over sort-ability.

From Google's Best practices for building containers:

If you have an advanced continuous delivery system and you release your software often, you probably don't use version numbers as described in the Semantic Versioning Specification. In this case, a common way of handling version numbers is to use the Git commit SHA-1 hash (or a short version of it) as the version number. By design, the Git commit hash is immutable and references a specific version of your software.

You can use this commit hash as a version number for your software, but also as a tag for the Docker image built from this specific version of your software. Doing so makes Docker images traceable: because in this case the image tag is immutable, you instantly know which specific version of your software is running inside a given container. In your continuous delivery pipeline, automate the update of the version number used for your deployments.

@rarkins
Copy link
Collaborator

rarkins commented Aug 4, 2020

In practice it causes many difficulties, such as not being able to easily determine which version is newer. You can see they don't practice what they preach: https://hub.docker.com/u/google

Anyway, Renovate does conceptually abort attempts to update if the existing version (in this case tag) does not satisfy what we expect the version to look like. Because Docker tagging is the wild west (and btw it's not just semver but also pep440 etc) our docker versioning scheme attempts to be very flexible - perhaps too flexible. I think we could add a special case that if an existing tag is a valid commit SHA then treat it as not a version and don't attempt to update it.

@rarkins rarkins added type:bug Bug fix of existing functionality priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others and removed type:feature Feature (new functionality) priority-4-low Low priority, unlikely to be done unless it becomes important to more people labels Aug 4, 2020
@rarkins
Copy link
Collaborator

rarkins commented Aug 4, 2020

Solution: in loose versioning, check if the string is a valid git commit SHA and return isValid=false

@stefee
Copy link
Contributor Author

stefee commented Aug 5, 2020

@rarkins
I can take a look at opening a PR for this if you're open to external contributions.

@rarkins
Copy link
Collaborator

rarkins commented Aug 5, 2020

@stefee yes, most definitely. I think it is probably best solved by adding an if condition here that returns null if it matches a valid git SHA. Maybe can check for both short (7 chars?) as well as long (20?).

@stefee
Copy link
Contributor Author

stefee commented Aug 5, 2020

Maybe can check for both short (7 chars?) as well as long (20?).

Will do 7-char hash as this is truncated version outputted from git rev-parse --short and 40-char hash which is the default SHA-1 length.

So I think this regex is a correct test for commit hash:

(^[a-z0-9]{7}$)|(^[a-z0-9]{40}$)

@rarkins
Copy link
Collaborator

rarkins commented Aug 5, 2020

Should be a-f not a-z.

Length 7-40 inclusive is probably most flexible too, i.e. /^[a-f0-9]{7,40}$/

One problem I see is that if people use date-based versions (e.g. 20200805 or 20200805115304 then they'd get mistaken for commit hashes. Perhaps our test should exclude any strings that parse as a number, to avoid this.

i.e. matches /^[a-f0-9]{7,40}$/ and doesn't match ^[0-9]+$

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 15, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
datasource:docker priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others type:bug Bug fix of existing functionality
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants