feat(git/auth): Support hostrules with username and password when authenticating git commands

[Shegox]

Changes

This pull request introduces the ability to authenticate git commands using a combination of username and password through host rules in addition to the currently supported token format.

Context

Fixes #24077

Documentation (please check one with an [x])

• I have updated the documentation, or
• No documentation update is required

How I've tested my work (please select one)

I have verified these changes via:

• Code inspection only, or
• Newly added/modified unit tests, or
• No unit tests but ran on a real repository, or
• Both unit tests + ran on a real repository

[viceice]

otherwise LGTM

[renovate-release]

🎉 This PR is included in version 36.79.0 🎉

The release is available on:

• GitHub release
• 36.79.0

Your semantic-release bot 📦🚀

[lcobucci]

Just tested the fix and I still get the authentication errors 😢
I'll do some local debugging and come back you.

Thanks @Shegox and @viceice for working on this 👍

[lcobucci]

Okay, after adding a whole lot of console.log() on a few places I've managed to find the issue... it's a tricky one, I think 😭

We were previously relying on the host types git-refs and git-tags to resolve the submodules.
The authentication changes aren't considering those host types, leading the creation of environment variables using the token as auth mechanism (RENOVATE_TOKEN is automatically promoted as a host rule).

So, essentially I have the following host rules:

[
  {
    hostType: 'github',
    matchHost: 'github.com',
    token: 'github-token',
    resolvedHost: 'github.com'
  },
  {
    hostType: 'git-refs',
    matchHost: 'mygitlab.com',
    password: 'mygitlab-pwd',
    username: 'renovate-bot',
    resolvedHost: 'mygitlab.com'
  },
  {
    hostType: 'git-tags',
    matchHost: 'mygitlab.com',
    password: 'mygitlab-pwd',
    username: 'renovate-bot',
    resolvedHost: 'mygitlab.com'
  },
  {
    matchHost: 'mygitlab.com',
    token: 'mygitlab-pwd',
    hostType: 'gitlab',
    resolvedHost: 'mygitlab.com'
  }
]

The host rule created from RENOVATE_TOKEN is legit for the API requests. However, not for the Git related requests.

Filtering for git-refs, renovate creates the following env vars (contents of newEnvironmentVariables):

{
  GIT_CONFIG_KEY_0: 'url.https://ssh:[email protected]/.insteadOf',
  GIT_CONFIG_VALUE_0: 'ssh://[email protected]/',
  GIT_CONFIG_KEY_1: 'url.https://git:[email protected]/.insteadOf',
  GIT_CONFIG_VALUE_1: '[email protected]:',
  GIT_CONFIG_KEY_2: 'url.https://[email protected]/.insteadOf',
  GIT_CONFIG_VALUE_2: 'https://github.com/',
  GIT_CONFIG_COUNT: '6',
  GIT_CONFIG_KEY_3: 'url.https://renovate-bot:[email protected]/.insteadOf',
  GIT_CONFIG_VALUE_3: 'ssh://[email protected]/',
  GIT_CONFIG_KEY_4: 'url.https://renovate-bot:[email protected]/.insteadOf',
  GIT_CONFIG_VALUE_4: '[email protected]:',
  GIT_CONFIG_KEY_5: 'url.https://renovate-bot:[email protected]/.insteadOf',
  GIT_CONFIG_VALUE_5: 'https://mygitlab.com/'
}

And manually running git ls-remote --symref https://mygitlab.com/path/to/my/repo.git HEAD with those variables work just fine. However, letting the command run leads to the password prompt of a completely weird username (oauth2) - which understandably fails on non-interactive sessions.

I also confirm that running git ls-remote --symref https://[email protected]/path/to/my/repo.git HEAD with those variables doesn't work at all (which makes sense because there's no match on https://oauth2@....

What do you think? We might still be missing some scenarios for getAuthenticationRules() or getGitEnvironmentVariables() - it seems like different components are messing up with the expectations here.

[lcobucci]

Btw, I did find oauth2 here: https://github.com/renovatebot/renovate/blob/main/lib/modules/platform/gitlab/index.ts#L248
Is this somehow affecting the URL generation?

[viceice]

please open a new discussion

[lcobucci]

please open a new discussion

@viceice the original hasn't been concluded IMHO so I just added context here. I believe it's quite counter-productive to open a new one.

[viceice]

you can of cause use the existing discussion, but not the PR. this PR is wrong place to discuss things.