yarn for enterprise git repos

[abhijeetNmishra]

I need to perform npm install from a enterprise git repo like below:
git+ssh://[email protected]:tech/library#develop

Its failing with below log
_error Command failed.
Exit code: 128
Command: git
Arguments: clone ssh://[email protected]:tech/library /Users//.yarn-cache/.tmp/bd1634df3d45c462cf0ca410896de658
Directory: /Users//MobileUIMod
Output:
Cloning into '/Users//.yarn-cache/.tmp/bd1634df3d45c462cf0ca410896de658'...
ssh: Could not resolve hostname git.abc.com:tech: nodename nor servname provided, or not known
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command._

Same is working fine with npm install.

[abhijeetNmishra]

This is package.json entry:
"lab": "git+ssh://[email protected]:tech/library#develop"

[Daniel15]

What does the :tech mean after the hostname? It doesn't look like a port number, is it part of the hostname?

[abhijeetNmishra]

@Daniel15 its a configured value, which we use internally , above is example version of URL in my package.json. I can't share exact URL as its enterprise thing.

[abhijeetNmishra]

@Daniel15 Thanks for the reply 👍 at least. Seeing # of issues reported, i wasn't expecting any reply soon.

[chase]

@Daniel15 The OpenSSH client doesn't follow the RFC for URIs, what follows the : after the domain is always a path. tech/library is a relative path. develop is the git branch.

Git follows the RFCs for all URIs.

[Daniel15]

Ahh, got it. Shouldn't the path begin with a /? I'm used to it being a full path - One of my repos uses a Git URL of [email protected]:/var/local/git/foo for example. I guess it can be a relative path if the user is chroot jailed.

[abhijeetNmishra]

bottom line is : same URL works with npm install but not with yarn

[chase]

@abhijeetNmishra The command is failing because the git clone URI is invalid per the RFC and what is supported by git.
This is a valid URI's format:
git+ssh://[user@]host.xz[:port]/path/to/repo.git/
See the git-clone man page: https://git-scm.com/docs/git-clone

It looks like NPM is introducing non-standard behavior.

[chrisirhc]

From that page that you linked:
This is supported by git. The : syntax allows shortcuts to full git urls.

[chase]

@chrisirhc That is an example for URL rewrite rules meant to be placed in a .gitconfig file.

I use these rewrite rules to switch from https: or git: on push, for example:

[url "[email protected]:"]
    insteadOf = "gh:"
    pushInsteadOf = "github:"
    pushInsteadOf = "git://github.com/"
    pushInsteadOf = "https://github.com/"
[url "git@bitbucket:"]
    insteadOf = "bit:"
    insteadOf = "git://bitbucket.org/"
    pushInsteadOf = "bitbucket:"
    pushInsteadOf = "https://bitbucket.org/"

The only shorthand supported is: [user@]host.xz:path/to/repo.git/ which is rewritten to ssh://[user@]host.xz/path/to/repo.git/

An alternative scp-like syntax may also be used with the ssh protocol:

[user@]host.xz:path/to/repo.git/

This syntax is only recognized if there are no slashes before the first colon. This helps differentiate a local path that contains a colon. For example the local path foo:bar could be specified as an absolute path or ./foo:bar to avoid being misinterpreted as an ssh url.

[chrisirhc]

@chase what I'm observing is that the rewrite rules aren't working (aren't being applied) when yarn attempts to clone. yarn seems to use the hostname as-is.

This is what caused a breakage when switching from npm to yarn.

@chase , in your example would you be able to use git+ssh://gh:some-gh-repo/#master as a package install URL (in package.json where version would be) and yarn would install it?

[chase]

@chrisirhc Hm, I can see where you're coming from.

This is how I see it:
The failure occurs from a git command not yarn, that means it isn't a valid URI per specification. Try running git clone ssh://[email protected]:tech/library in your terminal and you'll see the error that is being given by yarn.

There are no defaults sets of re-write rules to handle the URL in question, which seems to be a cross-breed of both scp and ssh style URLs.

---

After looking deeper into the inconsistency between npm and yarn, it seems that npm has a git URI normalization step. yarn handles URI inconsistencies in the resolvers it has, but it simply doesn't handle all the same cases.

The solution I've been planning on getting at is to integrate npm's normalization package in the proper areas, but the feedback I was looking for on the #development channel fell silent.

There are several issues rooting from incompatibility and it seems they keep popping up every day, but ultimately #513 covers this issue more clearly.

---

@chrisirhc, my example would not allow that, no.
It would allow a remote URL of gh:some-gh-repo/#master, but I would never place that in a package.json because it is not a standard-compliant URI.

[vernak2539]

Is this going to be picked up? would be very nice to get

[iddan]

Trying to install a newer version of a repo (i.e #1.0.4 -> #1.0.5) with:

yarn add https://****@github.com/****/****.git#v1.0.5

Yarn tries to run at ~/Library/Caches/Yarn/v1/.tmp/d796310ae7842a0e388e36b26b021016

git archive 5179795d2575196062582271d5c5a32c3fe03dc6

which fails because:

fatal: not a tree object

When I execute git reflog in the dir it shows:

63eab7a (HEAD -> master, origin/master, origin/HEAD) HEAD@{0}: pull: Fast-forward
7d1f0af HEAD@{1}: clone: from ******

Clearing cache, or even just the .tmp dir, seems to fix this. My guess is that cloning is done incorrectly.