Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Global packages no longer work after node.js update with homebrew #2064

Closed
publicarray opened this issue Nov 29, 2016 · 57 comments
Closed

Global packages no longer work after node.js update with homebrew #2064

publicarray opened this issue Nov 29, 2016 · 57 comments

Comments

@publicarray
Copy link

publicarray commented Nov 29, 2016

Do you want to request a feature or report a bug?
Bug

What is the current behavior?

When node and yarn are installed with homebrew yarn global bin returns a path that is tied to the a specific version of node. If node is updated with brew the symlinks are no longer valid, i.e global packages with binaries need to be reinstalled.

~ ❯ yarn global bin
/usr/local/Cellar/node/7.2.0/bin

Related issue: Homebrew/homebrew-core#7283

If the current behavior is a bug, please provide the steps to reproduce.

I've used brew reinstall to emulate an update, I'm not sure how different it is to brew upgrade node.

~ ❯ brew install node
~ ❯ node --version
v7.2.0
~ ❯ brew install yarn
~ ❯ yarn global bin
/usr/local/Cellar/node/7.2.0/bin
~ ❯ yarn global add bower
~ ❯ bower --version
1.8.0
~ ❯ brew reinstall node --HEAD
~ ❯ node --version
v8.0.0-head
~ ❯ yarn global bin
/usr/local/Cellar/node/HEAD-11b0872/bin
~ ❯ bower --version
command not found: bower

What is the expected behavior?

~ ❯ bower --version
1.8.0

Please mention your node.js, yarn and operating system version.

~ ❯ yarn --version
0.17.9
~ ❯ node --version
v7.2.0
~ ❯ sw_vers
ProductName: Mac OS X
ProductVersion: 10.12.1
BuildVersion: 16B2555
@nicgordon
Copy link

The same issue exists for node installed via nvm:

~ $ which node
/Users/nicgordon/.nvm/versions/node/v7.2.0/bin/node
~ $ yarn global bin
/Users/nicgordon/.nvm/versions/node/v7.2.0/bin
~ $ yarn global add gatsby
~ $ which gatsby
gatsby not found

However adding this prefix flag symlinks the installed module into /usr/local/bin and fixes the issue for me

~ $ yarn global add gatsby --prefix /usr/local
~ $ which gatsby
/usr/local/bin/gatsby

Checking the symlink shows me that the module was actually installed here:

/Users/nicgordon/.config/yarn/global/node_modules/.bin/gatsby

Found this solution here #1877

@publicarray
Copy link
Author

Thank you, however I personally think that your fix is more of a work around and not a permanent solution. I was looking to start a discussion of how this could be handled within yarn or brew so that yarn global packages are not effected on a node.js version change. On the other hand npm global packages function fine after a brew upgrade node.

@kamui545
Copy link

kamui545 commented Dec 10, 2016

I had the same issue several times too.
It would be nice to have this working without the prefix setting

@gutenye
Copy link

gutenye commented Jan 27, 2017

same issue here, is there a permanent solution?

@OmgImAlexis
Copy link

Got the same issue here.

@iddan
Copy link

iddan commented Jan 30, 2017

same issue

@bradley-curran
Copy link

Yarn adds a symlink of the executables to node's bin directory.
In my instance it was /usr/local/Cellar/node/7.2.1.
The actual place where the global packages are installed is ~/.config/yarn/global/node_modules/.

When I updated to node 7.4.0 from 7.2.1 the symlinks weren't added to the new version. I'm guessing you're seeing the same thing.

The problem is, there's no way of using yarn right now to fix this.

You could write a script that does something along the lines of:

  • Get the global packages directory for yarn
  • Find all the files that are executable
  • Symlink all the files into your current node directory

I'll write an npm package later that does all this.

@bradley-curran
Copy link

Hey guys,

I've written an npm package that should fix your yarn-installed global packages.

$ npm install -g fix-yarn-global-packages
$ fix-yarn-global-packages

That should fix all your global packages. Let me know if it doesn't work for you.

https://www.npmjs.com/package/fix-yarn-global-packages

@publicarray
Copy link
Author

publicarray commented Feb 10, 2017

Thanks @bradley-curran but I personally prefer to just add alias yarn-global='yarn global --prefix /usr/local' to my dotfiles until this is fixed.

@bradley-curran
Copy link

I've been doing some more digging and it seems the issue is that yarn doesn't read all npm config files that npm itself uses.

Yarn should read all files defined in npm's npmrc documentation

Currently yarn only reads config files from:

  • ./.npmrc
  • ~/.npmrc
  • <node directory>/.npmrc (<node directory> = /usr/local/Cellar/node/7.6.0/ for me)

The config files that also need to be read include:

  • $PREFIX/etc/npmrc ($PREFIX is usually /usr/local)
  • path/to/npm/itself/npmrc

The problem is that these files can't be found easily using the existing architecture afaict.
Ideally the existing config loading would be replaced with delegating to npm config ls -l and parsing the result rather than trying to read all the files.

Quick Fix

  • Open ~/.npmrc in your favourite text editor.
  • Add the line prefix = /usr/local

The Real Fix

Read all configuration files, ideally by delegating to npm config, rather than reading existing INI files. I'd be happy to contribute, though I'm not really sure who I can discuss implementation details with.

@publicarray
Copy link
Author

publicarray commented Feb 25, 2017

@bradley-curran thanks so much for looking into this, from now on I'll use the npmrc file. (I didn't know that the npmrc existed until now). Just a small word of warning that npm will also install global packages to the prefix set in in the npmrc and no longer in /usr/local/Cellar/node/7.6.0/ as previously.

@mkutny
Copy link

mkutny commented Mar 26, 2017

Agree it should not install into /usr/local/Cellar/node/version .

As a temporary fix, it's now possible to yarn config set prefix /usr/local -g

Although this prefix should be set by default on yarn installation.

@publicarray
Copy link
Author

Similar issue: #2639

@publicarray
Copy link
Author

publicarray commented Mar 27, 2017

Thanks @mkutny, is the -g flag supposed to do something? for me the config file that is modified is always ~/.yarnrc

@mkutny
Copy link

mkutny commented Mar 27, 2017

@publicarray , I just did it according to the doc which says yarn config set <key> <value> [-g|--global]

It seems that in this particular case it doesn't matter but for options which could be set for either local or global scope, it does.

@publicarray
Copy link
Author

@mkutny thanks, I should have read the docs 📖

@HenrikWL
Copy link

Can't this be fixed by having the homebrew formula do

yarn config set prefix /usr/local/

as a post-install?

@bradley-curran
Copy link

You can, if you're willing to assume that the prefix is /usr/local on all machines, which I'm not sure can be assumed.

@HenrikWL
Copy link

Good point. Within the context of Homebrew formulae, I do believe the global Homebrew-prefix is available in a variable. It would be better to use that.

@vdh
Copy link

vdh commented Apr 20, 2017

Please stop advocating the /usr/local prefix workaround. Some of us prefer to stick with the versioned bin directory (e.g. /usr/local/Cellar/node/7.9.0/bin) and keep /usr/local/bin clean. Editing $PATH is a whole lot easier than cleaning up dead symlinks from an overloaded /usr/local/bin directory.

@mkutny
Copy link

mkutny commented Apr 20, 2017

@vdh , it's not about which solution "is a whole lot easier". It's all about what is the right thing to do.

If you read the thread carefully there are currently several issues with global packages:

  • they are not immediately available after yarn global add (they need to be manually symlinked)
  • global packages won't work after node update ($PATH needs to be updated)
  • global packages will be removed if obsolete node version is purged (editing $PATH won't help at all)

Binaries in /usr/local/bin not only solve all these issues but are also consistent with Unix way of doing things in general and with Homebrew in particular.

@HenrikWL
Copy link

HenrikWL commented Apr 20, 2017

@vdh the solution to dead symlinks in /usr/local/bin is to fix the problem that causes dead symlinks in there, not to preserve broken behaviour in yarn. Like @mkutny says, there are several issues with yarn global the way it works now. I expect it to work like npm -g but it doesn't. If that's by design then that's ok, I'll just stick with npm -g for global packages and keep using yarn for project packages (for which it works great).

@partounian
Copy link

I'm not sure if this is related or not, but with zsh, I have an export on startup with the path from yarn global bin, but on startup I always get command not found: yarn how it calling yarn does work. On top of this with global packages that have their own command, I get command not found when trying that call them, but it all works fine when I reload zsh in the same terminal window.

@publicarray
Copy link
Author

publicarray commented Apr 20, 2017

@partounian I recommend you open a new issue because your problem is unrelated. Is it possible that you are calling yarn before you initialised the $PATH? e.g.

~/.zshrc:

yarn global bin
export PATH=$(brew --prefix)/bin:$PATH # run this first

@Daniel15
Copy link
Member

Adding ~/.config/yarn/global/node_modules/.bin to your $PATH is probably a reasonable workaround for now.

The issue is that Yarn symlinks the executables into the wrong directory (the Node.js directory rather than somewhere reasonable). It doesn't really make sense to use the Node.js directory for these scripts, and on many systems that directory is not even writable by regular users. I fixed this for Windows in #3233 but we still need fixes for other operating systems. #3458 seems reasonable as long as the user has write access to /usr/local/bin, although maybe something like ~/.yarn/bin would be better.

@KishanBagaria
Copy link
Contributor

KishanBagaria commented May 24, 2017

TL;DR Use a default directory for Windows/macOS. Ask user to set directory on Linux/others.

Windows and macOS
/usr/local/bin is writable in a default macOS installation and that's where Homebrew writes stuff. When node/npm is installed from Homebrew, Homebrew edits the prefix config in .npmrc so that npm links the global binaries in /usr/local/bin too.

For Windows and macOS, we can use %LocalAppData%\Yarn\bin and /usr/local/bin as default, because that's guaranteed to work out-of-the-box (unless the user is on a broken/unusual installation).

On macOS, Yarn global binaries will work without extending the PATH since /usr/local/bin is already present in default PATH. Although, the directories in default Windows PATH all need admin privileges, so Chocolatey or the user will have to extend the PATH with %LocalAppData%\Yarn\bin.

Linux et al.
For Linux and other unrecognized platforms, we shouldn't assume anything (any of the parent directories of process.execPath isn't reasonable or guaranteed to be writable as @Daniel15 suggested). We can test if /usr/local/bin is writable before using it. But there's thousands of configurations of Linux distros with different versions and different defaults. I dunno if that directory will work on all of them.
Instead, we can ask the user to set the prefix explicitly. Running any yarn global command on these platforms should throw an error and say something like

error you need to first run yarn config set prefix <global_prefix_dir>
info see https://yarnpkg.com/en/docs/cli/global

Or we can show an interactive inquirer.js prompt where the user can select one among many commonly-used prefixes:

* /usr/local/bin
* /usr/bin
* /bin
* ~/n/bin
* ~/bin
* <enter custom prefix>: ______

The current behaviour has been copied from npm which was written about 5 years ago and has seen no updates: https://github.com/npm/npm/blob/771c66884a0a1851cffb3bf132828f6ccef1c09c/lib/config/defaults.js#L92-L105

@BYK
Copy link
Member

BYK commented Jul 10, 2017

When node/npm is installed from Homebrew, Homebrew edits the prefix config in .npmrc so that npm links the global binaries in /usr/local/bin too.

Shall we honor this setting too?

Although, the directories in default Windows PATH all need admin privileges, so Chocolatey or the user will have to extend the PATH with %LocalAppData%\Yarn\bin.

I think this is a really subpar experience for Windows users. Node puts itself into the path along with npm right, why can't we use those directories directly?

@Daniel15
Copy link
Member

I think this is a really subpar experience for Windows users

The Windows installer (and Chocolatey package) automatically update the path, so this is mostly a non-issue.

Node puts itself into the path along with npm right, why can't we use those directories directly?

On most systems, it would require you to run Yarn as administrator, as Node is normally installed at C:\Program Files\nodejs and Node.js doesn't expose the Win32 API for requesting elevation (getting admin rights after the app has started, like "sudo")

@BYK
Copy link
Member

BYK commented Jul 10, 2017

@Daniel15 oh, okay so this clears my Windows-side questions :)

@BYK
Copy link
Member

BYK commented Jul 10, 2017

Well, with the exception of, when/if we install yarn via NPM or as a direct script. I'm not sure if we need elevated permissions when running npm install -g do we?

@Daniel15
Copy link
Member

I'm not sure if we need elevated permissions when running npm install -g do we?

I think it'd depend on where Node.js is installed and what the permissions are on that directory.

@Daniel15
Copy link
Member

Daniel15 commented Jul 10, 2017

Well, with the exception of, when/if we install yarn via NPM or as a direct script.

We could probably update Yarn so that it edits the path on first run. However I don't know if Node.js exposes an API for that. It's got extremely limited ffi capabilities without dealing with complex native modules.

@BYK
Copy link
Member

BYK commented Jul 12, 2017

Did some reading on https://docs.npmjs.com/files/folders#prefix-configuration and looks like the approach in the diff is correct. Just concerned about not using the npm folder (we use Yarn instead). I think this is fine until we get complaints.

@BYK BYK closed this as completed in #3721 Jul 17, 2017
BYK pushed a commit that referenced this issue Jul 17, 2017
**Sumary**

Refs #2064.

Uses `%LOCALAPPDATA%\Yarn\bin` and `/usr/local/bin` on POSIX systems if it's writeable, falling back to `~/.yarn/bin` if it is not.

**Test plan**

N/A. Should add automated tests.
@BYK BYK marked this as a duplicate of #3905 Jul 18, 2017
@BYK BYK marked this as a duplicate of #3728 Jul 18, 2017
@nodox
Copy link

nodox commented Mar 7, 2018

sudo yarn global add ... worked but I'm not sure that should be the best solution

@diegoazh
Copy link

I had the same issue in ubuntu after upgrade node version with nvm, in my dev proyects I fix it removing .lock file en running yarn cache clean and reinstall its. I think that if you remove the .lock file inside of ~/.config/yarn/global/ (in ubuntu) and reinstall all packages fix the error with global packages; also I recomend defining install location with yarn config set prefix ~/.yarn like this docs suggests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests