MacVim doesn't link to brewed python correctly

[xgalaxy]

MacVim isn't linking against the brewed python correctly.
Vim formula works fine. Related: #17896

If you edit the MacVim formula like the below snippit, MacVim will use the configuration directory specified but still links against system python, resulting in various issues using python based plugins.

--enable-pythoninterp=dynamic
--with-python-config-dir=/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/Current/lib/python2.7/config

However, if you remove the changes to the formula and instead temporarily symlink system python path to point to brewed python, install MacVim, and then remove the symlink then MacVim links against the proper python.

cd /System/Library/Frameworks/Python.framework/Versions
sudo mv Current Current-sys
sudo ln -s /usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7 Current

brew install macvim

sudo rm Current
sudo mv Current-sys Current

Fix was suggested here:
http://superuser.com/a/463474

There are several projects that are running into issues with this:
ycm-core/YouCompleteMe#18
powerline/powerline#39
python-mode/python-mode#87

In the cases above, the "solution" has been to first unlink python then install macvim and then relink python. But this isn't a solution at all. Its a bandaid, and a terrible one at that.

I'm not sure why Vim compiles and finds brewed python correctly and why MacVim doesn't. The formula between them appears to be nearly identical. It's likely an upstream issue with MacVim.

Failing anything homebrew can do to fix the issue, at least this will help document it for any body else that comes along in the future and runs into the same problem.

[adamv]

We can't leave an issue open as "documentation", so what should we do here?

[xgalaxy]

The only major difference between the Vim and MacVim formula that I saw was the Vim formula specifying --prefix to configure. Unfortunately this change doesn't seem to make a difference.

For reference, here is the output of MacVim / Vim --version under the 'Linking' section:

Vim:

Linking: /usr/bin/clang   -L.        -L/usr/local/lib -F/usr/local/Frameworks -o vim        -lncurses -liconv -framework Cocoa   -fstack-protector -L/usr/local/lib  -L/System/Library/Perl/5.12/darwin-thread-multi-2level/CORE -lperl -lm -lutil -lc -framework Python  -F/System/Library/Frameworks -framework Tcl -framework CoreFoundation -lruby -lobjc

MacVim:

Linking: cc   -L.   -L.        -arch x86_64 -L/usr/local/lib -o Vim -framework Cocoa -framework Carbon      -lncurses -liconv -framework Cocoa    -fstack-protector -L/usr/local/lib  -L/System/Library/Perl/5.12/darwin-thread-multi-2level/CORE -lperl -lm -lutil -lc -framework Python  -F/System/Library/Frameworks -framework Tcl -framework CoreFoundation -framework Ruby

Notice the lack of -F/usr/local/Frameworks for MacVim.

[jacknagel]

Given that macvim by its nature has a more complicated build system than vanilla vim, it's not surprising that it might ignore flags that we set... the standard procedure here is to try and reproduce the same problem outside of Homebrew. If it persists then the issue should be taken to macvim.

[uraimo]

Just installed YouCompleteMe with homebrew python/macvim, the workaround described by xgalaxy (thanks!) seems the only way to make this work on OSX at the moment. I'd update the FAQ pointing to this issue instead of #18. I agree with what others have already said, the proper fix would be to use homebrew python instead of the system one in macvim formula.

[vaz]

This isn't specific to homebrew, though. I ran into the same problem compiling MacVim by hand against a Python framework I'd installed in /Library... the only way I could compile it against that python was the symlinking trick xgalaxy describes. So I'd call this a MacVim issue (but if the MacVim formula can resolve it with a workaround in the meantime, that would be awesome... would have saved me time for sure).

[pencilcheck]

After a "brew update", I reinstall macvim to see if there is any changes that will fix the issue, but it seems like the changes has made python support disappear completely!!

Here is the Linking portion of my mvim --version

Linking: clang -L. -L. -L/usr/local/lib -o Vim -framework Cocoa -framework Carbon -lncurses -liconv -framework Cocoa -fstack-protector -L/usr/local/lib -L/System/Library/Perl/5.12/darwin-thread-multi-2level/CORE -lperl -lm -lutil -lc -F/System/Library/Frameworks -framework Tcl -framework CoreFoundation -framework Ruby

Notice no -framework Python at all.

[samueljohn]

Your system and your python? (brew --config)

[pencilcheck]

Mountain Lion 10.8.3

HOMEBREW_VERSION: 0.9.4
ORIGIN: http://github.com/mxcl/homebrew.git
HEAD: 046582f
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CELLAR: /usr/local/Cellar
CPU: dual-core 64-bit penryn
OS X: 10.8.3-x86_64
Xcode: 4.6.2
CLT: 4.6.0.0.1.1365549073
LLVM-GCC: build 2336
Clang: 4.2 build 425
X11: 2.7.4 => /opt/X11
System Ruby: 1.8.7-358
Perl: /usr/bin/perl
Python: /usr/local/bin/python => /usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/bin/python2.7
Ruby: /usr/local/bin/ruby => /usr/local/Cellar/ruby/2.0.0-p0/bin/ruby

[samueljohn]

-F/usr/local/Frameworks should be added implicitly (on default). I will check that.

[samueljohn]

perhaps I am wrong.

[pencilcheck]

Adding --with-python has no effect on the compilation.

[samueljohn]

I can't reproduce a failure. I built MacVim.app with system python and with brewed python.

Since there were some python changes, perhaps brew update, brew rm python and brew install python might help.

[pencilcheck]

Would that remove site-packages and existing libraries I installed?

[pencilcheck]

Ok, after updating brew and install python 2.7.5, macvim still doesn't seems to link to python

Linking: clang -L. -L. -L/usr/local/lib -o Vim -framework Cocoa -framework Carbon -lncurses -liconv -framework Cocoa -fstack-protector -L/usr/local/lib -L/System/Library/Perl/5.12/darwin-thread-multi-2level/CORE -lperl -lm -lutil -lc -F/System/Library/Frameworks -framework Tcl -framework CoreFoundation -framework Ruby

[samueljohn]

brew rm python does not remove your site-packages. That would suck :-)

Can anyone reproduce the link failure here? I can't so its hard for me to debug.
Open to suggestions.

[uraimo]

Starting from a working macvim i've replicated the issue removing/installing it (and following xgalaxy's procedure to swap libraries), the --version output is the same pencilcheck is seeing, no more -framework Python among the clang linking options (it was there before).
I tried to rollback the change done in 0b50110 adding the removed --enable-pythoninterp (even if i noticed it was added a few lines below...) but the --version output doesn't change.

The weird thing is that everything python-related (youcompleteme, powerline, :python "command") still works with both versions and that python/dyn is included among the available features:

Huge version with MacVim GUI.  Features included (+) or not (-):
+arabic +autocmd +balloon_eval +browse ++builtin_terms +byte_offset +cindent
+clientserver +clipboard +cmdline_compl +cmdline_hist +cmdline_info +comments
+conceal +cryptv +cscope +cursorbind +cursorshape +dialog_con_gui +diff
+digraphs +dnd -ebcdic +emacs_tags +eval +ex_extra +extra_search +farsi
+file_in_path +find_in_path +float +folding -footer +fork() +fullscreen
-gettext -hangul_input +iconv +insert_expand +jumplist +keymap +langmap
+libcall +linebreak +lispindent +listcmds +localmap -lua +menu +mksession
+modify_fname +mouse +mouseshape +mouse_dec -mouse_gpm -mouse_jsbterm
+mouse_netterm +mouse_sgr -mouse_sysmouse +mouse_urxvt +mouse_xterm +multi_byte
 +multi_lang -mzscheme +netbeans_intg +odbeditor +path_extra +perl
+persistent_undo +postscript +printer +profile +python/dyn -python3 +quickfix
+reltime +rightleft +ruby +scrollbind +signs +smartindent -sniff +startuptime
+statusline -sun_workshop +syntax +tag_binary +tag_old_static -tag_any_white
+tcl +terminfo +termresponse +textobjects +title +toolbar +transparency
+user_commands +vertsplit +virtualedit +visual +visualextra +viminfo +vreplace
+wildignore +wildmenu +windows +writebackup -X11 -xfontset +xim -xsmp
-xterm_clipboard -xterm_save

I'm not sure if what i'm seeing it's a consequence of a dirty environment or if i'm not testing the python support correctly, but afaik even w/o the linking options everything seems to work as usual.

[pencilcheck]

I am getting the same output as well, +python is in the options, but python libraries such as YouCompleteMe, and powerline don't work for my version, as it seems like the python is completely gone from MacVim.

VIM - Vi IMproved 7.3 (2010 Aug 15, compiled Jun 5 2013 01:18:33)
MacOS X (unix) version
Included patches: 1-754
Compiled by Homebrew
Huge version with MacVim GUI. Features included (+) or not (-):
+arabic +autocmd +balloon_eval +browse ++builtin_terms +byte_offset +cindent
+clientserver +clipboard +cmdline_compl +cmdline_hist +cmdline_info +comments
+conceal +cryptv +cscope +cursorbind +cursorshape +dialog_con_gui +diff
+digraphs +dnd -ebcdic +emacs_tags +eval +ex_extra +extra_search +farsi
+file_in_path +find_in_path +float +folding -footer +fork() +fullscreen
-gettext -hangul_input +iconv +insert_expand +jumplist +keymap +langmap
+libcall +linebreak +lispindent +listcmds +localmap -lua +menu +mksession
+modify_fname +mouse +mouseshape +mouse_dec -mouse_gpm -mouse_jsbterm
+mouse_netterm +mouse_sgr -mouse_sysmouse +mouse_urxvt +mouse_xterm +multi_byte
+multi_lang -mzscheme +netbeans_intg +odbeditor +path_extra +perl
+persistent_undo +postscript +printer +profile +python/dyn -python3 +quickfix
+reltime +rightleft +ruby +scrollbind +signs +smartindent -sniff +startuptime
+statusline -sun_workshop +syntax +tag_binary +tag_old_static -tag_any_white
+tcl +terminfo +termresponse +textobjects +title +toolbar +transparency
+user_commands +vertsplit +virtualedit +visual +visualextra +viminfo +vreplace
+wildignore +wildmenu +windows +writebackup -X11 -xfontset +xim -xsmp
-xterm_clipboard -xterm_save
system vimrc file: "$VIM/vimrc"
user vimrc file: "$HOME/.vimrc"
user exrc file: "$HOME/.exrc"
system gvimrc file: "$VIM/gvimrc"
user gvimrc file: "$HOME/.gvimrc"
system menu file: "$VIMRUNTIME/menu.vim"
fall-back for $VIM: "/Applications/MacVim.app/Contents/Resources/vim"
Compilation: clang -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_MACVIM -Wall -Wno-unknown-pragmas -pipe -DMACOS_X_UNIX -no-cpp-precomp -g -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -I/System/Library/Frameworks/Tcl.framework/Headers -D_REENTRANT=1 -D_THREAD_SAFE=1 -D_DARWIN_C_SOURCE=1
Linking: clang -L. -L. -L/usr/local/lib -o Vim -framework Cocoa -framework Carbon -lncurses -liconv -framework Cocoa -fstack-protector -L/usr/local/lib -L/System/Library/Perl/5.12/darwin-thread-multi-2level/CORE -lperl -lm -lutil -lc -F/System/Library/Frameworks -framework Tcl -framework CoreFoundation -framework Ruby

[samueljohn]

In macvim I can do :python print(":-)"), I did not went beyond that test.
All I changed in macvim is that python is linked dynamically because macvim kept picking the wrong (system) python.

What is true: Homebrew does not add -F for HOMEBREW_PREFIX/Frameworks or for python. I could add this for python, but most formulae use python-config to get this information.
If you locally add ENV['ldflags'] = "-F#{HOMEBREW_PREFIX}/Frameworks"?

[pencilcheck]

Sure, I will add that in my zshrc

[samueljohn]

Not there. In brew edit macvim inside the install method.

[pencilcheck]

Still doesn't work...

I add it inside the install method.

Linking: clang -L. -L. -L/usr/local/lib -o Vim -framework Cocoa -framework Carbon -lncurses -liconv -framework Cocoa -fstack-protector -L/usr/local/lib -L/System/Library/Perl/5.12/darwin-thread-multi-2level/CORE -lperl -lm -lutil -lc -F/System/Library/Frameworks -framework Tcl -framework CoreFoundation -framework Ruby

[samueljohn]

Above I see you have python 2.7.3. Can you please brew update and brew upgrade python?

[pencilcheck]

I already upgraded python to 2.7.5

[samueljohn]

In my configure output from brew install macvim --with-python3 -v:

checking --enable-pythoninterp argument... yes
checking for python... /homebrew/opt/python/bin/python
checking Python version... 2.7
checking Python is 1.4 or better... yep
checking Python's install prefix... /homebrew/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7
checking Python's execution prefix... /homebrew/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7
checking Python's configuration directory... /homebrew/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config
checking if -pthread should be used... no
checking if compile and link flags for Python are sane... yes
checking --enable-python3interp argument... yes
checking for python3... /homebrew/opt/python3/bin/python3
checking Python version... 3.3
checking Python's abiflags... m
checking Python's install prefix... /homebrew/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3
checking Python's execution prefix... /homebrew/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3
checking Python's configuration directory... /homebrew/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/config-3.3m

so it seems like MacVim has found the correct python versions (both even!).
I tried with --enable-pythoninterp=yes and --enable-pythoninterp=dynamic. In both cases I can type :python print('hello'). I have not tested plugins like powerline, though.

How is this for you?

[pencilcheck]

Running with brew install macvim -v since I didn't have python3 installed and here is part of the output:
Btw, the macvim formula still has the custom ENV['ldflags'] = "-F#{HOMEBREW_PREFIX}/Frameworks" in it.

checking --enable-pythoninterp argument... dynamic
checking for python... /usr/local/opt/python/bin/python
checking Python version... 2.7
checking Python is 1.4 or better... yep
checking Python's install prefix... /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7
checking Python's execution prefix... /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7
checking Python's configuration directory... /usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config

It's weird how it also detects the correct python, but it just keep crashing on start for powerline and youcompleteme.
I tried to reinstall/recompile powerline and youcompleteme under the new python, but still no avail.

On the other hand, the homebrew vim seems to work out of the box, no warnings, and no errors... so weird.

[samueljohn]

By stops working, what do you mean? I can start it up. See the gui and type :python print "test" and it works. But I have no powerline or other module.

However, I just pushed a change with --enable-pythoninterp=yes (instead of dynamic), and indeed, if I
xcrun otool -L $(brew --prefix)/Cellar/macvim/7.3-66/MacVim.app/Contents/MacOS/Vim (perhaps without xcrun if you don't have Xcode), then I see that macvim links to /Library/Frameworks/Python.framework/Versions/2.7/Python.

[pencilcheck]

An error occurred while importing the Powerline package. This could be caused by an invalid sys.path setting, or by an incompatible Python version (Powerline requires Python 2.6+ or 3.2+ to work). Please consult the troubleshooting section in the documentation for possible solutions.

This is only the first error I see when I start up, then the errors for YouCompleteMe and powerline comes in in never ending cycle, thus I can't even go in vim to type :python at all, or even insert a letter.

It's weird that running xcrun otool -L $(brew --prefix)/Cellar/macvim/7.3-66/MacVim.app/Contents/MacOS/Vim shows that it is linked to system python as well.

[pencilcheck]

Error look something like this once I get into mvim:

Error detected while processing function youcompleteme#Enable:
line 9:
Traceback (most recent call last):
Press ENTER or type command to continue
Error detected while processing function youcompleteme#Enable:
line 9:
File "", line 1, in
Press ENTER or type command to continue
Error detected while processing function youcompleteme#Enable:
line 9:
File "/Users/pencilcheck/.dotfiles/.vim/bundle/YouCompleteMe/autoload/../python/ycm/extra_conf_store.py", line 24, in
Press ENTER or type command to continue
Error detected while processing function youcompleteme#Enable:
line 9:
import random
Press ENTER or type command to continue
Error detected while processing function youcompleteme#Enable:
line 9:
ImportError: No module named random
Press ENTER or type command to continue
Error detected while processing function youcompleteme#Enable:
line 10:
Traceback (most recent call last):

[samueljohn]

Ah so it fails for that youcompleteme module and not if you type :python print "test"!
That is why I didn't detect that. Can you check inside of the .vim/bundle/YouCompleteMe/autoload if there is something that could hardcode a path to system python?

[samueljohn]

But if you brew unlink python these plugins for MacVim do work?

[pencilcheck]

Yea, brew unlink python these plugins will start working in MacVim but not homebrew vim anymore. I forgot which python it is linked to, let me check.

[pencilcheck]

It's not only YouCompleteMe, the same with powerline as well when I link homebrew python and launch MacVim. There is no hardcoded path in YouCompleteMe though, at least in autoload/YouCompleteMe.vim.

[pencilcheck]

Still have no idea how to fix this. It seems like MacVim completely ignores static linking of python

[oryband]

I've opened an issue on Macvim's development page.

[samueljohn]

Thanks for reporting upstream.

[tareqak]

I've run into the same problem while trying to install the HEAD version of vim i.e. brew install vim --HEAD. The workaround described here worked perfectly for my case.

[oryband]

Vim isn't Macvim, and I guess that's why it worked. But thanks anyways.. :)

On 9 ביונ 2013, at 09:42, Tareq A Khandaker [email protected] wrote:

I've run into the same problem while trying to install the HEAD version of vim i.e. brew install vim --HEAD. The workaround described here worked perfectly for my case.

—
Reply to this email directly or view it on GitHub.

[ricardodani]

I`ve got the same here.

Renamed the Current to Current-sys, but my brewed macvim never`s get brewed Python 2.7.5.

But with the brewed vim i`ve got the brewed Python.

Tried all theses aproach`s.

[samueljohn]

Hey all, please NEVER CHANGE STUFF IN /System, even if people on StackOverflow say so :-)

MacVim now works with a brewed Python 2.x since ee0d6c0.

I tested with the powerline. Works nice for me. So please repair your /System dirs, brew update and brew rm macvim and finally brew install macvim. Note, I had to remove the python3 option.

[pencilcheck]

Wow, thanks for the patch, it indeed works at least for powerline.

[vandrada]

The patch works for YouCompleteMe as well. At least for me.

[samueljohn]

Cool, thanks for the feedback!

[dcow]

After updating to Xcode5-DP3 I get this error, however, it manifests when launching vanilla vim. I tried uninstalling vim and python and updating and reinstalling to no avail.

[oryband]

Attention, this bug reappered as of today.

Fresh install of python 2.7.6, macvim 7.4 patch 1-52 (+ brew unlink && brew link at proper times), I start macvim and:

:py import sys; print (sys.version, sys.executable)
('2.7.5 (default, Aug 25 2013, 00:04:04) \n[GCC 4.2.1 Compatible Apple LLVM 5.0
(clang-500.0.68)]', '/usr/local/opt/python/bin/python2.7')

Note the big bad 2.7.5 there.
Need to reopen this issue. What happened that is causing this?

I guess this somehow relates to #20392
@samueljohn please note this.

[MikeMcQuaid]

@oryband Please brew update, uninstall python, uninstall macvim, reinstall python and macvim and if it's still an issue please create a new one.

[kittykatattack]

I had this same problem.
I used brew to install the latest version of Python (2.5)
Macvim was using an earlier version of Python that comes with OSX (Mavericks).
Macvim lost its link to the older Python when I installed the brew version.

Here's my solution.
I unlinked Python, removed the current Macvim installation, and used brew to re-install it.

brew unlink python
brew remove macvim
brew install macvim

All my bundles and .vimrc file stayed intact... it works!!

[oryband]

@kittykatattack The reason it works for you now is because in the meanwhile the bug was fixed. See the issue references above.