Black Vim Error Formatting File: Error invoking python_execute

[rollschild]

Describe the bug A clear and concise description of what the bug is.

When saving/formatting a python file via :Black in Vim, it produced the following error:

Error detected while processing function provider#python3#Call:
line   18:
Error invoking 'python_execute' on channel 4 (python3-script-host): 
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<string>", line 92, in Black
  File "<string>", line 128, in get_configs
AttributeError: module 'black' has no attribute 'find_pyproject_toml'

To Reproduce Steps to reproduce the behavior:

1. Open up some python file, writing something, and leave insert mode
2. Type :Black and hit Enter to save
3. Produce the error above

Expected behavior
Python code should be formatted without errors

Environment (please complete the following information):

• Version: 19.10b0
• OS and Python version: macOS 10.14.6 / Python 3.7.5

Does this bug also happen on master? To answer this, you have two options:
No I don't think so.
Additional context Add any other context about the problem here.

[duarteocarmo]

Same here!

[BurnhamG]

It looks like this function was added in #1273, so the Vim plugin was updated to call this function, while the installed package doesn't (yet) have this function. Looks like this comment holds the current solution:

You need to install black from sources instead of from PyPI, or downgrade the plugin to the last release. To upgrade black:

source ~/.vim/black/bin/activate  # make sure to install in the right venv
pip install --upgrade git+https://github.com/psf/black.git

Would be nice if these kinds of breaks could be avoided in the future, although I am not sure how. Problem is that Plug or Vundle pulls the black vim plugin from current master but black gets installed only for releases on PyPI. Perhaps you should point Vundle or whatever at a tagged release, if possible.

[evrenkutar]

for a temp local solution this works for me:

cd ~/.vim/bundle/black/
git checkout ce14fa8b497bae2b50ec48b3bd7022573a59cdb1

or to checkout to the version tag: git checkout 19.10b0 instead.

[rollschild]

for a temp local solution this works for me:

cd ~/.vim/bundle/black/
git checkout ce14fa8b497bae2b50ec48b3bd7022573a59cdb1

or to checkout to the version tag: git checkout 19.10b0 instead.

I'm using vim-plug so I did a cd ~/.vim/plugged/black. But this workaround works for me. Thanks!

[bkjohnson]

Here's a workaround using vim-plug that should be a bit easier to manage:

Plug 'psf/black', { 'tag': '19.10b0' }

[im-n1]

Just ran into this. This basicaly makes whole Black unusable in Vim.

[traviswaelbro]

Just ran into this. This basicaly makes whole Black unusable in Vim.

The problem is with the installed black package not having the functionality that the vim plugin is trying to use. Upgrade your installed package as @BurnhamG suggested and you shouldn't have any more issues in vim.

[rpdelaney]

The fix from @bkjohnson has the advantage of pinning the vim version to match the installed release. Thank you for that.

[mkinet]

I had the same issue and could solve it with the proposed fix. However, I observed that it only occurred in the python environments in which pynvim was installed.

[traviswaelbro]

I had the same issue and could solve it with the proposed fix. However, I observed that it only occurred in the python environments in which pynvim was installed.

For what it's worth, I also have pynvim installed in my case.

[rpdelaney]

I do not

[niuxuliang929]

使用以下解决方法vim-plug应该更容易管理：

Plug 'psf/black', { 'tag': '19.10b0' }

Your method is very good, thank you!

[dlukes]

With vim-plug, you can even use a wildcard tag, to avoid having to specify a particular tagged version and updating your init file when a new black version is released:

Plug 'psf/black', { 'tag': '*' }

[duarteocarmo]

@ambv sorry to at you, but do you have an idea when this could be potentially be fixed? Thanks for your great effort and amazing job!

[ldragan]

The solution from #1273 didn't work for me since I had black installed at the system level and that was the one picked up so I had to do something like

pip3 install --upgrade git+https://github.com/psf/black.git

which seems to have replaced my systemwide install of black with the tip of git.

[weaming]

I added this key mapping Alt + f to use external black command in $PATH:

au FileType python nnoremap <buffer> <a-f> :silent !black -S %<CR><CR>

No more black plugin!

[dlukes]

No more black plugin!

That can be noticeably slower though, as you incur black's startup time cost each time you format.

[sjoegren]

I ran into problems when Vim was upgraded to 8.2.587.

My temporary workaround:
I have the vim plugin installed with vim-plug, make sure it's using master branch (since the released black/vim-black aren't compatible with latest vim):

Plug 'psf/black'

Update vim-plugin:

:PlugUpdate black

Upgrade black used by the vim-plugin to the latest from master:

cd ~/.vim
black/bin/pip install --upgrade git+https://github.com/psf/black.git

This should work when there is a new release of Black on PyPI I reckon.

[dlukes]

@akselsjogren That gets you a bleeding edge version of black though, which might be asking for trouble. If you want to keep using the latest PyPI release of black, you can just ask vim-plug to install the plugin from the latest tag in the repo, which should correspond to the latest version released on PyPI: Plug 'psf/black', { 'tag': '*' } (and rerun PlugUpdate black obviously).

[sjoegren]

@akselsjogren That gets you a bleeding edge version of black though, which might be asking for trouble. If you want to keep using the latest PyPI release of black, you can just ask vim-plug to install the plugin from the latest tag in the repo, which should correspond to the latest version released on PyPI: Plug 'psf/black', { 'tag': '*' } (and rerun PlugUpdate black obviously).

As I wrote, the bleeding edge of master (8654e8d) is what is needed to get the plugin working with later vim versions.

[dlukes]

As I wrote, the bleeding edge of master (8654e8d) is what is needed to get the plugin working with later vim versions.

Oh right, sorry :) I misunderstood the purpose of your comment because that's a somewhat different issue -- the latest release of the plugin being incompatible with latest Vim, and how to get around that.

This issue is about the fact that current installation instructions are likely to result in different versions of the black Vim plugin vs. Python library (master vs. latest PyPI release). For which I think the safer solution is to stay off the bleeding edge if you can afford to (Neovim, Vim older than 8.2.587, thanks for pointing that out).

But maybe black devs will chime in to say that master is safe and the ultimate solution to this issue might be that the plugin switches to installing black from git+https instead of PyPI, who knows.

[erikr]

@dlukes I'm on macOS 10.15.4, vim 8.2 (installed via home-brew, +python3). I tried your fix but cannot run black in vim. This is frustrating, as I run black in vim without issues on my Ubuntu machines following the default installation instructions.

I put Plug 'psf/black', { 'tag': '*' } in my .vimrc, installed Black via vim -c PlugInstall, and the latest tag was 19.10b0:

However, when I run vim, I get the following error:

Error detected while processing /Users/erik/.vim/plugged/black/plugin/black.vim:
line   43:
E221: Marker cannot start with lower case letter
  File "<string>", line 1
    << endpython3
    ^
SyntaxError: invalid syntax
line   44:
E1042: import/export can only be used in vim9script
line   45:
E1042: import/export can only be used in vim9script
line   46:
E1042: import/export can only be used in vim9script
line   48:
E1077: Missing argument type for exec_prefix
line   49:
E488: Trailing characters:   try:
line   50:
E492: Not an editor command:     default = vim.eval("g:pymode_python").strip()
line   51:
E492: Not an editor command:   except vim.error:
line   52:
E492: Not an editor command:     default = ""
line   53:
E121: Undefined variable: default
line   54:
E133: :return not inside a function
line   56:
E133: :return not inside a function
line   57:
E133: :return not inside a function
line   59:
E1077: Missing argument type for venv_path
line   61:
E133: :return not inside a function
line   62:
E133: :return not inside a function
line   64:
E1077: Missing argument type for venv_path
line   66:
E133: :return not inside a function
line   67:
E133: :return not inside a function
line   69:
E1056: expected a type:
line  152:
E1057: Missing :enddef
line  153:
E171: Missing :endif
Press ENTER or type command to continue

If I:

1. edit $HOME/.vim/plugged/black/plugin/black.vim and change endpython3 to Endpython3, or
2. cd $HOME/.vim/plugged/black/plugin && git checkout master -- black.vim

and run vim, I get a new error:

Error detected while processing /Users/erik/.vim/plugged/black/plugin/black.vim:
line  147:
Traceback (most recent call last):
  File "<string>", line 63, in <module>
ModuleNotFoundError: No module named 'black'
Press ENTER or type command to continue

Do you or @akselsjogren have other suggestions? Thanks in advance for your time.

[erikr]

@weaming's workaround did not work for me.

I made my own workaround in ~/.vimrc to call black from the command line within vim by hitting F9:

nnoremap <F9> :silent execute "! /usr/local/bin/black % &" <bar> redraw!<CR>

but I still have to hit ctrl+L or enter a few times to get the screen to redraw.

[weaming]

@weaming's workaround did not work for me.

I made my own workaround in ~/.vimrc to call black from the command line within vim by hitting F9:

nnoremap <F9> :silent execute "! /usr/local/bin/black % &" <bar> redraw!<CR>

but I still have to hit ctrl+L or enter a few times to get the screen to redraw.

@erikr Why put the process into background using & ? I can't understand the purpose of <bar> redraw!.

[erikr]

@weaming Running the process in the background gives an illusion of speed, since this approach incurs startup time per @dlukes.

If I call the command without redraw, the screen goes blank and the output from black from the command line appears and the vim editor disappears until I redraw manually via ctrl + L:

[weaming]

@erikr Try au FileType python nnoremap <buffer> <F9> :silent !black -S %<CR><CR>.
First <CR> to execute command, second <CR> to press Enter for you automatically.

[dlukes]

@erikr The solution I suggested makes sure that both the Vim plugin and the Python library are set to the latest released version. Unfortunately, as @akselsjogren has pointed out, Vim 8.2.587 is incompatible with the latest released version of the plugin, so your best bet is to use the master branch version for both until the next release happens.

[dlukes]

By the way, I think this issue can now be closed thanks to #1318. For vim-plug, the officially recommended way to install the stable version of the plugin now seems to be Plug 'psf/black', { 'branch': 'stable' }.

[erikr]

@weaming @dlukes Thanks so much for both of your comments.

I cleaned up prior installs of black:

rm -rf ~/.vim/black
rm -rf ~/.vim/plugged/black

Then changed my .vimrc to Plug 'psf/black', { 'branch': 'stable' }.

Then ran vim -c PlugInstall successfully.

However I still get this error:

Error detected while processing /Users/erik/.vim/plugged/black/plugin/black.vim:
line   43:
E221: Marker cannot start with lower case letter
  File "<string>", line 1
    << endpython3

Any other steps you'd suggest?

[dlukes]

@erikr You can't currently use the stable branch of black with Vim 8.2.587, so the advice to Plug 'psf/black', { 'branch': 'stable' }, which solves the original problem that this issue is about, won't help you. You'll end up with versions of the black Vim plugin and Python library that are mutually compatible, but the plugin will still be incompatible with your Vim version (that's the error you're getting, and it's addressed in #1347 and #1348).

Please follow the instructions by akselsjogren that I've linked to in my previous reply, they should help you make sure that you have the latest master of both the Vim plugin and the Python library.

EDIT: And once #1348 makes it into a stable release, then consider switching to Plug 'psf/black', { 'branch': 'stable' }, as it might not be a good idea to stay on the bleeding edge if not needed :) Just remember to make sure you have the correct version of the black Python library alongside the plugin (e.g. by wiping both the virtualenv and the plugin and reinstalling, if you want to make extra sure).

[erikr]

Hi @dlukes Thanks for clarifying.

I am trying to perform the following instructions from @akselsjogren that you linked to:

Upgrade black used by the vim-plugin to the latest from master:

cd ~/.vim
black/bin/pip install --upgrade git+https://github.com/psf/black.git

However I do not have pip in bin:

$ tree ~/.vim/black
.
├── bin
│   ├── python
│   ├── python3
│   └── vim
├── include
├── lib
│   └── python3.8
└── pyvenv.cfg

#1375 seems similar.

What step am I missing here?

[dlukes]

@erikr That's weird, but I can confirm this happens with Homebrew Vim 8.2.654 on macOS (I normally use Neovim). It seems to be a completely unrelated problem though, so please file a different issue for that. Feel free to re-use any information provided in the Details section below :) (I made it collapsible because I'd rather not hijack this thread any more than it already has been, the issue mentioned in OP has been addressed and this issue should be closed.)

Details

When the virtualenv is created, vim gets somehow substituted instead of python3 as the executable name when ensurepip is run:

Please wait, one time setup for Black.
Creating a virtualenv in /Users/david/.vim/black...
(this path can be customized in .vimrc by setting g:black_virtualenv)
Error detected while processing /Users/david/.vim/plugged/black/plugin/black.vim:
line  195:
Traceback (most recent call last):
  File "<string>", line 87, in <module>
  File "<string>", line 70, in _initialize_black_env
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/venv/__init__.py", line 391, in create
    builder.create(env_dir)
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/venv/__init__.py", line 68, in create
    self._setup_pip(context)
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/venv/__init__.py", line 289, in _setup_pip
    subprocess.check_output(cmd, stderr=subprocess.STDOUT)
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 411, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/Users/david/.vim/black/bin/vim', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

On that last line, it should be '/Users/david/.vim/black/bin/python3', not '/Users/david/.vim/black/bin/vim'. So ensurepip fails and the installation of the black Python package aborts. Not sure why.

If you can't see that error message, wipe the black virtualenv and restart Vim. If it flies by too quickly, you can redisplay it with :messages.

[NISH1001]

Here's a workaround using vim-plug that should be a bit easier to manage:

Plug 'psf/black', { 'tag': '19.10b0' }

Hi, I upgraded all the vim-plug plugins recently. And it seems black was broken with the same error. I had to switch to that tag 19.

Is this happening with recent release too?

[vv9k]

Hi, I have a similar situation. I'm also getting errors about pyproject_toml with stable branch of Black. Specifying { 'tag': '19.10b0' } resolves the issue.

[bataak]

Guys, please just update your pip version to 20 and python3 to 3.9 then that problem will be solved. At least you can use the latest version of black on your vim.

[wwwehr]

for lua configs, this worked for me:

use { 'psf/black', tag = '19.10b0', cmd = {'Black'}}

[hauntsaninja]

I think this has been fixed by later releases and #1318