Argcomplete is not completing in ZSH when prefix is present

[sblask]

I use ZSH and in general bash completions work well enough. However there is a problem that is bothering me. I wouldn't mind looking into it, but I have no idea where to start. Consider the following piece of code:

#!/usr/bin/env python
import argparse
import argcomplete


if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument(
        '-n',
        '--name',
        help='',
        choices=['123', '145']
    )
    argcomplete.autocomplete(parser)
    parser.parse_args()

When I save it to test.py, make it executable and run eval "$(register-python-argcomplete test.py)" I get completions, but only if there is no prefix present. For example ./test.py <tab> gives me --name and the other arguments. ./test.py --name <tab> gives me 123 and 145. I'd expect the same with ./test.py --name 1<tab> but I don't get anything. It's the same with ./test.py --n. However, when I use subcommands, argcomplete does complete things when a prefix is present. When I enable debug, I get the following in ZSH:

LINE: './test.py --name 1'
POINT: 20
PREQUOTE: ''
PREFIX: ''
SUFFIX: ''
WORDS: ['./test.py', '--name', '1']
Active parsers: [MonkeyPatchedIntrospectiveArgumentParser(prog='test.py', usage=None, description=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)]
Visited positionals: [MonkeyPatchedIntrospectiveArgumentParser(prog='test.py', usage=None, description=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)]
invoking parser with ['--name', '1']

exception <class 'SystemExit'> 2 while parsing args
all active parsers: [MonkeyPatchedIntrospectiveArgumentParser(prog='test.py', usage=None, description=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)]
active_parser: MonkeyPatchedIntrospectiveArgumentParser(prog='test.py', usage=None, description=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)
optional options: ['-h', '--help', '-n', '--name']
next_positional: None
Active actions (L=0): []
active options: ['-h', '--help', '-n', '--name']
display completions: {'-h --help': 'show this help message and exit', '-n --name': ''}

Returning completions: ['-h', '--help', '-n', '--name']
./test.py

And this in BASH (where I do get completions too):

LINE: './test.py --name 1'
POINT: 18
PREQUOTE: ''
PREFIX: '1'
SUFFIX: ''
WORDS: ['./test.py', '--name']
Active parsers: [MonkeyPatchedIntrospectiveArgumentParser(prog='test.py', usage=None, description=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)]
Visited positionals: [MonkeyPatchedIntrospectiveArgumentParser(prog='test.py', usage=None, description=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)]
invoking parser with ['--name']

exception <class 'SystemExit'> 2 while parsing args
all active parsers: [MonkeyPatchedIntrospectiveArgumentParser(prog='test.py', usage=None, description=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)]
active_parser: MonkeyPatchedIntrospectiveArgumentParser(prog='test.py', usage=None, description=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)
optional options: []
next_positional: None
Active actions (L=1): [IntrospectAction(option_strings=['-n', '--name'], dest='name', nargs=None, const=None, default=None, type=None, choices=['123', '145'], help='', metavar=None)]
Resetting completions because IntrospectAction(option_strings=['-n', '--name'], dest='name', nargs=None, const=None, default=None, type=None, choices=['123', '145'], help='', metavar=None) must consume the next argument
Activating completion for IntrospectAction(option_strings=['-n', '--name'], dest='name', nargs=None, const=None, default=None, type=None, choices=['123', '145'], help='', metavar=None) <class 'argparse._StoreAction'>
Completions: ['123', '145']
active options: ['123', '145']
display completions: {'145': '', '123': ''}

Returning completions: ['123', '145']

So apparently the prefix is not recognised as such. Any idea where the difference is coming from and how to fix it?

[evanunderscore]

This looks like it's caused by the same bug I came across in the second half of #228, which would mean I wasn't entirely correct about it not affecting completions from the end of a line. Can you update zsh or manually apply the particular fix I describe there and see if it fixes your problem?

[sblask]

I changed /usr/share/zsh/functions/Completion/bashcompinit as described in #228 and it's still not working. I kind of expected that at least the POINT= value in the debug output would change, but it doesn't. Or is it not the same as COMP_POINT ? I find it also strange that POINT is off by two with ZSH compared to bash while the fix would have adjusted it by one.

[evanunderscore]

That is showing the value of COMP_POINT, so if it hasn't changed, the code you updated isn't being used. The fix adjusts it by one token, not one. The bugged code is counting the completion prefix and the space before it twice. If you try ./test.py --name 14, you'll see it shows 22 when it should be 19.

I reproduced the problem using your script, made the modification (bashcompinit was located at /usr/share/zsh/5.0.2/functions/bashcompinit; I don't remember how I set this up), ran bashcompinit, and now my debug output shows the point as 18 and completion works as expected.

It looks like the fix for this was first released in zsh 5.5. Perhaps it's worth adding this caveat to the proposed documentation change in #245.

[sblask]

After some serious grepping I found the problem. I use https://github.com/aws/aws-cli/blob/develop/bin/aws_zsh_completer.sh which has a copy-paste version of _bash_complete - without the fix. Thanks for looking into this, it works as expected now that I applied the fix to that file too. I'll open a ticket in the aws repo.

[evanunderscore]

No problem, thanks for working through this with me, and thanks for the detailed initial bug report - it really helped!

[haifengkao]

I solved this problem by updating zsh from 5.3 to 5.7
the default zsh shell on OS X 10.14.5 is 5.3, which is too old