Writing extra characters when beautifying python

[marcoslhc]

When I save any python code this is added at the beginning of the file:

�(B�[m�(B�[mfrom ...

[apt-itude]

I am experiencing the same issue. Running autopep8 from the command line does not produce the extra characters.

[demosdemon]

+1, is there any workaround until this fix is finished?

[Glavin001]

I am unable to reproduce this issue.

Are you using beautify on save?

What version of Atom, Atom Beautify, and autopep8 do you have?

[apt-itude]

Not using beautify on save.

OS: Mac OSX 10.10.2
Atom version: 0.186.0
Atom Beautify version: 0.21.5
autopep8 version: 1.1

I just verified that the bug is still occurring with all of the above.

[Glavin001]

I am not able to reproduce with:

OS: Mac OSX 10.10.2
Atom version: 0.187.0
Atom Beautify version: 0.21.5
autopep8 version: 1.1

How are you executing beautify?

• beautify on save, or
• right-click file and Beautify File, or
• running beautify with keyboard shortcut
• running beautify from command palette

Does the extra characters appear immediately after beautifying? Only on save? etc.

I need to try and diagnose at what stage of the lifecycle is this error occurring, which is significantly harder since I am not experiencing this issue.
Any information may be helpful.

[apt-itude]

Sorry I didn't mean to mislead you about beautify on save. It actually does occur during a beautify on save, I just wasn't using that feature. In fact, I just verified that the issue occurs for all of the above methods of executing beautify. The characters appear immediately after execution, not after a save.

I thought it might be a weird artifact of the fact that I am using sshfs, so the editor is running on Mac and the code lives in a VM, but I just tested with a local file on the Mac, and the error still occurs.

[demosdemon]

Atom: 0.187.0
Atom Beautify: 0.21.5
autopep8: 1.1.1
OSX: 10.10.3dp2

Regardless if beautify on save is on or not, or if I manually beautify, it takes the following script

#!/usr/bin/env python
# coding=utf-8

import setup
def main():
    pass
if __name__ == '__main__':
    main()

and replaces it with

(B�[m#!/usr/bin/env python
# coding=utf-8

import setup


def main():
    pass
if __name__ == '__main__':
    main()

I feel like it has something to do with the encoding, because the extra bits look suspiciously like the UTF BOM.

[Glavin001]

Could you remove the # coding=utf-8 line? You mention problem with encoding, what is that line supposed to do? Maybe autopep8 does not handle it well.

[demosdemon]

No effect, I actually added the coding line to force autopep8 to interpret the file as UTF8.

It's not autopep8, and I was able to confirm this by changing

python_autopep8_path: "echo; /usr/local/bin/autopep8"

to include the echo statement to force a newline. With the extra newline, the extra bytes appear then the newline from echo, then the autopep8 output.

[Glavin001]

I wonder if it has something to do with

[ -f ~/.bash_profile ] && source ~/.bash_profile;[ -f ~/.bashrc ] && source ~/.bashrc;

could you run that in your Terminal?

[apt-itude]

Do you mean we should run that in a terminal and then launch atom from the terminal? If so, that was unsuccessful as well.

[Glavin001]

Run that command from Terminal and see if there is any output.

Atom Beautify is simply reading files, and autopep8 is writing to them. Both are pretty basic operations, so I'm trying to find out where those special characters are coming from. You mentioned that it happens even before the echo and that command above is the only other thing that is executed before that echo.

[demosdemon]

I'm not at my computer at the moment, but I will test when I get back. I do know that my bash_profile is setup to echo some control characters that iterm interprets into some GUI actions, so that's probably it. The default OSX bash_profile has that as well for Terminal.app to display the working directory in the title bar.

On Mar 12, 2015, at 1:32 PM, Glavin Wiechert [email protected] wrote:

Run that command from Terminal and see if there is any output.

Atom Beautify is simply reading files, and autopep8 is writing to them. Both are pretty basic operations, so I'm trying to find out where those special characters are coming from. You mentioned that it happens even before the echo and that command above is the only other thing that is executed before that echo.

—
Reply to this email directly or view it on GitHub.

[demosdemon]

[!3921][demosdemon@Marry Anne]
[~]$ /bin/sh -c '[ -f ~/.bash_profile ] && source ~/.bash_profile;[ -f ~/.bashrc ] && source ~/.bashrc; echo; /usr/local/bin/autopep8 --aggressive "/var/folders/ml/9xg4h3kj0js5yvg2j95_vbmw0000gn/T/input115212-45911-1dp94ew" --max-line-length 99 --indent-size 4 --ignore E24' | hexdump -C
/Users/demosdemon/.functions: line 253: syntax error near unexpected token `('
/Users/demosdemon/.functions: line 253: `    python <(cat <<EOF'
00000000  1b 28 42 1b 5b 6d 0a 23  21 2f 75 73 72 2f 62 69  |.(B.[m.#!/usr/bi|
00000010  6e 2f 65 6e 76 20 70 79  74 68 6f 6e 0a 0a 69 6d  |n/env python..im|
00000020  70 6f 72 74 20 73 65 74  75 70 0a 0a 0a 64 65 66  |port setup...def|
00000030  20 6d 61 69 6e 28 29 3a  0a 20 20 20 20 70 61 73  | main():.    pas|
00000040  73 0a 69 66 20 5f 5f 6e  61 6d 65 5f 5f 20 3d 3d  |s.if __name__ ==|
00000050  20 27 5f 5f 6d 61 69 6e  5f 5f 27 3a 0a 20 20 20  | '__main__':.   |
00000060  20 6d 61 69 6e 28 29 0a                           | main().|
00000068

[!3922][demosdemon@Marry Anne]
[~]$ /bin/sh -c '[ -f ~/.bash_profile ] && source ~/.bash_profile;[ -f ~/.bashrc ] && source ~/.bashrc;' | hexdump -C
/Users/demosdemon/.functions: line 253: syntax error near unexpected token `('
/Users/demosdemon/.functions: line 253: `    python <(cat <<EOF'
00000000  1b 28 42 1b 5b 6d                                 |.(B.[m|
00000006

Yeah, that's it. Something's printing the \x1B(B\x1Bml. (The stderr messages aren't being picked up by Atom but they're from using sh over bash, like require('child_process').exec does)

[apt-itude]

I'm not sure where the control characters are coming from, but both of the following actions eliminate the problem:

1. Comment out line 54 of cli-beautify.coffee (cmd = $path + cmd) so that the $PATH is not modified
2. Launch Atom graphically instead of from the terminal (I always launch it from the terminal because my linters don't play nice if the environment isn't set up properly)

@demosdemon do you also start Atom from the terminal?

[demosdemon]

I've found the culprit in my bash_profile, somewhere along the line I have a sourced file that sets up my terminal colors and bash_prompt, in it there's a tput command, tput sgr0 # reset colors. That is where the control characters are coming from, at least for me.

[apt-itude]

Huh... Same thing here. I'd like to keep that tput command in there, though. Do you have a good solution without just removing that from bash_profile?

[demosdemon]

I've currently moved the tput the start of my set_prompt so its echoed every time bash wants to display the prompt (which does not happen in this case), overkill but it'll stop this from happening.

Other than that, in #233 I suggest making the path sourcing optional. I can't think of any other way to get the end goal of getting the path from the user's profile without the side effects of the profile echoing control characters.

My only thought is that since, in my case, I'm already providing python_autopep8_path, the path finding is not necessary. But that would require some legwork to accommodate every language that uses CLI.

Another thing to think about is that stdout may not be necessary, since the cmd is being called from /bin/sh, redirection works. So instead of telling CLI to get the stdout, redirect the output of autopep8 to the temporary outputFile created.

[Glavin001]

Published a fix to v0.22.2

Let me know if that works for you for now until I have time to properly implement #233 and #164.