README

Read Erlang source on stdin, reformat with erl_tidy, and write to stdout.
No options, you get what erl_prettypr:format/2 gives you.


                                  Back story


    The problem seemed simple: read Erlang source code on stdin and send a
    nicely formatted result to stdout.  In vim, I got used to this when I was
    learning Go:

            :%!gofmt

    Just type the code without any concern for formatting, and then format the
    entire file in one command.  I now have xmlfmt, jsfmt, javafmt and I
    wanted a similar utility for Erlang.

    I started with erl_tidy, which

            Tidies and pretty-prints Erlang source code, removing
            unused functions, updating obsolete constructs and
            function calls, etc.

    Seems perfect ... except that erl_tidy does not read from stdin.  I posted
    a question to the Erlang questions mailing list asking if I missed
    something and the answer was a clear no.  People pointed me to alternative
    approaches: an Emacs elisp module, a rebar3 module that wraps erl_tidy,
    and a state-machine/parser written in Erlang that the vim Erlang module
    uses.  The first two didn't solve my problem, and I didn't like the
    complexity of the third approach.

First attempt: use basic Erlang modules
    
    While Googling around, I found a short post showing how you can format
    Erlang code using erl_scan (string -> tokens), erl_parse (tokens -> form)
    and erl_pp (form -> string).  So I started coding that up.  It worked
    great. Initially.

    The first problem I hit was that erl_parse:parse_form/1 does not handle
    white space or comment tokens.  OK, no problem.  To keep moving forward I
    implemented the quick hack of only dropping comments that came inside a
    form, and keeping the ones that came before.  Not great, but I wanted to
    get something working.

    The next problem I hit was pre-processor constructs.  It turns out that
    erl_parse does not understand pre-processor bits either (macros, imports,
    etc).  So, another hack: when we hit a dot-terminated token sequence that
    includes a pre-processor construct, just print out the raw text that
    generated those tokens and leave it at that.

    When that was done, I decided dropping comments was not acceptable.  While
    searching around for how to re-insert comments, I came across a reference
    to the epp_dodger module and a function that re-inserts comments.  Both of
    which is used by erl_tidy.  It seemed stupid to re-write erl_tidy so I
    went back to square one.

Second attempt: teach erl_tidy about stdin

    Erlang can read stdin just fine.  In fact, most of the input functions in
    the io module read from stdin by default.  Erlang provides the standard_io
    atom that you can use for an IODevice argument.  I started hacking on
    erl_tidy, intending to use the "special" file name of a single dash (-) to
    tell erl_tidy to read from stdin.

    It was simple to add a new read_module("-", Opts) and pass standard_io to
    epp_dodger:parse/3.  But, the next chunk of erl_tidy logic reads comments
    from the same file again, which is not possible with stdin, since the
    stream was already consumed by parsing.

    I briefly looking to see if I could somehow turn a string into an IODevice
    (like you can in Java), but nothing turned up.  So, it seems like I have
    to write stdin to a file in order to use erl_tidy.

Third attempt: call erl_tidy:file/1 from a shell script.

    It was trivial to write a shell script that pipes stdin to a file.
    Calling erl_tidy from the shell script was not.  My first attempt looked
    like this:

            $ erl -run erl_tidy file $TMPFN

    which produced:

            =ERROR REPORT==== 9-May-2016::20:13:30 ===
            erl_comment_scan: bad filename: `['hmmmm_sup.erl']'

    and hung there, waiting in the Erlang interpreter.

    It turns out that when use the -run flag and pass it arguments, Erlang
    assumes the receiving function has one argument---a list.  That's why the
    filename in the error message has brackets around it ... it's an list not
    a string..

Final attempt: call escript from a shell script

    The final result is what you see in the repository now.   It's so easy to
    pipe stdin to a file in a shell script that I saw no reason to re-write
    that in Erlang.  So I added a short escript that unpacks the file name
    from the list and passes that to erl_tidy.

    The result: I was able to erlfmt all 1,795 source files under
    /usr/local/Cellar/erlang/18.2.1, with only one error:

            /usr/local/Cellar/erlang/18.2.1/lib/erlang/lib/wx-1.6/src/gen/gl.erl
            ./erlfmt: line 6: 25399 User defined signal 2: 31 ./erlfmt.escript "$TMPF"

    That file is a generated one, and is 971KB in size.  I got the above error
    when I had erl_tidy write the reformatted file to the temporary file.
    When instead I told erl_tidy to write the result to stdout, I got this
    error:

            escript: exception exit: badarg
              in function  erl_tidy:file/2 (erl_tidy.erl, line 295)
              in call from erl_eval:local_func/6 (erl_eval.erl, line 557)
              in call from escript:interpret/4 (escript.erl, line 787)
              in call from escript:start/1 (escript.erl, line 277)
              in call from init:start_it/1 
              in call from init:start_em/1 
            
    It successfully parsed the other 1,794 files, which gives success rate of
    0.99945.  Not too hot in the Erlang world, but I don't expect to be
    formatting files that big.  Good enough.

The back story back story.

    Why such a long README?  Erlang questions mailing list recently had an
    interesting thread on code documentation ("rhetorical structure of code").
    It was a long thread and a lot of things were said, but one in particular
    stuck with me: code documentation rarely shows the starts and stops, the
    litany of failed experiments that you encounter on the way to the final
    product.  And that those failed experiements are sometimes useful in the
    future.