Always generate line numbers debug information

[ysbaddaden]

Adds a Crystal::Debug flags enum to select the kinds of debug information to generate, and alters the build flags:

• default: only generate line numbers debug info (including function names);
• --debug: generate all debug info;
• --no-debug: don't generate any debug info.

For example, when compiling the crystal compiler, I see a 12% file increase for line numbers vs 45% for the full debug info (compile units, variables, ...)

• default: 26MB
• --debug: 34MB
• --no-debug: 23MB

[RX14]

I think this is a good default, however I would make --no-debug be the default with --release.

[ysbaddaden]

Why? LLVM means to have the best debugability even with aggresive optimizations. This adds line numbers to backtraces, which may lead to better reports.

[ysbaddaden]

Travis fails to compile specs on x86_64 linux ... while it works perfectly for me, and the error should happen on other platforms, too 😭 is the docker image outdated?

[asterite]

@ysbaddaden Yes, they are outdated. @jhass used to keep them up to date. Tomorrow I'll try to create docker images based on those ones and then use them in travis.

[asterite]

@ysbaddaden Actually, it seems the build is automatic. I don't know why it failed. I'll try downloading the image and checking if the none? method is there, or what's wrong...

[asterite]

Hmmm...

$ docker run -it --rm jhass/crystal-build-x86_64
root@c184614d3dcd:/# crystal version
Crystal 0.20.0 [b0cc6f7] (2016-11-22)

😕

[asterite]

I'd try to make this work without relying on Crystal 0.20.3. Maybe there was a problem with the deb package for x84-64, I don't know. But after I release 0.20.4 that should be fixed. If this can be implemented without none? then it will go into the 0.20.4 release, otherwise I'll merge it after the next release.

[ysbaddaden]

I tried to add a None member manually, but it then failed horribly on all platforms on Travis. No idea what happened again. Let's skip a release.

[asterite]

@ysbaddaden I'm not sure adding a None member will work. One last thing you can try is doing enum.value == 0, which should be equivalent and should compile.

[ysbaddaden]

I restarted the travis job. Let's see!

[ysbaddaden]

@asterite green at last!

[ysbaddaden]

@asterite merge?

[spalladino]

LGTM, merged. Thanks again @ysbaddaden!

[0x1eef]

@ysbaddaden i think it would be good if --release were to imply --no-debug because the binary will be smaller. --release targets a binary that could be distributed to a client or customer, so i don't think including debug information is that useful, and reduced binary size is a plus if you are offering downloads and so on.

[ysbaddaden]

• A 12% increase in binary size is negligible (some KB, a few MB at worst).
• LLVM has good support for debug even with aggressive optimizations.
• Line numbers in backtraces make better crash reports.

If you really want to shave a few KB off your binaries, at the expense of harder to decipher crash reports, you can pass --no-debug to change this default, then strip your binary.

[asterite]

@ysbaddaden This PR made the compiler not be able to reuse previous compilations, ever.

The main problem is that when code is inside a macro, to be able to debug it we generate a file named "macro#{some_id}.cr" and put that inside the debug info. The code for that is here. This some_id changes between compilations, right now it's file.object_id where file is VirtualFile.

Now, the culprit is not this PR but the way debug info is generated for macros, which was like that before this PR. Now it's more noticeable because debug info is always on (this is something that I personally like).

I'm not exactly sure how to solve this. Some ideas:

1. Not to emit debug info for macros, though I guess that will break some things.
2. Generate a single macros.cr file containing all macro expansions for the current program, but then line numbers need to be adjusted accordingly, and it's possible that if we change some code around then the order of execution of macros will change and then reusing a previous compilation won't be possible.
3. Only with Cyrstal::Debug::LineNumbers use the original_filename of the VirtualFile (through Location) though that will have the line numbers wrong when debugging or at exceptions, but in any case line numbers for backtraces where macros appear aren't very useful or informative.
4. Any other solution?

I think 3 is the easiest/fastest things to do to "fix" this (maybe even do it always, because stepping into macro code might not be very useful either), but I don't know what the general solution is. Macros are hard...

One quick way to notice if the compiler reuses a previous compilation is to do pp changed after this line. The second compilation of a same program should give all false. But with programs that use macros (basically, every program) then it always gives true now. I noticed this because compiling a program with an HTTP::Server in it was always slow. Previously it was only slow the first time. This is because HTTP::StaticFileHandler uses ECR, and that executes a macro run that compiles a program in release mode. The idea is that it compiles it in release mode because the second compilation will find it in the cache (the semantic phase is still done, but that's usually fast for macro run programs, though in any case I want to optimize that further by storing last modified dates of files or something like that). But now it won't find that in the cache, so the ECR/process program is compiled over and over, in release mode. An improvement we can do here is to compile macro run programs with Crystal::Debug::None, but that's just a hack, regular programs will still suffer from slow(er) compilations.

[asterite]

@ysbaddaden I'll soon send a PR to "fix" this, don't worry about a solution :-)

[ysbaddaden]

Great, because I really didn't have one... except maybe using a digest of the macro contents, instead of a random identifier.