dwarfdump errors on debug info generated by rustc 1.19 in Firefox build

[luser]

I was doing some inspection of Firefox debug symbols related to a crash, and dwarfdump errored out on me. The binary I was dumping is here, which is from a fairly recent Firefox nightly build (2017-08-16). I was running dwarfdump -S any=Element22AddSizeOfExcludingThis ./libxul.so.dbg when it crashed, but it looks like any dwarfdump command that causes it to read far enough into the DWARF will make it barf. Just asking it to print the DIEs for that CU makes it error pretty quickly: dwarfdump -u /builds/worker/workspace/build/src/media/libcubeb/cubeb-pulse-rs/src/lib.rs ./libxul.so.dbg.

I'm using a fairly recent dwarfdump on Ubuntu 17.04:

$ dwarfdump -V
 2016-11-24 14:21:17-08:00 

cc @michaelwoerister

[michaelwoerister]

I can reproduce with dwarfdump from Ubuntu 16.04:

$ dwarfdump -V
 Tue Apr 10 11:43:32 PDT 2012

[tromey]

Both readelf (from GNU binutils) and eu-readelf (from elfutils) have no trouble with this file. I wonder if this is a dwarfdump bug rather than a Rust bug. (I tried building dwarfdump from source here, but that build doesn't recognize the file type, dunno what's going on there.)

[cuviper]

@tromey, have you tried Fedora's own package? /usr/bin/dwarfdump is in libdwarf-tools.

[tromey]

Thanks @cuviper - I'd looked for it but didn't manage to find that package. I get:

dwarfdump ERROR:  reference form with no valid local ref?!, offset=<0x008cf760>:  DW_DLE_ATTR_FORM_OFFSET_BAD (119)

dwgrep is the simplest way to extract stuff from this enormous file, using it I found:

[8cf752]	inlined_subroutine
	abstract_origin	[8edf65] subprogram
	entry_pc	0
	ranges	<empty range>
	call_file	"/builds/worker/workspace/build/src/xpcom/threads/nsMemoryPressure.cpp"
	call_line	52
	sibling	[8cf7eb] inlined_subroutine
[8cf76a]	formal_parameter
	abstract_origin	[8edf7e] formal_parameter
	location	<no value>

It's hard to say which part is troubling dwarfdump without more information than dwgrep gives.

[tromey]

And note that this doesn't seem to come from a Rust CU at all (but to clarify, not saying this wasn't originally the case; it's possible I'm seeing a different failure than the OP)

[tromey]

I let eu-readelf grind on this and it says:

 [8cf752]      inlined_subroutine
               abstract_origin      (ref4) [8edf65]
               entry_pc             (addr) +000000000000000000 <xul57a1>
               ranges               (sec_offset) range list [ eb5d0]
               call_file            (data1) 99
               call_line            (data1) 52
               sibling              (ref4) [8cf7eb]

Still not that helpful; though I did check that those ref4s are defined.

[luser]

And note that this doesn't seem to come from a Rust CU at all (but to clarify, not saying this wasn't originally the case; it's possible I'm seeing a different failure than the OP)

Interesting! If you run:
dwarfdump -u /builds/worker/workspace/build/src/media/libcubeb/cubeb-pulse-rs/src/lib.rs ./libxul.so.dbg

does it still barf on that same CU?

The other possible culprit here is GCC 6, although I think that binary was built from a nightly before we switched (per https://hg.mozilla.org/mozilla-central/rev/db36870cee64).

[tromey]

does it still barf on that same CU?

Yes, that's how I got the error I saw.

The other possible culprit here is GCC 6, although I think that binary was built from a nightly before we switched (per https://hg.mozilla.org/mozilla-central/rev/db36870cee64).

dwgrep is fantastic.

$ dwgrep -e 'entry (child offset > 0x008cf700) (child offset < 0x8cf900) ?TAG_compile_unit @AT_producer' libxul.so.dbg  
GNU C++11 6.4.0 -mtune=generic -march=x86-64 -g -O3 -std=gnu++11 -fPIC -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -fprofile-use -fprofile-correction -fno-omit-frame-pointer

[philipc]

@tromey I think 0x008cf760 is referring to the offset value that is invalid, not the offset where the invalid value is. Have a look at the DIE that is offset 0x0000f6c2 within the CU that starts at 0x2aea982f. Here's what gimli's dwarfdump gives:

< 5><0x0000f6c2>            DW_TAG_formal_parameter
                              DW_AT_name                  f
                              DW_AT_decl_file             0x000000caUnable to get header for file 202

                              DW_AT_decl_line             0x00000001
                              DW_AT_type                  <0x008cf760>

This is within a unit that is only length 0x1a154. I'm not sure what form that DW_AT_type is, but if it is a unit-relative offset then it seems wrong.

[tromey]

I think 0x008cf760 is referring to the offset value that is invalid, not the offset where the invalid value is.

Haha, yeah, that makes sense, doesn't it.
I'll re-run eu-readelf and see what it says about the form, etc.

[tromey]

Just wanted to report back - after several fun adventures I still don't have a good answer; but I do think my earlier statements were wrong and there's a bug here.

[steveklabnik]

Triage: has anyone seen this lately? both rust and firefox are quite different, two years later.

[luser]

I tried with the libxul.so.dbg from Firefox 70 (https://s3-us-west-2.amazonaws.com/org.mozilla.crash-stats.symbols-public/v1/libxul.so/4B768AB4B534F6A880006855AE30129E0/libxul.so.dbg.gz) and the dwarfdump on Ubuntu 19.04 (2018-08-09 08:32:59-07:00) and it seems to be able to read the entire file without issue, so this doesn't seem to be an issue anymore.