Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

split dwarf doesn't work with crate dependencies #81024

Closed
davidtwco opened this issue Jan 14, 2021 · 5 comments · Fixed by #89819
Closed

split dwarf doesn't work with crate dependencies #81024

davidtwco opened this issue Jan 14, 2021 · 5 comments · Fixed by #89819
Assignees
@davidtwco
Labels
A-debuginfo Area: Debugging information in compiled programs (DWARF, PDB, etc.) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@davidtwco
Copy link
Member

As reported in #77117 (comment) (cc @philipc), Split DWARF currently fails when compiling a crate with a dependency.

To reproduce this, compile backtrace-rs with the following command:

$ RUSTFLAGS="-Z split-dwarf=split" cargo +nightly --verbose build
@davidtwco
Copy link
Member Author

davidtwco commented Jan 14, 2021 •
edited
Loading

As of writing, here's what I've figured out - using backtrace-rs as an example, as that was the crate which was used when reporting the bug:

Compiling backtrace-rs with RUSTFLAGS="-Z split-dwarf=split" cargo +nightly --verbose build produces the following output:

  Compiling autocfg v1.0.1
   Compiling libc v0.2.82
   Compiling gimli v0.23.0
   Compiling adler v0.2.3
   Compiling cfg-if v1.0.0
   Compiling rustc-demangle v0.1.18
   Compiling object v0.22.0
     Running `rustc --crate-name autocfg /home/david/.cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/autocfg-1.0.1/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -Cembed-bitcode=no -C debuginfo=2 -C metadata=f40b69de65cfa16a -C extra-filename=-f40b69de65cfa16a --out-dir /home/david/Projects/rust/backtrace-rs/target/debug/deps -L dependency=/home/david/Projects/rust/backtrace-rs/target/debug/deps --cap-lints allow -Z split-dwarf=split`
     Running `rustc --crate-name build_script_build /home/david/.cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/libc-0.2.82/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -Cembed-bitcode=no -C debuginfo=2 -C metadata=299960e7a1692464 -C extra-filename=-299960e7a1692464 --out-dir /home/david/Projects/rust/backtrace-rs/target/debug/build/libc-299960e7a1692464 -L dependency=/home/david/Projects/rust/backtrace-rs/target/debug/deps --cap-lints allow -Z split-dwarf=split`
     Running `rustc --crate-name gimli --edition=2018 /home/david/.cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/gimli-0.23.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -Cembed-bitcode=no -C debuginfo=2 --cfg 'feature="read"' -C metadata=1558b68d5687196a -C extra-filename=-1558b68d5687196a --out-dir /home/david/Projects/rust/backtrace-rs/target/debug/deps -L dependency=/home/david/Projects/rust/backtrace-rs/target/debug/deps --cap-lints allow -Z split-dwarf=split`
     Running `rustc --crate-name cfg_if --edition=2018 /home/david/.cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/cfg-if-1.0.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -Cembed-bitcode=no -C debuginfo=2 -C metadata=ce59338d636be0e5 -C extra-filename=-ce59338d636be0e5 --out-dir /home/david/Projects/rust/backtrace-rs/target/debug/deps -L dependency=/home/david/Projects/rust/backtrace-rs/target/debug/deps --cap-lints allow -Z split-dwarf=split`
     Running `rustc --crate-name adler /home/david/.cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/adler-0.2.3/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -Cembed-bitcode=no -C debuginfo=2 -C metadata=b66c5323933d2b31 -C extra-filename=-b66c5323933d2b31 --out-dir /home/david/Projects/rust/backtrace-rs/target/debug/deps -L dependency=/home/david/Projects/rust/backtrace-rs/target/debug/deps --cap-lints allow -Z split-dwarf=split`
     Running `rustc --crate-name rustc_demangle /home/david/.cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/rustc-demangle-0.1.18/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -Cembed-bitcode=no -C debuginfo=2 -C metadata=882a963021f29db3 -C extra-filename=-882a963021f29db3 --out-dir /home/david/Projects/rust/backtrace-rs/target/debug/deps -L dependency=/home/david/Projects/rust/backtrace-rs/target/debug/deps --cap-lints allow -Z split-dwarf=split`
     Running `rustc --crate-name object --edition=2018 /home/david/.cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/object-0.22.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -Cembed-bitcode=no -C debuginfo=2 --cfg 'feature="archive"' --cfg 'feature="coff"' --cfg 'feature="elf"' --cfg 'feature="macho"' --cfg 'feature="pe"' --cfg 'feature="read_core"' --cfg 'feature="unaligned"' -C metadata=fdd8b8c7c909b622 -C extra-filename=-fdd8b8c7c909b622 --out-dir /home/david/Projects/rust/backtrace-rs/target/debug/deps -L dependency=/home/david/Projects/rust/backtrace-rs/target/debug/deps --cap-lints allow -Z split-dwarf=split`
     Running `/home/david/Projects/rust/backtrace-rs/target/debug/build/libc-299960e7a1692464/build-script-build`
   Compiling miniz_oxide v0.4.3
     Running `rustc --crate-name build_script_build --edition=2018 /home/david/.cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/miniz_oxide-0.4.3/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -Cembed-bitcode=no -C debuginfo=2 -C metadata=6f3f48770204354e -C extra-filename=-6f3f48770204354e --out-dir /home/david/Projects/rust/backtrace-rs/target/debug/build/miniz_oxide-6f3f48770204354e -L dependency=/home/david/Projects/rust/backtrace-rs/target/debug/deps --extern autocfg=/home/david/Projects/rust/backtrace-rs/target/debug/deps/libautocfg-f40b69de65cfa16a.rlib --cap-lints allow -Z split-dwarf=split`
     Running `rustc --crate-name libc /home/david/.cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/libc-0.2.82/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -Cembed-bitcode=no -C debuginfo=2 -C metadata=55acfc5d67d2fc7b -C extra-filename=-55acfc5d67d2fc7b --out-dir /home/david/Projects/rust/backtrace-rs/target/debug/deps -L dependency=/home/david/Projects/rust/backtrace-rs/target/debug/deps --cap-lints allow -Z split-dwarf=split --cfg freebsd11 --cfg libc_priv_mod_use --cfg libc_union --cfg libc_const_size_of --cfg libc_align --cfg libc_core_cvoid --cfg libc_packedN --cfg libc_cfg_target_vendor`
error: linking dwarf objects with `rust-llvm-dwp` failed: exit code: 1
  |
  = note: "rust-llvm-dwp" "-e" "/home/david/Projects/rust/backtrace-rs/target/debug/build/miniz_oxide-6f3f48770204354e/build_script_build-6f3f48770204354e" "-o" "/home/david/Projects/rust/backtrace-rs/target/debug/build/miniz_oxide-6f3f48770204354e/build_script_build-6f3f48770204354e.dwp"
  = note:
  = note: error: No such file or directory


error: aborting due to previous error

     Running `/home/david/Projects/rust/backtrace-rs/target/debug/build/miniz_oxide-6f3f48770204354e/build-script-build`
^C⏎

llvm-dwp expects to be able to find all of the dwo files referenced by the linked build_script_build-6f3f48770204354e binary. Using readelf -wi, we can inspect the DW_AT_GNU_dwo_name attributes to find the expected filenames (prepending DW_AT_comp_dir for the complete path). rustc should have kept DWARF object files around at this point in compilation, so "No such file or directory" error suggests a failure in that mechanism. However, there are DWARF object files referenced from previous compilations:

  Compilation Unit @ offset 0x2259:
   Length:        0x30 (32-bit)
   Version:       4
   Abbrev Offset: 0x447
   Pointer Size:  8
 <0><2264>: Abbrev Number: 1 (DW_TAG_compile_unit)
    <2265>   DW_AT_stmt_list   : 0xa76
    <2269>   DW_AT_comp_dir    : (indirect string, offset: 0x16b6): /home/david/Projects/rust/backtrace-rs/target/debug/deps
    <226d>   DW_AT_GNU_dwo_name: (indirect string, offset: 0x16ef): autocfg-f40b69de65cfa16a.autocfg.4pya73nn-cgu.0.rcgu.dwo
    <2271>   DW_AT_GNU_dwo_id  : 0x532a2a4a633d9dea
    <2279>   DW_AT_GNU_ranges_base: 0xf0
    <227d>   DW_AT_low_pc      : 0x0
    <2285>   DW_AT_ranges      : 0x4f0
    <2289>   DW_AT_GNU_addr_base: 0x70

During earlier compilation of autocfg, the DWARF object file referenced would have been created, but then deleted at the end of the compilation. This explains why adding -Csave-temps to the RUSTFLAGS environment variable (as suggested in the initial report) fixes the issue. If we added -Csave-temps to only the failing compiler invocation then that would be insufficient to resolve the issue.

autocfg's compilation does not produce a DWARF package because rustc will only produce one when it is producing an output binary. Even if a DWARF package had been produced, it would have been insufficient to resolve the problem, as llvm-dwp only loads DWARF object files (DWARF package files are only consumed by the debugger, and DW_AT_GNU_dwo_name is never actually modified to point to the package file, debuggers just know to look for sibling .dwp files of binaries):

https://github.com/llvm/llvm-project/blob/5cf2696317afb0631a4a09414ae40a4e226a905e/llvm/tools/llvm-dwp/llvm-dwp.cpp#L510-L537

Therefore, it is necessary that DWARF object files from dependencies still exist during the compilation of dependent crates (in ./target/{debug,release}/deps, the output directory of the dependency's compilation).

I've come up with a some potential approaches which would resolve this issue, and I'd appreciate feedback:

Adding a DWARF object output type

Unlike other output files of the compiler, DWARF objects are not requested with the --emit flag:

rust/compiler/rustc_session/src/config.rs

Lines 241 to 252 in 4275ef6

#[derive(Clone, Copy, PartialEq, Eq, Hash, Debug, PartialOrd, Ord)]
#[derive(Encodable, Decodable)]
pub enum OutputType {
Bitcode,
Assembly,
LlvmAssembly,
Mir,
Metadata,
Object,
Exe,
DepInfo,
}

Instead, when Split DWARF is enabled: DWARF object files are produced whenever an object file is emitted (we produce DWARF objects alongside object files in the same LLVMRustWriteOutputFile invocation, by providing a buffer as an additional argument to LLVM's addPassesToEmitFile); DWARF package files are created whenever a OutputType::Exe is requested; and DWARF object files are removed whenever we would remove object files.

We could choose to add a OutputType::DwarfObject, which would allow Cargo to pass --emit=dwarf-object,dep-info,metadata,link to the invocation of rustc for dependencies. When --emit=dwarf-object is provided, DWARF object files will not be removed at the end of the compiler invocation. Necessarily when --emit=dwarf-object is requested, object files would have to be created, but these would be removed if only DWARF objects were explicitly requested.

Through this mechanism, we could ensure that the DWARF object files of dependencies continue to exist after the compilation session and thus would be available for dependent compilations.

Adding DWARF objects to rlibs

Alternatively, we could add DWARF object files to rlib files during linking, and then in the dependent compilation, before invoking llvm-dwp, read the DWARF object files from the rlib and write them to the expected location.

Adding DWARF objects to the rlib would require appending the following snippet..

    for dwo in codegen_results.modules.iter().filter_map(|m| m.dwarf_object.as_ref()) {
        ab.add_file(dwo);
    }

..after this part of the compiler:

rust/compiler/rustc_codegen_ssa/src/back/link.rs

Lines 302 to 314 in 4275ef6

fn link_rlib<'a, B: ArchiveBuilder<'a>>(
sess: &'a Session,
codegen_results: &CodegenResults,
flavor: RlibFlavor,
out_filename: &Path,
tmpdir: &MaybeTempDir,
) -> B {
info!("preparing rlib to {:?}", out_filename);
let mut ab = <B as ArchiveBuilder>::new(sess, out_filename, None);
for obj in codegen_results.modules.iter().filter_map(|m| m.object.as_ref()) {
ab.add_file(obj);
}

Reading DWARF objects from the rlib would require similar code to the following function, but with a different value for METADATA_FILENAME (instead, this would be the filename of the DWARF object which should be retrieved):

rust/compiler/rustc_codegen_llvm/src/metadata.rs

Lines 21 to 41 in 4275ef6

fn get_rlib_metadata(&self, _: &Target, filename: &Path) -> Result<MetadataRef, String> {
// Use ArchiveRO for speed here, it's backed by LLVM and uses mmap
// internally to read the file. We also avoid even using a memcpy by
// just keeping the archive along while the metadata is in use.
let archive =
ArchiveRO::open(filename).map(|ar| OwningRef::new(Box::new(ar))).map_err(|e| {
debug!("llvm didn't like `{}`: {}", filename.display(), e);
format!("failed to read rlib metadata in '{}': {}", filename.display(), e)
})?;
let buf: OwningRef<_, [u8]> = archive.try_map(|ar| {
ar.iter()
.filter_map(|s| s.ok())
.find(|sect| sect.name() == Some(METADATA_FILENAME))
.map(|s| s.data())
.ok_or_else(|| {
debug!("didn't find '{}' in the archive", METADATA_FILENAME);
format!("failed to read rlib metadata: '{}'", filename.display())
})
})?;
Ok(rustc_erase_owner!(buf))
}

(The snippet above is from rustc_codegen_llvm, rustc_codegen_cranelift has a similar function)

If we don't already, we would need to record the codegen modules produced for the dependency. This would be required to know which sections to load from the rlib and write out for the llvm-dwp invocation to succeed.

This is quite a bit more complicated than the first solution, but limits the required changes to only rustc.

Always keep DWARF objects for library crates

Similar to "Adding a DWARF object output type" described above, but instead of having Cargo explicitly request that DWARF objects are kept, infer that a crate is a dependency by checking the crate type is a library and that an rlib is also being produced.

I'd appreciate any feedback on which of the above solutions is desirable, or if anyone has any alternatives that I haven't thought of yet.

@davidtwco davidtwco added the A-debuginfo Area: Debugging information in compiled programs (DWARF, PDB, etc.) label Jan 14, 2021
@davidtwco davidtwco self-assigned this Jan 14, 2021
@bjorn3
Copy link
Member

bjorn3 commented Jan 14, 2021

Alternatively, we could add DWARF object files to rlib files during linking, and then in the dependent compilation, before invoking llvm-dwp, read the DWARF object files from the rlib and write them to the expected location.

I was under the impression that this already happend. At least my FCP proposal for -Csplit-dwarf was made with this assumption.

@davidtwco
Copy link
Member Author

Alternatively, we could add DWARF object files to rlib files during linking, and then in the dependent compilation, before invoking llvm-dwp, read the DWARF object files from the rlib and write them to the expected location.

I was under the impression that this already happend. At least my FCP proposal for -Csplit-dwarf was made with this assumption.

I assumed that the -Csplit-debuginfo proposal wouldn't enable Split DWARF initially, only -Zrun-dsymutil, since it had only been in nightly for a short time.

@bjorn3
Copy link
Member

bjorn3 commented Jan 14, 2021

-Csplit-debuginfo will be gated behind -Zunstable-options for all targets other than macOS, but it is supposed to work for both Linux and Windows.

@apiraino
Copy link
Contributor

Nominated for discussion for next week's compiler meeting (Zulip discussion)

@apiraino apiraino added T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. and removed I-nominated labels Jan 21, 2021
@davidtwco davidtwco mentioned this issue Sep 19, 2021
Closed
@Mark-Simulacrum Mark-Simulacrum mentioned this issue Oct 7, 2021
Closed
@davidtwco davidtwco mentioned this issue Oct 12, 2021
Merged
@bors bors closed this as completed in a77cc64 Jan 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@davidtwco davidtwco

Labels
A-debuginfo Area: Debugging information in compiled programs (DWARF, PDB, etc.) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

cg: split dwarf for crate dependencies davidtwco/rust
3 participants
@davidtwco @apiraino @bjorn3