Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rustc_mir: add a pass for fragmenting locals into their fields (aka SROA). #48300

Closed
wants to merge 4 commits into from

Conversation

eddyb
Copy link
Member

@eddyb eddyb commented Feb 17, 2018

In order to make other MIR optimizations more effective and/or easier to implement, this pass breaks up ("fragments") aggregate locals into smaller locals, ideally leaf fields of primitive or generic types.

This roughly corresponds to LLVM's SROA ("Scalar Replacement of Aggregates"), although "scalar" is less applicable here, as MIR doesn't distinguish between register-like SSA and memory.

Locals are fragmented only when all accesses are directly through field/downcast projections, so that the number of statements is unchanged (ignoring Storage{Live,Dead} ones).
For example, x = y; isn't transformed to x.f = y.f; for each field, instead it completely prevents x and y from being split up in any way.

Variable debuginfo is always preserved by this pass, by transforming it into a "composite" form, which maps pieces of an user variable to independent Places.
That corresponds to DWARF's DW_OP_piece "composition operator", which LLVM exposes as a more "random-access" DW_OP_LLVM_fragment (as each llvm.dbg.declare can only point to one alloca), indicating what byte range of the debugger-facing variable is being declared.

However, enums can't be broken up into their discriminant and variant fields, if any debuginfo refers to them, as there is more than one possible memory location for the enum bytes after the discriminant, and the debugger would have to inspect the discriminant to use the right variant's fragmented fields.
And LLVM doesn't currently (and couldn't easily AFAICT) support the more advanced DWARF features which would let us check the discriminant and use different locations based on it.
(While there is a workaround, namely faking type debuginfo for such cases, so the variants appear laid out like in a tuple instead of overlapping, it's complex enough that I don't want to tackle it in this PR)


As an example, this small snippet:

let mut pair = (1, 2);
pair = (pair.1, pair.0);

currently produces this MIR (after deaggregation, and some details omitted):

scope 1 {
    debug pair => _1;
}

bb0: {
    (_1.0: i32) = const 1i32;
    (_1.1: i32) = const 2i32;
    _2 = (_1.1: i32);
    _3 = (_1.0: i32);
    (_1.0: i32) = _2;
    (_1.1: i32) = _3;
}

but after this PR, the pair is replaced with two separate locals:

scope 1 {
    debug pair: (i32, i32) {
        .0 => _3,
        .1 => _4,
    };
}

bb0: {
    _3 = const 1i32;
    _4 = const 2i32;
    _1 = _4;
    _2 = _3;
    _3 = move _1;
    _4 = move _2;
}

The only thing left of the original pair is the debuginfo, which describes how a debugger can reconstruct the pair user variable, by combining the contents of _3 and _4.

More concretely, the debugger sees a single pair: (i32, i32) variable with contents drawn from:

  • _3, for bytes 0..4
  • _4, for bytes 4..8

But outside of a debugger, those two halves are completely independent.


r? @nikomatsakis cc @rust-lang/wg-mir-opt @michaelwoerister

TODO: address review comments (about missing code comments, and some notes to self)

@rust-highfive

This comment has been minimized.

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 17, 2018
@eddyb
Copy link
Member Author

eddyb commented Feb 17, 2018

r? @nikomatsakis

@eddyb
Copy link
Member Author

eddyb commented Feb 17, 2018

@bors try

@bors
Copy link
Contributor

bors commented Feb 17, 2018

⌛ Trying commit 9a18264 with merge b8d8ea6...

bors added a commit that referenced this pull request Feb 17, 2018
rustc_mir: add a pass for splitting locals into their fields (aka SROA).

**DO NOT MERGE**: based on #48052.
@bors
Copy link
Contributor

bors commented Feb 17, 2018

💔 Test failed - status-travis

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 17, 2018
@eddyb
Copy link
Member Author

eddyb commented Feb 19, 2018

@bors try

@bors
Copy link
Contributor

bors commented Feb 19, 2018

⌛ Trying commit 49f1ddaec0f5c745909c5797cc030eb5448473db with merge d2592cc1fd6fd56b975206a2d1c3e4f4978be3a6...

@bors
Copy link
Contributor

bors commented Feb 19, 2018

☀️ Test successful - status-travis
State: approved= try=True

@eddyb
Copy link
Member Author

eddyb commented Feb 20, 2018

This comparison is a bit more accurate (this PR is rebased on top of #48052).

Copy link
Contributor

@nikomatsakis nikomatsakis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pretty nifty. Left a bunch of requests, mostly for comments. Will take another pass after that.

src/librustc_mir/transform/split_local_fields.rs Outdated Show resolved Hide resolved
src/librustc_mir/transform/split_local_fields.rs Outdated Show resolved Hide resolved
src/librustc_mir/transform/split_local_fields.rs Outdated Show resolved Hide resolved
}

impl<'tcx> LocalPathCollector<'tcx> {
fn place_path(&mut self, place: &Place<'tcx>) -> Option<&mut LocalPath<'tcx>> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be nice to have a comment on this function -- even better if it can reference some kind of running example explaining what this optimization does.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, this setup is pretty elegant. I was getting worried about calls to make_opaque coming later and destroying the LocalPath instances that are returned by this function -- but of course the borrow checker prevents us from relying on them...

use syntax_pos::Span;
use transform::{MirPass, MirSource};

pub struct SplitLocalFields;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be great to have a doc comment here that explains what this optimization does and gives some examples. Ideally, those examples can be referenced from comments later on.

first..mir.local_decls.len()
}
}
}).collect::<IndexVec<Local, _>>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would have found it helpful to write the Range<Local> here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or -- better -- add a comment atop the let like:


// Map each local variable X to a range of locals L1..L2 that replaces it.
// If the local is opaque, then this will just be `X..X+1`.
// If the local is being "unrolled", then it will be a range of variables, one for each unrolled field.
let replacements = ...;

}
drop(replacements);

// Lastly, replace all the opaque paths with their new locals.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, ok, I think I get it. The "leaves" are always opaque...

None
}

fn split_into_locals(&mut self,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment plz. My take:


Given some local variable base_decl that is to be unrolled, creates N local variables -- one for each opaque leaf path reachable from base_decl. These are appending to local_decls.

}
}

fn project(&mut self, elem: &PlaceElem<'tcx>, variant_index: usize) -> Option<&mut Self> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment plz -- this seems to be a pretty key function. My take:


Tries to "unroll" P and create a local variable for its projection F. If P is opaque, or P cannot be unrolled, then returns None. Otherwise, returns Some with the LocalPath representing P.F (this path may in turn be unrolled or made opaque).

During the first phase, where we determine what to unroll, we invoke this function on each path that we see being used. For example, if we see a MIR statement like Use(a.b.c), we would invoke "project" b from a, and then c from that result.

Note that a projection can succeed at first, but then later have some base path made opaque, leading to a failure if the projection is re-attempted. For example, if we later saw a use of a.b, then it would be made opaque, and attempting to project c again would return None.

}

impl<'a, 'tcx> LocalPathReplacer<'a, 'tcx> {
fn replace(&mut self, place: &mut Place<'tcx>) -> Option<&mut LocalPath<'tcx>> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment plz =)

@nikomatsakis
Copy link
Contributor

@eddyb what's up with this code?

@shepmaster
Copy link
Member

Ping from triage, @eddyb ! We haven't heard from you in a while; will you be able to address some of the review comments soon?

@pietroalbini
Copy link
Member

Closing this PR for inactivity. @eddyb please reopen this if you have some time to finish the PR!

@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2020-01-29T13:05:10.8435997Z ========================== Starting Command Output ===========================
2020-01-29T13:05:10.8437800Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/ac045e1e-a13c-487a-9929-331a9e16401a.sh
2020-01-29T13:05:10.8437838Z 
2020-01-29T13:05:10.8443577Z ##[section]Finishing: Disable git automatic line ending conversion
2020-01-29T13:05:10.8450248Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/48300/merge to s
2020-01-29T13:05:10.8451989Z Task         : Get sources
2020-01-29T13:05:10.8452023Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-01-29T13:05:10.8452101Z Version      : 1.0.0
2020-01-29T13:05:10.8452135Z Author       : Microsoft
---
2020-01-29T13:05:11.8850352Z ##[command]git remote add origin https://github.com/rust-lang/rust
2020-01-29T13:05:11.8861303Z ##[command]git config gc.auto 0
2020-01-29T13:05:11.8863868Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2020-01-29T13:05:11.8866052Z ##[command]git config --get-all http.proxy
2020-01-29T13:05:11.8969945Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/48300/merge:refs/remotes/pull/48300/merge
---
2020-01-29T14:07:40.1065228Z .................................................................................................... 1700/9558
2020-01-29T14:07:45.4422463Z .................................................................................................... 1800/9558
2020-01-29T14:07:59.4364681Z .........................i.......................................................................... 1900/9558
2020-01-29T14:08:07.0568469Z .................................................................................................... 2000/9558
2020-01-29T14:08:23.3869930Z ...............iiiii................................................................................ 2100/9558
2020-01-29T14:08:34.1464604Z .................................................................................................... 2300/9558
2020-01-29T14:08:36.7457832Z .................................................................................................... 2400/9558
2020-01-29T14:08:42.2677275Z .................................................................................................... 2500/9558
2020-01-29T14:09:05.3028709Z .................................................................................................... 2600/9558
---
2020-01-29T14:11:55.3966188Z .................................................................................................... 4800/9558
2020-01-29T14:12:00.8962249Z ...........................................................i...............i........................ 4900/9558
2020-01-29T14:12:09.2857359Z .................................................................................................... 5000/9558
2020-01-29T14:12:18.6134150Z .................................................................................................... 5100/9558
2020-01-29T14:12:23.8721409Z ..i................................................................................................. 5200/9558
2020-01-29T14:12:36.0950502Z ...........................................................................ii.ii........i...i....... 5300/9558
2020-01-29T14:12:45.4782574Z .............i...................................................................................... 5500/9558
2020-01-29T14:12:56.1688428Z .................................................................................................... 5600/9558
2020-01-29T14:13:03.0275792Z ..............................................................i..................................... 5700/9558
2020-01-29T14:13:10.6677885Z .................................................................................................... 5800/9558
2020-01-29T14:13:10.6677885Z .................................................................................................... 5800/9558
2020-01-29T14:13:19.0602584Z .................................................................................................... 5900/9558
2020-01-29T14:13:28.3505641Z .....................................................ii...i..ii...........i......................... 6000/9558
2020-01-29T14:13:51.8334988Z .................................................................................................... 6200/9558
2020-01-29T14:13:59.7549501Z .................................................................................................... 6300/9558
2020-01-29T14:13:59.7549501Z .................................................................................................... 6300/9558
2020-01-29T14:14:08.7748112Z .................................................................................i..ii.............. 6400/9558
2020-01-29T14:14:40.3410171Z .................................................................................................... 6600/9558
2020-01-29T14:14:46.5068568Z .........................................................i.......................................... 6700/9558
2020-01-29T14:14:48.9404343Z .................................................................................................... 6800/9558
2020-01-29T14:14:51.4156379Z ........................................................i........................................... 6900/9558
---
2020-01-29T14:16:44.6891572Z .................................................................................................... 7600/9558
2020-01-29T14:16:50.5278018Z .................................................................................................... 7700/9558
2020-01-29T14:16:58.0494967Z .................................................................................................... 7800/9558
2020-01-29T14:17:09.7752253Z .................................................................................................... 7900/9558
2020-01-29T14:17:16.6228263Z ............iiiiiii.i............................................................................... 8000/9558
2020-01-29T14:17:32.4944001Z .................................................................................................... 8200/9558
2020-01-29T14:17:44.2298563Z .................................................................................................... 8300/9558
2020-01-29T14:17:58.8635766Z .................................................................................................... 8400/9558
2020-01-29T14:18:06.3132817Z .................................................................................................... 8500/9558
---
2020-01-29T14:20:43.2673360Z  finished in 8.024
2020-01-29T14:20:43.2902041Z Check compiletest suite=codegen mode=codegen (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-01-29T14:20:43.4643943Z 
2020-01-29T14:20:43.4645074Z running 169 tests
2020-01-29T14:20:46.7714311Z iiii......i........ii..iiii...i....i...........i............i..i..................i....i............ 100/169
2020-01-29T14:20:50.1687689Z i.i.i...iii..iiiiiiiiii.......................iii............ii......
2020-01-29T14:20:50.1778047Z 
2020-01-29T14:20:50.1778129Z  finished in 5.969
2020-01-29T14:20:50.1778514Z Check compiletest suite=codegen-units mode=codegen-units (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-01-29T14:20:50.1778556Z 
---
2020-01-29T14:20:51.6427898Z  finished in 2.362
2020-01-29T14:20:51.6636895Z Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-01-29T14:20:51.8337986Z 
2020-01-29T14:20:51.8339694Z running 9 tests
2020-01-29T14:20:51.8340811Z iiiiiiiii
2020-01-29T14:20:51.8341584Z 
2020-01-29T14:20:51.8345575Z  finished in 0.171
2020-01-29T14:20:51.8563765Z Check compiletest suite=incremental mode=incremental (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-01-29T14:20:52.0335566Z 
2020-01-29T14:20:52.0335566Z 
2020-01-29T14:20:52.0336037Z running 115 tests
2020-01-29T14:21:11.7301115Z ................................F................................................................... 100/115
2020-01-29T14:21:14.5858905Z ...............
2020-01-29T14:21:14.5859051Z failures:
2020-01-29T14:21:14.5859086Z 
2020-01-29T14:21:14.5859408Z ---- [incremental] incremental/hashes/enum_constructors.rs stdout ----
2020-01-29T14:21:14.5859446Z 
2020-01-29T14:21:14.5859721Z error in revision `cfail2`: test compilation failed although it shouldn't!
2020-01-29T14:21:14.5859773Z status: exit code: 1
2020-01-29T14:21:14.5860874Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/incremental/hashes/enum_constructors.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--cfg" "cfail2" "-C" "incremental=/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/enum_constructors/enum_constructors.inc" "-Z" "incremental-verify-ich" "-Z" "incremental-queries" "--error-format" "json" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/enum_constructors" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "query-dep-graph" "-Zincremental-ignore-spans" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/enum_constructors/auxiliary"
2020-01-29T14:21:14.5861479Z ------------------------------------------
2020-01-29T14:21:14.5861515Z 
2020-01-29T14:21:14.5861776Z ------------------------------------------
2020-01-29T14:21:14.5861823Z stderr:
2020-01-29T14:21:14.5861823Z stderr:
2020-01-29T14:21:14.5862062Z ------------------------------------------
2020-01-29T14:21:14.5862134Z error: `optimized_mir(change_constructor_path_struct_like)` should be clean but is not
2020-01-29T14:21:14.5862582Z    |
2020-01-29T14:21:14.5862582Z    |
2020-01-29T14:21:14.5862658Z LL | / pub fn change_constructor_path_struct_like() {
2020-01-29T14:21:14.5862706Z LL | |     let _ = Enum2::Struct {
2020-01-29T14:21:14.5862759Z LL | |         x: 0,
2020-01-29T14:21:14.5862819Z LL | |         y: 1,
2020-01-29T14:21:14.5862863Z LL | |         z: 2,
2020-01-29T14:21:14.5862947Z LL | | }
2020-01-29T14:21:14.5863004Z    | |_^
2020-01-29T14:21:14.5863032Z 
2020-01-29T14:21:14.5863032Z 
2020-01-29T14:21:14.5863084Z error: `optimized_mir(change_constructor_variant_struct_like)` should be clean but is not
2020-01-29T14:21:14.5863459Z    |
2020-01-29T14:21:14.5863459Z    |
2020-01-29T14:21:14.5863505Z LL | / pub fn change_constructor_variant_struct_like() {
2020-01-29T14:21:14.5863554Z LL | |     let _ = Enum2::Struct2 {
2020-01-29T14:21:14.5863614Z LL | |         x: 0,
2020-01-29T14:21:14.5863657Z LL | |         y: 1,
2020-01-29T14:21:14.5863699Z LL | |         z: 2,
2020-01-29T14:21:14.5863806Z LL | | }
2020-01-29T14:21:14.5864750Z    | |_^
2020-01-29T14:21:14.5864894Z 
2020-01-29T14:21:14.5864894Z 
2020-01-29T14:21:14.5864996Z error: `optimized_mir(change_constructor_path_tuple_like)` should be clean but is not
2020-01-29T14:21:14.5865562Z    |
2020-01-29T14:21:14.5866036Z LL | / pub fn change_constructor_path_tuple_like() {
2020-01-29T14:21:14.5866036Z LL | / pub fn change_constructor_path_tuple_like() {
2020-01-29T14:21:14.5866097Z LL | |     let _ = Enum2::Tuple(0, 1, 2);
2020-01-29T14:21:14.5866198Z    | |_^
2020-01-29T14:21:14.5866227Z 
2020-01-29T14:21:14.5866227Z 
2020-01-29T14:21:14.5866278Z error: `optimized_mir(change_constructor_variant_tuple_like)` should be clean but is not
2020-01-29T14:21:14.5866708Z    |
2020-01-29T14:21:14.5866708Z    |
2020-01-29T14:21:14.5866755Z LL | / pub fn change_constructor_variant_tuple_like() {
2020-01-29T14:21:14.5866818Z LL | |     let _ = Enum2::Tuple2(0, 1, 2);
2020-01-29T14:21:14.5866919Z    | |_^
2020-01-29T14:21:14.5866955Z 
2020-01-29T14:21:14.5866955Z 
2020-01-29T14:21:14.5867005Z error: `optimized_mir(change_constructor_path_c_like)` should be clean but is not
2020-01-29T14:21:14.5867824Z    |
2020-01-29T14:21:14.5867824Z    |
2020-01-29T14:21:14.5867870Z LL | / pub fn change_constructor_path_c_like() {
2020-01-29T14:21:14.5867934Z LL | |     let _ = Clike2::B;
2020-01-29T14:21:14.5868019Z    | |_^
2020-01-29T14:21:14.5868046Z 
2020-01-29T14:21:14.5868046Z 
2020-01-29T14:21:14.5868704Z error: `optimized_mir(change_constructor_variant_c_like)` should be clean but is not
2020-01-29T14:21:14.5869142Z    |
2020-01-29T14:21:14.5869142Z    |
2020-01-29T14:21:14.5869204Z LL | / pub fn change_constructor_variant_c_like() {
2020-01-29T14:21:14.5869255Z LL | |     let _ = Clike::C;
2020-01-29T14:21:14.5869359Z    | |_^
2020-01-29T14:21:14.5869406Z 
2020-01-29T14:21:14.5869456Z error: aborting due to 6 previous errors
2020-01-29T14:21:14.5869664Z 
---
2020-01-29T14:21:14.5870596Z test result: FAILED. 114 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out
2020-01-29T14:21:14.5870635Z 
2020-01-29T14:21:14.5875661Z 
2020-01-29T14:21:14.5875733Z 
2020-01-29T14:21:14.5877952Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/incremental" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "incremental" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-7/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "7.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2020-01-29T14:21:14.5878270Z 
2020-01-29T14:21:14.5878302Z 
2020-01-29T14:21:14.5878351Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2020-01-29T14:21:14.5878418Z Build completed unsuccessfully in 1:10:04
2020-01-29T14:21:14.5878418Z Build completed unsuccessfully in 1:10:04
2020-01-29T14:21:14.5878809Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:348:22
2020-01-29T14:21:14.5878889Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2020-01-29T14:21:14.5930993Z == clock drift check ==
2020-01-29T14:21:14.5951894Z   local time: Wed Jan 29 14:21:14 UTC 2020
2020-01-29T14:21:14.8953518Z   network time: Wed, 29 Jan 2020 14:21:14 GMT
2020-01-29T14:21:14.8953636Z == end clock drift check ==
2020-01-29T14:21:16.2243141Z 
2020-01-29T14:21:16.2396100Z ##[error]Bash exited with code '1'.
2020-01-29T14:21:16.2410566Z ##[section]Finishing: Run build
2020-01-29T14:21:16.2460519Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/48300/merge to s
2020-01-29T14:21:16.2462793Z Task         : Get sources
2020-01-29T14:21:16.2462847Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-01-29T14:21:16.2462911Z Version      : 1.0.0
2020-01-29T14:21:16.2462959Z Author       : Microsoft
2020-01-29T14:21:16.2462959Z Author       : Microsoft
2020-01-29T14:21:16.2463010Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
2020-01-29T14:21:16.2463082Z ==============================================================================
2020-01-29T14:21:16.7342082Z Cleaning any cached credential from repository: rust-lang/rust (GitHub)
2020-01-29T14:21:16.7392671Z ##[section]Finishing: Checkout rust-lang/rust@refs/pull/48300/merge to s
2020-01-29T14:21:16.7550638Z Cleaning up task key
2020-01-29T14:21:16.7551465Z Start cleaning up orphan processes.
2020-01-29T14:21:16.7679503Z Terminate orphan process: pid (3403) (python)
2020-01-29T14:21:16.8012025Z ##[section]Finishing: Finalize Job

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors
Copy link
Contributor

bors commented Jan 29, 2020

☀️ Try build successful - checks-azure
Build commit: 4efa7e626ffbc8d7ea613736e9e03c3809e95c9b (4efa7e626ffbc8d7ea613736e9e03c3809e95c9b)

@rust-timer
Copy link
Collaborator

Queued 4efa7e626ffbc8d7ea613736e9e03c3809e95c9b with parent edb3684, future comparison URL.

// conservative, until such uses have been definitely deemed UB.
if context.is_borrow() {
self.locals[place.local].make_opaque();
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to self: this is not catching Rvalue::AddressOf (which has a different context).

This could've been caught by tests, if we had &raw versions of ui/mir/mir_raw_fat_ptr.rs and ui/raw-fat-ptr.rs (which are the tests that failed before I added this special-case).
I doubt I'll do that (adding tests) in this PR, but maybe other people are interested.
cc @matthewjasper @RalfJung

Also I'm worried we might have code elsewhere that isn't handling &raw correctly, if it relies on either Borrow contexts (by pattern-matching or is_borrow) or Rvalue::Ref.
cc @ecstatic-morse @oli-obk

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verified that Rvalue::AddressOf was handled correctly in const checking via explicit match in the PR that introduced &raw. It seems clear to me that AddressOf should not be its own PlaceContext, but should instead map to SharedBorrow and Borrow. I can't say whether other passes in the compiler are matching on AddressOf correctly.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't mind there being something like BorrowOrAddrOf and is_borrow_or_addr_of if we want to be super explicit, but otherwise I think I agree.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we had &raw versions of ui/mir/mir_raw_fat_ptr.rs and ui/raw-fat-ptr.rs

Those two tests look almost identical (the former just has some more stuff)... any idea why we have that duplication? One was clearly created by copy-pasting the other, but there are no comments, unfortunately.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it has mir in the name it used to have an opt-in to force mir codegen back when we had two modes (and -Zorbit). I agree we should dedup, cc @Centril

@rust-timer
Copy link
Collaborator

Finished benchmarking try commit 4efa7e626ffbc8d7ea613736e9e03c3809e95c9b, comparison URL.

@bjorn3
Copy link
Member

bjorn3 commented Jan 29, 2020

For syn-opt the base and patched incremental benchmarks spent less time in ThinLTO, while the inflate-opt patched incremental benchmark spent more time in ThinLTO

@eddyb
Copy link
Member Author

eddyb commented Jan 30, 2020

@bjorn3 So, it's just noise then? That run shouldn't change anything (compared to master) other than make (de)serialization slightly slower, I'm confused.

I am going to make a PR with just the first two commits, in case the order the LLVM DIVariables are created, or something else, is causing perf changes.

@bjorn3
Copy link
Member

bjorn3 commented Jan 30, 2020

So, it's just noise then?

I think so.

var_debug_info.source_info.span,
"FIXME: implement fragmentation for {:?}",
var_debug_info,
),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is possible to hit without MIR inlining (because only this pass can create VarDebugInfoContents::Composite, so it must've run in a callee that got inlined), but I need to implement it before landing anyway.
(gather_debug_info_fragments is already set up for this, I just forgot to do it)

Copy link
Member

@wesleywiser wesleywiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be good to have a couple of new mir-opt tests which capture the before & after output of this pass especially the handling of SetDiscriminant for enums.

src/librustc_mir/transform/fragment_locals.rs Show resolved Hide resolved
src/librustc/mir/mod.rs Show resolved Hide resolved
src/librustc_mir/util/pretty.rs Show resolved Hide resolved
bors added a commit that referenced this pull request Feb 3, 2020
add raw-addr-of variant to mir_raw_fat_ptr

As suggested at #48300 (comment)

r? @eddyb
bors added a commit that referenced this pull request Feb 3, 2020
codegen: create DIVariables ahead of using them with llvm.dbg.declare.

Instead of having `rustc_codegen_llvm` bundle creation of a `DIVariable` and `llvm.dbg.declare` into a single operation, they are now two separate methods, and the `DIVariable` is created earlier, specifically when `mir::VarDebugInfo`s are being partitioned into locals.

While this isn't currently needed, it's a prerequisite for #48300, which adds fragmented debuginfo, where one `mir::VarDebugInfo` has multiple parts of itself mapped to different `mir::Place`s.
For debuggers to see one composite variable instead of several ones with the same name, we need to create a single `DIVariable` and share it between multiple `llvm.dbg.declare` calls, which are likely pointing to different MIR locals.
That makes the `per_local_var_debug_info` partitioning a good spot to do this in, as we can create *exactly* one `DIVariable` per `mir::VarDebugInfo`, and refer to it as many things as needed.

I'm opening this PR separately because I want to test its perf impact in isolation (see #48300 (comment)).

r? @nagisa or @oli-obk cc @michaelwoerister @nikomatsakis
@bors
Copy link
Contributor

bors commented Feb 3, 2020

☔ The latest upstream changes (presumably #68665) made this pull request unmergeable. Please resolve the merge conflicts.

@joelpalmer
Copy link

Triaged

@joelpalmer joelpalmer added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Mar 23, 2020
@jonas-schievink jonas-schievink added the A-mir-opt Area: MIR optimizations label Mar 29, 2020
@joelpalmer joelpalmer added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Apr 9, 2020
@Dylan-DPC-zz
Copy link

Closing this after a discussion with the author.

@Dylan-DPC-zz Dylan-DPC-zz added S-inactive Status: Inactive and waiting on the author. This is often applied to closed PRs. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Apr 9, 2020
@eddyb eddyb marked this pull request as draft April 10, 2020 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-mir-opt Area: MIR optimizations S-inactive Status: Inactive and waiting on the author. This is often applied to closed PRs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.