Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

april 2018 trait resolution regression #60010

Closed
pnkfelix opened this issue Apr 16, 2019 · 9 comments
Closed

april 2018 trait resolution regression #60010

pnkfelix opened this issue Apr 16, 2019 · 9 comments
Labels
A-trait-system Area: Trait system C-bug Category: This is a bug. regression-from-stable-to-stable Performance or correctness regression from one stable version to another. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@pnkfelix
Copy link
Member

pnkfelix commented Apr 16, 2019

(spawned off of #58291)

Reduced example (play):

use std::panic::RefUnwindSafe;

trait Database { type Storage; }
trait HasQueryGroup { }
trait Query<DB> { type Data; }
trait SourceDatabase { fn parse(&self) { loop { } } }

struct ParseQuery;
struct RootDatabase { _runtime: Runtime<RootDatabase>, }
struct Runtime<DB: Database> { _storage: Box<DB::Storage> }
struct SalsaStorage { _parse: <ParseQuery as Query<RootDatabase>>::Data, }

impl Database for RootDatabase  { type Storage = SalsaStorage; }
impl HasQueryGroup for RootDatabase {}
impl<DB> Query<DB> for ParseQuery where DB: SourceDatabase, DB: Database { type Data = RootDatabase; }
impl<T> SourceDatabase for T where T: RefUnwindSafe, T: HasQueryGroup {}

pub(crate) fn goto_implementation(db: &RootDatabase) -> u32 { db.parse(); loop { } }

fn main() { }

original code:

https://gist.github.com/dc3f0f8568ca093d9750653578bb8026

% rustc +nightly-2018-04-27 --allow dead_code  src/main.rs
% rustc +nightly-2018-04-28 --allow dead_code  src/main.rs
error[E0599]: no method named `parse` found for type `&RootDatabase` in the current scope
   --> src/main.rs:231:6
    |
231 | { db.parse(); loop { } }
    |      ^^^^^
    |
    = note: the method `parse` exists but the following trait bounds were not satisfied:
            `RootDatabase : SourceDatabase`
            `&RootDatabase : SourceDatabase`
            `RootDatabase : SourceDatabase`
    = help: items from traits can only be used if the trait is implemented and in scope
    = note: the following trait defines an item `parse`, perhaps you need to implement it:
            candidate #1: `SourceDatabase`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0599`.
%
@pnkfelix pnkfelix changed the title trait resolution regression april 2018 trait resolution regression Apr 16, 2019
@jonas-schievink jonas-schievink added A-trait-system Area: Trait system C-bug Category: This is a bug. regression-from-stable-to-stable Performance or correctness regression from one stable version to another. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Apr 16, 2019
@pnkfelix
Copy link
Member Author

This change in behavior might be a bugfix. I'm still trying to understand the further narrowed-down test case, where it seems like one of the crucial details is the handling of std::panic::RefUnwindSafe:

use std::panic::RefUnwindSafe;

trait Database { type Storage; }
trait HasQueryGroup { }
trait Query<DB> { type Data; }
trait SourceDatabase { fn parse(&self) { loop { } } }

struct ParseQuery;
struct RootDatabase { _runtime: Runtime<RootDatabase>, }
struct Runtime<DB: Database> { _storage: Box<DB::Storage> }
struct SalsaStorage { _parse: <ParseQuery as Query<RootDatabase>>::Data, }

impl Database for RootDatabase  { type Storage = SalsaStorage; }
impl HasQueryGroup for RootDatabase {}
impl<DB> Query<DB> for ParseQuery where DB: SourceDatabase, DB: Database { type Data = RootDatabase; }
impl<T> SourceDatabase for T where T: RefUnwindSafe, T: HasQueryGroup {}

pub(crate) fn goto_implementation(db: &RootDatabase) -> u32 { db.parse(); loop { } }

fn main() { }

@pnkfelix
Copy link
Member Author

pnkfelix commented Apr 16, 2019

(note in particular that in this example, we have a blanket implementation of SourceDatabase for any T that implements HasQueryGroup and RefUnwindSafe. There's an implementation of the former for RootDatabase, but not for the latter.

(But I'm not yet sure whether RefUnwindSafe is an OIBIT. Looking now.)

Update: okay, pub auto trait RefUnwindSafe, so I think RootDatabase implements it by default; I don't see any negative impls for Box. But I also am not 100% sure how that auto traits interact with fields defined via associated types of generic type parameters...

@pnkfelix
Copy link
Member Author

pnkfelix commented Apr 16, 2019

For reference, here is the log of changes between the two relevant nightlies. (I just transcribed this directly from #58291 (comment) )

% git log 7f3444e1b..686d0ae13 --author=bors --format=oneline

@pnkfelix
Copy link
Member Author

The two PR's that I think seem most likely to be at fault here are either #48995 or #50102

@pnkfelix
Copy link
Member Author

Okay I have now confirmed that this was injected by #48995

@pnkfelix
Copy link
Member Author

pnkfelix commented Apr 17, 2019

my current theory is that this is arising due to some interaction between inductive and co-inductive reasoning.

In particular, if you focus in on the impl<T> SourceDatabase for T where T: RefUnwindSafe, T: HasQueryGroup {}:

impl<T> SourceDatabase for T 
    where
        T: RefUnwindSafe, // [1]
        T: HasQueryGroup, // [2]
{}

commenting out either line [1] or line [2] above from the original code will cause the whole source input to be acccepted.

It is only when both are presented as preconditions on the blanket impl of SourceDatabase that we see a failure.


I extended the debug! output a bit to print out the whole trait obligation stack (with syntax elem1 :: elem2 :: elem3 :: []) and am seeing this in one of the compiler's attempts to prove that RootDatabase may implement SourceDatabase when its trying to resolve that parse method invocation:

DEBUG 2019-04-17T09:14:41Z: rustc::traits::select: evaluate_stack entry
stack: TraitObligationStack { obligation: Obligation(predicate=Binder(TraitPredicate(<RootDatabase as SourceDatabase>)),depth=9), fresh_trait_ref: Binder(<RootDatabase as SourceDatabase>) }
    :: TraitObligationStack { obligation: Obligation(predicate=Binder(TraitPredicate(<SalsaStorage as std::panic::RefUnwindSafe>)),depth=7), fresh_trait_ref: Binder(<SalsaStorage as std::panic::RefUnwindSafe>) }
    :: TraitObligationStack { obligation: Obligation(predicate=Binder(TraitPredicate(<*const SalsaStorage as std::panic::RefUnwindSafe>)),depth=6), fresh_trait_ref: Binder(<*const SalsaStorage as std::panic::RefUnwindSafe>) }
    :: TraitObligationStack { obligation: Obligation(predicate=Binder(TraitPredicate(<core::nonzero::NonZero<*const SalsaStorage> as std::panic::RefUnwindSafe>)),depth=5), fresh_trait_ref: Binder(<core::nonzero::NonZero<*const SalsaStorage> as std::panic::RefUnwindSafe>) }
    :: TraitObligationStack { obligation: Obligation(predicate=Binder(TraitPredicate(<std::ptr::Unique<SalsaStorage> as std::panic::RefUnwindSafe>)),depth=4), fresh_trait_ref: Binder(<std::ptr::Unique<SalsaStorage> as std::panic::RefUnwindSafe>) }
    :: TraitObligationStack { obligation: Obligation(predicate=Binder(TraitPredicate(<std::boxed::Box<SalsaStorage> as std::panic::RefUnwindSafe>)),depth=3), fresh_trait_ref: Binder(<std::boxed::Box<SalsaStorage> as std::panic::RefUnwindSafe>) }
    :: TraitObligationStack { obligation: Obligation(predicate=Binder(TraitPredicate(<Runtime<RootDatabase> as std::panic::RefUnwindSafe>)),depth=2), fresh_trait_ref: Binder(<Runtime<RootDatabase> as std::panic::RefUnwindSafe>) }
    :: TraitObligationStack { obligation: Obligation(predicate=Binder(TraitPredicate(<RootDatabase as std::panic::RefUnwindSafe>)),depth=1), fresh_trait_ref: Binder(<RootDatabase as std::panic::RefUnwindSafe>) }
    :: TraitObligationStack { obligation: Obligation(predicate=Binder(TraitPredicate(<RootDatabase as SourceDatabase>)),depth=0), fresh_trait_ref: Binder(<RootDatabase as SourceDatabase>) }
    :: []
DEBUG 2019-04-17T09:14:41Z: rustc::traits::select: evaluate_stack(Binder(<RootDatabase as SourceDatabase>)) --> recursive

that is, it thinks the attempt to prove RootDatabase implements SourceDatabase requires recursive reasoning: because proving that requires proving that RootDatabase: RefUnwindSafe, which bubbles down to Runtime<RootDatabase>: RefUnwindSafe and from there to SalsaStorage: RefUnwindSafe.

And that last bit, SalsaStorage: RefUnwindSafe, might be where we are hitting a problem, due to these definitions:

struct SalsaStorage { _parse: <ParseQuery as Query<RootDatabase>>::Data, }
// ...
impl<DB> Query<DB> for ParseQuery where DB: SourceDatabase { type Data = RootDatabase; }

so we now see why the compiler might reason that it has to prove RootDatabase: SourceDatabase when analyzing the structure of SalsaStorage: it needs to find the implementation of that trait so that it can find its associated ::Data type.


But for some reason we don't hit the above problem if we remove the T: HasQueryGroup where clause on the blanket impl of SourceDatabase.

@nikomatsakis
Copy link
Contributor

@pnkfelix I haven't had time to deeply look yet, but a few notes about induction/co-induction in general:

In short, a cycle in trait resolution is only considered "true" if all the traits involved are co-inductive (i.e., auto-traits). So if you have Foo: Send requires (indirectly, say) Foo: Send, that's ok.

But if you have Foo: Send requires Foo: Debug which requires Foo: Send, that's not ok.

In this case, then, I expect that cycle to be "non-true", because SourceDatabase is an inductive trait and hence proving that RootDatabase: SourceDatabase cannot require RootDatabase: SourceDatabase.

What I'm not sure about 100% is why the behavior changed here and whether the cycle ought to arise.

@nikomatsakis
Copy link
Contributor

Also curious:

If you do SourceDatabase::parse(db) instead of db.parse(), it compiles.

There are two distinct systems for trait evaluation (the "evaluate" code and the "confirm" code) which, I suppose, are disagreeing here. Digging a bit more.

bors added a commit that referenced this issue May 2, 2019
…tion, r=<try>

forego caching for all participants in cycles, apart from root node

This is a targeted fix for #60010, which uncovered a pretty bad failure of our caching strategy in the face of coinductive cycles. The problem is explained in the comment in the PR on the new field, `in_cycle`, but I'll reproduce it here:

> Starts out as false -- if, during evaluation, we encounter a
> cycle, then we will set this flag to true for all participants
> in the cycle (apart from the "head" node). These participants
> will then forego caching their results. This is not the most
> efficient solution, but it addresses #60010. The problem we
> are trying to prevent:
>
> - If you have `A: AutoTrait` requires `B: AutoTrait` and `C: NonAutoTrait`
> - `B: AutoTrait` requires `A: AutoTrait` (coinductive cycle, ok)
> - `C: NonAutoTrait` requires `A: AutoTrait` (non-coinductive cycle, not ok)
>
> you don't want to cache that `B: AutoTrait` or `A: AutoTrait`
> is `EvaluatedToOk`; this is because they were only considered
> ok on the premise that if `A: AutoTrait` held, but we indeed
> encountered a problem (later on) with `A: AutoTrait. So we
> currently set a flag on the stack node for `B: AutoTrait` (as
> well as the second instance of `A: AutoTrait`) to supress
> caching.
>
> This is a simple, targeted fix. The correct fix requires
> deeper changes, but would permit more caching: we could
> basically defer caching until we have fully evaluated the
> tree, and then cache the entire tree at once.

I'm not sure what the impact of this fix will be in terms of existing crates or performance: we were accepting incorrect code before, so there will perhaps be some regressions, and we are now caching less.

As the comment above notes, we could do a lot better than this fix, but that would involve more invasive rewrites. I thought it best to start with something simple.

r? @pnkfelix -- but let's do crater/perf run
cc @arielb1
Centril added a commit to Centril/rust that referenced this issue May 14, 2019
…r-investigation, r=pnkfelix

forego caching for all participants in cycles, apart from root node

This is a targeted fix for rust-lang#60010, which uncovered a pretty bad failure of our caching strategy in the face of coinductive cycles. The problem is explained in the comment in the PR on the new field, `in_cycle`, but I'll reproduce it here:

> Starts out as false -- if, during evaluation, we encounter a
> cycle, then we will set this flag to true for all participants
> in the cycle (apart from the "head" node). These participants
> will then forego caching their results. This is not the most
> efficient solution, but it addresses rust-lang#60010. The problem we
> are trying to prevent:
>
> - If you have `A: AutoTrait` requires `B: AutoTrait` and `C: NonAutoTrait`
> - `B: AutoTrait` requires `A: AutoTrait` (coinductive cycle, ok)
> - `C: NonAutoTrait` requires `A: AutoTrait` (non-coinductive cycle, not ok)
>
> you don't want to cache that `B: AutoTrait` or `A: AutoTrait`
> is `EvaluatedToOk`; this is because they were only considered
> ok on the premise that if `A: AutoTrait` held, but we indeed
> encountered a problem (later on) with `A: AutoTrait. So we
> currently set a flag on the stack node for `B: AutoTrait` (as
> well as the second instance of `A: AutoTrait`) to supress
> caching.
>
> This is a simple, targeted fix. The correct fix requires
> deeper changes, but would permit more caching: we could
> basically defer caching until we have fully evaluated the
> tree, and then cache the entire tree at once.

I'm not sure what the impact of this fix will be in terms of existing crates or performance: we were accepting incorrect code before, so there will perhaps be some regressions, and we are now caching less.

As the comment above notes, we could do a lot better than this fix, but that would involve more invasive rewrites. I thought it best to start with something simple.

r? @pnkfelix -- but let's do crater/perf run
cc @arielb1
@pnkfelix
Copy link
Member Author

I believe this is resolved by PR #60444, in the sense that we now issue:

error[E0275]: overflow evaluating the requirement `RootDatabase: SourceDatabase`
  --> src/main.rs:11:23
   |
11 | struct SalsaStorage { _parse: <ParseQuery as Query<RootDatabase>>::Data, }
   |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: required because of the requirements on the impl of `Query<RootDatabase>` for `ParseQuery`

error[E0275]: overflow evaluating the requirement `RootDatabase: SourceDatabase`
  --> src/main.rs:13:6
   |
13 | impl Database for RootDatabase  { type Storage = SalsaStorage; }
   |      ^^^^^^^^
   |
   = note: required because of the requirements on the impl of `Query<RootDatabase>` for `ParseQuery`
   = note: required because it appears within the type `SalsaStorage`

and furthermore, we do so independently of whatever the function body of fn goto_implementation contains (thus eliminating the oddity that niko noted in their previous comment).

Closing as fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-trait-system Area: Trait system C-bug Category: This is a bug. regression-from-stable-to-stable Performance or correctness regression from one stable version to another. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

3 participants