Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Be more permissive with required bounds on existential types #57896

Merged
merged 10 commits into from
Feb 19, 2019

Conversation

oli-obk
Copy link
Contributor

@oli-obk oli-obk commented Jan 25, 2019

fixes #54184

r? @pnkfelix

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jan 25, 2019
@rust-highfive

This comment has been minimized.

@oli-obk
Copy link
Contributor Author

oli-obk commented Jan 28, 2019

So... I was looking at the various examples and found one reason not to just outright treat any generic parameter equal to any other generic parameter, as long as all the other types match up. My considerations basically boil down to: if

fn foo<T>() -> Foo<T, T> { ... }
fn bar<T, U>() -> Foo<T, U> { ... }
fn mep<U, T>() -> Foo<T, U> { ... }

all are defining uses as long as the concrete type they return has the same structure. Then the following concrete return types are ok:

foo bar meh
Vec Vec<T> Vec<T>, Vec<U> Vec<T>, Vec<U>
T T T, U T, U
tuple (T, T) (T, U), (U, T), (T, T), (U, U) (T, U), (U, T), (T, T), (U, U)
tuple2 (T, u32, T) (T, u32, U), (U, u32, T), (T, u32, T), (U, u32, U) (T, u32, U), (U, u32, T), (T, u32, T), (U, u32, U)

So if we chose the tuple variant:

fn foo<T: Copy>(t: T) -> Foo<T, T> { (t, t) }
fn bar<T, U>(t: T, u: U) -> Foo<T, U> { (t, u) }
fn mep<U, T>(u: U, t: T) -> Foo<T, U> { (t, u) }

Means that, assuming we could reveal existential types, the following would compile:

let _: (&str, &str) = foo("foo"); // Foo<&str, &str>
let _: (&str, i32) = bar("", 42); // Foo<&str, i32>
let _: (&str, i32) = mep(42, ""); // Foo<&str, i32>
let _: (i32, &str) = bar(42, ""); // Foo<i32, &str>
let _: (i32, &str) = mep("", 42); // Foo<i32, &str>

Now if we swapped the (t, u) in mep:

fn foo<T: Copy>(t: T) -> Foo<T, T> { (t, t) }
fn bar<T, U>(t: T, u: U) -> Foo<T, U> { (t, u) }
fn mep<U, T>(u: U, t: T) -> Foo<T, U> { (u, t) }

We'd get

let _: (&str, &str) = foo("foo"); // Foo<&str, &str>
let _: (&str, i32) = bar("", 42); // Foo<&str, i32>
let _: (i32, &str) = mep(42, ""); // Foo<&str, i32>
let _: (i32, &str) = bar(42, ""); // Foo<i32, &str>
let _: (&str, i32) = mep("", 42); // Foo<i32, &str>

Which would mean that Foo<i32, &str> could resolve both to (i32, &str) and (&str, i32), which is clearly wrong.

Which means we need to take the generic parameters from the existential type and not the function parameters, because the latter are irrelevant.

For simplicity (and forward compat to a more lenient approach), I will make it so that

fn foo<T>() -> Foo<T, T> { ... }
fn bar<T, U>() -> Foo<T, U> { ... }

and

fn foo<T>() -> Foo<T, T> { ... }
fn mep<U, T>() -> Foo<T, U> { ... }

are forbidden, but

fn bar<T, U>() -> Foo<T, U> { ... }
fn mep<U, T>() -> Foo<T, U> { ... }

and

fn bar<T, U>() -> Foo<T, U> { ... }
fn mep<T, U>() -> Foo<U, T> { ... }

are allowed as long as there is an injective mapping from Foo<X, Y> to the concrete type

@oli-obk
Copy link
Contributor Author

oli-obk commented Jan 30, 2019

r? @cramertj This PR is ready now

@rust-highfive rust-highfive assigned cramertj and unassigned pnkfelix Jan 30, 2019
@bors

This comment has been minimized.

@cramertj
Copy link
Member

cramertj commented Feb 5, 2019

Talked at the all hands-- I'm still of the opinion that we can't allow defining uses which don't keep both type parameters as fully universal:

existential type X<A, B>;

fn foo<A>(a: A) -> X<A, u32> {
    (a, 5u32)
}

This could force the concrete type to be X<A,B> = (A, B), or it could be X<A, B> = (A, u32), we don't know which one it is (it could also be something like X<A, B> = (A, <A as SomeTrait>::Output))

@oli-obk
Copy link
Contributor Author

oli-obk commented Feb 13, 2019

So... we've come full circle. We are now again requiring defining uses to use all the generic parameters of the existential type in the concrete type.

@cramertj
Copy link
Member

I'm a bit confused by the latest error. It's fine to not use the generic parameter in the concrete type, but each generic parameter has to be left fully variant in the type signature of the function that includes the defining use. The example you added unambiguously (afaict) does not use type U, which is fine. The thing that wouldn't be fine would be if it is unclear whether or not it uses U.

@oli-obk
Copy link
Contributor Author

oli-obk commented Feb 14, 2019

Yea that was a brain-afk moment for me. I cleaned it up and it makes much more sense now (and still fixes the original issue that caused this PR)

Right now we are erroring on all potential defining uses that end up not being a defining use. As a next step (separate from this PR), we can instead collect all the potential sites and add them as notes to the "no defining use" error. If a single defining use site exists, we don't need to emit an error for the potential use sites (except if their concrete types conflict with the defining use sites)

@cramertj
Copy link
Member

Nice, this looks good to me! Thanks for the fixes :)

@bors r+

@bors
Copy link
Contributor

bors commented Feb 14, 2019

📌 Commit eb98d31 has been approved by cramertj

@bors
Copy link
Contributor

bors commented Feb 16, 2019

💔 Test failed - checks-travis

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 16, 2019
@rust-highfive
Copy link
Collaborator

The job dist-i686-linux of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
 95 82.1M   95 78.7M    0     0  2197k      0  0:00:38  0:00:36  0:00:02 2196k
 98 82.1M   98 81.1M    0     0  2205k      0  0:00:38  0:00:37  0:00:01 2391k
100 82.1M  100 82.1M    0     0  2209k      0  0:00:38  0:00:38 --:--:-- 2382k
[00:05:17] + cd gcc-4.8.5
[00:05:17] + sed -i 's|ftp://gcc\.gnu\.org/|http://gcc.gnu.org/|g' ./contrib/download_prerequisites
[00:05:17] --2019-02-16 06:54:49--  http://gcc.gnu.org/pub/gcc/infrastructure/mpfr-2.4.2.tar.bz2
[00:05:17] Resolving gcc.gnu.org... 209.132.180.131
[00:05:17] Connecting to gcc.gnu.org|209.132.180.131|:80... connected.
[00:05:17] HTTP request sent, awaiting response... 200 OK
---
[00:44:03]  ---> 53189f0c88f2
[00:44:03] Step 25/41 : RUN ./build-clang.sh
[00:44:03]  ---> Running in 3151550acbdb
[00:44:04] + source shared.sh
[00:44:04] + LLVM=032b00a5404865765cda7db3039f39d54964d8b0
[00:44:04] + LLD=3e4aa4e8671523321af51449e0569f455ef3ad43
[00:44:04] + CLANG=a6b9739069763243020f4ea6fe586bc135fde1f9
[00:44:04] + mkdir clang
[00:44:04] + cd clang
[00:44:04] + curl -L https://github.com/llvm-mirror/llvm/archive/032b00a5404865765cda7db3039f39d54964d8b0.tar.gz
[00:44:04] + tar xzf - --strip-components=1
[00:44:04]                                  Dload  Upload   Total   Spent    Left  Speed
[00:44:04] 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   158    0   158    0     0    762      0 --:--:-- --:--:-- --:--:--   806
---
100 9540k    0 9540k    0     0  4283k      0 --:--:--  0:00:02 --:--:-- 5600k
100 14.6M    0 14.6M    0     0  4644k      0 --:--:--  0:00:03 --:--:-- 5521k
100 17.1M    0 17.1M    0     0  4945k      0 --:--:--  0:00:03 --:--:-- 5867k
[00:44:16] + mkdir -p tools/lld
[00:44:16] + tar zxf - --strip-components=1 -C tools/lld
[00:44:16] + curl -L https://github.com/llvm-mirror/lld/archive/3e4aa4e8671523321af51449e0569f455ef3ad43.tar.gz
[00:44:16]                                  Dload  Upload   Total   Spent    Left  Speed
[00:44:16] 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   157    0   157    0     0    814      0 --:--:-- --:--:-- --:--:--   830
100   157    0   157    0     0    814      0 --:--:-- --:--:-- --:--:--   830
[00:44:16] 
100 1346k    0 1346k    0     0  2146k      0 --:--:-- --:--:-- --:--:-- 2146k
[00:44:16] + mkdir clang-build
[00:44:16] + cd clang-build
[00:44:16] + INC=/rustroot/include
[00:44:16] + INC=/rustroot/include:/rustroot/lib/gcc/x86_64-unknown-linux-gnu/4.8.5/include-fixed
[00:44:16] + INC=/rustroot/include:/rustroot/lib/gcc/x86_64-unknown-linux-gnu/4.8.5/include-fixed:/usr/include
[00:44:16] + hide_output cmake .. -DCMAKE_C_COMPILER=/rustroot/bin/gcc -DCMAKE_CXX_COMPILER=/rustroot/bin/g++ -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/rustroot -DLLVM_TARGETS_TO_BUILD=X86 -DC_INCLUDE_DIRS=/rustroot/include:/rustroot/lib/gcc/x86_64-unknown-linux-gnu/4.8.5/include-fixed:/usr/include
[00:44:46] Sat Feb 16 07:34:19 UTC 2019 - building ...
[00:44:53] + hide_output make -j10
[00:44:53] + set +x
[00:45:23] Sat Feb 16 07:34:55 UTC 2019 - building ...
---
[01:54:03]  ---> fed169bab981
[01:54:03] Step 32/41 : RUN ./build-perl.sh
[01:54:03]  ---> Running in 6550ce607346
[01:54:04] + source shared.sh
[01:54:04] + curl https://www.cpan.org/src/5.0/perl-5.28.0.tar.gz
[01:54:04]   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
[01:54:04]                                  Dload  Upload   Total   Spent    Left  Speed
[01:54:05] 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
---
[02:59:14] [RUSTC-TIMING] tendril test:false 1.030
[02:59:14]    Compiling clap v2.32.0
[02:59:14]    Compiling phf_generator v0.7.22
[02:59:14]    Compiling url v1.7.2
The job exceeded the maximum time limit for jobs, and has been terminated.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@oli-obk
Copy link
Contributor Author

oli-obk commented Feb 16, 2019

@bors retry timout

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 16, 2019
@bors
Copy link
Contributor

bors commented Feb 16, 2019

⌛ Testing commit eb98d31 with merge 235a5f7...

bors added a commit that referenced this pull request Feb 16, 2019
 Be more permissive with required bounds on existential types

fixes  #54184

r? @pnkfelix
@bors
Copy link
Contributor

bors commented Feb 16, 2019

💔 Test failed - checks-travis

@rust-highfive
Copy link
Collaborator

The job dist-i686-linux of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
 93 82.1M   93 77.0M    0     0  1547k      0  0:00:54  0:00:51  0:00:03 2758k
 97 82.1M   97 79.8M    0     0  1570k      0  0:00:53  0:00:52  0:00:01 2819k
100 82.1M  100 82.1M    0     0  1592k      0  0:00:52  0:00:52 --:--:-- 2861k
[00:05:28] + cd gcc-4.8.5
[00:05:28] + sed -i 's|ftp://gcc\.gnu\.org/|http://gcc.gnu.org/|g' ./contrib/download_prerequisites
[00:05:28] --2019-02-16 12:57:23--  http://gcc.gnu.org/pub/gcc/infrastructure/mpfr-2.4.2.tar.bz2
[00:05:28] Resolving gcc.gnu.org... 209.132.180.131
[00:05:28] Connecting to gcc.gnu.org|209.132.180.131|:80... connected.
[00:05:29] HTTP request sent, awaiting response... 200 OK
---
[00:42:26]  ---> a4e8d576cc17
[00:42:26] Step 25/41 : RUN ./build-clang.sh
[00:42:26]  ---> Running in 79a2496171af
[00:42:26] + source shared.sh
[00:42:26] + LLVM=032b00a5404865765cda7db3039f39d54964d8b0
[00:42:26] + LLD=3e4aa4e8671523321af51449e0569f455ef3ad43
[00:42:26] + CLANG=a6b9739069763243020f4ea6fe586bc135fde1f9
[00:42:26] + mkdir clang
[00:42:26] + cd clang
[00:42:26] + curl -L https://github.com/llvm-mirror/llvm/archive/032b00a5404865765cda7db3039f39d54964d8b0.tar.gz
[00:42:26] + tar xzf - --strip-components=1
[00:42:26]                                  Dload  Upload   Total   Spent    Left  Speed
[00:42:27] 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
---
100 8816k    0 8816k    0     0  4461k      0 --:--:--  0:00:01 --:--:-- 5694k
100 13.7M    0 13.7M    0     0  4748k      0 --:--:--  0:00:02 --:--:-- 5503k
100 17.1M    0 17.1M    0     0  5013k      0 --:--:--  0:00:03 --:--:-- 5716k
[00:42:38] + mkdir -p tools/lld
[00:42:38] + curl -L https://github.com/llvm-mirror/lld/archive/3e4aa4e8671523321af51449e0569f455ef3ad43.tar.gz
[00:42:38] + tar zxf - --strip-components=1 -C tools/lld
[00:42:38]                                  Dload  Upload   Total   Spent    Left  Speed
[00:42:38] 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   157    0   157    0     0    727      0 --:--:-- --:--:-- --:--:--   740
---
[00:42:39] + cd clang-build
[00:42:39] + INC=/rustroot/include
[00:42:39] + INC=/rustroot/include:/rustroot/lib/gcc/x86_64-unknown-linux-gnu/4.8.5/include-fixed
[00:42:39] + INC=/rustroot/include:/rustroot/lib/gcc/x86_64-unknown-linux-gnu/4.8.5/include-fixed:/usr/include
[00:42:39] + hide_output cmake .. -DCMAKE_C_COMPILER=/rustroot/bin/gcc -DCMAKE_CXX_COMPILER=/rustroot/bin/g++ -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/rustroot -DLLVM_TARGETS_TO_BUILD=X86 -DC_INCLUDE_DIRS=/rustroot/include:/rustroot/lib/gcc/x86_64-unknown-linux-gnu/4.8.5/include-fixed:/usr/include
[00:43:09] Sat Feb 16 13:35:04 UTC 2019 - building ...
[00:43:14] + hide_output make -j10
[00:43:14] + set +x
[00:43:44] Sat Feb 16 13:35:39 UTC 2019 - building ...
---
[01:50:32]  ---> 4a096348fc26
[01:50:32] Step 32/41 : RUN ./build-perl.sh
[01:50:32]  ---> Running in f043392dc0e4
[01:50:33] + source shared.sh
[01:50:33] + curl https://www.cpan.org/src/5.0/perl-5.28.0.tar.gz
[01:50:33]   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
[01:50:33]                                  Dload  Upload   Total   Spent    Left  Speed
[01:50:34] 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 16, 2019
@kennytm
Copy link
Member

kennytm commented Feb 16, 2019

@bors retry

3 hour timeout 🤔 (caused by rebuilding docker images)

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 16, 2019
@bors
Copy link
Contributor

bors commented Feb 19, 2019

⌛ Testing commit eb98d31 with merge 74e35d2...

bors added a commit that referenced this pull request Feb 19, 2019
 Be more permissive with required bounds on existential types

fixes  #54184

r? @pnkfelix
@bors
Copy link
Contributor

bors commented Feb 19, 2019

☀️ Test successful - checks-travis, status-appveyor
Approved by: cramertj
Pushing 74e35d2 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Feb 19, 2019
@bors bors merged commit eb98d31 into rust-lang:master Feb 19, 2019
Comment on lines +1404 to +1408
// type parameters are equal to any other type parameter for the purpose of
// concrete type equality, as it is possible to obtain the same type just
// by passing matching parameters to a function.
(ty::Param(_), ty::Param(_)) => true,
_ => t == p,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm suspicious of this entire approach. Only the outermost type could be a differing ty::Params, anything containing ty::Param would automatically hit t == p being false.

There are two ways to compare types deeply, and they're:

  1. exact equality (which, with the right substitutions, can be reached for types coming from different contexts)
  2. unification (ty::relate or full-on inference, which uses the former) - this is how e.g. impl matching works in the current trait system

"Walking" a type is a not a good way to achieve almost anything, the main legitimate usecase I've found is gathering leaves of some kind (e.g. ty::Infer(_)) or searching for a specific type.

Comment on lines +1384 to +1390
let indices = concrete_type
.subst(self.tcx, substs)
.walk()
.filter_map(|t| match &t.sty {
ty::Param(p) => Some(*index_map.get(p).unwrap()),
_ => None,
}).collect();
Copy link
Member

@eddyb eddyb Mar 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying to understand what this does, we have these facts:

  1. concrete_type, in the "generics domain" of the opaque type
  2. concrete_type.subst(self.tcx, substs), in the "generics domain" of the definer
  3. substs are required to be a 1:1 mapping from the opaque type to the definer
    • this implies index_map is the exact inverse of substs

All that means indices could be computed so:

                let indices = concrete_type
                    .walk()
                    .filter_map(|t| match &t.sty {
                        ty::Param(p) => Some(p.index),
                        _ => None,
                    }).collect();

If indices have to agree between definers, that means concrete_types are fully equal.

Am I missing something? This entire PR, other than checking substs, looks like a noop.

I'll attempt a revert of the parts that look like a noop, and generalize to handle const generics (if this is even the right place to do so).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #70272.

Comment on lines +323 to +326
/// Generic parameters on the opaque type as passed by this function.
/// For `existential type Foo<A, B>; fn foo<T, U>() -> Foo<T, U> { .. }` this is `[T, U]`, not
/// `[A, B]`
pub substs: &'tcx Substs<'tcx>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hang on, isn't substs checking handled by wfcheck?

/// Checks "defining uses" of opaque `impl Trait` types to ensure that they meet the restrictions
/// laid for "higher-order pattern unification".
/// This ensures that inference is tractable.
/// In particular, definitions of opaque types can only use other generics as arguments,
/// and they cannot repeat an argument. Example:
///
/// ```rust
/// type Foo<A, B> = impl Bar<A, B>;
///
/// // Okay -- `Foo` is applied to two distinct, generic types.
/// fn a<T, U>() -> Foo<T, U> { .. }
///
/// // Not okay -- `Foo` is applied to `T` twice.
/// fn b<T>() -> Foo<T, T> { .. }
///
/// // Not okay -- `Foo` is applied to a non-generic type.
/// fn b<T>() -> Foo<T, u32> { .. }
/// ```
///
fn check_opaque_types<'fcx, 'tcx>(

Wouldn't that make ResolvedOpaqueTy and most of this PR entirely redundant, if params were checked there?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I can't remove ResolvedOpaqueTy as it seems MIR borrowck needs the Substs, and I'd rather not touch that code if I can help it.

bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 4, 2020
typeck/type_of: let wfcheck handle generics in opaque types' substs.

I was working on rust-lang#70164, and `type_of`'s handling of opaque types seemed to be, by far, the trickiest use of `Ty::walk`, but I believe it wasn't doing anything (see rust-lang#57896 (comment) - I suspect, based on glancing at the PR discussion, that an early attempt was kept in, despite becoming just an overcomplicated way to do exactly the same as the previous simple type equality check).

I would've loved to remove `ResolvedOpaqueTy` (keep the `Ty` and lose the `Substs`), but it looks like the MIR borrowck part of the process needs that now, so it would've been added anyway since rust-lang#57896, even if that PR hadn't happened.

<hr/>

In the process, I've moved the remaining substitution validation to `wfcheck`, which was already handling lifetimes, and kept only `delay_span_bug`s in `type_of`, as an insurance policy.

I've added tests for lifetime and const cases, they seem to be checked correctly now.
(and more uniform than they were in rust-lang#63063 (comment))

However, the quality of the errors is maybe a bit worse, and they don't trigger when there are other errors (not sure if this is due to compilation stop points or something more specific to one opaque type).

r? @nikomatsakis cc @matthewjasper @oli-obk @Aaron1011
@oli-obk oli-obk deleted the permissive_existence branch March 16, 2021 12:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

generic existential types should (at most) warn, not error, if a type parameter is unused
9 participants