Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lifetime Capture Rules 2024 #3498

Merged

Conversation

traviscross
Copy link
Contributor

@traviscross traviscross commented Sep 25, 2023

In Rust 2024 and later editions, return position impl Trait (RPIT) opaque types will automatically capture all in-scope type and lifetime parameters. In preparation for this, new RPIT-like impl Trait features introduced into the Rust 2021 edition will also automatically capture all in-scope type and lifetime parameters.

Rendered

The lifetime capture rules discussed in this RFC were the subject of a
T-lang design meeting on 2023-07-26.  The consensus of that meeting
was to put these rules into effect for the Rust 2024 edition and for
all RPIT-like features stabilized ahead of the Rust 2024 edition.

The purpose of this RFC is to memorialize the consensus of that
meeting.
@traviscross
Copy link
Contributor Author

@rustbot labels +T-lang

@rustbot rustbot added the T-lang Relevant to the language team, which will review and decide on the RFC. label Sep 25, 2023
@traviscross traviscross changed the title Add RFC for lifetime capture rules 2024 Lifetime Capture Rules 2024 Sep 25, 2023
@traviscross
Copy link
Contributor Author

@rustbot labels +I-lang-nominated

@rustbot rustbot added the I-lang-nominated Indicates that an issue has been nominated for prioritizing at the next lang team meeting. label Sep 25, 2023
In Appendix B, we have a matrix of capturing effects.  For TAIT, we
had correctly marked the capturing of lifetime parameters from the
outer impl as being not applicable in Rust 2021, but we had
incorrectly marked this as being "yes" in Rust 2024.  It's of course
not applicable in both editions.  Let's fix that.
@zirconium-n
Copy link

Non-blocking

The TAIT as the solution to overcapturing part looks a little bit weird. This implies following two code blocks are not always equivalent.

// block 1
type T = /* xxx */ ;
fn foo</*...*/>()  -> T { /*...*/ }

// block 2
fn foo</*...*/>()  -> /* xxx */ { /*...*/ }

This could be interpreted as an inconsistency. I think we need to guide users to realize the distinction in some way, or it will create much confusion. (I guess most common use cases would be on the happy path though.)

@nikomatsakis
Copy link
Contributor

@rfcbot fcp merge

There hasn't been a lot of discussion here, but this decision has been discussed quite a bit elsewhere, including two design meetings and numerous bits of Zulip discussion. It's also important for consistency with async fn and RPIT in traits, as discussed in rust-lang/rust#115822. Let's do it!

@rfcbot
Copy link
Collaborator

rfcbot commented Oct 3, 2023

Team member @nikomatsakis has proposed to merge this. The next step is review by the rest of the tagged team members:

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns.
See this document for info about what commands tagged team members can give me.

@rfcbot rfcbot added proposed-final-comment-period Currently awaiting signoff of all team members in order to enter the final comment period. disposition-merge This RFC is in PFCP or FCP with a disposition to merge it. labels Oct 3, 2023
text/3498-lifetime-capture-rules-2024.md Outdated Show resolved Hide resolved
text/3498-lifetime-capture-rules-2024.md Show resolved Hide resolved
text/3498-lifetime-capture-rules-2024.md Outdated Show resolved Hide resolved
text/3498-lifetime-capture-rules-2024.md Show resolved Hide resolved
text/3498-lifetime-capture-rules-2024.md Outdated Show resolved Hide resolved
text/3498-lifetime-capture-rules-2024.md Outdated Show resolved Hide resolved
text/3498-lifetime-capture-rules-2024.md Show resolved Hide resolved
text/3498-lifetime-capture-rules-2024.md Show resolved Hide resolved
@tmandry
Copy link
Member

tmandry commented Oct 4, 2023

Thank you for writing this RFC about such a detailed and nuanced topic, @traviscross, and for your earlier work driving the design conversations around it.

I have left some feedback in comments above, but none of it is about the normative changes in the RFC itself. I feel confident that we have the right set of rules here.

What's more, I'm really excited to tell users (especially async users) that we just made all those niggling lifetime bounds in their RPITs, and the ensuing confusion described in this RFC, disappear. I believe this is really a case where, syntactically speaking, less is more.

@rfcbot reviewed

There is some subtlety in describing the mechanics of the outlives
rules for `impl Trait` opaque types.  In our description of these
rules for opaque types that include a specified outlives bound, we had
some words that were less clear than we had intended.  Let's change
those words around to make the meaning more clear.

Specifically, rather saying that "it" outlives the specified lifetime,
what we mean exactly is that the lifetime that's substituted for the
specified lifetime parameter must outlive that other lifetime.

Note that none of this language is normative.  These rules have all
been specified in earlier RFCs.
The outlives rules related to RPIT-like `impl Trait` opaque types can
be a bit subtle.  Even though this is only non-normative background
information, let's add an appendix that explains these rules less
concisely than in the main body and provides examples.
Unfortunately, some Markdown tools care that the internal link
identifiers are the same case as the headings with which they are
associated, some tools care that the identifiers are always lower
case, and some tools use case insensitive matching for these
identifiers.

There is no way to satisfy all of these tools simultaneously.
However, the tools we care about at the moment either use case
insensitive matching or care that the identifiers are lower case, so
let's convert all of the identifiers to lower case.
In this RFC, we're specifying the capture rules for new RPIT-like
`impl Trait` features that may be introduced ahead of the Rust 2024
edition.  We had described those as features "introduced into the Rust
2021 edition", but actually, new features are introduced where
possible into all editions, so it's more correct to describe the
features as being "introduced into earlier editions".  Let's make that
change.

(Thanks to @tmandry for the suggestion.)
@traviscross
Copy link
Contributor Author

traviscross commented Oct 4, 2023

Thanks @tmandry for the review, for your collaboration, and for the acknowledgment and appreciation you so kindly expressed. I too am excited by how these new rules change Rust for the better.

We never added the `Captures` trait to the standard library, and it's
unlikely that we ever will due to this RFC, so there is no one single
way to define it.  However, different people may have different
expectations about it based on how those people have seen it defined
in the past.

Let's add a forward reference in the first place we use `Captures` to
the section on the `Captures` trick where we define the trait, and
let's add a footnote in that section that notes the existence of other
plausible implementations.

(Thanks to @spastorino for pointing out that `rustc` internally
currently uses a different definition of this trait.)
We placed the reference to our footnote about how `Captures` can be
defined in various ways inside of the colon that leads into the
example.  Since this footnote is really about the example itself
rather than about the text introducing the example, let's put the
reference to the footnote outside of the colon.  This looks a bit
better typographically anyway.
We use the `Captures` trait for exposition before formally introducing
it in a later section.  In an earlier commit, we had added some text
in parentheses to explain this.

Rather than breaking up the flow with parenthesized text, let's use
footnotes for this.

In the second place that we do this, we're using `Captures` only for
exposition; real code would not need to use the trick here.  We'll
mention that in the footnote as well.
What we want to show with `async fn` is that it automatically captures
all in-scope type and lifetime parameters in the returned opaque
`Future`.  We're doing this to contrast it with RPIT in Rust 2021 and
earlier editions which does not capture in the returned opaque type
all in-scope lifetime parameters automatically.

However, our examples did not well demonstrate this, because the
examples used the lifetime parameters in the `async fn` return types,
which results in those lifetime parameters appearing in the associated
type of each returned opaque `Future`.  In the RPIT desugarings, the
lifetime parameters would therefore appear in the bounds of the `impl
Trait` opaque types, and so would be captured regardless.

To better draw the distinction we want to draw, let's change each
`async fn` example to simply return the unit type (`()`).  This
ensures that in the RPIT desugarings any lifetime parameters will not
be automatically captured under the rules of Rust 2021 and earlier
editions.

For each example, we'll use each of the type and lifetime parameters
in the body of the function so that they will appear in the returned
hidden type.  This ensures that the RPIT desugarings for the examples
where a lifetime parameter is captured cannot be expressed without
using one of the tricks.

(Thanks to @tmandry for raising this important point.)
In two examples involving `async fn`, we annotate the returned opaque
type in comments.  In these comments, we had notated the trait bounds,
but we had not included the keyword `impl`.  Let's include that
keyword for greater clarity.

We'll also add backticks around the annotated return type in the
second example.  These were already present in the first example.
We have a sentence that speculates that if `async fn` had happened
before RPIT that the original lifetime capture rules for RPIT might
have matched those of `async fn`.  Let's improve the wording of this
sentence to make it more clear what we're saying and why we're
saying it.
In a footnote, we describe how there are various ways that `Captures`
can be defined.  However, we should point out that `Captures<'a> +
Captures<'b>` is not equivalent to `Captures<(&'a (), &'b ())>` as
lifetimes do not participate in trait selection in Rust.  This is in
fact the main reason that we defined `Captures` to accept a type
parameter rather than a lifetime parameter in this document.  Let's
include this caveat in the footnote.

(Thanks to @compiler-errors for reminding everyone of this.)
Copy link
Member

@aliemjay aliemjay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to specify the behavior of capturing the lifetimes of for<..> binders. The current implementation for TAIT/RPITIT does not allow it, but I would think it is essential for consistency.

#![feature(unboxed_closures)]
fn foo(s: &str) -> &str {
    s
}
fn bar() -> impl for<'a> Fn<(&'a str,), Output = impl Sized> {
    // should we allow `impl Sized` to capture `'a`?
    foo
}

@runiq
Copy link

runiq commented Nov 2, 2023

Is there somewhere I can read more about the reasoning behind capturing any and all lifetime generic parameters in scope? As someone who only gets to use Rust occasionally, this feels excessively broad, and I don't really understand the reason for it.

@azergante
Copy link

I think Rust's explicitness and consistency is one of its strength as it makes it easy to reason about the behavior of a program.

Implicit captures go against this explicitness, and I am worried this is already a half-broken feature: it does what one want in some cases, but it has to be manually undone the rest of the time. C++ is full of these half-broken features, and it makes it unpleasant to work with. From experience it is very frustrating to find out that a tool has been doing some automagic behind one's back that now has to be manually undone.

These may be newbie questions but as a Rust user without full understanding of the issue (as most Rust users out there I suppose) I wonder if there are no other viable solutions:

  • is the -> Trait + Captures<...> explicit solution not good enough ? this could be made easier by providing Capture traits for various capture arities in the standard library
  • could the compiler infer the minimum capture lifetimes and types needed, and capture just these instead of capturing everything in the context? I suppose the compiler already has to know this information to typecheck / lifetime check the returned value

In any case, I think having implicit features that work only some of the time is not great.

@traviscross
Copy link
Contributor Author

@runiq:

Is there somewhere I can read more about the reasoning behind capturing any and all lifetime generic parameters in scope? As someone who only gets to use Rust occasionally, this feels excessively broad, and I don't really understand the reason for it.

For discussion of this, you would want to look to RFC 1951, RFC 2394, RFC 3425, and the "Capturing lifetimes in RPITIT" document linked in Appendix A of this RFC.

@azergante:

These may be newbie questions but as a Rust user without full understanding of the issue (as most Rust users out there I suppose) I wonder if there are no other viable solutions.... I think having implicit features that work only some of the time is not great.

The decision to make the captures automatic happened back in the RFCs mentioned above. Here, we're simply reconciling some inconsistencies in a way that also allowed RPITIT to happen. Those documents may answer some of your questions. If you want to discuss further how or why we might want to rethink those earlier decisions, it'd probably be better to open that discussion separately on Zulip or IRLO.

@rfcbot rfcbot added finished-final-comment-period The final comment period is finished for this RFC. to-announce and removed final-comment-period Will be merged/postponed/closed in ~10 calendar days unless new substational objections are raised. labels Nov 4, 2023
@rfcbot
Copy link
Collaborator

rfcbot commented Nov 4, 2023

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

We have now created a tracking issue for RFC 3498.

Let's add that tracking issue to the RFC itself.
@traviscross traviscross merged commit 4254063 into rust-lang:master Nov 4, 2023
@traviscross
Copy link
Contributor Author

We have merged this RFC and opened a tracking issue: rust-lang/rust#117587

For further updates, you may want to subscribe to that.

Thanks to all of the members of T-lang for being courageous enough to answer "the big question", to @tmandry for his collaboration on the earlier design document for the 2024 lifetime capture rules, to @compiler-errors for helpful discussions and insights on this topic, and to all who reviewed this RFC and provided useful feedback and suggestions. Thanks for working to make Rust better.

@ehuss ehuss added the A-edition-2024 Area: The 2024 edition label Nov 12, 2023
compiler-errors added a commit to compiler-errors/rust that referenced this pull request Dec 5, 2023
…ules_2024, r=TaKO8Ki

Implement 2024-edition lifetime capture rules RFC

Implements rust-lang/rfcs#3498.
compiler-errors added a commit to compiler-errors/rust that referenced this pull request Dec 5, 2023
…ules_2024, r=TaKO8Ki

Implement 2024-edition lifetime capture rules RFC

Implements rust-lang/rfcs#3498.
bors added a commit to rust-lang-ci/rust that referenced this pull request Dec 10, 2023
…es_2024, r=TaKO8Ki

Implement 2024-edition lifetime capture rules RFC

Implements rust-lang/rfcs#3498.
github-actions bot pushed a commit to rust-lang/miri that referenced this pull request Dec 12, 2023
…r=TaKO8Ki

Implement 2024-edition lifetime capture rules RFC

Implements rust-lang/rfcs#3498.
lnicola pushed a commit to lnicola/rust-analyzer that referenced this pull request Apr 7, 2024
…r=TaKO8Ki

Implement 2024-edition lifetime capture rules RFC

Implements rust-lang/rfcs#3498.
@traviscross traviscross mentioned this pull request Apr 25, 2024
RalfJung pushed a commit to RalfJung/rust-analyzer that referenced this pull request Apr 27, 2024
…r=TaKO8Ki

Implement 2024-edition lifetime capture rules RFC

Implements rust-lang/rfcs#3498.
bors added a commit to rust-lang-ci/rust that referenced this pull request Aug 20, 2024
…=spastorino

Stabilize opaque type precise capturing (RFC 3617)

This PR partially stabilizes opaque type *precise capturing*, which was specified in [RFC 3617](rust-lang/rfcs#3617), and whose syntax was amended by FCP in [rust-lang#125836](rust-lang#125836).

This feature, as stabilized here, gives us a way to explicitly specify the generic lifetime parameters that an RPIT-like opaque type captures.  This solves the problem of overcapturing, for lifetime parameters in these opaque types, and will allow the Lifetime Capture Rules 2024 ([RFC 3498](rust-lang/rfcs#3498)) to be fully stabilized for RPIT in Rust 2024.

### What are we stabilizing?

This PR stabilizes the use of a `use<'a, T>` bound in return-position impl Trait opaque types.  Such a bound fully specifies the set of generic parameters captured by the RPIT opaque type, entirely overriding the implicit default behavior.  E.g.:

```rust
fn does_not_capture<'a, 'b>() -> impl Sized + use<'a> {}
//                               ~~~~~~~~~~~~~~~~~~~~
//                This RPIT opaque type does not capture `'b`.
```

The way we would suggest thinking of `impl Trait` types *without* an explicit `use<..>` bound is that the `use<..>` bound has been *elided*, and that the bound is filled in automatically by the compiler according to the edition-specific capture rules.

All non-`'static` lifetime parameters, named (i.e. non-APIT) type parameters, and const parameters in scope are valid to name, including an elided lifetime if such a lifetime would also be valid in an outlives bound, e.g.:

```rust
fn elided(x: &u8) -> impl Sized + use<'_> { x }
```

Lifetimes must be listed before type and const parameters, but otherwise the ordering is not relevant to the `use<..>` bound.  Captured parameters may not be duplicated.  For now, only one `use<..>` bound may appear in a bounds list.  It may appear anywhere within the bounds list.

### How does this differ from the RFC?

This stabilization differs from the RFC in one respect: the RFC originally specified `use<'a, T>` as syntactically part of the RPIT type itself, e.g.:

```rust
fn capture<'a>() -> impl use<'a> Sized {}
```

However, settling on the final syntax was left as an open question.  T-lang later decided via FCP in [rust-lang#125836](rust-lang#125836) to treat `use<..>` as a syntactic bound instead, e.g.:

```rust
fn capture<'a>() -> impl Sized + use<'a> {}
```

### What aren't we stabilizing?

The key goal of this PR is to stabilize the parts of *precise capturing* that are needed to enable the migration to Rust 2024.

There are some capabilities of *precise capturing* that the RFC specifies but that we're not stabilizing here, as these require further work on the type system.  We hope to lift these limitations later.

The limitations that are part of this PR were specified in the [RFC's stabilization strategy](https://rust-lang.github.io/rfcs/3617-precise-capturing.html#stabilization-strategy).

#### Not capturing type or const parameters

The RFC addresses the overcapturing of type and const parameters; that is, it allows for them to not be captured in opaque types.  We're not stabilizing that in this PR.  Since all in scope generic type and const parameters are implicitly captured in all editions, this is not needed for the migration to Rust 2024.

For now, when using `use<..>`, all in scope type and const parameters must be nameable (i.e., APIT cannot be used) and included as arguments.  For example, this is an error because `T` is in scope and not included as an argument:

```rust
fn test<T>() -> impl Sized + use<> {}
//~^ ERROR `impl Trait` must mention all type parameters in scope in `use<...>`
```

This is due to certain current limitations in the type system related to how generic parameters are represented as captured (i.e. bivariance) and how inference operates.

We hope to relax this in the future, and this stabilization is forward compatible with doing so.

#### Precise capturing for return-position impl Trait **in trait** (RPITIT)

The RFC specifies precise capturing for RPITIT.  We're not stabilizing that in this PR.  Since RPITIT already adheres to the Lifetime Capture Rules 2024, this isn't needed for the migration to Rust 2024.

The effect of this is that the anonymous associated types created by RPITITs must continue to capture all of the lifetime parameters in scope, e.g.:

```rust
trait Foo<'a> {
    fn test() -> impl Sized + use<Self>;
    //~^ ERROR `use<...>` precise capturing syntax is currently not allowed in return-position `impl Trait` in traits
}
```

To allow this involves a meaningful amount of type system work related to adding variance to GATs or reworking how generics are represented in RPITITs.  We plan to do this work separately from the stabilization.  See:

- rust-lang#124029

Supporting precise capturing for RPITIT will also require us to implement a new algorithm for detecting refining capture behavior.  This may involve looking through type parameters to detect cases where the impl Trait type in an implementation captures fewer lifetimes than the corresponding RPITIT in the trait definition, e.g.:

```rust
trait Foo {
    fn rpit() -> impl Sized + use<Self>;
}

impl<'a> Foo for &'a () {
    // This is "refining" due to not capturing `'a` which
    // is implied by the trait's `use<Self>`.
    fn rpit() -> impl Sized + use<>;

    // This is not "refining".
    fn rpit() -> impl Sized + use<'a>;
}
```

This stabilization is forward compatible with adding support for this later.

### The technical details

This bound is purely syntactical and does not lower to a [`Clause`](https://doc.rust-lang.org/1.79.0/nightly-rustc/rustc_middle/ty/type.ClauseKind.html) in the type system.  For the purposes of the type system (and for the types team's curiosity regarding this stabilization), we have no current need to represent this as a `ClauseKind`.

Since opaques already capture a variable set of lifetimes depending on edition and their syntactical position (e.g. RPIT vs RPITIT), a `use<..>` bound is just a way to explicitly rather than implicitly specify that set of lifetimes, and this only affects opaque type lowering from AST to HIR.

### FCP plan

While there's much discussion of the type system here, the feature in this PR is implemented internally as a transformation that happens before lowering to the type system layer.  We already support impl Trait types partially capturing the in scope lifetimes; we just currently only expose that implicitly.

So, in my (errs's) view as a types team member, there's nothing for types to weigh in on here with respect to the implementation being stabilized, and I'd suggest a lang-only proposed FCP (though we'll of course CC the team below).

### Authorship and acknowledgments

This stabilization report was coauthored by compiler-errors and TC.

TC would like to acknowledge the outstanding and speedy work that compiler-errors has done to make this feature happen.

compiler-errors thanks TC for authoring the RFC, for all of his involvement in this feature's development, and pushing the Rust 2024 edition forward.

### Open items

We're doing some things in parallel here.  In signaling the intention to stabilize, we want to uncover any latent issues so we can be sure they get addressed.  We want to give the maximum time for discussion here to happen by starting it while other remaining miscellaneous work proceeds.  That work includes:

- [x] Look into `syn` support.
  - dtolnay/syn#1677
  - dtolnay/syn#1707
- [x] Look into `rustfmt` support.
  - rust-lang#126754
- [x] Look into `rust-analyzer` support.
  - rust-lang/rust-analyzer#17598
  - rust-lang/rust-analyzer#17676
- [x] Look into `rustdoc` support.
  - rust-lang#127228
  - rust-lang#127632
  - rust-lang#127658
- [x] Suggest this feature to RfL (a known nightly user).
- [x] Add a chapter to the edition guide.
  - rust-lang/edition-guide#316
- [x] Update the Reference.
  - rust-lang/reference#1577

### (Selected) implementation history

* rust-lang/rfcs#3498
* rust-lang/rfcs#3617
* rust-lang#123468
* rust-lang#125836
* rust-lang#126049
* rust-lang#126753

Closes rust-lang#123432.

cc `@rust-lang/lang` `@rust-lang/types`

`@rustbot` labels +T-lang +I-lang-nominated +A-impl-trait +F-precise_capturing

Tracking:

- rust-lang#123432

----

For the compiler reviewer, I'll leave some inline comments about diagnostics fallout :^)

r? compiler
bors added a commit to rust-lang-ci/rust that referenced this pull request Aug 20, 2024
…=spastorino

Stabilize opaque type precise capturing (RFC 3617)

This PR partially stabilizes opaque type *precise capturing*, which was specified in [RFC 3617](rust-lang/rfcs#3617), and whose syntax was amended by FCP in [rust-lang#125836](rust-lang#125836).

This feature, as stabilized here, gives us a way to explicitly specify the generic lifetime parameters that an RPIT-like opaque type captures.  This solves the problem of overcapturing, for lifetime parameters in these opaque types, and will allow the Lifetime Capture Rules 2024 ([RFC 3498](rust-lang/rfcs#3498)) to be fully stabilized for RPIT in Rust 2024.

### What are we stabilizing?

This PR stabilizes the use of a `use<'a, T>` bound in return-position impl Trait opaque types.  Such a bound fully specifies the set of generic parameters captured by the RPIT opaque type, entirely overriding the implicit default behavior.  E.g.:

```rust
fn does_not_capture<'a, 'b>() -> impl Sized + use<'a> {}
//                               ~~~~~~~~~~~~~~~~~~~~
//                This RPIT opaque type does not capture `'b`.
```

The way we would suggest thinking of `impl Trait` types *without* an explicit `use<..>` bound is that the `use<..>` bound has been *elided*, and that the bound is filled in automatically by the compiler according to the edition-specific capture rules.

All non-`'static` lifetime parameters, named (i.e. non-APIT) type parameters, and const parameters in scope are valid to name, including an elided lifetime if such a lifetime would also be valid in an outlives bound, e.g.:

```rust
fn elided(x: &u8) -> impl Sized + use<'_> { x }
```

Lifetimes must be listed before type and const parameters, but otherwise the ordering is not relevant to the `use<..>` bound.  Captured parameters may not be duplicated.  For now, only one `use<..>` bound may appear in a bounds list.  It may appear anywhere within the bounds list.

### How does this differ from the RFC?

This stabilization differs from the RFC in one respect: the RFC originally specified `use<'a, T>` as syntactically part of the RPIT type itself, e.g.:

```rust
fn capture<'a>() -> impl use<'a> Sized {}
```

However, settling on the final syntax was left as an open question.  T-lang later decided via FCP in [rust-lang#125836](rust-lang#125836) to treat `use<..>` as a syntactic bound instead, e.g.:

```rust
fn capture<'a>() -> impl Sized + use<'a> {}
```

### What aren't we stabilizing?

The key goal of this PR is to stabilize the parts of *precise capturing* that are needed to enable the migration to Rust 2024.

There are some capabilities of *precise capturing* that the RFC specifies but that we're not stabilizing here, as these require further work on the type system.  We hope to lift these limitations later.

The limitations that are part of this PR were specified in the [RFC's stabilization strategy](https://rust-lang.github.io/rfcs/3617-precise-capturing.html#stabilization-strategy).

#### Not capturing type or const parameters

The RFC addresses the overcapturing of type and const parameters; that is, it allows for them to not be captured in opaque types.  We're not stabilizing that in this PR.  Since all in scope generic type and const parameters are implicitly captured in all editions, this is not needed for the migration to Rust 2024.

For now, when using `use<..>`, all in scope type and const parameters must be nameable (i.e., APIT cannot be used) and included as arguments.  For example, this is an error because `T` is in scope and not included as an argument:

```rust
fn test<T>() -> impl Sized + use<> {}
//~^ ERROR `impl Trait` must mention all type parameters in scope in `use<...>`
```

This is due to certain current limitations in the type system related to how generic parameters are represented as captured (i.e. bivariance) and how inference operates.

We hope to relax this in the future, and this stabilization is forward compatible with doing so.

#### Precise capturing for return-position impl Trait **in trait** (RPITIT)

The RFC specifies precise capturing for RPITIT.  We're not stabilizing that in this PR.  Since RPITIT already adheres to the Lifetime Capture Rules 2024, this isn't needed for the migration to Rust 2024.

The effect of this is that the anonymous associated types created by RPITITs must continue to capture all of the lifetime parameters in scope, e.g.:

```rust
trait Foo<'a> {
    fn test() -> impl Sized + use<Self>;
    //~^ ERROR `use<...>` precise capturing syntax is currently not allowed in return-position `impl Trait` in traits
}
```

To allow this involves a meaningful amount of type system work related to adding variance to GATs or reworking how generics are represented in RPITITs.  We plan to do this work separately from the stabilization.  See:

- rust-lang#124029

Supporting precise capturing for RPITIT will also require us to implement a new algorithm for detecting refining capture behavior.  This may involve looking through type parameters to detect cases where the impl Trait type in an implementation captures fewer lifetimes than the corresponding RPITIT in the trait definition, e.g.:

```rust
trait Foo {
    fn rpit() -> impl Sized + use<Self>;
}

impl<'a> Foo for &'a () {
    // This is "refining" due to not capturing `'a` which
    // is implied by the trait's `use<Self>`.
    fn rpit() -> impl Sized + use<>;

    // This is not "refining".
    fn rpit() -> impl Sized + use<'a>;
}
```

This stabilization is forward compatible with adding support for this later.

### The technical details

This bound is purely syntactical and does not lower to a [`Clause`](https://doc.rust-lang.org/1.79.0/nightly-rustc/rustc_middle/ty/type.ClauseKind.html) in the type system.  For the purposes of the type system (and for the types team's curiosity regarding this stabilization), we have no current need to represent this as a `ClauseKind`.

Since opaques already capture a variable set of lifetimes depending on edition and their syntactical position (e.g. RPIT vs RPITIT), a `use<..>` bound is just a way to explicitly rather than implicitly specify that set of lifetimes, and this only affects opaque type lowering from AST to HIR.

### FCP plan

While there's much discussion of the type system here, the feature in this PR is implemented internally as a transformation that happens before lowering to the type system layer.  We already support impl Trait types partially capturing the in scope lifetimes; we just currently only expose that implicitly.

So, in my (errs's) view as a types team member, there's nothing for types to weigh in on here with respect to the implementation being stabilized, and I'd suggest a lang-only proposed FCP (though we'll of course CC the team below).

### Authorship and acknowledgments

This stabilization report was coauthored by compiler-errors and TC.

TC would like to acknowledge the outstanding and speedy work that compiler-errors has done to make this feature happen.

compiler-errors thanks TC for authoring the RFC, for all of his involvement in this feature's development, and pushing the Rust 2024 edition forward.

### Open items

We're doing some things in parallel here.  In signaling the intention to stabilize, we want to uncover any latent issues so we can be sure they get addressed.  We want to give the maximum time for discussion here to happen by starting it while other remaining miscellaneous work proceeds.  That work includes:

- [x] Look into `syn` support.
  - dtolnay/syn#1677
  - dtolnay/syn#1707
- [x] Look into `rustfmt` support.
  - rust-lang#126754
- [x] Look into `rust-analyzer` support.
  - rust-lang/rust-analyzer#17598
  - rust-lang/rust-analyzer#17676
- [x] Look into `rustdoc` support.
  - rust-lang#127228
  - rust-lang#127632
  - rust-lang#127658
- [x] Suggest this feature to RfL (a known nightly user).
- [x] Add a chapter to the edition guide.
  - rust-lang/edition-guide#316
- [x] Update the Reference.
  - rust-lang/reference#1577

### (Selected) implementation history

* rust-lang/rfcs#3498
* rust-lang/rfcs#3617
* rust-lang#123468
* rust-lang#125836
* rust-lang#126049
* rust-lang#126753

Closes rust-lang#123432.

cc `@rust-lang/lang` `@rust-lang/types`

`@rustbot` labels +T-lang +I-lang-nominated +A-impl-trait +F-precise_capturing

Tracking:

- rust-lang#123432

----

For the compiler reviewer, I'll leave some inline comments about diagnostics fallout :^)

r? compiler
github-actions bot pushed a commit to rust-lang/miri that referenced this pull request Aug 26, 2024
Stabilize opaque type precise capturing (RFC 3617)

This PR partially stabilizes opaque type *precise capturing*, which was specified in [RFC 3617](rust-lang/rfcs#3617), and whose syntax was amended by FCP in [#125836](rust-lang/rust#125836).

This feature, as stabilized here, gives us a way to explicitly specify the generic lifetime parameters that an RPIT-like opaque type captures.  This solves the problem of overcapturing, for lifetime parameters in these opaque types, and will allow the Lifetime Capture Rules 2024 ([RFC 3498](rust-lang/rfcs#3498)) to be fully stabilized for RPIT in Rust 2024.

### What are we stabilizing?

This PR stabilizes the use of a `use<'a, T>` bound in return-position impl Trait opaque types.  Such a bound fully specifies the set of generic parameters captured by the RPIT opaque type, entirely overriding the implicit default behavior.  E.g.:

```rust
fn does_not_capture<'a, 'b>() -> impl Sized + use<'a> {}
//                               ~~~~~~~~~~~~~~~~~~~~
//                This RPIT opaque type does not capture `'b`.
```

The way we would suggest thinking of `impl Trait` types *without* an explicit `use<..>` bound is that the `use<..>` bound has been *elided*, and that the bound is filled in automatically by the compiler according to the edition-specific capture rules.

All non-`'static` lifetime parameters, named (i.e. non-APIT) type parameters, and const parameters in scope are valid to name, including an elided lifetime if such a lifetime would also be valid in an outlives bound, e.g.:

```rust
fn elided(x: &u8) -> impl Sized + use<'_> { x }
```

Lifetimes must be listed before type and const parameters, but otherwise the ordering is not relevant to the `use<..>` bound.  Captured parameters may not be duplicated.  For now, only one `use<..>` bound may appear in a bounds list.  It may appear anywhere within the bounds list.

### How does this differ from the RFC?

This stabilization differs from the RFC in one respect: the RFC originally specified `use<'a, T>` as syntactically part of the RPIT type itself, e.g.:

```rust
fn capture<'a>() -> impl use<'a> Sized {}
```

However, settling on the final syntax was left as an open question.  T-lang later decided via FCP in [#125836](rust-lang/rust#125836) to treat `use<..>` as a syntactic bound instead, e.g.:

```rust
fn capture<'a>() -> impl Sized + use<'a> {}
```

### What aren't we stabilizing?

The key goal of this PR is to stabilize the parts of *precise capturing* that are needed to enable the migration to Rust 2024.

There are some capabilities of *precise capturing* that the RFC specifies but that we're not stabilizing here, as these require further work on the type system.  We hope to lift these limitations later.

The limitations that are part of this PR were specified in the [RFC's stabilization strategy](https://rust-lang.github.io/rfcs/3617-precise-capturing.html#stabilization-strategy).

#### Not capturing type or const parameters

The RFC addresses the overcapturing of type and const parameters; that is, it allows for them to not be captured in opaque types.  We're not stabilizing that in this PR.  Since all in scope generic type and const parameters are implicitly captured in all editions, this is not needed for the migration to Rust 2024.

For now, when using `use<..>`, all in scope type and const parameters must be nameable (i.e., APIT cannot be used) and included as arguments.  For example, this is an error because `T` is in scope and not included as an argument:

```rust
fn test<T>() -> impl Sized + use<> {}
//~^ ERROR `impl Trait` must mention all type parameters in scope in `use<...>`
```

This is due to certain current limitations in the type system related to how generic parameters are represented as captured (i.e. bivariance) and how inference operates.

We hope to relax this in the future, and this stabilization is forward compatible with doing so.

#### Precise capturing for return-position impl Trait **in trait** (RPITIT)

The RFC specifies precise capturing for RPITIT.  We're not stabilizing that in this PR.  Since RPITIT already adheres to the Lifetime Capture Rules 2024, this isn't needed for the migration to Rust 2024.

The effect of this is that the anonymous associated types created by RPITITs must continue to capture all of the lifetime parameters in scope, e.g.:

```rust
trait Foo<'a> {
    fn test() -> impl Sized + use<Self>;
    //~^ ERROR `use<...>` precise capturing syntax is currently not allowed in return-position `impl Trait` in traits
}
```

To allow this involves a meaningful amount of type system work related to adding variance to GATs or reworking how generics are represented in RPITITs.  We plan to do this work separately from the stabilization.  See:

- rust-lang/rust#124029

Supporting precise capturing for RPITIT will also require us to implement a new algorithm for detecting refining capture behavior.  This may involve looking through type parameters to detect cases where the impl Trait type in an implementation captures fewer lifetimes than the corresponding RPITIT in the trait definition, e.g.:

```rust
trait Foo {
    fn rpit() -> impl Sized + use<Self>;
}

impl<'a> Foo for &'a () {
    // This is "refining" due to not capturing `'a` which
    // is implied by the trait's `use<Self>`.
    fn rpit() -> impl Sized + use<>;

    // This is not "refining".
    fn rpit() -> impl Sized + use<'a>;
}
```

This stabilization is forward compatible with adding support for this later.

### The technical details

This bound is purely syntactical and does not lower to a [`Clause`](https://doc.rust-lang.org/1.79.0/nightly-rustc/rustc_middle/ty/type.ClauseKind.html) in the type system.  For the purposes of the type system (and for the types team's curiosity regarding this stabilization), we have no current need to represent this as a `ClauseKind`.

Since opaques already capture a variable set of lifetimes depending on edition and their syntactical position (e.g. RPIT vs RPITIT), a `use<..>` bound is just a way to explicitly rather than implicitly specify that set of lifetimes, and this only affects opaque type lowering from AST to HIR.

### FCP plan

While there's much discussion of the type system here, the feature in this PR is implemented internally as a transformation that happens before lowering to the type system layer.  We already support impl Trait types partially capturing the in scope lifetimes; we just currently only expose that implicitly.

So, in my (errs's) view as a types team member, there's nothing for types to weigh in on here with respect to the implementation being stabilized, and I'd suggest a lang-only proposed FCP (though we'll of course CC the team below).

### Authorship and acknowledgments

This stabilization report was coauthored by compiler-errors and TC.

TC would like to acknowledge the outstanding and speedy work that compiler-errors has done to make this feature happen.

compiler-errors thanks TC for authoring the RFC, for all of his involvement in this feature's development, and pushing the Rust 2024 edition forward.

### Open items

We're doing some things in parallel here.  In signaling the intention to stabilize, we want to uncover any latent issues so we can be sure they get addressed.  We want to give the maximum time for discussion here to happen by starting it while other remaining miscellaneous work proceeds.  That work includes:

- [x] Look into `syn` support.
  - dtolnay/syn#1677
  - dtolnay/syn#1707
- [x] Look into `rustfmt` support.
  - rust-lang/rust#126754
- [x] Look into `rust-analyzer` support.
  - rust-lang/rust-analyzer#17598
  - rust-lang/rust-analyzer#17676
- [x] Look into `rustdoc` support.
  - rust-lang/rust#127228
  - rust-lang/rust#127632
  - rust-lang/rust#127658
- [x] Suggest this feature to RfL (a known nightly user).
- [x] Add a chapter to the edition guide.
  - rust-lang/edition-guide#316
- [x] Update the Reference.
  - rust-lang/reference#1577

### (Selected) implementation history

* rust-lang/rfcs#3498
* rust-lang/rfcs#3617
* rust-lang/rust#123468
* rust-lang/rust#125836
* rust-lang/rust#126049
* rust-lang/rust#126753

Closes #123432.

cc `@rust-lang/lang` `@rust-lang/types`

`@rustbot` labels +T-lang +I-lang-nominated +A-impl-trait +F-precise_capturing

Tracking:

- rust-lang/rust#123432

----

For the compiler reviewer, I'll leave some inline comments about diagnostics fallout :^)

r? compiler
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-edition-2024 Area: The 2024 edition disposition-merge This RFC is in PFCP or FCP with a disposition to merge it. finished-final-comment-period The final comment period is finished for this RFC. T-lang Relevant to the language team, which will review and decide on the RFC. to-announce
Projects
None yet
Development

Successfully merging this pull request may close these issues.