Improve the remaining rough corners of the module system

[Kimundi]

Motivation

The module system (specifically name resolution) still has a few rough edges, footguns and corner cases that can be removed in a way that doesn't change its core semantics. Similarly, error messages and doc generation need to be smarter about the module system in some areas.

This issue proposes all changes that the author regards as improvements, and they would build on prior work like #116 and #385.

This is not a RFC due to its size, and lack of details. Rather, the idea is to paint the big picture, and have a central place to discuss wether this is a direction the module system should be taken or not.

Proposals

• Change wildcard import semantics to be shadow-able by other items and non-wildcard imports.

  • The shadowing applies to any access to paths in the module, including other use items.
  • Don't error on non-disjoint wildcard imports, only error on access of a name
    publicly visible through more more than one of them
    if the names refer to different logical items.
    Recommend to use an explicit import to choose between them in the error message.
  • This would mitigate the dangers of extending the set of exported items in a module, since
    downstream users using wildcard imports can always clarify or guard in advance
    with an explicit import.
  • This would enable the liberal use of wildcard imports for things like custom preludes, test modules, etc.
  • This would address issue Multiple globs are basically unusable #553 and
    Allow pub use re-imports/exports of already privately imported symbols #1285
  • This should be completely backwards compatible to implement.
  • The unused import lint would trigger for any wildcard import through
    which zero names are resolved.

• Expose private imports as regular importable items to submodules

  • This makes imports non-special in regard to privacy and other imports.
  • This enables seamless moving around of items by replacing them with a import,
    breaking no other code on the inside of a module.
  • This allows concise patterns:
    • A single use super::*; in a test submodule to import everything
      needed to test the parent.
    • use self::submodule::*; to flatten a module hierarchy.
    • mod my_prelude { pub use other_prelude::*; pub use my_items::*; }
      to easily merge modules together.
  • This is backwards compatible if implemented at the same time as the wildcard import change,
    since any names becoming visible with this change would not conflict with existing imports:
    • Existing non-wildcard imports would shadow the additional names in existing wildcard imports.
    • Existing non-wildcard imports will not change meaning and
      existing modules will not fail to compile since imports are currently strictly
      forbidden from shadowing other items.

• Make imports in non-module scopes see items defined in non-module scopes

  • This is about making this print "inner a" twice:

    fn a() {
        println!("outer a");
    }
    
    fn main() {
        {
            fn a() {
                println!("inner a");
            }
    
            use self::a as b;
    
            a(); // Currently prints "inner a"
            b(); // Currently prints "outer a"
        }
    }

  • This would address issues like Ability to refer to a path relative to the current scope in use declarations #959 and extern crate foo with use self::foo doesn't work inside a function rust#27626.

  • This would remove the corner case of imports not composing well
    with items in a nested scope.

  • This is a breaking change.

  • However, the breakage scenario is obscure and unidiomatic
    enough that a crater run
    and community outreach might likely discover no regressions.

• Address recursive mutual import issues.

  • This is basically a bugfix.
  • This has, however, caused enough trouble to affect decisions about
    language semantics in the past.
  • I am not informed enough about resolve itself to know wether
    this is algorithmically feasible to implement in rustc, but
    as far as I abstractly know about the problem, it should be doable.
    (Say with a fixpoint algorithm gathering the set of possible imports from wildcards, kept finite by normalizing the paths to the canonical path of a item)
  • Mutual recursive wildcard imports that actually reach items
    should become a non-error with the new wildcard import semantics.
    This should work, and doesn't today:

    mod a { pub use b::*; pub struct a_item; }
    mod b { pub use a::*; pub struct b_item; }
    use a::b_item;
    use b::a_item;

  • Mutual recursive (wildcard) imports should be detected and produce a
    nice error message. This should fail to compile
    (and already does today, though not with a nice message):

    mod a { pub use b::*; }
    mod b { pub use a::*; }
    use a::b_item;
    use b::a_item;

    mod a { pub use b::c; }
    mod b { pub use a::c; }

• Error messages and rustdoc output should treat imports and paths of imported
  items smarter.

  • This does not change language semantics.
  • This describes heuristics to improve error messages and docs.
  • Each item has a "canonical path" in its crate that uniquely describes its definition:

    mod a {
        pub struct b; // canonical path <crate>::a::b
    }
    use a::b; // non-canonical path <crate>::b for canonical path <crate>::a::b

  • Each resolvable name is potentially a chain of reexports "terminating"
    in a non-import item:

    mod a {
        pub struct b;
    }
    use a::b as c;
    use c as d;
    // Visible names in <crate>:
    // a              terminating in a
    // a::b           terminating in a::b
    // c -> a::b      terminating in a::b
    // d -> c -> a::b terminating in a::b

  extern crate statements count as a reexport here.

  • Each chain of imports in a resolvable name potentially terminates in a
    item private from the location of the start of the chain.
    In this case, the last visible reexport in the chain is defined
    as "visibly-terminating":

    mod a {
        pub use self::c as b;
        struct c;
    }
    // Visible names in <crate>:
    // a            terminating in a,    visibly-terminating in a
    // a::b -> a::c terminating in a::c, visibly-terminating in a::b

  • Currently, if the compiler prints the path of an item in a error message
    it is <name_of_the_crate>::<canonical_path_in_the_crate>. Example:

    use std::rc::Rc as RefCount;
  
    fn main() {
        let () = RefCount::new(5);
    }

    <anon>:4:9: 4:11 error: mismatched types:
    expected `alloc::rc::Rc<_>`,
       found `()`
    (expected struct `alloc::rc::Rc`,
               found ()) [E0308]
  

  • A more useful heuristic would be to print
    <resolved-name-in-current-contex> defined at <visibly-terminating-path>.
    For example:

    use std::rc::Rc as RefCount;
  
    fn main() {
        let () = RefCount::new(5);
    }

    <anon>:4:9: 4:11 error: mismatched types:
    expected `RefCount<_>` defined at std::rc::Rc,
       found `()`
    (expected struct `RefCount` defined at `std::rc::Rc`,
               found ()) [E0308]
  

  • If there is no visibly-terminating-item, print
    extern crate <name_of_the_crate>::<visibly-terminating-path-in-that-crate>
    similarly to before. Maybe also add a hash or version number to clear up
    issues like in Two different versions of a crate interacting leads to unhelpful error messages rust#22750.
    • This indicates to the user that the mentioned item is not part of the
      current crate in any way, and that he would have to add a extern crate
      item or resolve a version mismatch to change that.
  • Rustdoc should, for each crate it generates documentation for:
    • Treat all use statements that are visibly-terminating in the
      reexport chain of any other use item in that crate as its terminating item.
      • This uniformly resolves issues like presenting the libstd facade in the
        docs.
      • If the terminating item is defined in another crate and rustdoc
        currently generates documentation for that one as well (to make cross-crate links),
        also append a "defined at "
        note that links there.
        • This documents the relationship between dependencies
    • Treat all other use statements uniformly, for example by showing them simply
      as a "reexport" linking to the visibly-terminating item.
    • Treat items or links to items that don't have a visibly-terminating import
      referring to them in the current crate like today.
      • This handles the common situation of visible definitions or types
        defined in another crate that is not also reexported,
        like the return type of a pub fn foo() -> Rc<u32> in a crate
        not reexporting std or alloc.

Precedence

• In Multiple globs are basically unusable #553 @paulp makes the important
  comment to look at what Scala did in this whole space, and the semantics
  described at http://www.scala-lang.org/files/archive/spec/2.11/02-identifiers-names-and-scopes.html,
  where applicable, seem to mostly match the semantics of Rusts name resolution if the
  tweaks described above get applied.
• @nikomatsakis has an older unfinished RFC about the new wildcard import semantic.

[felixc]

As discussed in #1285, a revision of these features would greatly ease the pain of writing wrapper libraries that wish to glob-import the items being wrapped, but also publicly re-export a subset of them.

The ability to do:

use foo::*;
pub use foo::Bar;

would be very much appreciated.

👍

[ticki]

👍

[nrc]

Several of these suggestions are addressed by #1560

[Kimundi]

Indeed, a lot has been done in the last year :)

[ebkalderon]

Just curious, what is the current status of these efforts?

[sfackler]

"Expose private imports as regular importable items to submodules" is done, as is much of "Address recursive mutual import issues". Not sure about the others off the top of my head.