tracking issue for const_indexing stabilization

[oli-obk]

The feature called const_indexing allows the indexing operation to be performed while calculating a true constant like an array length.

See the implementation at #25570

[debris]

I've updated rust nightly today and i've stumbled upon const_indexing issue in my lib.

const indexing is an unstable feature

The code was compiling fine on 1.4.0, beta and old nightly, so I assume it might be the issue with new feature itself, not my code. After quick investigation I worked around the issue in this commit: debris/tiny-keccak@591d611 . Hope it helps with stabilization!

[oli-obk]

oh... not good. I know what's going on. You are using the index operation on the rhs of a bitshift operation. So during check_const the rhs is const_evaled, which triggers this... Fixing this makes the code rather ugly. Basically I need to make sure that during compiler checks no feature gate errors are emitted but simply const eval fails...

[frewsxcv]

A similar regression also happened here: ende76/brotli-rs#14

[nikomatsakis]

@eddyb do you see any reason not to stabilize this? Seems like we're going to wind up with a model that supports this sort of thing, at least in some cases.

[eddyb]

The actual addresses would be different... probably fine though, we already do the same thing in trans.
EDIT: ignore me, this only matters in trans. If qualify_consts passes then miri likely can eval it.
@oli-obk @solson Any objections regarding miri? Doubt it but just in case.

[solson]

Indexing will force allocations in miri, unlike most things supported by legacy const eval, but it wouldn't be the only thing (e.g. I think *&42 is valid in consts today). It sounds fine.

[quadrupleslap]

Is this going to be stabilised soon? I tried it on stable, and it works without any issues, but it's feature-gated only on nightly. Why does it seem to work on stable?

[oli-obk]

@quadrupleslap: that sounds buggy, can you give an example code exhibiting that behaviour?

[quadrupleslap]

Okay, my bad, I was using them in different ways. I think this behaviour is buggy, though:

const HELLO: &'static [u8; 5] = b"hello";
const H: u8 = HELLO[0];

fn main() {
    println!("'{:?}' starts with '{:?}'!", HELLO, H);
    
    // Comment the let and match out and everything works.
    // It looks like constants aren't guaranteed to be constant.
    let n = 3;
    match n {
        H => {
            println!("Why does it only error if it's used in a constant position?");
        },
        _ => ()
    }
    
    // I think if indexing in constants is unstable, rustc should error
    // even if the constant isn't used in a constant position.
}

[oli-obk]

That's just a symptom of our 3 separate const evaluator implementations. (HIR folding, trans MIR interpretation, pattern creation)

[quadrupleslap]

I... what? That's terrifying, but okay, I guess. Thanks for the read!

[oli-obk]

Don't worry, there's work being done to merge them into a single one.

[eddyb]

Well, I think there's only two? But the HIR one can work both before type-checking and after (differently).

[oli-obk]

I would like to stabilize const indexing in the process of moving from old ctfe to miri. While the feature gate could be kept, it would require some work which would simply be useless if we're going to stabilize const indexing anyway.

[cramertj]

FCP'ing since #46882 would stabilize.

@rfcbot fcp merge

[rfcbot]

Team member @cramertj has proposed to merge this. The next step is review by the rest of the tagged teams:

• @aturon
• @cramertj
• @eddyb
• @nikomatsakis
• @nrc
• @pnkfelix
• @withoutboats

No concerns currently listed.

Once these reviewers reach consensus, this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[nikomatsakis]

I'm going to use this feature as a kind of "proving ground" -- I feel frustrated that we often don't have a lot of documentaiton on precisely what is being stabilized and what the current behavior is. Can somebody -- perhaps @oli-obk -- write-up something that says:

• What the new behavior is and what some of the interesting edge cases or limits are
• What are some test cases that show the new behavior and those limits
  • these ought to be pointers to tests in the code base, so that we can be sure that behavior won't regress

[oli-obk]

Did you have something in mind like this?

New behaviour and edge cases

The new behaviour is that similar to accessing a tuple field, you can now access an array element. The only difference is that you can compute which element you want instead of having to write it out literally.

Similar to tuples, the array is computed entirely beforehand and the indexing only happens if all elements of the array are computed correctly.

Similar to tuples again, accessing an an out of bounds element will produce a compiler error. Unlike tuples, since the index depends on a computation, you can cause the indexing to fail depending on the platform. E.g. [1, 2, 3, 4, 5][std::size_of::<usize>()] will produce a compile-time error on 64 bit systems, but not on 32 bit systems. But that is nothing new, since you can already produce an error with 0xFFFFFFFF - std::usize::MAX

Emulating if with arrays

While you can emulate branching with arrays and bool casting, you won't gain much out of it, because all branches are evaluated, irrelevant of the condition. The following will produce an error unless X and Y are both 0, because either X - Y will overflow or Y - X will overflow (assuming unsigned types for X and Y)

[X - Y, Y - X][(X < Y) as usize]

So this differs from the assumed potential future support for if which would not error in

if X < Y { Y - X } else { X - Y }

Test cases

rust/src/test/run-pass/issue-29914.rs

Lines 12 to 15 in b1363a7

	const ARR: [usize; 5] = [5, 4, 3, 2, 1];
	fn main() {
	assert_eq!(3, ARR[ARR[3]]);

rust/src/test/run-pass/array_const_index-1.rs

Lines 14 to 17 in b1363a7

	const ARR: [i32; 6] = [42, 43, 44, 45, 46, 47];
	const IDX: usize = 3;
	const VAL: i32 = ARR[IDX];
	const BLUB: [i32; (ARR[0] - 41) as usize] = [5];

rust/src/test/compile-fail/const-array-oob-arith.rs

Lines 13 to 18 in b1363a7

	const ARR: [i32; 6] = [42, 43, 44, 45, 46, 47];
	const IDX: usize = 3;
	const VAL: i32 = ARR[IDX];
	const BONG: [i32; (ARR[0] - 41) as usize] = [5];
	const BLUB: [i32; (ARR[0] - 40) as usize] = [5]; //~ ERROR: mismatched types
	const BOO: [i32; (ARR[0] - 41) as usize] = [5, 99]; //~ ERROR: mismatched types

rust/src/test/compile-fail/const-err-early.rs

Lines 18 to 19 in b1363a7

	pub const E: u8 = [5u8][1];
	//~^ ERROR index out of bounds: the len is 1 but the index is 1

rust/src/test/run-pass/const-fn.rs

Lines 31 to 33 in b1363a7

	const fn generic_arr<T: Copy>(t: [T; 1]) -> T {
	t[0]
	}

rust/src/test/compile-fail/const-array-oob.rs

Lines 15 to 20 in e1cb9ba

	const FOO: [usize; 3] = [1, 2, 3];
	const BAR: usize = FOO[5]; // no error, because the error below occurs before regular const eval
	const BLUB: [u32; FOO[4]] = [5, 6];
	//~^ ERROR constant evaluation error [E0080]
	//~| index out of bounds: the len is 3 but the index is 4

rust/src/test/compile-fail/lint-exceeding-bitshifts.rs

Lines 72 to 73 in b1363a7

	let n = 1i64 >> [63][0];
	let n = 1i64 >> [64][0]; //~ ERROR: bitshift exceeds the type's number of bits

[oli-obk]

@nikomatsakis I added a test to the PR: 83e6d3c

[nikomatsakis]

@oli-obk thanks!

@rfcbot reviewed

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[mrhota]

@oli-obk @nikomatsakis in order to stabilize, don't we need at least an open PR on rust-lang-nursery/reference documenting the to-be-stabilized feature? The Unstable Book hardly even has anything about this.

[mrhota]

you also need a PR on the unstable book removing the page about this feature

[oli-obk]

@mrhota: while discussion around the semantics were quite extensive, the feature itself is not more special than any of the mathematical operations. They all have error modes (e.g. overflow or index out of bounds), all input arguments are validated entirely before the result is computed, even if not all of them are needed (e.g. {5u8 - 6; 42} + 3 will error even though the 5u8 - 6 will never show up in the result.

Additionally, the reference already mentions array indexing in https://doc.rust-lang.org/nightly/reference/expressions.html#constant-expressions

[nikomatsakis]

I can believe that the existing documentation is sufficient -- I've not looked closely, but I doubt that the precise borders of what constitutes a legal constant expression are too well-defined.

But at minimum removing the "unstable page" makes sense!

[rfcbot]

The final comment period is now complete.

[nikomatsakis]

@oli-obk is there a PR for stabilizing this work? I guess landing miri would probably do it...?

[oli-obk]

Yes this is part of the miri pr