Tracking Issue for raw slice getters (slice_ptr_get)

[RalfJung]

This is a tracking issue for indexing methods on raw slices: get_unchecked(_mut) and as_(mut_/non_null_)ptr on raw slices (mutable and const raw pointers and NonNull).

The feature gate for the issue is #![feature(slice_ptr_get)].

Public API

impl<T> *mut [T] {
    pub const fn as_mut_ptr(self) -> *mut T {}
    pub unsafe fn get_unchecked_mut<I>(self, index: I) -> *mut I::Output where I: SliceIndex<[T]>;
}

impl<T> *const [T] {
    pub const fn as_ptr(self) -> *const T {}
    pub unsafe fn get_unchecked<I>(self, index: I) -> *const I::Output where I: SliceIndex<[T]>;
}

impl<T> NonNull<[T]> {
    pub const fn as_non_null_ptr(self) -> NonNull<T> {}
    pub const fn as_mut_ptr(self) -> *mut T {}
    pub unsafe fn get_unchecked_mut<I>(self, index: I) -> NonNull<I::Output> where I: SliceIndex<[T]>;
}

History / Steps

• Initial PR: add (unchecked) indexing methods to raw (and NonNull) slices #73986
• Add NonNull::as_mut_ptr: Add as_mut_ptr to NonNull<[T]> #75248
• Final commenting period (FCP)
• Stabilization PR

Open questions

• Potential blockers: rustc performs auto-ref when a raw pointer would be enough #73987, Missing unsizing coercions for raw slice pointers #74679
• Should this use arbitrary_self_types (Arbitrary self types v2 rfcs#3519)

[RalfJung]

#74679 is a potential blocker for this.

[RalfJung]

With #75152, the allocator trait started making use of this.

[TimDiekmann]

Especially with AllocRef using it, it would come in handy if we could also add

impl<T> NonNull<[T]> {
    #[inline]
    fn as_mut_ptr(self) -> *mut T {
        self.as_non_null_ptr().as_ptr()
    }
}

If we want this I'd make a PR.

[RalfJung]

Seems reasonable to me -- @Amanieu what do you think?

[TimDiekmann]

An alternative name would be as_raw_ptr to avoid confusion with as_ptr, which already returns *mut T. This would also apply to *const [T] and *mut [T].

[Amanieu]

I don't particularly care for the name bikeshed but I agree that this method would be useful.

[RalfJung]

While using these methods for an upcoming PR, I noticed one thing: there is no const-to-mut coercion for raw pointers (at least not for slices, or maybe method lookup just is unable to use it), so one has to use get_unchecked_mut on *mut [T].

I wonder if it would make sense to just not have the mut in the name (but it would likely be inconsistent).

[steffahn]

There’s discussion here about adding general *const T -> *mut T and *mut T -> *const T helper methods (as a way to avoid as casts). The method name as_mut_ptr sounds like it does exactly that, but it's currently used here for *mut [T] -> *mut T. Reading the current NonNull documentation was also highly confusing; particularly when looking at the method list in the side-bar.

I would personally appreciate e.g. some mention to slice in the name of the as_[…]ptr family of methods; in any case, I just mainly wanted to note here (just so it won't be forgotten) that in case such general conversion methods are going to be wanted, there might be interest in a different meaning for the method name as_mut_ptr.

[RalfJung]

These methods are needed to work around the problem described in #99437. Though interestingly, to preserve the full semantics of the original code but fix the aliasing problems, we would want bounds-checked getters on raw slices.

[kornelski]

@steffahn these methods could be named cast_mut() and cast_shared() (or is it const?), alluding to ptr::cast(). cast is the replacement for the as operator in pointers. In types like slice.as_ptr() and vec.as_ptr() this method changes the type, but not shared/mut status.

NonNull is weird tho. Maybe it could get a redundant as_mut_ptr() just to reduce the weirdness?

[JonathanWoollett-Light]

Has there been any progress? It appears all required work is done? This feature will be useful in adding safety munmap, madvise and other mman functions in https://crates.io/crates/nix specifically extension work to nix-rust/nix#2000 as mentioned nix-rust/nix#2000 (comment).

[Ralith]

The API docs for get_unchecked and get_unchecked_mut say:

Calling this method [...] when self is not dereferenceable is undefined behavior

This is too strong, right? self is not dereferenceable when e.g. an exclusive reference to any of its elements exists, but it should be sound to use these methods to construct pointers to other elements, or even to an already-referenced element if the constructed pointer isn't dereferenced. Was the intention to instead specify that the pointer must address allocated memory?

[RalfJung]

"dereferenceable" generally means "points to memory that is inbounds of an allocation".

However, these functions take references, so it is indeed UB to call them when there are aliasing conflicts: get_unchecked acts like a read of the entire slice, and get_unchecked_mut acts like a write of the entire slice. These rules are not final, but until Rust has a precise aliasing model you should follow these conservative rules to ensure maximal chance of being compatible with the future memory model.

[Ralith]

"dereferenceable" generally means "points to memory that is inbounds of an allocation".

That's surprising; in Rust API documentation, of a pointer, I'd expect it to mean "it is legal to convert this pointer into a reference".

these functions take references

Are we talking about the same functions? <*const [T]>::get_unchecked takes a raw pointer by value, and passes it on to SliceIndex::get_unchecked:

rust/library/core/src/ptr/const_ptr.rs

Line 1693 in 0ea7ddc

unsafe { index.get_unchecked(self) }

which adds the index and returns the pointer:

rust/library/core/src/slice/index.rs

Line 238 in 0ea7ddc

slice.as_ptr().add(self)

I don't see a single reference being accepted, constructed or manipulated at any stage here.

[RalfJung]

Are we talking about the same functions?

Sorry, I thought you meant slice::get_unchecked. I didn't realize which issue this is in 😂

That's surprising; in Rust API documentation, of a pointer, I'd expect it to mean "it is legal to convert this pointer into a reference".

No, that's not what it means. It's not "referenceable" after all, it's "dereferenceable", referring to the dereference operator *. Though the semantics of that got relaxed recently so dereferencing can be allowed on pointers that are not dereferenceable... maybe we need a new term.^^

[RalfJung]

I think this has been sitting around long enough, clearly a fix for #73987 will not magically materialize. And even with #73987 unresolved, as long as we don't let users write arbitrary-self methods on raw pointers, we can still change the autoref behavior later to use raw pointers when that is enough. So while these methods might have a footgun, they are still better than the status quo.

So I propose we move towards stabilizing the raw pointer part of this. (I am less sure about the NonNull part.)

@rust-lang/types are you aware of anything that would block stabilizing a function that uses a raw slice pointer self type? (AFAIK these would be the first stable functions that do that.)

[lcnr]

#71146 (comment) applies to this feature as well

[the8472]

This happened to come up in a libs-api meeting. We want to note that stabilization of this feature will likely be blocked on the arbitrary_self_types feature (rust-lang/rfcs#3519) since that will allow moving these methods to

impl [T] {
   fn as_mut_ptr(*const Self) -> *mut T 
}

[RalfJung]

Yeah I was about to ask, given @joshtriplett's reply here, what the implications are for this feature.

However, I think we should really offer some way to work with raw slice pointers. The current situation is pretty unfortunate and nothing has improved for many years. If we can't stabilize this as-is, does @rust-lang/libs-api have any alternatives we could pursue?

[the8472]

Well, part of the issue is that similar methods have been proposed for array pointers. We'd either have to duplicate them or see what kind of deref-like coercions self types will provide.

[RalfJung]

or see what kind of deref-like coercions self types will provide

So far, Rust has largely avoided coercions on raw pointers. That's pretty orthogonal to arbitrary-self-types, so I don't expect rust-lang/rfcs#3519 to change that. So another T-lang RFC would be needed to have such coercions on raw ptr methods I think.

[Rua]

Is there any interest in implementing a method that does perform bounds checking, while still operating on pointers rather than references?

[RalfJung]

That would make sense to me, yeah. I am not in t-libs-api though, so this should probably go through an ACP.

[Rua]

I ran into an issue with this. get_unchecked is implemented for pointers to slices, but not for pointers to fixed-size arrays. We probably want to implement it for arrays too?

[yotamofek]

I ran into an issue with this. get_unchecked is implemented for pointers to slices, but not for pointers to fixed-size arrays. We probably want to implement it for arrays too?

Would love to have get_unchecked(_mut) on pointers to arrays, they could maybe live under the array_ptr_get feature. Two notes: 1) you could go through the as_(mut_)slice methods from that feature; 2) if implemented, stabilization of methods on pointers to arrays will likely be blocked on the arbitrary_self_type features, as discussed in the ACP.