Add Vec visualization to understand capacity #76066

pickfire · 2020-08-29T11:29:26Z

Visualize vector while differentiating between stack and heap.

Inspired by cheats.rs, this is probably the first place beginner go, they could understand stack and stack, length and capacity with this. Not sure if adding this means we should add to other places too.

r? @rust-lang/rustdoc

Visualize vector while differentiating between stack and heap.

pickfire · 2020-08-29T11:44:11Z

cc @jyn514 @GuillaumeGomez Do you think it look nice?

Weird it didn't assign anyone
r? @jyn514

library/alloc/src/vec.rs

GuillaumeGomez · 2020-08-29T11:59:47Z

cc @rust-lang/docs

leonardo-m · 2020-08-29T12:02:08Z

See also:
#42412

pickfire · 2020-08-29T12:12:38Z

cc @rust-lang/docs

Oh, it doesn't work for me. @GuillaumeGomez have link but I don't have. T_T

library/alloc/src/vec.rs

jyn514 · 2020-08-31T13:58:33Z

Visualize vector while differentiating between stack and heap.

Inspired by cheats.rs, this is probably the first place beginner go, they could understand stack and stack, length and capacity with this. Not sure if adding this means we should add to other places too.

Hmm ... maybe this belongs in the book instead, in the section on stack and heap? https://doc.rust-lang.org/book/ch04-01-what-is-ownership.html

Anyways I don't think I'm the best person to review this, maybe @steveklabnik would be interested?

Co-authored-by: Joshua Nelson <[email protected]>

pickfire · 2020-08-31T14:07:09Z

We could put it there but we also mention about Vec here which I don't think we should put it specifically in the book.

steveklabnik · 2020-08-31T17:30:46Z

I am still worried that this exposes details that are not stable.

the8472 · 2020-08-31T17:54:23Z

Most fundamentally, Vec is and always will be a (pointer, capacity, length) triplet. No more, no less. The order of these fields is completely unspecified

So the ascii chart should make clear that it is only an example of a possible implementation, not the real one. And it probably should show an incorrect layout just to make sure that anyone who even tries to rely on it will immediately run into a problem.

pickfire · 2020-09-01T01:31:00Z

I am still worried that this exposes details that are not stable.

Yes, it should mention that ABI is not stable yet.

So the ascii chart should make clear that it is only an example of a possible implementation, not the real one. And it probably should show an incorrect layout just to make sure that anyone who even tries to rely on it will immediately run into a problem.

Incorrect layout?

library/alloc/src/vec.rs

the8472 · 2020-09-01T09:07:43Z

So the ascii chart should make clear that it is only an example of a possible implementation, not the real one. And it probably should show an incorrect layout just to make sure that anyone who even tries to rely on it will immediately run into a problem.

Incorrect layout?

It should give an order of fields that does not match the current implementation.

pickfire · 2020-09-01T17:16:11Z

For example?

the8472 · 2020-09-01T17:25:46Z

E.g. put the pointer last. In the current implementation (assuming no reordering by the compiler) it is the first element of the struct.

pickfire · 2020-09-02T00:11:01Z

@the8472 I already mentioned that ABI is not stable so layout is not guaranteed.

library/alloc/src/vec.rs

LukasKalbertodt · 2020-10-26T22:14:50Z

library/alloc/src/vec.rs

@@ -194,6 +194,22 @@ use crate::raw_vec::RawVec;
 /// can be slow. For this reason, it is recommended to use [`Vec::with_capacity`]
 /// whenever possible to specify how big the vector is expected to get.
 ///
+/// In summary, a vector containing 1, 2 with capacity 4 can be visualized as:
+/// ```text
+/// Stack      ptr   capacity    len


Maybe just remove "Stack". We simply don't know where those three values live. We just know that the lower box lives on the heap.

Co-authored-by: Lukas Kalbertodt <[email protected]>

crlf0710 · 2020-11-13T12:48:18Z

Triage: What's the current status on this?

jyn514

@crlf0710 this needs a new reviewer, I think Lukas has left the libs team.

jyn514 · 2020-11-13T13:37:02Z

library/alloc/src/vec.rs

@@ -194,6 +194,22 @@ use crate::raw_vec::RawVec;
 /// can be slow. For this reason, it is recommended to use [`Vec::with_capacity`]
 /// whenever possible to specify how big the vector is expected to get.
 ///
+/// In summary, a vector containing 1, 2 with capacity 4 can be visualized as:
+/// ```text
+/// Stack      ptr   capacity    len


Try 'Layout' instead, maybe?

pickfire · 2020-11-14T05:54:05Z

Try 'Layout' instead, maybe?

Lucas suggested something too, maybe we should wait for the new reviewer to say. Feel like this is something that can be bikeshed on, I personally would use 'Heap'.

JohnCSimon · 2020-11-30T16:28:00Z

ping from triage: @pickfire can you please resolve the merge conflict? thank you.
Also I've commented in Triage that this PR is still waiting on a new reviewer.

library/alloc/src/vec.rs

pickfire · 2020-12-01T11:32:18Z

@JohnCSimon Fix conflict

jyn514 · 2020-12-01T13:07:02Z

CI is now failing unfortunately, you probably need to fix tidy. Also it would be nice to squash the commits.

Dylan-DPC-zz · 2020-12-02T15:03:15Z

r? @m-ou-se

m-ou-se

Looks like a useful addition!

A bit later in the text, in the "Guarantees" section, this layout is explained in words. Maybe this visualisation should be put there instead? (Right after the sentence ending in logically uninitialized, contiguous elements.)

(It's a bit of a shame that this has to use +---+ to draw the boxes, but none of the alternatives (an svg, raw html, a markdown table, or unicode box drawing characters) work well. I couldn't find something better that is usable and reliable in both the browser and terminal.)

m-ou-se · 2020-12-02T15:22:25Z

library/alloc/src/vec.rs

@@ -194,6 +194,22 @@ use crate::raw_vec::RawVec;
 /// can be slow. For this reason, it is recommended to use [`Vec::with_capacity`]
 /// whenever possible to specify how big the vector is expected to get.
 ///
+/// In summary, a vector containing 1, 2 with capacity 4 can be visualized as:
+/// ```text
+/// Stack      ptr   capacity    len


Maybe something like "The Vec object itself" and "The data on the heap".

m-ou-se · 2020-12-02T15:23:42Z

library/alloc/src/vec.rs

+/// ```text
+/// Stack      ptr   capacity    len
+///        +--------+--------+--------+
+///        | 0x0123 |      4 |      2 |


into_raw_parts and from_raw_parts use (ptr, length, capacity), not (ptr, capacity, length). It might be good to use the same order here.

This is intentionally different, see #76066 (comment).

@jyn514 That comment suggests to use a different layout than the real layout. The 'real' layout is (RawVec, len), which is effectively a (ptr, cap, len). Changing it to (ptr, len, cap) as I suggest does make it different from the actual layout, just like the8472 asked for.

Weird, I didn't notice this, I thought I used len followed by capacity.

m-ou-se · 2020-12-02T15:26:54Z

library/alloc/src/vec.rs

+/// - Note: the ABI is not stable and `Vec` makes no guarantees about its memory 
+///   layout (including the order of fields). See [the section about 
+///   guarantees](#guarantees).


The layout of the part on the heap is pretty strict. Let's make clear it's only the Vec object itself (the (ptr, size, cap) part) that's unspecified. A mention of into_raw_parts/from_raw_parts and/or .as_ptr()/.len()/.capacity() would also be nice here, as those are the correct way of interacting with these fields.

Maybe you can suggest a change here?

pickfire · 2020-12-03T02:39:54Z

CI is now failing unfortunately, you probably need to fix tidy. Also it would be nice to squash the commits.

Yes, I am thinking of creating a new PR for this.

Maybe something like "The Vec object itself" and "The data on the heap".

What?

oli-obk · 2020-12-03T11:41:31Z

CI is now failing unfortunately, you probably need to fix tidy. Also it would be nice to squash the commits.

Yes, I am thinking of creating a new PR for this.

Note that you can force push to the same branch after sqashing in order to reuse the same PR, it's not technically necessary to start a new PR. Opening a new PR makes it harder to follow the discussion as one needs to jump between PRs now and potentially requires verbatim copying comments from the old PR.

m-ou-se · 2020-12-03T12:48:47Z

What?

Excuse me?

Too long.

There's plenty of space to put more words, if that makes things clearer.

"The data on the heap" is incorrect, since it could be in the stack too.

The elements of the vector (the second part of your diagram) are on the heap. Only the Vec object itself can be on the stack.

I am thinking of creating a new PR for this.

Next time please just force-push to the same branch if you need to rebase or squash, otherwise it's hard to keep track of the review comments.

pickfire · 2020-12-03T17:51:47Z

Note that you can force push to the same branch after sqashing in order to reuse the same PR, it's not technically necessary to start a new PR. Opening a new PR makes it harder to follow the discussion as one needs to jump between PRs now and potentially requires verbatim copying comments from the old PR.

Yes, but it is more troublesome, the branch name is also weird.

Next time please just force-push to the same branch if you need to rebase or squash, otherwise it's hard to keep track of the review comments.

No, I still want to see the condition first, in this case it would be easier to create a new pull request. But if other case where I already have a branch I will force-push.

lcnr · 2020-12-03T18:15:28Z

but it is more troublesome

To locally change a pull request you opened using the github api you can use

git fetch remote-name
git checkout remote-name/patch-15 -b patch-15

Splitting the conversation over two pull requests, especially if there are some open threads on the old one, has a noticeable negative impact on the review process. So while it might be a bit easier for you to open a new PR, I strongly advise you to use the commands above in the future and keep everything in one place. By doing so you prevent potential oversights without wasting valuable time of both the reviewers and yourself.

Visualize vector while differentiating between stack and heap. Inspired by cheats.rs, as this is probably the first place beginner go, they could understand stack and heap, length and capacity with this. Not sure if adding this means we should add to other places too. Superseeds rust-lang#76066

Add Vec visualization to understand capacity Visualize vector while differentiating between stack and heap. Inspired by cheats.rs, as this is probably the first place beginner go, they could understand stack and heap, length and capacity with this. Not sure if adding this means we should add to other places too. Superseeds rust-lang#76066 r? `@m-ou-se` cc `@the8472` I put back the order of the fields as it feels weird, the note already explains that the order of fields is not guaranteed

Add Vec visualization to understand capacity

9c1415e

Visualize vector while differentiating between stack and heap.

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 29, 2020

rust-highfive assigned jyn514 Aug 29, 2020

GuillaumeGomez reviewed Aug 29, 2020

View reviewed changes

library/alloc/src/vec.rs Outdated Show resolved Hide resolved

Center align ptr and len box vec doc

ea337c6

jyn514 added A-docs Area: documentation for any part of the project, including the compiler, standard library, and tools T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. labels Aug 29, 2020

the8472 reviewed Aug 29, 2020

View reviewed changes

library/alloc/src/vec.rs Show resolved Hide resolved

pickfire added 2 commits August 30, 2020 22:37

Link uninitialized memory to MaybeUninit

f2bd46b

Vec visualization separate description

53af7b8

jyn514 reviewed Aug 31, 2020

View reviewed changes

library/alloc/src/vec.rs Outdated Show resolved Hide resolved

Link Vec visualization to core::mem::MaybeUninit

a919516

Co-authored-by: Joshua Nelson <[email protected]>

jyn514 mentioned this pull request Aug 31, 2020

Stabilize hint::spin_loop #76097

Merged

jyn514 assigned steveklabnik and unassigned jyn514 Aug 31, 2020

pickfire commented Sep 1, 2020

View reviewed changes

library/alloc/src/vec.rs Outdated Show resolved Hide resolved

Vec visualization mention unstable ABI

55ad7f2

rust-highfive assigned LukasKalbertodt Oct 15, 2020

LukasKalbertodt suggested changes Oct 26, 2020

View reviewed changes

Use alphabetical example in vec visualization

79918c1

Co-authored-by: Lukas Kalbertodt <[email protected]>

jyn514 reviewed Nov 13, 2020

View reviewed changes

JohnCSimon added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 30, 2020

pickfire commented Dec 1, 2020

View reviewed changes

library/alloc/src/vec.rs Show resolved Hide resolved

pickfire added 2 commits December 1, 2020 19:29

Fix vec doc merge conflict

62ba5b1

Merge branch 'master' into patch-15

702662e

rust-highfive assigned m-ou-se and unassigned LukasKalbertodt Dec 2, 2020

m-ou-se reviewed Dec 2, 2020

View reviewed changes

pickfire mentioned this pull request Dec 3, 2020

Add Vec visualization to understand capacity #79655

Merged

pickfire closed this Dec 3, 2020

pickfire deleted the patch-15 branch December 3, 2020 03:00

Add Vec visualization to understand capacity #76066

Add Vec visualization to understand capacity #76066

Conversation

pickfire commented Aug 29, 2020

pickfire commented Aug 29, 2020

GuillaumeGomez commented Aug 29, 2020

leonardo-m commented Aug 29, 2020

pickfire commented Aug 29, 2020 • edited Loading

jyn514 commented Aug 31, 2020

pickfire commented Aug 31, 2020

steveklabnik commented Aug 31, 2020

the8472 commented Aug 31, 2020

pickfire commented Sep 1, 2020

the8472 commented Sep 1, 2020

pickfire commented Sep 1, 2020

the8472 commented Sep 1, 2020

pickfire commented Sep 2, 2020

Choose a reason for hiding this comment

crlf0710 commented Nov 13, 2020

jyn514 left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pickfire commented Nov 14, 2020

JohnCSimon commented Nov 30, 2020

pickfire commented Dec 1, 2020

jyn514 commented Dec 1, 2020

Dylan-DPC-zz commented Dec 2, 2020

m-ou-se left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pickfire commented Dec 3, 2020

oli-obk commented Dec 3, 2020

m-ou-se commented Dec 3, 2020

pickfire commented Dec 3, 2020

lcnr commented Dec 3, 2020

pickfire commented Aug 29, 2020 •

edited

Loading

jyn514 left a comment •

edited

Loading