#[repr(Interoperable_2024)]

[ckaran]

Proposal

There is an ongoing discussion about creating a new stable ABI layout. The idea is not to stabilize #[repr(Rust)], but to instead create a new layout of the form #[repr(Interoperable_2024)], where the suffix is the edition year to permit future updates if/when needed. The idea is that the stabilized ABI layout will be widely known and shared between different compiler vendors, chip vendors, etc., so that instead of having to use #[repr(C)] (which is platform dependent), we can have a truly independent layout whose form is well-specified and allows code developed by different vendors to interoperate with one another.

Summary and problem statement

As has been noted elsewhere, C isn't just a programming language, it's been beaten beyond recognition into a communications protocol between languages. For Rust to interoperate with Swift, we have to use something like #[repr(C)] to beat our data into a form that both languages recognize. The issue is that C was never designed with this in mind. On some platforms the ABI is well defined, but on others, it's basically whatever the first compiler decided it was going to be. This makes interoperability a nightmare as you have to read through the compiler's source to understand what the ABI is, and that's assuming you have access to the source code. If you don't, then you have to do some reverse engineering, which is not guaranteed to catch all corner cases.

Motivation, use-cases, and solution sketches

I want to define an ABI spec that can be shared between different platforms and languages. @Kornel gave some examples in a sibling discussion discussing how a stabilized ABI could help library authors.

I don't have any solid solution, or even solution sketches. I believe that a working group needs to be set up that has the following goals it wants to accomplish:

1. Interoperable_2024 - A fully stabilized representation that is an alternative to repr(C). This will not replace repr(C), it sits beside it. Because this is fully specified on all platforms, library authors can target it for stability. Vendors of other rust compilers can interoperate with rustc. This could be particularly useful for things like rust-gpu, where rust-gpu might compile a chunk of code for the GPU, rustc compiles for the CPU, and the linker can still make the two work with each other. The limitation is that only a subset of Rust can cross the ABI because this is a minimum viable product that is an alternative to repr(C), not an extension of it. Get buy-in from other vendors and other languages.
2. Interoperable_2027 - Extends Interoperable_2024 with more modern features (TBD what they are). These should be useful beyond Rust. Get more buy-in from other vendors and other languages. At this point the working group is operating like a normative body, getting as many groups to cooperate as is possible.
3. Further work as needed.

Links and related work

The initial discussion started in the middle of https://internals.rust-lang.org/t/discussion-editions-in-rust-gcc-and-other-rust-compilers/16608, which was hijacked from the the original topic there.

At several people's suggestion, I created a new topic at https://internals.rust-lang.org/t/repr-interoperable-2024/16763.

@Kornel Linked to it in the library discussion at https://internals.rust-lang.org/t/for-a-library-author-how-maintenance-of-a-stable-abi-would-look-like/16769.

https://users.rust-lang.org/t/modular-abi-for-rust/42590
https://internals.rust-lang.org/t/a-stable-modular-abi-for-rust/12347
https://internals.rust-lang.org/t/when-is-the-abi-stable/10420/2
https://rust-lang.github.io/unsafe-code-guidelines/layout/structs-and-tuples.html
https://internals.rust-lang.org/t/dynamically-linking-rust-crates-to-rust-crates/10369
https://crates.io/crates/abi_stable
https://internals.rust-lang.org/t/making-repr-un-enum-ffi-safe-pre-rfc/5325

Other (sort of) related posts:

https://internals.rust-lang.org/t/add-repr-inherit-type/16626
https://internals.rust-lang.org/t/question-about-c-abi-stability/12449
https://internals.rust-lang.org/t/pre-rfc-repr-c-for-traits/12598

Initial people involved

I'm not sure how will own this in the end (I can push on it, but I don't have sufficient technical expertise to lead it), but I think all of the people that have posted to either https://internals.rust-lang.org/t/repr-interoperable-2024/16763 or https://internals.rust-lang.org/t/for-a-library-author-how-maintenance-of-a-stable-abi-would-look-like/16769 would be interested.

What happens now?

This issue is part of the lang-team initiative process. Once this issue is filed, a Zulip topic will be opened for discussion, and the lang-team will review open proposals in its weekly triage meetings. You should receive feedback within a week or two.

This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

[rustbot]

This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

[programmerjake]

hope you don't mind, leaving a link to my proposed struct layout algorithm so it doesn't get forgotten:
https://rust-lang.zulipchat.com/#narrow/stream/243200-t-lang.2Fmajor-changes/topic/.23.5Brepr.28Interoperable_2024.29.5D.20lang-team.23165/near/296078795
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=03eb7d67bd8883208c71afceeee32324

[ckaran]

@programmerjake Do you mind if I add your comments/links to https://internals.rust-lang.org/t/repr-interoperable-2024/16763? I wish we had a place that was the place for summarizing/taking notes, but until and unless Rust decides it wants to take this under its umbrella, IRLO is where I want to keep them so others can come up to speed quickly.

That said, while I still enjoy the idea from an academic perspective¹, I don't think it's appropriate for the interoperable spec, at least at this time. The issue is that we need to get a lot of different groups onboard with the idea of even writing a new interoperable spec, let alone making changes to it. My personal goal for 2024 is actually much, much simpler: first, let's partition the current C ABI into the 'safe' and 'unsafe' parts. These will have analogous meanings to 'safe' and 'unsafe' in Rust; if you use the 'safe' ABI, then it is possible to mechanically enforce some set of guarantees (where we need to decide what guarantees we want to enforce first). The 'unsafe' part of the ABI is unsafe for a reason; the programmers that use it are assumed to be using them correctly, and they can't be mechanically checked. @joshtriplett would you be onboard with this simpler goal for 2024?

Once we have enough people and representatives of different languages/compilers on board², and a decent process for making decisions, we can start pushing forwards on growing the spec in new directions.

Footnotes

1. Because, let's face, this idea is cool! ↩

2. No, I do not have a definition of 'enough', we'll have to work it out as we go along. ↩

[nikomatsakis]

I'm going to close this issue as we've adopted a new process and are no longer using major-change proposals. I think that the right next step here would be to create a lang-team experiment, and I would recommend discussing that with @joshtriplett.

Thanks for the proposal!

[programmerjake]

@programmerjake Do you mind if I add your comments/links to https://internals.rust-lang.org/t/repr-interoperable-2024/16763? I wish we had a place that was the place for summarizing/taking notes, but until and unless Rust decides it wants to take this under its umbrella, IRLO is where I want to keep them so others can come up to speed quickly.

Sure, go ahead. Sorry I took soo long to reply, I didn't notice your message till now.

[ckaran]

Thank you!

[joshtriplett]

I put together a proposal for a related experimental feature gate at rust-lang/rust#105586. Providing a cross-reference so that folks here can look at that.

I don't think that proposal is exactly what everyone in this thread is looking for; notably, that proposal is for a strict superset of the C ABI, rather than only the safe bits of the C ABI. However, I want to make sure people can find that thread and participate if interested.

[ckaran]

@joshtriplett thank you! I wish I had more time to work on this, but work has really picked up in the past couple of months, and it looks like it won't slow down for at least the next several months. I'll help if and where I can, but I definitely won't be able to do as much I would like to.