Compilation error: can't find crate for std; the x86_64-linux-kernel target may not be installed
#27
Labels
• docs
Related to `Documentation/rust/`, `samples/`, generated docs, doctests, typos...
Comments
|
This should resolve itself, when #25 gets resolved. |
kloenk
added
the
• docs
|
@wecing It may work now since we don't depend on |
|
@ojeda Thank you! Unfortunately it still doesn't work for me: Since the previous error message I saw was about rust Not sure what was causing that failure, but it probably doesn't matter anymore. |
Oh, not good. But without that change it also does not work. So no idea on how to fix it from my side at this time. I changed it so, it detects if it's a module or a builtin. |
ojeda
added a commit
that referenced
this issue
Dec 9, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure. Way less files around!
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Offline builds, always; i.e. there is no "online compilation"
anymore (fixes #17).
- No more interleaved Cargo output (fixes #29).
- One less nightly dependency (Cargo's `build-std`); since now we manage
the cross-compilation ourselves (should fix #27).
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
following what the user selected for C (no Cargo profiles).
- Added Kconfig menu for tweaking `rustc` options, like overflow checks.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.9 while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is one more nightly feature used (the new Rust mangling scheme),
but we know that one will be stable (and the default one, later on).
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
There are a few TODOs that we can improve later if we agree on this:
- Kbuild:
+ Actually use the `*.d` files.
+ Complete `make clean`.
+ Support single-object compilation.
+ Pass `objtool` to make the ORC unwinder work.
+ Echo the building of the rust/* libraries and the bindgen call.
- Figure out how to pick symbols to export automatically from
Rust code instead of managing the list by hand.
Perhaps we could use a no-op macro on the Rust code, which is then
parse by a script to pick up the symbols:
pub fn foo() {}
export_symbol!(foo);
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Dec 9, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure. Way less files around!
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Offline builds, always; i.e. there is no "online compilation"
anymore (fixes #17).
- No more interleaved Cargo output (fixes #29).
- One less nightly dependency (Cargo's `build-std`); since now we manage
the cross-compilation ourselves (should fix #27).
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
following what the user selected for C (no Cargo profiles).
- Added Kconfig menu for tweaking `rustc` options, like overflow checks.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.9 while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is one more nightly feature used (the new Rust mangling scheme),
but we know that one will be stable (and the default one, later on).
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
There are a few TODOs that we can improve later if we agree on this:
- Kbuild:
+ Actually use the `*.d` files.
+ Complete `make clean`.
+ Support single-object compilation.
+ Pass `objtool` to make the ORC unwinder work.
+ Echo the building of the rust/* libraries and the bindgen call.
- Figure out how to pick symbols to export automatically from
Rust code instead of managing the list by hand.
Perhaps we could use a no-op macro on the Rust code, which is then
parse by a script to pick up the symbols:
pub fn foo() {}
export_symbol!(foo);
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Dec 9, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure. Way less files around!
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Offline builds, always; i.e. there is no "online compilation"
anymore (fixes #17).
- No more interleaved Cargo output (fixes #29).
- One less nightly dependency (Cargo's `build-std`); since now we manage
the cross-compilation ourselves (should fix #27).
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
following what the user selected for C (no Cargo profiles).
- Added Kconfig menu for tweaking `rustc` options, like overflow checks.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.9 while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is one more nightly feature used (the new Rust mangling scheme),
but we know that one will be stable (and the default one, later on).
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
There are a few TODOs that we can improve later if we agree on this:
- Kbuild:
+ Actually use the `*.d` files.
+ Complete `make clean`.
+ Support single-object compilation.
+ Pass `objtool` to make the ORC unwinder work.
+ Echo the building of the rust/* libraries and the bindgen call.
- Figure out how to pick symbols to export automatically from
Rust code instead of managing the list by hand.
Perhaps we could use a no-op macro on the Rust code, which is then
parse by a script to pick up the symbols:
pub fn foo() {}
export_symbol!(foo);
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Dec 9, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure. Way less files around!
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Offline builds, always; i.e. there is no "online compilation"
anymore (fixes #17).
- No more interleaved Cargo output (fixes #29).
- One less nightly dependency (Cargo's `build-std`); since now we manage
the cross-compilation ourselves (should fix #27).
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
following what the user selected for C (no Cargo profiles).
- Added Kconfig menu for tweaking `rustc` options, like overflow checks.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.9 while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is one more nightly feature used (the new Rust mangling scheme),
but we know that one will be stable (and the default one, later on).
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
There are a few TODOs that we can improve later if we agree on this:
- Kbuild:
+ Actually use the `*.d` files.
+ Complete `make clean`.
+ Support single-object compilation.
+ Pass `objtool` to make the ORC unwinder work.
+ Echo the building of the rust/* libraries and the bindgen call.
- Figure out how to pick symbols to export automatically from
Rust code instead of managing the list by hand.
Perhaps we could use a no-op macro on the Rust code, which is then
parse by a script to pick up the symbols:
pub fn foo() {}
export_symbol!(foo);
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Dec 12, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, etc. all trigger
recompilation of the proper things.
+ Works as expected with parallel support (`-j`).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No more interleaved Cargo output (fixes #29).
- One less nightly dependency (Cargo's `build-std`); since now we manage
the cross-compilation ourselves (should fix #27).
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
following what the user selected for C (no Cargo profiles).
- Added Kconfig menu for tweaking `rustc` options, like overflow checks.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.9 while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is one more nightly feature used (the new Rust mangling scheme),
but we know that one will be stable (and the default one, later on).
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Dec 16, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
following what the user selected for C (no Cargo profiles).
- Added Kconfig menu for tweaking relevant `rustc` options, like overflow
checks, debug assertions, optimization level, etc.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.9 while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
pushed a commit
that referenced
this issue
Dec 16, 2020
If cm_create_timewait_info() fails, the timewait_info pointer will contain an error value and will be used in cm_remove_remote() later. general protection fault, probably for non-canonical address 0xdffffc0000000024: 0000 [#1] SMP KASAN PTI KASAN: null-ptr-deref in range [0×0000000000000120-0×0000000000000127] CPU: 2 PID: 12446 Comm: syz-executor.3 Not tainted 5.10.0-rc5-5d4c0742a60e #27 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014 RIP: 0010:cm_remove_remote.isra.0+0x24/0×170 drivers/infiniband/core/cm.c:978 Code: 84 00 00 00 00 00 41 54 55 53 48 89 fb 48 8d ab 2d 01 00 00 e8 7d bf 4b fe 48 89 ea 48 b8 00 00 00 00 00 fc ff df 48 c1 ea 03 <0f> b6 04 02 48 89 ea 83 e2 07 38 d0 7f 08 84 c0 0f 85 fc 00 00 00 RSP: 0018:ffff888013127918 EFLAGS: 00010006 RAX: dffffc0000000000 RBX: fffffffffffffff4 RCX: ffffc9000a18b000 RDX: 0000000000000024 RSI: ffffffff82edc573 RDI: fffffffffffffff4 RBP: 0000000000000121 R08: 0000000000000001 R09: ffffed1002624f1d R10: 0000000000000003 R11: ffffed1002624f1c R12: ffff888107760c70 R13: ffff888107760c40 R14: fffffffffffffff4 R15: ffff888107760c9c FS: 00007fe1ffcc1700(0000) GS:ffff88811a600000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000001b2ff21000 CR3: 000000010f504001 CR4: 0000000000370ee0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: cm_destroy_id+0x189/0×15b0 drivers/infiniband/core/cm.c:1155 cma_connect_ib drivers/infiniband/core/cma.c:4029 [inline] rdma_connect_locked+0x1100/0×17c0 drivers/infiniband/core/cma.c:4107 rdma_connect+0x2a/0×40 drivers/infiniband/core/cma.c:4140 ucma_connect+0x277/0×340 drivers/infiniband/core/ucma.c:1069 ucma_write+0x236/0×2f0 drivers/infiniband/core/ucma.c:1724 vfs_write+0x220/0×830 fs/read_write.c:603 ksys_write+0x1df/0×240 fs/read_write.c:658 do_syscall_64+0x33/0×40 arch/x86/entry/common.c:46 entry_SYSCALL_64_after_hwframe+0x44/0xa9 Fixes: a977049 ("[PATCH] IB: Add the kernel CM implementation") Link: https://lore.kernel.org/r/[email protected] Reviewed-by: Maor Gottlieb <[email protected]> Reported-by: Amit Matityahu <[email protected]> Signed-off-by: Leon Romanovsky <[email protected]> Signed-off-by: Jason Gunthorpe <[email protected]>
ojeda
added a commit
that referenced
this issue
Dec 16, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
following what the user selected for C (no Cargo profiles).
- Added Kconfig menu for tweaking relevant `rustc` options, like overflow
checks, debug assertions, optimization level, etc.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.9 while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Dec 20, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels.
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks, debug assertions, etc.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Updated with `toolchain` matrix support: now we test building
with GCC, Clang or a full LLVM toolchain.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK. Also enable
`-C opt-level=0` to test that such an extreme works and also
to see how much bloated everything becomes.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.10 and `LLVM=1` while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Dec 20, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels.
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks, debug assertions, etc.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Updated with `toolchain` matrix support: now we test building
with GCC, Clang or a full LLVM toolchain.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK. Also enable
`-C opt-level=0` to test that such an extreme works and also
to see how much bloated everything becomes.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.10 and `LLVM=1` while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Dec 20, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels.
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks, debug assertions, etc.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Updated with `toolchain` matrix support: now we test building
with GCC, Clang or a full LLVM toolchain.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK. Also enable
`-C opt-level=0` to test that such an extreme works and also
to see how much bloated everything becomes.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.10 and `LLVM=1` while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Dec 20, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels.
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks, debug assertions, etc.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Updated with `toolchain` matrix support: now we test building
with GCC, Clang or a full LLVM toolchain.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK. Also enable
`-C opt-level=0` to test that such an extreme works and also
to see how much bloated everything becomes.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.10 and `LLVM=1` while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Dec 20, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels.
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks, debug assertions, etc.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Updated with `toolchain` matrix support: now we test building
with GCC, Clang or a full LLVM toolchain.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK. Also enable
`-C opt-level=0` to test that such an extreme works and also
to see how much bloated everything becomes.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.10 and `LLVM=1` while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Dec 20, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels.
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks, debug assertions, etc.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Updated with `toolchain` matrix support: now we test building
with GCC, Clang or a full LLVM toolchain.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK. Also enable
`-C opt-level=0` to test that such an extreme works and also
to see how much bloated everything becomes.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.10 and `LLVM=1` while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Dec 20, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels.
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks, debug assertions, etc.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Updated the quick start guide.
- Updated CI `.config`s:
+ Add the new options and test with 2 builtins and 2 loadables.
At the same time, remove the matrix test for builtin/loadable.
+ Updated with `toolchain` matrix support: now we test building
with GCC, Clang or a full LLVM toolchain.
+ Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
that mimic more what a developer would have. Running the CI
will be slightly slower, but should be OK. Also enable
`-C opt-level=0` to test that such an extreme works and also
to see how much bloated everything becomes.
+ Release: disabled `EXPERT` and changed a few things to make it
look more like a normal configuration.
+ Also update both configs to v5.10 and `LLVM=1` while I was at it.
(I could have split a few of these ones off into another PR,
but anyway it is for the CI only and I had already done it).
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Dec 28, 2020
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels:
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Support for `arm64` architecture:
+ Added to the CI: BusyBox is cross-compiled on the fly (increased
timeout a bit to match).
+ Requires weakening of a few compiler builtins and adding
`copy_{from,to}_user` helpers.
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks and debug assertions.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Do not export any helpers' symbols.
- Updated the quick start guide.
- Updated CI:
+ Now we always test with 2 builtins and 2 loadables Rust example
drivers, removing the matrix test for builtin/loadable.
+ Added `toolchain` to matrix: now we test building with GCC,
Clang or a full LLVM toolchain.
+ Added `arch` to matrix: now we test both arm64 and x86_64.
+ Added `rustc` to matrix: now we test with a very recent nightly
as well.
+ Only build `output == build` once to reduce the number
of combinations.
+ Debug x86_64 config: more things enabled (debuginfo, kgdb,
unit testing, etc.) that mimic more what a developer would have.
Running the CI will be slightly slower, but should be OK.
Also enable `-C opt-level=0` to test that such an extreme works
and also to see how much bloated everything becomes.
+ Release x86_64 config: disabled `EXPERT` and changed a few things
to make it look more like a normal desktop configuration,
although it is still pretty minimal.
+ The configs for arm64 are `EXPERT` and `EMBEDDED` ones,
very minimal, for the particular CPU we are simulating.
+ Update configs to v5.10.
+ Use `$GITHUB_ENV` to simplify.
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
- The hack to get the proper flags for bindgen on GCC builds.
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Jan 2, 2021
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels:
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Support for `arm64` architecture:
+ Added to the CI: BusyBox is cross-compiled on the fly (increased
timeout a bit to match).
+ Requires weakening of a few compiler builtins and adding
`copy_{from,to}_user` helpers.
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks and debug assertions.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Do not export any helpers' symbols.
- Updated the quick start guide.
- Updated CI:
+ Now we always test with 2 builtins and 2 loadables Rust example
drivers, removing the matrix test for builtin/loadable.
+ Added `toolchain` to matrix: now we test building with GCC,
Clang or a full LLVM toolchain.
+ Added `arch` to matrix: now we test both arm64 and x86_64.
+ Added `rustc` to matrix: now we test with a very recent nightly
as well.
+ Only build `output == build` once to reduce the number
of combinations.
+ Debug x86_64 config: more things enabled (debuginfo, kgdb,
unit testing, etc.) that mimic more what a developer would have.
Running the CI will be slightly slower, but should be OK.
Also enable `-C opt-level=0` to test that such an extreme works
and also to see how much bloated everything becomes.
+ Release x86_64 config: disabled `EXPERT` and changed a few things
to make it look more like a normal desktop configuration,
although it is still pretty minimal.
+ The configs for arm64 are `EXPERT` and `EMBEDDED` ones,
very minimal, for the particular CPU we are simulating.
+ Update configs to v5.10.
+ Use `$GITHUB_ENV` to simplify.
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
- The hack to get the proper flags for bindgen on GCC builds.
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Jan 3, 2021
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels:
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Support for `arm64` architecture:
+ Added to the CI: BusyBox is cross-compiled on the fly (increased
timeout a bit to match).
+ Requires weakening of a few compiler builtins and adding
`copy_{from,to}_user` helpers.
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks and debug assertions.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Do not export any helpers' symbols.
- Updated the quick start guide.
- Updated CI:
+ Now we always test with 2 builtins and 2 loadables Rust example
drivers, removing the matrix test for builtin/loadable.
+ Added `toolchain` to matrix: now we test building with GCC,
Clang or a full LLVM toolchain.
+ Added `arch` to matrix: now we test both arm64 and x86_64.
+ Added `rustc` to matrix: now we test with a very recent nightly
as well.
+ Only build `output == build` once to reduce the number
of combinations.
+ Debug x86_64 config: more things enabled (debuginfo, kgdb,
unit testing, etc.) that mimic more what a developer would have.
Running the CI will be slightly slower, but should be OK.
Also enable `-C opt-level=0` to test that such an extreme works
and also to see how much bloated everything becomes.
+ Release x86_64 config: disabled `EXPERT` and changed a few things
to make it look more like a normal desktop configuration,
although it is still pretty minimal.
+ The configs for arm64 are `EXPERT` and `EMBEDDED` ones,
very minimal, for the particular CPU we are simulating.
+ Update configs to v5.10.
+ Use `$GITHUB_ENV` to simplify.
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
- The hack to get the proper flags for bindgen on GCC builds.
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Jan 10, 2021
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels:
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Support for `arm64` architecture:
+ Added to the CI: BusyBox is cross-compiled on the fly (increased
timeout a bit to match).
+ Requires weakening of a few compiler builtins and adding
`copy_{from,to}_user` helpers.
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks and debug assertions.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Do not export any helpers' symbols.
- Updated the quick start guide.
- Updated CI:
+ Now we always test with 2 builtins and 2 loadables Rust example
drivers, removing the matrix test for builtin/loadable.
+ Added `toolchain` to matrix: now we test building with GCC,
Clang or a full LLVM toolchain.
+ Added `arch` to matrix: now we test both arm64 and x86_64.
+ Added `rustc` to matrix: now we test with a very recent nightly
as well.
+ Only build `output == build` once to reduce the number
of combinations.
+ Debug x86_64 config: more things enabled (debuginfo, kgdb,
unit testing, etc.) that mimic more what a developer would have.
Running the CI will be slightly slower, but should be OK.
Also enable `-C opt-level=0` to test that such an extreme works
and also to see how much bloated everything becomes.
+ Release x86_64 config: disabled `EXPERT` and changed a few things
to make it look more like a normal desktop configuration,
although it is still pretty minimal.
+ The configs for arm64 are `EXPERT` and `EMBEDDED` ones,
very minimal, for the particular CPU we are simulating.
+ Update configs to v5.10.
+ Use `$GITHUB_ENV` to simplify.
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
- The hack to get the proper flags for bindgen on GCC builds.
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Jan 10, 2021
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels:
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Support for `arm64` architecture:
+ Added to the CI: BusyBox is cross-compiled on the fly (increased
timeout a bit to match).
+ Requires weakening of a few compiler builtins and adding
`copy_{from,to}_user` helpers.
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks and debug assertions.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Do not export any helpers' symbols.
- Updated the quick start guide.
- Updated CI:
+ Now we always test with 2 builtins and 2 loadables Rust example
drivers, removing the matrix test for builtin/loadable.
+ Added `toolchain` to matrix: now we test building with GCC,
Clang or a full LLVM toolchain.
+ Added `arch` to matrix: now we test both arm64 and x86_64.
+ Added `rustc` to matrix: now we test with a very recent nightly
as well.
+ Only build `output == build` once to reduce the number
of combinations.
+ Debug x86_64 config: more things enabled (debuginfo, kgdb,
unit testing, etc.) that mimic more what a developer would have.
Running the CI will be slightly slower, but should be OK.
Also enable `-C opt-level=0` to test that such an extreme works
and also to see how much bloated everything becomes.
+ Release x86_64 config: disabled `EXPERT` and changed a few things
to make it look more like a normal desktop configuration,
although it is still pretty minimal.
+ The configs for arm64 are `EXPERT` and `EMBEDDED` ones,
very minimal, for the particular CPU we are simulating.
+ Update configs to v5.10.
+ Use `$GITHUB_ENV` to simplify.
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
- The hack to get the proper flags for bindgen on GCC builds.
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Jan 16, 2021
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels:
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Support for `arm64` architecture:
+ Added to the CI: BusyBox is cross-compiled on the fly (increased
timeout a bit to match).
+ Requires weakening of a few compiler builtins and adding
`copy_{from,to}_user` helpers.
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks and debug assertions.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Do not export any helpers' symbols.
- Updated the quick start guide.
- Updated CI:
+ Now we always test with 2 builtins and 2 loadables Rust example
drivers, removing the matrix test for builtin/loadable.
+ Added `toolchain` to matrix: now we test building with GCC,
Clang or a full LLVM toolchain.
+ Added `arch` to matrix: now we test both arm64 and x86_64.
+ Added `rustc` to matrix: now we test with a very recent nightly
as well.
+ Only build `output == build` once to reduce the number
of combinations.
+ Debug x86_64 config: more things enabled (debuginfo, kgdb,
unit testing, etc.) that mimic more what a developer would have.
Running the CI will be slightly slower, but should be OK.
Also enable `-C opt-level=0` to test that such an extreme works
and also to see how much bloated everything becomes.
+ Release x86_64 config: disabled `EXPERT` and changed a few things
to make it look more like a normal desktop configuration,
although it is still pretty minimal.
+ The configs for arm64 are `EXPERT` and `EMBEDDED` ones,
very minimal, for the particular CPU we are simulating.
+ Update configs to v5.10.
+ Use `$GITHUB_ENV` to simplify.
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
- The hack to get the proper flags for bindgen on GCC builds.
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Jan 19, 2021
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels:
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Support for `arm64` architecture:
+ Added to the CI: BusyBox is cross-compiled on the fly (increased
timeout a bit to match).
+ Requires weakening of a few compiler builtins and adding
`copy_{from,to}_user` helpers.
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks and debug assertions.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Do not export any helpers' symbols.
- Updated the quick start guide.
- Updated CI:
+ Now we always test with 2 builtins and 2 loadables Rust example
drivers, removing the matrix test for builtin/loadable.
+ Added `toolchain` to matrix: now we test building with GCC,
Clang or a full LLVM toolchain.
+ Added `arch` to matrix: now we test both arm64 and x86_64.
+ Added `rustc` to matrix: now we test with a very recent nightly
as well.
+ Only build `output == build` once to reduce the number
of combinations.
+ Debug x86_64 config: more things enabled (debuginfo, kgdb,
unit testing, etc.) that mimic more what a developer would have.
Running the CI will be slightly slower, but should be OK.
Also enable `-C opt-level=0` to test that such an extreme works
and also to see how much bloated everything becomes.
+ Release x86_64 config: disabled `EXPERT` and changed a few things
to make it look more like a normal desktop configuration,
although it is still pretty minimal.
+ The configs for arm64 are `EXPERT` and `EMBEDDED` ones,
very minimal, for the particular CPU we are simulating.
+ Update configs to v5.10.
+ Use `$GITHUB_ENV` to simplify.
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
- The hack to get the proper flags for bindgen on GCC builds.
Signed-off-by: Miguel Ojeda <[email protected]>
ojeda
added a commit
that referenced
this issue
Jan 19, 2021
This is a big PR, but most of it is interdependent to the rest.
- Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
`liballoc`, `libcompiler_builtins`.
+ The Rust modules are now much smaller since they do not contain
several copies of those libraries. Our example `.ko` on release
is just 12 KiB, down from 1.3 MiB. For reference:
`vmlinux` on release w/ Rust is 23 MiB (compressed: 2.1 MiB)
`vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
i.e. the bulk is now shared.
+ Multiple builtin modules are now supported since their symbols
do not collide against each other (fixes #9).
+ Faster compilation (less crates to compile & less repetition).
+ We achieve this by compiling all the shared code to `.rlib`s
(and the `.so` for the proc macro). For loadable modules,
we need to rely on the upcoming v0 Rust mangling scheme,
plus we need to export the Rust symbols needed by the `.ko`s.
- Simpler, flat file structure: now a small driver may only need
a single file like `drivers/char/rust_example.rs`, like in C.
+ All the `rust/*` and `driver/char/rust_example/*` files moved
to fit in the new structure: less files around.
- Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
+ Also helps with the faster compilation.
- Dependency handling integration with `Kbuild`/`fixdep`.
+ Changes to the Rust standard library, kernel headers (bindings),
`rust/` source files, `.rs` changes, command-line changes,
flag changes, etc. all trigger recompilation as needed.
+ Works as expected with parallel support (`-j`).
- Automatic generation of the `exports.c` list:
+ Instead of manually handling the list, all non-local functions
available in `core`, `alloc` and `kernel` are exported, so all
modules should work, regardless of what they need, and without
failing linking due to symbols in the manual list not existing
(e.g. due to differences in config options).
+ They are a lot, though:
* ~6k Rust symbols vs. ~4k C symbols in release.
* However, 4k of those are `bindings_raw` (i.e. duplicated C
ones), which shouldn't be exported. Thus we should look
into making `bindings_raw` private to the crate (at the
moment, the (first) Rust example requires
`<kernel::bindings...::miscdevice as Default>::default`).
+ Licensing:
* `kernel`'s symbols are exported as GPL.
* `core`'s and `alloc`'s symbols are exported as non-GPL so
that third-parties can build Rust modules as long as they
write their own kernel support infrastructure, i.e. without
taking advantage of `kernel`. This seemed to make the most
sense compared to other exports from the kernel, plus it
follows more closely the original licence of the crates.
- Support for GCC-compiled kernels:
+ The generated bindings do not have meaningful differences in our
release config, between GCC 10.1 and Clang 11.
+ Other configs (e.g. our debug one) may add/remove types and functions.
That is fine unless we use them form our bindings.
+ However, there are config options that may not work (e.g.
the randstruct GCC plugin if we use one of those structs).
- Support for `arm64` architecture:
+ Added to the CI: BusyBox is cross-compiled on the fly (increased
timeout a bit to match).
+ Requires weakening of a few compiler builtins and adding
`copy_{from,to}_user` helpers.
- Support for custom `--sysroot` via `KRUSTCFLAGS`.
- Proper `make clean` support.
- Offline builds by default (there is no "online compilation" anymore;
fixes #17).
- No interleaved Cargo output (fixes #29).
- No nightly dependency on Cargo's `build-std`; since now we manage
the cross-compilation ourselves (should fix #27).
- "Big" kallsyms symbol support:
+ I already raised ksym names from 128 to 256 back when I wrote the first
integration. However, Rust symbols can be huge in debug/non-optimized,
so I increased it again to 512; plus the module name from 56 to 248.
+ In turn, this required tuning the table format to support 2-byte lengths
for ksyms. Compression at generation and kernel decompression is covered,
although it may be the case that some script/tool also requires changes
to understand the new table format.
- Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
is added to enable/disable it manually, regardless of whether one has
`rustc` available or not (`CONFIG_HAS_RUST`).
- Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
by default following what the user selected for C, but customizable
through a Kconfig menu. As well as options for tweaking overflow
checks and debug assertions.
- This rewrite of the Kbuild support is cleaner, i.e. less hacks
in general handling paths (e.g. no more `shell readlink` for `O=`).
- Duplicated the example driver 3 times so that we can test in the CI
that 2 builtins and 2 loadables work, all at the same time.
- Do not export any helpers' symbols.
- Updated the quick start guide.
- Updated CI:
+ Now we always test with 2 builtins and 2 loadables Rust example
drivers, removing the matrix test for builtin/loadable.
+ Added `toolchain` to matrix: now we test building with GCC,
Clang or a full LLVM toolchain.
+ Added `arch` to matrix: now we test both arm64 and x86_64.
+ Added `rustc` to matrix: now we test with a very recent nightly
as well.
+ Only build `output == build` once to reduce the number
of combinations.
+ Debug x86_64 config: more things enabled (debuginfo, kgdb,
unit testing, etc.) that mimic more what a developer would have.
Running the CI will be slightly slower, but should be OK.
Also enable `-C opt-level=0` to test that such an extreme works
and also to see how much bloated everything becomes.
+ Release x86_64 config: disabled `EXPERT` and changed a few things
to make it look more like a normal desktop configuration,
although it is still pretty minimal.
+ The configs for arm64 are `EXPERT` and `EMBEDDED` ones,
very minimal, for the particular CPU we are simulating.
+ Update configs to v5.10.
+ Use `$GITHUB_ENV` to simplify.
- Less `extern crate`s needed since we pass it via `rustc`
(closer to idiomatic 2018 edition Rust code).
Things to note:
- There is two more nightly features used:
+ The new Rust mangling scheme: we know it will be stable
(and the default on, later on).
+ The binary dep-info output: if we remove all other nightly
features, this one can easily go too.
- The hack at `exports.c` to export symbols to loadable modules.
- The hack at `allocator.rs` to get the `__rust_*()` functions.
- The hack to get the proper flags for bindgen on GCC builds.
Signed-off-by: Miguel Ojeda <[email protected]>
AltF02
pushed a commit
to AltF02/linux
that referenced
this issue
Oct 5, 2021
Minimal selftest which implements a small BPF policy program to the connect(2) hook which rejects TCP connection requests to port 60123 with EPERM. This is being attached to a non-root cgroup v2 path. The test asserts that this works under cgroup v2-only and under a mixed cgroup v1/v2 environment where net_classid is set in the former case. Before fix: # ./test_progs -t cgroup_v1v2 test_cgroup_v1v2:PASS:server_fd 0 nsec test_cgroup_v1v2:PASS:client_fd 0 nsec test_cgroup_v1v2:PASS:cgroup_fd 0 nsec test_cgroup_v1v2:PASS:server_fd 0 nsec run_test:PASS:skel_open 0 nsec run_test:PASS:prog_attach 0 nsec test_cgroup_v1v2:PASS:cgroup-v2-only 0 nsec run_test:PASS:skel_open 0 nsec run_test:PASS:prog_attach 0 nsec run_test:PASS:join_classid 0 nsec (network_helpers.c:219: errno: None) Unexpected success to connect to server test_cgroup_v1v2:FAIL:cgroup-v1v2 unexpected error: -1 (errno 0) Rust-for-Linux#27 cgroup_v1v2:FAIL Summary: 0/0 PASSED, 0 SKIPPED, 1 FAILED After fix: # ./test_progs -t cgroup_v1v2 Rust-for-Linux#27 cgroup_v1v2:OK Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Daniel Borkmann <[email protected]> Signed-off-by: Alexei Starovoitov <[email protected]> Acked-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
ojeda
pushed a commit
that referenced
this issue
Mar 14, 2022
high_memory used to be initialized in mem_init, way after setup_bootmem. But a call to dma_contiguous_reserve in this function gives rise to the below warning because high_memory is equal to 0 and is used at the very beginning at cma_declare_contiguous_nid. It went unnoticed since the move of the kasan region redefined KERN_VIRT_SIZE so that it does not encompass -1 anymore. Fix this by initializing high_memory in setup_bootmem. ------------[ cut here ]------------ virt_to_phys used for non-linear address: ffffffffffffffff (0xffffffffffffffff) WARNING: CPU: 0 PID: 0 at arch/riscv/mm/physaddr.c:14 __virt_to_phys+0xac/0x1b8 Modules linked in: CPU: 0 PID: 0 Comm: swapper Not tainted 5.17.0-rc1-00007-ga68b89289e26 #27 Hardware name: riscv-virtio,qemu (DT) epc : __virt_to_phys+0xac/0x1b8 ra : __virt_to_phys+0xac/0x1b8 epc : ffffffff80014922 ra : ffffffff80014922 sp : ffffffff84a03c30 gp : ffffffff85866c80 tp : ffffffff84a3f180 t0 : ffffffff86bce657 t1 : fffffffef09406e8 t2 : 0000000000000000 s0 : ffffffff84a03c70 s1 : ffffffffffffffff a0 : 000000000000004f a1 : 00000000000f0000 a2 : 0000000000000002 a3 : ffffffff8011f408 a4 : 0000000000000000 a5 : 0000000000000000 a6 : 0000000000f00000 a7 : ffffffff84a03747 s2 : ffffffd800000000 s3 : ffffffff86ef4000 s4 : ffffffff8467f828 s5 : fffffff800000000 s6 : 8000000000006800 s7 : 0000000000000000 s8 : 0000000480000000 s9 : 0000000080038ea0 s10: 0000000000000000 s11: ffffffffffffffff t3 : ffffffff84a035c0 t4 : fffffffef09406e8 t5 : fffffffef09406e9 t6 : ffffffff84a03758 status: 0000000000000100 badaddr: 0000000000000000 cause: 0000000000000003 [<ffffffff8322ef4c>] cma_declare_contiguous_nid+0xf2/0x64a [<ffffffff83212a58>] dma_contiguous_reserve_area+0x46/0xb4 [<ffffffff83212c3a>] dma_contiguous_reserve+0x174/0x18e [<ffffffff83208fc2>] paging_init+0x12c/0x35e [<ffffffff83206bd2>] setup_arch+0x120/0x74e [<ffffffff83201416>] start_kernel+0xce/0x68c irq event stamp: 0 hardirqs last enabled at (0): [<0000000000000000>] 0x0 hardirqs last disabled at (0): [<0000000000000000>] 0x0 softirqs last enabled at (0): [<0000000000000000>] 0x0 softirqs last disabled at (0): [<0000000000000000>] 0x0 ---[ end trace 0000000000000000 ]--- Fixes: f7ae023 ("riscv: Move KASAN mapping next to the kernel mapping") Signed-off-by: Alexandre Ghiti <[email protected]> Cc: [email protected] Signed-off-by: Palmer Dabbelt <[email protected]>
ojeda
pushed a commit
that referenced
this issue
Apr 12, 2022
when get fiemap starting from MAX_LFS_FILESIZE, (maxbytes - *len) < start will always true , then *len set zero. because of start offset is beyond file size, for erofs filesystem it will always return iomap.length with zero,iomap iterate will enter infinite loop. it is necessary cover this corner case to avoid this situation. ------------[ cut here ]------------ WARNING: CPU: 7 PID: 905 at fs/iomap/iter.c:35 iomap_iter+0x97f/0xc70 Modules linked in: xfs erofs CPU: 7 PID: 905 Comm: iomap Tainted: G W 5.17.0-rc8 #27 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014 RIP: 0010:iomap_iter+0x97f/0xc70 Code: 85 a1 fc ff ff e8 71 be 9c ff 0f 1f 44 00 00 e9 92 fc ff ff e8 62 be 9c ff 0f 0b b8 fb ff ff ff e9 fc f8 ff ff e8 51 be 9c ff <0f> 0b e9 2b fc ff ff e8 45 be 9c ff 0f 0b e9 e1 fb ff ff e8 39 be RSP: 0018:ffff888060a37ab0 EFLAGS: 00010293 RAX: 0000000000000000 RBX: ffff888060a37bb0 RCX: 0000000000000000 RDX: ffff88807e19a900 RSI: ffffffff81a7da7f RDI: ffff888060a37be0 RBP: 7fffffffffffffff R08: 0000000000000000 R09: ffff888060a37c20 R10: ffff888060a37c67 R11: ffffed100c146f8c R12: 7fffffffffffffff R13: 0000000000000000 R14: ffff888060a37bd8 R15: ffff888060a37c20 FS: 00007fd3cca01540(0000) GS:ffff888108780000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000020010820 CR3: 0000000054b92000 CR4: 00000000000006e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: <TASK> iomap_fiemap+0x1c9/0x2f0 erofs_fiemap+0x64/0x90 [erofs] do_vfs_ioctl+0x40d/0x12e0 __x64_sys_ioctl+0xaa/0x1c0 do_syscall_64+0x35/0x80 entry_SYSCALL_64_after_hwframe+0x44/0xae </TASK> ---[ end trace 0000000000000000 ]--- watchdog: BUG: soft lockup - CPU#7 stuck for 26s! [iomap:905] Reported-by: Hulk Robot <[email protected]> Signed-off-by: Guo Xuenan <[email protected]> Reviewed-by: Christoph Hellwig <[email protected]> [djwong: fix some typos] Reviewed-by: Darrick J. Wong <[email protected]> Signed-off-by: Darrick J. Wong <[email protected]>
ojeda
pushed a commit
that referenced
this issue
May 4, 2022
+new file mode 100644 +WARNING: Missing or malformed SPDX-License-Identifier tag in line 1 +#27: FILE: Documentation/virt/kvm/x86/errata.rst:1: Opportunistically update all other non-added KVM documents and remove a new extra blank line at EOF for x86/errata.rst. Signed-off-by: Like Xu <[email protected]> Message-Id: <[email protected]> Signed-off-by: Paolo Bonzini <[email protected]>
ojeda
pushed a commit
that referenced
this issue
May 4, 2022
Given a sufficiently large number of actions, while copying and reserving memory for a new action of a new flow, if next_offset is greater than MAX_ACTIONS_BUFSIZE, the function reserve_sfa_size() does not return -EMSGSIZE as expected, but it allocates MAX_ACTIONS_BUFSIZE bytes increasing actions_len by req_size. This can then lead to an OOB write access, especially when further actions need to be copied. Fix it by rearranging the flow action size check. KASAN splat below: ================================================================== BUG: KASAN: slab-out-of-bounds in reserve_sfa_size+0x1ba/0x380 [openvswitch] Write of size 65360 at addr ffff888147e4001c by task handler15/836 CPU: 1 PID: 836 Comm: handler15 Not tainted 5.18.0-rc1+ #27 ... Call Trace: <TASK> dump_stack_lvl+0x45/0x5a print_report.cold+0x5e/0x5db ? __lock_text_start+0x8/0x8 ? reserve_sfa_size+0x1ba/0x380 [openvswitch] kasan_report+0xb5/0x130 ? reserve_sfa_size+0x1ba/0x380 [openvswitch] kasan_check_range+0xf5/0x1d0 memcpy+0x39/0x60 reserve_sfa_size+0x1ba/0x380 [openvswitch] __add_action+0x24/0x120 [openvswitch] ovs_nla_add_action+0xe/0x20 [openvswitch] ovs_ct_copy_action+0x29d/0x1130 [openvswitch] ? __kernel_text_address+0xe/0x30 ? unwind_get_return_address+0x56/0xa0 ? create_prof_cpu_mask+0x20/0x20 ? ovs_ct_verify+0xf0/0xf0 [openvswitch] ? prep_compound_page+0x198/0x2a0 ? __kasan_check_byte+0x10/0x40 ? kasan_unpoison+0x40/0x70 ? ksize+0x44/0x60 ? reserve_sfa_size+0x75/0x380 [openvswitch] __ovs_nla_copy_actions+0xc26/0x2070 [openvswitch] ? __zone_watermark_ok+0x420/0x420 ? validate_set.constprop.0+0xc90/0xc90 [openvswitch] ? __alloc_pages+0x1a9/0x3e0 ? __alloc_pages_slowpath.constprop.0+0x1da0/0x1da0 ? unwind_next_frame+0x991/0x1e40 ? __mod_node_page_state+0x99/0x120 ? __mod_lruvec_page_state+0x2e3/0x470 ? __kasan_kmalloc_large+0x90/0xe0 ovs_nla_copy_actions+0x1b4/0x2c0 [openvswitch] ovs_flow_cmd_new+0x3cd/0xb10 [openvswitch] ... Cc: [email protected] Fixes: f28cd2a ("openvswitch: fix flow actions reallocation") Signed-off-by: Paolo Valerio <[email protected]> Acked-by: Eelco Chaudron <[email protected]> Signed-off-by: David S. Miller <[email protected]>
y86-dev
pushed a commit
to y86-dev/linux
that referenced
this issue
Oct 3, 2022
Broadcom 4377 controllers found in Apple x86 Macs with the T2 chip claim to support extended scanning when querying supported states, < HCI Command: LE Read Supported St.. (0x08|0x001c) plen 0 Rust-for-Linux#27 [hci0] 2.971839 > HCI Event: Command Complete (0x0e) plen 12 Rust-for-Linux#28 [hci0] 2.972730 LE Read Supported States (0x08|0x001c) ncmd 1 Status: Success (0x00) States: 0x000003ffffffffff [...] LE Set Extended Scan Parameters (Octet 37 - Bit 5) LE Set Extended Scan Enable (Octet 37 - Bit 6) [...] , but then fail to actually implement the extended scanning: < HCI Command: LE Set Extended Sca.. (0x08|0x0041) plen 8 Rust-for-Linux#105 [hci0] 5.460776 Own address type: Random (0x01) Filter policy: Accept all advertisement (0x00) PHYs: 0x01 Entry 0: LE 1M Type: Active (0x01) Interval: 11.250 msec (0x0012) Window: 11.250 msec (0x0012) > HCI Event: Command Complete (0x0e) plen 4 Rust-for-Linux#106 [hci0] 5.461777 LE Set Extended Scan Parameters (0x08|0x0041) ncmd 1 Status: Unknown HCI Command (0x01) Signed-off-by: Sven Peter <[email protected]>
psomas
pushed a commit
to psomas/linux-rs
that referenced
this issue
Dec 30, 2022
We use uprobe in aarch64_be, which we found the tracee task would exit due to SIGILL when we enable the uprobe trace. We can see the replace inst from uprobe is not correct in aarch big-endian. As in Armv8-A, instruction fetches are always treated as little-endian, we should treat the UPROBE_SWBP_INSN as little-endian。 The test case is as following。 bash-4.4# ./mqueue_test_aarchbe 1 1 2 1 10 > /dev/null & bash-4.4# cd /sys/kernel/debug/tracing/ bash-4.4# echo 'p:test /mqueue_test_aarchbe:0xc30 %x0 %x1' > uprobe_events bash-4.4# echo 1 > events/uprobes/enable bash-4.4# bash-4.4# ps PID TTY TIME CMD 140 ? 00:00:01 bash 237 ? 00:00:00 ps [1]+ Illegal instruction ./mqueue_test_aarchbe 1 1 2 1 100 > /dev/null which we debug use gdb as following: bash-4.4# gdb attach 155 (gdb) disassemble send Dump of assembler code for function send: 0x0000000000400c30 <+0>: .inst 0xa00020d4 ; undefined 0x0000000000400c34 <+4>: mov x29, sp 0x0000000000400c38 <+8>: str w0, [sp, Rust-for-Linux#28] 0x0000000000400c3c <+12>: strb w1, [sp, Rust-for-Linux#27] 0x0000000000400c40 <+16>: str xzr, [sp, Rust-for-Linux#40] 0x0000000000400c44 <+20>: str xzr, [sp, Rust-for-Linux#48] 0x0000000000400c48 <+24>: add x0, sp, #0x1b 0x0000000000400c4c <+28>: mov w3, #0x0 // #0 0x0000000000400c50 <+32>: mov x2, #0x1 // Rust-for-Linux#1 0x0000000000400c54 <+36>: mov x1, x0 0x0000000000400c58 <+40>: ldr w0, [sp, Rust-for-Linux#28] 0x0000000000400c5c <+44>: bl 0x405e10 <mq_send> 0x0000000000400c60 <+48>: str w0, [sp, Rust-for-Linux#60] 0x0000000000400c64 <+52>: ldr w0, [sp, Rust-for-Linux#60] 0x0000000000400c68 <+56>: ldp x29, x30, [sp], Rust-for-Linux#64 0x0000000000400c6c <+60>: ret End of assembler dump. (gdb) info b No breakpoints or watchpoints. (gdb) c Continuing. Program received signal SIGILL, Illegal instruction. 0x0000000000400c30 in send () (gdb) x/10x 0x400c30 0x400c30 <send>: 0xd42000a0 0xfd030091 0xe01f00b9 0xe16f0039 0x400c40 <send+16>: 0xff1700f9 0xff1b00f9 0xe06f0091 0x03008052 0x400c50 <send+32>: 0x220080d2 0xe10300aa (gdb) disassemble 0x400c30 Dump of assembler code for function send: => 0x0000000000400c30 <+0>: .inst 0xa00020d4 ; undefined 0x0000000000400c34 <+4>: mov x29, sp 0x0000000000400c38 <+8>: str w0, [sp, Rust-for-Linux#28] 0x0000000000400c3c <+12>: strb w1, [sp, Rust-for-Linux#27] 0x0000000000400c40 <+16>: str xzr, [sp, Rust-for-Linux#40] Signed-off-by: junhua huang <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Will Deacon <[email protected]>
fbq
pushed a commit
that referenced
this issue
Sep 25, 2023
The following processes run into a deadlock. CPU 41 was waiting for CPU 29 to handle a CSD request while holding spinlock "crashdump_lock", but CPU 29 was hung by that spinlock with IRQs disabled. PID: 17360 TASK: ffff95c1090c5c40 CPU: 41 COMMAND: "mrdiagd" !# 0 [ffffb80edbf37b58] __read_once_size at ffffffff9b871a40 include/linux/compiler.h:185:0 !# 1 [ffffb80edbf37b58] atomic_read at ffffffff9b871a40 arch/x86/include/asm/atomic.h:27:0 !# 2 [ffffb80edbf37b58] dump_stack at ffffffff9b871a40 lib/dump_stack.c:54:0 # 3 [ffffb80edbf37b78] csd_lock_wait_toolong at ffffffff9b131ad5 kernel/smp.c:364:0 # 4 [ffffb80edbf37b78] __csd_lock_wait at ffffffff9b131ad5 kernel/smp.c:384:0 # 5 [ffffb80edbf37bf8] csd_lock_wait at ffffffff9b13267a kernel/smp.c:394:0 # 6 [ffffb80edbf37bf8] smp_call_function_many at ffffffff9b13267a kernel/smp.c:843:0 # 7 [ffffb80edbf37c50] smp_call_function at ffffffff9b13279d kernel/smp.c:867:0 # 8 [ffffb80edbf37c50] on_each_cpu at ffffffff9b13279d kernel/smp.c:976:0 # 9 [ffffb80edbf37c78] flush_tlb_kernel_range at ffffffff9b085c4b arch/x86/mm/tlb.c:742:0 #10 [ffffb80edbf37cb8] __purge_vmap_area_lazy at ffffffff9b23a1e0 mm/vmalloc.c:701:0 #11 [ffffb80edbf37ce0] try_purge_vmap_area_lazy at ffffffff9b23a2cc mm/vmalloc.c:722:0 #12 [ffffb80edbf37ce0] free_vmap_area_noflush at ffffffff9b23a2cc mm/vmalloc.c:754:0 #13 [ffffb80edbf37cf8] free_unmap_vmap_area at ffffffff9b23bb3b mm/vmalloc.c:764:0 #14 [ffffb80edbf37cf8] remove_vm_area at ffffffff9b23bb3b mm/vmalloc.c:1509:0 #15 [ffffb80edbf37d18] __vunmap at ffffffff9b23bb8a mm/vmalloc.c:1537:0 #16 [ffffb80edbf37d40] vfree at ffffffff9b23bc85 mm/vmalloc.c:1612:0 #17 [ffffb80edbf37d58] megasas_free_host_crash_buffer [megaraid_sas] at ffffffffc020b7f2 drivers/scsi/megaraid/megaraid_sas_fusion.c:3932:0 #18 [ffffb80edbf37d80] fw_crash_state_store [megaraid_sas] at ffffffffc01f804d drivers/scsi/megaraid/megaraid_sas_base.c:3291:0 #19 [ffffb80edbf37dc0] dev_attr_store at ffffffff9b56dd7b drivers/base/core.c:758:0 #20 [ffffb80edbf37dd0] sysfs_kf_write at ffffffff9b326acf fs/sysfs/file.c:144:0 #21 [ffffb80edbf37de0] kernfs_fop_write at ffffffff9b325fd4 fs/kernfs/file.c:316:0 #22 [ffffb80edbf37e20] __vfs_write at ffffffff9b29418a fs/read_write.c:480:0 #23 [ffffb80edbf37ea8] vfs_write at ffffffff9b294462 fs/read_write.c:544:0 #24 [ffffb80edbf37ee8] SYSC_write at ffffffff9b2946ec fs/read_write.c:590:0 #25 [ffffb80edbf37ee8] SyS_write at ffffffff9b2946ec fs/read_write.c:582:0 #26 [ffffb80edbf37f30] do_syscall_64 at ffffffff9b003ca9 arch/x86/entry/common.c:298:0 #27 [ffffb80edbf37f58] entry_SYSCALL_64 at ffffffff9ba001b1 arch/x86/entry/entry_64.S:238:0 PID: 17355 TASK: ffff95c1090c3d80 CPU: 29 COMMAND: "mrdiagd" !# 0 [ffffb80f2d3c7d30] __read_once_size at ffffffff9b0f2ab0 include/linux/compiler.h:185:0 !# 1 [ffffb80f2d3c7d30] native_queued_spin_lock_slowpath at ffffffff9b0f2ab0 kernel/locking/qspinlock.c:368:0 # 2 [ffffb80f2d3c7d58] pv_queued_spin_lock_slowpath at ffffffff9b0f244b arch/x86/include/asm/paravirt.h:674:0 # 3 [ffffb80f2d3c7d58] queued_spin_lock_slowpath at ffffffff9b0f244b arch/x86/include/asm/qspinlock.h:53:0 # 4 [ffffb80f2d3c7d68] queued_spin_lock at ffffffff9b8961a6 include/asm-generic/qspinlock.h:90:0 # 5 [ffffb80f2d3c7d68] do_raw_spin_lock_flags at ffffffff9b8961a6 include/linux/spinlock.h:173:0 # 6 [ffffb80f2d3c7d68] __raw_spin_lock_irqsave at ffffffff9b8961a6 include/linux/spinlock_api_smp.h:122:0 # 7 [ffffb80f2d3c7d68] _raw_spin_lock_irqsave at ffffffff9b8961a6 kernel/locking/spinlock.c:160:0 # 8 [ffffb80f2d3c7d88] fw_crash_buffer_store [megaraid_sas] at ffffffffc01f8129 drivers/scsi/megaraid/megaraid_sas_base.c:3205:0 # 9 [ffffb80f2d3c7dc0] dev_attr_store at ffffffff9b56dd7b drivers/base/core.c:758:0 #10 [ffffb80f2d3c7dd0] sysfs_kf_write at ffffffff9b326acf fs/sysfs/file.c:144:0 #11 [ffffb80f2d3c7de0] kernfs_fop_write at ffffffff9b325fd4 fs/kernfs/file.c:316:0 #12 [ffffb80f2d3c7e20] __vfs_write at ffffffff9b29418a fs/read_write.c:480:0 #13 [ffffb80f2d3c7ea8] vfs_write at ffffffff9b294462 fs/read_write.c:544:0 #14 [ffffb80f2d3c7ee8] SYSC_write at ffffffff9b2946ec fs/read_write.c:590:0 #15 [ffffb80f2d3c7ee8] SyS_write at ffffffff9b2946ec fs/read_write.c:582:0 #16 [ffffb80f2d3c7f30] do_syscall_64 at ffffffff9b003ca9 arch/x86/entry/common.c:298:0 #17 [ffffb80f2d3c7f58] entry_SYSCALL_64 at ffffffff9ba001b1 arch/x86/entry/entry_64.S:238:0 The lock is used to synchronize different sysfs operations, it doesn't protect any resource that will be touched by an interrupt. Consequently it's not required to disable IRQs. Replace the spinlock with a mutex to fix the deadlock. Signed-off-by: Junxiao Bi <[email protected]> Link: https://lore.kernel.org/r/[email protected] Reviewed-by: Mike Christie <[email protected]> Cc: [email protected] Signed-off-by: Martin K. Petersen <[email protected]>
metaspace
pushed a commit
to metaspace/linux
that referenced
this issue
Mar 12, 2024
It appears the client object tree has no locking unless I've missed something else. Fix races around adding/removing client objects, mostly vram bar mappings. 4562.099306] general protection fault, probably for non-canonical address 0x6677ed422bceb80c: 0000 [#1] PREEMPT SMP PTI [ 4562.099314] CPU: 2 PID: 23171 Comm: deqp-vk Not tainted 6.8.0-rc6+ Rust-for-Linux#27 [ 4562.099324] Hardware name: Gigabyte Technology Co., Ltd. Z390 I AORUS PRO WIFI/Z390 I AORUS PRO WIFI-CF, BIOS F8 11/05/2021 [ 4562.099330] RIP: 0010:nvkm_object_search+0x1d/0x70 [nouveau] [ 4562.099503] Code: 90 90 90 90 90 90 90 90 90 90 90 90 90 66 0f 1f 00 0f 1f 44 00 00 48 89 f8 48 85 f6 74 39 48 8b 87 a0 00 00 00 48 85 c0 74 12 <48> 8b 48 f8 48 39 ce 73 15 48 8b 40 10 48 85 c0 75 ee 48 c7 c0 fe [ 4562.099506] RSP: 0000:ffffa94cc420bbf8 EFLAGS: 00010206 [ 4562.099512] RAX: 6677ed422bceb814 RBX: ffff98108791f400 RCX: ffff9810f26b8f58 [ 4562.099517] RDX: 0000000000000000 RSI: ffff9810f26b9158 RDI: ffff98108791f400 [ 4562.099519] RBP: ffff9810f26b9158 R08: 0000000000000000 R09: 0000000000000000 [ 4562.099521] R10: ffffa94cc420bc48 R11: 0000000000000001 R12: ffff9810f02a7cc0 [ 4562.099526] R13: 0000000000000000 R14: 00000000000000ff R15: 0000000000000007 [ 4562.099528] FS: 00007f629c5017c0(0000) GS:ffff98142c700000(0000) knlGS:0000000000000000 [ 4562.099534] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4562.099536] CR2: 00007f629a882000 CR3: 000000017019e004 CR4: 00000000003706f0 [ 4562.099541] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 4562.099542] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 4562.099544] Call Trace: [ 4562.099555] <TASK> [ 4562.099573] ? die_addr+0x36/0x90 [ 4562.099583] ? exc_general_protection+0x246/0x4a0 [ 4562.099593] ? asm_exc_general_protection+0x26/0x30 [ 4562.099600] ? nvkm_object_search+0x1d/0x70 [nouveau] [ 4562.099730] nvkm_ioctl+0xa1/0x250 [nouveau] [ 4562.099861] nvif_object_map_handle+0xc8/0x180 [nouveau] [ 4562.099986] nouveau_ttm_io_mem_reserve+0x122/0x270 [nouveau] [ 4562.100156] ? dma_resv_test_signaled+0x26/0xb0 [ 4562.100163] ttm_bo_vm_fault_reserved+0x97/0x3c0 [ttm] [ 4562.100182] ? __mutex_unlock_slowpath+0x2a/0x270 [ 4562.100189] nouveau_ttm_fault+0x69/0xb0 [nouveau] [ 4562.100356] __do_fault+0x32/0x150 [ 4562.100362] do_fault+0x7c/0x560 [ 4562.100369] __handle_mm_fault+0x800/0xc10 [ 4562.100382] handle_mm_fault+0x17c/0x3e0 [ 4562.100388] do_user_addr_fault+0x208/0x860 [ 4562.100395] exc_page_fault+0x7f/0x200 [ 4562.100402] asm_exc_page_fault+0x26/0x30 [ 4562.100412] RIP: 0033:0x9b9870 [ 4562.100419] Code: 85 a8 f7 ff ff 8b 8d 80 f7 ff ff 89 08 e9 18 f2 ff ff 0f 1f 84 00 00 00 00 00 44 89 32 e9 90 fa ff ff 0f 1f 84 00 00 00 00 00 <44> 89 32 e9 f8 f1 ff ff 0f 1f 84 00 00 00 00 00 66 44 89 32 e9 e7 [ 4562.100422] RSP: 002b:00007fff9ba2dc70 EFLAGS: 00010246 [ 4562.100426] RAX: 0000000000000004 RBX: 000000000dd65e10 RCX: 000000fff0000000 [ 4562.100428] RDX: 00007f629a882000 RSI: 00007f629a882000 RDI: 0000000000000066 [ 4562.100432] RBP: 00007fff9ba2e570 R08: 0000000000000000 R09: 0000000123ddf000 [ 4562.100434] R10: 0000000000000001 R11: 0000000000000246 R12: 000000007fffffff [ 4562.100436] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 [ 4562.100446] </TASK> [ 4562.100448] Modules linked in: nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables libcrc32c nfnetlink cmac bnep sunrpc iwlmvm intel_rapl_msr intel_rapl_common snd_sof_pci_intel_cnl x86_pkg_temp_thermal intel_powerclamp snd_sof_intel_hda_common mac80211 coretemp snd_soc_acpi_intel_match kvm_intel snd_soc_acpi snd_soc_hdac_hda snd_sof_pci snd_sof_xtensa_dsp snd_sof_intel_hda_mlink snd_sof_intel_hda snd_sof kvm snd_sof_utils snd_soc_core snd_hda_codec_realtek libarc4 snd_hda_codec_generic snd_compress snd_hda_ext_core vfat fat snd_hda_intel snd_intel_dspcfg irqbypass iwlwifi snd_hda_codec snd_hwdep snd_hda_core btusb btrtl mei_hdcp iTCO_wdt rapl mei_pxp btintel snd_seq iTCO_vendor_support btbcm snd_seq_device intel_cstate bluetooth snd_pcm cfg80211 intel_wmi_thunderbolt wmi_bmof intel_uncore snd_timer mei_me snd ecdh_generic i2c_i801 [ 4562.100541] ecc mei i2c_smbus soundcore rfkill intel_pch_thermal acpi_pad zram nouveau drm_ttm_helper ttm gpu_sched i2c_algo_bit drm_gpuvm drm_exec mxm_wmi drm_display_helper drm_kms_helper drm crct10dif_pclmul crc32_pclmul nvme e1000e crc32c_intel nvme_core ghash_clmulni_intel video wmi pinctrl_cannonlake ip6_tables ip_tables fuse [ 4562.100616] ---[ end trace 0000000000000000 ]--- Signed-off-by: Dave Airlie <[email protected]> Cc: [email protected]
ojeda
pushed a commit
that referenced
this issue
Apr 29, 2024
Running a lot of VK CTS in parallel against nouveau, once every few hours you might see something like this crash. BUG: kernel NULL pointer dereference, address: 0000000000000008 PGD 8000000114e6e067 P4D 8000000114e6e067 PUD 109046067 PMD 0 Oops: 0000 [#1] PREEMPT SMP PTI CPU: 7 PID: 53891 Comm: deqp-vk Not tainted 6.8.0-rc6+ #27 Hardware name: Gigabyte Technology Co., Ltd. Z390 I AORUS PRO WIFI/Z390 I AORUS PRO WIFI-CF, BIOS F8 11/05/2021 RIP: 0010:gp100_vmm_pgt_mem+0xe3/0x180 [nouveau] Code: c7 48 01 c8 49 89 45 58 85 d2 0f 84 95 00 00 00 41 0f b7 46 12 49 8b 7e 08 89 da 42 8d 2c f8 48 8b 47 08 41 83 c7 01 48 89 ee <48> 8b 40 08 ff d0 0f 1f 00 49 8b 7e 08 48 89 d9 48 8d 75 04 48 c1 RSP: 0000:ffffac20c5857838 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 00000000004d8001 RCX: 0000000000000001 RDX: 00000000004d8001 RSI: 00000000000006d8 RDI: ffffa07afe332180 RBP: 00000000000006d8 R08: ffffac20c5857ad0 R09: 0000000000ffff10 R10: 0000000000000001 R11: ffffa07af27e2de0 R12: 000000000000001c R13: ffffac20c5857ad0 R14: ffffa07a96fe9040 R15: 000000000000001c FS: 00007fe395eed7c0(0000) GS:ffffa07e2c980000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000011febe001 CR4: 00000000003706f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ... ? gp100_vmm_pgt_mem+0xe3/0x180 [nouveau] ? gp100_vmm_pgt_mem+0x37/0x180 [nouveau] nvkm_vmm_iter+0x351/0xa20 [nouveau] ? __pfx_nvkm_vmm_ref_ptes+0x10/0x10 [nouveau] ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] ? __lock_acquire+0x3ed/0x2170 ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] nvkm_vmm_ptes_get_map+0xc2/0x100 [nouveau] ? __pfx_nvkm_vmm_ref_ptes+0x10/0x10 [nouveau] ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] nvkm_vmm_map_locked+0x224/0x3a0 [nouveau] Adding any sort of useful debug usually makes it go away, so I hand wrote the function in a line, and debugged the asm. Every so often pt->memory->ptrs is NULL. This ptrs ptr is set in the nv50_instobj_acquire called from nvkm_kmap. If Thread A and Thread B both get to nv50_instobj_acquire around the same time, and Thread A hits the refcount_set line, and in lockstep thread B succeeds at refcount_inc_not_zero, there is a chance the ptrs value won't have been stored since refcount_set is unordered. Force a memory barrier here, I picked smp_mb, since we want it on all CPUs and it's write followed by a read. v2: use paired smp_rmb/smp_wmb. Cc: <[email protected]> Fixes: be55287 ("drm/nouveau/imem/nv50: embed nvkm_instobj directly into nv04_instobj") Signed-off-by: Dave Airlie <[email protected]> Signed-off-by: Danilo Krummrich <[email protected]> Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
ojeda
pushed a commit
that referenced
this issue
May 27, 2024
When request_irq() fails, error path calls vp_del_vqs(). There, as vq is present in the list, free_irq() is called for the same vector. That causes following splat: [ 0.414355] Trying to free already-free IRQ 27 [ 0.414403] WARNING: CPU: 1 PID: 1 at kernel/irq/manage.c:1899 free_irq+0x1a1/0x2d0 [ 0.414510] Modules linked in: [ 0.414540] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 6.9.0-rc4+ #27 [ 0.414540] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 0.414540] RIP: 0010:free_irq+0x1a1/0x2d0 [ 0.414540] Code: 1e 00 48 83 c4 08 48 89 e8 5b 5d 41 5c 41 5d 41 5e 41 5f c3 cc cc cc cc 90 8b 74 24 04 48 c7 c7 98 80 6c b1 e8 00 c9 f7 ff 90 <0f> 0b 90 90 48 89 ee 4c 89 ef e8 e0 20 b8 00 49 8b 47 40 48 8b 40 [ 0.414540] RSP: 0000:ffffb71480013ae0 EFLAGS: 00010086 [ 0.414540] RAX: 0000000000000000 RBX: ffffa099c2722000 RCX: 0000000000000000 [ 0.414540] RDX: 0000000000000000 RSI: ffffb71480013998 RDI: 0000000000000001 [ 0.414540] RBP: 0000000000000246 R08: 00000000ffffdfff R09: 0000000000000001 [ 0.414540] R10: 00000000ffffdfff R11: ffffffffb18729c0 R12: ffffa099c1c91760 [ 0.414540] R13: ffffa099c1c916a4 R14: ffffa099c1d2f200 R15: ffffa099c1c91600 [ 0.414540] FS: 0000000000000000(0000) GS:ffffa099fec40000(0000) knlGS:0000000000000000 [ 0.414540] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 0.414540] CR2: 0000000000000000 CR3: 0000000008e3e001 CR4: 0000000000370ef0 [ 0.414540] Call Trace: [ 0.414540] <TASK> [ 0.414540] ? __warn+0x80/0x120 [ 0.414540] ? free_irq+0x1a1/0x2d0 [ 0.414540] ? report_bug+0x164/0x190 [ 0.414540] ? handle_bug+0x3b/0x70 [ 0.414540] ? exc_invalid_op+0x17/0x70 [ 0.414540] ? asm_exc_invalid_op+0x1a/0x20 [ 0.414540] ? free_irq+0x1a1/0x2d0 [ 0.414540] vp_del_vqs+0xc1/0x220 [ 0.414540] vp_find_vqs_msix+0x305/0x470 [ 0.414540] vp_find_vqs+0x3e/0x1a0 [ 0.414540] vp_modern_find_vqs+0x1b/0x70 [ 0.414540] init_vqs+0x387/0x600 [ 0.414540] virtnet_probe+0x50a/0xc80 [ 0.414540] virtio_dev_probe+0x1e0/0x2b0 [ 0.414540] really_probe+0xc0/0x2c0 [ 0.414540] ? __pfx___driver_attach+0x10/0x10 [ 0.414540] __driver_probe_device+0x73/0x120 [ 0.414540] driver_probe_device+0x1f/0xe0 [ 0.414540] __driver_attach+0x88/0x180 [ 0.414540] bus_for_each_dev+0x85/0xd0 [ 0.414540] bus_add_driver+0xec/0x1f0 [ 0.414540] driver_register+0x59/0x100 [ 0.414540] ? __pfx_virtio_net_driver_init+0x10/0x10 [ 0.414540] virtio_net_driver_init+0x90/0xb0 [ 0.414540] do_one_initcall+0x58/0x230 [ 0.414540] kernel_init_freeable+0x1a3/0x2d0 [ 0.414540] ? __pfx_kernel_init+0x10/0x10 [ 0.414540] kernel_init+0x1a/0x1c0 [ 0.414540] ret_from_fork+0x31/0x50 [ 0.414540] ? __pfx_kernel_init+0x10/0x10 [ 0.414540] ret_from_fork_asm+0x1a/0x30 [ 0.414540] </TASK> Fix this by calling deleting the current vq when request_irq() fails. Fixes: 0b0f9dc ("Revert "virtio_pci: use shared interrupts for virtqueues"") Signed-off-by: Jiri Pirko <[email protected]> Message-Id: <[email protected]> Signed-off-by: Michael S. Tsirkin <[email protected]>
ojeda
pushed a commit
that referenced
this issue
Oct 7, 2024
Wesley reported an issue: ================================================================== EXT4-fs (dm-5): resizing filesystem from 7168 to 786432 blocks ------------[ cut here ]------------ kernel BUG at fs/ext4/resize.c:324! CPU: 9 UID: 0 PID: 3576 Comm: resize2fs Not tainted 6.11.0+ #27 RIP: 0010:ext4_resize_fs+0x1212/0x12d0 Call Trace: __ext4_ioctl+0x4e0/0x1800 ext4_ioctl+0x12/0x20 __x64_sys_ioctl+0x99/0xd0 x64_sys_call+0x1206/0x20d0 do_syscall_64+0x72/0x110 entry_SYSCALL_64_after_hwframe+0x76/0x7e ================================================================== While reviewing the patch, Honza found that when adjusting resize_bg in alloc_flex_gd(), it was possible for flex_gd->resize_bg to be bigger than flexbg_size. The reproduction of the problem requires the following: o_group = flexbg_size * 2 * n; o_size = (o_group + 1) * group_size; n_group: [o_group + flexbg_size, o_group + flexbg_size * 2) o_size = (n_group + 1) * group_size; Take n=0,flexbg_size=16 as an example: last:15 |o---------------|--------------n-| o_group:0 resize to n_group:30 The corresponding reproducer is: img=test.img rm -f $img truncate -s 600M $img mkfs.ext4 -F $img -b 1024 -G 16 8M dev=`losetup -f --show $img` mkdir -p /tmp/test mount $dev /tmp/test resize2fs $dev 248M Delete the problematic plus 1 to fix the issue, and add a WARN_ON_ONCE() to prevent the issue from happening again. [ Note: another reproucer which this commit fixes is: img=test.img rm -f $img truncate -s 25MiB $img mkfs.ext4 -b 4096 -E nodiscard,lazy_itable_init=0,lazy_journal_init=0 $img truncate -s 3GiB $img dev=`losetup -f --show $img` mkdir -p /tmp/test mount $dev /tmp/test resize2fs $dev 3G umount $dev losetup -d $dev -- TYT ] Reported-by: Wesley Hershberger <[email protected]> Closes: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2081231 Reported-by: Stéphane Graber <[email protected]> Closes: https://lore.kernel.org/all/[email protected]/ Tested-by: Alexander Mikhalitsyn <[email protected]> Tested-by: Eric Sandeen <[email protected]> Fixes: 665d3e0 ("ext4: reduce unnecessary memory allocation in alloc_flex_gd()") Cc: [email protected] Signed-off-by: Baokun Li <[email protected]> Reviewed-by: Jan Kara <[email protected]> Link: https://patch.msgid.link/[email protected] Signed-off-by: Theodore Ts'o <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Compiling the current head of the
rustbranch failed for me. Here is my setup:Is there something missing in Documentation/rust/quick-start.rst?
The text was updated successfully, but these errors were encountered: