Commits on Jan 19, 2021

1. Use rustc directly instead of cargo …

   This is a big PR, but most of it is interdependent to the rest.
   
     - Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
       `liballoc`, `libcompiler_builtins`.
   
         + The Rust modules are now much smaller since they do not contain
           several copies of those libraries. Our example `.ko` on release
           is just 12 KiB, down from 1.3 MiB. For reference:
   
               `vmlinux` on release w/  Rust is 23 MiB (compressed: 2.1 MiB)
               `vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)
   
           i.e. the bulk is now shared.
   
         + Multiple builtin modules are now supported since their symbols
           do not collide against each other (fixes #9).
   
         + Faster compilation (less crates to compile & less repetition).
   
         + We achieve this by compiling all the shared code to `.rlib`s
           (and the `.so` for the proc macro). For loadable modules,
           we need to rely on the upcoming v0 Rust mangling scheme,
           plus we need to export the Rust symbols needed by the `.ko`s.
   
     - Simpler, flat file structure: now a small driver may only need
       a single file like `drivers/char/rust_example.rs`, like in C.
   
         + All the `rust/*` and `driver/char/rust_example/*` files moved
           to fit in the new structure: less files around.
   
     - Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.
   
         + Also helps with the faster compilation.
   
     - Dependency handling integration with `Kbuild`/`fixdep`.
   
         + Changes to the Rust standard library, kernel headers (bindings),
           `rust/` source files, `.rs` changes, command-line changes,
           flag changes, etc. all trigger recompilation as needed.
   
         + Works as expected with parallel support (`-j`).
   
     - Automatic generation of the `exports.c` list:
   
         + Instead of manually handling the list, all non-local functions
           available in `core`, `alloc` and `kernel` are exported, so all
           modules should work, regardless of what they need, and without
           failing linking due to symbols in the manual list not existing
           (e.g. due to differences in config options).
   
         + They are a lot, though:
   
             * ~6k Rust symbols vs. ~4k C symbols in release.
   
             * However, 4k of those are `bindings_raw` (i.e. duplicated C
               ones), which shouldn't be exported. Thus we should look
               into making `bindings_raw` private to the crate (at the
               moment, the (first) Rust example requires
               `<kernel::bindings...::miscdevice as Default>::default`).
   
         + Licensing:
   
             * `kernel`'s symbols are exported as GPL.
   
             * `core`'s and `alloc`'s symbols are exported as non-GPL so
               that third-parties can build Rust modules as long as they
               write their own kernel support infrastructure, i.e. without
               taking advantage of `kernel`. This seemed to make the most
               sense compared to other exports from the kernel, plus it
               follows more closely the original licence of the crates.
   
     - Support for GCC-compiled kernels:
   
       + The generated bindings do not have meaningful differences in our
         release config, between GCC 10.1 and Clang 11.
   
       + Other configs (e.g. our debug one) may add/remove types and functions.
         That is fine unless we use them form our bindings.
   
       + However, there are config options that may not work (e.g.
         the randstruct GCC plugin if we use one of those structs).
   
     - Support for `arm64` architecture:
   
       + Added to the CI: BusyBox is cross-compiled on the fly (increased
         timeout a bit to match).
   
       + Requires weakening of a few compiler builtins and adding
         `copy_{from,to}_user` helpers.
   
     - Support for custom `--sysroot` via `KRUSTCFLAGS`.
   
     - Proper `make clean` support.
   
     - Offline builds by default (there is no "online compilation" anymore;
       fixes #17).
   
     - No interleaved Cargo output (fixes #29).
   
     - No nightly dependency on Cargo's `build-std`; since now we manage
       the cross-compilation ourselves (should fix #27).
   
     - "Big" kallsyms symbol support:
   
       + I already raised ksym names from 128 to 256 back when I wrote the first
         integration. However, Rust symbols can be huge in debug/non-optimized,
         so I increased it again to 512; plus the module name from 56 to 248.
   
       + In turn, this required tuning the table format to support 2-byte lengths
         for ksyms. Compression at generation and kernel decompression is covered,
         although it may be the case that some script/tool also requires changes
         to understand the new table format.
   
     - Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
       is added to enable/disable it manually, regardless of whether one has
       `rustc` available or not (`CONFIG_HAS_RUST`).
   
     - Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
       by default following what the user selected for C, but customizable
       through a Kconfig menu. As well as options for tweaking overflow
       checks and debug assertions.
   
     - This rewrite of the Kbuild support is cleaner, i.e. less hacks
       in general handling paths (e.g. no more `shell readlink` for `O=`).
   
     - Duplicated the example driver 3 times so that we can test in the CI
       that 2 builtins and 2 loadables work, all at the same time.
   
     - Do not export any helpers' symbols.
   
     - Updated the quick start guide.
   
     - Updated CI:
   
         + Now we always test with 2 builtins and 2 loadables Rust example
           drivers, removing the matrix test for builtin/loadable.
   
         + Added `toolchain` to matrix: now we test building with GCC,
           Clang or a full LLVM toolchain.
   
         + Added `arch` to matrix: now we test both arm64 and x86_64.
   
         + Added `rustc` to matrix: now we test with a very recent nightly
           as well.
   
         + Only build `output == build` once to reduce the number
           of combinations.
   
         + Debug x86_64 config: more things enabled (debuginfo, kgdb,
           unit testing, etc.) that mimic more what a developer would have.
           Running the CI will be slightly slower, but should be OK.
           Also enable `-C opt-level=0` to test that such an extreme works
           and also to see how much bloated everything becomes.
   
         + Release x86_64 config: disabled `EXPERT` and changed a few things
           to make it look more like a normal desktop configuration,
           although it is still pretty minimal.
   
         + The configs for arm64 are `EXPERT` and `EMBEDDED` ones,
           very minimal, for the particular CPU we are simulating.
   
         + Update configs to v5.10.
   
         + Use `$GITHUB_ENV` to simplify.
   
     - Less `extern crate`s needed since we pass it via `rustc`
       (closer to idiomatic 2018 edition Rust code).
   
   Things to note:
   
     - There is two more nightly features used:
   
         + The new Rust mangling scheme: we know it will be stable
           (and the default on, later on).
   
         + The binary dep-info output: if we remove all other nightly
           features, this one can easily go too.
   
     - The hack at `exports.c` to export symbols to loadable modules.
   
     - The hack at `allocator.rs` to get the `__rust_*()` functions.
   
     - The hack to get the proper flags for bindgen on GCC builds.
   
   Signed-off-by: Miguel Ojeda <[email protected]>

   

   ojeda committed Jan 19, 2021
   Configuration menu

   • View commit details
   • Copy full SHA for 7c6155b
   • Browse repository at this point

   Copy the full SHA

   7c6155b View commit details

   Browse the repository at this point in the history