Skip to content

Commit

Permalink
Use rustc directly instead of cargo
Browse files Browse the repository at this point in the history
This is a big PR, but most of it is interdependent to the rest.

  - Shared Rust infrastructure: `libkernel`, `libmodule`, `libcore`,
    `liballoc`, `libcompiler_builtins`.

      + The Rust modules are now much smaller since they do not contain
        several copies of those libraries. Our example `.ko` on release
        is just 12 KiB, down from 1.3 MiB. For reference:

            `vmlinux` on release w/  Rust is 23 MiB (compressed: 2.1 MiB)
            `vmlinux` on release w/o Rust is 22 MiB (compressed: 1.9 MiB)

        i.e. the bulk is now shared.

      + Multiple builtin modules are now supported since their symbols
        do not collide against each other (fixes #9).

      + Faster compilation (less crates to compile & less repetition).

      + We achieve this by compiling all the shared code to `.rlib`s
        (and the `.so` for the proc macro). For loadable modules,
        we need to rely on the upcoming v0 Rust mangling scheme,
        plus we need to export the Rust symbols needed by the `.ko`s.

  - Simpler, flat file structure: now a small driver may only need
    a single file like `drivers/char/rust_example.rs`, like in C.

      + All the `rust/*` and `driver/char/rust_example/*` files moved
        to fit in the new structure: less files around.

  - Only `rust-lang/{rust,rust-bindgen,compiler-builtins}` as dependencies.

      + Also helps with the faster compilation.

  - Dependency handling integration with `Kbuild`/`fixdep`.

      + Changes to the Rust standard library, kernel headers (bindings),
        `rust/` source files, `.rs` changes, command-line changes,
        flag changes, etc. all trigger recompilation as needed.

      + Works as expected with parallel support (`-j`).

  - Automatic generation of the `exports.c` list:

      + Instead of manually handling the list, all non-local functions
        available in `core`, `alloc` and `kernel` are exported, so all
        modules should work, regardless of what they need, and without
        failing linking due to symbols in the manual list not existing
        (e.g. due to differences in config options).

      + They are a lot, though:

          * ~6k Rust symbols vs. ~4k C symbols in release.

          * However, 4k of those are `bindings_raw` (i.e. duplicated C
            ones), which shouldn't be exported. Thus we should look
            into making `bindings_raw` private to the crate (at the
            moment, the (first) Rust example requires
            `<kernel::bindings...::miscdevice as Default>::default`).

      + Licensing:

          * `kernel`'s symbols are exported as GPL.

          * `core`'s and `alloc`'s symbols are exported as non-GPL so
            that third-parties can build Rust modules as long as they
            write their own kernel support infrastructure, i.e. without
            taking advantage of `kernel`. This seemed to make the most
            sense compared to other exports from the kernel, plus it
            follows more closely the original licence of the crates.

  - Support for GCC-compiled kernels.

    + The generated bindings do not have meaningful differences in our
      release config, between GCC 10.1 and Clang 11.

    + Other configs (e.g. our debug one) may add/remove types and functions.
      That is fine unless we use them form our bindings.

    + However, there are config options that may not work (e.g.
      the randstruct GCC plugin if we use one of those structs).

  - Proper `make clean` support.

  - Offline builds by default (there is no "online compilation" anymore;
    fixes #17).

  - No interleaved Cargo output (fixes #29).

  - No nightly dependency on Cargo's `build-std`; since now we manage
    the cross-compilation ourselves (should fix #27).

  - "Big" kallsyms symbol support:

    + I already raised ksym names from 128 to 256 back when I wrote the first
      integration. However, Rust symbols can be huge in debug/non-optimized,
      so I increased it again to 512; plus the module name from 56 to 248.

    + In turn, this required tuning the table format to support 2-byte lengths
      for ksyms. Compression at generation and kernel decompression is covered,
      although it may be the case that some script/tool also requires changes
      to understand the new table format.

  - Since now a kernel can be "Rust-enabled", a new `CONFIG_RUST` option
    is added to enable/disable it manually, regardless of whether one has
    `rustc` available or not (`CONFIG_HAS_RUST`).

  - Improved handling of `rustc` flags (`opt-level`, `debuginfo`, etc.),
    by default following what the user selected for C, but customizable
    through a Kconfig menu. As well as options for tweaking overflow
    checks, debug assertions, etc.

  - This rewrite of the Kbuild support is cleaner, i.e. less hacks
    in general handling paths (e.g. no more `shell readlink` for `O=`).

  - Duplicated the example driver 3 times so that we can test in the CI
    that 2 builtins and 2 loadables work, all at the same time.

  - Updated the quick start guide.

  - Updated CI `.config`s:

      + Add the new options and test with 2 builtins and 2 loadables.
        At the same time, remove the matrix test for builtin/loadable.

      + Updated with `toolchain` matrix support: now we test building
        with GCC, Clang or a full LLVM toolchain.

      + Debug: more things enabled (debuginfo, kgdb, unit testing, etc.)
        that mimic more what a developer would have. Running the CI
        will be slightly slower, but should be OK. Also enable
        `-C opt-level=0` to test that such an extreme works and also
        to see how much bloated everything becomes.

      + Release: disabled `EXPERT` and changed a few things to make it
        look more like a normal configuration.

      + Also update both configs to v5.10 and `LLVM=1` while I was at it.

    (I could have split a few of these ones off into another PR,
    but anyway it is for the CI only and I had already done it).

  - Less `extern crate`s needed since we pass it via `rustc`
    (closer to idiomatic 2018 edition Rust code).

Things to note:

  - There is two more nightly features used:

      + The new Rust mangling scheme: we know it will be stable
        (and the default on, later on).

      + The binary dep-info output: if we remove all other nightly
        features, this one can easily go too.

  - The hack at `exports.c` to export symbols to loadable modules.

  - The hack at `allocator.rs` to get the `__rust_*()` functions.

Signed-off-by: Miguel Ojeda <[email protected]>
  • Loading branch information
ojeda committed Dec 20, 2020
1 parent 00c1ce0 commit 82d14e2
Show file tree
Hide file tree
Showing 56 changed files with 1,094 additions and 521 deletions.
64 changes: 35 additions & 29 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ jobs:

strategy:
matrix:
mode: [debug, release]
module: [builtin, loadable]
config: [debug, release]
toolchain: [gcc, clang, llvm]
outputdir: [src, build]

steps:
Expand All @@ -20,29 +20,32 @@ jobs:
- run: wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
- run: sudo add-apt-repository 'deb http://apt.llvm.org/focal/ llvm-toolchain-focal-11 main'
- run: sudo apt-get update -y
- run: sudo apt-get install -y clang-11 libelf-dev qemu-system-x86 busybox-static
- run: sudo apt-get install -y clang-11 lld-11 libelf-dev qemu-system-x86 busybox-static
- run: rustup default nightly-2020-08-27
- run: rustup component add rustfmt
- run: rustup component add rust-src
- run: git clone --depth 1 --branch 0.1.36 https://github.com/rust-lang/compiler-builtins.git $(rustc --print sysroot)/lib/rustlib/src/compiler-builtins

# Build
- run: cp .github/workflows/kernel-${{ matrix.mode }}.config .config

- if: matrix.module == 'loadable'
run: sed -i -E 's/^(CONFIG_RUST_EXAMPLE=)(y)$/\1m/g' .config
- run: cp .github/workflows/kernel-${{ matrix.config }}.config .config

- if: matrix.outputdir == 'build'
run: mkdir build && mv .config build/.config

- if: matrix.outputdir == 'src'
run: make CC=clang-11 LLVM_CONFIG_PATH=llvm-config-11 -j3
- if: matrix.outputdir == 'build'
run: make O=build CC=clang-11 LLVM_CONFIG_PATH=llvm-config-11 -j3
- if: matrix.toolchain == 'gcc' && matrix.outputdir == 'src'
run: make -j3
- if: matrix.toolchain == 'gcc' && matrix.outputdir == 'build'
run: make -j3 O=build
- if: matrix.toolchain == 'clang' && matrix.outputdir == 'src'
run: make -j3 LLVM_CONFIG_PATH=llvm-config-11 CC=clang-11
- if: matrix.toolchain == 'clang' && matrix.outputdir == 'build'
run: make -j3 O=build LLVM_CONFIG_PATH=llvm-config-11 CC=clang-11
- if: matrix.toolchain == 'llvm' && matrix.outputdir == 'src'
run: make -j3 LLVM_CONFIG_PATH=llvm-config-11 LLVM=1
- if: matrix.toolchain == 'llvm' && matrix.outputdir == 'build'
run: make -j3 O=build LLVM_CONFIG_PATH=llvm-config-11 LLVM=1

# Run
- if: matrix.module == 'builtin'
run: sed -i '/rust_example/d' .github/workflows/qemu-initramfs.desc

- if: matrix.outputdir == 'build'
run: sed -i 's:drivers/:build/drivers/:' .github/workflows/qemu-initramfs.desc

Expand All @@ -52,28 +55,31 @@ jobs:
run: build/usr/gen_init_cpio .github/workflows/qemu-initramfs.desc > qemu-initramfs.img

- if: matrix.outputdir == 'src'
run: qemu-system-x86_64 -kernel arch/x86/boot/bzImage -initrd qemu-initramfs.img -cpu Cascadelake-Server -smp 2 -nographic -no-reboot -append "console=ttyS0 ${{ matrix.module == 'builtin' && 'rust_example.my_i32=123321' || '' }}" | tee qemu-stdout.log
run: qemu-system-x86_64 -kernel arch/x86/boot/bzImage -initrd qemu-initramfs.img -cpu Cascadelake-Server -smp 2 -nographic -no-reboot -append "console=ttyS0 rust_example.my_i32=123321 rust_example_2.my_i32=234432" | tee qemu-stdout.log
- if: matrix.outputdir == 'build'
run: qemu-system-x86_64 -kernel build/arch/x86/boot/bzImage -initrd qemu-initramfs.img -cpu Cascadelake-Server -smp 2 -nographic -no-reboot -append "console=ttyS0 ${{ matrix.module == 'builtin' && 'rust_example.my_i32=123321' || '' }}" | tee qemu-stdout.log
run: qemu-system-x86_64 -kernel build/arch/x86/boot/bzImage -initrd qemu-initramfs.img -cpu Cascadelake-Server -smp 2 -nographic -no-reboot -append "console=ttyS0 rust_example.my_i32=123321 rust_example_2.my_i32=234432" | tee qemu-stdout.log

# Check
- run: grep -F 'Rust Example (init)' qemu-stdout.log
- run: "grep 'my_i32: \\+123321' qemu-stdout.log"
- if: matrix.module == 'loadable'
run: grep -F 'Rust Example (exit)' qemu-stdout.log
- run: grep -F '] Rust Example (init)' qemu-stdout.log
- run: grep -F '] [2] Rust Example (init)' qemu-stdout.log
- run: grep -F '] [3] Rust Example (init)' qemu-stdout.log
- run: grep -F '] [4] Rust Example (init)' qemu-stdout.log

# Report
- if: matrix.outputdir == 'src' && matrix.module == 'loadable'
run: ls -l drivers/char/rust_example/rust_example.ko
- if: matrix.outputdir == 'build' && matrix.module == 'loadable'
run: ls -l build/drivers/char/rust_example/rust_example.ko
- run: "grep -F '] my_i32: 123321' qemu-stdout.log"
- run: "grep -F '] [2] my_i32: 234432' qemu-stdout.log"
- run: "grep -F '] [3] my_i32: 345543' qemu-stdout.log"
- run: "grep -F '] [4] my_i32: 456654' qemu-stdout.log"

- run: grep -F '] [3] Rust Example (exit)' qemu-stdout.log
- run: grep -F '] [4] Rust Example (exit)' qemu-stdout.log

# Report
- if: matrix.outputdir == 'src'
run: ls -l vmlinux arch/x86/boot/bzImage
run: ls -l drivers/char/rust_example.o drivers/char/rust_example_3.ko rust/*.o vmlinux arch/x86/boot/bzImage
- if: matrix.outputdir == 'build'
run: ls -l build/vmlinux build/arch/x86/boot/bzImage
run: ls -l build/drivers/char/rust_example.o build/drivers/char/rust_example_3.ko build/rust/*.o build/vmlinux build/arch/x86/boot/bzImage

- if: matrix.outputdir == 'src'
run: size vmlinux
run: size drivers/char/rust_example.o drivers/char/rust_example_3.ko rust/*.o vmlinux
- if: matrix.outputdir == 'build'
run: size build/vmlinux
run: size build/drivers/char/rust_example.o build/drivers/char/rust_example_3.ko build/rust/*.o build/vmlinux
Loading

0 comments on commit 82d14e2

Please sign in to comment.