Move PAM generation to build time, optimizing download size

.github/workflows/ci.yml

@@ -15,50 +15,18 @@ jobs:
       - name: Checkout code
         uses: actions/checkout@v4
 
+      # Rust Dependencies
       - name: Install Stable Rust
         uses: actions-rust-lang/setup-rust-toolchain@v1
         with:
           toolchain: stable
-
-      # Rust Dependencies
-      - name: Cache Cargo installed binaries
-        uses: actions/cache@v4
-        id: cache-cargo
-        with:
-          path: ~/cargo-bin
-          key: rust-tools-20250106-001
-      - name: Install svd2rust
-        if: steps.cache-cargo.outputs.cache-hit != 'true'
-        run: cargo install svd2rust --version 0.28.0 --locked
-      - name: Install cargo-form
-        if: steps.cache-cargo.outputs.cache-hit != 'true'
-        run: cargo install form --version 0.8.0 --locked
-      - name: Install atdf2svd
-        if: steps.cache-cargo.outputs.cache-hit != 'true'
-        run: cargo install atdf2svd --version 0.5.0 --locked
-      - name: Install svdtools
-        if: steps.cache-cargo.outputs.cache-hit != 'true'
-        run: cargo install svdtools --version 0.4.0 --locked
-      - name: Copy tools to cache directory
-        if: steps.cache-cargo.outputs.cache-hit != 'true'
-        run: |
-          mkdir ~/cargo-bin
-          cp ~/.cargo/bin/svd2rust ~/cargo-bin
-          cp ~/.cargo/bin/form ~/cargo-bin
-          cp ~/.cargo/bin/atdf2svd ~/cargo-bin
-          cp ~/.cargo/bin/svdtools ~/cargo-bin
-      - name: Put new cargo binary directory into path
-        run: echo "$HOME/cargo-bin" >> $GITHUB_PATH
-
       - name: Install Nightly Rust
         uses: actions-rust-lang/setup-rust-toolchain@v1
         with:
-          toolchain: nightly-2023-08-08
-          components: rustfmt
+          toolchain: nightly-2023-12-28
+          components: rust-src,rustfmt
 
       # Actual test run
-      - name: Generate chip description sources
-        run: make RUSTUP_TOOLCHAIN=nightly-2023-08-08
       - name: Test-compile the crate
         run: cargo check --all-features
 
@@ -73,7 +41,7 @@ jobs:
         with:
           name: avr-device
           path: |
-            svd/
+            target/avr-atmega328p/debug/build/avr-device-*/out/svd/
             target/package/avr-device-*.crate
             macros/target/package/avr-device-macros-*.crate
 

.gitignore

@@ -2,9 +2,3 @@
 /macros/target/
 **/*.rs.bk
 Cargo.lock
-
-svd/
-.deps/
-src/devices/*/*
-src/generic.rs
-__pycache__/

Cargo.toml

@@ -16,6 +16,10 @@ include = [
     "/LICENSE-*",
     "/README.md",
     "/build.rs",
+    "/patch/**/*.yaml",
+    "/vendor/*.atdf",
+    "/vendor/LICENSE",
+    "/examples/**/src/*.rs"
 ]
 
 [package.metadata.docs.rs]
@@ -94,3 +98,10 @@ critical-section = { version = "1.1.1", optional = true }
 path = "macros/"
 version = "=0.7.0"
 optional = true
+
+[build-dependencies]
+svd2rust = "=0.28.0"
+svdtools = "=0.4.0"
+atdf2svd = "=0.4.0"
+prettyplease = "=0.1.22"
+syn = { version = "1", default-features = false, features = ["full", "parsing"] }

README.md

@@ -38,48 +38,64 @@ Via the feature you can select which chip you want the register specifications f
 |                 |              |                   |               | `attiny2313a` |
 
 ## Build Instructions
-The version on `crates.io` is pre-built.  The following is only necessary when trying to build this crate from source.
-
-You need to have [atdf2svd][] (= 0.5.0), [svd2rust][] (= 0.28), [form][] (>= 0.8), [rustfmt][](for the *nightly* toolchain) and [svdtools][] (= 0.4.0) installed:
-```bash
-cargo install atdf2svd --version 0.5.0 --locked
-cargo install svd2rust --version 0.28.0 --locked
-cargo install form
-rustup component add --toolchain nightly rustfmt
-cargo install svdtools --version 0.4.0 --locked
-```
-
-[atdf2svd]: https://github.com/Rahix/atdf2svd
-[svd2rust]: https://github.com/rust-embedded/svd2rust
-[form]: https://github.com/djmcgill/form
-[rustfmt]: https://github.com/rust-lang/rustfmt
-[svdtools]: https://github.com/stm32-rs/svdtools
-
-Next, clone this repo and build the device definitions:
-```bash
-git clone https://github.com/Rahix/avr-device
-cd avr-device
-make
-# You can build for just one specific chip using
-# make atmega32u4
-# I suggest building documentation as well
-cargo +nightly doc --features <chip> --open
-```
-
-## Internals
-*avr-device* is generated using [`atdf2svd`](https://github.com/Rahix/atdf2svd) and [`svd2rust`](https://github.com/rust-embedded/svd2rust).  The vendor-provided *atdf* files can be found in `vendor/`.  The intermediate svd files are patched by `svdpatch.py` (Adapted from [`svdpatch.py`](https://github.com/stm32-rs/stm32-rs/blob/master/scripts/svdpatch.py) in [stm32-rs](https://github.com/stm32-rs/stm32-rs)) with device-dependent patches in `patch/`, mainly to improve undescriptive names and missing descriptions.
+The PACs (Peripheral Access Crates, or really modules, in our case) **are not**
+checked into git. Rather, we generate them at build time, via an automated
+process implemented in [`build.rs`](./build.rs). It takes the ATDF files
+Microchip (former Atmel) provides plus some patches of our own making as inputs,
+and outputs a module generated from those device descriptions. These inputs
+**are** checked-in. The process is similar to what the `*bindgen` crates
+provide, just has more steps. So, in short, building should be a matter of
+selecting the features and running cargo.
 
 ### Adding a new Chip
-To add a new chip, download the *atdf* from <http://packs.download.atmel.com/> (or [avr-mcu/packs/](https://github.com/avr-rust/avr-mcu/tree/master/packs)) and place it in `vendor/` ***note: file name may need to be modified***.  Be sure to name it like the Rust module that should be generated.  Next, you need to integrate it into the base crate and build system.  Follow what was done in commit [290613454fbd ("Add basic support for ATmega64")](https://github.com/Rahix/avr-device/commit/290613454fbdc5e4ac98e53deccaf74dafc88963).  Please adhere to the alphabetical sorting that is present so far.
-
-Next, you **must** create a `<chipname>.yaml` in `patch/` which has at least the following content:
-```yaml
-_svd: ../svd/<chipname>.svd
-```
-
-If more patches need to be applied (most likely!), they should be added into this file as well.  The patching format is documented in the [`svdtools` README](https://github.com/stm32-rs/svdtools#device-and-peripheral-yaml-format).  Ideally, try to reuse the exisiting patches in `patch/common/` or `patch/timer/`.
-
-Finally, try building the crate for your MCU with `make <chipname>`.
+To add a new chip:
+
+1. Download the ATDF from <http://packs.download.atmel.com/> and place it in
+   `vendor/`. Be sure to name it like the Rust module that should be generated.
+2. Add a feature of the same name to `Cargo.toml` (it should enable
+   `device-selected`);
+3. Add any needed patches to a yaml file with the same name under the `patch`
+   directory, ideally by including some of the snippets present in
+   `patch/common` and `patch/timer`; The format is decribed
+   [here](https://github.com/rust-embedded/svdtools#device-and-peripheral-yaml-format),
+   but it should not include the top-level `_svd` key, as that's handled by the
+   build system; If patching is unneeded (it's almost always needed!), the file
+   can be omitted.
+4. Include the module into the tree, in [`devices.rs`](./src/devices.rs),
+   following the format used by other modules in that file;
+5. Finally, try building the crate for your MCU with
+   `cargo build --features <mcu>,rt`.
+6. Also check the built documentation for inconsistencies, via
+   `cargo doc --features <mcu>,rt --open` (it will pop up in your browser).
+
+## Internals
+Since the vendor does not provide SVDs we can pass to [`svd2rust`][], we
+generate one via [`atdf2svd`][]. The sequence is as follows:
+
+1. Check which MCUs are known to the crate
+   ([build.rs:get_available_mcus](./build.rs#L21-L40));
+2. Select which to build for by checking enabled features
+   ([build.rs:select_mcu](./build.rs#L42-L62));
+3. Generate the Rust module ([build.rs:build_mcu_module](./build.rs#L64-L148));
+
+   Substeps are:
+   1. Register inputs with cargo;
+   2. Get a temporary directory;
+   3. Apply `atdf2svd`;
+   4. If a yaml patch exists, use it via [`svdtools`][] and read the new content
+      / else, read the content of the unpatched file to continue;
+   5. Get the output directory;
+   6. Apply `svd2rust`;
+   7. Run [`prettyplease`][] on the module to make it readable in [`docs.rs`][];
+4. It will be included from `$OUT_DIR/pac/<mcu>.rs` into the path
+   `avr_device::devices::<mcu>` (private), and re-exported as
+   `avr_device::<mcu>` (public).
+
+[`atdf2svd`]: https://github.com/Rahix/atdf2svd
+[`svd2rust`]: https://github.com/rust-embedded/svd2rust
+[`svdtools`]: https://github.com/rust-embedded/svdtools
+[`prettyplease`]: https://github.com/dtolnay/prettyplease
+[`docs.rs`]: https://docs.rs/avr-device/latest/avr_device
 
 ## License
 *avr-device* is licensed under either of