Add NoAllocBufferSegments, suitable for no_alloc environments #423
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
It is similar to BufferSegments, but it avoids any allocations.
|
With this PR I can use capnp in the linux kernel, I've tested it and it works (with the honorable mention of removing the f32/f64 match arms in |
ariel-miculas
added a commit
to ariel-miculas/linux
that referenced
this pull request
Jul 11, 2023
…etadata Puzzlefs uses capnproto for storing the filesystem metadata, defined in manifest.capnp and metadata.capnp. The files manifest_capnp.rs and metadata_capnp.rs were generated with (Cap'n Proto version 0.10.4): ``` capnp compile -orust manifest.capnp capnp compile -orust metadata.capnp ``` , formatted with rustfmt and I have also added the following lines: ``` ``` to avoid warnings. Ideally, manifest_capnp.rs and metadata_capnp.rs should be automatically generated at build time. However, this means the capnp binary becomes a dependency for the build and I didn't want this. Besides, the metadata schemas are not expected to change frequently, so it's acceptable to manually regenerate the two rust files when this happens. The code is adapted from the puzzlefs FUSE driver [1]. The data structures required for the filesystem metadata are defined in types.rs. Each structure has a `from_capnp` method used to convert between the capnp memory layout and the native Rust structures. This leaves room for improvement [2]. The deserializing code also uses `NoAllocBufferSegments` which is not yet merged in capnproto-rust upstream [3]. When a new release is published, then we'll be able to use capnproto-rust in the kernel with only a few patches to it. inode.rs implements the PuzzleFS structure which contains a list of metadata layers. Link: https://github.com/project-machine/puzzlefs [1] Link: https://lore.kernel.org/rust-for-linux/[email protected]/ [2] Link: capnproto/capnproto-rust#423 [3] Signed-off-by: Ariel Miculas <[email protected]>
ariel-miculas
added a commit
to ariel-miculas/linux
that referenced
this pull request
Jul 13, 2023
…etadata Puzzlefs uses capnproto for storing the filesystem metadata, defined in manifest.capnp and metadata.capnp. The files manifest_capnp.rs and metadata_capnp.rs were generated with (Cap'n Proto version 0.10.4): ``` capnp compile -orust manifest.capnp capnp compile -orust metadata.capnp ``` , formatted with rustfmt and I have also added the following lines: ``` ``` to avoid warnings. Ideally, manifest_capnp.rs and metadata_capnp.rs should be automatically generated at build time. However, this means the capnp binary becomes a dependency for the build and I didn't want this. Besides, the metadata schemas are not expected to change frequently, so it's acceptable to manually regenerate the two rust files when this happens. The code is adapted from the puzzlefs FUSE driver [1]. The data structures required for the filesystem metadata are defined in types.rs. Each structure has a `from_capnp` method used to convert between the capnp memory layout and the native Rust structures. This leaves room for improvement [2]. The deserializing code also uses `NoAllocBufferSegments` which is not yet merged in capnproto-rust upstream [3]. When a new release is published, then we'll be able to use capnproto-rust in the kernel with only a few patches to it. inode.rs implements the PuzzleFS structure which contains a list of metadata layers. Link: https://github.com/project-machine/puzzlefs [1] Link: https://lore.kernel.org/rust-for-linux/[email protected]/ [2] Link: capnproto/capnproto-rust#423 [3] Signed-off-by: Ariel Miculas <[email protected]>
ariel-miculas
added a commit
to ariel-miculas/linux
that referenced
this pull request
Jul 19, 2023
…etadata Puzzlefs uses capnproto for storing the filesystem metadata, defined in manifest.capnp and metadata.capnp. The files manifest_capnp.rs and metadata_capnp.rs were generated with (Cap'n Proto version 0.10.4): ``` capnp compile -orust manifest.capnp capnp compile -orust metadata.capnp ``` , formatted with rustfmt and I have also added the following lines: ``` ``` to avoid warnings. Ideally, manifest_capnp.rs and metadata_capnp.rs should be automatically generated at build time. However, this means the capnp binary becomes a dependency for the build and I didn't want this. Besides, the metadata schemas are not expected to change frequently, so it's acceptable to manually regenerate the two rust files when this happens. The code is adapted from the puzzlefs FUSE driver [1]. The data structures required for the filesystem metadata are defined in types.rs. Each structure has a `from_capnp` method used to convert between the capnp memory layout and the native Rust structures. This leaves room for improvement [2]. The deserializing code also uses `NoAllocBufferSegments` which is not yet merged in capnproto-rust upstream [3]. When a new release is published, then we'll be able to use capnproto-rust in the kernel with only a few patches to it. inode.rs implements the PuzzleFS structure which contains a list of metadata layers. Link: https://github.com/project-machine/puzzlefs [1] Link: https://lore.kernel.org/rust-for-linux/[email protected]/ [2] Link: capnproto/capnproto-rust#423 [3] Signed-off-by: Ariel Miculas <[email protected]>
ariel-miculas
added a commit
to ariel-miculas/linux
that referenced
this pull request
Jul 27, 2023
…etadata Puzzlefs uses capnproto for storing the filesystem metadata, defined in manifest.capnp and metadata.capnp. The files manifest_capnp.rs and metadata_capnp.rs were generated with (Cap'n Proto version 0.10.4): ``` capnp compile -orust manifest.capnp capnp compile -orust metadata.capnp ``` , formatted with rustfmt and I have also added the following lines: ``` ``` to avoid warnings. Ideally, manifest_capnp.rs and metadata_capnp.rs should be automatically generated at build time. However, this means the capnp binary becomes a dependency for the build and I didn't want this. Besides, the metadata schemas are not expected to change frequently, so it's acceptable to manually regenerate the two rust files when this happens. The code is adapted from the puzzlefs FUSE driver [1]. The data structures required for the filesystem metadata are defined in types.rs. Each structure has a `from_capnp` method used to convert between the capnp memory layout and the native Rust structures. This leaves room for improvement [2]. The deserializing code also uses `NoAllocBufferSegments` which is not yet merged in capnproto-rust upstream [3]. When a new release is published, then we'll be able to use capnproto-rust in the kernel with only a few patches to it. inode.rs implements the PuzzleFS structure which contains a list of metadata layers. Link: https://github.com/project-machine/puzzlefs [1] Link: https://lore.kernel.org/rust-for-linux/[email protected]/ [2] Link: capnproto/capnproto-rust#423 [3] Signed-off-by: Ariel Miculas <[email protected]>
ariel-miculas
added a commit
to ariel-miculas/linux
that referenced
this pull request
Jul 27, 2023
…etadata Puzzlefs uses capnproto for storing the filesystem metadata, defined in manifest.capnp and metadata.capnp. The files manifest_capnp.rs and metadata_capnp.rs were generated with (Cap'n Proto version 0.10.4): ``` capnp compile -orust manifest.capnp capnp compile -orust metadata.capnp ``` , formatted with rustfmt and I have also added the following lines: ``` ``` to avoid warnings. Ideally, manifest_capnp.rs and metadata_capnp.rs should be automatically generated at build time. However, this means the capnp binary becomes a dependency for the build and I didn't want this. Besides, the metadata schemas are not expected to change frequently, so it's acceptable to manually regenerate the two rust files when this happens. The code is adapted from the puzzlefs FUSE driver [1]. The data structures required for the filesystem metadata are defined in types.rs. Each structure has a `from_capnp` method used to convert between the capnp memory layout and the native Rust structures. This leaves room for improvement [2]. The deserializing code also uses `NoAllocBufferSegments` which is not yet merged in capnproto-rust upstream [3]. When a new release is published, then we'll be able to use capnproto-rust in the kernel with only a few patches to it. inode.rs implements the PuzzleFS structure which contains a list of metadata layers. Link: https://github.com/project-machine/puzzlefs [1] Link: https://lore.kernel.org/rust-for-linux/[email protected]/ [2] Link: capnproto/capnproto-rust#423 [3] Signed-off-by: Ariel Miculas <[email protected]>
|
@dwrensha when could you take a look at this? |
|
I expect to have time in about two weeks. |
|
Thanks! |
|
I made |
ariel-miculas
added a commit
to ariel-miculas/linux
that referenced
this pull request
Aug 31, 2023
…etadata Puzzlefs uses capnproto for storing the filesystem metadata, defined in manifest.capnp and metadata.capnp. The files manifest_capnp.rs and metadata_capnp.rs were generated with (Cap'n Proto version 0.10.4): ``` capnp compile -o/home/amiculas/work/capnproto-rust/target/release/capnpc-rust manifest.capnp capnp compile -o/home/amiculas/work/capnproto-rust/target/release/capnpc-rust metadata.capnp ``` , formatted with rustfmt and I have also added the following lines: ``` #![allow(unreachable_pub)] ``` to avoid warnings. In `/home/amiculas/work/capnproto-rust` I have the latest version of master for the capnproto-rust repo. Ideally, manifest_capnp.rs and metadata_capnp.rs should be automatically generated at build time. However, this means the capnp binary becomes a dependency for the build and I didn't want this. Besides, the metadata schemas are not expected to change frequently, so it's acceptable to manually regenerate the two rust files when this happens. The code is adapted from the puzzlefs FUSE driver [1]. The data structures required for the filesystem metadata are defined in types.rs. Each structure has a `from_capnp` method used to convert between the capnp memory layout and the native Rust structures. This leaves room for improvement [2]. The deserializing code also uses `NoAllocBufferSegments`, which is merged in capnproto-rust upstream [3], but a new release hasn't yet been created. When the release is published, then we'll be able to use capnproto-rust in the kernel with only a few patches to it. inode.rs implements the PuzzleFS structure which contains a list of metadata layers. Link: https://github.com/project-machine/puzzlefs [1] Link: https://lore.kernel.org/rust-for-linux/[email protected]/ [2] Link: capnproto/capnproto-rust#423 [3] Signed-off-by: Ariel Miculas <[email protected]>
|
@ariel-miculas in #475 I change the |
|
Looks straightforward to me, thanks for the heads-up. |
ariel-miculas
added a commit
to ariel-miculas/linux
that referenced
this pull request
Mar 26, 2024
…etadata Puzzlefs uses capnproto for storing the filesystem metadata, defined in manifest.capnp and metadata.capnp. The files manifest_capnp.rs and metadata_capnp.rs were generated with (Cap'n Proto version 0.10.4): ``` capnp compile -orust manifest.capnp capnp compile -orust metadata.capnp ``` , formatted with rustfmt and I have also added the following lines: ``` ``` to avoid warnings. Ideally, manifest_capnp.rs and metadata_capnp.rs should be automatically generated at build time. However, this means the capnp binary becomes a dependency for the build and I didn't want this. Besides, the metadata schemas are not expected to change frequently, so it's acceptable to manually regenerate the two rust files when this happens. The code is adapted from the puzzlefs FUSE driver [1]. The data structures required for the filesystem metadata are defined in types.rs. Each structure has a `from_capnp` method used to convert between the capnp memory layout and the native Rust structures. This leaves room for improvement [2]. The deserializing code also uses `NoAllocBufferSegments` which is not yet merged in capnproto-rust upstream [3]. When a new release is published, then we'll be able to use capnproto-rust in the kernel with only a few patches to it. inode.rs implements the PuzzleFS structure which contains a list of metadata layers. Link: https://github.com/project-machine/puzzlefs [1] Link: https://lore.kernel.org/rust-for-linux/[email protected]/ [2] Link: capnproto/capnproto-rust#423 [3] Signed-off-by: Ariel Miculas <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
It is similar to BufferSegments, but it avoids any allocations.