Rework CLI interface #263

mkroening · 2022-01-01T20:35:33Z

Reopens #243, which was closed accidentally.

This heavily reworks the CLI interface using StructOpt as well as Params for creating uhyve instances programmatically.

See the commit messages for details.

codecov · 2022-01-01T20:36:53Z

Codecov Report

Merging #263 (2612691) into master (eae6749) will decrease coverage by 0.70%.
The diff coverage is 24.24%.

@@            Coverage Diff             @@
##           master     #263      +/-   ##
==========================================
- Coverage   64.15%   63.44%   -0.71%     
==========================================
  Files          16       16              
  Lines        2427     2380      -47     
==========================================
- Hits         1557     1510      -47     
  Misses        870      870

Impacted Files	Coverage Δ
src/bin/uhyve.rs	`0.50% <0.00%> (+0.03%)`	⬆️
src/lib.rs	`100.00% <ø> (ø)`
src/vm.rs	`76.17% <33.33%> (-2.62%)`	⬇️
src/params.rs	`67.16% <67.16%> (ø)`
src/linux/uhyve.rs	`66.47% <94.11%> (-0.85%)`	⬇️
src/linux/vcpu.rs	`70.35% <100.00%> (+0.37%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eae6749...2612691. Read the comment docs.

stlankes · 2022-01-03T21:36:32Z

@mkroening @jbreitbart @jounathaen are missing on the list of authors. Why do we enable "Kernel Samepage Merging" on default?

mkroening · 2022-01-03T22:37:43Z

I rebased the PR and opened #277.

We can discuss KSM with the rest of the CLI changes on friday, if you don't mind. :)

277: Cargo.toml: Add authors r=stlankes a=mkroening I added `@mkroening,` `@jbreitbart` and `@jounathaen` as requested in #263 (comment). Co-authored-by: Martin Kröning <[email protected]>

stlankes · 2022-01-03T22:38:17Z

I am on vacation :-)

mkroening · 2022-01-03T22:41:44Z

Ah, I see. Have a nice vacation then! :D

Maybe I'll find some motivation in the next days to write some prose on all of these changes then. This PR should not be very urgent, right?

These tests are obsolete once the guest memory minimum is enforced. Replacing these tests is tracked in hermit-os#265.

mkroening · 2022-01-07T21:09:50Z

I expanded the commit messages. See those for details on each commit.

Regarding turning on KSM by default: As far as I know this matches QEMU's default behavior. Also as KSM has been designed especially for hypervisors, I see no reason not to enable KSM by default. Our --no-thp matches -redhat-disable-THP, --no-ksm matches -redhat-disable-KSM.

stlankes · 2022-01-10T10:25:56Z

It takes time... For large VMs is this useful. For small binaries with small memory footprints, I would say, this is just overhead with nearly no benefit. But this is more my feeling....

jbreitbart · 2022-01-10T12:04:30Z

It takes time... For large VMs is this useful. For small binaries with small memory footprints, I would say, this is just overhead with nearly no benefit. But this is more my feeling....

Now if only someone would have added a way to benchmark functionality 😜

mkroening · 2022-01-10T17:11:20Z

Very good point. I'll be doing some benchmarks then. :D

mkroening · 2022-01-12T21:58:55Z

@stlankes, I did some benchmarks and further investigation into KSM.

The KSM daemon is disabled by default. Without having activated the KSM daemon, no overhead is measurable. In that case, the kernel only registers the memory range for whenever the KSM daemon is started in the future.

Activating the KSM daemon had no effect on hello_world. rusty-demo's runtime changed from 223.74 ms to 226.37 ms (an increase of 2.63 ms or 1,1 %).

Since the KSM daemon has to be enabled explicitly system-wide before having any effect, us advising KSM regions is a sensible default in my opinion. The user could still opt-out application-wise via --no-ksm.

What do you think?

mkroening · 2022-01-21T15:46:43Z

I'll compare the default KSM setting with Firecracker and QEMU's microvm.

* Have structs for args to modularize arg handling * Improve argument docs * Improve error messages on faulty args * Colorize help output * Rename `--disable-hugepages` to `--no-thp` * Corresponds to QEMU's `-redhat-disable-THP` * Remove `HERMIT_HUGEPAGES` * Replace `--mergeable` with `--ksm` * Corresponds to QEMU's `-redhat-disable-KSM` * Remove `HERMIT_MERGEABLE` * Remove `HERMIT_VERBOSE` * Rename `-c, --cpus` to `-c, --cpu-count` * Rename `HERMIT_CPUS` to `HERMIT_CPU_COUNT` * Rename `-s, --gdb_port` to `-s, --gdb-port` * Rename `-m, --memsize` to `-m, --memory-size` * Rename `HERMIT_MEM` to `HERMIT_MEMORY_SIZE` * Disable `--nic` since networking is currently not supported ``` uhyve 0.0.29 Stefan Lankes <[email protected]>, Martin Kröning <[email protected]>, Jens Breitbart <[email protected]>, Jonathan Klimt <[email protected]> A minimal hypervisor for RustyHermit USAGE: uhyve [OPTIONS] <KERNEL> [KERNEL_ARGS]... ARGS: <KERNEL> The kernel to execute <KERNEL_ARGS>... Arguments to forward to the kernel OPTIONS: -h, --help Print help information -s, --gdb-port <GDB_PORT> GDB server port Starts a GDB server on the provided port and waits for a connection. [env: HERMIT_GDB_PORT=] -v, --verbose Print kernel messages -V, --version Print version information MEMORY: --ksm Kernel Samepage Merging Advise the kernel to enable Kernel Samepage Merging [KSM] on the virtual RAM. [KSM]: https://www.kernel.org/doc/html/latest/admin-guide/mm/ksm.html -m, --memory-size <MEMORY_SIZE> Guest RAM size [env: HERMIT_MEMORY_SIZE=] [default: "64.00 MiB"] --no-thp No Transparent Hugepages Don't advise the kernel to enable Transparent Hugepages [THP] on the virtual RAM. [THP]: https://www.kernel.org/doc/html/latest/admin-guide/mm/transhuge.html CPU: -a, --affinity <CPUs> Bind guest vCPUs to host cpus A list of host CPU numbers onto which the guest vCPUs should be bound to obtain performance benefits. List items may be single numbers or inclusive ranges. List items may be separated with commas or spaces. # Examples * `--affinity "0 1 2"` * `--affinity 0-1,2` -c, --cpu-count <CPU_COUNT> Number of guest CPUs [env: HERMIT_CPU_COUNT=] [default: 1] ``` ``` uhyve 0.0.29 Stefan Lankes <[email protected]> Martin Kröning <[email protected]> Jens Breitbart <[email protected]> Jonathan Klimt <[email protected]> USAGE: uhyve [FLAGS] [OPTIONS] <KERNEL> [ARGUMENTS]... FLAGS: --disable-hugepages Disable the usage of huge pages --mergeable Enable kernel feature to merge same pages -v, --verbose Print also kernel messages -h, --help Prints help information -V, --version Prints version information OPTIONS: -c, --cpus <CPUS> Number of guest processors [env: HERMIT_CPUS=] -a, --affinity <cpulist> A list of CPUs delimited by commas onto which the virtual CPUs should be bound. This may improve performance. -s, --gdb_port <GDB_PORT> Enables GDB-Stub on given port [env: HERMIT_GDB_PORT=] -m, --memsize <MEM> Memory size of the guest [env: HERMIT_MEM=] --nic <NETIF> Name of the network interface [env: HERMIT_NETIF=] ARGS: <KERNEL> Sets path to the kernel <ARGUMENTS>... Arguments of the unikernel ```

This improves constructing Uhyve instances programmatically.

This disables GDB and network parameters on macos.

This allows supplying kernel arguments programatically and unifies argument parsing in the frontend using clap. This avoids parsing kernel arguments from the environment unconditionally.

mkroening · 2022-01-21T17:38:22Z

I disabled KSM per default.

mkroening · 2022-01-21T17:44:05Z

bors r+

bors · 2022-01-21T17:51:33Z

Build succeeded:

mkroening force-pushed the structopt branch 9 times, most recently from 80b368c to 246ba1c Compare January 2, 2022 16:27

mkroening marked this pull request as ready for review January 2, 2022 18:52

mkroening force-pushed the structopt branch from 246ba1c to f7ed7ec Compare January 2, 2022 18:57

mkroening mentioned this pull request Jan 3, 2022

Bump clap from 2.34.0 to 3.0.0 #270

Closed

mkroening force-pushed the structopt branch 7 times, most recently from 7f10994 to 3167651 Compare January 3, 2022 20:04

mkroening force-pushed the structopt branch from 9e42710 to 937bd2d Compare January 3, 2022 22:30

mkroening mentioned this pull request Jan 3, 2022

Cargo.toml: Add authors #277

Merged

bors bot added a commit that referenced this pull request Jan 3, 2022

Merge #277

eae6749

277: Cargo.toml: Add authors r=stlankes a=mkroening I added `@mkroening,` `@jbreitbart` and `@jounathaen` as requested in #263 (comment). Co-authored-by: Martin Kröning <[email protected]>

mkroening added 2 commits January 4, 2022 00:06

linux/uhyve: Use cmp::min

b6b1193

Remove memory size unit tests

80f9c2a

These tests are obsolete once the guest memory minimum is enforced. Replacing these tests is tracked in hermit-os#265.

mkroening force-pushed the structopt branch from 937bd2d to 7da74cf Compare January 3, 2022 23:07

mkroening force-pushed the structopt branch 3 times, most recently from 82e75e2 to fbbeecb Compare January 7, 2022 21:09

mkroening mentioned this pull request Jan 10, 2022

Bump clap from 2.34.0 to 3.0.5 #279

Closed

mkroening requested review from stlankes and jounathaen January 10, 2022 10:04

mkroening force-pushed the structopt branch from fbbeecb to 35759d8 Compare January 17, 2022 22:36

mkroening mentioned this pull request Jan 17, 2022

Bump clap from 2.34.0 to 3.0.7 #282

Closed

mkroening added 4 commits January 21, 2022 18:34

Rework Uhyve construction params

435b586

This improves constructing Uhyve instances programmatically.

Params: Enable parameter only on supported OS

542ef80

This disables GDB and network parameters on macos.

Supply kernel arguments as parameter

2612691

This allows supplying kernel arguments programatically and unifies argument parsing in the frontend using clap. This avoids parsing kernel arguments from the environment unconditionally.

mkroening force-pushed the structopt branch from 35759d8 to 2612691 Compare January 21, 2022 17:37

hermit-os deleted a comment from bors bot Jan 21, 2022

bors bot merged commit 2d87f9c into hermit-os:master Jan 21, 2022

mkroening deleted the structopt branch January 21, 2022 17:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rework CLI interface #263

Rework CLI interface #263

mkroening commented Jan 1, 2022 •

edited

Loading

codecov bot commented Jan 1, 2022 •

edited

Loading

stlankes commented Jan 3, 2022

mkroening commented Jan 3, 2022

stlankes commented Jan 3, 2022

mkroening commented Jan 3, 2022

mkroening commented Jan 7, 2022 •

edited

Loading

stlankes commented Jan 10, 2022

jbreitbart commented Jan 10, 2022

mkroening commented Jan 10, 2022

mkroening commented Jan 12, 2022

mkroening commented Jan 21, 2022

mkroening commented Jan 21, 2022

mkroening commented Jan 21, 2022

bors bot commented Jan 21, 2022

Rework CLI interface #263

Rework CLI interface #263

Conversation

mkroening commented Jan 1, 2022 • edited Loading

codecov bot commented Jan 1, 2022 • edited Loading

Codecov Report

stlankes commented Jan 3, 2022

mkroening commented Jan 3, 2022

stlankes commented Jan 3, 2022

mkroening commented Jan 3, 2022

mkroening commented Jan 7, 2022 • edited Loading

stlankes commented Jan 10, 2022

jbreitbart commented Jan 10, 2022

mkroening commented Jan 10, 2022

mkroening commented Jan 12, 2022

mkroening commented Jan 21, 2022

mkroening commented Jan 21, 2022

mkroening commented Jan 21, 2022

bors bot commented Jan 21, 2022

mkroening commented Jan 1, 2022 •

edited

Loading

codecov bot commented Jan 1, 2022 •

edited

Loading

mkroening commented Jan 7, 2022 •

edited

Loading