labs: Makefile: Add scp and ssh targets for easier module transfer on images with networking #240
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sergiu121
requested changes
Mar 4, 2021
tools/labs/Makefile
Outdated
| @@ -31,6 +33,24 @@ copy: $(YOCTO_IMAGE) | |||
| sudo umount $(TEMPDIR) | |||
| rmdir $(TEMPDIR) | |||
|
|
|||
| scp: ip | |||
| scp -r skels root@$(shell cat .ip): | |||
There was a problem hiding this comment.
scp should only copy the .ko files, like the make copy command.
There was a problem hiding this comment.
scp can't create parent directories and we don't have rsync on the provided images. I can do it by sshing first to create the required directories and then using scp to copy the files. Would that be ok?
tools/labs/Makefile
Outdated
| GUEST_IP=`arp -e -n -i tap0 | tail -1 | cut -d ' ' -f 1`; \ | ||
| done; \ | ||
| echo ""; \ | ||
| echo -n $$GUEST_IP > .ip |
There was a problem hiding this comment.
Why make this ip file hidden? Why not just output the IP value?
There was a problem hiding this comment.
I extracted the ip target so I don't have duplicate code for finding the ip in the scp and ssh targets. Saving the ip in a hidden file was the only way I could think of making it accesible to the other targets. Do you have any suggestions about this?
There was a problem hiding this comment.
You can do a make -s ip to get the output.
There was a problem hiding this comment.
@VladNastase You could add a script in tools/labs/qemu to get the ip address and call that for the scp and ssh targets, e.g.
ssh:
IP=$$(qemu/get_vm_ip.sh); if [ -n "$IP" ]; then ssh ...
… images with networking
|
Hi, I found a reference to use I suppose this PR is a prerequisite? Or is there any "one-liner" command I don't know about that I can use to copy files from host to the emulator? Thank you! |
vlasakm
added a commit
to vlasakm/linux
that referenced
this pull request
Jan 12, 2023
TL;DR: In one window run `make -j$(ncproc) console` and `cd` to some lab's subdirectory (`skels/...`). In second window `cd` to matching `skels/...` subdirectory, edit source files and compile with something like `kmake` (`alias kmake='make -C "$HOME/src/linux/" M="$(pwd)"'`) as needed. The `skels` directory is shared between the host and the guest, thus in the first window, so you can just `insmod` and `rmmod` the compiled modules. You can kill the VM by `CTRL-a x` (if you made some writes from the VM it might be a good idea to run `sync` first). Samba and QEMU are required. Full description: To be honest I don't like the current QEMU setup. I am sure there are things it does that I don't understand yet, because I have not yet finished all the labs, but in any case I think that the setup can be improved. Some things in particular I don't like about the current setup: - "Huge" opaque `.ext4` images are used, even though the contents of the root file system are not that large. - While running QEMU newly built modules can't be copied to the image. - Mounting and unmounting the `.ext4` image for copying the modules requires `sudo`. - The networking setup seems too complex, requires `sudo` and was broken (at least for me - IIRC I didn't get IP through DHCP), thus I also didn't get `ssh` to work. I also seem to be not the only one having issues with this: linux-kernel-labs#240 (comment) - `dnsmasq` and `nttctp` mostly don't start correctly (they are not killed correctly by the previous run) - this isn't a problem on my end, as demonstrated by the output at https://linux-kernel-labs.github.io/refs/heads/master/info/vm.html, which shows the same issues. - Running `minicom` is required to access the serial port, thus at least two terminals are required for working with the VM (not a huge problem for me personally, since I use `tmux`, but I know some people complain about this). The setup also seems unnecessarily complex. - I remember a lot of the `.mon` `.pts` files being left uncleaned in some cases. - I recall warnigns about some of the entries added to `/etc/inittab` being ignored. - Even though root login requires no password I have to enter the `root` username. In this commit I introdoce an alternative way of running QEMU through a new `run-qemu.sh` script. The setup is laregely independent and its user interface consists of `console` and `gui` targets. I tried to make the script parameterizable through environment variables (inherited from Make variables), though it may be argued that the default values should be encoded in Makefile rather than in the script like they are now. I have no strong opinions about that, it's' just that the current state allows running the script in standalone fashion. What the setup brings: - A rootfs is extracted from the official Yocto Project tarball and kept in a directory that is shared through [Samba as network share](https://www.kernel.org/doc/html/latest/filesystems/cifs/cifsroot.html). The `skels` directory is shared as well. Thus the modules can be freely tweaked / compiled / ran from either the host or guest. - The QEMU stdio serial console setup is used (`ttyS0` on the kernel side). This means that running QEMU results in the serial console being mapped directly to standard input and output of the terminal - `minicom` is not needed. This is the console mode (`make console`). - The setup allows also allows the virtual machine to be run in graphical mode (`make gui`). - Root is logged in automatically in `console` mode (though similar thing could be done for the `gui` mode). - Although Samba (`smbd`) is required, `sudo` or root access is not. - Networking through QEMU's default [SLIRP backend](https://wiki.qemu.org/Documentation/Networking#User_Networking_.28SLIRP.29). DHCP is handled by the kernel, which overcomes some problems I had with the System V init system in the Yocto images. - The compilation can largely be done with something like this `kmake` alias: `alias kmake='make -C "$HOME/src/linux/" M="$(pwd)"'` (customize as needed). Though this is not enough for some labs (I no longer remember the details, but I think it was some of the earlier labs which had dependencies between modules, I think I used the classic `make build` for that. Known issues: - SSH support is currently missing. This both requires more featureful Yocto images and is IMO unnecessary, since it wouldn't bring much benefit over the console mode. Though it can be easily achieved by using QEMU option like `-nic user,hostfwd=tcp::2222-:22`, which would allow SSH'ing into the guest by something like `ssh -p 2222 root@localhost`. - I used a slightly less advanced setup while doing the labs, so the lab workflow with this particular setup is largely untested. There may be problems with file permissions due to the samba share. - The guest seems to fail to shutdown correctly in a timely manner. I just took the habbit of killing qemu with `CTRL-a` followed by `x`, potentially running `sync` first to ensure my work is saved (though rarely did I actually modify anything on the guest side). [The former setup](720bd64) I used contains some details of the SSH setup if anyone is interested in that. It was the basis for this PR, so some ideas can be seen there (Samba share for `skels`), but I didn't take particular care with [the kernel config](0290919) and the automounting didn't really work (the `init` would try to mount the filesystem before networking was up). What I evaluated and didn't use in the end: - At first I tried to extend my former setup by just automounting the Samba share. I didn't manage to do this - the (non)workings of init scripts seem to be beyond me. If anyone is interested here are a few pointers: [[1]](https://unix.stackexchange.com/questions/169697/how-does-netdev-mount-option-in-etc-fstab-work), `/etc/inittab`, `/etc/init.d/mountall.sh`, `/etc/init.d/mountnfs.sh`. - I tried using `9p` [[2]](https://wiki.qemu.org/Documentation/9p), [[3]](https://wiki.qemu.org/Documentation/9psetup) [[4]](https://wiki.qemu.org/Documentation/9p_root_fs) which is built into QEMU and can be compiled into the kernel. With `mapped-xattr` security model it would be too cumbersome to create the rootfs, and `passthrough` would require root privileges. It is also very slow. There are also some problems with trying to use it as rootfs, maybe specific to `linux-kernel-labs` kernel version or config. Ask me if interested. [[5]](https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg07184.html) [[6]](https://lore.kernel.org/linux-fsdevel/[email protected]/) [[7]](https://lore.kernel.org/all/[email protected]/T/) - QEMU has an option to setup the Samba share on its own, though I found a custom config (based on the QEMU one) to be easier - allows customization like multiple shares, unix extensions, different port, etc.
vlasakm
added a commit
to vlasakm/linux
that referenced
this pull request
Jan 13, 2023
TL;DR: In one window run `make -j$(ncproc) console` and `cd` to some lab's subdirectory (`skels/...`). In second window `cd` to matching `skels/...` subdirectory, edit source files and compile with something like `kmake` (`alias kmake='make -C "$HOME/src/linux/" M="$(pwd)"'`) as needed. The `skels` directory is shared between the host and the guest, thus in the first window, so you can just `insmod` and `rmmod` the compiled modules. You can kill the VM by `CTRL-a x` (if you made some writes from the VM it might be a good idea to run `sync` first). Samba and QEMU are required. Full description: To be honest I don't like the current QEMU setup. I am sure there are things it does that I don't understand yet, because I have not yet finished all the labs, but in any case I think that the setup can be improved. Some things in particular I don't like about the current setup: - "Huge" opaque `.ext4` images are used, even though the contents of the root file system are not that large. - While running QEMU newly built modules can't be copied to the image. - Mounting and unmounting the `.ext4` image for copying the modules requires `sudo`. - The networking setup seems too complex, requires `sudo` and was broken (at least for me - IIRC I didn't get IP through DHCP), thus I also didn't get `ssh` to work. I also seem to be not the only one having issues with this: linux-kernel-labs#240 (comment) - `dnsmasq` and `nttctp` mostly don't start correctly (they are not killed correctly by the previous run) - this isn't a problem on my end, as demonstrated by the output at https://linux-kernel-labs.github.io/refs/heads/master/info/vm.html, which shows the same issues. - Running `minicom` is required to access the serial port, thus at least two terminals are required for working with the VM (not a huge problem for me personally, since I use `tmux`, but I know some people complain about this). The setup also seems unnecessarily complex. - I remember a lot of the `.mon` `.pts` files being left uncleaned in some cases. - I recall warnigns about some of the entries added to `/etc/inittab` being ignored. - Even though root login requires no password I have to enter the `root` username. In this commit I introdoce an alternative way of running QEMU through a new `run-qemu.sh` script. The setup is laregely independent and its user interface consists of `console` and `gui` targets. I tried to make the script parameterizable through environment variables (inherited from Make variables), though it may be argued that the default values should be encoded in Makefile rather than in the script like they are now. I have no strong opinions about that, it's' just that the current state allows running the script in standalone fashion. What the setup brings: - A rootfs is extracted from the official Yocto Project tarball and kept in a directory that is shared through [Samba as network share](https://www.kernel.org/doc/html/latest/filesystems/cifs/cifsroot.html). The `skels` directory is shared as well. Thus the modules can be freely tweaked / compiled / ran from either the host or guest. - The QEMU stdio serial console setup is used (`ttyS0` on the kernel side). This means that running QEMU results in the serial console being mapped directly to standard input and output of the terminal - `minicom` is not needed. This is the console mode (`make console`). - The setup allows also allows the virtual machine to be run in graphical mode (`make gui`). - Root is logged in automatically in `console` mode (though similar thing could be done for the `gui` mode). - Although Samba (`smbd`) is required, `sudo` or root access is not. - Networking through QEMU's default [SLIRP backend](https://wiki.qemu.org/Documentation/Networking#User_Networking_.28SLIRP.29). DHCP is handled by the kernel, which overcomes some problems I had with the System V init system in the Yocto images. - The compilation can largely be done with something like this `kmake` alias: `alias kmake='make -C "$HOME/src/linux/" M="$(pwd)"'` (customize as needed). Though this is not enough for some labs (I no longer remember the details, but I think it was some of the earlier labs which had dependencies between modules, I think I used the classic `make build` for that. Known issues: - SSH support is currently missing. This both requires more featureful Yocto images and is IMO unnecessary, since it wouldn't bring much benefit over the console mode. Though it can be easily achieved by using QEMU option like `-nic user,hostfwd=tcp::2222-:22`, which would allow SSH'ing into the guest by something like `ssh -p 2222 root@localhost`. - I used a slightly less advanced setup while doing the labs, so the lab workflow with this particular setup is largely untested. There may be problems with file permissions due to the samba share. - The guest seems to fail to shutdown correctly in a timely manner. I just took the habbit of killing qemu with `CTRL-a` followed by `x`, potentially running `sync` first to ensure my work is saved (though rarely did I actually modify anything on the guest side). [The former setup](720bd64) I used contains some details of the SSH setup if anyone is interested in that. It was the basis for this PR, so some ideas can be seen there (Samba share for `skels`), but I didn't take particular care with [the kernel config](0290919) and the automounting didn't really work (the `init` would try to mount the filesystem before networking was up). What I evaluated and didn't use in the end: - At first I tried to extend my former setup by just automounting the Samba share. I didn't manage to do this - the (non)workings of init scripts seem to be beyond me. If anyone is interested here are a few pointers: [[1]](https://unix.stackexchange.com/questions/169697/how-does-netdev-mount-option-in-etc-fstab-work), `/etc/inittab`, `/etc/init.d/mountall.sh`, `/etc/init.d/mountnfs.sh`. - I tried using `9p` [[2]](https://wiki.qemu.org/Documentation/9p), [[3]](https://wiki.qemu.org/Documentation/9psetup) [[4]](https://wiki.qemu.org/Documentation/9p_root_fs) which is built into QEMU and can be compiled into the kernel. With `mapped-xattr` security model it would be too cumbersome to create the rootfs, and `passthrough` would require root privileges. It is also very slow. There are also some problems with trying to use it as rootfs, maybe specific to `linux-kernel-labs` kernel version or config. Ask me if interested. [[5]](https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg07184.html) [[6]](https://lore.kernel.org/linux-fsdevel/[email protected]/) [[7]](https://lore.kernel.org/all/[email protected]/T/) - QEMU has an option to setup the Samba share on its own, though I found a custom config (based on the QEMU one) to be easier - allows customization like multiple shares, unix extensions, different port, etc.
dbaluta
pushed a commit
that referenced
this pull request
Jan 16, 2023
TL;DR: In one window run `make -j$(ncproc) console` and `cd` to some lab's subdirectory (`skels/...`). In second window `cd` to matching `skels/...` subdirectory, edit source files and compile with something like `kmake` (`alias kmake='make -C "$HOME/src/linux/" M="$(pwd)"'`) as needed. The `skels` directory is shared between the host and the guest, thus in the first window, so you can just `insmod` and `rmmod` the compiled modules. You can kill the VM by `CTRL-a x` (if you made some writes from the VM it might be a good idea to run `sync` first). Samba and QEMU are required. Full description: To be honest I don't like the current QEMU setup. I am sure there are things it does that I don't understand yet, because I have not yet finished all the labs, but in any case I think that the setup can be improved. Some things in particular I don't like about the current setup: - "Huge" opaque `.ext4` images are used, even though the contents of the root file system are not that large. - While running QEMU newly built modules can't be copied to the image. - Mounting and unmounting the `.ext4` image for copying the modules requires `sudo`. - The networking setup seems too complex, requires `sudo` and was broken (at least for me - IIRC I didn't get IP through DHCP), thus I also didn't get `ssh` to work. I also seem to be not the only one having issues with this: #240 (comment) - `dnsmasq` and `nttctp` mostly don't start correctly (they are not killed correctly by the previous run) - this isn't a problem on my end, as demonstrated by the output at https://linux-kernel-labs.github.io/refs/heads/master/info/vm.html, which shows the same issues. - Running `minicom` is required to access the serial port, thus at least two terminals are required for working with the VM (not a huge problem for me personally, since I use `tmux`, but I know some people complain about this). The setup also seems unnecessarily complex. - I remember a lot of the `.mon` `.pts` files being left uncleaned in some cases. - I recall warnigns about some of the entries added to `/etc/inittab` being ignored. - Even though root login requires no password I have to enter the `root` username. In this commit I introdoce an alternative way of running QEMU through a new `run-qemu.sh` script. The setup is laregely independent and its user interface consists of `console` and `gui` targets. I tried to make the script parameterizable through environment variables (inherited from Make variables), though it may be argued that the default values should be encoded in Makefile rather than in the script like they are now. I have no strong opinions about that, it's' just that the current state allows running the script in standalone fashion. What the setup brings: - A rootfs is extracted from the official Yocto Project tarball and kept in a directory that is shared through [Samba as network share](https://www.kernel.org/doc/html/latest/filesystems/cifs/cifsroot.html). The `skels` directory is shared as well. Thus the modules can be freely tweaked / compiled / ran from either the host or guest. - The QEMU stdio serial console setup is used (`ttyS0` on the kernel side). This means that running QEMU results in the serial console being mapped directly to standard input and output of the terminal - `minicom` is not needed. This is the console mode (`make console`). - The setup allows also allows the virtual machine to be run in graphical mode (`make gui`). - Root is logged in automatically in `console` mode (though similar thing could be done for the `gui` mode). - Although Samba (`smbd`) is required, `sudo` or root access is not. - Networking through QEMU's default [SLIRP backend](https://wiki.qemu.org/Documentation/Networking#User_Networking_.28SLIRP.29). DHCP is handled by the kernel, which overcomes some problems I had with the System V init system in the Yocto images. - The compilation can largely be done with something like this `kmake` alias: `alias kmake='make -C "$HOME/src/linux/" M="$(pwd)"'` (customize as needed). Though this is not enough for some labs (I no longer remember the details, but I think it was some of the earlier labs which had dependencies between modules, I think I used the classic `make build` for that. Known issues: - SSH support is currently missing. This both requires more featureful Yocto images and is IMO unnecessary, since it wouldn't bring much benefit over the console mode. Though it can be easily achieved by using QEMU option like `-nic user,hostfwd=tcp::2222-:22`, which would allow SSH'ing into the guest by something like `ssh -p 2222 root@localhost`. - I used a slightly less advanced setup while doing the labs, so the lab workflow with this particular setup is largely untested. There may be problems with file permissions due to the samba share. - The guest seems to fail to shutdown correctly in a timely manner. I just took the habbit of killing qemu with `CTRL-a` followed by `x`, potentially running `sync` first to ensure my work is saved (though rarely did I actually modify anything on the guest side). [The former setup](vlasakm@720bd64) I used contains some details of the SSH setup if anyone is interested in that. It was the basis for this PR, so some ideas can be seen there (Samba share for `skels`), but I didn't take particular care with [the kernel config](vlasakm@0290919) and the automounting didn't really work (the `init` would try to mount the filesystem before networking was up). What I evaluated and didn't use in the end: - At first I tried to extend my former setup by just automounting the Samba share. I didn't manage to do this - the (non)workings of init scripts seem to be beyond me. If anyone is interested here are a few pointers: [[1]](https://unix.stackexchange.com/questions/169697/how-does-netdev-mount-option-in-etc-fstab-work), `/etc/inittab`, `/etc/init.d/mountall.sh`, `/etc/init.d/mountnfs.sh`. - I tried using `9p` [[2]](https://wiki.qemu.org/Documentation/9p), [[3]](https://wiki.qemu.org/Documentation/9psetup) [[4]](https://wiki.qemu.org/Documentation/9p_root_fs) which is built into QEMU and can be compiled into the kernel. With `mapped-xattr` security model it would be too cumbersome to create the rootfs, and `passthrough` would require root privileges. It is also very slow. There are also some problems with trying to use it as rootfs, maybe specific to `linux-kernel-labs` kernel version or config. Ask me if interested. [[5]](https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg07184.html) [[6]](https://lore.kernel.org/linux-fsdevel/[email protected]/) [[7]](https://lore.kernel.org/all/[email protected]/T/) - QEMU has an option to setup the Samba share on its own, though I found a custom config (based on the QEMU one) to be easier - allows customization like multiple shares, unix extensions, different port, etc. Signed-off-by: Michal Vlasák <[email protected]> Signed-off-by: Daniel Baluta <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.