Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DietPi-Software | WireGuard: Now available for install #2398

Merged
merged 17 commits into from
Jan 11, 2019
Merged

DietPi-Software | WireGuard: Now available for install #2398

merged 17 commits into from
Jan 11, 2019

Conversation

MichaIng
Copy link
Owner

@MichaIng MichaIng commented Jan 6, 2019

Status: WIP

NB: Currently only works on x86_64 and RPi's. Since kernel headers are required, we need to add those for other devices, before the WireGuard kernel module can be compiled.

Testing:

  • VM Stretch
  • VM Jessie
  • VM Buster

Reference: https://github.com/Fourdee/DietPi/issues/2052

Commit list/description:

  • DietPi-Software | WireGuard: Now available for install

+ DietPi-Software | WireGuard: Now available for install
+ DietPi-Software | WireGuard: APT update after adding sid repo
+ DietPi-Software | WireGuard: Fix sources list location
+ DietPi-Software | WireGuard: Add "qrencode" install, which allows easy mobile client setup via QR code on console
@MichaIng
Copy link
Owner Author

MichaIng commented Jan 7, 2019

🈴 Buster x86_64: Module build is skipped, since current kernel and headers from repo do not match:

root@VM-Buster:~# dpkg -l | grep '[[:blank:]]linux'
ii  linux-base                    4.5                         all          Linux image base package
ii  linux-compiler-gcc-8-x86      4.19.12-1                   amd64        Compiler for Linux on x86 (meta-package)
ii  linux-headers-4.19.0-1-amd64  4.19.12-1                   amd64        Header files for Linux 4.19.0-1-amd64
ii  linux-headers-4.19.0-1-common 4.19.12-1                   all          Common header files for Linux 4.19.0-1
ii  linux-headers-amd64           4.19+101                    amd64        Header files for Linux amd64 configuration (meta-package)
ii  linux-image-4.18.0-2-amd64    4.18.10-2+b1                amd64        Linux 4.18 for 64-bit PCs
ii  linux-image-amd64             4.18+99                     amd64        Linux for 64-bit PCs (meta-package)
ii  linux-kbuild-4.19             4.19.12-1                   amd64        Kbuild infrastructure for Linux 4.19

G_AGDUG resolves it, since it upgrades the kernel package (linux-image-amd64), regardless of changed dependency (linux-image-4.19.0-1-amd64). This even seems to create the wireguard module automatically as an install trigger. Makes somehow sense, so if the kernel package is updated, one does not always need to manually update the modules as well.


This is a general issue, since in most cases we have the kernel installed via meta package, which is not updated via ordinary G_AGUG, due to changed dependency (new actual kernel package).
And the header packages do not depend on the kernel packages.
So either we need to find the exact header package that matches the currently installed kernel, or we need to run G_AGDUG to assure up-to-date kernel package first.

Second issue: After updating the kernel module, a reboot is required to apply it to the system, okay no big issue, since on interactive call, dietpi-software anyway triggers a reboot.

@Fourdee
Perhaps on update and software install start, we should do G_AGDUG instead of G_AGUG? This is theoretically a bid more risky, due to changed dependencies (and kernel upgrades), but should usually not cause issues on stable repo. Hmm, it would currently regularly cause issues on my Buster RPi (not Buster VM), due to ongoing dependency inconsistencies in Raspbian Buster...
Perhaps only force a kernel package update + dependencies on WireGuard install as well... G_AGI linux-image-amd64 should do it.

🈺 Okay G_AGI linux-image-amd64 works, somehow:

Loading new wireguard-0.0.20181218 DKMS files...
Building for 4.18.0-2-amd64 4.19.0-1-amd64
Module build for kernel 4.18.0-2-amd64 was skipped since the
kernel headers for this kernel does not seem to be installed.
Building initial module for 4.19.0-1-amd64
Done.
  • It automatically selects the kernel headers, which are installed. But it looks like, if old headers were installed before, it would build the module for the old kernel. Question is now, after reboot, when new kernel is active, the module is missing for it, of if the installer creates the module for all installed headers.

Partly resolved with: Fourdee@3ccd572

  • Another test required to check, if module is build for a freshly installed kernel in every case.
    🈯️ Jep, module is compiled for all available kernel header versions, so no issue, if older headers were installed before:
Building initial module for 4.9.0-8-amd64
Done.

wireguard.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/4.9.0-8-amd64/kernel/net/

depmod....

DKMS: install completed.
Building initial module for 4.18.0-0.bpo.3-amd64
Done.

wireguard.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/4.18.0-0.bpo.3-amd64/kernel/net/

depmod....

DKMS: install completed.

🈺 On RPi headers are no meta packages, thus don't need to be installed manually. But an issue occurs, if kernel was updated via rpi-update. rpi-source might resolve this issue: https://github.com/notro/rpi-source/wiki

@MichaIng
Copy link
Owner Author

MichaIng commented Jan 7, 2019

@Fourdee
Does NordVPN support WireGuard as well? We could then let user choose.

+ DietPi-Software | Tiny
+ DietPi-Software | WireGuard: Install kernel meta packages via G_AGI to force install of new dependencies; Check whether a reboot is required due to upgraded kernel, before starting WireGuard, otherwise inform user.
@Fourdee
Copy link
Collaborator

Fourdee commented Jan 7, 2019

@MichaIng

Appears NordVPN are looking into Wireguard: https://www.pr.com/press-release/769043

Perhaps on update and software install start, we should do G_AGDUG instead of G_AGUG?

Originally, G_AGDUG is left for dietpi-update, as Meveric kernel packages required it. Ideally wanted to keep these as part of the update system so we could track any issues.

But yep, overall makes sense to G_AGDUG in dietpi-software aswell.

@MichaIng
Copy link
Owner Author

MichaIng commented Jan 7, 2019

@Fourdee
dietpi-update currently as well does G_AGUG only: https://github.com/Fourdee/DietPi/blob/dev/dietpi/dietpi-update#L328

I think this basically should stay as is now, mainly due to the dependency issues I regularly run into on Raspbian Buster: If some dependency is missing (which might as well occur on stable branch, theoretically), a forced (non-interactive) dist-upgrade leads to dependants being purged instead of upgrade being skipped. Perhaps there is a way to work around this via apt(-get) command arguments, so that manually installed dependants are never removed, but dependency upgrades skipped instead. As long as this is not the case, I think we should never force a dist-upgrade, to prevent possibly broken systems.

For WireGuard this is not not required anymore with latest commits, since G_AGI has the same effect than G_AGDUG, but only for the installed packages, which is safe in case of kernel meta packages.

@Fourdee
Copy link
Collaborator

Fourdee commented Jan 7, 2019

@MichaIng

dietpi-update currently as well does G_AGUG only:

Whoopsie! 😄

As long as this is not the case, I think we should never force a dist-upgrade, to prevent possibly broken systems.

Yep, good point 👍

@MichaIng
Copy link
Owner Author

MichaIng commented Jan 7, 2019

Another issue when rpi-update was used, i assume:

  • We can rerun rpi-update and rpi-source both, to assure used kernel and headers match.
  • But I guess, if user manually runs rpi-update afterwards, or via dietpi-config, no WireGuard module update/install to new kernel will be triggered, especially also because new headers are missing.

Best we can do, I think, is implement rpi-source into G_RPI_UPDATE, so check if it's available and, in case, runs it directly afterwards. Then check, if WireGuard is installed and run dpkg-reconfigure wireguard to install the module to new kernel.

If rpi-update is not installed, of course we can skip all this on WireGuard install.
Damn complicated on RPi with different moreless usual kernel sources. Alternatively we can reinstall the APT kernel package, if rpi-update is installed, thus we cannot assure that no custom kernel version is in place.


Okay at least on WireGuard install the way is now clear: notro/rpi-source#40 (comment)
- [ ] If rpi-update is installed, use rpi-source to assure the fitting headers are installed.
- [ ] Otherwise assume that Raspbian kernel package is installed (better add it to G_AGI again, to be failsafe) and install the related headers package via APT.
- [ ] As mentioned above, add rpi-source and WireGuard module rebuild to G_RPI_UPDATE (thus dietpi-config option).

🈺 Damn it's a python script... I'm not keen to install python just for workaround this issue. Hmm the more I think about it, the more I think we should remove any rpi-update from our scripts and globals. This is really not recommended (bleeding edge, possible unstable kernel) and nowadays the kernel APT package is updated quite regularly. Instead ask user, if rpi-update is detected, to either reinstall/overwrite with APT package kernel or to otherwise manually take care the required kernel headers and module build.

@HiDef888
Copy link

HiDef888 commented Jan 8, 2019

Thanks guys! I haven't tried it yet but I'm going to!
I'm currently having some issues with mono :( chomping up all my memory. Working on figuring out why.

PIA is also supposed to roll out support once it's audited

+ DietPi-Config | Firmware update: On RPi give choice and do recommend to NOT use "rpi-update", but stay with APT firmware packages  instead
+ DietPi-Config | Firmware update: Check firmware (via /lib/modules) before and after update attempt, to check if it actually changed, before setting reboot requirement
+ DietPi-Config | Minor coding
+ DietPi-Set_Hardware | Do not use "rpi-update" anymore. APT packages provide regular stable updates based on the same firmware sources. Use "raspberrypi-kernel-headers" package instead of "rpi-source" to build custom kernel modules.
+ DietPi-Software | WireGuard: On RPi, if "rpi-update" is detected and APT did not override with new stable firmware, inform user about possibly failing kernel module and options to resolve
+ DietPi-Software | WireGuard: "raspberrypi-kernel-headers" never needs to be reinstalled, even if "rpi-source" was used
+ DietPi-Globals | Remove G_RPI_UPDATE, which is not recommended without special reason anyway and causes issues, when correct headers are required to build kernel modules
+ DietPi-Config | Purge "rpi-update", if APT based firmware was chosen
+ DietPi-Config | Remove ".G_RPI_UPDATE" flag creation, which is not required anymore
@MichaIng
Copy link
Owner Author

MichaIng commented Jan 8, 2019

@Fourdee
Okay, due to some research and thinking about this, I suggest we should never recommend or use rpi-update within our scripts. It is never required, potentially unstable and APT kernel packages are updated nowadays very frequently, possibly offering a more current firmware than user installed via rpi-update once.

If left the choice to use rpi-update inside the RPi firmware update menu, made clear that this is not recommended, offering usual APT based update instead. Although I personally would remove the option completely, to leave it's use very clear to end user responsibility.
The previous implementation lead to wrong assumptions, that rpi-update is a regular or even recommended way to update ones RPi kernel, which is definitely not!

One case where we used rpi-update + rpi-source to build a kernel module within dietpi-set_hardware, I replaced this by APT packages (including raspberrypi-kernel-headers), in case overriding any non-APT kernel, to assure headers match.

Do you agree?

+ DietPi-Patch | Remove obsolete rpi-update flag
+ DietPi-Patch | Patch Samba cache to RAM manually, to prevent whole "dietpi-software reinstall" with config override (even we place backup)
@Fourdee
Copy link
Collaborator

Fourdee commented Jan 9, 2019

@MichaIng

Do not recommended/force use of rpi-update

Yep agree 👍

One case where we used rpi-update + rpi-source to build a kernel module within dietpi-set_hardware, I replaced this by APT packages (including raspberrypi-kernel-headers), in case overriding any non-APT kernel, to assure headers match.

Yep, rpi-update was required at the time as latest kernel was needed which included support for the soundcard I believe. However, should be fine using APT now.

@garret
Copy link

garret commented Jan 9, 2019

DietPi confirms to be one of the best distro.
ProtonVPN is pushing hard to integrate Wireguard. Hopefully, they will get it soon.

@MichaIng
Copy link
Owner Author

MichaIng commented Jan 9, 2019

@Fourdee or anyone with RPi free for testing
Could you try the following, if anyhow possible:

  • Install WireGuard on an RPi with an older kernel + headers version
  • Upgrade kernel and headers packages and see, if the WireGuard module is rebuilt automatically, like it is the case on x86_64?

I am just thinking how to achieve this practically, since there are no older kernel packages available and no backports or buster repo. Perhaps only time will show, when users install WireGuard and next firmware packages are released 🤔.

There is this RPi dac module that we build manually, when selected. I guess this needs to be redone after every kernel update currently, right?


Same question is for Odroids. We have kernel packages in Meveric's repo, header packages are surely available as well. But would be good to know, if WireGuard kernel module rebuild is triggered automatically or one needs to do this manually.

@MichaIng
Copy link
Owner Author

MichaIng commented Jan 9, 2019

@garret
Thanks 😃

Currently the install script will configure WireGuard to be the server and create matching client configurations to pass all network traffic through the VPN. It would be great to have some choices there:

  1. Configure WireGuard as above
  2. As above, but only having requests to the servers local network passed through the VPN, while other www requests from clients are not forwarded though the VPN. Useful, if you only need to for secure file transfer to you local network, e.g. between two work places, University and home or such.
  3. As above, but only having requests to the VPN server machine itself passed through the tunnel. Useful, if you only need secure file transfer to this machine, or you run Pi-hole on it and want to use it as DNS server to have ads blocked everywhere.
  4. Configure WireGuard as client, to connect to public/external VPN providers. This should be very easy, as long as they support it, only by adding their domain/DNS and global key to your config. Usually you want this for privacy, so all requests should be tunnelled then.

We could either add a whiptail menu to allow user choice while install, and/or we add a dietpi.txt entry that allows choice (+ local machine domain/public IP + VPN port), to then have the install non-interactive, or we leave configuration to end user, by adding commented code parts to to the config file, to enable one of the above use cases.

@Fourdee
Copy link
Collaborator

Fourdee commented Jan 10, 2019

@MichaIng

There is this RPi dac module that we build manually, when selected. I guess this needs to be redone after every kernel update currently, right?

Yep, if the module is not built into the stock kernel, the only solution is to hold kernel packages, prevent any updates.

Else, each kernel update would break compiled module.

@MichaIng
Copy link
Owner Author

MichaIng commented Jan 10, 2019

@Fourdee
Okay, so either we set the kernel packages on hold then, or, since there are other cases where one needs to update them, inform user that the soundcard needs to be re-selected, if a kernel update occurred.

Or we try to automate this within G_AG*, if RPi + kernel update was detected.

@Fourdee
Copy link
Collaborator

Fourdee commented Jan 11, 2019

@MichaIng

We could:

root@DietPi:~# uname -r
4.4.154

Save to file, and if that does not match at time of install, then we could prompt user to reinstall?

But probably best to hold packages regardless.

EDIT:

Ah see you did something similar 👍

if [[ $old_firmware == $(ls /lib/modules/) ]]; then

EDIT: Lets just hold packages.

+ reboot if kernel mismatch before wireguard install
+ testing
@Fourdee Fourdee merged commit bf35386 into dev Jan 11, 2019
@Fourdee
Copy link
Collaborator

Fourdee commented Jan 11, 2019

@MichaIng

Many thanks 👍

Will merge, run tests and changes if required.

@MichaIng
Copy link
Owner Author

@Fourdee
Jep [[ $old_firmware == $(ls /lib/modules/) ]] is to check the "installed" kernel, while uname -r shows the active version, which does not change before reboot.

# - If APT updates did not override "rpi-update" kernel version, headers might mismatch and WireGuard module missing
if [[ $old_firmware == $(ls /lib/modules/) ]]; then
G_WHIP_MSG 'Wireguard:\n\nKernel update detected, a reboot is required before Wireguard module can be built with latest kernel.\n\nOnce the system has rebooted, reselect Wireguard for installation to continue.'
aSOFTWARE_INSTALL_STATE[$software_id]=0
Copy link
Owner Author

@MichaIng MichaIng Jan 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Fourdee
Reboot not required here. The module will be build for all installed kernels, regardless of which one is active. So after reboot (which is done anyway after dietpi-software), WireGuard will work.

Also setting the packages on hold is contra-productive. WireGuard rebuilds the module automatically, when the kernel packages are updated due to some APT trigger, or did this not work on RPi (was the case on x86_64 at least)? So setting kernel packages on hold is required only for manually build modules.

Copy link
Collaborator

@Fourdee Fourdee Jan 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MichaIng

Reboot not required here. The module will be build for all installed kernels, regardless of which one is active. So after reboot (which is done anyway after dietpi-software), WireGuard will work.

Excellent, i'll make the change 👍
In this case, modprobe wireguard will never fail.

Copy link
Collaborator

@Fourdee Fourdee Jan 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MichaIng

WireGuard rebuilds the module automatically, when the kernel packages are updated due to some APT trigger, or did this not work on RPi (was the case on x86_64 at least)?

Will need to verify this is case.

Does with Odroids

@MichaIng MichaIng deleted the wireguard branch January 11, 2019 20:15
@MichaIng
Copy link
Owner Author

MichaIng commented Jan 11, 2019

Further device support:

  • http://fuzon.co.uk/meveric/dists/all
  • Odroid N1: linux-image-arm64-odroid-n1 linux-headers-armhf-odroid-n1
  • Odroid C1: linux-image-arm64-odroid-c1 linux-headers-armhf-odroid-c1
  • Odroid C2: linux-image-arm64-odroid-c2 linux-headers-armhf-odroid-c2
  • Odroid XU3/XU4/HC1/HC2: linux-image-4.14-armhf-odroid-xu4 linux-headers-4.14-armhf-odroid-xu4
  • Pine64 RockPro64: linux-rockpro64
  • Pine63 Rock A64: linux-rock64
    • Are headers included with the Pine64 packages?

@MichaIng
Copy link
Owner Author

@Fourdee
Odroid U3 is missing. Also we do not install kernel packages for in within DietPi-PREP. Not sure if it's linux-image-armhf-odroid-u from http://fuzon.co.uk/meveric/dists/all/u/binary-armhf/Packages?
But this is kernel v3.X, not sure if there is a newer available somwhere?

I am afraid it's for U1 actually, similar to linux-image-arm64-odroid-xu being an old kernel, obviously for XU1.

@Fourdee
Copy link
Collaborator

Fourdee commented Jan 13, 2019

@MichaIng

Odroid U3 is missing.

Yep, unable to check as lack the device, and, no longer made. We only have 6 installs on survey, so I believe we can leave it for now.

@MichaIng
Copy link
Owner Author

@Fourdee
Ah good point. Also we do not offer an official image for it, didn't have a look at this.

Meveric Jessie image available only, so makes sense that kernel version is frozen at 3.X: https://forum.odroid.com/viewtopic.php?f=79&t=17770

Jep, leave it out from support until someone asks for it and is willing verify kernel+header packages and do required tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants