Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Verify downloaded packages and include GPG keys in installer #66

Merged
merged 6 commits into from
Jul 16, 2014

Conversation

jturner314
Copy link
Contributor

This pull request is a superset of #63 and is rebased off of the latest master. Compared to #63, this includes a few small fixes, uses a temporary keyring instead of the user's personal keyring, adds additional functionality to fix issue #64, and improves the interface of update.sh. The individual commit messages provide more detail.

Please double-check the fingerprints in update.sh. I think they are accurate but don't have a good way to verify them directly.

Before, during the build process, packages were downloaded over http
without authentication. Now, the Release file and packages are properly
verified. If verification fails, a warning will be presented to the
user with the option to ignore it.

Note that in order to perform GPG verification of the Release file, the
user must have the Raspbian signing key in their keyring. The key can
be obtained from public keyservers (0x9165938D90FDDD2E) or from
https://archive.raspbian.org/raspbian.public.key.
This also switches from using the user's personal keyring to using a
temporary keyring for `update.sh` and fingerprints hardcoded in
`update.sh`.
Before, the key was being downloaded insecurely with wget during
installation without verification. Now, the key that was downloaded and
verified by `update.sh` is included in the installer image so that it
doesn't need to be downloaded at install time.
Before, the output was very verbose, particularly with wget downloads.
Now, much less output is generated so that it is easier to monitor.
Additionally, added confirmation messages for successful verifications
and changed verification problems from warnings to errors.
Before, `update.sh` would exit without any messages if not all required
packages were found in the package lists. Now, it tells the user what
went wrong.
@diederikdehaas
Copy link
Member

I (finally) got around to testing your PR and got to say: "It's looking good!" 👍

But I'm a bit puzzled by some lines in the bootstrap.log file, particularly line 1297 and line 1310.
Especially since in the beginning you can see the lines P: Retrieving <packagename>.

This is quite likely due to my lack of knowledge on this, but could you take a look and see if there's any harm in it?

Don't know if it's relevant, but apt-key list produces the following output:

# apt-key list
/etc/apt/trusted.gpg
--------------------
pub   2048R/90FDDD2E 2012-04-01
uid                  Mike Thompson (Raspberry Pi Debian armhf ARMv6+VFP) <[email protected]>
sub   2048R/5373DC6C 2012-04-01

pub   2048R/7FA3303E 2012-06-17
uid                  Raspberry Pi Archive Signing Key
sub   2048R/EDD83D6C 2012-06-17

So there's also a 'Raspberry Pi Archive Signing Key', besides Mike Thomson's key. Should something be done with that key?

@jturner314
Copy link
Contributor Author

But I'm a bit puzzled by some lines in the bootstrap.log file, particularly line 1297 and line 1310.
Especially since in the beginning you can see the lines P: Retrieving <packagename>.

Good catch on those warnings. I just took a look at the source for cdebootstrap and the log file you posted. Here is what I think is happening:

  1. cdebootstrap downloads all of the packages to /var/cache/bootstrap/ and validates them at the very beginning of the process, which is why you see the P: Retrieving <packagename> and P: Validating <packagename> lines.

  2. cdebootstrap extracts some essential packages directly to the root filesystem from previously-downloaded packages.

  3. cdebootstrap uses dpkg to install and configure some essential, previously-downloaded packages and helper scripts.

  4. The postinst script for cdebootstrap-helper-apt writes the following to /etc/sources.list:

    deb bootstrap: . main
    

    which, as far as I can tell, forces APT to use the previously downloaded packages in /var/cache/bootstrap.

  5. cdebootstrap uses APT to install the remaining packages, which were downloaded in step 1.

  6. cdebootstrap removes its helper scripts, configures the actual sources.list, etc.

So, in summary, it appears that cdebootstrap downloads and verifies everything at the beginning, so the warning messages that you pointed out are not a concern. In fact, the warnings appear to be expected because line 173 in cdebootstrap/src/install.c is the following. (Note the AllowUnauthenticated.)

  strcpy (buf, "apt-get install --yes -o APT::Get::AllowUnauthenticated=true -o APT::Install-Recommends=false");

I still find the warnings strange, though -- after all, if Release and Release.gpg were present as suggested by lines 1232-1233 of the log, why wouldn't authentication work? The upstream cdebootstrap developers would probably be able to clarify this.

There is a --debug option for cdebootstrap, which may help provide more details. Unfortunately, I don't have my Raspberry Pi available for testing anymore, but if someone posts the output with --debug, I can help look through it.

So there's also a 'Raspberry Pi Archive Signing Key', besides Mike Thomson's key. Should something be done with that key?

That's the raspberrypi.org signing key, which is necessary to install packages securely from archive.raspberrypi.org. (See issue #64.)

@diederikdehaas
Copy link
Member

On Tuesday 08 July 2014 21:07:05 Jim Turner wrote:

So, in summary, it appears that cdebootstrap downloads and verifies
everything at the beginning, so the warning messages that you pointed out
are not a concern. In fact, the warnings appear to be expected

Thanks for your explanation :)

There is a --debug option for cdebootstrap

Yep and also --verbose. Problem is that it is only logged on screen, but not
to the logfile.
I have filed a bug for that: http://bugs.debian.org/754867 but it won't help us
now ;-)

So there's also a 'Raspberry Pi Archive Signing Key', besides Mike
Thomson's key. Should something be done with that key?
That's the raspberrypi.org signing key, which is necessary to install
packages securely from archive.raspberrypi.org. (See issue #64.)

That must have been a brain fart on my side or sth like that LOL. You're
absolutely right.

I expect to merge your PR today :)

@jturner314
Copy link
Contributor Author

Yep and also --verbose. Problem is that it is only logged on screen, but not to the logfile.

I didn't realize that. It should be possible to use the shell and tee to copy the screen output to a logfile somewhere:

cdebootstrap-static --arch=armf $cdebootstrap_cmdline   ...   2>&1 | tee /path/to/logfile

I expect to merge your PR today :)

Sounds great! Thanks for reviewing my PR so thoroughly and for all of your work on this project.

@diederikdehaas
Copy link
Member

Ohh, definitely going to try the tee suggestion. I knew the existence of it, but haven't studied it yet.

I do/did test it thoroughly, but the merge was also delayed by a new kernel version popping up all of a sudden and a new version of cdebootstrap. That one had a bug in it. Thankfully, plugwash uploaded a fixed version quite quickly after I reported it :-)

@diederikdehaas diederikdehaas merged commit 7b78f6f into debian-pi:master Jul 16, 2014
@jturner314 jturner314 deleted the use-gpg branch July 16, 2014 22:44
@diederikdehaas
Copy link
Member

Just wanted to let you know that your code works excellent!
Up until just now it always 'Just worked (tm)', but now I got an error:

$ ./build.sh 
Downloading and importing raspbian.public.key...
######################################################################## 100.0%
Verifying raspbian.public.key... OK

Downloading and importing raspberrypi.gpg.key...
######################################################################## 100.0%
Verifying raspberrypi.gpg.key... OK

Downloading Release file and its signature...
######################################################################## 100.0%
######################################################################## 100.0%
Verifying Release file... OK

Downloading firmware package list...
######################################################################## 100.0%
Verifying firmware package list... OK

Downloading main package list...
######################################################################## 100.0%
Verifying main package list... OK

Searching for required packages...
  raspberrypi-bootloader-nokernel  raspberrypi-bootloader-nokernel_1.20140107-1~nokernel1_armhf.deb
  btrfs-tools                      btrfs-tools_3.14.1-1_armhf.deb
  busybox-static                   busybox-static_1.22.0-6_armhf.deb
  cdebootstrap-static              cdebootstrap-static_0.6.1_armhf.deb
  dosfstools                       dosfstools_3.0.26-2_armhf.deb
  e2fslibs                         e2fslibs_1.42.10-1.1_armhf.deb
  e2fsprogs                        e2fsprogs_1.42.10-1.1_armhf.deb
  f2fs-tools                       f2fs-tools_1.3.0-1_armhf.deb
  gpgv                             gpgv_1.4.18-2_armhf.deb
  libblkid1                        libblkid1_2.20.1-5.8_armhf.deb
  libbz2-1.0                       libbz2-1.0_1.0.6-5_armhf.deb
  libc6                            libc6_2.19-7_armhf.deb
  libcomerr2                       libcomerr2_1.42.10-1.1_armhf.deb
  libgcc1                          libgcc1_4.9.0-7_armhf.deb
  liblzo2-2                        liblzo2-2_2.06-1.2_armhf.deb
  libuuid1                         libuuid1_2.20.1-5.8_armhf.deb
  linux-image-3.10-3-rpi           linux-image-3.10-3-rpi_3.10.11-1+rpi7_armhf.deb
  raspbian-archive-keyring         raspbian-archive-keyring_20120528.2_all.deb
  zlib1g                           zlib1g_1.2.8.dfsg-1+rpi1_armhf.deb

Downloading packages...
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
######################################################################## 100.0%
Verifying downloaded packages... libcomerr2_1.42.10-1.1_armhf.deb: FAILED
e2fsprogs_1.42.10-1.1_armhf.deb: FAILED
e2fslibs_1.42.10-1.1_armhf.deb: FAILED
sha256sum: WARNING: 3 computed checksums did NOT match
ERROR
The checksums of the downloaded packages don't match the package lists!

And that was indeed correct, since when I checked the archives, I found version 1.42.11-2.
Running update.sh again downloaded the new version and no errors where reported then.

hoehnp pushed a commit to hoehnp/raspbian-ua-netinst that referenced this pull request Jul 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants