Debian 12 aka bookworm template

[fepitre]

Debian 12 aka bookworm: https://www.debian.org/releases/bookworm/
Right now it is in final freeze, final release planned for 2022-04-21

Tasks:

• build all packages
• build the template
• document
• upload to testing repo
• migrate to stable repo
• announce

If any issue affects bookworm specifically (build failures, things that worked fine before etc), please add reference to this issue too.

cc @marmarek

[unman]

On Fri, Dec 17, 2021 at 07:19:06AM -0800, Frédéric Pierret wrote: Debian 12 aka `bookworm`: https://www.debian.org/releases/bookworm/ Right now it is in final freeze, final release planned for 2020-04-21

I think that the copy/paste has gone too far, no? ;-)

[sidhussmann]

On my journey to support debian-testing (bookworm) templates on Qubes R4.0 I chose following strategy:

1. setup a Fedora VM for Qubes Builder (https://www.qubes-os.org/doc/qubes-builder/)
2. build current Debian stable (bullseye)
3. use the same procedure to build (bookworm)
4. see where it fails, fix, create PRs

However, I'm struggling at step 2 and the build for Debian 11 fails.

Steps to reproduce:

[user@dev-qubes-fedora ~]$ git clone git://github.com/QubesOS/qubes-builder.git
[user@dev-qubes-fedora ~]$ cd qubes-builder/
[user@dev-qubes-fedora qubes-builder]$ git tag -v `git describe`
object c209be5e4a3cafae882bea4e438e48552805ee52
type commit
tag mm_c209be5e
tagger Marek Marczykowski-Górecki <[email protected]> 1639101164 +0100

Tag for commit c209be5e4a3cafae882bea4e438e48552805ee52
gpg: Signature made Fri 10 Dec 2021 02:52:44 AM CET
gpg:                using RSA key 0064428F455451B3EBE78A7F063938BA42CFA724
gpg: Good signature from "Marek Marczykowski-Górecki (Qubes OS signing key) <[email protected]>" [full]

Then I create a builder.conf with following content:

TEMPLATE_ONLY ?= 1

# Release Version
# 2 - Release 2
# 3 - Release 3
# Default: 3
RELEASE := 4.0

# GIT_PREFIX - Git release repo prefix
# Default: R2: qubes-r2/  R3: qubes-r3/
GIT_PREFIX := QubesOS/qubes-

# DISTS_VM - Choose the templates to build.  Multiple templates can be defined 
# to build.  Uncomment tempalte dist to build.
# Default: fc20
DISTS_VM := 
DISTS_VM += bullseye

DEBUG = 0
VERBOSE = 0

# Include template.conf configuration.  Contains most of the rules to build
# the templates
include example-configs/templates.conf

# Used to identify which additional makefile configuration scripts are being used
about::
	@echo "debian.conf"

Then I successfully prepare the build with make install-deps && make get-sources.
When running make qubes-vm the build fails with following error after a couple of minutes:

...
The following packages will be REMOVED:
  xen-build-deps
The following NEW packages will be installed:
  bcc bin86 mailcap mime-support
0 upgraded, 4 newly installed, 1 to remove and 0 not upgraded.
1 not fully installed or removed.
Need to get 257 kB of archives.
After this operation, 620 kB of additional disk space will be used.
Get:1 https://deb.debian.org/debian bullseye/main amd64 mailcap all 3.69 [31.7 kB]
Get:2 https://deb.debian.org/debian bullseye/main amd64 mime-support all 3.66 [10.9 kB]
Get:3 https://deb.debian.org/debian bullseye/main amd64 bin86 amd64 0.16.17-3.3 [93.5 kB]
Get:4 https://deb.debian.org/debian bullseye/main amd64 bcc amd64 0.16.17-3.3 [121 kB]
Fetched 257 kB in 0s (2632 kB/s)
E: Can not write log (Is /dev/pts mounted?) - posix_openpt (19: No such device)
(Reading database ... 34873 files and directories currently installed.)
Removing xen-build-deps (2001:4.8.5-36) ...
Selecting previously unselected package mailcap.
(Reading database ... 34869 files and directories currently installed.)
Preparing to unpack .../archives/mailcap_3.69_all.deb ...
Unpacking mailcap (3.69) ...
Selecting previously unselected package mime-support.
Preparing to unpack .../mime-support_3.66_all.deb ...
Unpacking mime-support (3.66) ...
Selecting previously unselected package bin86.
Preparing to unpack .../bin86_0.16.17-3.3_amd64.deb ...
Unpacking bin86 (0.16.17-3.3) ...
Selecting previously unselected package bcc.
Preparing to unpack .../bcc_0.16.17-3.3_amd64.deb ...
Unpacking bcc (0.16.17-3.3) ...
Setting up bin86 (0.16.17-3.3) ...
Setting up bcc (0.16.17-3.3) ...
Setting up mailcap (3.69) ...
Setting up mime-support (3.66) ...
Processing triggers for man-db (2.9.4-2) ...
mk-build-deps: Unable to install xen-build-deps at /usr/bin/mk-build-deps line 457.
mk-build-deps: Unable to install all build-dep packages
make[2]: *** [/home/user/qubes-builder/qubes-src/builder-debian//Makefile-legacy.debian:45: dist-build-dep] Error 1
make[1]: *** [Makefile.generic:197: packages] Error 1
make: *** [Makefile:265: vmm-xen-vm] Error 1

There are two things that got my attention: First, why is the script removing xen-build-deps, when it seems to be needed. Second, why is there an error E: Can not write log (Is /dev/pts mounted?) - posix_openpt (19: No such device). These two things seem completely unrelated to me.

I'm pretty sure this should just work and I have an error in my builder.conf. However, I can't seem to spot it. I use a Fedora 34 VM for building, but it also fails with Fedora 32.

@fepitre could you please provide your builder.conf as reference?

Attached the build-logs:

mgmt-salt-vm-bullseye.log
vmm-xen-vm-bullseye.log

[fepitre]

@sidhussmann here: https://gist.github.com/fepitre/801d10f7c162d63400b289f543dad4ae. More generally, I would recommend to use directly example configs qubes-os-r4.0.conf. We don't spend that much time on updating to resolving current config examples as we are in creating a new builder from scratch.

[fepitre]

@marmarek I've added references to other PR that I forgot to add in description few days ago.

[andrewdavidwong]

Debian 12 has now officially been released: https://www.debian.org/News/2023/20230610

[adrelanos]

This is blocking the release of Whonix 17 (based on Debian bookworm).

[unman]

bookworm builds and work fine. What's missing is salt-ssh package - see #7896

[andrewdavidwong]

I also see that the minimal template is now available in the testing repo for Qubes 4.2. I believe that covers testing availability for all supported release combinations. Thanks!

[k4r4b3y]

When can we expect debian 12 (both minimal and normal) to become available in QubesOS 4.1?

[andrewdavidwong]

@k4r4b3y: That depends on how many bugs are found during testing, how severe they are, and how long they take to fix. Aside from that, I don't believe we have a policy governing the timing of template releases the way we do for our release schedule. (But, in this case, I suppose that release schedule might provide an indirect clue, since Debian 12 is supposed to be ready in time for the 4.2 stable release.)

[p1llule]

Hello,

I suspect something wrong with the debian-12-minimal template for qubes 4.1 :

user@host:~/QubesIncoming$ wc -l debian-11-minimal/debian-11-minimal.packages 
418 debian-11-minimal/debian-11-minimal.packages
user@host:~/QubesIncoming$ wc -l debian-12-minimal/debian-12-minimal.packages 
795 debian-12-minimal/debian-12-minimal.packages
user@host:~/QubesIncoming$ wc -l debian-12/debian-12.packages 
1364 debian-12/debian-12.packages

There is many more packages installed that it used to be.
Cups server, firefox, etc.

Cdlt.

[lubellier]

There is many more packages installed that it used to be.
Cups server, firefox, etc.

@p1llule : see the #8330 issue.

[k4r4b3y]

There is many more packages installed that it used to be.
Cups server, firefox, etc.

@p1llule : see the #8330 issue.

This issue seems to be resolved now. Can we expect debian 12 hitting qubes os sooner?

[marmarek]

Debian 12 template seems to be stable enough now. @andrewdavidwong can you prepare the announcement?

[lubellier]

Sorry, late report... I see an issue with the debian-12 template and qvm-run.

With Qubes-OS 4.1, myAppVM is a debian-12 template based AppVM, and with the below command :

[user@dom0 ~]$ qvm-shutdown --wait myAppVM ; qvm-run -v -a --service -- myAppVM qubes.StartApp+firefox-esr

Approximately, 1 time by 5, Firefox doesn't open. The AppVM launched (Qube state is Running), and qvm-run doesn't give back the prompt, no error message. I don't see this case with my debian-11 template based AppVMs.

Someone else see this problem?

[lubellier]

More details :

• same issue with a new created AppVM and launching qubes.StartApp+org.gnome.Terminal.
• when the AppVM doen't open the application, if I launch again the qvm-run command, the application opens
• when the issue occurs, no special trace in /var/log/qubes/*myAppVM*.log

[unman]

Also late - on 4.1 I see exactly this *most* of the time. I have numerous scripts launching disposables with qubes-run-terminal - the disposables almost always close down, without launching any terminal. On a standard qube, the same -- but the second call will open the terminal. Also reproduce with firefox.

[marmarek]

This looks like an issue that was fixed in R4.2 already. Apparently it affects not only Ubuntu, but also debia-12.

[andrewdavidwong]

@marmarek, will debian-12-xfce also be released for Qubes 4.1 or only for 4.2?

[marmarek]

@marmarek, will debian-12-xfce also be released for Qubes 4.1 or only for 4.2?

Currently no (only R4.2). I can build it, but I would push it to stable later, to give it more time for testing (we haven't published debian xfce templates for R4.1 before).

[andrewdavidwong]

@marmarek, @fepitre: Looks like we forgot to migrate these from testing to stable.

[andrewdavidwong]

By the way, I also see that the "document" step is unchecked in the first post. Is action required there?

[andrewdavidwong]

The packages have been migrated.

[andrewdavidwong]

@marmarek: Multiple users report getting the message "Error canonicalizing file: failed to fill whole buffer" when attempting to download debian-12-minimal.

[TheWanderer1983]

With Qubes-OS 4.1, myAppVM is a debian-12 template based AppVM, and with the below command :

[user@dom0 ~]$ qvm-shutdown --wait myAppVM ; qvm-run -v -a --service -- myAppVM qubes.StartApp+firefox-esr

Approximately, 1 time by 5, Firefox doesn't open. The AppVM launched (Qube state is Running), and qvm-run doesn't give back >the prompt, no error message. I don't see this case with my debian-11 template based AppVMs.

Someone else see this problem?

I updated the debian minimal 11 templates in Qubes 4.1 to bookworm and I'm having the same issue. I was planning to do a fresh 4.2 install and copy over VMs so isn't a big concern. Given its fixed in 4.2.

[marmarek]

debian-12 templates are in stable repo for both R4.1 and R4.2 for quite some time already.