From 7d85ee75dab9f763bfe3bbca118ebd600a99487b Mon Sep 17 00:00:00 2001 From: James Le Cuirot Date: Fri, 5 Jul 2024 10:15:03 +0100 Subject: [PATCH 1/2] Skip Catalyst stage2 as recommended by upstream Gentoo From https://wiki.gentoo.org/wiki/Catalyst/Stage_Creation#Build_Stage3: > It is not necessary to build stage2 in order to build stage3. Gentoo > release engineering does not build stage2, and you should not need to > unless you're intentionally building a stage2 as your goal. We can now sync portage-stable/scripts with upstream because bootstrap.sh is only used during stage2, and the changes we had are no longer relevant. It seems likely the changes were already redundant anyway. Signed-off-by: James Le Cuirot --- .../workflows/portage-stable-packages-list | 5 +--- bootstrap_sdk | 29 +++++++------------ build_library/catalyst.sh | 24 ++------------- .../portage-stable/scripts/bootstrap.sh | 13 +-------- 4 files changed, 16 insertions(+), 55 deletions(-) diff --git a/.github/workflows/portage-stable-packages-list b/.github/workflows/portage-stable-packages-list index c956de77425..d687dd55462 100644 --- a/.github/workflows/portage-stable-packages-list +++ b/.github/workflows/portage-stable-packages-list @@ -474,10 +474,7 @@ net-vpn/wireguard-tools perl-core/File-Temp profiles - -# The bootstrap script has some modifications, so we can't sync scripts directory yet. -# -# scripts +scripts sec-keys/openpgp-keys-gentoo-release diff --git a/bootstrap_sdk b/bootstrap_sdk index 36bda6ecca9..15ac6ce9bda 100755 --- a/bootstrap_sdk +++ b/bootstrap_sdk @@ -18,23 +18,16 @@ # will be linked against the SEED SDK libraries, NOT against libraries built # in stage 1. # -# 3. stage2: Run portage-stable/scripts/bootstrap.sh -# This rebuilds the toolchain using Gentoo bootstrapping, ensuring it's not linked -# to or otherwise influenced by whatever was in the "seed" tarball. -# The toolchain rebuild may contain updated package ebuilds from -# third_party/(portage-stable|coreos-overlay). -# This and all following stages use portage-stable and coreos-overlay -# from third_party/... (see 1.) +# 3. stage2: This is skipped as recommended by upstream Gentoo. # -# 4. stage3: Run emerge -e system to rebuild everything using the fresh updated -# toolchain from 3., using the normal USE flags provided by the profile. This -# will also pull in assorted base system packages that weren't included -# in the minimal environment stage1 created. +# 4. stage3: Run emerge -e system to rebuild everything using the normal USE +# flags provided by the profile. This will also pull in assorted base system +# packages that weren't included in the minimal environment stage1 created. # # 5. stage4: Install any extra packages or other desired tweaks. For the # sdk we just install all the packages normally make_chroot.sh does. # -# Usage: bootstrap_sdk [stage1 stage2 etc] +# Usage: bootstrap_sdk [stage1 stage3 etc] # By default all four stages will be built using the latest stage4 as a seed. SCRIPT_ROOT=$(dirname $(readlink -f "$0")) @@ -96,7 +89,7 @@ cp "${BUILD_LIBRARY_DIR}/toolchain_util.sh" "${ROOT_OVERLAY}/tmp" # Stage 1 uses "known-good" ebuilds (from both coreos-overlay and portage-stable) -# to build a minimal toolchain (USE="-*") for stage 2. +# to build a minimal toolchain (USE="-*") for stage 1. # # No package updates must happen in stage 1, so we use the portage-stable and # coreos-overlay paths included with the current SDK (from the SDK chroot's @@ -104,14 +97,14 @@ cp "${BUILD_LIBRARY_DIR}/toolchain_util.sh" "${ROOT_OVERLAY}/tmp" # 'cork enter', i.e. the SDK we run ./bootstrap_sdk in. # # Using ebuilds from the above mentioned sources will ensure that stage 1 builds -# a minimal stage 2 from known-good ebuild versions - the same ebuild versions +# a stage 3 from known-good ebuild versions - the same ebuild versions # that were used to build the very SDK we run ./bootstrap_sdk in. # # DANGER ZONE # # Stage 1 lacks proper isolation and will link all packages built for -# stage 2 against its own seed libraries ("/" in the catalyst chroot) instead of against libraries -# installed into the FS root of the stage 2 seed ("/tmp/stage1root" in the catalyst chroot). +# stage 3 against its own seed libraries ("/" in the catalyst chroot) instead of against libraries +# installed into the FS root of the stage 3 seed ("/tmp/stage1root" in the catalyst chroot). # This is why we must prevent any updated package ebuilds to "leak" into stage 1, hence we use # "known good" ebuild repo versions outlined above. # @@ -119,10 +112,10 @@ cp "${BUILD_LIBRARY_DIR}/toolchain_util.sh" "${ROOT_OVERLAY}/tmp" # for either (or both) portage and overlay. The command line options # --stage1-portage-path and --stage1-overlay-path may be used to specify # a repo path known to work for stage1. In that case the stage1 seed (i.e. the seed SDK) -# will be updated prior to starting to build stage 2. +# will be updated prior to starting to build stage 3. # NOTE that this should never be used to introduce library updates in stage 1. All binaries # produced in stage 1 are linked against libraries in the seed tarball, NOT libraries produced -# by stage one. Therefore, these binaries will cease to work in stage 2 when linked against +# by stage 1. Therefore, these binaries will cease to work in stage 3 when linked against # outdated "seed tarball" libraries which have been updated to newer versions in stage 1. catalyst_build diff --git a/build_library/catalyst.sh b/build_library/catalyst.sh index 4512091dfc7..050358320e4 100644 --- a/build_library/catalyst.sh +++ b/build_library/catalyst.sh @@ -110,14 +110,6 @@ EOF catalyst_stage_default 1 } -catalyst_stage2() { -cat < "$TEMPDIR/stage2.spec" - build_stage stage2 - used_seed=1 - fi - if [[ "$STAGES" =~ stage3 ]]; then if [[ $used_seed -eq 1 ]]; then - SEED="${TYPE}/stage2-${ARCH}-latest" + SEED="${TYPE}/stage1-${ARCH}-latest" fi info " stage3.spec" catalyst_stage3 > "$TEMPDIR/stage3.spec" diff --git a/sdk_container/src/third_party/portage-stable/scripts/bootstrap.sh b/sdk_container/src/third_party/portage-stable/scripts/bootstrap.sh index e07310e8e8e..61bbb0ab9a0 100755 --- a/sdk_container/src/third_party/portage-stable/scripts/bootstrap.sh +++ b/sdk_container/src/third_party/portage-stable/scripts/bootstrap.sh @@ -263,10 +263,6 @@ for atom in portage.settings.packages: [[ -z ${myZLIB} ]] && myZLIB="sys-libs/zlib" [[ -z ${myNCURSES} ]] && myNCURSES="sys-libs/ncurses" [[ -z ${myOS_HEADERS} ]] && myOS_HEADERS="$(portageq expand_virtual / virtual/os-headers)" -# Flatcar: install curl with BOOTSTRAP_USE=ssl to fetch from https URLs -[[ -z ${myCURL} ]] && myCURL="net-misc/curl" -# Flatcar: upgrade to openssl-3 before system rebuild in stage3 -[[ -z ${myOPENSSL} ]] && myOPENSSL="dev-libs/openssl" # Do we really want gettext/nls? [[ ${USE_NLS} != 1 ]] && myGETTEXT= @@ -281,10 +277,6 @@ einfo "Using libc : ${myLIBC}" einfo "Using texinfo : ${myTEXINFO}" einfo "Using zlib : ${myZLIB}" einfo "Using ncurses : ${myNCURSES}" -# Flatcar: install curl with BOOTSTRAP_USE=ssl to fetch from https URLs -einfo "Using curl : ${myCURL}" -# Flatcar: upgrade to openssl-3 before system rebuild in stage3 -einfo "Using openssl : ${myOPENSSL}" echo ------------------------------------------------------------------------------- show_status 1 Configuring environment echo ------------------------------------------------------------------------------- @@ -316,12 +308,9 @@ if [ ${BOOTSTRAP_STAGE} -le 2 ] ; then STRAP_EMERGE_OPTS="${STRAP_EMERGE_OPTS} --resume" cp /var/run/bootstrap-mtimedb /var/cache/edb else - # Flatcar: install curl with BOOTSTRAP_USE=ssl to fetch from https URLs STRAP_EMERGE_POSARGS="\ ${myOS_HEADERS} ${myTEXINFO} ${myGETTEXT} ${myBINUTILS} \ - ${myGCC} ${myLIBC} ${myCURL} ${myBASELAYOUT} ${myZLIB}" - # Flatcar: upgrade to openssl-3 before system rebuild in stage3 - STRAP_EMERGE_POSARGS="${STRAP_EMERGE_POSARGS} ${myOPENSSL}" + ${myGCC} ${myLIBC} ${myBASELAYOUT} ${myZLIB}" fi ${V_ECHO} emerge ${STRAP_EMERGE_OPTS} ${STRAP_EMERGE_POSARGS} || cleanup 1 echo ------------------------------------------------------------------------------- From 44223cbbe0fb27b342172b84cdd70d5c9ef67fba Mon Sep 17 00:00:00 2001 From: James Le Cuirot Date: Thu, 1 Aug 2024 10:18:06 +0100 Subject: [PATCH 2/2] Drop obsolete comments about how Catalyst stage 1 is built Signed-off-by: James Le Cuirot --- bootstrap_sdk | 31 ------------------------------- 1 file changed, 31 deletions(-) diff --git a/bootstrap_sdk b/bootstrap_sdk index 15ac6ce9bda..812a86209ee 100755 --- a/bootstrap_sdk +++ b/bootstrap_sdk @@ -87,37 +87,6 @@ mkdir -p "${ROOT_OVERLAY}/tmp" chmod 1777 "${ROOT_OVERLAY}/tmp" cp "${BUILD_LIBRARY_DIR}/toolchain_util.sh" "${ROOT_OVERLAY}/tmp" - -# Stage 1 uses "known-good" ebuilds (from both coreos-overlay and portage-stable) -# to build a minimal toolchain (USE="-*") for stage 1. -# -# No package updates must happen in stage 1, so we use the portage-stable and -# coreos-overlay paths included with the current SDK (from the SDK chroot's -# /var/gentoo/repos/). "Current SDK" refers to the SDK we entered with -# 'cork enter', i.e. the SDK we run ./bootstrap_sdk in. -# -# Using ebuilds from the above mentioned sources will ensure that stage 1 builds -# a stage 3 from known-good ebuild versions - the same ebuild versions -# that were used to build the very SDK we run ./bootstrap_sdk in. -# -# DANGER ZONE -# -# Stage 1 lacks proper isolation and will link all packages built for -# stage 3 against its own seed libraries ("/" in the catalyst chroot) instead of against libraries -# installed into the FS root of the stage 3 seed ("/tmp/stage1root" in the catalyst chroot). -# This is why we must prevent any updated package ebuilds to "leak" into stage 1, hence we use -# "known good" ebuild repo versions outlined above. -# -# In special circumstances it may be required to circumvent this and use custom paths -# for either (or both) portage and overlay. The command line options -# --stage1-portage-path and --stage1-overlay-path may be used to specify -# a repo path known to work for stage1. In that case the stage1 seed (i.e. the seed SDK) -# will be updated prior to starting to build stage 3. -# NOTE that this should never be used to introduce library updates in stage 1. All binaries -# produced in stage 1 are linked against libraries in the seed tarball, NOT libraries produced -# by stage 1. Therefore, these binaries will cease to work in stage 3 when linked against -# outdated "seed tarball" libraries which have been updated to newer versions in stage 1. - catalyst_build if [[ "$STAGES" =~ stage4 ]]; then