Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Packaging] Use CentOS Stream 8 to build el8 RPM package #20918

Merged
merged 12 commits into from
Mar 15, 2022
Merged

[Packaging] Use CentOS Stream 8 to build el8 RPM package #20918

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
74c2205
el8
jiasli Jan 6, 2022
521ceea
update
jiasli Jan 6, 2022
756e664
Merge remote-tracking branch 'azure/dev' into el8
jiasli Feb 9, 2022
36d3389
Add comment
jiasli Feb 9, 2022
ec44277
comment out docker save
jiasli Feb 9, 2022
3ca5d20
Rename yum to rpm
jiasli Feb 9, 2022
7b5a07e
Merge remote-tracking branch 'azure/dev' into el8
jiasli Feb 9, 2022
340c4be
Do not use matrix
jiasli Feb 9, 2022
0b6287c
Update azure-pipelines.yml
jiasli Feb 9, 2022
e88e373
Merge remote-tracking branch 'azure/dev' into el8
jiasli Mar 11, 2022
7a6e98c
update
jiasli Mar 11, 2022
5050e7e
Update test_rpm_in_docker.sh
jiasli Mar 11, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions azure-pipelines.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -700,17 +700,25 @@ jobs:
displayName: Build Yum Package
pool:
vmImage: 'ubuntu-20.04'
strategy:
matrix:
CentOS7:
image: centos7
CentOSStream8:
image: centos_stream8
steps:
- task: Bash@3
displayName: 'Build Rpm Package'
inputs:
targetType: 'filePath'
filePath: scripts/release/rpm/pipeline.sh
env:
IMAGE: $(image)
- task: PublishPipelineArtifact@0
displayName: 'Publish Artifact: yum'
inputs:
TargetPath: $(Build.ArtifactStagingDirectory)
ArtifactName: yum
ArtifactName: yum-$(image)


- job: TestYumPackage
Expand All @@ -732,7 +740,7 @@ jobs:
displayName: 'Download Build Artifacts'
inputs:
TargetPath: '$(Build.ArtifactStagingDirectory)/yum'
artifactName: yum
artifactName: yum-centos7


- bash: |
Expand Down
0 scripts/release/rpm/Dockerfile.centos → scripts/release/rpm/Dockerfile.centos7
View file
Open in desktop
File renamed without changes.
24 changes: 24 additions & 0 deletions scripts/release/rpm/Dockerfile.centos_stream8
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
ARG tag=stream8

FROM quay.io/centos/centos:${tag} AS build-env
ARG cli_version=dev

RUN yum update -y
RUN yum install -y wget rpm-build gcc libffi-devel python3-devel openssl-devel make bash diffutils patch dos2unix python3-virtualenv perl
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

coreutils is not compatible with preinstalled coreutils-single, so we don't need to install coreutils.

# dnf list installed | grep coreutils
coreutils-single.x86_64                       8.30-12.el8                     @baseos

# yum install -y coreutils
Last metadata expiration check: 0:13:00 ago on Thu Jan  6 09:04:58 2022.
Error:
 Problem: problem with installed package coreutils-single-8.30-12.el8.x86_64
  - package coreutils-8.30-12.el8.x86_64 conflicts with coreutils-single provided by coreutils-single-8.30-12.el8.x86_64
  - package coreutils-8.30-12.el8.x86_64 conflicts with coreutils-single provided by coreutils-single-8.30-10.el8.x86_64
  - package coreutils-8.30-12.el8.x86_64 conflicts with coreutils-single provided by coreutils-single-8.30-8.el8.x86_64
  - package coreutils-8.30-12.el8.x86_64 conflicts with coreutils-single provided by coreutils-single-8.30-9.el8.x86_64
  - cannot install the best candidate for the job
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)

Copy link
Member Author

@jiasli jiasli Jan 6, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perl is not installed by rpm-build on CentOS Stream 8, so we should manually install it.


WORKDIR /azure-cli

COPY . .

RUN dos2unix ./scripts/release/rpm/azure-cli.spec && \
REPO_PATH=$(pwd) CLI_VERSION=$cli_version rpmbuild -v -bb --clean scripts/release/rpm/azure-cli.spec && \
cp /root/rpmbuild/RPMS/x86_64/azure-cli-${cli_version}-1.*.x86_64.rpm /azure-cli-dev.rpm

FROM quay.io/centos/centos:${tag} AS execution-env

RUN yum update -y
RUN yum install -y python3 python3-virtualenv

COPY --from=build-env /azure-cli-dev.rpm ./
RUN rpm -i ./azure-cli-dev.rpm && \
az --version
24 changes: 14 additions & 10 deletions scripts/release/rpm/pipeline.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,34 +6,38 @@ set -exv

: "${BUILD_STAGINGDIRECTORY:?BUILD_STAGINGDIRECTORY environment variable not set.}"

# IMAGE should be one of 'centos7', 'centos_stream8'
: "${IMAGE:?IMAGE environment variable not set.}"

CLI_VERSION=`cat src/azure-cli/azure/cli/__main__.py | grep __version__ | sed s/' '//g | sed s/'__version__='// | sed s/\"//g`

# Create a container image that includes the source code and a built RPM using this file.
docker build \
--target build-env \
--build-arg cli_version=${CLI_VERSION} \
-f ./scripts/release/rpm/Dockerfile.centos \
-t azure/azure-cli:centos7-builder \
-f ./scripts/release/rpm/Dockerfile.${IMAGE} \
-t azure/azure-cli:${IMAGE}-builder \
.

# Continue the previous build, and create a container that has the current azure-cli build but not the source code.
docker build \
--build-arg cli_version=${CLI_VERSION} \
-f ./scripts/release/rpm/Dockerfile.centos \
-t azure/azure-cli:centos7 \
-f ./scripts/release/rpm/Dockerfile.${IMAGE} \
-t azure/azure-cli:${IMAGE} \
.

# Extract the built RPM so that it can be distributed independently.
docker run \
azure/azure-cli:centos7-builder \
cat /root/rpmbuild/RPMS/x86_64/azure-cli-${CLI_VERSION}-1.el7.x86_64.rpm \
> ${BUILD_STAGINGDIRECTORY}/azure-cli-${CLI_VERSION}-1.el7.x86_64.rpm
# The RPM file looks like azure-cli-2.32.0-1.el7.x86_64.rpm
id=$(docker create azure/azure-cli:${IMAGE}-builder)
# https://docs.docker.com/engine/reference/commandline/cp/
# Append /. so that the x86_64 folder's content is copied, instead of x86_64 folder itself.
docker cp $id:/root/rpmbuild/RPMS/x86_64/. ${BUILD_STAGINGDIRECTORY}

# Save these too a staging directory so that later build phases can choose to save them as Artifacts or publish them to
# a registry.
#
# The products of `docker save` can be rehydrated using `docker load`.
mkdir -p ${BUILD_STAGINGDIRECTORY}/docker
docker save azure/azure-cli:centos7-builder | gzip > ${BUILD_STAGINGDIRECTORY}/docker/azure_azure-cli_centos7-builder.tar.gz &
docker save azure/azure-cli:centos7 | gzip > ${BUILD_STAGINGDIRECTORY}/docker/azure_azure-cli_centos7.tar.gz &
docker save azure/azure-cli:${IMAGE}-builder | gzip > ${BUILD_STAGINGDIRECTORY}/docker/azure_azure-cli_${IMAGE}-builder.tar.gz &
docker save azure/azure-cli:${IMAGE} | gzip > ${BUILD_STAGINGDIRECTORY}/docker/azure_azure-cli_${IMAGE}.tar.gz &
wait