Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cd): automatic AUR release #3081

Closed
wants to merge 69 commits into from
Closed
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
69c9927
feat(cd): AUR test packaging
Zimbrando Jan 10, 2024
2c8609f
fix(cd): test-aur-packaging job
Zimbrando Jan 15, 2024
6c7dc98
fix(cd): PKGBUILD softlink
Zimbrando Jan 15, 2024
118fbfc
fix(cd): pkgbuild generation task dependency
Zimbrando Jan 18, 2024
6194d75
fix(cd): missing pkg
Zimbrando Jan 19, 2024
5454013
feat(cd): add aur package publish action
Zimbrando Jan 20, 2024
e0d2f43
chore(cd): add maintainer tag
Zimbrando Jan 25, 2024
975e1ee
perf(cd): add version in AUR commit message
Zimbrando Jan 26, 2024
a1a9da6
fix(cd): missing quotes in template
Zimbrando Jan 26, 2024
8056a1b
chore(cd): merge with #3079
Zimbrando Jan 29, 2024
e459c72
perf(aur): add bash publish AUR script
Zimbrando Jan 31, 2024
4f312c7
perf: test-aur optimization
Zimbrando Jan 31, 2024
5884511
fix: add parameters check
Zimbrando Jan 31, 2024
f3f99cd
fix: UUOC in publish script
Zimbrando Jan 31, 2024
ac3037e
Merge branch 'master' into dev-aur
DanySK Jan 31, 2024
ff4004a
Update alchemist-full/PKGBUILD.template url
Zimbrando Jan 31, 2024
dd26b14
Update alchemist-full/PKGBUILD.template license
Zimbrando Jan 31, 2024
12c08cc
Merge branch 'master' into dev-aur
DanySK Jan 31, 2024
d829a53
perf: md5 sum, workflow
Zimbrando Jan 31, 2024
5c1a1be
Merge branch 'master' into dev-aur
DanySK Jan 31, 2024
2494ce9
Merge branch 'master' into dev-aur
DanySK Jan 31, 2024
e08abf1
Merge branch 'master' into dev-aur
DanySK Feb 1, 2024
84ecedf
Merge branch 'master' into dev-aur
DanySK Feb 1, 2024
642a9a8
Merge branch 'master' into dev-aur
DanySK Feb 1, 2024
2e77b31
Merge branch 'master' into dev-aur
DanySK Feb 1, 2024
98aeea7
Merge branch 'master' into dev-aur
DanySK Feb 1, 2024
1ac4fe7
fix: remove test-packages condition
Zimbrando Feb 1, 2024
a6c51a7
ci(release): wrap variable expansion in "
DanySK Feb 1, 2024
24a4295
ci(release): add a build-ref computation job
DanySK Feb 1, 2024
32dcd53
ci(release): remove concurrency groups from select-java-version
DanySK Feb 1, 2024
7685704
ci(release): add a comment for the build job
DanySK Feb 1, 2024
f1f9cfa
ci(release): use the build ref to determine the concurrency group of …
DanySK Feb 1, 2024
3231567
ci: unify the generation of packages and fat jars
DanySK Feb 1, 2024
f7b994f
ci(release): use the build ref to determine the concurrency group of …
DanySK Feb 1, 2024
d3cf9a5
ci(release): use the build ref to determine the concurrency group of …
DanySK Feb 1, 2024
6ffbc1d
ci(release): use the build ref to determine the concurrency group of …
DanySK Feb 1, 2024
6dd01c5
ci(release): use the build ref to determine the concurrency group of …
DanySK Feb 1, 2024
936c7f8
ci(release): use the build ref to determine the concurrency group of …
DanySK Feb 1, 2024
7697559
ci: use the build ref to determine the artefact names
DanySK Feb 1, 2024
6b3ea8c
ci: fix broken release concurrency
DanySK Feb 1, 2024
a5290de
ci: separate jobs with an emtpy line
DanySK Feb 1, 2024
0eccf0a
ci: fix build ref output syntax
DanySK Feb 1, 2024
1594d59
ci: unify the ci preparation
DanySK Feb 1, 2024
f4790f9
ci: unify the ci preparation
DanySK Feb 1, 2024
f6f4d4c
ci: favor the commit sha over the build id for artefact identification
DanySK Feb 1, 2024
9735257
ci: test for the existence of the `jpackageFull` task upfront
DanySK Feb 1, 2024
9b9b8a2
Merge branch 'master' into dev-aur
DanySK Feb 1, 2024
d5add3c
build: apply shadowjar only inside the alchemist-full
DanySK Feb 1, 2024
7b7f9c6
Merge branch 'master' into dev-aur
DanySK Feb 1, 2024
663bd7f
build: move the fatJar verification on assemble
DanySK Feb 1, 2024
34dfc3d
build: disable distTar and distZip
DanySK Feb 1, 2024
72e5182
ci: try a may-deploy step
DanySK Feb 1, 2024
d6e0610
ci: build a more robust ci preparation step
DanySK Feb 1, 2024
d5fc8d6
Merge branch 'master' into dev-aur
DanySK Feb 1, 2024
5d66033
Merge branch 'master' into dev-aur
DanySK Feb 1, 2024
16475a4
Merge branch 'master' into dev-aur
DanySK Feb 2, 2024
c5cdbe7
Merge branch 'master' into dev-aur
DanySK Feb 2, 2024
b691e57
Merge branch 'master' into dev-aur
DanySK Feb 3, 2024
1e1884f
Merge branch 'master' into dev-aur
DanySK Feb 3, 2024
0c8a44c
Merge branch 'master' into dev-aur
DanySK Feb 3, 2024
0ec9553
Merge branch 'master' into dev-aur
DanySK Feb 4, 2024
703c45f
Merge branch 'master' into dev-aur
DanySK Feb 4, 2024
75cd8c4
Merge branch 'master' into dev-aur
DanySK Feb 4, 2024
2c066b9
Merge branch 'master' into dev-aur
DanySK Feb 4, 2024
a3aa8cf
Merge branch 'master' into dev-aur
DanySK Feb 4, 2024
78aa620
Merge branch 'master' into dev-aur
DanySK Feb 4, 2024
9853278
Merge branch 'master' into dev-aur
DanySK Feb 5, 2024
96ae558
Merge branch 'master' into dev-aur
DanySK Feb 5, 2024
9e12fe1
Merge branch 'master' into dev-aur
DanySK Feb 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 49 additions & 1 deletion .github/workflows/build-and-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,50 @@ jobs:
- name: Test package
shell: bash
run: ./gradlew testJpackageOutput --info
# Test PKGBUILD for aur release
test-aur-package:
needs:
- select-java-version
- build
runs-on: ubuntu-22.04
container:
image: archlinux:base-devel
Zimbrando marked this conversation as resolved.
Show resolved Hide resolved
if: >-
github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository
Zimbrando marked this conversation as resolved.
Show resolved Hide resolved
concurrency:
group: ${{ github.workflow }}-test-aur-release-${{ github.event.number || github.ref }}
cancel-in-progress: true
steps:
- name: Setup arch
run: |
sudo pacman --noconfirm -Syu
sudo pacman --noconfirm -Sy git rpm-tools
- run: git config --system --add safe.directory /__w/Alchemist/Alchemist # https://github.com/actions/checkout/issues/1169
- name: Checkout
uses: danysk/[email protected]
- name: Setup Java
uses: actions/[email protected]
with:
java-version: ${{ needs.select-java-version.outputs.java-version }}
distribution: 'temurin'
- name: Download packages
uses: actions/[email protected]
with:
pattern: installer-package-Linux
path: build/package/
merge-multiple: true
- name: Generate PKGBUILD
run: ./gradlew generatePKGBUILD -DgeneratePKGBUILD.testSource=true
- name: Makepkg Build and Check
uses: edlanglois/pkgbuild-action@v1
with:
pkgdir: build/package
- name: Test installation
working-directory: build/package
run: |
sudo pacman -U --noconfirm alchemist-*.zst
(sudo pacman -Q | grep -q alchemist && echo "Success! Alchemist found in the package list") || exit 1
(/usr/bin/alchemist --help | grep -q options_list && echo "Success! Alchemist available") || exit 2
# Builds the website and verifies link sanity
build-website:
needs:
Expand Down Expand Up @@ -220,6 +264,7 @@ jobs:
- build-website
- test-deploy
- test-packages
- test-aur-package
runs-on: ubuntu-22.04
if: >-
!github.event.repository.fork
Expand Down Expand Up @@ -291,7 +336,9 @@ jobs:
maven-central-password: ${{ secrets.MAVEN_CENTRAL_PASSWORD }}
signing-key: ${{ secrets.SIGNING_KEY }}
signing-password: ${{ secrets.SIGNING_PASSWORD }}

custom-secret-0: ${{ secrets.AUR_USERNAME }}
custom-secret-1: ${{ secrets.AUR_EMAIL }}
custom-secret-2: ${{ secrets.AUR_SSH_PRIVATE_KEY }}
success:
runs-on: ubuntu-22.04
needs:
Expand All @@ -300,6 +347,7 @@ jobs:
- release
- test-deploy
- test-packages
- test-aur-package
if: >-
always() && (
contains(join(needs.*.result, ','), 'failure')
Expand Down
26 changes: 26 additions & 0 deletions alchemist-full/PKGBUILD.template
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Maintainer: Marco Sternini <[email protected]>
# Maintainer: Danilo Pianini <[email protected]>
pkgname=alchemist
pkgver={%}
pkgrel=1
pkgdesc="An extensible simulator for pervasive computing"
arch=('x86_64')
url="https://github.com/AlchemistSimulator/Alchemist"
Zimbrando marked this conversation as resolved.
Show resolved Hide resolved
license=('GPL')
Zimbrando marked this conversation as resolved.
Show resolved Hide resolved
depends=('glibc' 'libxrender' 'libxtst' 'libxi' 'libx11' 'freetype2' 'alsa-lib' 'libxext')
makedepends=()
source=("{%}")
md5sums=('{%}')

package() {
mkdir -p "${pkgdir}/usr/share/${pkgname}"
mkdir -p "${pkgdir}/usr/lib/"
mkdir -p "${pkgdir}/usr/bin/"

# Copy the application in usr/lib
cp -r "${srcdir}/usr/share/licenses/" "${pkgdir}/usr/share/alchemist/"
cp -r "${srcdir}/opt/${pkgname}/" "${pkgdir}/usr/lib/"

# Create a soft link from usr/bin to the application launcher
ln -s "/usr/lib/${pkgname}/bin/${pkgname}" "${pkgdir}/usr/bin/${pkgname}"
}
37 changes: 37 additions & 0 deletions alchemist-full/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import Util.isMultiplatform
import Util.isWindows
import org.panteleyev.jpackage.ImageType
import org.panteleyev.jpackage.JPackageTask
import java.io.ByteArrayOutputStream

plugins {
application
Expand Down Expand Up @@ -145,6 +146,42 @@ tasks.register<Exec>("testJpackageOutput") {
finalizedBy(deleteJpackageOutput)
}

val generatePKGBUILD by tasks.registering(Exec::class) {
Zimbrando marked this conversation as resolved.
Show resolved Hide resolved
group = "Publishing"
description = "Generates a valid PKGBUILD by replacing values in the template file"
val inputFile = file("${project.projectDir}/PKGBUILD.template")
val outputDir = file(rootProject.layout.buildDirectory.map { it.dir("package") }.get().asFile.path)
val tokenToReplace = "{%}"
var replacementValues: List<String>
val testSourceParam = System.getProperty("generatePKGBUILD.testSource", "false")
val interceptOutput = ByteArrayOutputStream()
standardOutput = interceptOutput
doFirst {
val version = rootProject.version.toString().substringBefore('-')
commandLine("md5sum", "-b", "${rootProject.layout.buildDirectory.get().asFile.resolve("package").path}/${rootProject.name}-$version-1.x86_64.rpm")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need -1 at the end?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Jpackage outputs packages following that syntax. I suppose it's the release number.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There must be a reason, can you investigate? If there is no reason for it to have -1, I'd like to remove it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a naming convention for .rpm packages, .deb packages follow something similar too

This is tradition and not a restriction, however. Like the version number, the only restriction is that dashes are not allowed. source

I found the jpackage setting, but it seems impossible to disable.
We can modify it to whatever string we would like

Zimbrando marked this conversation as resolved.
Show resolved Hide resolved
}
doLast {
val exit = executionResult.get().exitValue
require(exit == 0)
val md5 = String(interceptOutput.toByteArray(), Charsets.UTF_8).split(' ')[0]
val version = rootProject.version.toString().substringBefore('-')
val fileContent = inputFile.readText()
replacementValues = listOf(
version, // pkgver
if (testSourceParam.toBoolean()) { "${rootProject.name}-$version-1.x86_64.rpm" } else { "https://github.com/AlchemistSimulator/Alchemist/releases/download/$version/${rootProject.name}-$version-1.x86_64.rpm" }, // source name
md5, // md5sums
)
val replacedContent = replacementValues.foldIndexed(fileContent) { _, content, value ->
content.replaceFirst(tokenToReplace, value)
}
outputDir.mkdirs()
val file = outputDir.resolve("PKGBUILD")
file.createNewFile()
file.writeText(replacedContent)
}
mustRunAfter(jpackageFull)
}

tasks.withType<AbstractArchiveTask> {
duplicatesStrategy = DuplicatesStrategy.WARN
}
Expand Down
60 changes: 60 additions & 0 deletions publishToAUR.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
#!/bin/bash

# AUR publishing script

set -o errexit
set -o nounset
set -o pipefail

Help()
{
# Display Help
echo "AUR publishing script for alchemist"
echo
echo "Syntax: publishToAUR pkgbuild commit_username commit_email ssh_private_key"
echo
}

if [[ $# -ne 4 ]]; then
Help
exit 1
fi

aur_repo_dir="/tmp/aur-repo"
main_repo_dir=$(pwd)

pkgname="alchemist"
pkgbuild="$1"
username="$2"
email="$3"
ssh_key="$4"

echo "-- Setting up ssh configuration"
ssh-keyscan -v -t "rsa" aur.archlinux.org >> ~/.ssh/known_hosts
echo "$ssh_key" > ~/.ssh/aur
chmod -vR 600 ~/.ssh/aur*
ssh-keygen -vy -f ~/.ssh/aur > ~/.ssh/aur.pub
echo "Host aur.archlinux.org
IdentityFile ~/.ssh/aur
User $username" > ~/.ssh/config

echo "-- Cloning the AUR repository in $aur_repo_dir"
git clone -v "https://aur.archlinux.org/${pkgname}.git" $aur_repo_dir

(
cd $aur_repo_dir
git config user.name "$username"
git config user.email "$email"
cp -r "$main_repo_dir/$pkgbuild" $aur_repo_dir
# Retrieve the version from the PKGBUILD
version=$(< PKGBUILD grep pkgver | cut -d'=' -f 2)
makepkg --printsrcinfo > .SRCINFO
echo "-- Committing the update to version $version"

git add -fv PKGBUILD .SRCINFO
git commit -m "Update to version $version"
git remote add aur "ssh://[email protected]/${pkgname}.git"

echo "-- Publishing the version $version"
git push -v aur master
)
2 changes: 2 additions & 0 deletions release.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ git -C build/website/ push || exit 3
RELEASE_ON_CENTRAL="./gradlew uploadKotlinOSSRH uploadKotlinMultiplatform uploadJvm uploadJs release --parallel"
eval "$RELEASE_ON_CENTRAL" || eval "$RELEASE_ON_CENTRAL" || eval "$RELEASE_ON_CENTRAL" || exit 5
./gradlew publishKotlinOSSRHPublicationToGithubRepository --continue || true
./gradlew generatePKGBUILD
Zimbrando marked this conversation as resolved.
Show resolved Hide resolved
./publishToAUR.sh build/package/PKGBUILD $CUSTOM_SECRET_0 $CUSTOM_SECRET_1 "$CUSTOM_SECRET_2" || exit 9
`
var config = require('semantic-release-preconfigured-conventional-commits');
config.plugins.push(
Expand Down
Loading