From 0eb549a5842ef714b4983e0f2f2e1865c3cb0e3e Mon Sep 17 00:00:00 2001 From: Jose Luis Rivero Date: Wed, 22 Nov 2023 19:57:24 +0100 Subject: [PATCH 1/3] [WIP] Document the job names Signed-off-by: Jose Luis Rivero --- jenkins-scripts/dsl/README.md | 101 ++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 jenkins-scripts/dsl/README.md diff --git a/jenkins-scripts/dsl/README.md b/jenkins-scripts/dsl/README.md new file mode 100644 index 000000000..717416ea2 --- /dev/null +++ b/jenkins-scripts/dsl/README.md @@ -0,0 +1,101 @@ +# Job generation + +## Naming schema for the jobs generated + + * Separator: `-` reserved symbol. The different parts can not use the `-` symbol + * Parts: chars between separators or between a separator and the beginning or the end of the job name + * Section: a group of parts that have some meaning about the job goal + +### Categories of jobs + + * Jobs affecting the binary packages + * Jobs affecting the continuous integration of Gazebo libraries + +### Jobs affecting the binary packages + +Sections composing the name: + +``` +{package-name}-{action} +``` + +**Sections:** + +--- + +`{package_name}`: + * Parts: 1 + * Name of the binary package used as a target for the job action. Examples: `gz-fuel-tools9-` or `sdformat15-` + +--- + +`{action}`: + * Parts: 1..N + * Part 1: kind of actions to perform on the `{package_name}` defined + * Part 2..N: modifiers for the kind of actions in part 1. Examples: `-install-pkg-` or `-source` + +__Knowing actions and their parts__: + + * `debbuilder`: Linux builders for `{package_name}` + * `source`: job creating the sources when releasing `{package_name}` + * `install`: check for the correct installation of the Linux `{package_name}` + * `pkg`: TODO: remove pkg or merge with install name + * `{distro}`: Debian/Ubuntu distribution name used by the job. Example`: `jammy` + * `{arch}`: `{package_name}` architecture tested. Example `arm64` + * `install_bottle`: check for the correct installation of Mac `{package_name}` + * `{mac_flavour}`: variant of Mac package manager used by the job. Example`: `homebrew` + * `{arch}`: `{package_name}` architecture tested. Exmaple `amd64` + +### Jobs affecting the continuous integration of Gazebo libraries + +Sections composing the name: + +``` +{lib_name}-{testing_type}-{platform} +``` + +**Sections:** + +`{lib_name}`: + * Parts: 1 + * Name (without major versions) of the Gazebo library using underscores instead of hyphens. Examples: `gz_cmake` or `gz_fuel_tools` + +--- + +`{testing_type}`: + * Parts: 1..N + * Part 1: kind of testing job. Examples: ci, install, etc. + * Part 2..N: modifiers for the kind of testing job in part 1 + +__Knowing types and parts__: + + * `ci`: build and test that runs on + * `pr_any`: branch/gitref of a repository is defined as a job input parameter. `ci-pr_any-` + * `{branch}`: branch/gitref explicit name. Examples: `-ci-gz-cmake4-` or `-ci-gz-sim8-` + * `ci_asan`: build and test with asan sanitizers on + * `{branch}`: branch/gitref explicit name. `-ci_asan-sdf12-` + * `abichecker`: + * `any_to_any-ubuntu`: abi checking job for two branches defined a job input parameters. `-abichecker-any_to_any-ubuntu-` + * TODO: remove the ubuntu part using a migration strategy for existing PR + +Due to problems with the max length of paths in the Windows command line, the Windows jobs use a different schema: + * `{number}`: Windows short-name implying `-ci-{branch}` being branch library_name+{number}. Example: `-5-` on gz-common will imply branch gz-common5 + * `pr` Windows short-name implying `-ci-pr_any-`. Example: `-pr-` on gz-physics will imply the PR job on gz-physics + +--- + +TODO: merge this with the binary package platforms + +`{platform}` + * Parts: 1..2 + * Platform where the testing is done by the job + +__Knowing platforms and parts__: + + * `{distro}`: Debian/Ubuntu distribution name used by the job. Example: `jammy` + * `{arch}`: `{lib_name}` architecture tested. Example `arm64` + * `{mac_flavour}`: variant of Mac package manager used by the job. Example: `homebrew` + * `{arch}`: `{lib_name}` architecture tested. Example `amd64` + +Due to problems with the max length of paths in the Windows command line, the Windows jobs use a different schema: + * `win`: windows short-name implying colcon builds on amd64 using vcpkg packages. From 47fa1d3726126807f0b3d676b0e49c1e609dbefc Mon Sep 17 00:00:00 2001 From: Jose Luis Rivero Date: Thu, 23 Nov 2023 17:06:06 +0100 Subject: [PATCH 2/3] Typo Signed-off-by: Jose Luis Rivero --- jenkins-scripts/dsl/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/jenkins-scripts/dsl/README.md b/jenkins-scripts/dsl/README.md index 717416ea2..186e7f7bc 100644 --- a/jenkins-scripts/dsl/README.md +++ b/jenkins-scripts/dsl/README.md @@ -40,10 +40,10 @@ __Knowing actions and their parts__: * `source`: job creating the sources when releasing `{package_name}` * `install`: check for the correct installation of the Linux `{package_name}` * `pkg`: TODO: remove pkg or merge with install name - * `{distro}`: Debian/Ubuntu distribution name used by the job. Example`: `jammy` + * `{distro}`: Debian/Ubuntu distribution name used by the job. Example: `jammy` * `{arch}`: `{package_name}` architecture tested. Example `arm64` * `install_bottle`: check for the correct installation of Mac `{package_name}` - * `{mac_flavour}`: variant of Mac package manager used by the job. Example`: `homebrew` + * `{mac_flavour}`: variant of Mac package manager used by the job. Example: `homebrew` * `{arch}`: `{package_name}` architecture tested. Exmaple `amd64` ### Jobs affecting the continuous integration of Gazebo libraries From d1cf9a053a92561fa676775071adc995000f6307 Mon Sep 17 00:00:00 2001 From: Jose Luis Rivero Date: Thu, 23 Nov 2023 18:08:48 +0100 Subject: [PATCH 3/3] Refactor common sections Signed-off-by: Jose Luis Rivero --- jenkins-scripts/dsl/README.md | 113 ++++++++++++++++++++++------------ 1 file changed, 73 insertions(+), 40 deletions(-) diff --git a/jenkins-scripts/dsl/README.md b/jenkins-scripts/dsl/README.md index 186e7f7bc..256220055 100644 --- a/jenkins-scripts/dsl/README.md +++ b/jenkins-scripts/dsl/README.md @@ -2,78 +2,124 @@ ## Naming schema for the jobs generated - * Separator: `-` reserved symbol. The different parts can not use the `-` symbol + * Separator: `-` reserved symbol. The different parts can not use the `-` symbol with the exception of + the {package_name} section. * Parts: chars between separators or between a separator and the beginning or the end of the job name * Section: a group of parts that have some meaning about the job goal ### Categories of jobs - * Jobs affecting the binary packages - * Jobs affecting the continuous integration of Gazebo libraries + 0. Common sections + 1. Jobs affecting releasing + 2. Jobs affecting the continuous integration of Gazebo libraries + 3. Jobs affecting the testing and building of the source code -### Jobs affecting the binary packages +### 0. Common sections -Sections composing the name: +Parts composing composing the **package_name** section: + +`{package_name}`: + * Parts: 1 + * Name of the gazebo library with its major version (as appearing in CMakeLists) used as a target for the job action. + Using dashes is aceptable only in this case. Examples: `gz-fuel-tools9-` or `sdformat15-` +Parts composing the **platform** section: ``` -{package-name}-{action} +{so_variant}-{arch} ``` +Due to problems with the max length of paths in the Windows command line, the Windows jobs use a different schema: + * `win`: windows short-name implying colcon builds on amd64 using vcpkg packages. + **Sections:** --- -`{package_name}`: +`{so_variant}` + * Parts: 2 + * Platform description relative to the operative system and/or package manager. Examples: + ubuntu distributions `jammy` and Mac Brew `homebrew` + * TODO: separate concepts of OS and provisioner or build system + +--- + +`arch` * Parts: 1 - * Name of the binary package used as a target for the job action. Examples: `gz-fuel-tools9-` or `sdformat15-` + * Architecture used in the build. Examples `arm64` or `amd64` + + +### 1. Jobs affecting releasing + +Sections composing the name: + +``` +{package_name}-{release_stage} +``` + +**Sections:** + + `{package_name}`: see common sections --- -`{action}`: - * Parts: 1..N - * Part 1: kind of actions to perform on the `{package_name}` defined - * Part 2..N: modifiers for the kind of actions in part 1. Examples: `-install-pkg-` or `-source` +`{release_stage}`: + * Parts: 1 + * Release stage for the `{package_name}` defined. __Knowing actions and their parts__: * `debbuilder`: Linux builders for `{package_name}` * `source`: job creating the sources when releasing `{package_name}` - * `install`: check for the correct installation of the Linux `{package_name}` - * `pkg`: TODO: remove pkg or merge with install name - * `{distro}`: Debian/Ubuntu distribution name used by the job. Example: `jammy` - * `{arch}`: `{package_name}` architecture tested. Example `arm64` - * `install_bottle`: check for the correct installation of Mac `{package_name}` - * `{mac_flavour}`: variant of Mac package manager used by the job. Example: `homebrew` - * `{arch}`: `{package_name}` architecture tested. Exmaple `amd64` -### Jobs affecting the continuous integration of Gazebo libraries +### 2. Jobs affecting the testing of the binary packages + +``` +{package_name}-{testing_type}-{platform} +``` + +**Sections:** + +`{package_name}`: see common sections + +--- + +`{testing_type}`: literals: `-install-pkg-` or `-install_bottle-` + +--- + +`{platform}`: see common sections + + +### 3. Jobs affecting the testing and building of the source code Sections composing the name: ``` -{lib_name}-{testing_type}-{platform} +{lib_name}-{testing_type}-{platform} ``` **Sections:** `{lib_name}`: - * Parts: 1 - * Name (without major versions) of the Gazebo library using underscores instead of hyphens. Examples: `gz_cmake` or `gz_fuel_tools` + * Parts: 1 + * Name (without major versions) of the Gazebo library using underscores instead of hyphens. + Examples: `gz_cmake` or `gz_fuel_tools` --- -`{testing_type}`: +`{testing_type}`: * Parts: 1..N * Part 1: kind of testing job. Examples: ci, install, etc. * Part 2..N: modifiers for the kind of testing job in part 1 -__Knowing types and parts__: +__Knowing testing types and parts__: * `ci`: build and test that runs on * `pr_any`: branch/gitref of a repository is defined as a job input parameter. `ci-pr_any-` * `{branch}`: branch/gitref explicit name. Examples: `-ci-gz-cmake4-` or `-ci-gz-sim8-` * `ci_asan`: build and test with asan sanitizers on * `{branch}`: branch/gitref explicit name. `-ci_asan-sdf12-` + * `{platorm}`: see common sections * `abichecker`: * `any_to_any-ubuntu`: abi checking job for two branches defined a job input parameters. `-abichecker-any_to_any-ubuntu-` * TODO: remove the ubuntu part using a migration strategy for existing PR @@ -81,21 +127,8 @@ __Knowing types and parts__: Due to problems with the max length of paths in the Windows command line, the Windows jobs use a different schema: * `{number}`: Windows short-name implying `-ci-{branch}` being branch library_name+{number}. Example: `-5-` on gz-common will imply branch gz-common5 * `pr` Windows short-name implying `-ci-pr_any-`. Example: `-pr-` on gz-physics will imply the PR job on gz-physics - ---- - -TODO: merge this with the binary package platforms -`{platform}` - * Parts: 1..2 - * Platform where the testing is done by the job +--- -__Knowing platforms and parts__: - - * `{distro}`: Debian/Ubuntu distribution name used by the job. Example: `jammy` - * `{arch}`: `{lib_name}` architecture tested. Example `arm64` - * `{mac_flavour}`: variant of Mac package manager used by the job. Example: `homebrew` - * `{arch}`: `{lib_name}` architecture tested. Example `amd64` +`{platform}`: see common sections -Due to problems with the max length of paths in the Windows command line, the Windows jobs use a different schema: - * `win`: windows short-name implying colcon builds on amd64 using vcpkg packages.