ign -> gz : Release Tools Migrate DSL #750

methylDragon · 2022-06-24T01:01:02Z

Part of: #746

This PR attempts to migrate internal variables inside DSL files to whittle down the number of occurrences of ign/ignition.

jenkins-scripts/dsl/ignition.dsl

jenkins-scripts/dsl/ignition_collection.dsl

chapulina · 2022-06-24T03:05:14Z

source-repo-scripts/bump_dependency.bash

@@ -347,8 +347,10 @@ for ((i = 0; i < "${#LIBRARIES[@]}"; i++)); do
  cd ${TEMP_DIR}/homebrew-simulation
  startFromCleanBranch ${BUMP_BRANCH} master

+  # NOTE(CH3): This won't interfere with $LIB=gzXXX so it's fine
  # expand ign-* to ignition-*
  FORMULA_BASE=${LIB/ign/ignition}


We'll only use this script to bump gz now, from Gazebo-H onwards, so I think it's ok to remove all ign / ignition

j-rivero · 2022-08-03T17:56:08Z

Not sure what is the goal of this PR @methylDragon . I see changes in variable names inside the DSL but not sure if we are trying to deal with other script names in release-tools. Could you please complete the PR description with the goals of the changes, please?

chapulina · 2022-08-03T18:07:28Z

I see changes in variable names inside the DSL but not sure if we are trying to deal with other script names in release-tools

We've been trying to change everything we can to reduce the clutter when grepping for ign. That includes internal variables that aren't external-facing. This makes it easier to find and maintain the places where ign is still needed, and modernizes the codebase so we don't have legacy ign references hanging around.

As for the external facing stuff, we need a plan to migrate the names and URLs that show up in http://build.osrfoundation.org. Job names like _dsl_ignition, https://build.osrfoundation.org/job/ignition_gazebo-ci-main-focal-amd64/, etc. We'll need to update jobs and badges in the source code when we make these changes.

methylDragon · 2022-08-17T19:34:14Z

There was a change in #779 that I'd like to tweak

                  software_name="gz-${ign_sw}"
                  [[ ${ign_sw} == 'gazebo' ]] && software_name="gz-sim"
                  /bin/bash -xe "./scripts/jenkins-scripts/lib/project-default-devel-homebrew-amd64.bash" "\${software_name}"

We can leverage variable expansion, it's more concise, I think, if gazebo -> sim is the only case we're considering

${gz_sw/sim/gazebo}

I'm making this change during my rebase of this branch (since it's a conflict), do let me know if you'd rather I revert it, and I will

EDIT: I ended up going back to using software_name because I don't understand the syntax enough and I couldn't get the substitutions to work for some reason...

Signed-off-by: methylDragon <[email protected]>

Find: (ign(ition)?[-_])\$\{gz([^/|-|\}]*)\} Replace: $1${gz$3/sim/gazebo} Signed-off-by: methylDragon <[email protected]>

methylDragon · 2022-08-17T19:47:17Z

See: e3a684b for more changes!

I spotted more internal vars!

methylDragon · 2022-08-17T22:34:09Z

RFC: I think special attention needs to be paid to 8776491

I'm not sure of the syntax or if what I'm doing makes sense, but tests pass

Signed-off-by: methylDragon <[email protected]>

j-rivero · 2022-08-19T16:49:34Z

Run a test on #795 the real XML changes displayed are:

diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/ignition_citadel-ci-main-homebrew-amd64.xml /tmp/pr_xml_configuration/ignition_citadel-ci-main-homebrew-amd64.xml
--- /tmp/current_xml_configuration/ignition_citadel-ci-main-homebrew-amd64.xml	2022-08-19 16:46:00.323093363 +0000
+++ /tmp/pr_xml_configuration/ignition_citadel-ci-main-homebrew-amd64.xml	2022-08-19 16:44:48.238185025 +0000
@@ -64,7 +64,7 @@
         <hudson.tasks.Shell>
             <command>#!/bin/bash -xe
 
-/bin/bash -xe "./scripts/jenkins-scripts/lib/project-default-devel-homebrew-amd64.bash" "gz-citadel"
+/bin/bash -xe "./scripts/jenkins-scripts/lib/project-default-devel-homebrew-amd64.bash" "ignition-citadel"
 </command>
         </hudson.tasks.Shell>
     </builders>
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/ignition_fortress-ci-main-homebrew-amd64.xml /tmp/pr_xml_configuration/ignition_fortress-ci-main-homebrew-amd64.xml
--- /tmp/current_xml_configuration/ignition_fortress-ci-main-homebrew-amd64.xml	2022-08-19 16:46:00.879101044 +0000
+++ /tmp/pr_xml_configuration/ignition_fortress-ci-main-homebrew-amd64.xml	2022-08-19 16:44:48.810193675 +0000
@@ -64,7 +64,7 @@
         <hudson.tasks.Shell>
             <command>#!/bin/bash -xe
 
-/bin/bash -xe "./scripts/jenkins-scripts/lib/project-default-devel-homebrew-amd64.bash" "gz-fortress"
+/bin/bash -xe "./scripts/jenkins-scripts/lib/project-default-devel-homebrew-amd64.bash" "ignition-fortress"
 </command>
         </hudson.tasks.Shell>
     </builders>
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/ignition_garden-ci-main-homebrew-amd64.xml /tmp/pr_xml_configuration/ignition_garden-ci-main-homebrew-amd64.xml
--- /tmp/current_xml_configuration/ignition_garden-ci-main-homebrew-amd64.xml	2022-08-19 16:46:01.431108670 +0000
+++ /tmp/pr_xml_configuration/ignition_garden-ci-main-homebrew-amd64.xml	2022-08-19 16:44:49.366202084 +0000
@@ -64,7 +64,7 @@
         <hudson.tasks.Shell>
             <command>#!/bin/bash -xe
 
-/bin/bash -xe "./scripts/jenkins-scripts/lib/project-default-devel-homebrew-amd64.bash" "gz-garden"
+/bin/bash -xe "./scripts/jenkins-scripts/lib/project-default-devel-homebrew-amd64.bash" "ignition-garden"
 </command>
         </hudson.tasks.Shell>
     </builders>

j-rivero · 2022-08-19T17:00:42Z

Run a test on #795 the real XML changes displayed are:

Should be fixed with 662c857

source-repo-scripts/bump_dependency.bash

Co-authored-by: methylDragon <[email protected]>

j-rivero

I think we are good to go.

Blast545 · 2022-08-19T18:31:10Z

Minor comment: I see that two of the files changed are named jenkins-scripts/dsl/ignition.dsl and jenkins-scripts/dsl/ignition_collection.dsl, should these names be changed to gazebo as well?

Context question: after this is deployed, do we lose history of the buildfarm jobs with the old ignition names?

@Crola1702 FYI: Waiting for this to be merged before fixing the internal buildfarm tools.

j-rivero · 2022-08-22T14:24:53Z

Minor comment: I see that two of the files changed are named jenkins-scripts/dsl/ignition.dsl and jenkins-scripts/dsl/ignition_collection.dsl, should these names be changed to gazebo as well?

We are not renaming files in this PR, just changing the internal variable names if I'm not wrong.

Context question: after this is deployed, do we lose history of the buildfarm jobs with the old ignition names?

AFAIK we are not changing any job name too. If you check the work in #795, the artifacts generated are the files changed and content changed in these files. Should be completely empty now.

methylDragon requested a review from j-rivero as a code owner June 24, 2022 01:01

methylDragon mentioned this pull request Jun 24, 2022

ign -> gz Partial Release Tools Migration #746

Closed

chapulina added the ign to gz Renaming Ignition to Gazebo. label Jun 24, 2022

chapulina reviewed Jun 24, 2022

View reviewed changes

methylDragon force-pushed the migrate_dsl branch from f9a79e7 to b3db625 Compare June 25, 2022 06:45

methylDragon force-pushed the migrate_dsl branch 2 times, most recently from 8041e7d to 576ff9a Compare August 4, 2022 20:09

methylDragon added 3 commits August 17, 2022 12:35

Migrate homebrew simulation and DSL replacements in bump_dependency

ae3790e

Signed-off-by: methylDragon <[email protected]>

TRIAL: Migrate ign_collection_name, ign_sw DSL vars

90c072a

Signed-off-by: methylDragon <[email protected]>

Prevent cases of ign-sim

1e8b18b

Find: (ign(ition)?[-_])\$\{gz([^/|-|\}]*)\} Replace: $1${gz$3/sim/gazebo} Signed-off-by: methylDragon <[email protected]>

methylDragon force-pushed the migrate_dsl branch from 3fa6b4c to 86fc254 Compare August 17, 2022 19:47

methylDragon force-pushed the migrate_dsl branch 15 times, most recently from d54d275 to 9fef16d Compare August 17, 2022 22:07

methylDragon force-pushed the migrate_dsl branch 2 times, most recently from 2d0a33f to a7a46f5 Compare August 17, 2022 22:33

Fix gz_sw

8776491

Signed-off-by: methylDragon <[email protected]>

methylDragon force-pushed the migrate_dsl branch from a7a46f5 to 8776491 Compare August 17, 2022 22:35

j-rivero added 3 commits August 18, 2022 21:43

Merge branch 'master' into migrate_dsl

d710629

Fix var name

4c0b7eb

Fix support to keep ign branches

23a2e96

methylDragon assigned j-rivero Aug 18, 2022

j-rivero added 8 commits August 18, 2022 23:56

Fix spacing and return brew trick

3f8ea64

Fix variable name

b78b57d

Fix Windows generation

4dff9c4

Fix debbuilder creation

5f9f834

Fix Windows CI

b0c84e2

Fix install jobs

e0d8b93

Fix problem with var name

94ec5a3

Use gz_sw variable instead of software name

3fdaea7

j-rivero mentioned this pull request Aug 19, 2022

(ci testing only, no merge) 750 + DSL testing in 794 #795

Closed

Restore gz- prefix for collection_name

662c857

methylDragon commented Aug 19, 2022

View reviewed changes

source-repo-scripts/bump_dependency.bash Outdated Show resolved Hide resolved

methylDragon commented Aug 19, 2022

View reviewed changes

source-repo-scripts/bump_dependency.bash Outdated Show resolved Hide resolved

Update source-repo-scripts/bump_dependency.bash

825bcaa

Co-authored-by: methylDragon <[email protected]>

j-rivero approved these changes Aug 19, 2022

View reviewed changes

Blast545 approved these changes Aug 19, 2022

View reviewed changes

j-rivero merged commit 891cf11 into master Aug 22, 2022

j-rivero deleted the migrate_dsl branch August 22, 2022 16:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ign -> gz : Release Tools Migrate DSL #750

ign -> gz : Release Tools Migrate DSL #750

methylDragon commented Jun 24, 2022 •

edited

Loading

chapulina Jun 24, 2022

j-rivero commented Aug 3, 2022

chapulina commented Aug 3, 2022

methylDragon commented Aug 17, 2022 •

edited

Loading

methylDragon commented Aug 17, 2022 •

edited

Loading

methylDragon commented Aug 17, 2022 •

edited

Loading

j-rivero commented Aug 19, 2022

j-rivero commented Aug 19, 2022

j-rivero left a comment

Blast545 commented Aug 19, 2022

j-rivero commented Aug 22, 2022

ign -> gz : Release Tools Migrate DSL #750

ign -> gz : Release Tools Migrate DSL #750

Conversation

methylDragon commented Jun 24, 2022 • edited Loading

chapulina Jun 24, 2022

Choose a reason for hiding this comment

j-rivero commented Aug 3, 2022

chapulina commented Aug 3, 2022

methylDragon commented Aug 17, 2022 • edited Loading

methylDragon commented Aug 17, 2022 • edited Loading

methylDragon commented Aug 17, 2022 • edited Loading

j-rivero commented Aug 19, 2022

j-rivero commented Aug 19, 2022

j-rivero left a comment

Choose a reason for hiding this comment

Blast545 commented Aug 19, 2022

j-rivero commented Aug 22, 2022

methylDragon commented Jun 24, 2022 •

edited

Loading

methylDragon commented Aug 17, 2022 •

edited

Loading

methylDragon commented Aug 17, 2022 •

edited

Loading

methylDragon commented Aug 17, 2022 •

edited

Loading