From 39e9019fb266887c66776517d261b54572711a07 Mon Sep 17 00:00:00 2001 From: Lorenzo Natali Date: Mon, 4 Mar 2024 17:08:19 +0100 Subject: [PATCH] Release Automation Process (#9555) * Add actions * Updated workflows for deprecations and security * Fixed changelog stuff * renamed release workflow file * Fixed concurrency * Separated release steps * Update for uniformity the scripts. Fixed wrong java modules set * Fixed wrong version system in binary * Fixed version update script * Updated documentation. Last fixes * Removed file committed in error * Improved docs * Improved docs * Improved docs * Minor fixes to doc * Fixed release procedure missing paragraph * Update procedure for linking latest stable doc * Update release_steps.md * Update .github/ISSUE_TEMPLATE/release_steps.md * Apply suggestions from code review Co-authored-by: Matteo V. --------- Co-authored-by: Matteo V. --- .github/ISSUE_TEMPLATE/release_steps.md | 83 +++++++------ .github/workflows/create_release.yml | 152 ++++++++++++++++++++++++ .github/workflows/cut_major_branch.yml | 97 +++++++++++++++ .github/workflows/post_release.yml | 48 ++++++++ .github/workflows/pre_release.yml | 149 +++++++++++++++++++++++ binary/bin-war/pom.xml | 4 +- binary/pom.xml | 4 +- docs/developer-guide/release.md | 146 ++++++++++++++++++++--- mkdocs.yml | 2 +- 9 files changed, 628 insertions(+), 57 deletions(-) create mode 100644 .github/workflows/create_release.yml create mode 100644 .github/workflows/cut_major_branch.yml create mode 100644 .github/workflows/post_release.yml create mode 100644 .github/workflows/pre_release.yml diff --git a/.github/ISSUE_TEMPLATE/release_steps.md b/.github/ISSUE_TEMPLATE/release_steps.md index e2b5519299..91531f3081 100644 --- a/.github/ISSUE_TEMPLATE/release_steps.md +++ b/.github/ISSUE_TEMPLATE/release_steps.md @@ -19,18 +19,24 @@ This steps have to be followed always when preparing a new release. ## New stable branch creation -If stable release (YYYY.XX.00) follow these sub-steps: - -- [ ] create a branch `YYYY.XX.xx` from master branch (`xx` is really `xx`, example: 2018.01.xx). This is the new **stable branch** -- [ ] Change [MapStore2-QA-Build](http://build.geosolutionsgroup.com/view/MapStore/job/MapStore/view/MapStore%20QA/job/MapStore2-QA-Build/) by updating the `branch` parameter in the build configuration page to `YYYY.XX.xx` -- [ ] on MapStore **stable branch** - - [ ] Fix `pom.xml` files to make sure that no `-SNAPSHOT` **dependencies** are used anymore. -- [ ] on MapStore **master branch** - - [ ] increase version of java modules. (`mvn versions:set -DnewVersion= -DprocessAllModules -DgenerateBackupPoms=false`). Where `` increases the major number. (e.g. `1.3-SNAPSHOT` --> `1.4-SNAPSHOT`) - - [ ] Manually update project pom templates to use `mapstore-services` of `` to the new one. (`projects/templates/web/pom.xml`). - - [ ] Increment version of `package.json` on master **0.<x-incremented>.0** with the command `npm version minor --git-tag-version=false` +**Only** if you need to create a new stable major release (YYYY.XX.00), you need to create a branch for it. Check the following: + +- [ ] Run the [`Cut Release Branch`](https://github.com/geosolutions-it/MapStore2/actions/workflows/cut_major_branch.yml) workflow on github. + With the following Parameters: + + - [ ] Use workflow from branch `master` + - [ ] MapStore branch name to use: `YYYY.XX.xx` + - [ ] Version of *MapFish Print*, *GeoStore* and *HTTP-Proxy* accordingly to the [MapStore release calendar](https://github.com/geosolutions-it/MapStore2/wiki/MapStore-Release-Calendars) + - [ ] use the default value for the other parameters +- [ ] Wait for the process to complete. At the end: + - A Pull request will be created to the master + - A new branch named `YYYY.XX.xx` with fixed versions +- [ ] Merge the incoming PR created by the workflow - [ ] Create on [ReadTheDocs](https://readthedocs.org/projects/mapstore/) project the version build for `YYYY.XX.xx` (click on "Versions" and activate the version of the branch) -- [ ] create a branch with the same name (`YYYY.XX.xx`) in [MapStoreExtension](https://github.com/geosolutions-it/MapStoreExtension) repository. +- [ ] Run the [`Cut Release Branch`](https://github.com/geosolutions-it/MapStoreExtension/actions/workflows/cut_release_branch.yml) workflow on MapStoreExtension project, indicating: + - [ ] Use workflow from branch `master` + - [ ] MapStore branch name to use: `YYYY.XX.xx` + - [ ] main branch `master` (default) ## Before the Release @@ -48,14 +54,18 @@ If stable release (YYYY.XX.00) follow these sub-steps: - [ ] `npm run start:app`, then check that an empty homepage loads correctly - [ ] Test [Binary](http://build.geosolutionsgroup.com/view/MapStore/job/MapStore/view/MapStore%20QA/job/MapStore2-QA-Build/) (take the mapstore2--qa-bin.zip, from latest build) -## Release +## Prepare Release -- [ ] On **stable** branch, do and merge a PR for updating: - - [ ] Update `CHANGELOG.md` [Instructions](https://mapstore.readthedocs.io/en/latest/developer-guide/release/#changelog-generation) - - [ ] Update the version of java modules on the stable branch to a stable, incremental version. Run `mvn versions:set -DnewVersion= -DprocessAllModules -DgenerateBackupPoms=false` to update package version, where `` is the version of the java packages (e.g. `1.3.1`). (`mvn:release:prepare` may also work. TODO: check this command) - - [ ] Manually update project pom templates to use `mapstore-services` of ``. `project/standard/templates/web/pom.xml` -- [ ] on **master branch** do and merge a PR for updating: - - [ ] Update `CHANGELOG.md` [Instructions](https://mapstore.readthedocs.io/en/latest/developer-guide/release/#changelog-generation) +- [ ] Run [`Prepare Release`](https://github.com/geosolutions-it/MapStore2/actions/workflows/pre_release.yml) workflow on github actions with the following parameters: + - Use workflow from `branch` **YYYY.XX.xx** (the release branch) + - Version to release **YYYY.XX.mm** (the effective number of the release) + - MapStore version for changelog generation **YYYY.XX.mm** (the effective number of the previous release) + - version to fix for the java module, accordingly with release schedule (e.g. `1.7.0`) + - use the default value for the other parameters +- [ ] Wait for the process to complete. At the end: + - a new commit will be added to the release branch tagged as `vYYYY.XX.mm`. This commit will contain the changelog and the updated version of the java modules. + - a pull request will be created on master with the changelog updates +- [ ] Merge the incoming PR created by the workflow for updating changelog on Master ## MapStore Stable deploy @@ -70,28 +80,27 @@ If stable release (YYYY.XX.00) follow these sub-steps: - [ ] After "MapStore2-Stable-Build" finished, Launch [MapStore2-Stable-Deploy](http://build.geosolutionsgroup.com/view/MapStore/job/MapStore/view/MapStore%20Stable/job/MapStore2-Stable-Deploy/) to install the latest stable version on official demo - [ ] test the change has been applied, login on https://mapstore.geosolutionsgroup.com and verify that the layers from `gs-stable` are visible without errors (typically authentication errors that was caused by the wrong auth-key). - ## Build and publishing release -- [ ] Create a [github draft release](https://github.com/geosolutions-it/MapStore2/releases) - - [ ] `branch` **YYYY.XX.xx** - - [ ] `tag` **vYYYY.XX.mm** (create a new tag from UI after entering this value) - - [ ] `release` name equal to tag **vYYYY.XX.mm** - - [ ] `description` describe the major changes and add links of the Changelog paragraph. -- [ ] Launch [MapStore2-Stable-Releaser](http://build.geosolutionsgroup.com/view/MapStore/job/MapStore/view/MapStore%20Stable/job/MapStore2-Stable-Releaser/) Jenkins job with **YYYY.XX.mm** for the version and **YYYY.XX.xx** for the branch to build and **wait the end**. **Note:** Using the MapStore2 Releaser allows to write the correct version number into the binary packages. In the overview of this job you can find and download : - - [ ] the latest `mapstore.war` - - [ ] the latest binary `mapstore2-YYYY.XX.mm-bin.zip` - - [ ] the printing bundle `mapstore-printing.zip` - - [ ] Upload to draft release - - [ ] the updated binary `mapstore2-YYYY.XX.mm-bin.zip` - - [ ] the `mapstore.war` package - - [ ] `mapstore-printing.zip` on github release +- [ ] Run [`Create Release`](https://github.com/geosolutions-it/MapStore2/actions/workflows/create_release.yml) workflow on github actions with the following parameters: + - Use workflow from `branch` **YYYY.XX.xx** (the release branch) + - Version to release **YYYY.XX.mm** (the effective number of the release) +- [ ] Launch [MapStore2-Stable-Releaser](http://build.geosolutionsgroup.com/view/MapStore/job/MapStore/view/MapStore%20Stable/job/MapStore2-Stable-Releaser/) Jenkins job with + - **YYYY.XX.mm** for the version + - **YYYY.XX.xx** for the branch to build +- [ ] Wait the end of the 2 process + +When the processes are finished, the release is ready to be published on github in draft mode. + +- [ ] Open the new release in draft from [here](https://github.com/geosolutions-it/MapStore2/releases) +- [ ] Update the link to Docker in the release notes with the link to the latest stable release (search the new tag on [docker hub](https://hub.docker.com/r/geosolutionsit/mapstore2/tags) ) +- [ ] Update the description of the release details - [ ] Publish the release - [ ] create on [ReadTheDocs](https://readthedocs.org/projects/mapstore/) project the version build for `vYYYY.XX.mm` (click on "Versions" and activate the version of the tag, created when release was published) - [ ] Update `Default version` to point the release version in the `Advanced Settings` menu of the [ReadTheDocs](https://readthedocs.org/dashboard/mapstore/advanced/) admin panel - ## Build and publish MapStoreExtension release + - [ ] [Create a draft release](https://github.com/geosolutions-it/MapStoreExtension/releases/new) for [MapstoreExtension](https://github.com/geosolutions-it/MapStoreExtension) with the same name and tag - [ ] target of the release is **stable branch** aligned to latest commit in stable branch of main mapstore repo - [ ] tag is **vYYYY.XX.mm** @@ -103,10 +112,10 @@ If stable release (YYYY.XX.00) follow these sub-steps: - [ ] Link the MapStore extension release in the MapStore release ## Finalize Release -- [ ] Prepare a PR MapStore **stable branch** **YYYY.XX.xx** in order to : - - [ ] reset versions of java modules to `-SNAPSHOT` (`mvn versions:set -DnewVersion= -DprocessAllModules -DgenerateBackupPoms=false`) where `` is the version to set. (e.g. `1.2-SNAPSHOT`). - - [ ] Manually update project pom templates to use `mapstore-services` of ``. `project/standard/templates/web/pom.xml` - - [ ] on `package.json` increasing the minor "version" number. **0.x.<number-of-minor-version>** + +- [ ] Run the [`Post Release`](https://github.com/geosolutions-it/MapStore2/actions/workflows/post_release.yml) workflow on github with the following parameters: + - Use workflow from branch `YYYY.XX.xx` (the release branch) + - Version of Java Packages to restore accordingly with release calendar with `-SNAPSHOT` E.g. `1.7-SNAPSHOT` - [ ] Write to the mailing list about the current release news and the next release major changes - [ ] Optional - prepare a PR for updating release procedure, if needed - [ ] Close this issue diff --git a/.github/workflows/create_release.yml b/.github/workflows/create_release.yml new file mode 100644 index 0000000000..7edee440a5 --- /dev/null +++ b/.github/workflows/create_release.yml @@ -0,0 +1,152 @@ +# This workflow will build a Java project with Maven, and cache/restore any dependencies to improve the workflow execution time +# For more information see: https://help.github.com/actions/language-and-framework-guides/building-and-testing-java-with-maven + +name: Create Release + +on: + workflow_dispatch: + inputs: + version: + description: 'Version to release. (format: `YYYY.MM.mm`)' + required: true +jobs: + ################ + # Build + ################ + build: + # The type of runner that the job will run on + runs-on: ubuntu-latest + steps: + ################ + # Protect master branch + ################ + - name: Check branch + if: ${{ github.repository != 'geosolutions-it/MapStore2' || github.ref == 'master' }} + uses: actions/github-script@v3 + with: + script: | + core.setFailed('This workflow can not run on master branch') + - uses: actions/checkout@v3 + - name: "checking out" + uses: actions/checkout@v3 + with: + ref: ${{ github.ref }} + - name: "setting up npm" + uses: actions/setup-node@v3 + with: + node-version: '16.x' + - name: "setting up Java" + uses: actions/setup-java@v1 + with: + java-version: '11.x' + ############ + # CACHING + ########## + - name: "cache node modules" + uses: actions/cache@v1 + env: + cache-name: cache-node-modules + with: + path: ~/.npm # npm cache files are stored in `~/.npm` on Linux/macOS + key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package.json') }} + restore-keys: | + ${{ runner.os }}-build-${{ env.cache-name }}- + ${{ runner.os }}-build- + ${{ runner.os }}- + - name: "cache maven dependencies" + uses: actions/cache@v1 + with: + path: ~/.m2/repository + key: mapstore-${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} + restore-keys: | + mapstore-${{ runner.os }}-maven- + - name: Build + id: "build" + run: "./build.sh ${{ github.event.inputs.version }} binary,printingbundle" + - name: "Upload war" + uses: actions/upload-artifact@v3 + with: + name: war + path: product/target/mapstore.war + - name: "Upload binary" + uses: actions/upload-artifact@v3 + with: + name: binary + path: "binary/target/mapstore2-${{ github.event.inputs.version }}-bin.zip" + - name: "Upload printing" + uses: actions/upload-artifact@v3 + with: + name: printing + path: "java/printing/target/mapstore-printing.zip" + release: + runs-on: ubuntu-latest + needs: build + steps: + - name: "Download war" + uses: actions/download-artifact@v3 + with: + path: artifacts/ + - name: Display structure of downloaded files + run: ls -R + working-directory: artifacts + - name: Get current date + id: date + run: echo "::set-output name=date::$(date +'%Y-%m-%d')" + - name: create_release + id: create_release + uses: actions/create-release@v1 + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # This token is provided by Actions, you do not need to create your own token + with: + commitish: ${{ github.ref }} + tag_name: "v${{ github.event.inputs.version }}" + release_name: "v${{ github.event.inputs.version }}" + body: | + ## Main Features + + - ... + + ## Main Improvements + + - ... + + ## Useful links related to **[v${{ github.event.inputs.version }}](https://github.com/geosolutions-it/MapStore2/tree/v${{ github.event.inputs.version }})** + - **[Full Changelog](https://github.com/geosolutions-it/MapStore2/compare/v${{ github.event.inputs.previousVersion }}...v${{ github.event.inputs.version }})** + - **[Implemented enhancements](https://github.com/geosolutions-it/MapStore2/issues?q=is%3Aissue+milestone%3A%22${{ github.event.inputs.version }}%22+is%3Aclosed+label%3Aenhancement)** + - **[Fixed bugs](https://github.com/geosolutions-it/MapStore2/issues?q=is%3Aissue+milestone%3A%22${{ github.event.inputs.version }}%22+is%3Aclosed+label%3Abug)** + - **[Closed issues](https://github.com/geosolutions-it/MapStore2/issues?q=is%3Aissue+milestone%3A%22${{ github.event.inputs.version }}%22+is%3Aclosed)** + - **[MapStore Extension release v${{ github.event.inputs.version }}](https://github.com/geosolutions-it/MapStoreExtension/releases/tag/v${{ github.event.inputs.version }})** + - **[Docker image v${{ github.event.inputs.version }}](xxx)** `< TODO: add this link manually` + - **[MapStore documentation v${{ github.event.inputs.version }}](https://docs.mapstore.geosolutionsgroup.com/en/v${{ github.event.inputs.version }}/)** + draft: true + prerelease: false + - name: Upload Release war + id: upload-release-war + uses: actions/upload-release-asset@v1 + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + with: + upload_url: ${{ steps.create_release.outputs.upload_url }} # This pulls from the CREATE RELEASE step above, referencing it's ID to get its outputs object, which include a `upload_url`. See this blog post for more info: https://jasonet.co/posts/new-features-of-github-actions/#passing-data-to-future-steps + asset_path: artifacts/war/mapstore.war + asset_name: mapstore.war + asset_content_type: application/zip + - name: Upload Release binary + id: upload-release-binary + uses: actions/upload-release-asset@v1 + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + with: + upload_url: ${{ steps.create_release.outputs.upload_url }} # This pulls from the CREATE RELEASE step above, referencing it's ID to get its outputs object, which include a `upload_url`. See this blog post for more info: https://jasonet.co/posts/new-features-of-github-actions/#passing-data-to-future-steps + asset_path: "artifacts/binary/mapstore2-${{github.event.inputs.version}}-bin.zip" + asset_name: "mapstore2-${{github.event.inputs.version}}-bin.zip" + asset_content_type: application/zip + - name: Upload Release printing + id: upload-release-printing + uses: actions/upload-release-asset@v1 + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + with: + upload_url: ${{ steps.create_release.outputs.upload_url }} # This pulls from the CREATE RELEASE step above, referencing it's ID to get its outputs object, which include a `upload_url`. See this blog post for more info: https://jasonet.co/posts/new-features-of-github-actions/#passing-data-to-future-steps + asset_path: artifacts/printing/mapstore-printing.zip + asset_name: mapstore-printing.zip + asset_content_type: application/zip diff --git a/.github/workflows/cut_major_branch.yml b/.github/workflows/cut_major_branch.yml new file mode 100644 index 0000000000..064bab8ba8 --- /dev/null +++ b/.github/workflows/cut_major_branch.yml @@ -0,0 +1,97 @@ +name: Cut Release Branch +on: + workflow_dispatch: + inputs: + release-branch: + description: MapStore branch name to use (YYYY.MM.xx). E.g. 2024.01.xx + required: true + mapfish-version: + description: Mapfish print version to use (e.g. 2.3-SNAPSHOT) + required: true + default: '2.3-SNAPSHOT' + geostore-version: + description: GeoStore version to use (e.g. 2.0.0). + required: true + default: '2.0.0' + http-proxy-version: + description: Http proxy version to use (e.g. 2.4). + required: true + default: '1.4.0' + main-branch: + description: main branch + default: master + pr-options: + description: Options for Pull request + default: --squash --auto --delete-branch +jobs: + cut-release: + name: Create release branch and PRs into main main branch + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v3 + with: + ref: ${{ github.event.inputs.main-branch }} + token: ${{ secrets.GITHUB_TOKEN }} + - name: Create release branch and generate PR body + id: create-branch + env: + MAPFISH_GROUP: org.mapfish.print + MAPFISH_VERSION: ${{ github.event.inputs.mapfish-version }} + GEOSTORE_GROUP: it.geosolutions.geostore + GEOSTORE_VERSION: ${{ github.event.inputs.geostore-version }} + HTTP_PROXY_GROUP: proxy + HTTP_PROXY_VERSION: ${{ github.event.inputs.http-proxy-version }} + RELEASE_BRANCH: ${{ github.event.inputs.release-branch }} + + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + MAIN_BRANCH: ${{ github.event.inputs.main-branch }} + PR_OPTIONS: ${{ github.event.inputs.pr-options }} + RUN_ID: ${{ github.run_id }} + + run: | + # script will go here + echo "Initializing git" + # Optional + git config user.name github-actions + git config user.email github-actions@github.com + BRANCH_NAME="${RELEASE_BRANCH}" + echo "creating branch is $BRANCH_NAME" + git checkout -b "$BRANCH_NAME" + # Find all pom.xml files and update-dependencies on them + echo "Updating versions on release branch" + # Find all pom.xml files and update-dependencies on them + echo "Updating versions on release branch" + POM_FILES=$(git ls-files . | grep 'pom\.xml$' | grep -v 'project/standard/templates/backend/pom\.xml$'| grep -v 'project/standard/templates/pom\.xml$');# note: exclues one file not involved that is not valid pom.xml, because it is a template + for POM_FILE in $POM_FILES; do + mvn versions:use-dep-version -f $POM_FILE -Dincludes=$MAPFISH_GROUP -DdepVersion=$MAPFISH_VERSION -DforceVersion=true -DgenerateBackupPoms=false -DautoVersionSubmodules=true -Pprinting + mvn versions:use-dep-version -f $POM_FILE -Dincludes=$GEOSTORE_GROUP -DdepVersion=$GEOSTORE_VERSION -DforceVersion=true -DgenerateBackupPoms=false -DautoVersionSubmodules=true -Pprinting + mvn versions:use-dep-version -f $POM_FILE -Dincludes=$HTTP_PROXY_GROUP -DdepVersion=$HTTP_PROXY_VERSION -DforceVersion=true -DgenerateBackupPoms=false -DautoVersionSubmodules=true -Pprinting + done + echo $POM_FILES | xargs git add + git commit -m "Set versions of main dependencies to a stable version" + git push --set-upstream origin "$BRANCH_NAME" + echo "branch created" + echo "creating bump changes" + git checkout "$MAIN_BRANCH" + OLD_JAVA_VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) + echo "Incrementing java packages versions" + mvn -B release:update-versions -DautoVersionSubmodules=true -Pprinting,binary,printingbundle + NEXT_JAVA_VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) + echo "Java packages versions updated from $OLD_JAVA_VERSION to $NEXT_JAVA_VERSION" + ### increase dependency of project to new version + echo "Updating project dependency to new version" + mvn versions:use-dep-version -f project/standard/templates/web/pom.xml -Dincludes=it.geosolutions.mapstore -DdepVersion=$NEXT_JAVA_VERSION -DforceVersion=true -DgenerateBackupPoms=false + # Increase release minor version + echo "Updating project version to new version in package.json" + npm version minor --git-tag-version=false + pr_branch_name="bump-${release-branch}-${RUN_ID}" + echo "Creating a temp PR on branch: ${pr_branch_name}" + git checkout -b "${pr_branch_name}" + find . -name 'pom.xml' | xargs git add + git add package.json + git commit -m "Bump versions on master for release-branch" + git push origin "${pr_branch_name}" + pr_url=$(gh pr create -B "${MAIN_BRANCH}" -H "${pr_branch_name}" --title "[github-action] Bump version for next release" --body "This automatic pull request bumps version of ${MAIN_BRANCH} branch for java packages and for package.json after creating the $release-branc") + sleep 10 + #gh pr merge "$pr_url" ${PR_OPTIONS} diff --git a/.github/workflows/post_release.yml b/.github/workflows/post_release.yml new file mode 100644 index 0000000000..27c2ce2062 --- /dev/null +++ b/.github/workflows/post_release.yml @@ -0,0 +1,48 @@ +# This workflow will build a Java project with Maven, and cache/restore any dependencies to improve the workflow execution time +# For more information see: https://help.github.com/actions/language-and-framework-guides/building-and-testing-java-with-maven + +name: Post Release + +on: + workflow_dispatch: + inputs: + snapshot: + description: 'Version of Java packages to restore. e.g. 2.1-SNAPSHOT' + required: true +jobs: + set-versions: + concurrency: post-release-${{ github.ref }} + runs-on: ubuntu-latest + steps: + - name: Check branch + if: ${{ github.repository != 'geosolutions-it/geostore' || github.ref == 'master' }} + uses: actions/github-script@v3 + with: + script: | + core.setFailed('This workflow can not run on master branch') + - uses: actions/checkout@v3 + - name: Set up JDK 11 + uses: actions/setup-java@v3 + with: + java-version: '11.x' + distribution: 'adopt' + cache: maven + - name: Restore Snapshots + env: + SNAPSHOT_VERSION: ${{ github.event.inputs.snapshot }} + run: | + # restore versions snapshot + echo "Initializing git" + git config user.name github-actions + git config user.email github-actions@github.com + echo + echo "Restoring snapshot version tp $SNAPSHOT_VERSION" + mvn versions:set -DnewVersion=$SNAPSHOT_VERSION -DprocessAllModules -DgenerateBackupPoms=false -Pprinting,binary,printingbundle + mvn versions:use-dep-version -f project/standard/templates/web/pom.xml -Dincludes=it.geosolutions.mapstore -DdepVersion=$SNAPSHOT_VERSION -DforceVersion=true -DgenerateBackupPoms=false + npm version patch --git-tag-version=false + echo "Committing changes" + find . -name 'pom.xml' | xargs git add + git add package.json + git commit -m "Restore java packages to $SNAPSHOT_VERSION and update package.json" + git push origin ${{ github.ref_name }} + echo "Snapshots version restored" diff --git a/.github/workflows/pre_release.yml b/.github/workflows/pre_release.yml new file mode 100644 index 0000000000..32e54b7a59 --- /dev/null +++ b/.github/workflows/pre_release.yml @@ -0,0 +1,149 @@ +# This workflow will build a Java project with Maven, and cache/restore any dependencies to improve the workflow execution time +# For more information see: https://help.github.com/actions/language-and-framework-guides/building-and-testing-java-with-maven + +name: Prepare Release + +on: + workflow_dispatch: + inputs: + version: + description: 'Version to release. (format: `YYYY.MM.mm`)' + required: true + previous-ms-version: + description: 'previous MapStore version for changelog generation. (format: `YYYY.MM.mm`)' + required: true + java-modules-version: + description: 'version to fix for the java module, accordingly with release schedule (e.g. `1.7.0`)' + required: true + main-branch: + description: 'main branch name' + required: true + default: 'master' +jobs: + ################ + # Fix versions + ############### + fix-version: + concurrency: release-${{ github.ref }} + runs-on: ubuntu-latest + steps: + ################ + # Protect master branch + ################ + - name: Check branch + if: ${{ github.repository != 'geosolutions-it/MapStore2' || github.ref == 'master' }} + uses: actions/github-script@v3 + with: + script: | + core.setFailed('This workflow can not run on master branch') + - uses: actions/checkout@v3 + - name: Set up JDK 11 + uses: actions/setup-java@v3 + with: + java-version: '11.x' + distribution: 'adopt' + cache: maven + - name: 'Fix versions, commit and push new tag' + env: + LAST_MS_VERSION: ${{ github.event.inputs.previous-ms-version }} + NEW_MS_VERSION: ${{ github.event.inputs.version }} + JAVA_MODULES_NEW_VERSION: ${{ github.event.inputs.java-modules-version }} + run: | + + ## Generate changelog + function update_changelog () { + echo "Update changelog" + markdown_text=$(npm -s run generate:changelog $1 $2) + # Text file path + file_path="CHANGELOG.md" + # read old content, excluding the title + existing_content=$(cat "$file_path" | tail -n +2) + + # add again the title and the new content concatenated to the old one + new_content="# Change Log\n${markdown_text}\n${existing_content}" + + # overwrite the file with the new content + echo -e "$new_content" > "$file_path" + + echo "Changelog updated" + } + # read snapshot version + CURRENT_JAVA_VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) + echo "current java version: $CURRENT_JAVA_VERSION" + + # Update snapshot version + echo "updating modules to version: $JAVA_MODULES_NEW_VERSION" + mvn versions:set -DnewVersion=$JAVA_MODULES_NEW_VERSION -DprocessAllModules -DgenerateBackupPoms=false -Pprinting,binary,printingbundle # note: this do not set binary profile version it is used to generate artifact name ( to be fixed in the future ) + + # Update project dependency + echo "updating project pom.xml dependency from mapstore packages to version: $JAVA_MODULES_NEW_VERSION" + mvn versions:use-dep-version -f project/standard/templates/web/pom.xml -Dincludes=it.geosolutions.mapstore -DdepVersion=$JAVA_MODULES_NEW_VERSION -DforceVersion=true -DgenerateBackupPoms=false + # Update changelog + echo "Updating changelog" + update_changelog $LAST_MS_VERSION $NEW_MS_VERSION + echo "Initializing git" + # Initializing git + git config user.name github-actions + git config user.email github-actions@github.com + + # Commit changes + echo "Committing changes" + find . -name 'pom.xml' | xargs git add + git add CHANGELOG.md + git commit -m "Version Release ${NEW_MS_VERSION}" + git tag v${NEW_MS_VERSION} # create tag + git push origin ${{ github.ref_name }} --tags # push tag + update-main-changelog: + runs-on: ubuntu-latest + needs: fix-version + steps: + - name: "checking out" + uses: actions/checkout@v3 + with: + ref: ${{ inputs.main-branch }} + - name: "setting up npm" + uses: actions/setup-node@v3 + with: + node-version: '16.x' + - name: 'Updte changelog on master' + env: + LAST_MS_VERSION: ${{ github.event.inputs.previous-ms-version }} + NEW_MS_VERSION: ${{ github.event.inputs.version }} + JAVA_MODULES_NEW_VERSION: ${{ github.event.inputs.java-modules-version }} + GH_TOKEN: ${{ github.token }} + run: | + ## Generate changelog + function update_changelog () { + echo "Update changelog" + markdown_text=$(npm -s run generate:changelog $1 $2) + # Text file path + file_path="CHANGELOG.md" + # read old content, excluding the title + existing_content=$(cat "$file_path" | tail -n +2) + + # add again the title and the new content concatenated to the old one + new_content="# Change Log\n${markdown_text}\n${existing_content}" + + # overwrite the file with the new content + echo -e "$new_content" > "$file_path" + + echo "Changelog updated" + } + echo "Initializing git" + # Initializing git + git config user.name github-actions + git config user.email github-actions@github.com + + # Update master branch with CHANGELOG.md + pr_branch_name="update-changelog-${{ github.run_id}}" + git checkout -b "${pr_branch_name}" + # Update changelog + update_changelog $LAST_MS_VERSION $NEW_MS_VERSION + git add CHANGELOG.md + git commit -m "Update Changelog for version ${{ inputs.version }}" + git push origin "${pr_branch_name}" + echo "Sending pull request" + pr_url=$(gh pr create -B "${MAIN_BRANCH}" -H "${pr_branch_name}" --title "[github-action] Update changelog for release ${NEW_MS_VERSION}" --body "This automatic pull request update CHANGELOG.md on ${MAIN_BRANCH} branch") + + + diff --git a/binary/bin-war/pom.xml b/binary/bin-war/pom.xml index 8a0bd7fa74..662f4659d5 100644 --- a/binary/bin-war/pom.xml +++ b/binary/bin-war/pom.xml @@ -3,14 +3,14 @@ it.geosolutions.mapstore mapstore-bin-war war - ${mapstore2.version} + 1.7-SNAPSHOT MapStore 2 Release Module WAR Creates the war for the binary package, adding customization (e.g. h2 database) http://www.geo-solutions.it UTF-8 8.5.69 - ${project.version} + ${mapstore2.version} diff --git a/binary/pom.xml b/binary/pom.xml index 70104ae3eb..794c3e2846 100644 --- a/binary/pom.xml +++ b/binary/pom.xml @@ -3,7 +3,7 @@ it.geosolutions.mapstore mapstore-binary pom - ${mapstore2.version} + 1.7-SNAPSHOT MapStore 2 Binary Module Project to create the binary package http://www.geo-solutions.it @@ -11,7 +11,7 @@ UTF-8 8.5.96 - ${project.version} + ${mapstore2.version} diff --git a/docs/developer-guide/release.md b/docs/developer-guide/release.md index ca9a91df3d..cdaad2eabd 100644 --- a/docs/developer-guide/release.md +++ b/docs/developer-guide/release.md @@ -7,31 +7,147 @@ To create a new MapStore release, you need to: Here below some details about changelog generation and naming conventions. -## Changelog generation +## Naming conventions -Generate new changelog by running this: +### release and tag + +- **vYYYY.XX.mm** name of the release and tag. (e.g. `v2022.01.01`) +- **YYYY** is the year, +- **XX** is the incremental number of the release for the current year (starting from 01) +- **mm** is an incremental value (starting from 00) to increment for minor releases + +### stable branch + +- **YYYY.XX.xx** name of stable branch (e.g. `2022.01.xx` ) +- **YYYY** is the year +- **XX** is the incremental number of the release for the current year (starting from 01) +- **xx** is the fixed text `xx` + +## Release procedure details + +### Github actions + +The github actions are configured to automate the release process. + +In particular the process is composed by 4 steps: + +- **Cut Release Branch** creates a new branch from master with the name of the stable branch +- **Prepare Release** updates the version of the packages and creates the changelog +- **Create Release** creates the github draft release +- **Post Release** restores the snapshots + +In this process the user has to manually finalize the release by adding the release description and the binaries to the release. +Everything is listed in the checklist of the issue created at the beginning. + +Here a flow chart that describes the process: + +```mermaid +flowchart TD + Issue[Create MapStore Release Issue] --> X[...] + X --> Cut + Cut[[GH Action: Cut Release Branch]] -->| + create branch + fix dependencies versions + PR to master with packages versions update + | Fixes + Fixes[Fixes...] --> Tests[Tests...] + Tests --> start(((start release))) + start --> PrepareRelease + PrepareRelease[[GH Action: Prepare Release]] --> | + Fix versions + Changelog + Tag + PR to master with changelog updates + | CreateRelease + CreateRelease[[GH Action: Create Release]] --> | + Create github Release + generate binaries to attach to the release + |release + release((release + created)) -->PostRelease[[Post Release]] + PostRelease[[GH Action: Post Release]] --> | + restores snapshots + | finish(((end))) + release --> FinalizeRelease[Finalize release manually] + FinalizeRelease -->| + refine release description with features + docker + jenkins + communication... + | Publish + finish -- next minor release --> Fixes +``` + +### Git graph structure + +The procedure produces the following git graph structure: + +```mermaid +gitGraph TB: + commit + branch 2023.01.xx + checkout 2023.01.xx + commit id: "2023.01 initial commit" + checkout main + commit id: "feature-1" + commit id: "fix-1" + checkout 2023.01.xx + cherry-pick id:"fix-1" + commit id: "fix-versions-1+changelog" + commit id: "2023.01.00-release" tag: "v2023.02.00" + commit id: "restore-snaphots-1" + checkout main + commit id: "Changelog for version v2023.01.00" + commit id: "fix-2" + commit id: "feature-2" + commit id: "feature-3" + checkout 2023.01.xx + cherry-pick id:"fix-2" + commit id: "fix-versions-2+changelog" + commit id: "2023.01.01-release" tag: "v2023.02.01" + commit id: "restore-snaphots-2" + checkout main + commit id: "Changelog for version v2023.02.01" + commit + commit + branch 2023.02.xx + checkout 2023.02.xx + commit id: "2023.02 initial commit" + checkout main + commit id: "fix-3" + checkout 2023.02.xx + cherry-pick id:"fix-3" +``` + +### Changelog generation + +The changelog generation is now automated. Anyway it is possible to generate the changelog manually by running the following command: ```sh npm run generate:changelog # usage -# generate:changelog 2022.01.00 2022.02.00 +# e.g. npm run generate:changelog 2022.01.00 2022.02.00 ``` -## Release Checklist +This produces some text that can be used to update the changelog file. -### naming conventions +### Handling errors -#### release and tag +If in a certain point we notice some errors, the separation of the tasks allows to restart the process from the point where the error occurred. -- **vYYYY.XX.mm** name of the release and tag. (e.g. `v2022.01.01`) -- **YYYY** is the year, -- **XX** is the incremental number of the release for the current year (starting from 01) -- **mm** is an incremental value (starting from 00) to increment for minor releases +For instance if after tag creation we notice some problem or we have some failures, we can fix the problem, delete and recreate the tag manually and recreate the release by running the `Create Release` action. -#### stable branch +### Testing and fixing release procedure -- **YYYY.XX.xx** name of stable branch (e.g. `2022.01.xx` ) -- **YYYY** is the year -- **XX** is the incremental number of the release for the current year (starting from 01) -- **xx** is the fixed text `xx` +For testing and fixing the release procedure it is possible to run the release procedure on your fork. +anyway "Create, Prepare and Post Release" actions are configured to run only on the main repository. This is done to avoid that the release procedure is triggered automatically by forks in some cases. +So on your fork you will need to manually edit the workflow files to enable the actions to run on your fork. + +In particular you will need to edit some lines like this: + +```yaml +if: ${{ github.event_name == 'push' && github.repository == 'geosolutions-it/MapStore2' }} +``` + +Removing the condition or changing the repository name. diff --git a/mkdocs.yml b/mkdocs.yml index 4c0d5ea46f..4ebd23d438 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -42,7 +42,7 @@ extra: link: 'https://it.linkedin.com/company/geosolutions-s-a-s-' extra_javascript: - 'https://code.jquery.com/jquery-3.3.1.min.js' - - 'https://unpkg.com/mermaid@9.1.3/dist/mermaid.min.js' + - 'https://unpkg.com/mermaid@10.6.0/dist/mermaid.min.js' extra_css: - 'theme/css/extra.css' repo_name: 'geosolutions-it/MapStore2'