Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(build): Derive application versions from git #11863

Merged
merged 22 commits into from
Dec 13, 2022
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/api-test-lint-deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,8 @@ jobs:
if: github.event_name == 'push'
steps:
- uses: 'actions/checkout@v3'
with:
fetch-depth: 0
- uses: 'actions/setup-node@v3'
with:
node-version: '16'
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/app-test-build-deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,8 @@ jobs:
runs-on: ${{ matrix.os }}
steps:
- uses: 'actions/checkout@v3'
with:
fetch-depth: 0
- uses: 'actions/setup-node@v3'
with:
node-version: '16'
Expand Down Expand Up @@ -154,6 +156,7 @@ jobs:
APPLE_ID: ${{ secrets.OT_APP_APPLE_ID }}
APPLE_ID_PASSWORD: ${{ secrets.OT_APP_APPLE_ID_PASSWORD }}
HOST_PYTHON: python
OPENTRONS_PROJECT: robot-stack
run: |
make -C app-shell dist-${{ matrix.os }}

Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/docs-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@ jobs:
runs-on: 'ubuntu-22.04'
steps:
- uses: 'actions/checkout@v3'
with:
fetch-depth: 0
- uses: 'actions/setup-node@v3'
with:
node-version: '16'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/hardware-lint-test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:
steps:
- name: Checkout opentrons repo
uses: 'actions/checkout@v3'

fetch-depth: 0
- name: Setup Node
uses: 'actions/setup-node@v3'
with:
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/hardware-testing-protocols.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ jobs:
steps:
- name: Checkout opentrons repo
uses: 'actions/checkout@v3'
fetch-depth: 0

- name: Setup Node
uses: 'actions/setup-node@v3'
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/ll-test-build-deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,8 @@ jobs:
if: github.event_name != 'pull_request'
steps:
- uses: 'actions/checkout@v3'
with:
fetch-depth: 0
- uses: 'actions/setup-node@v3'
with:
node-version: '16'
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/pd-test-build-deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,8 @@ jobs:
if: github.event_name != 'pull_request'
steps:
- uses: 'actions/checkout@v3'
with:
fetch-depth: 0
- uses: 'actions/setup-node@v3'
with:
node-version: '16'
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/shared-data-test-lint-deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,8 @@ jobs:
if: github.event_name == 'push'
steps:
- uses: 'actions/checkout@v3'
with:
fetch-depth: 0
- uses: 'actions/setup-node@v3'
with:
node-version: '16'
Expand Down
106 changes: 45 additions & 61 deletions RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@
Below you will find instructions for release processes for projects within our monorepo. The main goal of our process is to
neatly document any changes that may happen during QA, such as bug fixes, and separate production concerns from our development branch.

## Releasing Robot Software Stack
## Releasing Robot Software Stacks

Our release process is still a work-in-progress. The app and API projects are currently versioned together to ensure interoperability.

1. Ensure you have a buildroot release created in GitHub with all the changes you want in this release, if any. If there are no buildroot changes, you don't have to create a new release; the last tag in buildroot is used for release builds.
1. Ensure you have a release created in GitHub for the robot stack you're releasing - buildroot for ot-2, oe-core for ot-3 - with all the changes you want in this release, if any. If there are no system changes, you don't have to create a new release; the last tag in the system repo is used for release builds.
2. Checkout `edge` and make a release branch, without any new changes. The branch name should match `release_*` to make `make bump` usage easier and make it clear this is a release.

```shell
Expand All @@ -18,38 +18,27 @@ git push --set-upstream origin release_${version}
```

3. Open a PR into `release` for your empty release branch.
4. Create a new branch for your initial version bump:
4. Seek input on the user-facing release notes at `app-shell/build/release-notes.md` to add the new notes for the app and user-facing release notes at `api/release-notes.md` to add the new notes for the robot software. These changes should get wordsmithing approval and use a `chore` merge commit. This can happen in parallel, but should be done before the alpha process concludes.

```shell
git checkout -b chore_bump-${version}
```
5. Create a tag using `make bump` for the new version. See the notes on [`make bump` usage](#make-bump-usage) below. Since this is entering qa, you'll want to bump the version to the target release version with an `-alpha.0` prerelease tag.

5. In the bump branch, bump the version to the appropriate alpha (read [the section below](#make-bump-usage) carefully)
6. Inspect version bumps
7. Edit the user-facing changelog at `app-shell/build/release-notes.md` to add the new notes for the app
8. Edit the user-facing changelog at `api/release-notes.md` to add the new notes for the robot software
9. `git add --all`
10. `git cz`
6. Review the tag with `git show ${project}@${version}` (or `git show v${version}` for the robot-stack project). If it points to the commit you meant, push the tag - this starts the build process. This is a release candidate that will undergo QA.

- Type: `chore`
- Scope: `release`
- Message: `${version}`
```shell
git push origin ${project}@${version}
```

11. Gather reviews on release notes until everybody is satisfied. Check the docs at sandbox.docs.opentrons.com/release\_\${version}
12. Once your chore bump branch is ready, squash merge the `chore_bump-${version}` into the `release_${version}` branch.
13. Tag the release branch as the version you just bumped to; this is a release candidate that will undergo QA:
or

```shell
git tag -a v${version} -m 'chore(release): ${version}'
git push origin v${version}
```

Changelogs for the release are automatically generated when the tag is pushed and sent to the release page in github.

14. Run QA on this release. If issues are found, create PRs targeted on the release branch. To create new alpha releases, repeat steps 4-14.
15. Once QA is a pass, bump to the target release version (review [the section below](#make-bump-usage) again)
7. Run QA on this release. If issues are found, create PRs targeted on the release branch. To create new alpha releases, repeat steps 4-7.

16. Do a NORMAL MERGE into `release`. Do NOT squash or rebase. This should be done from your local command line (and will succeed as long as the release PR is reviewed and status checks have passed):
8. Once QA is a pass, do a NORMAL MERGE into `release`. Do NOT squash or rebase. This should be done from your local command line (and will succeed as long as the release PR is reviewed and status checks have passed):

```shell
# note: make sure you have pulled the latest changes for branch
Expand All @@ -59,7 +48,7 @@ git merge --ff-only release_${version}
git push origin release
```

17. Tag the release:
10. Run [`make bump](#make-bump-usage) to tag the release and start release builds:

```shell
git tag -a v${version} -m 'chore(release): ${version}'
Expand All @@ -68,20 +57,20 @@ git push origin v${version}

The tag push will kick off release builds and create a release page where those builds and automatically generated in-depth changelogs will be posted.

18. Open a PR of `release` into `edge`. Give the PR a name like `chore(release): Merge changes from ${version} into edge`. Once it passes, on the command line merge it into `edge`:
11. Open a PR of `release` into `edge`. Give the PR a name like `chore(release): Merge changes from ${version} into edge`. Once it passes, on the command line merge it into `edge`:

```shell
git checkout edge
git pull
git merge --no-ff release
```

19. Use the PR title for the merge commit title. You can then `git push origin edge`, which will succeed as ong as the PR is approved and status checks pass.
20. Release the docs for this version (see below under Releasing Web Projects)
12. Use the PR title for the merge commit title. You can then `git push origin edge`, which will succeed as ong as the PR is approved and status checks pass.
13. Release the docs for this version (see below under Releasing Web Projects)

## Releasing Robot Software Stack Hotfixes

1. Ensure you have a buildroot release created in GitHub with all the changes you want to see, if any. If there aren't any, you don't have to create a new buildroot release; by default, the last tag is used for release builds.
1. Ensure you have a system release created in GitHub (buildroot for OT2, oe-core for OT3) with all the changes you want to see, if any. If there aren't any, you don't have to create a new release; by default, the last tag is used for release builds.
2. Checkout `release` and make a release branch, without any new changes. The branch name should match `hotfix_*` to make it clear this is a hotfix, and make `make bump` usage simpler.

```shell
Expand All @@ -92,35 +81,15 @@ git push --set-upstream origin hotfix_${version}
```

3. Target the hotfix PRs on this branch.
4. Once the fixes have been merged into the original hotfix branch, create a new branch for your initial version bump:
4. Wordsmith the release notes in `app-shell/build/release-notes.md` and `api/release-notes.md` in a PR that uses the `chore` commit type.
5. Once the fixes and release notes have been merged into the hotfix branch, bump to an alpha version to begin qa using [`make bump`](#make-bump-usage).
6. Inspect the created tag and then push it:

```shell
git checkout -b chore_bump-${version}
```

5. In the bump branch, bump the version to the appropriate alpha (read [the section below](#make-bump-usage) carefully)
6. Inspect version bumps
7. Edit the user-facing changelog at `app-shell/build/release-notes.md` to add the new notes for the app
8. Edit the user-facing changelog at `api/release-notes.md` to add the new notes for the robot software
9. `git add --all`
10. `git cz`

- Type: `chore`
- Scope: `release`
- Message: `${version}`

11. Push this commit
12. Once your chore bump branch is ready, squash merge the `chore_bump-${version}` into the `hotfix_${version}` branch.
13. Tag the release branch as the version you just bumped to; this is a release candidate that will undergo QA:

```bash
git tag -a v${version} -m 'chore(release): ${version}'
git push origin v${version}
git push ${tag}
```

14. Run QA on this release. If issues are found, create PRs targeted on the release branch. To create new alpha releases, repeat steps 4-14.
15. Once QA is a pass, bump to your target version (review [the section below](#make-bump-usage) again)
16. Do a NORMAL MERGE into `release`. Do NOT squash or rebase. This should be done from your local command line (and will succeed as long as the release PR is reviewed and status checks have passed):
7. Once QA is a pass, do a NORMAL MERGE into `release`. Do NOT squash or rebase. This should be done from your local command line (and will succeed as long as the release PR is reviewed and status checks have passed):

```shell
# note: make sure you have pulled the latest changes for branch
Expand All @@ -130,28 +99,43 @@ git merge --ff-only release_${version}
git push origin release
```

17. Tag the release:

```shell
git tag -a v${version} -m 'chore(release): ${version}'
git push origin v${version}
```
8. Tag the release using [`make bump`](#make-bump-usage)

Pushing the tag will create release builds and a github release page with the in-depth changelogs.

18. Open a PR of `release` into `edge`. Give the PR a name like `chore(release): Merge changes from ${version} into edge`. Once it passes, on the command line merge it into `edge`:
9. Open a PR of `release` into `edge`. Give the PR a name like `chore(release): Merge changes from ${version} into edge`. Once it passes, on the command line merge it into `edge`:

```shell
git checkout edge
git pull
git merge --no-ff release
```

19. Use the PR title for the merge commit title. You can then `git push origin edge`, which will succeed as long as the PR is approved and status checks pass.
20. Release the docs for this version (see below under Releasing Web Projects)
10. Use the PR title for the merge commit title. You can then `git push origin edge`, which will succeed as long as the PR is approved and status checks pass.
11. Release the docs for this version (see below under Releasing Web Projects)

### tag usage

If you know the version (at this stage, it should be a new minor or patch version with an alpha tag) you can specify it with `version` and `project`, where `project` is one of our multi-application projects: `docs`, `robot-stack`, `labware-library`, `protocol-designer`, `ot3`:

```shell
make bump project=${project} version=${version}
```

i.e.

If not, you can follow the prompts:

```shell
make bump
Enter the project (labware-library,docs,robot-stack,ot3,protocol-designer):
The current version of ${project} is ${current-version}
The next prerelease version of ${project} would be ${version}
The next release version of ${project} would be ${version}
Enter version exactly or bump type (prerelease,alpha,beta,candidate,release)
version:
```

`make bump` runs `lerna version` (with git tag and push disabled) to bump all required files. You can pass options to lerna with the `version` environment variable. See the [lerna version docs][lerna-version] for available options. The most important options are:

- First positional argument: bump type _or_ explicit version
Expand Down
4 changes: 2 additions & 2 deletions api-client/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"private": true,
"name": "@opentrons/api-client",
"description": "Opentrons robot API client for Node.js and the browser",
"version": "6.2.0",
"version": "0.0.0-dev",
"license": "Apache-2.0",
"main": "dist/api-client.js",
"module": "dist/api-client.mjs",
Expand All @@ -13,7 +13,7 @@
"./dist/api-client.mjs": "./dist/api-client.browser.mjs"
},
"dependencies": {
"@opentrons/shared-data": "6.2.0",
"@opentrons/shared-data": "link:../shared-data",
"axios": "^0.21.1"
}
}
1 change: 0 additions & 1 deletion api/MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
include src/opentrons/package.json
graft src/opentrons/config
graft src/opentrons/resources
exclude pyproject.toml
42 changes: 20 additions & 22 deletions api/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,17 @@ sphinx_build := $(pipenv) run sphinx-build -W --keep-going
# github.com/Opentrons/opentrons/issues/6135
sphinx_build_allow_warnings := $(pipenv) run sphinx-build

ot_project := $(OPENTRONS_PROJECT)
project_rs_default = $(if $(ot_project),$(ot_project),robot-stack)
project_ot3_default = $(if $(ot_project),$(ot_project),ot3)

# Find the version of the wheel from package.json using a helper script. We
# Find the version of the wheel from git using a helper script. We
# use python here so we can use the same version normalization that will be
# used to create the wheel.
wheel_file = dist/$(call python_get_wheelname,api,opentrons,$(BUILD_NUMBER))
wheel_file = dist/$(call python_get_wheelname,api,$(project_rs_default),opentrons,$(BUILD_NUMBER))

# Find the version of the sdist file from package.json using a helper script.
sdist_file = dist/$(call python_get_sdistname,api,opentrons)
# Find the version of the sdist file from git using a helper script.
sdist_file = dist/$(call python_get_sdistname,api,$(project_ot3_default),opentrons)

# These variables are for simulating python protocols
sim_log_level ?= info
Expand All @@ -51,18 +54,12 @@ ssh_opts ?= $(default_ssh_opts)
twine_auth_args := --username $(pypi_username) --password $(pypi_password)
twine_repository_url ?= $(pypi_test_upload_url)

# Source discovery
# For the python sources
ot_py_sources := $(filter %.py,$(shell $(SHX) find src/opentrons/))
# And the out of tree shared data
ot_shared_data_sources := $(filter %.json,$(shell $(SHX) find ../shared-data/))
# And the arbitrary stuff in resources
ot_resources := $(filter %,$(shell $(SHX) find src/opentrons/resources))
ot_sources := $(ot_py_sources) $(ot_shared_data_sources) $(ot_resources)

# Defined separately than the clean target so the wheel file doesn’t have to
# depend on a PHONY target
clean_cmd = $(SHX) rm -rf build dist .coverage coverage.xml '*.egg-info' '**/__pycache__' '**/*.pyc' 'src/**/.mypy_cache'
clean_cmd = $(SHX) rm -rf build .coverage coverage.xml '*.egg-info' '**/__pycache__' '**/*.pyc' 'src/**/.mypy_cache'
clean_wheel_cmd = $(clean_cmd) dist/*.whl
clean_sdist_cmd = $(clean_cmd) dist/*.tar.gz
clean_all_cmd = $(clean_cmd) dist

plot_type ?=

Expand All @@ -82,28 +79,29 @@ setup-ot2:

.PHONY: clean
clean: docs-clean
$(clean_cmd)
$(clean_all_cmd)

.PHONY: teardown
teardown:
$(pipenv) --rm

$(wheel_file): setup.py $(ot_sources)
$(clean_cmd)
.PHONY: wheel
wheel: export OPENTRONS_PROJECT=$(project_rs_default)
wheel:
$(clean_wheel_cmd)
$(python) setup.py $(wheel_opts) bdist_wheel
$(SHX) rm -rf build
$(SHX) ls dist

wheel: $(wheel_file)

$(sdist_file): setup.py $(ot_sources)
$(clean_cmd)
.PHONY: sdist
sdist: export OPENTRONS_PROJECT=$(project_ot3_default)
sdist:
$(clean_sdist_cmd)
$(python) setup.py sdist
$(SHX) rm -rf build
$(SHX) ls dist

sdist: $(sdist_file)

.PHONY: test
test:
$(pytest) $(tests) $(test_opts)
Expand Down
Loading