Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish React Native Bridge to Bintray from CI #3230

Merged
merged 33 commits into from
Mar 26, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
95a7349
Point Gutenberg to the RN composite build support branch
mokagio Mar 3, 2021
85772fc
Track changes to package-lock.json from running `npm install`
mokagio Mar 3, 2021
9d45990
Add CI steps to build and publish the RN Android bridge to Bintray
mokagio Jan 21, 2021
186ff1a
Adjust CI Bintray workflow to new Gutenberg project setup
mokagio Mar 3, 2021
a218a58
DEBUG - Allow publishing from branches with no PR
mokagio Mar 3, 2021
27ae5b2
Re-enable Bintray publishing step
mokagio Mar 3, 2021
176a3d8
Add node_modules to cache
mokagio Mar 4, 2021
2f4ad07
Try to cache gutenberg's node_modules instead
mokagio Mar 4, 2021
dd4d06c
Update gutenberg submodule to commit removing the waitForJitpack call
oguzkocer Mar 4, 2021
a88ca61
Rename PREFIX to VERSION in CI config for clarity
mokagio Mar 5, 2021
ef265a2
Remove gutenberg npm install from ci
oguzkocer Mar 4, 2021
98105ea
Point Gutenberg submodule to latest composite build iteration
mokagio Mar 8, 2021
59ebed0
Re-enable full pipeline + make build bundle step dependent on tests
mokagio Mar 9, 2021
98d313c
Add more documentation to the new Android RN bridge jobs
mokagio Mar 9, 2021
f1b1ee6
Rename Android AAR publishing jobs for more clarity
mokagio Mar 9, 2021
d5b55ac
Run `npm install` as requested by "Check Correctness" CI job
mokagio Mar 9, 2021
17d1c54
Run `npm install` after deleting the node_modules folder
mokagio Mar 9, 2021
c92143a
Remove debug code from CircleCI config
mokagio Mar 9, 2021
7450902
Reset changes in package-lock.json
mokagio Mar 9, 2021
ea0e970
Run `npm install` and track result
mokagio Mar 9, 2021
554e10b
Adds WPAndroid Integration document
oguzkocer Mar 15, 2021
56cd006
Add semantic line breaks
oguzkocer Mar 16, 2021
21e81ff
Modify and apply wpandroid integration doc suggestions by Gio
oguzkocer Mar 16, 2021
6a5ca7e
Improve WPAndroid Integration document per Matt's PR feedback
oguzkocer Mar 16, 2021
ba88fd3
Change how it works sections to be titles in WPAndroid integration
oguzkocer Mar 16, 2021
cf789fb
Run Bintray job on Linux + Build on tags + Bail early
mokagio Mar 17, 2021
ca6390a
Update Gutenberg to latest composite build commit
mokagio Mar 17, 2021
4d3148d
Adds RN Android bridge integration documentation (#3262)
mokagio Mar 17, 2021
4a492ba
Update Gutenberg submodule to latest version of composite build work
mokagio Mar 26, 2021
e39c6a6
Merge remote-tracking branch 'origin/develop' into mokagio/publish-to…
mokagio Mar 26, 2021
9354207
Update the Gutenberg – Fixes standalone react-native-editor build in CI
mokagio Mar 26, 2021
17d9031
Updates after gutenberg merge
mchowning Mar 26, 2021
c43d2ba
Use tab in .gitmodules
mchowning Mar 26, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 109 additions & 2 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,11 +49,11 @@ parameters:
android-docker-image:
type: string
# Hash points to previous version with node 12. When everything works with node 14 it can be removed
default: "circleci/android:api-29-node@sha256:71d61d6c21b29948d57120f476a83cc322a280979bce355c5a0ad771293ca380"
default: "circleci/android:api-29-node@sha256:71d61d6c21b29948d57120f476a83cc322a280979bce355c5a0ad771293ca380"
linux-machine-image:
type: string
# Latest supported ubuntu image from https://circleci.com/docs/2.0/configuration-reference/#available-machine-images
default: "ubuntu-2004:202010-01"
default: "ubuntu-2004:202010-01"

jobs:
checks:
Expand Down Expand Up @@ -253,6 +253,104 @@ jobs:
include_job_number_field: false
include_project_field: false
failure_message: ':red_circle: Scheduled tests failed on iOS device!'
android-build-and-publish-react-native-bridge:
# This job builds the JS bunle required by the Android version of the
# react-native-bridge package, builds the AAR for that package, and uploads
# it to Bintray.
#
# Running on a Ubuntu machine to have more memory compared to the Docker +
# Node or Docker + Android images that CircleCI offers (circleci/node)
#
# Also running all in one job so that we can bail out early under cetrain
# conditions and save CI cycles.
#
# See: https://github.com/wordpress-mobile/gutenberg-mobile/pull/3038#issuecomment-765334675
machine:
image: << pipeline.parameters.linux-machine-image >>
steps:
- run:
name: Evaluate whether to run
command: |
# This is a partial duplication of the logic to compute the version
# to use for the AAR from below. It's useful to have it here
# because there are conditions under which we shouldn't run the job
# that can't be captured in a CircleCI filter.
#
# Exiting here saves a non-trivial amount of CI time because of how
# heavy the repo is to checkout (it takes at least 1 minute) and
# how time consuming running `npm ci` is.
Comment on lines +274 to +281
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per comment, I tried having this in a dedicated script, but the repo checkout takes at least 1 minute.

On the other hand, the builds that would actually exit from here are a small % of the total builds running on this repo 🤔 Maybe by taking that into consideration the cost in CI has less weight than the duplication cost here?

Keen to hear your thoughts. It's a straightforward change to make, but it might be best to make it in a followup PR, just so we can get this moving along.

if [[ -z ${CIRCLE_TAG+x} ]]; then
if [[ -n "$CIRCLE_PULL_REQUEST" ]]; then
echo "Build initiated from a branch with a pull request: proceeding..."
elif [[ $CIRCLE_BRANCH == "develop" ]]; then
echo "Build initiated on $CIRCLE_BRANCH: proceeding..."
else
# We end up here on the first push of a new branch, when there
# isn't a PR open for it yet.
echo "Build initiated from a feature branch with no pull request: aborting."
circleci-agent step halt
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This build is an example of this scenario

image

Unfortunately, CircleCI doesn't treat halts differently and the UI might be misleading as it looks like the build passed.

fi
else
echo "Build initiated from a tag: proceeding..."
fi
- checkout
- checkout-submodules
- run:
# Setting up Android before fetching the Node dependencies because
# this step is faster, so if it fails we'll learn about it sooner and
# avoid wasting cycles.
name: Setup Android Tooling
command: .circleci/setup-android-on-ubuntu.sh
Comment on lines +298 to +303
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ceyhun suggested looking into spinning up a Docker container to avoid this manual setup.

I definitely want to look into it but opted for this approach which I already know works for this first version (which has already been dragging along for a while 😞 )

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mokagio 🙇 This already looks awesome and we can surely improve it later once we have it in place and working first 👍

- npm-install
- run:
name: Build JavaScript Bundle
command: npm run bundle:android
# The job can take a long time, so let's bump the no_output_timeout
# to more than the default 10m
no_output_timeout: 20m
- run:
name: Build React Native Bridge & Upload to Bintray
environment:
# This value is set in the setup script, but I'm not sure whether
# it's available after executing it
ANDROID_SDK_ROOT: /usr/lib/android-sdk
command: |
# We publish from PRs as well as the main branch and tags to help
# developers to iterate faster.
#
# The builds from a PR have a different version so that they can be
# deleted by
# https://github.com/Automattic/bintray-garbage-collector/
# once their PR is merged.
#
# If we're not on a PR but a merge commit on the develop branch the
# version will be the "develop-" followed by the commit SHA1.
#
# On a tag, the version should be the tag value.
#
# If we're on a different branch that has no open PR, we should
# not build and exit early instead.
if [[ -z "${CIRCLE_TAG+x}" ]]; then
if [[ -n "$CIRCLE_PULL_REQUEST" ]]; then
PR_NUMBER=$(basename $CIRCLE_PULL_REQUEST)
# CIRCLE_SHA1 should always be available.
VERSION="$PR_NUMBER-$CIRCLE_SHA1"
elif [[ $CIRCLE_BRANCH == "develop" ]]; then
VERSION="$CIRCLE_BRANCH-$CIRCLE_SHA1"
else
# This happens on the first push of a new branch, when there
# isn't a PR open for it yet.
echo "Running on a feature branch with no open PR."
echo "The checks previous to this step should not have let this happen."
exit 1
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I decided to fail the build here because if this happen there's something wrong in the setup and we should know about it sooner rather than later.

fi
else
VERSION=$CIRCLE_TAG
fi

cd gutenberg/packages/react-native-bridge/android

./publish-aztec-and-bridge.sh $VERSION

workflows:
gutenberg-mobile:
Expand Down Expand Up @@ -301,6 +399,15 @@ workflows:
filters:
branches:
only: /^dependabot/submodules/.*/
- android-build-and-publish-react-native-bridge:
name: Build Android RN Bridge & Publish to Bintray
filters:
branches:
# Don't run on trunk, otherwise, run on every other branch
ignore: trunk
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason I set this to not run on trunk is that from my understanding of our workflows discussions, we never need to access a build from that branch.

Copy link
Contributor

@mchowning mchowning Mar 17, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that all of our release tags are made on trunk. I may just not be understanding how this CI job is used.

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mchowning!

Every time a tag is pushed, this workflow will run and upload an AAR with the version being the tag name. See this build for an example: it started from GitHub receiving the tag mokagio-test-20210317-173215 and uploaded a binary to Bintray with version mokagio-test-20210317-173215.

Do you ever need to reference in WordPress Android a commit from gutenbreg-mobile's trunk that has not a matching tag? If that's the case, I can easily make builds on trunk behave like builds on develop: publish a trunk-< commit SHA1 > version to Bintray.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you ever need to reference in WordPress Android a commit from gutenbreg-mobile's trunk that has not a matching tag?

I don't think so. As long as all the tags get pushed we should be fine. 👍

tags:
# Run for every tag
only: /.*/
Comment on lines +408 to +410
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's an example of build from a tag: mokagio-test-20210317-173215.


ui-tests-full-scheduled:
jobs:
Expand Down
68 changes: 68 additions & 0 deletions .circleci/setup-android-on-ubuntu.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
#!/bin/bash

set -e

# Update the packages list, just in case
sudo apt update

# Android depends on Java. Luckily, the Ubuntu machine that CircleCI provides
# already includes it. If that wouldn't the case, we'd need to install
# openjdk-11-jdk.

# Install the Android SDK
sudo apt install android-sdk

# The Android SDK gets installed in /usr/lib/. Some of the tools expect the SDK
# location to be available via a known environment variable, so let's export
# it.
export ANDROID_SDK_ROOT=/usr/lib/android-sdk

# Next, we need to install the command line tools to access the sdkmanager tool
# to 1) install the platform and build-tools that the project needs and 2)
# agree to the licenses so we can run the tools.
#
# For some reason, the SDK doesn't come with the command line tools. They are
# not shipped via apt either. We need to download them manually.
#
# You can see what the latest version is at:
# https://developer.android.com/studio#command-tools
#
# See also:
# https://stackoverflow.com/questions/53994924/sdkmanager-command-not-found-after-installing-android-sdk
cmd_line_tools_version=6858069
cmd_line_tools_zip_name="commandlinetools-linux-${cmd_line_tools_version}_latest.zip"
wget https://dl.google.com/android/repository/$cmd_line_tools_zip_name
unzip $cmd_line_tools_zip_name

# Move the command line tools in the location they expect to be in to work.
#
# Alternatively, we could keep them here but then we'd have to pass the
# $ANDROID_SDK_ROOT location to each invocation of the sdkmanager command line
# tool using the --sdk_root option. Doing so would also result in the extra
# platform and build tool we'd install to end up in / rather than in the
# correct subfolder of $ANDROID_SDK_ROOT.
cmdline_tools_root=$ANDROID_SDK_ROOT/cmdline-tools
sudo mkdir $cmdline_tools_root
cmdline_tools_location=$cmdline_tools_root/latest
sudo mv ./cmdline-tools $cmdline_tools_location

# Store the path to sdkmanager for easier access in the rest of the script.
#
# This script is meant to run on a CI machine that gets torn down on every run.
#
# If this was a development machine instead, we would add the whole command line tools bin folder to the path by adding this to the RC file of the shell in use (e.g.: .bashrc or .zshrc):
#
# export PATH=$PATH:$cmdline_tools_location/bin/
sdkmanager_bin=$cmdline_tools_location/bin/sdkmanager

# We need a specific version of the Android platform and build tools.
build_tools_version="29.0.2"
platform_version="28"
yes | sudo $sdkmanager_bin --install "build-tools;$build_tools_version"
yes | sudo $sdkmanager_bin --install "platforms;android-$platform_version"
# Apparently, we also need platform version 29?
# See https://app.circleci.com/pipelines/github/wordpress-mobile/gutenberg-mobile/10739/workflows/a96e43eb-6d64-490e-87ab-ef1f1df2eb1d/jobs/56631
yes | sudo $sdkmanager_bin --install "platforms;android-29"

# Accept the licenses
yes | sudo $sdkmanager_bin --licenses
119 changes: 119 additions & 0 deletions WPAndroid Integration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# WordPress Android Integration Guide

### Summary

WPAndroid by default integrates `react-native-bridge` as a binary dependency that's fetched from a remote maven repo.
The CI in this repo will deploy a new version each time a commit is pushed to `develop` or an open PR, as well as each time a tag is created.
This is the only required dependency.
Whenever a new version is deployed, we can simply update `ext.gutenbergMobileVersion` in [build.gradle](https://github.com/wordpress-mobile/WordPress-Android/blob/develop/build.gradle).

During development, we can use composite builds and dependency substitution to point WPAndroid to our local checkout of `gutenberg-mobile`.
To do this, we need to copy the [local-builds.gradle-example file](https://github.com/wordpress-mobile/WordPress-Android/blob/develop/local-builds.gradle-example) in WPAndroid renaming it to `local-builds.gradle` and change the `localGutenbergMobilePath` value.
Building the project as such will make WPAndroid build the source code for `react-native-bridge`, `react-native-aztec` and other sublibraries from the locally checked out folder.
It'll also expect the metro server to be running to fetch the JS bundle.

- [Work with local gutenberg-mobile checkout](#work-with-local-gutenberg-mobile-checkout)
- [Deploy a new version of `react-native-bridge` from CI](#deploy-a-new-version-of-react-native-bridge-from-ci)
- [Manually deploy a new version of `react-native-bridge`](#manually-deploy-a-new-version-of-react-native-bridge)
- [Update WPAndroid's binary version](#update-wpandroids-binary-version)
- [How to](#how-to)
- [How to test a gutenberg-mobile PR in WPAndroid](how-to-test-a-gutenberg-mobile-pr-in-wpandroid)
- [How to share WIP WPAndroid APK](how-to-share-wip-wpandroid-apk)
- [How to checkout gutenberg-mobile to deployed binary version](#how-to-checkout-gutenberg-mobile-to-deployed-binary-version)

---

### Work with local gutenberg-mobile checkout

From your gutenberg-mobile checkout:

- Make sure you run `npm install` or `npm ci`
- Run metro server with `npm run start:reset`

From your WordPress-Android checkout:

- Copy [local-builds.gradle-example](https://github.com/wordpress-mobile/WordPress-Android/blob/develop/local-builds.gradle-example) renaming it to `local-builds.gradle`
- Update `localGutenbergMobilePath`to your local `gutenberg-mobile`checkout
- Run the project

#### How it works:

When [settings.gradle](https://github.com/wordpress-mobile/WordPress-Android/blob/develop/settings.gradle) finds `localGutenbergMobilePath` in `local-builds.gradle`, it'll substitute the below binary dependency with the local folder:

```
implementation "$rootProject.gradle.ext.gutenbergMobileBinaryPath:$rootProject.ext.gutenbergMobileVersion"
```

`react-native-bridge` is setup to include all the other projects, so no other action is necessary.

---

### Deploy a new version of `react-native-bridge` from CI

There are a few different ways to do this:

1. Open a new PR or push a new commit to an open PR which will be published as `<PR number>-<commit full SHA1>`
2. Merge a PR to `develop` which will be published as `<develop>-<commit full SHA1>`
3. Create a new tag which will be published as `<tag name>`

#### How it works:

CI will run the following commands:

- `npm ci`
- `npm run bundle:android`
- `gutenberg/packages/react-native-bridge/android/publish-aztec-and-bridge.sh <version>`

---

### Manually deploy a new version of `react-native-bridge`

_**IMPORTANT:** This should only be used for testing.
We shouldn't commit a manually deployed version to WPAndroid's `develop` because the process bypasses the guarantees CI would otherwise give us._

- Make sure you run `npm install` or `npm ci` first
- Run `npm run bundle:android` so the JS bundle is created
- Run `publish-aztec-and-bridge.sh` script in `react-native-bridge`, with the version name as its argument. i.e `./gutenberg/packages/react-native-bridge/android/publish-aztec-and-bridge.sh locally-built-by-<your_name>-<commit full SHA1>`

#### How it works:

Besides checking a few things, `publish-aztec-and-bridge.sh` script will clean the project, copy the bundled JS file into its assets, publish `react-native-aztec` and use that version to build and publish `react-native-bridge`.

---

### Update WPAndroid's binary version

- Find [the version deployed from CI](#deploy-a-new-version-of-react-native-bridge-from-ci) or [the version you've manually deployed](#manually-deploy-a-new-version-of-react-native-bridge)
- Update `ext.gutenbergMobileVersion` property in [build.gradle](https://github.com/wordpress-mobile/WordPress-Android/blob/develop/build.gradle) to the new version

In order to test this, make sure `localGutenbergMobilePath` in your `local-builds.gradle` file is commented out as otherwise the binary version will be ignored.

---

## How to

### How to test a gutenberg-mobile PR in WPAndroid

Assuming that there is no open WPAndroid PR yet:

1. Setup WPAndroid to [work with local gutenberg-mobile checkout](#work-with-local-gutenberg-mobile-checkout)
2. Switch `gutenberg-mobile` to the branch you're testing
3. Run the app

---

### How to share a WIP WPAndroid APK

1. Either (a) open a `gutenberg-mobile` PR which will [publish a new version](#deploy-a-new-version-of-react-native-bridge-from-ci) or (b) [manually publish a new version](#manually-deploy-a-new-version-of-react-native-bridge)
2. Open a WPAndroid PR [updating the binary version](#update-wpandroids-binary-version)
3. Peril will publish a comment to the WPAndroid PR with a link to the APK

Alternatively, once the `react-native-bridge` version is published, we can locally create the APK.
**Note that there is no way to use composite build to create a WPAndroid APK that can be shared. This is because composite build only works with the local metro server.**

---

### How to checkout gutenberg-mobile to deployed binary version

Whenever the CI publishes `react-native-bridge`, it will either include the full commit hash or the tag in its name.
So, we can checkout simply checkout the commit hash or the tag.
2 changes: 1 addition & 1 deletion gutenberg
Submodule gutenberg updated 225 files