Publish React Native Bridge to Bintray from CI #3230

mokagio · 2021-03-08T23:38:02Z

Before merging this PR, we need to merge React native composite and binary build support WordPress/gutenberg#29488 into Gutenberg's trunk and update the submodule has here.
Before merging this PR, it needs to implement the versioning strategy specified here

This PR adds CI jobs to build and publish the AAR for the react-native-bridge package (which lives in Gutenberg).

I originally planned to open this against the gutenberg-composite-build project WIP branch, but given this doesn't change anything in the codebase itself other than the Gutenberg submodule hash, I think develop is appropriate.

To test:

Verify that CI builds
Head over to Test - Use RN Gutenberg bridge built from the gutenberg-mobile CI WordPress-Android#14215 which uses the binary published from here and:

Verify it builds in CI
Verify you can run the app and use the editor locally, making sure the localGutenbergMobile path in your local-builds.gradle is commented out, otherwise it'll kick off a composite build, which won't use the binary.

PR submission checklist:

I have considered adding unit tests where possible. (N.A.)
I have considered if this change warrants user-facing release notes more info and have added them to RELEASE-NOTES.txt if necessary. (N.A.)

See WordPress/gutenberg#29488

This is a key step in allowing the Android app to build without the overhead of the `gutenberg-mobile` submodule. Currently, there is a manual step to unlock the process if the Android tests passed. That's because of how long it takes for the build step to run: ~15 minutes. Eventually, it would be great to have some intelligence in the PR to be able to determine whether it should start a build for the Android bridge.

See WordPress/gutenberg#29488

See wordpress-mobile/gutenberg-mobile#3230

peril-wordpress-mobile · 2021-03-09T03:30:37Z

Wanna run full suite of Android and iOS UI tests? Click here and 'Approve' CI job!

See https://app.circleci.com/pipelines/github/wordpress-mobile/gutenberg-mobile/10804/workflows/c1cfe5f8-4ccc-4ed7-9f37-ffbeb09e1c5c/jobs/56874

mokagio · 2021-03-09T04:18:17Z

.circleci/config.yml

+            # The build process for the react-navite-bridge requires the App.js
+            # bundle and also access to the node_modules folder.


Should I a link to the line of code that requires node_modules diff on GitHub once the PR for that is merged?

Suggested change

# The build process for the react-navite-bridge requires the App.js

# bundle and also access to the node_modules folder.

# The build process for the react-native-bridge requires the App.js and the node_modules folder.

We technically don't need the node_modules once the bundle is created. However, without it configuration gets a bit messy. I don't think it's worth adding a link here and I am hoping we can work towards not needing the node_modules folder.

.circleci/config.yml

mokagio · 2021-03-09T04:21:28Z

I'm not sure where all those package-lock.json changes came from... I checked out a new branch from develop, pointed Gutenberg to the same hash as here, deleted the node_modules folder, and run npm install and nothing changed in the file 🤔

loremattei · 2021-03-10T11:19:49Z

.circleci/config.yml

+    steps:
+      - checkout
+      - checkout-submodules
+      - npm-install


The name is misleading :-) This actually runs npm ci as expected 👍

loremattei · 2021-03-10T11:40:39Z

.gitmodules

@@ -1,6 +1,6 @@
 [submodule "gutenberg"]
 	path = gutenberg
-	url = ../../WordPress/gutenberg.git
+	url = [email protected]:oguzkocer/gutenberg.git


I'm pretty sure this is intentional to test this PR, but I'm leaving this comment just to make sure this is not something which fell between the cracks :-)

Correct & great idea!

See WordPress/gutenberg#29488

See wordpress-mobile/gutenberg-mobile#3230

mokagio

@mchowning @cameronvoell @ceyhun @loremattei here's an updated version of the workflow that incorporates the versioning workflow discussed internally and most of your suggestions.

Can I ask you to have another look?

I updated the test PR on WordPress Android to use the binary from the latest CI build, if you want to verify it builds and works.

mokagio · 2021-03-17T06:27:20Z

.circleci/config.yml

+            # This is a partial duplication of the logic to compute the version
+            # to use for the AAR from below. It's useful to have it here
+            # because there are conditions under which we shouldn't run the job
+            # that can't be captured in a CircleCI filter.
+            #
+            # Exiting here saves a non-trivial amount of CI time because of how
+            # heavy the repo is to checkout (it takes at least 1 minute) and
+            # how time consuming running `npm ci` is.


As per comment, I tried having this in a dedicated script, but the repo checkout takes at least 1 minute.

On the other hand, the builds that would actually exit from here are a small % of the total builds running on this repo 🤔 Maybe by taking that into consideration the cost in CI has less weight than the duplication cost here?

Keen to hear your thoughts. It's a straightforward change to make, but it might be best to make it in a followup PR, just so we can get this moving along.

mokagio · 2021-03-17T06:28:40Z

.circleci/config.yml

+      - run:
+          # Setting up Android before fetching the Node dependencies because
+          # this step is faster, so if it fails we'll learn about it sooner and
+          # avoid wasting cycles.
+          name: Setup Android Tooling
+          command: .circleci/setup-android-on-ubuntu.sh


@ceyhun suggested looking into spinning up a Docker container to avoid this manual setup.

I definitely want to look into it but opted for this approach which I already know works for this first version (which has already been dragging along for a while 😞 )

Thanks @mokagio 🙇 This already looks awesome and we can surely improve it later once we have it in place and working first 👍

mokagio · 2021-03-17T06:29:29Z

.circleci/config.yml

+                # isn't a PR open for it yet.
+                echo "Running on a feature branch with no open PR."
+                echo "The checks previous to this step should not have let this happen."
+                exit 1


I decided to fail the build here because if this happen there's something wrong in the setup and we should know about it sooner rather than later.

mokagio · 2021-03-17T06:31:07Z

.circleci/config.yml

+          filters:
+            branches:
+              # Don't run on trunk, otherwise, run on every other branch
+              ignore: trunk


The reason I set this to not run on trunk is that from my understanding of our workflows discussions, we never need to access a build from that branch.

I believe that all of our release tags are made on trunk. I may just not be understanding how this CI job is used.

Thanks @mchowning!

Every time a tag is pushed, this workflow will run and upload an AAR with the version being the tag name. See this build for an example: it started from GitHub receiving the tag mokagio-test-20210317-173215 and uploaded a binary to Bintray with version mokagio-test-20210317-173215.

Do you ever need to reference in WordPress Android a commit from gutenbreg-mobile's trunk that has not a matching tag? If that's the case, I can easily make builds on trunk behave like builds on develop: publish a trunk-< commit SHA1 > version to Bintray.

Do you ever need to reference in WordPress Android a commit from gutenbreg-mobile's trunk that has not a matching tag?

I don't think so. As long as all the tags get pushed we should be fine. 👍

mokagio · 2021-03-17T08:36:59Z

.circleci/config.yml

+                # We end up here on the first push of a new branch, when there
+                # isn't a PR open for it yet.
+                echo "Build initiated from a feature branch with no pull request: aborting."
+                circleci-agent step halt


This build is an example of this scenario

Unfortunately, CircleCI doesn't treat halts differently and the UI might be misleading as it looks like the build passed.

mokagio · 2021-03-17T08:41:48Z

.circleci/config.yml

+            tags:
+              # Run for every tag
+              only: /.*/


Here's an example of build from a tag: mokagio-test-20210317-173215.

…-bintray-with-script-merge-test-both-on-latest

mokagio · 2021-03-26T03:49:10Z

@cameronvoell @mchowning @oguzkocer the tests still fail, even after syncing with develop and the latest WordPress/gutenberg#29488, which is synced with Gutenberg's trunk.

This is the failure on both steps:

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':app:testDebugUnitTest'.
> Could not resolve all task dependencies for configuration ':app:debugUnitTestRuntimeClasspath'.
   > Could not find org.wordpress-mobile:hermes-debug-mirror:0.61.5.
     Searched in the following locations:
       - file:/home/circleci/.m2/repository/org/wordpress-mobile/hermes-debug-mirror/0.61.5/hermes-debug-mirror-0.61.5.pom
       - file:/home/circleci/project/gutenberg/node_modules/react-native/android/org/wordpress-mobile/hermes-debug-mirror/0.61.5/hermes-debug-mirror-0.61.5.pom
       - https://dl.google.com/dl/android/maven2/org/wordpress-mobile/hermes-debug-mirror/0.61.5/hermes-debug-mirror-0.61.5.pom
       - https://jcenter.bintray.com/org/wordpress-mobile/hermes-debug-mirror/0.61.5/hermes-debug-mirror-0.61.5.pom
       - https://jitpack.io/org/wordpress-mobile/hermes-debug-mirror/0.61.5/hermes-debug-mirror-0.61.5.pom
     Required by:
         project :app > project :android:react-native-bridge

diff of the error output between the two steps to show it's the same

4,5c4,5
< Could not determine the dependencies of task ':app:testDebugUnitTest'.
< > Could not resolve all task dependencies for configuration ':app:debugUnitTestRuntimeClasspath'.
---
> Could not determine the dependencies of task ':app:processDebugResources'.
> > Could not resolve all task dependencies for configuration ':app:debugRuntimeClasspath'.

🤔

I bet the issue has to do with this project not having the S3 repository where our mirror lives. Let's see if I can do something about it...

mokagio · 2021-03-26T05:50:37Z

All green in CI 🎉 Thanks @oguzkocer for the help.

I'm going to turn this "Ready for review" even though the "Before merging this PR, we need to merge WordPress/gutenberg#29488 into Gutenberg's trunk and update the submodule has here" hasn't been checked yet.

@mchowning @cameronvoell let me know if you think we should merge before or after the Gutenberg counterpart lands into Gutenberg's trunk. I'm writing this in my Friday EOD, so feel free to take the PR over and merge it if you want, without waiting for me.

mchowning

mokagio and others added 11 commits March 3, 2021 16:31

Point Gutenberg to the RN composite build support branch

95a7349

See WordPress/gutenberg#29488

Track changes to package-lock.json from running npm install

85772fc

Adjust CI Bintray workflow to new Gutenberg project setup

186ff1a

See WordPress/gutenberg#29488

DEBUG - Allow publishing from branches with no PR

a218a58

Re-enable Bintray publishing step

27ae5b2

Add node_modules to cache

176a3d8

Try to cache gutenberg's node_modules instead

2f4ad07

Update gutenberg submodule to commit removing the waitForJitpack call

dd4d06c

Rename PREFIX to VERSION in CI config for clarity

a88ca61

Remove gutenberg npm install from ci

ef265a2

This was referenced Mar 8, 2021

Update Gutenberg submodule with version generating AAR for Bintray #3010

Closed

Build JS bundle and upload AAR to Bintray in one step on CI #3111

Closed

Point Gutenberg submodule to latest composite build iteration

98105ea

mokagio changed the title ~~Point Gutenberg to the RN composite build support branch~~ Publish React Native Bridge to Bintray from CI Mar 9, 2021

mokagio added a commit to wordpress-mobile/WordPress-Android that referenced this pull request Mar 9, 2021

Test - Use RN Gutenberg bridge built from the gutenberg-mobile CI

181e76f

See wordpress-mobile/gutenberg-mobile#3230

mokagio mentioned this pull request Mar 9, 2021

Test - Use RN Gutenberg bridge built from the gutenberg-mobile CI wordpress-mobile/WordPress-Android#14215

Closed

mokagio added a commit to wordpress-mobile/WordPress-Android that referenced this pull request Mar 9, 2021

Test - Use RN Gutenberg bridge built from the gutenberg-mobile CI

014275d

See wordpress-mobile/gutenberg-mobile#3230

mokagio added 3 commits March 9, 2021 14:18

Re-enable full pipeline + make build bundle step dependent on tests

59ebed0

Add more documentation to the new Android RN bridge jobs

98d313c

Rename Android AAR publishing jobs for more clarity

f1b1ee6

mokagio mentioned this pull request Mar 9, 2021

Add CI step to publish Android React Native Bridge to Bintray #3038

Closed

2 tasks

mokagio added 2 commits March 9, 2021 14:46

Run npm install as requested by "Check Correctness" CI job

d5b55ac

See https://app.circleci.com/pipelines/github/wordpress-mobile/gutenberg-mobile/10804/workflows/c1cfe5f8-4ccc-4ed7-9f37-ffbeb09e1c5c/jobs/56874

Run npm install after deleting the node_modules folder

17d1c54

mokagio commented Mar 9, 2021

View reviewed changes

.circleci/config.yml Outdated Show resolved Hide resolved

mokagio commented Mar 9, 2021

View reviewed changes

.circleci/config.yml Outdated Show resolved Hide resolved

Remove debug code from CircleCI config

c92143a

loremattei reviewed Mar 10, 2021

View reviewed changes

oguzkocer and others added 7 commits March 15, 2021 17:19

Adds WPAndroid Integration document

554e10b

Add semantic line breaks

56cd006

Modify and apply wpandroid integration doc suggestions by Gio

21e81ff

Improve WPAndroid Integration document per Matt's PR feedback

6a5ca7e

Change how it works sections to be titles in WPAndroid integration

ba88fd3

Run Bintray job on Linux + Build on tags + Bail early

cf789fb

Update Gutenberg to latest composite build commit

ca6390a

See WordPress/gutenberg#29488

mokagio added a commit to wordpress-mobile/WordPress-Android that referenced this pull request Mar 17, 2021

Test - Use RN Gutenberg bridge built from the gutenberg-mobile CI

5c8c934

See wordpress-mobile/gutenberg-mobile#3230

mokagio mentioned this pull request Mar 17, 2021

Adds WPAndroid Integration document #3262

Merged

Adds RN Android bridge integration documentation (#3262)

4d3148d

mokagio commented Mar 17, 2021

View reviewed changes

mokagio requested review from ceyhun and loremattei March 17, 2021 08:53

oguzkocer approved these changes Mar 24, 2021

View reviewed changes

oguzkocer mentioned this pull request Mar 24, 2021

Add instructions for publish access token to WordPress Android Integration Guide #3302

Open

mokagio added 2 commits March 26, 2021 11:00

Update Gutenberg submodule to latest version of composite build work

4a492ba

Merge remote-tracking branch 'origin/develop' into mokagio/publish-to…

e39c6a6

…-bintray-with-script-merge-test-both-on-latest

Update the Gutenberg – Fixes standalone react-native-editor build in CI

9354207

mokagio marked this pull request as ready for review March 26, 2021 05:50

Updates after gutenberg merge

17d9031

mchowning approved these changes Mar 26, 2021

View reviewed changes

mchowning enabled auto-merge March 26, 2021 21:41

Use tab in .gitmodules

c43d2ba

mchowning merged commit 2d631f6 into develop Mar 26, 2021

mchowning deleted the mokagio/publish-to-bintray-with-script branch March 26, 2021 22:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Publish React Native Bridge to Bintray from CI #3230

Publish React Native Bridge to Bintray from CI #3230

mokagio commented Mar 8, 2021 •

edited

Loading

peril-wordpress-mobile bot commented Mar 9, 2021 •

edited

Loading

mokagio Mar 9, 2021

oguzkocer Mar 9, 2021

mokagio commented Mar 9, 2021

loremattei Mar 10, 2021

loremattei Mar 10, 2021

mokagio Mar 11, 2021

mokagio left a comment

mokagio Mar 17, 2021

mokagio Mar 17, 2021

ceyhun Mar 19, 2021

mokagio Mar 17, 2021

mokagio Mar 17, 2021

mchowning Mar 17, 2021 •

edited

Loading

mokagio Mar 18, 2021

mchowning Mar 18, 2021

mokagio Mar 17, 2021

mokagio Mar 17, 2021

mokagio commented Mar 26, 2021 •

edited

Loading

mokagio commented Mar 26, 2021

mchowning left a comment

		# The build process for the react-navite-bridge requires the App.js
		# bundle and also access to the node_modules folder.

	# The build process for the react-navite-bridge requires the App.js
	# bundle and also access to the node_modules folder.
	# The build process for the react-native-bridge requires the App.js and the node_modules folder.

Publish React Native Bridge to Bintray from CI #3230

Publish React Native Bridge to Bintray from CI #3230

Conversation

mokagio commented Mar 8, 2021 • edited Loading

peril-wordpress-mobile bot commented Mar 9, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mokagio commented Mar 9, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mokagio left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mchowning Mar 17, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mokagio commented Mar 26, 2021 • edited Loading

mokagio commented Mar 26, 2021

mchowning left a comment

Choose a reason for hiding this comment

mokagio commented Mar 8, 2021 •

edited

Loading

peril-wordpress-mobile bot commented Mar 9, 2021 •

edited

Loading

mchowning Mar 17, 2021 •

edited

Loading

mokagio commented Mar 26, 2021 •

edited

Loading