Android ide build and CHIPTest build into CI (#10371)

* added IDE build for CHIPTool and script build for CHIPTest in build_example.py * fix spell and toBoolean issue to fix CHIPTool & CHIPTest script build in build_example.py * fixed restyled-io and ci * sync android build document * fixed restyled-io * added document for building CHIPTest * fixed restyled-io & ci
project-chip · Oct 12, 2021 · c2b7217 · c2b7217
1 parent 8689443
commit c2b7217
Show file tree

Hide file tree

Showing 19 changed files with 312 additions and 161 deletions.
diff --git a/.github/.wordlist.txt b/.github/.wordlist.txt
@@ -137,6 +137,7 @@ ChipImResponder
 ChipLight
 ChipMessageLayer
 CHIPTool
+CHIPTest
 chmod
 chrpath
 cipd
@@ -532,6 +533,8 @@ matterBuildSrcDir
 matterc
 matterd
 MatterLock
+matterSdkSourceBuild
+matterUTestLib
 MaxInterval
 MaxRtrAdvInterval
 mbedTLS

diff --git a/docs/guides/android_chiptool_building.md → docs/guides/android_building.md b/docs/guides/android_chiptool_building.md → docs/guides/android_building.md
@@ -1,15 +1,16 @@
-# Building Android CHIPTool
+# Building Android
 
-Android CHIPTool is an application for Android for commissioning and controlling
-CHIP accessory devices.
+There are following Apps on Android
 
-CHIPTool offers the following features:
-
--   Scan a CHIP QR code and display payload information to the user
--   Read the NFC tag containing CHIP onboarding information
--   Commission a CHIP device
--   Send echo requests to the CHIP echo server
--   Send on/off cluster requests to a CHIP device
+-   CHIPTool - Android CHIPTool is an application for Android for commissioning
+    and controlling CHIP accessory devices. It offers the following features: -
+    Scan a CHIP QR code and display payload information to the user - Read the
+    NFC tag containing CHIP onboarding information - Commission a CHIP device -
+    Send echo requests to the CHIP echo server - Send on/off cluster requests to
+    a CHIP device
+-   CHIPTest
+    -   Android CHIPTest is an application for Android for running CHIP's unit
+        tests
 
 <hr>
 
@@ -26,8 +27,8 @@ CHIPTool offers the following features:
 
 ## Source files
 
-You can find source files of the Android CHIPTool application in the
-`src/android/CHIPTool` directory.
+You can find source files of the Android applications in the `src/android/`
+directory.
 
 <hr>
 
@@ -75,7 +76,11 @@ Complete the following steps to prepare the CHIP build:
 
 <a name="building-scripts"></a>
 
-## Building Android CHIPTool from scripts
+## Building Android CHIPTool
+
+<a name="chiptool_scripts_build"></a>
+
+### From scripts
 
 This is the simplest option. In the command line, run the following command from
 the top CHIP directory:
@@ -87,20 +92,20 @@ the top CHIP directory:
 See the table above for other values of `TARGET_CPU`.
 
 The debug Android package `app-debug.apk` will be generated at
-`out/android-$TARGET_CPU-chip_tool/outputs/apk/debug/`, and can be installed
+`out/android-$TARGET_CPU-chip-tool/outputs/apk/debug/`, and can be installed
 with
 
 ```shell
-adb install out/android-$TARGET_CPU-chip_tool/outputs/apk/debug/app-debug.apk
+adb install out/android-$TARGET_CPU-chip-tool/outputs/apk/debug/app-debug.apk
 ```
 
-You can use Android Studio to edit the Android CHIPTool app itself, but you will
-not be able to edit CHIP Android code from `src/controller/java`, or other CHIP
-C++ code within Android Studio.
+You can use Android Studio to edit the Android CHIPTool app itself and run it
+after build_examples.py, but you will not be able to edit CHIP Android code from
+`src/controller/java`, or other CHIP C++ code within Android Studio.
 
 <a name="building-studio"></a>
 
-## Building Android CHIPTool from Android Studio
+### From Android Studio
 
 This option allows Android Studio to build the core CHIP code from source, which
 allows us to directly edit core CHIP code in-IDE.
@@ -113,14 +118,15 @@ allows us to directly edit core CHIP code in-IDE.
 
     See the table above for other values of `TARGET_CPU`.
 
-2. Modify the `matterBuildSrcDir` variable in
+2. Modify the `matterSdkSourceBuild` variable to true and `matterBuildSrcDir`
+   point to the appropriate output directory (e.g.
+   `../../../../out/android_arm64`) in
    [src/android/CHIPTool/gradle.properties](https://github.com/project-chip/connectedhomeip/blob/master/src/android/CHIPTool/gradle.properties)
-   to point to the appropriate output directory (e.g. `out/android_arm64`).
 
-3. Open the project in Android Studio and run **Sync Project with Gradle
+3) Open the project in Android Studio and run **Sync Project with Gradle
    Files**.
 
-4. Use one of the following options to build an Android package:
+4) Use one of the following options to build an Android package:
 
     - Click **Make Project** in Android Studio.
     - Run the following command in the command line:
@@ -142,3 +148,21 @@ or
 ```shell
 (cd src/android/CHIPTool && ./gradlew installDebug)
 ```
+
+## Building Android CHIPTest
+
+### From scripts
+
+The steps are similar with [CHIPTool scripts build](#chiptool_scripts_build)
+
+```shell
+./scripts/build/build_examples.py --target android-arm64-chip-test build
+```
+
+You can modify the `matterUTestLib` variable to the test lib in
+[src/android/CHIPTest/gradle.properties](https://github.com/project-chip/connectedhomeip/blob/master/src/android/CHIPTest/gradle.properties)
+to change target to test.
+
+### From Android Studio
+
+Not supported yet
diff --git a/scripts/build/build/targets.py b/scripts/build/build/targets.py
@@ -14,7 +14,7 @@
 
 import os
 
-from builders.android import AndroidBoard, AndroidBuilder
+from builders.android import AndroidBoard, AndroidApp, AndroidBuilder
 from builders.efr32 import Efr32Builder, Efr32App, Efr32Board
 from builders.esp32 import Esp32Builder, Esp32Board, Esp32App
 from builders.host import HostBuilder, HostApp, HostBoard
@@ -129,10 +129,12 @@ def NrfTargets():
 def AndroidTargets():
     target = Target('android', AndroidBuilder)
 
-    yield target.Extend('arm-chip-tool', board=AndroidBoard.ARM)
-    yield target.Extend('arm64-chip-tool', board=AndroidBoard.ARM64)
-    yield target.Extend('x64-chip-tool', board=AndroidBoard.X64)
-    yield target.Extend('x86-chip-tool', board=AndroidBoard.X86)
+    yield target.Extend('arm-chip-tool', board=AndroidBoard.ARM, app=AndroidApp.CHIP_TOOL)
+    yield target.Extend('arm64-chip-tool', board=AndroidBoard.ARM64, app=AndroidApp.CHIP_TOOL)
+    yield target.Extend('x64-chip-tool', board=AndroidBoard.X64, app=AndroidApp.CHIP_TOOL)
+    yield target.Extend('x86-chip-tool', board=AndroidBoard.X86, app=AndroidApp.CHIP_TOOL)
+    yield target.Extend('androidstudio-chip-tool', board=AndroidBoard.AndroidStudio, app=AndroidApp.CHIP_TOOL)
+    yield target.Extend('arm64-chip-test', board=AndroidBoard.ARM64, app=AndroidApp.CHIP_TEST)
 
 
 ALL = []

diff --git a/scripts/build/builders/android.py b/scripts/build/builders/android.py
@@ -25,6 +25,7 @@ class AndroidBoard(Enum):
     ARM64 = auto()
     X64 = auto()
     X86 = auto()
+    AndroidStudio = auto()
 
     def TargetCpuName(self):
         if self == AndroidBoard.ARM:
@@ -35,6 +36,8 @@ def TargetCpuName(self):
             return 'x64'
         elif self == AndroidBoard.X86:
             return 'x86'
+        elif self == AndroidBoard.AndroidStudio:
+            return "arm64"
         else:
             raise Exception('Unknown board type: %r' % self)
 
@@ -50,12 +53,32 @@ def AbiName(self):
         else:
             raise Exception('Unknown board type: %r' % self)
 
+    def IsIde(self):
+        if self == AndroidBoard.AndroidStudio:
+            return True
+        else:
+            return False
+
+
+class AndroidApp(Enum):
+    CHIP_TOOL = auto()
+    CHIP_TEST = auto()
+
+    def AppName(self):
+        if self == AndroidApp.CHIP_TOOL:
+            return "CHIPTool"
+        elif self == AndroidApp.CHIP_TEST:
+            return "CHIPTest"
+        else:
+            raise Exception('Unknown app type: %r' % self)
+
 
 class AndroidBuilder(Builder):
 
-    def __init__(self, root, runner, board: AndroidBoard):
+    def __init__(self, root, runner, board: AndroidBoard, app: AndroidApp):
         super(AndroidBuilder, self).__init__(root, runner)
         self.board = board
+        self.app = app
 
     def validate_build_environment(self):
         for k in ['ANDROID_NDK_HOME', 'ANDROID_HOME']:
@@ -112,10 +135,14 @@ def generate(self):
                 for key, value in gn_args.items()
             ]))
 
-            self._Execute([
-                'gn', 'gen', '--check', '--fail-on-unused-args', self.output_dir, args
-            ],
-                title='Generating ' + self.identifier)
+            gn_gen = [
+                'gn', 'gen', '--check', '--fail-on-unused-args', self.output_dir, args,
+            ]
+            if self.board.IsIde():
+                gn_gen += ['--ide=json',
+                           '--json-ide-script=//scripts/examples/gn_to_cmakelists.py']
+
+            self._Execute(gn_gen, title='Generating ' + self.identifier)
 
             self._Execute([
                 'bash', '-c',
@@ -125,76 +152,97 @@ def generate(self):
                 title='Accepting NDK licenses')
 
     def _build(self):
-        self._Execute(['ninja', '-C', self.output_dir],
-                      title='Building JNI ' + self.identifier)
-
-        # JNILibs will be copied as long as they reside in src/main/jniLibs/ABI:
-        #    https://developer.android.com/studio/projects/gradle-external-native-builds#jniLibs
-        # to avoid redefined in IDE mode, copy to another place and add that path in build.gradle
-
-        # We do NOT use python builtins for copy, so that the 'execution commands' are available
-        # when using dry run.
-        jnilibs_dir = os.path.join(
-            self.root, 'src/android/CHIPTool/app/libs/jniLibs', self.board.AbiName())
-        libs_dir = os.path.join(self.root, 'src/android/CHIPTool/app/libs')
-        self._Execute(['mkdir', '-p', jnilibs_dir],
-                      title='Prepare Native libs ' + self.identifier)
-
-        # TODO: Runtime dependencies should be computed by the build system rather than hardcoded
-        # GN supports getting these dependencies like:
-        #   gn desc out/android-x64-chip_tool/ //src/controller/java runtime_deps
-        #   gn desc out/android-x64-chip_tool/ //src/setup_payload/java runtime_deps
-        # However  this assumes that the output folder has been populated, which will not be
-        # the case for `dry-run` executions. Hence this harcoding here.
-        #
-        #   If we unify the JNI libraries, libc++_shared.so may not be needed anymore, which could
-        # be another path of resolving this inconsistency.
-        for libName in ['libSetupPayloadParser.so', 'libCHIPController.so', 'libc++_shared.so']:
-            self._Execute(['cp', os.path.join(self.output_dir, 'lib', 'jni', self.board.AbiName(
-            ), libName), os.path.join(jnilibs_dir, libName)])
-
-        jars = {
-            'CHIPController.jar': 'src/controller/java/CHIPController.jar',
-            'SetupPayloadParser.jar': 'src/setup_payload/java/SetupPayloadParser.jar',
-            'AndroidPlatform.jar': 'src/platform/android/AndroidPlatform.jar'
-
-        }
-        for jarName in jars.keys():
-            self._Execute(['cp', os.path.join(
-                self.output_dir, 'lib', jars[jarName]), os.path.join(libs_dir, jarName)])
-
-        # App compilation
-        self._Execute([
-            '%s/src/android/CHIPTool/gradlew' % self.root, '-p',
-            '%s/src/android/CHIPTool' % self.root,
-            '-PbuildDir=%s' % self.output_dir, 'assembleDebug'
-        ],
-            title='Building APP ' + self.identifier)
+        if self.board.IsIde():
+            # App compilation IDE
+            # TODO: Android Gradle with module and -PbuildDir= will caused issue, remove -PbuildDir=
+            self._Execute([
+                '%s/src/android/%s/gradlew' % (self.root,
+                                               self.app.AppName()), '-p',
+                '%s/src/android/%s' % (self.root, self.app.AppName()),
+                '-PmatterBuildSrcDir=%s' % self.output_dir,
+                '-PmatterSdkSourceBuild=true',
+                'assembleDebug'
+            ],
+                title='Building APP ' + self.identifier)
+        else:
+            self._Execute(['ninja', '-C', self.output_dir],
+                          title='Building JNI ' + self.identifier)
+
+            # JNILibs will be copied as long as they reside in src/main/jniLibs/ABI:
+            #    https://developer.android.com/studio/projects/gradle-external-native-builds#jniLibs
+            # to avoid redefined in IDE mode, copy to another place and add that path in build.gradle
+
+            # We do NOT use python builtins for copy, so that the 'execution commands' are available
+            # when using dry run.
+            jnilibs_dir = os.path.join(
+                self.root, 'src/android/', self.app.AppName(), 'app/libs/jniLibs', self.board.AbiName())
+            libs_dir = os.path.join(
+                self.root, 'src/android/', self.app.AppName(), 'app/libs')
+            self._Execute(['mkdir', '-p', jnilibs_dir],
+                          title='Prepare Native libs ' + self.identifier)
+
+            # TODO: Runtime dependencies should be computed by the build system rather than hardcoded
+            # GN supports getting these dependencies like:
+            #   gn desc out/android-x64-chip_tool/ //src/controller/java runtime_deps
+            #   gn desc out/android-x64-chip_tool/ //src/setup_payload/java runtime_deps
+            # However  this assumes that the output folder has been populated, which will not be
+            # the case for `dry-run` executions. Hence this harcoding here.
+            #
+            #   If we unify the JNI libraries, libc++_shared.so may not be needed anymore, which could
+            # be another path of resolving this inconsistency.
+            for libName in ['libSetupPayloadParser.so', 'libCHIPController.so', 'libc++_shared.so']:
+                self._Execute(['cp', os.path.join(self.output_dir, 'lib', 'jni', self.board.AbiName(
+                ), libName), os.path.join(jnilibs_dir, libName)])
+
+            jars = {
+                'CHIPController.jar': 'src/controller/java/CHIPController.jar',
+                'SetupPayloadParser.jar': 'src/setup_payload/java/SetupPayloadParser.jar',
+                'AndroidPlatform.jar': 'src/platform/android/AndroidPlatform.jar'
+
+            }
+            for jarName in jars.keys():
+                self._Execute(['cp', os.path.join(
+                    self.output_dir, 'lib', jars[jarName]), os.path.join(libs_dir, jarName)])
+
+            # App compilation
+            self._Execute([
+                '%s/src/android/%s/gradlew' % (self.root,
+                                               self.app.AppName()), '-p',
+                '%s/src/android/%s' % (self.root,
+                                       self.app.AppName()),
+                '-PmatterBuildSrcDir=%s' % self.output_dir,
+                '-PmatterSdkSourceBuild=false',
+                '-PbuildDir=%s' % self.output_dir, 'assembleDebug'
+            ],
+                title='Building APP ' + self.identifier)
 
     def build_outputs(self):
         outputs = {
-            'CHIPController.jar':
-                os.path.join(self.output_dir, 'lib',
-                             'src/controller/java/CHIPController.jar'),
-            'AndroidPlatform.jar':
-                os.path.join(self.output_dir, 'lib',
-                             'src/platform/android/AndroidPlatform.jar'),
-            'SetupPayloadParser.jar':
-                os.path.join(self.output_dir, 'lib',
-                             'src/setup_payload/java/SetupPayloadParser.jar'),
-            'ChipTool-debug.apk':
+            self.app.AppName() + '-debug.apk':
                 os.path.join(self.output_dir, 'outputs', 'apk', 'debug',
                              'app-debug.apk'),
-
-            'jni/%s/libSetupPayloadParser.so' % self.board.AbiName():
-                os.path.join(self.output_dir, 'lib', 'jni',
-                             self.board.AbiName(), 'libSetupPayloadParser.so'),
-            'jni/%s/libCHIPController.so' % self.board.AbiName():
-                os.path.join(self.output_dir, 'lib', 'jni',
-                             self.board.AbiName(), 'libCHIPController.so'),
-            'jni/%s/libc++_shared.so' % self.board.AbiName():
-                os.path.join(self.output_dir, 'lib', 'jni',
-                             self.board.AbiName(), 'libc++_shared.so'),
         }
+        if not self.board.IsIde():
+            outputs.update({
+                'CHIPController.jar':
+                    os.path.join(self.output_dir, 'lib',
+                                 'src/controller/java/CHIPController.jar'),
+                'AndroidPlatform.jar':
+                    os.path.join(self.output_dir, 'lib',
+                                 'src/platform/android/AndroidPlatform.jar'),
+                'SetupPayloadParser.jar':
+                    os.path.join(self.output_dir, 'lib',
+                                 'src/setup_payload/java/SetupPayloadParser.jar'),
+
+                'jni/%s/libSetupPayloadParser.so' % self.board.AbiName():
+                    os.path.join(self.output_dir, 'lib', 'jni',
+                                 self.board.AbiName(), 'libSetupPayloadParser.so'),
+                'jni/%s/libCHIPController.so' % self.board.AbiName():
+                    os.path.join(self.output_dir, 'lib', 'jni',
+                                 self.board.AbiName(), 'libCHIPController.so'),
+                'jni/%s/libc++_shared.so' % self.board.AbiName():
+                    os.path.join(self.output_dir, 'lib', 'jni',
+                                 self.board.AbiName(), 'libc++_shared.so'),
+            })
 
         return outputs