Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: add new bash scripts for test artifacts generation #1095

Merged
merged 12 commits into from
Sep 15, 2020
11 changes: 11 additions & 0 deletions .env
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/usr/bin/env bash

FLANK_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]-$0}")" && pwd)"
TEST_PROJECTS="$FLANK_ROOT/test_projects"
FLANK_FIXTURES_TMP="$FLANK_ROOT/test_runner/src/test/kotlin/ftl/fixtures/tmp"

. "$TEST_PROJECTS/ops.sh"

function bash_debug() {
set -euxo pipefail
}
29 changes: 23 additions & 6 deletions docs/test_artifacts.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,25 @@
# Test artifacts
Test artifacts are necessary for CI and local testing.

### Generating test artifacts
```bash
cd ... # your flank repositry root
source .env
update_test_artifacts android ios # [ android | go | ios | all ]
```
`update_test_artifacts` function will generate artifacts and copy them to `test_runner/src/test/kotlin/ftl/fixtures/tmp/` directory.

### Test projects
All source code for test artifacts is located in [test_projects](../test_projects/) directory.

Each test project can contain own `ops.sh` script used building and coping generated artifacts.

`update_test_artifacts` function is located in [test_projects/ops.sh](../test_projects/ops.sh) script.

# Refactor
This is proposal of test artifacts refactoring.

## Context
Test artifacts are necessary for CI and local testing.
## Context
Working on different issues sometimes requires different implementations of test artifacts that may exist in same time.

### Problem
@@ -33,9 +50,9 @@ Sometimes downloading may fail, but this not aborting test run.
For remote storage options see [host_binaries_solutions_comparison.md](./host_binaries_solutions_comparison.md)

## Solution proposal
1. Move source code of test artifacts to one repo.
We should consider flank repo because test_app is already there or dedicated repo linked to flank as git submodule.
2. Prepare missing build scripts and adjust existing.
We need dedicated scripts for each artifact group (iOS, Android or different types of each) and one to execute all of them.
1. ~~Move source code of test artifacts to one repo.
We should consider flank repo because test_app is already there or dedicated repo linked to flank as git submodule.~~
2. ~~Prepare missing build scripts and adjust existing.
We need dedicated scripts for each artifact group (iOS, Android or different types of each) and one to execute all of them.~~
3. Choose the best solution for hosting remote artifacts. See `Technical research` section.
4. Integrate artifacts synchronization with remote storage
9 changes: 7 additions & 2 deletions flank-scripts/bash/flankScripts
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
#!/usr/bin/env bash

DIR=`dirname "$BASH_SOURCE"`
dir=$(dirname "$BASH_SOURCE")
scriptsJar="$dir/flankScripts.jar"

/usr/bin/env java -jar "$DIR/flankScripts.jar" "$@"
if [ ! -f "$scriptsJar" ]; then
"$dir/buildFlankScripts.sh"
fi

/usr/bin/env java -jar "$scriptsJar" "$@"
4 changes: 3 additions & 1 deletion test_projects/android/bash/generate-multi-modules-apks.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
#!/usr/bin/env bash

echo "DEPRECATED!!! use functions from test_projects/android/ops.sh"

set -euxo pipefail

PROJECT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )/../../.." >/dev/null 2>&1 && pwd )"
@@ -30,4 +32,4 @@ MULTI_MODULES_DIR="$PROJECT_DIR/test_runner/src/test/kotlin/ftl/fixtures/tmp/apk
:multi-modules:testModule20:assembleAndroidTest \

mkdir -p "$MULTI_MODULES_DIR"
find multi-modules -type f -name "*.apk" -exec cp {} "$MULTI_MODULES_DIR" \;
find multi-modules -type f -name "*.apk" -exec cp {} "$MULTI_MODULES_DIR" \;
2 changes: 2 additions & 0 deletions test_projects/android/bash/generate_apks.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
#!/usr/bin/env bash

echo "DEPRECATED!!! use functions from test_projects/android/ops.sh"

set -euxo pipefail

APK_OUTPUT=$1
2 changes: 2 additions & 0 deletions test_projects/android/bash/generate_duplicated_names_apks.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
#!/usr/bin/env bash

echo "DEPRECATED!!! use functions from test_projects/android/ops.sh"

set -euxo pipefail

DIR=`dirname "$BASH_SOURCE"`
116 changes: 116 additions & 0 deletions test_projects/android/ops.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
#!/usr/bin/env bash

function base_app_apk() {
local dir=$TEST_PROJECTS_ANDROID
local outputDir="$FLANK_FIXTURES_TMP/apk"

for arg in "$@"; do case "$arg" in

'--generate' | '-g')
"$dir/gradlew" -p "$dir" app:assemble
;;

'--copy' | '-c')
local apkIn="$dir/app/build/outputs/apk/singleSuccess/debug/app-single-success-debug.apk"
local apkOut="$outputDir/app-debug.apk"

mkdir -p "$outputDir"
cp "$apkIn" "$apkOut"
;;

esac done
}

# depends on base_app_apk
function base_test_apks() {
local dir=$TEST_PROJECTS_ANDROID

for arg in "$@"; do case "$arg" in

'--generate' | '-g')
"$dir/gradlew" -p "$dir" app:assembleAndroidTest
;;

'--copy' | '-c')
local outputDir="$FLANK_FIXTURES_TMP/apk"

mkdir -p "$outputDir"
cp "$dir"/app/build/outputs/apk/androidTest/**/debug/*.apk "$outputDir/"
;;

esac done
}

# depends on base_app_apk
function duplicated_names_apks() {
local dir=$TEST_PROJECTS_ANDROID

for arg in "$@"; do case "$arg" in

'--generate' | '-g')
"$dir/gradlew" -p "$dir" \
dir0:testModule:assembleAndroidTest \
dir1:testModule:assembleAndroidTest \
dir2:testModule:assembleAndroidTest \
dir3:testModule:assembleAndroidTest
;;

'--copy' | '-c')
local outputDir="$FLANK_FIXTURES_TMP/apk"
local testIn="$dir/app/build/outputs/apk/androidTest/**/debug/*.apk"

mkdir -p "$outputDir"
local dir=$(dirname "${BASH_SOURCE[0]-$0}")
local outputDir="$FLANK_FIXTURES_TMP/apk/duplicated_names/"

for index in 0 1 2 3; do
moduleName="dir$index"
apkDir="$outputDir/$moduleName/"
mkdir -p "$apkDir"
cp "$dir/$moduleName"/testModule/build/outputs/apk/**/debug/*.apk "$apkDir"
done
;;

esac done
}

function multi_module_apks() {
local dir=$TEST_PROJECTS_ANDROID
local outputDir="$FLANK_FIXTURES_TMP/apk/multi-modules/"

for arg in "$@"; do case "$arg" in

'--generate' | '-g')
"$dir/gradlew" -p "$dir" \
:multi-modules:multiapp:assemble \
:multi-modules:testModule1:assembleAndroidTest \
:multi-modules:testModule2:assembleAndroidTest \
:multi-modules:testModule3:assembleAndroidTest \
:multi-modules:testModule4:assembleAndroidTest \
:multi-modules:testModule5:assembleAndroidTest \
:multi-modules:testModule6:assembleAndroidTest \
:multi-modules:testModule7:assembleAndroidTest \
:multi-modules:testModule8:assembleAndroidTest \
:multi-modules:testModule9:assembleAndroidTest \
:multi-modules:testModule10:assembleAndroidTest \
:multi-modules:testModule11:assembleAndroidTest \
:multi-modules:testModule12:assembleAndroidTest \
:multi-modules:testModule13:assembleAndroidTest \
:multi-modules:testModule14:assembleAndroidTest \
:multi-modules:testModule15:assembleAndroidTest \
:multi-modules:testModule16:assembleAndroidTest \
:multi-modules:testModule17:assembleAndroidTest \
:multi-modules:testModule18:assembleAndroidTest \
:multi-modules:testModule19:assembleAndroidTest \
:multi-modules:testModule20:assembleAndroidTest
;;

'--copy' | '-c')
mkdir -p "$outputDir"
find "$dir/multi-modules" -type f -name "*.apk" -exec cp {} "$outputDir" \;
;;

esac done
}

echo "Android test projects ops loaded"
Original file line number Diff line number Diff line change
@@ -283,18 +283,18 @@
TargetAttributes = {
2CB7314C1C29E54A00CF35C1 = {
CreatedOnToolsVersion = 7.2;
DevelopmentTeam = H86XEK7ZTM;
DevelopmentTeam = AD2V26JBWL;
jan-goral marked this conversation as resolved.
Show resolved Hide resolved
LastSwiftMigration = 0800;
TestTargetID = 5F5A53781ADE67D500F81DF0;
};
5F5A53781ADE67D500F81DF0 = {
CreatedOnToolsVersion = 6.3;
DevelopmentTeam = H86XEK7ZTM;
DevelopmentTeam = AD2V26JBWL;
LastSwiftMigration = 0800;
};
5FDE05571B0DAA090037B82F = {
CreatedOnToolsVersion = 6.3.2;
DevelopmentTeam = H86XEK7ZTM;
DevelopmentTeam = AD2V26JBWL;
TestTargetID = 5F5A53781ADE67D500F81DF0;
};
};
@@ -485,12 +485,12 @@
CODE_SIGN_IDENTITY = "iPhone Developer";
DEBUG_INFORMATION_FORMAT = dwarf;
DEFINES_MODULE = NO;
DEVELOPMENT_TEAM = H86XEK7ZTM;
DEVELOPMENT_TEAM = AD2V26JBWL;
ENABLE_TESTABILITY = YES;
INFOPLIST_FILE = EarlGreyExampleSwiftTests/Info.plist;
IPHONEOS_DEPLOYMENT_TARGET = 9.2;
LD_RUNPATH_SEARCH_PATHS = "$(inherited) @executable_path/Frameworks @loader_path/Frameworks";
PRODUCT_BUNDLE_IDENTIFIER = com.google.earlgrey.samples.EarlGreySwiftTests;
PRODUCT_BUNDLE_IDENTIFIER = io.gogoapps.earlgrey.samples.EarlGreySwiftTests;
PRODUCT_MODULE_NAME = EarlGreyExampleTestsSwift;
PRODUCT_NAME = "$(TARGET_NAME)";
PROVISIONING_PROFILE = "";
@@ -509,11 +509,11 @@
COPY_PHASE_STRIP = NO;
DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym";
DEFINES_MODULE = NO;
DEVELOPMENT_TEAM = H86XEK7ZTM;
DEVELOPMENT_TEAM = AD2V26JBWL;
INFOPLIST_FILE = EarlGreyExampleSwiftTests/Info.plist;
IPHONEOS_DEPLOYMENT_TARGET = 9.2;
LD_RUNPATH_SEARCH_PATHS = "$(inherited) @executable_path/Frameworks @loader_path/Frameworks";
PRODUCT_BUNDLE_IDENTIFIER = com.google.earlgrey.samples.EarlGreySwiftTests;
PRODUCT_BUNDLE_IDENTIFIER = io.gogoapps.earlgrey.samples.EarlGreySwiftTests;
PRODUCT_MODULE_NAME = EarlGreyExampleTestsSwift;
PRODUCT_NAME = "$(TARGET_NAME)";
PROVISIONING_PROFILE = "";
@@ -619,12 +619,12 @@
CODE_SIGN_IDENTITY = "iPhone Developer";
"CODE_SIGN_IDENTITY[sdk=iphoneos*]" = "iPhone Developer";
DEFINES_MODULE = YES;
DEVELOPMENT_TEAM = H86XEK7ZTM;
DEVELOPMENT_TEAM = AD2V26JBWL;
EMBEDDED_CONTENT_CONTAINS_SWIFT = YES;
INFOPLIST_FILE = "$(SRCROOT)/EarlGreyExample/Info.plist";
IPHONEOS_DEPLOYMENT_TARGET = 8.0;
LD_RUNPATH_SEARCH_PATHS = "$(inherited) @executable_path/Frameworks";
PRODUCT_BUNDLE_IDENTIFIER = com.google.earlgrey.samples.EarlGrey3;
PRODUCT_BUNDLE_IDENTIFIER = io.gogoapps.earlgrey.samples.EarlGrey;
PRODUCT_MODULE_NAME = EarlGreyExampleSwift;
PRODUCT_NAME = "$(TARGET_NAME)";
SWIFT_INSTALL_OBJC_HEADER = NO;
@@ -642,12 +642,12 @@
CODE_SIGN_IDENTITY = "iPhone Developer";
"CODE_SIGN_IDENTITY[sdk=iphoneos*]" = "iPhone Developer";
DEFINES_MODULE = YES;
DEVELOPMENT_TEAM = H86XEK7ZTM;
DEVELOPMENT_TEAM = AD2V26JBWL;
EMBEDDED_CONTENT_CONTAINS_SWIFT = YES;
INFOPLIST_FILE = "$(SRCROOT)/EarlGreyExample/Info.plist";
IPHONEOS_DEPLOYMENT_TARGET = 8.0;
LD_RUNPATH_SEARCH_PATHS = "$(inherited) @executable_path/Frameworks";
PRODUCT_BUNDLE_IDENTIFIER = com.google.earlgrey.samples.EarlGrey3;
PRODUCT_BUNDLE_IDENTIFIER = io.gogoapps.earlgrey.samples.EarlGrey;
PRODUCT_MODULE_NAME = EarlGreyExampleSwift;
PRODUCT_NAME = "$(TARGET_NAME)";
SWIFT_INSTALL_OBJC_HEADER = NO;
@@ -666,7 +666,7 @@
"CODE_SIGN_IDENTITY[sdk=iphoneos*]" = "iPhone Developer";
DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym";
DEFINES_MODULE = NO;
DEVELOPMENT_TEAM = H86XEK7ZTM;
DEVELOPMENT_TEAM = AD2V26JBWL;
EMBEDDED_CONTENT_CONTAINS_SWIFT = YES;
FRAMEWORK_SEARCH_PATHS = "$(inherited)";
GCC_PREPROCESSOR_DEFINITIONS = (
@@ -682,7 +682,7 @@
INFOPLIST_FILE = EarlGreyExample/Info.plist;
IPHONEOS_DEPLOYMENT_TARGET = 8.3;
LD_RUNPATH_SEARCH_PATHS = "$(inherited) @executable_path/Frameworks @loader_path/Frameworks";
PRODUCT_BUNDLE_IDENTIFIER = com.google.earlgrey.samples.EarlGreyTests;
PRODUCT_BUNDLE_IDENTIFIER = io.gogoapps.earlgrey.samples.EarlGreyTests;
PRODUCT_MODULE_NAME = EarlGreyExampleTests;
PRODUCT_NAME = "$(TARGET_NAME)";
SWIFT_OBJC_BRIDGING_HEADER = "";
@@ -704,7 +704,7 @@
COPY_PHASE_STRIP = NO;
DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym";
DEFINES_MODULE = NO;
DEVELOPMENT_TEAM = H86XEK7ZTM;
DEVELOPMENT_TEAM = AD2V26JBWL;
EMBEDDED_CONTENT_CONTAINS_SWIFT = YES;
FRAMEWORK_SEARCH_PATHS = "$(inherited)";
HEADER_SEARCH_PATHS = (
@@ -716,7 +716,7 @@
INFOPLIST_FILE = EarlGreyExample/Info.plist;
IPHONEOS_DEPLOYMENT_TARGET = 8.3;
LD_RUNPATH_SEARCH_PATHS = "$(inherited) @executable_path/Frameworks @loader_path/Frameworks";
PRODUCT_BUNDLE_IDENTIFIER = com.google.earlgrey.samples.EarlGreyTests;
PRODUCT_BUNDLE_IDENTIFIER = io.gogoapps.earlgrey.samples.EarlGreyTests;
PRODUCT_MODULE_NAME = EarlGreyExampleTests;
PRODUCT_NAME = "$(TARGET_NAME)";
SWIFT_OBJC_BRIDGING_HEADER = "";
Original file line number Diff line number Diff line change
@@ -11,6 +11,13 @@
selectedDebuggerIdentifier = "Xcode.DebuggerFoundation.Debugger.LLDB"
selectedLauncherIdentifier = "Xcode.DebuggerFoundation.Launcher.LLDB"
shouldUseLaunchSchemeArgsEnv = "NO">
<EnvironmentVariables>
<EnvironmentVariable
key = "DYLD_INSERT_LIBRARIES"
value = "@executable_path/EarlGrey.framework/EarlGrey"
isEnabled = "YES">
</EnvironmentVariable>
</EnvironmentVariables>
<Testables>
<TestableReference
skipped = "NO">
@@ -23,15 +30,6 @@
</BuildableReference>
</TestableReference>
</Testables>
<AdditionalOptions>
</AdditionalOptions>
<EnvironmentVariables>
<EnvironmentVariable
key = "DYLD_INSERT_LIBRARIES"
value = "@executable_path/EarlGrey.framework/EarlGrey"
isEnabled = "YES">
</EnvironmentVariable>
</EnvironmentVariables>
</TestAction>
<LaunchAction
buildConfiguration = "Debug"
@@ -43,8 +41,6 @@
debugDocumentVersioning = "YES"
debugServiceExtension = "internal"
allowLocationSimulation = "YES">
<AdditionalOptions>
</AdditionalOptions>
</LaunchAction>
jan-goral marked this conversation as resolved.
Show resolved Hide resolved
<ProfileAction
buildConfiguration = "Release"
71 changes: 71 additions & 0 deletions test_projects/ios/EarlGreyExample/ops.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
#!/usr/bin/env bash

EARL_GREY_EXAMPLE="$TEST_PROJECTS_IOS/EarlGreyExample"

function setup_ios_env() {
if ! [ -x "$(command -v xcpretty)" ]; then
gem install cocoapods -v 1.9.3
fi
(cd "$EARL_GREY_EXAMPLE" && pod install)
}

function install_xcpretty() {
if ! [ -x "$(command -v xcpretty)" ]; then
gem install xcpretty
jan-goral marked this conversation as resolved.
Show resolved Hide resolved
fi
}

function universal_framework() {
"$EARL_GREY_EXAMPLE/universal_framework.sh"
}

function earl_grey_example() {
local dir=$EARL_GREY_EXAMPLE
local buildDir="$dir/build"

for arg in "$@"; do case "$arg" in

'--generate' | '-g')

install_xcpretty

rm -rf "$buildDir"

xcodebuild build-for-testing \
-allowProvisioningUpdates \
-workspace "$dir/EarlGreyExample.xcworkspace" \
-scheme "EarlGreyExampleSwiftTests" \
-derivedDataPath "$buildDir" \
-sdk iphoneos |
xcpretty

xcodebuild build-for-testing \
-allowProvisioningUpdates \
-workspace "$dir/EarlGreyExample.xcworkspace" \
-scheme "EarlGreyExampleTests" \
-derivedDataPath "$buildDir" \
-sdk iphoneos |
xcpretty
;;

'--copy' | '-c')

local productsDir="$dir/build/Build/Products"

cp -Rf "$productsDir"/*-iphoneos "$FLANK_FIXTURES_TMP/"

cp "$productsDir"/*.xctestrun "$FLANK_FIXTURES_TMP/"

cp \
"$productsDir/Debug-iphoneos/EarlGreyExampleSwift.app/PlugIns/EarlGreyExampleTests.xctest/EarlGreyExampleTests" \
"$FLANK_FIXTURES_TMP/objc/"

cp \
"$productsDir/Debug-iphoneos/EarlGreyExampleSwift.app/PlugIns/EarlGreyExampleSwiftTests.xctest/EarlGreyExampleSwiftTests" \
"$FLANK_FIXTURES_TMP/swift/"
;;

esac done
}

echo "iOS test projects ops loaded"
32 changes: 32 additions & 0 deletions test_projects/ops.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/usr/bin/env bash

TEST_PROJECTS_ANDROID="$TEST_PROJECTS/android"
TEST_PROJECTS_IOS="$TEST_PROJECTS/ios"

. "$TEST_PROJECTS_ANDROID/ops.sh"
. "$TEST_PROJECTS_IOS/EarlGreyExample/ops.sh"

function update_test_artifacts() {

for arg in "$@"; do case "$arg" in

android)
base_app_apk --generate --copy
base_test_apks --generate --copy
;;

ios)
setup_ios_env
earl_grey_example --generate --copy
;;

go)
cp -R "$FLANK_ROOT/test_projects/gohello" "$FLANK_FIXTURES_TMP/"
;;

all)
update_test_artifacts android ios go
;;

esac done
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
gcloud:
app: ./src/test/kotlin/ftl/fixtures/tmp/apk/duplicated_names/app-debug.apk
app: ./src/test/kotlin/ftl/fixtures/tmp/apk/app-debug.apk
test: ./src/test/kotlin/ftl/fixtures/tmp/apk/duplicated_names/dir0/testModule-debug-androidTest.apk
use-orchestrator: false
flank: