Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Generate tasks diagram #2102

Merged
merged 4 commits into from
Jul 26, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 6 additions & 13 deletions corellium/domain/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,22 +9,15 @@ This module is specifying public API and internal implementation of Flank-Corell
* Public API - files inside [flank.corellium.domain](./src/main/kotlin/flank/corellium/domain)
* Internal functions - nested packages inside [flank.corellium.domain](./src/main/kotlin/flank/corellium/domain)

## Design
## Execution

### Stateful execution
Execution can be represented as a graph of tasks relations without cycles.

Following specification is suitable for complicated long-running use-cases, when becomes convenient to split execution into smaller atomic chunks of work.

#### Definition:
#### Version from master branch:

* `Execution` is process which is creating initial `State`, and is running the set of `steps` on it in specific `Context`.
* `Execution` can pass `Context` to `Step` if needed.
* `Step` is a suspendable operation that is receiving `State` as the only argument.
* `Step` must return received or new `State`.
* `Step` can generate side effects.
* `State` is a structure used for sharing data between preceding and following `steps`.
* `Context` is providing arguments and functions for `step`.
![TestAndroid.execute graph](http://www.plantuml.com/plantuml/proxy?cache=no&fmt=svg&src=https://raw.githubusercontent.com/Flank/flank/master/corellium/domain/TestAndroid-execute.puml)

#### Utility:
### New version draft:

* [`Transform.kt`](src/main/kotlin/flank/corellium/domain/util/Transform.kt)
![TestAndroid.execute graph](http://www.plantuml.com/plantuml/proxy?cache=no&fmt=svg&src=https://raw.githubusercontent.com/Flank/flank/2083_Add_module_tool-execution-parallel-plantuml/corellium/domain/TestAndroid-execute.puml)
44 changes: 44 additions & 0 deletions corellium/domain/TestAndroid-execute.puml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
@startuml

skinparam componentStyle rectangle

note as N #ffffff
* Brighter tasks are required by the darker tasks.
* The brightness means how fast the task will start.
* White tasks are starting first.
end note

[Authorize] #fbfbfb
[OutputDir] #fbfbfb
[ParseApkInfo] #fbfbfb
[ParseTestCases] #fbfbfb
[LoadPreviousDurations] #dfdfdf
[PrepareShards] #c3c3c3
[DumpShards] #a7a7a7
[InvokeDevices] #a7a7a7
[InstallApks] #8b8b8b
[ExecuteTests] #6f6f6f
[GenerateReport] #535353
[CompleteTests] #373737

[DumpShards] --> [PrepareShards]
[DumpShards] --> [OutputDir]
[ExecuteTests] --> [PrepareShards]
[ExecuteTests] --> [ParseApkInfo]
[ExecuteTests] --> [Authorize]
[ExecuteTests] --> [InvokeDevices]
[ExecuteTests] --> [InstallApks]
[CompleteTests] --> [GenerateReport]
[CompleteTests] --> [DumpShards]
[GenerateReport] --> [ExecuteTests]
[GenerateReport] --> [OutputDir]
[InstallApks] --> [Authorize]
[InstallApks] --> [PrepareShards]
[InstallApks] --> [InvokeDevices]
[InvokeDevices] --> [Authorize]
[InvokeDevices] --> [PrepareShards]
[LoadPreviousDurations] --> [ParseTestCases]
[PrepareShards] --> [ParseTestCases]
[PrepareShards] --> [LoadPreviousDurations]

@enduml
1 change: 1 addition & 0 deletions corellium/domain/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ dependencies {

testImplementation(Dependencies.JUNIT)
testImplementation(project(":corellium:adapter"))
testImplementation(project(":tool:execution:parallel:plantuml"))
}

tasks.test {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
package flank.corellium.domain

import flank.exection.parallel.plantuml.generatePlanUml
import org.junit.Test

class AndroidTestDiagram {

@Test
fun generate() {
TestAndroid.run { generatePlanUml(execute - context.validate) }
}
}
1 change: 1 addition & 0 deletions settings.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ include(
":tool:log",
":tool:log:format",
":tool:execution:parallel",
":tool:execution:parallel:plantuml",
":tool:execution:synchronized",
":tool:analytics",
":tool:analytics:mixpanel",
Expand Down
17 changes: 17 additions & 0 deletions tool/execution/parallel/plantuml/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile

plugins {
kotlin(Plugins.Kotlin.PLUGIN_JVM)
}

repositories {
mavenCentral()
}

tasks.withType<KotlinCompile> { kotlinOptions.jvmTarget = "1.8" }

dependencies {
api(project(":tool:execution:parallel"))
implementation(Dependencies.KOTLIN_COROUTINES_CORE)
testImplementation(Dependencies.JUNIT)
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
package flank.exection.parallel.plantuml

import flank.exection.parallel.Tasks
import flank.exection.parallel.plantuml.internal.generatePlanUmlFile
import flank.exection.parallel.plantuml.internal.generatePlantUmlString
import java.io.File

/**
* Generates plantuml file and saves on drive.
*
* @receiver Source object of execution. Used for generating filename and reducing tasks names.
* @param tasks Graph of tasks required to generate diagram.
* @param dir Optional path to directory for generated file.
*/
fun Any.generatePlanUml(
tasks: Tasks,
dir: String = ""
): File = generatePlanUmlFile(
tasks = tasks,
path = File(dir).resolve(javaClass.simpleName).absolutePath + "-execute.puml",
prefixToRemove = javaClass.name
)

/**
* Generates plantuml file and saves on drive.
*
* @param tasks Graph of tasks required to generate diagram.
* @param path Path to generated file.
* @param prefixToRemove Optional prefix to remove from each task name.
*/
fun generatePlanUml(
tasks: Tasks,
path: String,
prefixToRemove: String = "",
): File = generatePlanUmlFile(
tasks = tasks,
path = path,
prefixToRemove = prefixToRemove
)

/**
* Generates plantuml string.
*
* @receiver Source object of execution. Used for reducing tasks names.
* @param tasks Graph of tasks required to generate diagram.
*/
fun Any.generatePlantUml(
tasks: Tasks
): String = generatePlantUmlString(
tasks = tasks,
prefixToRemove = javaClass.name
)

/**
* Generates plantuml string.
*
* @param tasks Graph of tasks required to generate diagram.
* @param prefixToRemove Optional prefix to remove from each task name.
*/
fun generatePlantUml(
tasks: Tasks,
prefixToRemove: String = "",
): String = generatePlantUmlString(
tasks = tasks,
prefixToRemove = prefixToRemove
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
package flank.exection.parallel.plantuml.internal

import flank.exection.parallel.Tasks
import java.io.File

internal fun generatePlanUmlFile(
tasks: Tasks,
path: String,
prefixToRemove: String,
): File {
val plant = generatePlantUmlString(tasks, prefixToRemove)
println(plant)
return File(path).apply { writeText(plant) }
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
package flank.exection.parallel.plantuml.internal

import flank.exection.parallel.Parallel
import flank.exection.parallel.Tasks
import java.awt.Color

// =================== Internal API ===================

internal fun generatePlantUmlString(
tasks: Tasks,
prefixToRemove: String = "",
): String {
val graph: Graph = tasks.associate { task -> task.signature.type to task.signature.args }
val depth: Map<Node, Int> = graph.calculateDepth()
val maxDepth: Int = depth.values.maxOrNull() ?: 0
val colors: Colors = depth.mapValues { (_, value) -> calculateColor(maxDepth, value) }
val name = fun Any.() = javaClass.name.removePrefix("$prefixToRemove$").replace('$', '.').let { "[$it]" }

return """
@startuml

skinparam componentStyle rectangle

note as N #ffffff
* Brighter tasks are required by the darker tasks.
* The brightness means how fast the task will start.
* White tasks are starting first.
end note

${colors.printColors(name)}

${graph.printRelations(name)}

@enduml

""".trimIndent()
}

// =================== Private implementation ===================

private typealias Graph = Map<Node, Set<Node>>
private typealias Node = Parallel.Type<*>
private typealias Colors = Map<Node, String>

private fun calculateColor(maxDepth: Int, value: Int): String {
val c = (COLOR_MAX / maxDepth) * (maxDepth - value) + COLOR_OFFSET
return Integer.toHexString(Color(c, c, c).rgb).drop(2) // drop alpha
}

private const val COLOR_MAX = 200
private const val COLOR_OFFSET = 55

private fun <T> Map<T, Set<T>>.calculateDepth(): Map<T, Int> {
var jump = 0
val state = (values.flatten() - keys).toMutableList()
val remaining: MutableMap<T, Set<T>> = toMutableMap()
val depth = mutableMapOf<T, Int>()

while (depth.size < size) {
val current = remaining
.filterValues { state.containsAll(it) }.keys
.onEach { depth[it] = jump }
state += current
remaining -= current
jump++
}

return depth
}

private fun Colors.printColors(
name: Node.() -> String
) = toList().joinToString("\n") { (node, value) ->
"${node.name()} #$value"
}

private fun Graph.printRelations(
name: Node.() -> String
) = filterValues { dependencies ->
dependencies.isNotEmpty()
}.toList().joinToString("\n") { (node, dependencies) ->
dependencies.joinToString("\n") { dep ->
node.name() + " --> " + dep.name()
}
}