Improve logging and concurrency in controller

.travis.yml

@@ -33,7 +33,7 @@ install:
 - test/e2e/travis-kube-setup.sh
 - glide -h > /dev/null || go get github.com/Masterminds/glide
 # Needed for some integration tests
-- nats-streaming-server -h > /dev/null || get github.com/nats-io/nats-streaming-server
+- nats-streaming-server -h > /dev/null || go get github.com/nats-io/nats-streaming-server
 
 before_script:
 # Build

Docs/README.md

@@ -1,7 +1,7 @@
 # Fission Workflow
 
-- Overview
+- [Overview](../README.md)
 - [Terminology](./terminology.md) 
 - [System Architecture](./architecture.md)
-- Installation
-- Roadmap
+- [Installation](../INSTALL.md)
+- [Roadmap](./roadmap.md)

Docs/architecture.md

@@ -3,6 +3,7 @@
 This document describes the high-level architecture of Fission Workflows.
 
 ![System Architecture](./assets/sysarch.png)
+
 *System Architecture of Fission Workflows*
 
 ### API (Server)

Docs/controller-design.md

@@ -5,7 +5,7 @@ It makes use of the scheduler to determine the strategy towards the completion o
 It makes use of the InvocationApi, FunctionApi and WorkflowApi to perform actions.
 
 ## Control Flow
-The controller can best be described as an interruptible control loop.
+The controller can best be described as an interruptable control loop.
 It contains the following ways of managing the lifecycle of invocations.
 - Long control loop
 - Short control loop
@@ -25,7 +25,8 @@ This is meant to capture any invocations of which the notification (see notifica
 The notification resembles more or less an interrupt. 
 Whenever a relevant event occurs for an invocation, the controller receives a notification of this event.
 Note that notifications are entirely optional, and only serve to reduce the latency of a workflow invocation.
-In case of errors or an overload of notifications, the short or long control loop will pick up the invocation
+In case of errors or an overload of notifications, the short or long control loop will pick up the invocation.
+The controller is not obligated to handle a notification.
 
 
 ## Procedure

Docs/terminology.md

@@ -16,3 +16,12 @@ Throughout the system it is often also called a "function"".
 
 #### Workflow Engine
 The core part of the Workflow Management System, responsible for the scheduling and controlling of workflow.
+
+
+## Common abbreviations
+```
+wf   = workflow
+wfi  = worfklow invocation
+ctrl = controller
+expr = expression
+```

Docs/wip/expression-parser.md

@@ -0,0 +1,10 @@
+# Expression Parser
+
+TODO
+
+Contents
+- General concept
+- Use cases
+- Syntax
+- functions
+- advanced: adding custom functions

Docs/wip/instrumentation.md

@@ -0,0 +1,14 @@
+# Instrumentation
+
+This document contains description and high-level documentation on the instrumentation of the system.
+The instrumentation here is considered to encompass tracing, logging and metrics.
+
+## Logging
+
+TODO
+
+Common fields:
+- ctrl: name of the controller if applicable
+- wf: id of the workflow if applicable
+- wfi: id of the workflow invocation if applicable
+- component: name of component (controller, api, apiserver, fnenv)

hack/docker-publish.sh

@@ -25,10 +25,12 @@ NS=$1
 echo "Using tag: ${TAG} and org ${NS}"
 BUNDLE_IMAGE=${NS}/fission-workflows-bundle
 ENV_IMAGE=${NS}/workflow-env
+BUILD_ENV_IMAGE=${NS}/workflow-build-env
 
 # Check if images with tags exist
 local_image_exists ${BUNDLE_IMAGE} ${TAG}
 local_image_exists ${ENV_IMAGE} ${TAG}
+local_image_exists ${BUILD_ENV_IMAGE} ${TAG}
 
 # Publish
 read -p "Publish images with '${TAG}' to Dockerhub namespace '${NS}'? " -n 1 -r
@@ -40,4 +42,6 @@ then
     docker push ${BUNDLE_IMAGE}:${TAG}
     echo "Publishing ${ENV_IMAGE}:${TAG}"
     docker push ${ENV_IMAGE}:${TAG}
+    echo "publishing ${BUILD_ENV_IMAGE}:${TAG}"
+    docker push ${BUILD_ENV_IMAGE}:${TAG}
 fi

pkg/controller/expr/expr.go

@@ -19,7 +19,7 @@ type Resolver interface {
 }
 
 var (
-	RESOLVING_TIMEOUT = time.Duration(100) * time.Millisecond
+	ResolvingTimeout = time.Duration(100) * time.Millisecond
 
 	ErrTimeOut = errors.New("expression resolver timed out")
 )
@@ -96,7 +96,7 @@ func (oe *JavascriptExpressionParser) Resolve(rootScope interface{}, currentScop
 	}
 
 	go func() {
-		<-time.After(RESOLVING_TIMEOUT)
+		<-time.After(ResolvingTimeout)
 		scoped.Interrupt <- func() {
 			panic(ErrTimeOut)
 		}