Scenario-player v0.2

customersim/.dockerignore

@@ -0,0 +1,13 @@
+**/.git/
+**/.idea/
+*.iws
+
+
+**/__pycache__/
+*.py[cod]
+*$py.class
+
+*.so
+
+**/.venv*
+**/venv/

customersim/.environment.variables.sh

@@ -0,0 +1,8 @@
+export STORE_HEIGHT=10
+export STORE_WIDTH=6
+
+export CUSTOMERS_AVERAGE_IN_STORE=6
+export CUSTOMERS_LIST_FILE='customers.csv'
+
+export MQTT_HOST='127.0.0.1'
+export MQTT_NAME=test1

customersim/Dockerfile

@@ -0,0 +1,6 @@
+FROM tiangolo/uvicorn-gunicorn-fastapi:python3.8
+
+COPY requirements.txt .
+RUN pip install -r requirements.txt
+
+COPY ./app /app/app

customersim/README.md

@@ -1,37 +1,182 @@
-# Flask Sample Application
+# Functionality 
+This service generates messages that simulate customer behaviour in a reatail shop:
+* customer entering the store
+* customer movement 
+* customer exiting the store
 
-This repository provides a sample Python web application implemented using the Flask web framework and hosted using ``gunicorn``. It is intended to be used to demonstrate deployment of Python web applications to OpenShift 3.
 
-## Implementation Notes
+## Table of contents
+* [Functionality](#functionality)
+* [Event payloads](#event-payloads)
+    * [customer/enter](#customerenter)
+    * [customer/move](#customermove)
+    * [customer/exit](#customerexit)
 
-This sample Python application relies on the support provided by the default S2I builder for deploying a WSGI application using the ``gunicorn`` WSGI server. The requirements which need to be satisfied for this to work are:
+* [Development](#development)
+  * [Dependencies](#dependencies)
+  * [Service configuration](#service-configuration)
+  * [Running the service](#running-the-service)
+  * [Testing with MQTT broker in docker](#testing-with-mqtt-broker-in-docker)
+  * [Testing without MQTT](#testing-without-mqtt)
+  * [Mock event endpoints](#mock-event-endpoints)
 
-* The WSGI application code file needs to be named ``wsgi.py``.
-* The WSGI application entry point within the code file needs to be named ``application``.
-* The ``gunicorn`` package must be listed in the ``requirements.txt`` file for ``pip``.
+* [Deployment](#deployment)
+  * [Docker image](#docker-image)
+  * [Connecting to a secured broker](#connecting-to-a-secured-broker)
 
-In addition, the ``.s2i/environment`` file has been created to allow environment variables to be set to override the behaviour of the default S2I builder for Python.
 
-* The environment variable ``APP_CONFIG`` has been set to declare the name of the config file for ``gunicorn``.
 
-## Deployment Steps
+## Event payloads
+The service assumes the following data will be provided with given event types.
 
-To deploy this sample Python web application from the OpenShift web console, you should select ``python:2.7``, ``python:3.3``, ``python:3.4`` or ``python:latest``, when using _Add to project_. Use of ``python:latest`` is the same as having selected the most up to date Python version available, which at this time is ``python:3.4``.
+This script generates the following MQTT messages
 
-The HTTPS URL of this code repository which should be supplied to the _Git Repository URL_ field when using _Add to project_ is:
+### customer/enter
 
-* https://github.com/OpenShiftDemos/os-sample-python.git
+```
+{ 
+  id: --ID representing customer--, 
+  ts: --timestamp of the entrance, in seconds since epoch-- 
+}
+```
+
+### customer/move
+
+```
+{ 
+ id: --ID representing customer--,
+ ts: --timestamp of the move, in seconds since epoch--,
+  x: --x coordinate of location sensor that fired--,
+  y: --y coordinate of location sensor that fired--
+}
+```
+
+### customer/exit
+
+```
+{ 
+ id: --ID representing customer--,
+ ts: --timestamp of the exit, in seconds since epoch--
+}
+```
+
+
+
+# Development
+
+## Dependencies
+
+Dependencies of the project are contained in [requirements.txt](requirements.txt) file. All the packages are publicly
+available.
+
+All the packages can be installed with:
+`pip install -f requirements.txt`
+
+## Service configuration
+
+The service reads the following **environment variables**:
+
+| Variable               | Description                          |  Default      |
+|------------------------|--------------------------------------|--------------:|
+| STORE_HEIGHT          |                                       | 10            |
+| STORE_WIDTH           |                                       | 6             |
+| CUSTOMERS_AVERAGE_IN_STORE |  					            | 6 		    |
+| CUSTOMERS_LIST_FILE   |                                       | customers.csv |
+| MQTT_HOST             |                                       | -             |
+| MQTT_PORT             |               	                    | 1883          |
+| MQTT_NAME             |                	                    | demoClient    |
+| ENTER_TOPIC           |                                       | customer/enter|
+| MOVE_TOPIC            |                                       | customer/move |
+| EXIT_TOPIC            |                                       | customer/exit |
+
+(Parameters with `-` in "Default" column are required.)
+
+Use [log_config.py](./app/utils/log_config.py) to **configure logging behaviour**. 
+By default, console and file handlers are used. The file appender writes to `messages.log`.
+
+
+## Running the service
 
-If using the ``oc`` command line tool instead of the OpenShift web console, to deploy this sample Python web application, you can run:
+For my development I created a project with dedicated virtual environment (Python 3.8, all the dependencies installed
+there).
+
+The code reads sensitive information (tokens, secrets) from environment variables. They need to be set accordingly in
+advance.
+`environment.variables.sh` can be used for that purpose. Then, in order to run the service the following commands can be
+used:
 
 ```
-oc new-app https://github.com/OpenShiftDemos/os-sample-python.git
+$ . .environment.variables.sh
+$ . venv/bin/activate
+(venv)$ uvicorn app.main:app --host 0.0.0.0 --reload --reload-dir app
 ```
+> Please, note `reload-dir` switch. Without it the reloader goes into an infinite loop because it detects log file changes (messages.log).
+
+## Testing with MQTT broker in docker
 
-In this case, because no language type was specified, OpenShift will determine the language by inspecting the code repository. Because the code repository contains a ``requirements.txt``, it will subsequently be interpreted as including a Python application. When such automatic detection is used, ``python:latest`` will be used.
+Quick way to **set up a simple MQTT broker** is to use Docker containers:
+```shell
+docker run -d --rm --name mosquitto -p 1883:1883 eclipse-mosquitto
+```
+or
+```shell
+docker run -it -p 1883:1883 --name mosquitto eclipse-mosquitto mosquitto -c /mosquitto-no-auth.conf
+```
 
-If needing to select a specific Python version when using ``oc new-app``, you should instead use the form:
+To **publish to a topic**:
 
+```shell
+docker exec mosquitto mosquitto_pub -h 127.0.0.1 -t test -m "test message"
 ```
-oc new-app python:2.7~https://github.com/OpenShiftDemos/os-sample-python.git
+
+To **subscribe to a topic**:
+```shell
+docker exec mosquitto mosquitto_sub -h 127.0.0.1 -t test
 ```
+
+### Testing without MQTT
+There is an environment variable, `TESTING_MOCK_MQTT`, that will create an MQTT client mock instead of trying to connect
+to a real MQTT broker. Instead of publishing the messages, they will be simply logged/printed out.
+
+This may be helpful for local development or testing.
+
+### Producing test messages
+
+```shell
+curl http://127.0.0.1:8000/produce_entry -d '{"id": "997", "ts": 192326400}'
+ ```
+
+```shell
+curl http://127.0.0.1:8000/produce_exit -d '{"id": "997", "ts": 192326400}'
+ ```
+
+```shell
+curl http://127.0.0.1:8000/produce_move -d '{"id": "997", "ts": 192326400, "x": 2, "y": 3}'
+ ```
+
+
+# Deployment
+
+## Docker image
+The docker image for the service is [Dockerfile](Dockerfile).
+It is based on FastAPI "official" image.
+See https://github.com/tiangolo/uvicorn-gunicorn-fastapi-docker
+for the details on configuring the container (http port, log level, etc.)
+
+In order to build the image use:
+```
+docker build -t customersim-service:0.0.1 .
+```
+
+> Set image name (`customersim-service`) and tag (`0.0.1`) according to
+> your needs.
+
+To run the service as a Docker container run:
+```
+docker run -d -e LOG_LEVEL="warning"  --name customersim-service customersim-service:0.0.1
+
+```
+
+## Connecting to a secured broker
+**TODO** Add info about setting user/password
+**TODO** Add info about using client certificates (TLS)

customersim/app/__init__.py

@@ -0,0 +1,4 @@
+import logging
+
+logger = logging.getLogger(__name__)
+logger.addHandler(logging.NullHandler())

customersim/app/config.py

@@ -0,0 +1,34 @@
+import os
+import sys
+
+from app import logger
+
+
+def validate_and_crash(variable, message):
+    if not variable:
+        logger.error(message)
+        sys.exit(message)
+
+
+logger.info('Reading environment variables...')
+
+STORE_HEIGHT = int(os.getenv('STORE_HEIGHT', 10))
+STORE_WIDTH = int(os.getenv('STORE_WIDTH', 6))
+
+CUSTOMERS_AVERAGE_IN_STORE = int(os.getenv('CUSTOMERS_AVERAGE_IN_STORE', 6))
+CUSTOMERS_LIST_FILE = os.getenv('CUSTOMERS_LIST_FILE', 'customers.csv')
+
+MQTT_HOST = os.getenv('MQTT_HOST')
+MQTT_PORT = int(os.getenv('MQTT_PORT', 1883))
+MQTT_NAME = os.getenv('MQTT_NAME', 'demoClient')
+
+CUSTOMER_ENTER_TOPIC = os.getenv('ENTER_TOPIC', 'customer/enter')
+CUSTOMER_EXIT_TOPIC = os.getenv('EXIT_TOPIC', 'customer/exit')
+CUSTOMER_MOVE_TOPIC = os.getenv('MOVE_TOPIC', 'customer/move')
+
+TESTING_MOCK_MQTT = os.getenv('TESTING_MOCK_MQTT', 'false')
+TESTING_MOCK_MQTT = TESTING_MOCK_MQTT.lower() in ['1', 'yes', 'true']
+
+
+REQUIRED_PARAM_MESSAGE = 'Cannot read {} env variable. Please, make sure it is set before starting the service.'
+validate_and_crash(MQTT_HOST, REQUIRED_PARAM_MESSAGE.format('MQTT_HOST'))

customersim/app/domain_model.py

@@ -0,0 +1,70 @@
+import datetime
+import random
+
+
+# Represents a "square" in the store at a particular location, and contain all valid moves from that location
+class Location:
+    def __init__(self, x: int, y: int, width: int, height: int):
+        self.x = x
+        self.y = y
+        # a list of all valid moves. Each move is a tuple of the form
+        # ("adjacent x location", "adjacent y location", "is this closer to the exit?")
+        self.validMoves = [(a, b, True if a <= self.x and b <= self.y else False) for a in
+                           range(max(self.x - 1, 0), min(self.x + 2, width)) for b in
+                           range(max(self.y - 1, 0), min(self.y + 2, height)) if not (a == self.x and b == self.y)]
+
+
+class Store:
+    def __init__(self, width: int, height: int):
+        self.height = height
+        self.width = width
+        self.locations = [[Location(x, y, width, height) for y in range(0, height)] for x in range(0, width)]
+
+
+class Customer:
+    def __init__(self, store, customer_id: str, name: str):
+        self.store = store
+        # customers enter and exit from the bottom left corner of the store
+        self.currentLocation = store.locations[0][0]
+        # the *average* amount of time this customer will spend on a square
+        self.meanDwellTime = random.uniform(1, 20)
+        # how consistently the customer spends that time. Higher means more inconsistent
+        self.consistency = random.uniform(1, 5)
+        self.nextMoveTime = self.get_next_move_time()
+        self.isExiting = False
+        # the time this customer will start to exit
+        self.exitTime = datetime.datetime.now() + datetime.timedelta(0, random.uniform(1, 600))
+        self.id = customer_id
+        self.name = name
+
+    def get_next_move_time(self):
+        # amount of time spent at a location is a random value picked from a gaussian distribution,
+        # with a mean equal to the customer's average dwell time and a standard deviation
+        # equal to the customer's consistency
+        return datetime.datetime.now() + datetime.timedelta(0, random.gauss(self.meanDwellTime, self.consistency))
+
+    def move(self):
+        # if the customer is exiting, only move to an adjacent location that is towards the exit.
+        # If they are already at the door, don't move
+        if self.isExiting:
+            if self.currentLocation.x == 0 and self.currentLocation.y == 0:
+                (newX, newY) = (0, 0)
+            else:
+                (newX, newY, isTowardsExit) = random.choice(
+                    [(x, y, e) for (x, y, e) in self.currentLocation.validMoves if e is True])
+        else:
+            # if the customer is not exiting, pick any adjacent location
+            (newX, newY, isTowardsExit) = random.choice(self.currentLocation.validMoves)
+
+        self.currentLocation = self.store.locations[newX][newY]
+
+    def tick(self):
+        if not self.isExiting and self.exitTime < datetime.datetime.now():
+            self.isExiting = True
+
+        if self.nextMoveTime < datetime.datetime.now():
+            self.nextMoveTime = self.get_next_move_time()
+            self.move()
+            return True
+
+        return False

customersim/app/events_model.py

@@ -0,0 +1,32 @@
+from pydantic import BaseModel
+
+
+class CustomerEnterEvent(BaseModel):
+    """
+    id: --ID representing customer--,
+    ts: --timestamp of the entrance, in seconds since epoch--
+    """
+    id: str
+    ts: int
+
+
+class CustomerExitEvent(BaseModel):
+    """
+    id: --ID representing customer--,
+    ts: --timestamp of the exit, in seconds since epoch--
+    """
+    id: str
+    ts: int
+
+
+class CustomerMoveEvent(BaseModel):
+    """
+    id: --ID representing customer--,
+    ts: --timestamp of the move, in seconds since epoch--,
+     x: --x coordinate of location sensor that fired--,
+     y: --y coordinate of location sensor that fired--
+    """
+    id: str
+    ts: int
+    x: int
+    y: int