Visualization app v0.2.0

.gitignore

@@ -26,6 +26,7 @@ hs_err_pid*
 *__pycache__
 **/.git/
 **/.idea/
+**/.vscode/
 *.iws
 
 
@@ -39,4 +40,4 @@ hs_err_pid*
 **/venv/
 
 *.priv.*
-*.priv
+*.priv

visualization-app/.dockerignore

@@ -0,0 +1,13 @@
+**/.git/
+**/.idea/
+*.iws
+
+
+**/__pycache__/
+*.py[cod]
+*$py.class
+
+*.so
+
+**/.venv*
+**/venv/

visualization-app/.environment.variables.sh

@@ -0,0 +1,3 @@
+export MQTT_HOST='127.0.0.1'
+export MQTT_NAME=test1
+export SCENARIO_PLAYER_SCENARIO_ENDPOINT='http://localhost:8004/scenario'

visualization-app/Dockerfile

@@ -0,0 +1,6 @@
+FROM tiangolo/uvicorn-gunicorn-fastapi:python3.8
+
+COPY requirements.txt .
+RUN pip install -r requirements.txt
+
+COPY ./app /app/app

visualization-app/README.md

@@ -0,0 +1,112 @@
+# Project description
+This project was a part of broader demo. That broader demo analyzed customers movement in a retail store, determined their behaviour (for example: "customer stopped in men's clothes department") and use Machine Learning to model for purchase/product recommendation. The customer location was determined by movement sensors placed in the store.
+
+This service customer behaviour in a retail shop:
+
+* customer entering the store
+* customer movement
+* customer exiting the store
+* by generating proper MQTT messages.
+
+In this case, we are using a big store as an example, however, for the demo purposes, we predict customers' behaviors only in 5 areas: Women's, Men's, Boy's, Girl's, and Sport.
+
+# Functionality
+
+This application visualize the store and customers movement in it. The app also allows creating new scenarios and sending them to the scenario-player.
+
+This is the UI for the demo described above.
+
+This application uses a plan which is 1808px wide and 1315px high. In the plan below there are main areas borders (limit points) described.
+There are also the entrance and exit points shown.
+
+![alt text](docs/store-plan-pts.png)
+
+## Table of contents
+
+# Usage
+
+This is a web service (implemented with FastAPI). By default, it works on port 8000. (See instructions for details on configuring and running the service.)
+
+When starting, it does the following:
+
+* connects to MQTT server
+* exposes http endpoints with the UI
+* exposes REST API (HTTP + WS) for the UI
+* waits for, registers, and pass to the UI users movement
+
+## Dependencies
+
+Dependencies of the project are contained in requirements.txt file. All the packages are publicly available.
+
+All the packages can be installed with: pip install -f requirements.txt
+
+This application assumes running MQTT broker.
+
+## Service configuration
+
+The service reads the following **environment variables**:
+
+| Variable              | Description                           |  Default      |
+|-----------------------|---------------------------------------|--------------:|
+| CUSTOMERS_LIST_FILE   |                                       | app/resources/customers.json |
+| MQTT_HOST             |                                       | -             |
+| MQTT_PORT             |               	                    | 1883          |
+| MQTT_NAME             |                	                    | demoVisClient |
+| ENTER_TOPIC           |                                       | customer/enter|
+| MOVE_TOPIC            |                                       | customer/move |
+| EXIT_TOPIC            |                                       | customer/exit |
+| SCENARIO_PLAYER_SCENARIO_ENDPOINT | full address (ex: `http://localhost:8004/scenario`) to the scenario-player's `scenario` endpoint | - |
+
+
+(Parameters with `-` in "Default" column are required.)
+
+Use [log_config.py](./app/utils/log_config.py) to **configure logging behaviour**. 
+By default, console and file handlers are used. The file appender writes to `messages.log`.
+
+## Running the service
+
+### Development
+
+`environment.variables.sh` can be used for that purpose. Then, in order to run the service the following commands can be
+used:
+
+```
+$ . .environment.variables.sh
+$ . venv/bin/activate
+(venv)$ uvicorn app.main:app --host 0.0.0.0 --reload --reload-dir app
+```
+> Please, note `reload-dir` switch. Without it the reloader goes into an infinite loop because it detects log file changes (messages.log).
+
+### Production
+
+// To be done
+
+## App interfaces
+
+| Endpoint/topic/address | Type       | Description                           |
+|------------------------|------------|---------------------------------------|
+| `/`                    | Web (HTTP) | The main UI page                      |
+| `/phone/{customer_id}` | Web (HTTP) | Customer's phone simulation           |
+| `/health`              | HTTP       | Service Health                        |
+| `/api/new_scenario`    | HTTP       | Proxy to the scenario-player          |
+| `/api/customers`       | HTTP       | Returns list of a few customers for demo purposes |
+| `/ws/movement`         | Websocket  | Passes customers movements thru when received from MQTT topic |
+| `/ws/coupons`          | Websocket  | Passes coupons info thru when recieved from MQTT topic |
+| {CUSTOMER_ENTER_TOPIC} | MQTT       | Recieves `enter` events from Phone App or scenario-player |
+| {CUSTOMER_EXIT_TOPIC}  | MQTT       | Recieves `exit` events from Phone App or scenario-player |
+| {CUSTOMER_MOVE_TOPIC}  | MQTT       | Recieves `move` events from Phone App or scenario-player |
+
+## Using the UI
+
+When service is up and running go to the main path of it to see the UI, for example: `http://localhost:8000`
+
+There are 2 main tabs - "Store preview" and "Create customer scenario".
+
+1st one allows observing customers' movements in the store. On the right-hand side you can switch between the "Mobile app" and "Events log" view.
+"Mobile app" lets you choosing the customer to observe. After choosing one its color on the plan changes. You can also see the simulation of
+the client's phone application. "Events log" allows following the messages sent to the application via MQTT and then passed to the UI via Websockets.
+
+2nd tab - "Create customer scenario" allows creating a new scenario for the selected customer. Select the user, then the pin type, and put some
+pins on the plan. "Movement" pin lets you create points where the customer occurs for a short while. "Focus" points will generate multiple "Movement"
+points to simulate the client's focus on the product. Next, select Customer step time (how long does it take to move between "movement" points).
+Now start the simulation.

visualization-app/app/__init__.py

@@ -0,0 +1,4 @@
+import logging
+
+logger = logging.getLogger(__name__)
+logger.addHandler(logging.NullHandler())

visualization-app/app/config.py

@@ -0,0 +1,29 @@
+import os
+import sys
+
+from app import logger
+
+
+def validate_and_crash(variable, message):
+    if not variable:
+        logger.error(message)
+        sys.exit(message)
+
+
+logger.info('Reading environment variables...')
+
+CUSTOMERS_LIST_FILE = os.getenv('CUSTOMERS_LIST_FILE', 'app/resources/customers.json')
+
+MQTT_HOST = os.getenv('MQTT_HOST')
+MQTT_PORT = int(os.getenv('MQTT_PORT', 1883))
+MQTT_NAME = os.getenv('MQTT_NAME', 'demoVisClient')
+
+CUSTOMER_ENTER_TOPIC = os.getenv('ENTER_TOPIC', 'customer/enter')
+CUSTOMER_EXIT_TOPIC = os.getenv('EXIT_TOPIC', 'customer/exit')
+CUSTOMER_MOVE_TOPIC = os.getenv('MOVE_TOPIC', 'customer/move')
+
+SCENARIO_PLAYER_SCENARIO_ENDPOINT = os.getenv('SCENARIO_PLAYER_SCENARIO_ENDPOINT')
+
+REQUIRED_PARAM_MESSAGE = 'Cannot read {} env variable. Please, make sure it is set before starting the service.'
+validate_and_crash(SCENARIO_PLAYER_SCENARIO_ENDPOINT, REQUIRED_PARAM_MESSAGE.format('SCENARIO_PLAYER_SCENARIO_ENDPOINT'))
+validate_and_crash(MQTT_HOST, REQUIRED_PARAM_MESSAGE.format('MQTT_HOST'))

visualization-app/app/data_models.py

@@ -0,0 +1,34 @@
+from pydantic import BaseModel
+from typing import List, Optional
+
+from app.events_model import CustomerEvent
+
+
+class Location(BaseModel):
+    x: int
+    y: int
+
+
+class Point(BaseModel):
+    type: str
+    location: Location
+    timestamp: Optional[int]
+
+
+class CustomerDescription(BaseModel):
+    customer_id: str
+    name: Optional[str]
+    gender: Optional[str]
+    age_bucket: Optional[str]
+    preferred_vendors: Optional[List[str]]
+
+
+class Scenario(BaseModel):
+    customer: CustomerDescription
+    path: Optional[List[Point]]
+
+
+class CustomerEventExtended(BaseModel):
+    customer: CustomerDescription
+    location: Optional[Location]
+    event_type: str

visualization-app/app/events_model.py

@@ -0,0 +1,40 @@
+from pydantic import BaseModel
+from typing import Optional
+
+
+class CustomerEvent(BaseModel):
+    id: str
+    ts: int
+    x: Optional[int]
+    y: Optional[int]
+
+
+class CustomerEnterEvent(CustomerEvent):
+    """
+    id: --ID representing customer--,
+    ts: --timestamp of the entrance, in seconds since epoch--
+    """
+    id: str
+    ts: int
+
+
+class CustomerExitEvent(CustomerEvent):
+    """
+    id: --ID representing customer--,
+    ts: --timestamp of the exit, in seconds since epoch--
+    """
+    id: str
+    ts: int
+
+
+class CustomerMoveEvent(CustomerEvent):
+    """
+    id: --ID representing customer--,
+    ts: --timestamp of the move, in seconds since epoch--,
+     x: --x coordinate of location sensor that fired--,
+     y: --y coordinate of location sensor that fired--
+    """
+    id: str
+    ts: int
+    x: int
+    y: int

visualization-app/app/log_config.py

@@ -0,0 +1,25 @@
+import logging
+import os
+
+
+# TODO move to config
+LOG_FILENAME = "messages.log"
+LOG_FORMAT = "%(asctime)s [%(threadName)-12.12s] [%(levelname)-5.5s]\t%(message)s"
+LOG_LEVEL = os.environ.get('LOG_LEVEL', 'INFO').upper()
+
+assert LOG_LEVEL in ['DEBUG', 'INFO', 'WARNING', 'ERROR']
+
+
+def configure_logger():
+    logging.basicConfig(format=LOG_FORMAT, level=LOG_LEVEL)
+    # Basic console logger
+    logger = logging.getLogger("app")
+    # File logger
+    log_formatter = logging.Formatter(LOG_FORMAT)
+    file_handler = logging.FileHandler(LOG_FILENAME, encoding='UTF-8')
+    file_handler.setFormatter(log_formatter)
+    logger.addHandler(file_handler)
+    # Configuration done
+    logger.debug("Logger configured...")
+    return logger
+