Documentation update and cleanup (#38)

DEVELOPMENT.md

@@ -0,0 +1,200 @@
+# Development
+
+This document describes development techniques that can improve and fasten this process.
+Here you can find information corelated with more than one component.
+There might be some additional data
+
+## Table of contents
+
+* [Deploy and set up external components](#deploy-and-set-up-external-components)
+  * [Testing with Kafka in docker](#testing-with-kafka-in-docker)
+  * [Testing with MQTT broker in docker](#testing-with-mqtt-broker-in-docker)
+  * [Testing with Postgres in docker](#testing-with-postgres-in-docker)
+
+## Deploy and set up external components
+
+The solution uses external services and componets:
+
+* Kafka/OpenShift Streams
+* MQTT/AMQ Broker
+* Postgres database
+
+For development purposes components can be deployed localy with Docker containers.
+
+Prerequsitions:
+
+* Docker installed
+* docker-compose installed
+
+### Testing with Kafka in docker
+
+You may need this setup to test:
+
+* Quarkus/MQTT->Kafka bridge
+
+#### Setup
+
+One of the quickest way to set up Kafka cluster for development purposes is to use Docker containers.
+
+The procedure to **set up the cluster** boils down to:
+
+```shell
+curl --silent --output docker-compose.yml \
+  https://raw.githubusercontent.com/confluentinc/cp-all-in-one/6.1.0-post/cp-all-in-one/docker-compose.yml
+
+docker-compose up -d
+```
+
+See https://docs.confluent.io/platform/current/quickstart/ce-docker-quickstart.html for the details.
+
+To **create a topic**:
+
+```shell
+docker-compose exec broker kafka-topics --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic ENTRY_EVENT
+```
+
+where
+
+* `broker` is the name of the container hosting Kafka broker instance
+* `localhost:9092` is the broker's URL
+* `ENTRY_EVENT` is the topic name
+
+To **produce some testing messages**, one can issue the following command:
+
+```shell
+docker-compose exec broker \
+  bash -c "seq 10 | kafka-console-producer --request-required-acks 1 --broker-list localhost:9092 --topic ENTRY_EVENT && echo 'Produced 10 messages.'"
+```
+or   
+```shell
+docker-compose exec broker \
+  bash -c "echo '{\"event_type\":\"customer focus\",\"event_timestamp\":\"2001-12-17T09:30:47.0\",\"payload\":{\"customer_id\":3,\"category\":\"Boys\"}}' | kafka-console-producer --request-required-acks 1 --broker-list localhost:9092 --topic FOCUS_EVENTS && echo 'Message produced.'"
+```
+
+where
+
+* `broker` is the name of the container hosting Kafka broker instance
+* `ENTRY_EVENT` is the topic name
+* `localhost:9092` is the broker's URL
+
+### Testing with MQTT broker in docker
+
+You may need this setup to test:
+
+* Recommendation Service
+* Visualization Application
+* Mobile Application
+* Scenario Player
+
+#### Setup
+
+To **run the container** with mosquitto borker:
+
+```shell
+docker run -d --rm --name mosquitto -p 1883:1883 eclipse-mosquitto
+```
+or
+```shell
+docker run -it -p 1883:1883 --name mosquitto eclipse-mosquitto mosquitto -c /mosquitto-no-auth.conf
+```
+
+To **publish to a topic**:
+
+```shell
+docker exec mosquitto mosquitto_pub -h 127.0.0.1 -t test -m "test message"
+```
+
+To **subscribe to a topic**:
+```shell
+docker exec mosquitto mosquitto_sub -h 127.0.0.1 -t test
+```
+
+### Testing with Postgres in docker
+
+You may need this setup to test:
+
+* Recommendation Service
+
+#### Setup
+
+CSV files are available in the [../training-with-artificial-data/data_0409_0/data4db/](../training-with-artificial-data/data_0409_0/data4db/) path.
+
+To **run the container**:
+
+Go to `training-with-artificial-data/data_0419_0/data4db` and run
+
+```shell
+docker run -v $PWD:/usr/local/pgsql/data -e POSTGRES_PASSWORD=root -p 5432:5432 -d postgres
+```
+
+To **create a tables**:
+
+Install postgresql client. If you are using Ubuntu, you can use the command
+
+```shell
+sudo apt-get install postgresql
+```
+
+Connect to the database using
+
+```shell
+psql -h 127.0.0.1 -p 5432 -U postgres
+```
+
+Create tables
+
+```sql
+CREATE TABLE coupon_info (
+  coupon_id INT,
+  coupon_type VARCHAR(16),
+  department VARCHAR(10),
+  discount INT,
+  how_many_products_required INT,
+  start_date VARCHAR(10),
+  end_date VARCHAR(10),
+  product_mean_price REAL,
+  products_available INT,
+  PRIMARY KEY (coupon_id)
+);
+
+CREATE_TABLE product_info (
+    product_id INT,
+    name VARCHAR(256),
+    category VARCHAR(50),
+    sizes VARCHAR(50),
+    vendor VARCHAR(50),
+    description VARCHAR(256), 
+    buy_price REAL,
+    department VARCHAR(10),
+    PRIMARY KEY (product_id)
+);
+
+CREATE_TABLE coupon_product (
+    coupon_id INT,
+    product_id INT,
+    FOREIGN KEY (coupon_id) REFERENCES coupon_info(coupon_id),
+    FOREIGN KEY (product_id) REFERENCES products(product_id)
+)
+
+CREATE TABLE customer_info (
+  ustomer_id INT,
+  gender VARCHAR(1),
+  age INT,
+  mean_buy_price REAL,
+  total_coupons_used: INT,
+  mean_discount_received: REAL,
+  unique_products_bought INT,
+  unique_products_bought_with_coupons: INT,
+  total_items_bought:
+  INT, PRIMARY KEY (customer_id)
+);
+```
+
+Fill DB with data:
+
+```sql
+COPY coupon_info FROM '<<DATA_PATH>>/coupon_info.csv' DELIMITER ',' CSV HEADER;
+COPY product_info FROM '<<DATA_PATH>>/products.csv' DELIMITER ',' CSV HEADER;
+COPY coupon_product FROM '<<DATA_PATH>>/coupon_product.csv' DELIMITER ',' CSV HEADER;
+COPY customer_info FROM '<<DATA_PATH>>/customer_info.csv' DELIMITER ',' CSV HEADER;
+```

README.md

@@ -1,6 +1,33 @@
 # Retail Store of the Future
 
-Disclaimer! This solution was created for demo purposes only. It contains simplifications and must not be used for production purposes!
+**Disclaimer!** This solution was created for demo purposes only. It contains simplifications
+and must not be used for production purposes!
+
+This solution shows some potential use-cases and capabilities of RedHat and SAP components in
+Intel-driven environments.
+
+In the Retail Store of the Future, the customer is in the center of the action.
+Thanks to the advanced technologies the store can "feel" customer's needs and help
+her/him to make better purchasing decisions.
+
+This solution uses AI techniques to increase customer satisfaction by proposing the most
+interesting coupons while moving through the store. In the proposed scenario the system also detects
+browsing customers and sends the assistant to the customer.
+
+The "Retail Store of the Future" project shows a potential architecture and components that could
+be used in a real-world scenario. Depending on the final solution and needs some elements
+could be changed, added, or removed.
+
+## Table of contents
+
+* [Solution diagram and components description](#solution-diagram-and-components-description)
+  * [Training](#training)
+  * [Prediction Service](#prediction-service)
+  * [Recommendation Service](#recommendation-service)
+  * [Customers simulator](#customers-simulator)
+  * [Visualization app](#visualization-app)
+* [Development](#development)
+* [Deployment and production](#deployment-and-production)
 
 ## Solution diagram and components description
 
@@ -10,30 +37,52 @@ Disclaimer! This solution was created for demo purposes only. It contains simpli
 
 The training part is made in the jupyter notebooks using Intel DAAL libraries.
 
-### Prediction service
+All scripts and necessary components are available in
+(training-with-artificial-data)[./training-with-artificial-data] path.
+
+There was a special data generator created for training purposes. It is placed in the
+(artificial-data-generator)[./artificial-data-generator] path.
+
+### Prediction Service
 
 Prediction service handles the model.
 
 It provides a REST API that can be called to retrieve predictions for coupons.
 
-Check [Prediction service README](prediction-service/README.md) for details.
+Check [Prediction Service](./prediction-service) path for details.
 
-### Recommendation service
+### Recommendation Service
 
-Recommendation service listens for MQTT requests.
+Recommendation service provides MQTT interfaces for the recommendation process.
 
-When the entry event occurs it calls (TODO) the central resource for client data. The component stores the data in a cache - database (TODO).
+It observes MQTT topics, pulls data, creates the context, and sends it to the Prediction Service.
 
-When the focus event occurs the component gets client and coupon data from the cache (TODO) and calls prediction service.
-The result of the prediction is pushed to MQTT.
+The response is interpreted, processed, and sent to the suitable MQTT topic.
 
-Check [Recommendation service README](recommendation-service/README.md) for details.
-MQTT topics description and schema can also be found there.
+### Customers simulator
+
+This simulator was made for demo purposes. It can simulate customers' movement through the store.
+See more details in the [README file](scenario-player/README.md)
 
 ### Visualization app
 
-The application was made for demo purposes. [More details int app's README](visualization-app/README.md).
+This application provides the interface for the store's crew. However, for demo purposes, it was also
+extended to simulate customer's mobile app. [More details](visualization-app).
 
-### Customers simulator
+Visualization app contains:
+
+* Store interface (customers preview, alerts)
+* Simulator interface (you can create a new scenario for customers simulator using UI)
+* Mobile app simulation (shows customer's mobile app behavior while moving through the store)
+
+## Development
+
+Each component has its own instructions describing the development, deployment, structure, interfaces, and more.
+Please find the particular component README for details.
+
+As shown in the diagram, there are additional, external components used in this solution. You can set up
+your own development environment using containers. The development instruction can be found (here)[DEVELOPMENT.md].
+
+## Deployment and production
 
-This simulator was made for demo purposes. See more details in the [README file](scenario-player/README.md)
+The solution is adapted to run in an OpenShift environment. The deployment process is described in (infra)[./infra] path.

artificial-data-generator/README.md

@@ -11,13 +11,17 @@ It makes a few assumptions for that purpose:
 1. People in the age 61 and above buys products from the 2/3 to the end of the list* more often,
 1. The customer has personal preferences which can change the probability of buying products by up to 40% (20% on average) in 20% of cases.
 
-
 > *There are actually 3 lists - vendors, departments, and categories. The order in all lists is random but always the same for all generated customers. Every customer has his/her  own list of preferences for all 3 types. All lists are generated the same way and using the same assumptions described above.
 
 All numbers and functions described above can be changed using the config.py file.
 
+## Table of contents
+
 ## The algorithm
 
+* [The algorithm](#the-algorithm)
+* [Usage](#usage)
+
 Here's an approximate algorithm of the generator. For more details please check the code.
 
 1. Generate customers