Skip to content

Commit

Permalink
Project documentation has been updated.
Browse files Browse the repository at this point in the history
  • Loading branch information
nuald committed Nov 17, 2017
1 parent 7557c53 commit 9959fec
Show file tree
Hide file tree
Showing 5 changed files with 29 additions and 8 deletions.
3 changes: 1 addition & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
*.class
*.log
.DS_Store
target/

# User-specific stuff:
Expand Down
34 changes: 28 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,20 @@ The system layers:

- Speed: based on the standard deviation
- Batch: Random Forest classification (using [Smile](https://haifengl.github.io/smile/) engine)
- Serving: Akka/Scala cluster with in-memory Redis database and persistance with Cassandra
- Serving: [Akka](https://akka.io/) cluster with in-memory Redis database and persistence with Cassandra

The data flow:

1. MQTT messages are produced by the IoT emulator (`Producer` actor)
2. MQTT subscriber saves the messages into Cassandra database (`Consumer` actor)
3. The Random Forest model is constantly trained by the messages (`Trainer` actor)
4. HTTP endpoint requests the computation using the trained model and heuristics (`Analyzer` actor)
1. MQTT messages are produced by the IoT emulator ([Producer](src/main/scala/mqtt/Producer.scala) actor)
2. MQTT subscriber saves the messages into Cassandra database ([Consumer](src/main/scala/mqtt/Consumer.scala) actor)
3. The Random Forest model is constantly trained by the messages ([Trainer](src/main/scala/analyzer/Trainer.scala) actor)
4. HTTP endpoint requests the computation using the trained model and heuristics ([Analyzer](src/main/scala/analyzer/Analyzer.scala) actor)

# Table of Contents

* [A Sample of Lambda architecture project](#a-sample-of-lambda-architecture-project)
* [Requirements](#requirements)
* [Cluster client requirements](#cluster-client-requirements)
* [Usage](#usage)
* [IoT emulation](#iot-emulation)
* [Interactive processing](#interactive-processing)
Expand All @@ -43,6 +44,16 @@ Optionally you may install:
- [Graphviz](http://www.graphviz.org/) visualization software (its dot utility is used for
the Decision Tree visualization in the sample REPL session)
- [Hey](https://github.com/rakyll/hey) HTTP load generator (used for the performance tests)
- [Scala](https://www.scala-lang.org/download/) shell (used for [the helper script](start.sc) for clustering)

### Cluster client requirements

The cluster clients only uses Scala shell and SBT. To verify the installations, please use:

$ sbt compile
$ scala start.sc client --dry-run

It should create the target jar (`target/scala-2.12` directory) and configurations (`target/env` directory).

## Usage

Expand All @@ -62,10 +73,17 @@ Run the system (for the convenience, all microservices are packaged into the one

$ sbt run

Please note that Random Forest classification requires at least two classes in the input
data (that means that the analyzed messages should contain anomalies). Please use the
MQTT producer described below to generate anomalies,
otherwise the [Trainer](src/main/scala/analyzer/Trainer.scala) shows the errors.

### IoT emulation

Modify the sensor values with the Producer: http://localhost:8081

![](resources/img/producer.png?raw=true)

Verify the messages by subscribing to the required MQTT topic:

$ mosquitto_sub -t sensors/power
Expand All @@ -74,6 +92,8 @@ Verify the messages by subscribing to the required MQTT topic:

Verify the data stores with the Dashboard: http://localhost:8080

![](resources/img/timeline.png?raw=true)

Verify the entries data store using CQL:

$ cqlsh -e "select * from sandbox.entry limit 10;"
Expand Down Expand Up @@ -239,4 +259,6 @@ To manually get the performance metrics please use the hey tool:
$ hey -n 500 -c 10 -t 10 http://127.0.0.1:8082/
$ hey -n 500 -c 10 -t 10 http://127.0.0.1:8082/stress

Otherwise the stats is available via the Dashboard.
Otherwise the stats is available via the Dashboard: http://localhost:8080

![](resources/img/performance.png?raw=true)
Binary file added resources/img/performance.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added resources/img/producer.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added resources/img/timeline.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 9959fec

Please sign in to comment.