Userguide update

.github/ISSUE_TEMPLATE/bug_report.md

@@ -21,7 +21,7 @@ A clear and concise description of what the bug is.
 Provide the environment in which the bug has happened (minikube on a workstation, full fledged Kubernetes cluster, ...)
 
 * **OS** (e.g. from `/etc/os-release`)
-* **VM driver** (e.g. cat ~/.minikube/machines/minikube/config.json | grep DriverName)
+* **VM driver** (e.g. `cat ~/.minikube/machines/minikube/config.json | grep DriverName`)
 * **Minikube version** (e.g. `minikube version`)
 
 **What happened**:
@@ -34,6 +34,14 @@ Provide the environment in which the bug has happened (minikube on a workstation
 
 **Output of `minikube logs` (if applicable)**:
 
+**Output of Kubectl for pods, events**
+
+```bash
+kubectl get events --all-namespaces
+kubectl get events -n fadi
+kubectl get pods -n fadi
+kubectl logs fadi-nifi
+```
 
 **Anything else we need to know**:
 

FAQ.md

@@ -10,9 +10,13 @@ FAQ - Frequently asked questions
 
 In case you encounter an issue with FADI, have a feature request or any other question, feel free to [open an issue](https://github.com/cetic/fadi/issues/new/choose).
 
+## How can I extend FADI
+
+FADI relies on Helm to integrate the various service together. To add another service to the stack, you can package it inside a [Helm chart](https://helm.sh/docs/howto/) and [add it to your own FADI chart](helm/README.md).
+
 ## Why "FADI"?
 
-FADI is the acronym for "Framework d'Analyse de Données Industrielles" ("A Framework for Industrial Data Analysis")
+FADI is the acronym for "Framework for Automating the Deployment and orchestration of container-based Infrastructures"
 
 ## FADI is not working
 
@@ -22,7 +26,15 @@ Please make sure the following steps have been taken beforehand:
 
 * update Minikube to the latest version
 * update Helm to the latest version
-* check the logs (`minikube logs`) for any suspicious error message
+* check the logs for any suspicious error message:
+
+```bash
+minikube logs
+kubectl get events --all-namespaces
+kubectl get events -n fadi
+kubectl get pods -n fadi
+kubectl logs fadi-nifi
+```
 
 ## OSx - slow installation
 
@@ -41,6 +53,8 @@ See
 * https://github.com/cetic/fadi/blob/feature/documentation/doc/REVERSEPROXY.md for the reverse proxy configuration guide 
 * https://github.com/cetic/fadi/issues/81 for port forwarding instructions
 
+In a Minikube setting, make sure the ingress plugin is enabled (`minikube addons enable ingress`), and populate your `/etc/hosts` file accordingly.
+
 ## How to configure Kubernetes Storage Class?
 
 If you encounter the error `pod has unbound PersistentVolumeClaims`, make sure you have a **default StorageClass** in your cluster.
@@ -79,4 +93,4 @@ kubectl patch storageclass local-path -p '{"metadata": {"annotations":{"storagec
 See also:
 
  * Kubernetes official documentation on storage classes: https://kubernetes.io/docs/concepts/storage/storage-classes/
- * https://github.com/cetic/helm-fadi/issues/15
+ * https://github.com/cetic/helm-fadi/issues/15

INSTALL.md

@@ -26,8 +26,6 @@ The deployment of the FADI stack is achieved with:
 * [Helm v3](https://helm.sh/).
 * [Kubernetes](https://kubernetes.io/).
 
-![](doc/images/architecture/helm-architecture.png)
-
 ## 1. Local installation
 
 This type of installation provides a quick way to test the platform, and also to adapt it to your needs.
@@ -70,7 +68,7 @@ To get the Kubernetes dashboard, type:
 minikube dashboard
 ```
 
-This will open a browser window with the [Kubernetes Dashboard](http://127.0.0.1:40053/api/v1/namespaces/kube-system/services/http:kubernetes-dashboard:/proxy/), it should look like this:
+This will open a browser window with the Kubernetes Dashboard:
 
 ![Minikube initial dashboard](doc/images/installation/minikube_dashboard.png)
 
@@ -195,7 +193,16 @@ It is also possible to create the Kubernetes cluster in command line, see: https
 ## 4. Troubleshooting
 
 * Installation logs are located in the `helm/deploy.log` file.
-* Enable local monitoring in minikube: `minikube addons enable metrics-server`
+* Check the Minikube and Kubernetes logs:
+```bash
+minikube logs
+kubectl get events --all-namespaces
+kubectl get events -n fadi
+kubectl get pods -n fadi
+kubectl logs fadi-nifi-xxxxx -n fadi
+```
+* Enable [metrics server](https://kubernetes.io/docs/tasks/debug-application-cluster/resource-metrics-pipeline/#metrics-server) in minikube: `minikube addons enable metrics-server`
+* The [FAQ](FAQ.md) provides some guidance on common issues
 * For Windows users, please refer to the following [issue](https://github.com/cetic/fadi/issues/55).
 
 ## 5. Continuous integration (CI) and deployment (CD)

README.md

@@ -8,10 +8,7 @@
 
 ## What is FADI?
 
-FADI is a Cloud Native platform for Big Data based on mature open source tools.
-The FADI project is dedicated to making the deployment of Big Data tools simple, portable and scalable. 
-The goal is to provide a straightforward way to deploy open-source systems for Big Data to various infrastructures (private and public clouds). 
-Anywhere you can run [Kubernetes](https://kubernetes.io/), you should be able to run FADI.
+The purpose of FADI is to provide a straightforward way to deploy integrated and modular Big Data systems to various infrastructures (private and public clouds). FADI relies on the [Kubernetes](https://kubernetes.io/) container orchestration engine for portability and scalability.
 
 ![FADI carousel](doc/images/carousel/carousel.gif)
 
@@ -70,4 +67,4 @@ This project follows the [all-contributors](https://github.com/all-contributors/
 You can find out more about FADI on the [CETIC website](https://cetic.be/FADI)
 
 ## License
-[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fcetic%2Ffadi.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2Fcetic%2Ffadi?ref=badge_large)
+[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fcetic%2Ffadi.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2Fcetic%2Ffadi?ref=badge_large)

doc/LOGGING.md

@@ -43,9 +43,7 @@ To create the index pattern and monitor the logs, follow these simple steps:
 
 For more details you can always visit the [Elastic-stack official documentation](https://www.elastic.co/guide/index.html).
 
-
 ### LDAP Authentication 
-================
 
 KIBANA is not compatible with ldap which means it can't be linked directly, to authenticate against the ldap server before accessing KIBANA we're using [nginx-ldap-auth](https://github.com/nginxinc/nginx-ldap-auth).
 > The nginx-ldap-auth software is a reference implementation of a method for authenticating users who request protected resources from servers proxied by NGINX Plus. It includes a daemon (ldap-auth) that communicates with an authentication server which is in this case OpenLDAP.
@@ -55,5 +53,5 @@ The kibana service isn't accessible directly, to get to it you have to access ng
 ```
 minikube service fadi-nginx-ldapauth-proxy
 ```
-for more info: [nginx plus authenticate users](https://www.nginx.com/blog/nginx-plus-authenticate-users/).
 
+For more informations, see this blog post: [nginx plus authenticate users](https://www.nginx.com/blog/nginx-plus-authenticate-users/).

doc/MONITORING.md

@@ -0,0 +1,64 @@
+Monitoring
+==========
+
+<p align="left";>
+	<a href="https://www.elastic.co" alt="elk">
+	    <img src="images/logos/zabbix_logo.png" align="center" alt="ELK logo" width="200px" />
+    </a>
+</p>
+
+**[ZABBIX](https://www.zabbix.com)** Zabbix is an open-source monitoring software tool for diverse IT components, including networks, servers, virtual machines and cloud services. Zabbix provides monitoring metrics, among others network utilization, CPU load and disk space consumption.
+
+## Zabbix components
+
+### Zabbix Server
+
+Zabbix server is the central process of Zabbix software.
+
+The server performs the polling and trapping of data, it calculates triggers, sends notifications to users. It is the central component to which Zabbix agents and proxies report data on availability and integrity of systems. The server can itself remotely check networked services (such as web servers and mail servers) using simple service checks.
+
+### Zabbix Agent
+
+Zabbix agent is deployed on a monitoring target to actively monitor local resources and applications (hard drives, memory, processor statistics etc).
+
+### Zabbix Web ( frontend )
+
+Zabbix web interface is a part of Zabbix software. It is used to manage resources under monitoring and view monitoring statistics.
+
+### Zabbix Proxy 
+
+Zabbix proxy is a process that may collect monitoring data from one or more monitored devices and send the information to the Zabbix server, essentially working on behalf of the server. All collected data is buffered locally and then transferred to the Zabbix server the proxy belongs to.
+
+## How to use
+
+Make sure to enable zabbix in the `values.yaml` file, then to access the front-end use the following command:
+
+```
+minikube service fadi-zabbix-web
+```
+
+The default username/password are `Admin`/`zabbix`, once connected make sure the zabbix-server is working on the global view, to see the received metrics ( or graphs ) head to the tab `Latest data` under `monitoring` and then click on select on **host groups** and **hosts** and choose your `zabbix servers`.
+
+![zabbix](images/carousel/zabbix.gif)
+
+## LDAP Authentication 
+
+
+By default, internal [Zabbix authentication](https://www.zabbix.com/documentation/4.0/manual/web_interface/frontend_sections/administration/authentication) is used globally. To change to LDAP - select LDAP as Default authentication and enter **authentication details** in the LDAP settings tab. 
+
+The default **authentication details** for FADI:
+
+```
+LDAP host: fadi-openldap
+Port: 389
+Base DN: dc=ldap,dc=cetic,dc=be
+Search attribute: cn
+Bind DN: cn=admin,dc=ldap,dc=cetic,dc=be
+Bind password: password1
+User password: password1
+```
+
+
+![zabbix](images/carousel/zabbix-auth.gif)
+
+For more details you can always go to: [Zabbix Documentation 4.0](https://www.zabbix.com/documentation/4.0/manual/introduction).

doc/REVERSEPROXY.md

@@ -34,10 +34,10 @@ kubectl get clusterrole traefik-ingress-controller 2> /dev/null || kubectl creat
 
 Take a look at the [sample file](/helm/traefik/rbac-config.yaml).
 
-Then, you can install Traefik with Helm: (If you want further information, you can follow this [tutorial](https://docs.traefik.io/v1.3/user-guide/kubernetes/#deploy-trfik-using-helm-chart))
+Then, you can install Traefik with Helm: (If you want further information, you can follow this [tutorial](https://docs.traefik.io/v1.7/user-guide/kubernetes/#deploy-traefik-using-helm-chart))
 
 ```
-helm upgrade --install traefik stable/traefik -f ./traefik/values.yaml --namespace kube-system --tiller-namespace tiller
+helm upgrade --install traefik stable/traefik -f ./traefik/values.yaml --namespace kube-system
 ```
 
 The values file can be found [here](/helm/traefik/values.yaml).
@@ -81,4 +81,4 @@ grafana:
 
 You should now be able to access Grafana through the domain name you have chosen: `http(s)://grafana.yourdomain.com`
 
-Next you will also want to configure SSL access to your services. For that, have a look at the [security documentation](/doc/SECURITY.md). 
+Next you will also want to configure SSL access to your services. For that, have a look at the [security documentation](/doc/SECURITY.md).