From a9a6d69ec73e427788aed57f5192b51676890381 Mon Sep 17 00:00:00 2001 From: Amen AYADI <37482206+AyadiAmen@users.noreply.github.com> Date: Wed, 19 Feb 2020 13:53:09 +0100 Subject: [PATCH] Update logging doc (#103) * Update INSTALL.md * CI/CD with minikube * ldap documentation elastic-stack ldap documentation * Details on JHub LDAP documentation * typo in jhub ldap doc Co-authored-by: Sebastien Dupont --- INSTALL.md | 43 +++++++++++++++++++++++++++++++++++++------ doc/LOGGING.md | 19 +++++++++++++++++-- doc/USERMANAGEMENT.md | 4 ++++ 3 files changed, 58 insertions(+), 8 deletions(-) diff --git a/INSTALL.md b/INSTALL.md index b53abd1..593edf9 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -13,9 +13,9 @@ FADI - Installation This page describes how to install the FADI platform -1. on a laptop/workstation, using [Minikube](https://github.com/kubernetes/minikube) -2. on a generic Kubernetes cluster -3. on Google Kubernetes Engine (GKE) +1. on a laptop/workstation, using [Minikube](https://github.com/kubernetes/minikube) for local development +2. on a generic Kubernetes cluster for a self-hosted installation +3. on Google Kubernetes Engine (GKE) as an example of public cloud The last sections describes how to automate the deployment using Gitlab-CI, and configure various parts of FADI (user management, ...). @@ -93,7 +93,6 @@ minikube addons enable ingress cd helm # you can edit values.yaml file to customise the stack ./deploy.sh -# see deploy.log for connection information to the various services # specify the fadi namespace to see the different pods kubectl config set-context minikube --namespace fadi ``` @@ -182,7 +181,7 @@ Note that depending on your workstation size and network connection, this could The creation of a GKE environment can be done with [Terraform](https://www.terraform.io/) or manually. -See the Terraform scripts for the creation of the Kubernetes cluster [here](/terraform) and its documentation [here](/terraform/README.md). +See the [Terraform scripts](/terraform) for the creation of the Kubernetes cluster and its [documentation](/terraform/README.md). To manually create a Kubernetes cluster (GKE): @@ -203,6 +202,38 @@ It is also possible to create the Kubernetes cluster in command line, see: https See [.gitlab-ci.sample.yml](.gitlab-ci.sample.yml) for an example CI setup with [Gitlab-CI](https://about.gitlab.com/product/continuous-integration/). +A lightweight alternative to a proper Kubernetes cluster (for example for continuous integration or testing purposes) would be to install FADI in minikube (single node). + +Setup a server (VM or bare metal) with the following specifications: + +* 20 GB RAM +* 8 CPUs +* Debian 9 +* Docker, minikube, git, vim, kubectl, ... + * for port-forwarding: `sudo apt-get install socat` + +Launch minikube (in this case with `vm-driver` as `none`, see limitations of this approach [here](https://minikube.sigs.k8s.io/docs/reference/drivers/none/)): + +``` +sudo minikube start --vm-driver=none +# now install FADI as usual: +git clone https://github.com/cetic/fadi.git fadi +cd fadi +kubectl config set-context minikube +minikube addons enable ingress +cd helm +# you can edit values.yaml file to customise the stack +./deploy.sh +# specify the fadi namespace to see the different pods +kubectl config set-context minikube --namespace fadi +``` + +Open minikube to the outside world (make sure you know what you are doing here): + +``` +kubectl proxy --address='0.0.0.0' --disable-filter=true +``` + ## 6. Additional configuration See the [user management documentation](doc/USERMANAGEMENT.md) for information on how to configure user identification and authorization (LDAP, RBAC, ...). @@ -213,4 +244,4 @@ See the [reverse proxy documentation](doc/REVERSEPROXY.md) for information on ho See the [security documentation](doc/SECURITY.md) for information on how to configure SSL. -Seel the [TSimulus documentation](doc/TSIMULUS.md) for information on how to simulate sensors and generate realistic data with [TSimulus](https://github.com/cetic/TSimulus). +See the [TSimulus documentation](doc/TSIMULUS.md) for information on how to simulate sensors and generate realistic data with [TSimulus](https://github.com/cetic/TSimulus). diff --git a/doc/LOGGING.md b/doc/LOGGING.md index b3fe1fb..6f9e44c 100644 --- a/doc/LOGGING.md +++ b/doc/LOGGING.md @@ -17,7 +17,7 @@ Despite each one of these four technologies being a separate project, they have ![Elastic-stack](/doc/images/installation/elastic_stack.png) -To access the **Kibana** web interface, you have to go through the nginx-ldapauth-proxy, you can use this command: +To access the **Kibana** web interface, you have to go through the [nginx-ldapauth-proxy](###-LDAP-Authentication), you can use this command: ``` minikube service fadi-nginx-ldapauth-proxy @@ -41,4 +41,19 @@ To create the index pattern and monitor the logs, follow these simple steps: ![Kibana Logs](/doc/images/installation/kibana_logs.png) -For more details you can always visit the [Elastic-stack official documentation](https://www.elastic.co/guide/index.html). \ No newline at end of file +For more details you can always visit the [Elastic-stack official documentation](https://www.elastic.co/guide/index.html). + + +### LDAP Authentication +================ + +KIBANA is not compatible with ldap which means it can't be linked directly, to authenticate against the ldap server before accessing KIBANA we're using [nginx-ldap-auth](https://github.com/nginxinc/nginx-ldap-auth). +> The nginx-ldap-auth software is a reference implementation of a method for authenticating users who request protected resources from servers proxied by NGINX Plus. It includes a daemon (ldap-auth) that communicates with an authentication server which is in this case OpenLDAP. + +The kibana service isn't accessible directly, to get to it you have to access nginx-ldap-auth, authenticate using your username/password and if successful you'll be redirected to the kibana service, to do so run this command: + +``` +minikube service fadi-nginx-ldapauth-proxy +``` +for more info: [nginx plus authenticate users](https://www.nginx.com/blog/nginx-plus-authenticate-users/). + diff --git a/doc/USERMANAGEMENT.md b/doc/USERMANAGEMENT.md index 52bbcba..ccad531 100644 --- a/doc/USERMANAGEMENT.md +++ b/doc/USERMANAGEMENT.md @@ -58,6 +58,10 @@ where `{username}` will be overwrought by the value the user passes as username which means if david isn't in the developers group or the admins group, he will not be able to sign in. +A sample configuration can be found in the `jupyterhub:auth` section of the default FADI [`values.yaml` file](https://github.com/cetic/helm-fadi/blob/master/values.yaml) + +More details on using LDAP with JupyterHub in the [Jupyter documentation](https://z2jh.jupyter.org/en/stable/authentication.html#authenticating-with-ldap), + ### Superset Superset uses **Flask-AppBuilder** Security for the LDAP authentication, in order to activate we need to pass the configuration inside python config `configFile.py`.