From 84db7ce720ff7727ebe5414792238d038ecc159c Mon Sep 17 00:00:00 2001 From: prasadtalasila Date: Tue, 7 Nov 2023 14:53:27 +0100 Subject: [PATCH] Updates docs, install scripts and config - Updates documentation links to release v0.3 branch. - Fixes mistakes in install script and docs. - Updates gateway config to have libms as well. --- deploy/README.md | 57 +++++++++++++------ deploy/config/gateway/fileConfig.yml | 11 ++-- deploy/single-script-install.sh | 2 +- docs/admin/client/CLIENT.md | 8 +-- docs/admin/host.md | 11 ++-- docs/admin/servers/lib/LIB-MS.md | 10 ++++ docs/admin/services.md | 1 - docs/admin/trial.md | 2 +- docs/admin/vagrant/single-machine.md | 2 +- docs/admin/vagrant/two-machines.md | 3 +- docs/index.md | 2 +- docs/redirect-page.html | 5 ++ servers/config/gateway/dynamic/fileConfig.yml | 29 ++++------ servers/lib/README.md | 28 ++++++--- 14 files changed, 107 insertions(+), 64 deletions(-) diff --git a/deploy/README.md b/deploy/README.md index 0a688d8f2..06f7dbf5e 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -1,6 +1,6 @@ # DTaaS on Linux Operating System -This directory contains code for running DTaaS application +These are installation instructions for running DTaaS application on a Ubuntu Server 22.04 Operating System. The setup requires a machine which can spare 16GB RAM, 8 vCPUs and 50GB Hard Disk space. @@ -9,9 +9,16 @@ A dummy **foo.com** URL has been used for illustration. Please change this to your unique website URL. It is assumed that you are going to serve the application in only HTTPS mode. +A successful installation will create a setup +similar to the one shown in the figure. + +![Single host install](../docs/admin/single-host.png) + Please follow these steps to make this work in your local environment. Download the codebase as zip file into your computer and unzip the same -into a directory named **DTaaS**. The rest of the instructions assume +into a directory named **DTaaS**. +Alternatively, clone this git repository into your computer. +The rest of the instructions assume that your working directory is **DTaaS**. ## Configuration @@ -22,20 +29,28 @@ The first step is to decide on the number of users and their usenames. The traefik gateway configuration has a template for two users. You can modify the usernames in the template to the usernames chosen by you. -### The traefik gateway server +### Traefik gateway server -You can run the Run the Traefik gateway server in both and +You can run the Traefik gateway server in both and HTTPS and HTTPS mode to experience the DTaaS application. The installation guide assumes that you can run the application in HTTPS mode. -The Traefik gateway configuration is at [fileConfig](../config/gateway/fileConfig.yml). -Change `localhost` to `foo.com` and user1/user2 to the usernames chosen by you. +The Traefik gateway configuration is at _[fileConfig](config/gateway/fileConfig.yml)_. +Change `foo.com` to your local hostname and user1/user2 to the usernames chosen by you. **NOTE**: Do not use `http://` or `https://` -in [fileConfig](../config/gateway/fileConfig.yml). +in [fileConfig](config/gateway/fileConfig.yml). #### Authentication +This step requires `htpasswd` commandline utility. If +it is not available on your system, please install the same by using + +```bash +sudo apt-get install -y apache2-utils +``` + +You can now proceed with update of the gateway authentication setup. The dummy username is `foo` and the password is `bar`. Please change this before starting the gateway. @@ -46,24 +61,25 @@ htpasswd deploy/config/gateway/auth password: ``` -The user credentials added in [auth](../config/gateway/auth) should match -the usernames in [fileConfig](../config/gateway/fileConfig.yml). +The user credentials added in _[config/gateway/auth](config/gateway/auth)_ +should match the usernames in +_[config/gateway/fileConfig](config/gateway/fileConfig.yml)_. -## Configure lib microservice +## Lib microservice The library microservice requires configuration. -A template of this configuration file is given in _config/lib_ file. +A template of this configuration file is given in _[config/lib](config/lib)_ file. Please modify this file as per your needs. The first step in this configuration is to prepare the a filesystem for users. An example file system in `files/` directory. You can rename the top-level user1/user2 to the usernames chosen by you. -Add an environment file named .env in lib for the library microservice. +Add an environment file named `.env` in lib for the library microservice. An example `.env` file is given below. The simplest possibility is to use `local` mode with the following example. The filepath is the absolute filepath to `files/` directory. -You can copy this configuration into _config/lib_ file to get started. +You can copy this configuration into _[config/lib](config/lib)_ file to get started. ```env PORT='4001' @@ -74,7 +90,7 @@ APOLLO_PATH='/lib' GRAPHQL_PLAYGROUND='true' ``` -## Configure React Client Website +## React Client Website ### Gitlab OAuth application @@ -101,11 +117,11 @@ the [Authentication page](client/auth.md). ### Update Client Config -Change the React website configuration in _deploy/config/client/env.js_. +Change the React website configuration in _[config/client/env.js](config/client/env.js)_. ```js window.env = { - REACT_APP_ENVIRONMENT: 'dev', + REACT_APP_ENVIRONMENT: 'prod', REACT_APP_URL: 'https://foo.com/', REACT_APP_URL_BASENAME: 'dtaas', REACT_APP_URL_DTLINK: '/lab', @@ -141,3 +157,12 @@ You can run this script multiple times until the installation is successful. ## Access the application Now you should be able to access the DTaaS application at: _https://foo.com_ + +## References + +Image sources: [Ubuntu logo](https://logodix.com/linux-ubuntu), +[Traefik logo](https://www.laub-home.de/wiki/Traefik_SSL_Reverse_Proxy_f%C3%BCr_Docker_Container), +[ml-workspace](https://github.com/ml-tooling/ml-workspace), +[nodejs](https://www.metachris.com/2017/01/how-to-install-nodejs-7-on-ubuntu-and-centos/), +[reactjs](https://krify.co/about-reactjs/), +[nestjs](https://camunda.com/blog/2019/10/nestjs-tx-email/) diff --git a/deploy/config/gateway/fileConfig.yml b/deploy/config/gateway/fileConfig.yml index 9190f365b..efd35bb28 100644 --- a/deploy/config/gateway/fileConfig.yml +++ b/deploy/config/gateway/fileConfig.yml @@ -24,11 +24,11 @@ http: - basic-auth service: user2 - vis: + libms: entryPoints: - http - rule: 'Host(`foo.com`) && PathPrefix(`/vis`)' - service: grafana + rule: 'Host(`foo.com`) && PathPrefix(`/lib`)' + service: libms # Middleware: Basic authentication @@ -55,8 +55,7 @@ http: servers: - url: "http://localhost:8091" - grafana: + libms: loadBalancer: servers: - - url: "http://localhost:3000" - + - url: "http://localhost:4001" \ No newline at end of file diff --git a/deploy/single-script-install.sh b/deploy/single-script-install.sh index 6cf781007..4ff80e6be 100755 --- a/deploy/single-script-install.sh +++ b/deploy/single-script-install.sh @@ -116,7 +116,7 @@ else git clone https://github.com/INTO-CPS-Association/DTaaS.git DTaaS cd DTaaS || exit git fetch --all - git checkout feature/distributed-demo + git checkout release-v0.3 fi TOP_DIR=$(pwd) diff --git a/docs/admin/client/CLIENT.md b/docs/admin/client/CLIENT.md index a96a5d0bd..dbc2e0437 100644 --- a/docs/admin/client/CLIENT.md +++ b/docs/admin/client/CLIENT.md @@ -99,10 +99,10 @@ one [ML Workspace](https://github.com/ml-tooling/ml-workspace) serving the following routes. ```js -https:foo.com//lab -https:foo.com//terminals/main -https:foo.com//tools/vnc/?password=vncpassword -https:foo.com//tools/vscode/ +https://foo.com//lab +https://foo.com//terminals/main +https://foo.com//tools/vnc/?password=vncpassword +https://foo.com//tools/vscode/ ``` The `username` is the user workspace created using ML Workspace docker container. diff --git a/docs/admin/host.md b/docs/admin/host.md index 71a90a23d..4ec9ec66c 100644 --- a/docs/admin/host.md +++ b/docs/admin/host.md @@ -38,9 +38,9 @@ The first step is to decide on the number of users and their usenames. The traefik gateway configuration has a template for two users. You can modify the usernames in the template to the usernames chosen by you. -### The traefik gateway server +### Traefik gateway server -You can run the Run the Traefik gateway server in both +You can run the Traefik gateway server in both HTTP and HTTPS mode to experience the DTaaS application. The installation guide assumes that you can run the application in HTTPS mode. @@ -80,7 +80,7 @@ password: The user credentials added in _deploy/config/gateway/auth_ should match the usernames in _deploy/config/gateway/fileConfig.yml_. -## Configure lib microservice +## Lib microservice The library microservice requires configuration. A template of this configuration file is given in _deploy/config/lib_ file. @@ -90,6 +90,7 @@ The first step in this configuration is to prepare the a filesystem for users. An example file system in `files/` directory. You can rename the top-level user1/user2 to the usernames chosen by you. +Add an environment file named `.env` in lib for the library microservice. An example `.env` file is given below. The simplest possibility is to use `local` mode with the following example. The filepath is the absolute filepath to `files/` directory. @@ -104,7 +105,7 @@ APOLLO_PATH='/lib' GRAPHQL_PLAYGROUND='true' ``` -## Configure React Client Website +## React Client Website ### Gitlab OAuth application @@ -131,7 +132,7 @@ Change the React website configuration in _deploy/config/client/env.js_. ```js window.env = { - REACT_APP_ENVIRONMENT: 'dev', + REACT_APP_ENVIRONMENT: 'prod', REACT_APP_URL: 'https://foo.com/', REACT_APP_URL_BASENAME: 'dtaas', REACT_APP_URL_DTLINK: '/lab', diff --git a/docs/admin/servers/lib/LIB-MS.md b/docs/admin/servers/lib/LIB-MS.md index 8cee47ae5..49974732c 100644 --- a/docs/admin/servers/lib/LIB-MS.md +++ b/docs/admin/servers/lib/LIB-MS.md @@ -96,6 +96,16 @@ yarn build # build the application yarn start # start the application ``` +#### Config flag + +If the environment file is named something other than `.env`, +the filename must be specifed with the command `-c, --config `, +when starting the application. For instance, + +```sh +yarn start -c ".env.development" +``` + You can press `Ctl+C` to halt the application. If you wish to run the microservice in the background, use diff --git a/docs/admin/services.md b/docs/admin/services.md index 9af78dc1b..676a799ba 100644 --- a/docs/admin/services.md +++ b/docs/admin/services.md @@ -63,7 +63,6 @@ at the following ports / URLs. | RabbitMQ Broker | services.foo.com:5672 | | RabbitMQ Broker Management Website | services.foo.com:15672 | | MQTT Broker | services.foo.com:1883 | -|| The firewall and network access settings of corporate / cloud network need to be configured to allow external access to the services. Otherwise the users of DTaaS diff --git a/docs/admin/trial.md b/docs/admin/trial.md index 39dc168be..15da97711 100644 --- a/docs/admin/trial.md +++ b/docs/admin/trial.md @@ -43,7 +43,7 @@ for getting the Gitlab OAuth application details. ## Install ```bash -wget https://raw.githubusercontent.com/INTO-CPS-Association/DTaaS/feature/distributed-demo/deploy/single-script-install.sh +wget https://raw.githubusercontent.com/INTO-CPS-Association/DTaaS/release-v0.3/deploy/single-script-install.sh bash single-script-install.sh ``` diff --git a/docs/admin/vagrant/single-machine.md b/docs/admin/vagrant/single-machine.md index 9d068ad1e..876017509 100644 --- a/docs/admin/vagrant/single-machine.md +++ b/docs/admin/vagrant/single-machine.md @@ -69,7 +69,7 @@ Set a cronjob inside the vagrant virtual machine to remote the conflicting default route. ```bash -wget https://raw.githubusercontent.com/INTO-CPS-Association/DTaaS/feature/distributed-demo/deploy/vagrant/route.sh +wget https://raw.githubusercontent.com/INTO-CPS-Association/DTaaS/release-v0.3/deploy/vagrant/route.sh sudo bash route.sh ``` diff --git a/docs/admin/vagrant/two-machines.md b/docs/admin/vagrant/two-machines.md index b8ce4eabb..37ab817e0 100644 --- a/docs/admin/vagrant/two-machines.md +++ b/docs/admin/vagrant/two-machines.md @@ -99,7 +99,6 @@ you can see the following services active within server2 (_services.foo.com_). | MQTT communication service | services.foo.com:1883 | | RabbitMQ communication service | services.foo.com:5672 | | RabbitMQ management service | services.foo.com:15672 | -|| ### Install DTaaS Application @@ -108,7 +107,7 @@ Execute the following commands from terminal ```bash vagrant up --provision dtaas vagrant ssh dtaas -wget https://raw.githubusercontent.com/INTO-CPS-Association/DTaaS/feature/distributed-demo/deploy/vagrant/route.sh +wget https://raw.githubusercontent.com/INTO-CPS-Association/DTaaS/release-v0.3/deploy/vagrant/route.sh sudo bash route.sh ``` diff --git a/docs/index.md b/docs/index.md index 52daafde2..8eb4f20cc 100644 --- a/docs/index.md +++ b/docs/index.md @@ -17,7 +17,7 @@ It is also possible to share the services offered by one DT with other users. There is an overview of the software available in the form of [slides](https://odin.cps.digit.au.dk/into-cps/dtaas/assets/DTaaS-short-intro.pdf), [video](https://odin.cps.digit.au.dk/into-cps/dtaas/assets/videos/DTaaS-short-intro.mp4), -and [feature walkthrough](https://odin.cps.digit.au.dk/into-cps/dtaas/assets/videos/dtaas-v0.2.0-demo.mp4). +and [feature walkthrough](https://odin.cps.digit.au.dk/into-cps/dtaas/assets/videos/dtaas-v0.3-demo.mp4). ## License diff --git a/docs/redirect-page.html b/docs/redirect-page.html index b7b4bd918..d53a7ed1d 100644 --- a/docs/redirect-page.html +++ b/docs/redirect-page.html @@ -60,6 +60,11 @@ Online PDF + + Version 0.3.0 + Online + PDF + Version 0.2.0 Online diff --git a/servers/config/gateway/dynamic/fileConfig.yml b/servers/config/gateway/dynamic/fileConfig.yml index dbb7319af..86c2d0ba7 100644 --- a/servers/config/gateway/dynamic/fileConfig.yml +++ b/servers/config/gateway/dynamic/fileConfig.yml @@ -24,47 +24,38 @@ http: - basic-auth service: user2 - vis: - entryPoints: - - http - rule: 'Host(`localhost`) && PathPrefix(`/vis`)' - service: grafana - - lib: + libms: entryPoints: - http rule: 'Host(`localhost`) && PathPrefix(`/lib`)' - service: lib + service: libms + # Middleware: Basic authentication middlewares: basic-auth: basicAuth: - usersFile: '/etc/traefik/auth' + usersFile: "/etc/traefik/auth" removeHeader: true + services: dtaas: loadBalancer: servers: - - url: 'http://localhost:4000' + - url: "http://localhost:4000" user1: loadBalancer: servers: - - url: 'http://localhost:8090' + - url: "http://localhost:8090" user2: loadBalancer: servers: - - url: 'http://localhost:8091' - - grafana: - loadBalancer: - servers: - - url: 'http://localhost:3000' + - url: "http://localhost:8091" - lib: + libms: loadBalancer: servers: - - url: 'http://localhost:4001' + - url: "http://localhost:4001" \ No newline at end of file diff --git a/servers/lib/README.md b/servers/lib/README.md index 578810179..5951a6fff 100644 --- a/servers/lib/README.md +++ b/servers/lib/README.md @@ -43,14 +43,11 @@ APOLLO_PATH='/lib' or '' GRAPHQL_PLAYGROUND='false' or 'true' ``` -If the environment file is named something other than `.env`, -the filename must be specifed with the command `-c, --config `, -when starting the application. For instance, - -```sh -yarn start -c ".env.development" -``` +The `LOCAL_PATH` variable is the absolute filepath to the +location of the local directory which will be served to users +by the Library microservice. +The `GITLAB_URL`, `GITLAB_GROUP` and `TOKEN` are only relevant for `gitlab` mode. The `TOKEN` should be set to your GitLab Group access API token. For more information on how to create and use your access token, [gitlab page](https://docs.gitlab.com/ee/user/group/settings/group_access_tokens.html). @@ -58,6 +55,15 @@ For more information on how to create and use your access token, Once you've generated a token, copy it and replace the value of `TOKEN` with your token for the gitlab group, can be found. +Replace the default values the appropriate values for your setup. + +**NOTE**: + +1. When \__MODE=local_, only _LOCAL_PATH_ is used. + Other environment variables are unused. +1. When _MODE=gitlab_, _GITLAB_URL, TOKEN_, + and _GITLAB_GROUP_ are used; _LOCAL_PATH_ is unused. + ## User Commands ```bash @@ -66,6 +72,14 @@ yarn build # build the application yarn start # start the application ``` +If the environment file is named something other than `.env`, +the filename must be specifed with the command `-c, --config `, +when starting the application. For instance, + +```sh +yarn start -c ".env.development" +``` + You can press `Ctl+C` to halt the application. If you wish to run the microservice in the background, use