Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add path prefixed gitlab-ce instance #905

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
046888f
Add path prefixed gitlab-ce instance
aryanpingle Sep 10, 2024
c9e0c51
Add http/s explanations
aryanpingle Sep 11, 2024
f90b13b
feat: Inline gitlab.rb configs into compose
aryanpingle Sep 17, 2024
3271231
refactor: Use variables for gitlab/.env
aryanpingle Sep 17, 2024
8738a2f
feat: Make gitlab use local directory
aryanpingle Sep 17, 2024
a664587
fix: Fix typo in gitlab .gitignore
aryanpingle Sep 17, 2024
eea7580
feat: Revert gitlab-ce to 16.4.1-ce.0
aryanpingle Sep 17, 2024
f47c36a
fix: Fix forward-auth DNS issue
aryanpingle Sep 17, 2024
41b3256
Refactor gitlab compose
aryanpingle Sep 18, 2024
53a6db4
Revert server env file modifications
aryanpingle Sep 18, 2024
b9c1bdc
Add README to gitlab directory
aryanpingle Sep 18, 2024
2367c3e
style: Fix style issue in gitlab README
aryanpingle Sep 18, 2024
3e8504b
Rename gitlab compose file (+ remove obsolete version label)
aryanpingle Sep 20, 2024
bc13e26
Add INTEGRATION.md
aryanpingle Sep 23, 2024
7b2c5a6
Adds code to serve DTaaS over https on localhost
prasadtalasila Sep 23, 2024
c8b40f6
Add OAuth token configuration step to integration guide
aryanpingle Sep 23, 2024
2cf7e3f
Fixes the localhost integration of gitlab
prasadtalasila Sep 25, 2024
6defdc6
Merge 'pr-905' into feature/distributed-demo
aryanpingle Sep 25, 2024
2aeb139
Edit INTEGRATION.md for localhost installations
aryanpingle Sep 25, 2024
141a143
Update client.dockerfile
aryanpingle Oct 2, 2024
7ac6581
Revert accidental commit "Update client.dockerfile"
aryanpingle Oct 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -165,3 +165,6 @@ runner.yml

# command scripts for runner
servers/execution/runner/lifecycle*

# certs
*.pem
17 changes: 10 additions & 7 deletions deploy/README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
# Digital Twin as a Service
# Install

These are installation instructions for running DTaaS application.
The setup requires a machine which can spare 16GB
RAM, 8 vCPUs and 50GB Hard Disk space.

The application can be installed using
__docker with compose plugin or docker desktop__
on all the supported Operating Systems.
The DTaaS can be installed in different ways.
Each version serves a different purpose.

There are two installation scenarios each with a dedicated set of instructions:

1. Single user [localhost](./docker/LOCALHOST.md)
1. Multi-user [web application](./docker/SERVER.md)
| Installation Setup | Purpose |
|:-----|:-----|
| [localhost](docker/LOCALHOST.md) | Install DTaaS on your computer for a single user; does not need a web server. _This setup does not require domain name._ |
| [Server](docker/SERVER.md) | Install DTaaS on server for multiple users. |
| [One vagrant machine](vagrant/single-machine/README.md) | Install DTaaS on a virtual machine; can be used for single or multiple users. |
| [Two vagrant machines](vagrant/two-machine/README.md) | Install DTaaS on two virtual machines; can be used for single or multiple users. |
| | The core DTaaS application is installed on the first virtual machine and all the services (RabbitMQ, MQTT, InfluxDB, Grafana and MongoDB) are installed on second virtual machine. |
2 changes: 1 addition & 1 deletion deploy/config/client/env.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,4 @@ if (typeof window !== 'undefined') {
REACT_APP_LOGOUT_REDIRECT_URI: 'https://foo.com/',
REACT_APP_GITLAB_SCOPES: 'openid profile read_user read_repository api',
};
};
};
2 changes: 1 addition & 1 deletion deploy/docker/.env.server
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ CLIENT_SECRET='xx'
OAUTH_SECRET='random-secret-string'
CLIENT_CONFIG='/Users/<Username>/DTaaS/deploy/config/client/env.js'
username1='user1'
username2='user2'
username2='user2'
159 changes: 159 additions & 0 deletions deploy/docker/LOCALHOST.SECURE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
# Install DTaaS on localhost

The installation instructions provided in this README are
ideal for running the **DTaaS on both localhost served over HTTPS connection**.
The intention is to integrate Gitlab into DTaaS so that both these
are running on localhost.

This installation is ideal for single users intending to use
DTaaS on their own computers.

## Design

An illustration of the docker containers used and the authorization
setup is shown here.

![Traefik OAuth](./localhost-https.png)

## Requirements

The installation requirements to run this docker version of the DTaaS are:

- docker desktop / docker CLI with compose plugin
- [mkcert](https://github.com/FiloSottile/mkcert)

## Clone Codebase

```bash
git clone https://github.com/INTO-CPS-Association/DTaaS.git
cd DTaaS
```

:clipboard: file pathnames

1. The filepaths shown here follow POSIX convention.
The installation procedures also work with Windows
paths.
1. The description below refers to filenames. All the file
paths mentioned below are relatively to the top-level
**DTaaS** directory.

## Configuration

### Create User Workspace

The existing filesystem for installation is setup for `user1`.
A new filesystem directory needs to be created for the selected user.

Please execute the following commands from the top-level directory
of the DTaaS project.

```bash
cp -R files/user1 files/username
```

where _username_ is the selected username to be created (in next steps)
on Gitlab running at <https://localhost/gitlab>.

### Obtain TLS / HTTPS Certificate

Use [mkcert](https://github.com/FiloSottile/mkcert) to generate
TLS certificates using
[this guide](https://kifarunix.com/create-locally-trusted-ssl-certificates-with-mkcert-on-ubuntu-20-04/).
The certificates need to be generated for `localhost`.

The names of the certificates must be `fullchain.pem` and `privkey.pem`.
The `fullchain.pem` corresponds to public certificate and
the `privkey.pem` corresponds to private key.

### Add TLS Certificates to Traefik

Copy the two certificate files into:

- `certs/localhost/fullchain.pem`
- `certs/localhost/privkey.pem`

Traefik will run with self-issued certificates if the above two certificates
are either not found or found invalid.

### Start DTaaS

The commands to start and stop the appliation are:

```bash
docker compose -f compose.local.secure.yml --env-file .env.server up -d
docker compose -f compose.local.secure.yml --env-file .env.server down
```

To restart only a specific container, for example `client``

```bash
docker compose -f compose.server.secure.yml --env-file .env.server up -d --force-recreate client
```

### Start Gitlab

Use the instructions provided in
[gitlab integration](../services/gitlab/README.md) to bring up
Gitlab on localhost and the Gitlab service will be available at <https://localhost/gitlab>

### OAuth2 Application Registration

The Gitlab integration requires authorization setup for frontend website.
The details of Oauth2 app for the frontend website are in
[client docs](../../docs/admin/client/auth.md).

Please see
[gitlab oauth provider](https://docs.gitlab.com/ee/integration/oauth_provider.html)
documentation for further help with creating these two OAuth applications.

### Configure Docker Compose

The docker compose configuration is in `deploy/docker/.env.local`;
it is a sample file.
It contains environment variables
that are used by the docker compose files.
It can be updated to suit your local installation scenario.
It contains the following environment variables.

Edit all the fields according to your specific case.

| URL Path | Example Value | Explanation |
|:------------|:---------------|:---------------|
| DTAAS_DIR | '/home/Desktop/DTaaS' | Full path to the DTaaS directory. This is an absolute path with no trailing slash. |
| username1 | 'user1' | Your gitlab username |
| CLIENT_CONFIG | '/home/Desktop/DTaaS/deploy/config/client/env.local.js' | Full path to env.js file for client |

:clipboard: The path examples given here are for Linux OS.
These paths can be Windows OS compatible paths as well.

### Restart DTaaS

The commands to start and stop the appliation are:

```bash
docker compose -f compose.local.secure.yml --env-file .env.server down
docker compose -f compose.local.secure.yml --env-file .env.server up -d
```

## Use

The application will be accessible at:
<https://localhost> from web browser.
Sign in using your <https://localhost/gitlab> account.

All the functionality of DTaaS should be available to you
through the single page client now.

## Limitations

The [library microservice](../../docs/admin/servers/lib/docker.md) is not
included in the localhost installation scenario.

## References

Image sources:
[Traefik logo](https://www.laub-home.de/wiki/Traefik_SSL_Reverse_Proxy_f%C3%BCr_Docker_Container),
[ml-workspace](https://github.com/ml-tooling/ml-workspace),
[reactjs](https://krify.co/about-reactjs/),
[gitlab](https://gitlab.com)
3 changes: 2 additions & 1 deletion deploy/docker/LOCALHOST.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Install DTaaS on localhost

The installation instructions provided in this README are
ideal for running the DTaaS on both localhost.
ideal for running the **DTaaS on both localhost served over HTTP connection**.

This installation is ideal for single users intending to use
DTaaS on their own computers.

Expand Down
3 changes: 2 additions & 1 deletion deploy/docker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ on both localhost and server.

There are two installation scenarios each with a dedicated set of instructions:

1. Single user [localhost](./LOCALHOST.md)
1. Single user on [insecure localhost](./LOCALHOST.md)
1. Single user on [secure localhost](./LOCALHOST.SECURE.md)
1. Multi-user [web application](./SERVER.md) serving application over **HTTP or HTTPS**

Please choose an appropriate installation scenario.
93 changes: 47 additions & 46 deletions deploy/docker/SERVER.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,9 @@ The installation requirements to run this docker version of the DTaaS are:
### Domain name

The DTaaS software is a web application and is preferably hosted
on a server with a domain name like <http:>_foo.com_</http:>.
on a server with a domain name like _foo.com_.
It is also possible to use an IP address in place of domain name.

### TLS / HTTPS Certificate

It is possible to add HTTPS option to the DTaaS software installation.
Creation of the required TLS certificates is possible through
[certbot](https://certbot.eff.org/).

### OAuth Provider

**[Gitlab Instance](https://about.gitlab.com/install/)** -
Expand All @@ -40,7 +34,8 @@ use [gitlab.com](https://gitlab.com) itself.

Create user accounts in a linked gitlab instance for all the users.

The default docker compose file contains two - _user1_ and _user2_.
The default docker compose configuration file contains
two users - _user1_ and _user2_.
These names need to be changed to suitable usernames.

### OAuth2 Application Registration
Expand Down Expand Up @@ -91,7 +86,7 @@ Three following configuration files need to be updated.
### Docker Compose

The docker compose configuration is in `deploy/docker/.env.server`.
it is a sample file.
It is a sample file.
It contains environment variables
that are used by the docker compose files.
It can be updated to suit your local installation scenario.
Expand Down Expand Up @@ -198,16 +193,13 @@ such routes are not served by traefik; it will give **404 server response**.

## Access Rights Over Files

<!-- markdownlint-disable MD046 -->
<!-- prettier-ignore -->
!!! warning
The default setting in docker compose file exposes
all user files at <http://foo.com/lib/files>.
All files of all the users are readable-writable by
all logged in users.
The `compose.server.yml` / `compose.server.secure.yml` file needs to be
updated to expose another directory like common assets directory.
<!-- markdownlint-enable MD046 -->
:warning:
The default setting in docker compose file exposes
all user files at <http://foo.com/lib/files>.
All files of all the users are readable-writable by
all logged in users.
The `compose.server.yml/compose.server.secure.yml` file needs to be
updated to expose another directory like common assets directory.

If you wish to reduce this scope to only **common assets**,
please change,
Expand All @@ -225,12 +217,42 @@ The change in the last line. The `${DTAAS_DIR}/files`
got replaced by `${DTAAS_DIR}/files/common`. With this change, only
common files are readable-writable by all logged in users.

### Add TLS Certificates
## Serve over HTTP

This docker compose file serves the DTaaS application over HTTP.

The commands to start and stop the appliation are:

```bash
docker compose -f compose.server.yml --env-file .env.server up -d
docker compose -f compose.server.yml --env-file .env.server down
```

To restart only a specific container, for example `client``

```bash
docker compose -f compose.server.yml --env-file .env.server up -d --force-recreate client
```

## Serve Over HTTPS

The extra-steps related TLS certificate are needed only if
DTaaS application is provided over HTTPS connection.

### Obtain TLS / HTTPS Certificate

Obtain the required TLS certificate either through
[certbot](https://certbot.eff.org/) or directly from
online certificate providers.

The application can be served on HTTPS connection for which TLS certificates
are needed. The certificates need to be issued for `foo.com` or `*.foo.com`.
The names of the certificates must be `fullchain.pem` and `privkey.pem`. Copy
these two certificate files into:
The certificates need to be issued for `foo.com` or `*.foo.com`.
The names of the certificates must be `fullchain.pem` and `privkey.pem`.
The `fullchain.pem` corresponds to public certificate and
the `privkey.pem` corresponds to private key.

### Add TLS Certificates to Traefik

Copy the two certificate files into:

- `certs/foo.com/fullchain.pem`
- `certs/foo.com/privkey.pem`
Expand All @@ -252,28 +274,7 @@ tls:
- default
```

## Run

### Over HTTP

This docker compose file serves application over HTTP.

The commands to start and stop the appliation are:

```bash
docker compose -f compose.server.yml --env-file .env.server up -d
docker compose -f compose.server.yml --env-file .env.server down
```

To restart only a specific container, for example `client``

```bash
docker compose -f compose.server.yml --env-file .env.server up -d --force-recreate client
```

### Over HTTPS

This docker compose file serves application over HTTP.
### Run

The commands to start and stop the appliation are:

Expand Down
Empty file.
Loading