Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create script to install The Combine on a standalone system #2976

Merged
merged 112 commits into from
Mar 25, 2024
Merged
Show file tree
Hide file tree
Changes from 107 commits
Commits
Show all changes
112 commits
Select commit Hold shift + click to select a range
5d44cad
Set Kubernetes version in ansible configs
jmgrady Oct 5, 2023
2afb7ef
Add service to write wired ethernet addess to external device
jmgrady Oct 5, 2023
056ab95
Update rancher ui config
jmgrady Oct 5, 2023
fdd94ea
Remove backend init container
jmgrady Oct 5, 2023
0053d9f
WIP: Script to update running version of The Combine
jmgrady Oct 5, 2023
582da0a
Merge branch 'master' into nuc-init
jmgrady Oct 5, 2023
a9b2391
Check for login credentials when a non-released version is requested
jmgrady Oct 5, 2023
42d6fbe
Merge branch 'master' into nuc-init
jmgrady Nov 8, 2023
23ef564
Merge branch 'master' into nuc-init
jmgrady Nov 10, 2023
69cfc9c
Merge branch 'master' into nuc-init
jmgrady Nov 17, 2023
ab46ec1
Install on localhost
jmgrady Nov 21, 2023
e0d8cd8
Add playbook for wifi access point only
jmgrady Nov 21, 2023
3dcb204
Remove microk8s, fix lookup on user/group on localhost
jmgrady Nov 21, 2023
17f4f4a
create playbook for desktop installation
jmgrady Nov 27, 2023
2fc5f87
Comment out docker installation
jmgrady Nov 27, 2023
bddcfb6
Set link_kubeconfige to true for localhost
jmgrady Dec 4, 2023
8a80028
Fix regex for IP address
jmgrady Dec 4, 2023
806d0bf
Remove desktop group vars
jmgrady Dec 4, 2023
d69ed66
Update react from 17 to 18 (#2791)
imnasnainaec Nov 17, 2023
26e9677
[DataCleanup] Implement changes, completed for ReviewEntries (#2743)
imnasnainaec Nov 30, 2023
2d6cbd4
Export lift-ranges semantic domains in other languages (#2783)
imnasnainaec Dec 1, 2023
6503756
Dependabot updates for December 2023 (#2822)
jmgrady Dec 1, 2023
73789f6
Expand DeleteCell testing (#2823)
imnasnainaec Dec 4, 2023
4ed6a04
Organize ReviewEntriesTableState; Tidy other things (#2794)
imnasnainaec Dec 4, 2023
59a40f8
Add local.thecombine.app to list of certificates for offline use (#2825)
jmgrady Dec 4, 2023
d8975e7
Add role to install helm
jmgrady Dec 5, 2023
a7f0b14
Correct install of helm
jmgrady Dec 5, 2023
f6f1447
Add initial helm repo setup
jmgrady Dec 5, 2023
76b1b1f
Update helm_install role
jmgrady Dec 5, 2023
9c8f6ae
Fix unpacking of helm command
jmgrady Dec 5, 2023
09e66d6
Fix target architecture
jmgrady Dec 5, 2023
5ee202d
Fix helm installation
jmgrady Dec 5, 2023
e9b60f8
Merge remote-tracking branch 'origin/standalone-install' into standal…
jmgrady Dec 5, 2023
083b0af
Setup makeself to build installation self-extractor
jmgrady Dec 7, 2023
c00881d
Merge branch 'master' into standalone-install
jmgrady Jan 3, 2024
7d3fe5b
Add combinectl command to installation
jmgrady Jan 22, 2024
fad873b
Add desktop install target
jmgrady Jan 22, 2024
9b3f94e
Improve password prompts
jmgrady Jan 22, 2024
1537bfe
Merge branch 'master' into standalone-install
jmgrady Jan 22, 2024
3bd61b7
Rename "docker_install" role to "container_engine"
jmgrady Feb 11, 2024
32c88a7
Separate hostname from DNS name for The Combine for local install
jmgrady Feb 11, 2024
db2939a
Update script to install the combine
jmgrady Feb 11, 2024
d233a97
change 'public_dns_name' back to 'k8s_dns_name'
jmgrady Feb 11, 2024
fec3f3f
Comment out test of WiFi access point (temporary)
jmgrady Feb 11, 2024
0167eb8
Move install of base helm charts to installation script
jmgrady Feb 11, 2024
3e757b6
Make Kubeconfig file only readable to owner
jmgrady Feb 11, 2024
aeff438
Update repo for installing kubectl
jmgrady Feb 11, 2024
5345e44
Script to manage Combine installation
jmgrady Feb 11, 2024
c565b51
Merge branch 'master' into standalone-install
jmgrady Feb 11, 2024
f9531b0
Improve display of status for The Combine
jmgrady Feb 11, 2024
a4f5da5
Add enabled/disabled info to combinectl status
jmgrady Feb 13, 2024
39785bb
Merge branch 'master' into standalone-install
jmgrady Feb 13, 2024
d7d25de
Restore conditional reboots
jmgrady Feb 13, 2024
84e8c60
Add input arguments; resume from last install step
jmgrady Feb 14, 2024
a348deb
copy install scripts to local storage
jmgrady Feb 14, 2024
fd86571
Only update server in kubeconfig when not local
jmgrady Feb 16, 2024
af75b1b
Build combinectl from template
jmgrady Feb 16, 2024
c812091
Initial readme
jmgrady Feb 17, 2024
5d0454a
Clean up prompting during install
jmgrady Feb 17, 2024
b0db872
Move installed file back to subfolder of home directory
jmgrady Feb 19, 2024
6ce694f
Remove if up/down commands; improve service status messages
jmgrady Feb 20, 2024
2cca83d
Add terminal profiles to developer tools
jmgrady Feb 23, 2024
074df35
Specify natwork management tool
jmgrady Feb 23, 2024
63586aa
Add KUBECONFIG to user profile
jmgrady Feb 23, 2024
acce5d5
Fix missing "
jmgrady Feb 23, 2024
4c4bf8f
Ansible code cleanup
jmgrady Feb 28, 2024
5992dd5
Migrate from netplan to network-manager
jmgrady Feb 28, 2024
4eb5eb1
Fix pattern for WiFi interfaces
jmgrady Feb 28, 2024
d0c417e
Assert that there is only one WiFi interface present
jmgrady Feb 28, 2024
8be4441
Add state to wait for combine cluster to come up
jmgrady Feb 28, 2024
9565fa6
Merge branch 'master' into standalone-install
jmgrady Feb 28, 2024
ff206db
Delete obsolete ReviewEntriesReducer.test.tsx
jmgrady Feb 28, 2024
36191dc
Prettier reformat
jmgrady Feb 28, 2024
c5d94f3
Update setting of ~/.kube/config to support local & remote access
jmgrady Feb 29, 2024
3767c44
Set k8s_user to current for ansible install; remove debugging output
jmgrady Feb 29, 2024
7cbbbea
Rename readme file for installer
jmgrady Mar 7, 2024
85feb93
Improve user environment setup
jmgrady Mar 7, 2024
9dc46a2
set route metrics for virtual IP device
jmgrady Mar 7, 2024
9c41963
Remove unhelpful messages from output when performing a standard inst…
jmgrady Mar 7, 2024
a54f680
Add script for uninstall
jmgrady Mar 7, 2024
1b1b7a7
remove auto & noauto options; restart WiFi when combine is stopped
jmgrady Mar 7, 2024
4c529c9
Clean-up from review
jmgrady Mar 7, 2024
78c60db
Merge branch 'master' into standalone-install
jmgrady Mar 8, 2024
dbf9c0d
Merge branch 'master' into standalone-install
jmgrady Mar 14, 2024
1ce2060
Remove template for sudoer file - no longer used
jmgrady Mar 14, 2024
9567947
Remove shutdown-the-combine function - did too many things not really…
jmgrady Mar 14, 2024
111e820
Add ability to print or customize the WiFi passphrase
jmgrady Mar 14, 2024
09b1ce6
Fill in installation instructions
jmgrady Mar 18, 2024
84f8944
Pass Combine version to installer
jmgrady Mar 18, 2024
b01f674
Add ability to print/set WiFi Password
jmgrady Mar 18, 2024
9678248
Convert README.md to README.pdf
jmgrady Mar 18, 2024
d48f17f
Add 'sudo' to connection delete
jmgrady Mar 18, 2024
aada4b6
Add Desktop configuration for The Combine charts
jmgrady Mar 18, 2024
517801d
Updates to install script
jmgrady Mar 18, 2024
e15b069
Update project readme
jmgrady Mar 18, 2024
c5e6838
Address markdown lint issue
jmgrady Mar 18, 2024
9403b26
move combinectl.sh from templates to files - there are no templated f…
jmgrady Mar 19, 2024
a266d3b
Remove interactive part of make-combine-installer.sh
jmgrady Mar 19, 2024
be324df
Add instructions to make script executable
jmgrady Mar 19, 2024
acadac4
set SSID to thecombine_ap for desktop installations
jmgrady Mar 19, 2024
45ba3a5
Add comments
jmgrady Mar 19, 2024
003f2cd
Remove dead code; add comments to script
jmgrady Mar 19, 2024
b14c3aa
Remove copy of scripts to secondary directory for running
jmgrady Mar 19, 2024
272c196
Add note for Linux versiont that have been tested
jmgrady Mar 20, 2024
362d900
Merge branch 'master' into standalone-install
jmgrady Mar 20, 2024
82337d8
Update from review
jmgrady Mar 21, 2024
6a2f5d4
Updates from review
jmgrady Mar 22, 2024
9d46838
Fix patterns for updating /etc/hosts
jmgrady Mar 22, 2024
91d6278
Turn off install_ip_viewer for localhost
jmgrady Mar 22, 2024
3bb5473
Update from review
jmgrady Mar 24, 2024
5dd7afb
Update comment from review
jmgrady Mar 25, 2024
08a6faa
Merge branch 'master' into standalone-install
jmgrady Mar 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,11 @@ src/resources/dictionaries/*.txt
deploy/scripts/semantic_domains/json/*.json
database/semantic_domains/*

# Combine installer
installer/combine-installer.run
installer/makeself-*
installer/README.pdf

# Kubernetes Configuration files
**/site_files/
**/charts/*.tgz
Expand Down
29 changes: 29 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ A rapid word collection tool. See the [User Guide](https://sillsdev.github.io/Th
6. [Inspect Database](#inspect-database)
7. [Add or Update Dictionary Files](#add-or-update-dictionary-files)
8. [Cleanup Local Repository](#cleanup-local-repository)
9. [Generate Installer Script for The Combine](#generate-installer-script-for-the-combine-linux-only)
3. [Setup Local Kubernetes Cluster](#setup-local-kubernetes-cluster)
1. [Install Rancher Desktop](#install-rancher-desktop)
2. [Install Docker Desktop](#install-docker-desktop)
Expand Down Expand Up @@ -127,6 +128,17 @@ A rapid word collection tool. See the [User Guide](https://sillsdev.github.io/Th
`dotnet tool update --global dotnet-reportgenerator-globaltool --version 5.0.4`
11. [dotnet-project-licenses](https://github.com/tomchavakis/nuget-license)
`dotnet tool update --global dotnet-project-licenses`
12. Tools for generating the self installer (Linux only):

- [makeself](https://makeself.io/) - a tool to make self-extracting archives in Unix
- [pandoc](https://pandoc.org/installing.html#linux) - a tool to convert Markdown documents to PDF.
- `weasyprint` a PDF engine for `pandoc`.

These can be installed on Debian-based distributions by running:

```console
sudo apt install -y makeself pandoc weasyprint
```

### Prepare the Environment

Expand Down Expand Up @@ -505,6 +517,23 @@ of development setup errors. Run from within a Python virtual environment.
python scripts/cleanup_local_repo.py
```

### Generate Installer Script for The Combine (Linux only)

To generate the installer script, run the following commands starting in the project top level directory:

```console
cd installer
./make-combine-installer.sh combine-release-number
```

where `combine-release-number` is the Combine release to be installed, e.g. `v1.2.0`.

To update the PDF copy of the installer README.md file, run the following from the `installer` directory:

```console
pandoc --pdf-engine=weasyprint README.md -o README.pdf
```

## Setup Local Kubernetes Cluster

This section describes how to create a local Kubernetes cluster using either _Rancher Desktop_ or _Docker Desktop_.
Expand Down
14 changes: 12 additions & 2 deletions deploy/ansible/group_vars/nuc/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,17 @@ image_pull_secret: aws-login-credentials
app_namespace: thecombine

k8s_user: sillsdev
k8s_group: sillsdev

################################################
# Helm Installation
################################################
install_helm: no

################################################
# Support Tool Settings
################################################
install_ip_viewer: yes
install_combinectl: yes

#######################################
# Ingress configuration
Expand All @@ -30,7 +40,7 @@ ingress_namespace: ingress-nginx
# The server name will direct traffic to the production
# server since it is used to get the certificates for the
# NUC.
public_dns_name: "{{ ansible_hostname }}"
k8s_dns_name: "{{ ansible_hostname }}"

################################################
# Ethernet settings
Expand Down
11 changes: 11 additions & 0 deletions deploy/ansible/group_vars/server/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,17 @@ create_namespaces: []
# k8s namespaces
app_namespace: thecombine

################################################
# Helm Installation
################################################
install_helm: no

################################################
# Support Tool Settings
################################################
install_ip_viewer: no
install_combinectl: no

#######################################
# Ingress configuration
ingress_namespace: ingress-nginx
Expand Down
62 changes: 62 additions & 0 deletions deploy/ansible/host_vars/localhost/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
#################################################
# Host specific configuration items for localhost
################################################

################################################
# Configure Kubernetes cluster
################################################

# Specify which Kubernetes engine to install -
# one of k3s or none.
k8s_engine: k3s

image_pull_secret: aws-login-credentials

# k8s namespaces
app_namespace: thecombine

k8s_user: "{{ ansible_user_id }}"

################################################
# Helm Installation
################################################
install_helm: yes

################################################
# Support Tool Settings
################################################
install_ip_viewer: yes
install_combinectl: yes

#######################################
# Ingress configuration
ingress_namespace: ingress-nginx

# For the NUCs we want to use the ansible host name
# since that is how we can connect on the local network
# The server name will direct traffic to the production
# server since it is used to get the certificates for the
# NUC.
k8s_dns_name: "local"

################################################
# Ethernet settings
################################################
eth_optional: yes

################################################
# WiFi access point settings
################################################
has_wifi: yes
ap_domain: thecombine.app
ap_ssid: "thecombine_ap"
ap_passphrase: "Combine2020"
ap_gateway: "10.10.10.1"
ap_hostname: "local"
test_wifi: false
################################################
# hardware monitoring settings
################################################
include_hw_monitoring: no
history_days: 60
5 changes: 5 additions & 0 deletions deploy/ansible/hosts.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
---
all:
hosts:
localhost:
ansible_connection: local
kubecfgdir: local
combine_server_name: local.thecombine.app
link_kubeconfig: true
children:
nuc:
hosts:
Expand Down
54 changes: 54 additions & 0 deletions deploy/ansible/playbook_desktop_setup.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
##############################################################
# Playbook: playbook_desktop_setup.yml
#
# playbook_desktop_setup.yml installs the packages and
# configuration files that are required to run TheCombine
# as containers managed by a Kubernetes cluster on localhost.
#
##############################################################

- name: Configure hardware for The Combine
hosts: localhost
gather_facts: yes
become: yes

vars_files:
- "vars/config_common.yml"

tasks:
- name: Update packages
apt:
update_cache: yes
upgrade: "yes"

- name: Setup WiFi Access Point
import_role:
name: wifi_ap
when: has_wifi

- name: Enable hardware monitoring
import_role:
name: monitor_hardware
when: include_hw_monitoring

- name: Configure Network Interfaces
import_role:
name: network_config

- name: Install Container Engine
import_role:
name: container_engine

- name: Install Kubernetes
import_role:
name: k8s_install

- name: Install Helm
import_role:
name: helm_install
when: install_helm

- name: Setup Support Tool
import_role:
name: support_tools
35 changes: 0 additions & 35 deletions deploy/ansible/playbook_dev_tools.yaml

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
---
##############################################################
# Playbook: playbook_kube_install.yml
# Playbook: playbook_nuc_setup.yml
#
# playbook_kube_install.yml installs the packages and
# playbook_nuc_setup.yml installs the packages and
# configuration files that are required to run TheCombine
# as Docker containers managed by a Kubernetes cluster.
# as Docker containers managed by a Kubernetes cluster on
# a target PC, such as an Intel NUC.
#
##############################################################

- name: Configure hardware for The Combine
hosts: all
hosts: nuc
gather_facts: yes
become: yes

vars_files:
- "vars/config_common.yml"
- "vars/packages.yml"

tasks:
- name: Update packages
Expand All @@ -37,16 +37,14 @@
import_role:
name: network_config

- name: Install Docker Subsystem
- name: Install Container Engine
import_role:
name: docker_install
name: container_engine

- name: Install Kubernetes Tools
import_role:
name: k8s_install

- name: Get Kubernetes Configuration
- name: Setup Support Tool
import_role:
name: k8s_config
tags:
- kubeconfig
name: support_tools
3 changes: 3 additions & 0 deletions deploy/ansible/roles/container_engine/defaults/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
---
container_packages:
- containerd.io
Original file line number Diff line number Diff line change
@@ -1,16 +1,13 @@
---
##############################################################
# Role: docker_install
# Role: container_engine
#
# Install the Docker Engine, Docker Compose and all their
# pre-requisite packages.
# Install the container engine and pre-requisite packages.
#
# The Docker Engine is installed by adding the repo from
# The container engine is installed by adding the repo from
# docker.com to our apt sources and installing the relevant
# package.
# packages.
#
# Docker Compose is installed by downloading the ZIP package
# from GitHub and extracting it to /usr/local/bin
##############################################################
- name: Update apt cache.
apt:
Expand All @@ -27,7 +24,6 @@
- gnupg
- lsb-release
state: present
notify: reboot target

- name: Create keyring directory
file:
Expand All @@ -50,6 +46,16 @@

- name: Install Docker Packages
apt:
name: "{{ docker_packages }}"
name: "{{ container_packages }}"
update_cache: yes
notify: reboot target

- name: Check if reboot is required
stat:
path: /var/run/reboot-required
register: reboot_required

- name: Reboot
reboot:
when:
- reboot_required.stat.exists
- ansible_connection != "local"
6 changes: 0 additions & 6 deletions deploy/ansible/roles/docker_install/defaults/main.yml

This file was deleted.

3 changes: 0 additions & 3 deletions deploy/ansible/roles/docker_install/handlers/main.yml

This file was deleted.

5 changes: 5 additions & 0 deletions deploy/ansible/roles/helm_install/defaults/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
helm_version: v3.13.2
helm_arch: linux-amd64

helm_download_dir: /opt/helm-{{ helm_version }}-{{ helm_arch }}
Loading
Loading