Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workflow build Docker image with arm64,amd64 #124

Merged
merged 4 commits into from
Jan 4, 2024

Conversation

Generator
Copy link
Contributor

Summary

  • Fixes [Feature Request] arm64/armv7 docker/ghcr build #117
  • Github Workflow to build docker images for amd64(x86-64) and arm64
  • Github Workflow docker image with tags: latest, master, version, other branches (only dev)
  • Added entrypoint.sh, start with custom UserID/GroupID
  • Run Cabernet as user
  • Rework docker-compose
  • Rework dockerfile
  • Smaller x86-64 image (old 397MB, new 280MB)
  • Reformatted README.md with better instructions for docker
  • Reformatted docker-compose.yml

Notes

You can use single Dockerfile, other files are reluctant.
However is possible to update the app within the container, is recommended for the user update the image instead.
Won't be possible to build for arm(32bit) due to python cryptography system requirements.
Additional files or directories to include on docker must be added to .dockerignore (ex: !filename.ext !somedir/)
Docker images tested on x86-64 and arm64 (Raspberry Pi 4)

Add entrypoint.sh, start with custom UserID/GroupID
Rework docker-compose
Rework dockerfile
@cookieisland
Copy link
Contributor

cookieisland commented Dec 23, 2023

Only mapping the data directory and not the entire cabernet folder will cause problems. The recommendation is to map the entire cabernet source folder to /app. There are multiple reasons for this. Here are two:

  1. Docker does not like a container to change/update itself unless the changes are in a docker volume or in a bind mount. The update/upgrade process built into cabernet modfies not just /app/data but also /app/plugins_ext, /app/lib etc. Eventually these updates will cause the filesystem where docker is installed to run out of disk space as /var/lib/docker/overlay2 keeps growing and growing with every cabernet update. See this link for more on what happens if a docker image updates itself https://buisteven.medium.com/debugging-docker-overlay2-out-of-space-d1edc2ea412f This was discussed some time ago and here is a link to that discussion: Docker instructions are NOT working for cabernet, please help. #60 (comment)

  2. When you pull a new cabernet docker image you will lose your installed plugins because they are not in /app/data. They are in /app/plugins_ext. If you do not map the entire source tree you will need to reinstall your plugins every time you pull a new docker image (and this will make Add automatic docker build from github action #1 above worse because you are again updating the docker image)

I notice you also removed the mapping for /root/.cabernet/key.txt from the docker compose file. This will cause problems for people who have turned on encryption. This file is where the cryptography key is stored when your turn on the cabernet "use_encryption" option. If this file is not mapped via a docker volume or bind mount it will be lost when a new docker image of cabernet is pulled. When you lose this key, cabernet will fail because it will not be able to decrypt the encrypted data after the docker image pull.

@rocky4546
Copy link
Contributor

Agree with cookieisland. encryption key MUST be mounted in a volume and not be lost. If it is lost, anything encrypted would not be useable. Also, the entire Cabernet source must also be mounted in a volume due to the way Docker handles updates.

@rocky4546
Copy link
Contributor

Also, the hostname called "your-ip" is required to have a space. I do not care if its called your ip, my ip, their ip or ip address, it must contain a space. This is to handle those with no brain cells that think "your-ip" is an actual hostname. I had a issue written up on this and now require a space in the hostname area.

@cookieisland
Copy link
Contributor

cookieisland commented Dec 24, 2023

Note that when you switch users from root to cabernet that will likely change the location of the encryption key.txt file. People using the current docker container which runs as the root have their encryption key located at "/root/.cabernet/key.txt"
However when they switch to your proposed docker container their encryption key will likely then be located in the home directory for the user 'cabernet' which I believe would be "/home/cabernet/.cabernet/key.txt". This will cause problems/support issues if the user doesn't modify their docker compose file to remap their key.txt file to the correct new location when they transition from the current docker image to your proposed new docker image.

Backward compatibility for old key.txt path
Additional documentation provided on README.md
@Generator
Copy link
Contributor Author

Generator commented Dec 25, 2023

New commit addressing PR issues

Changes

  • Provided additional documentation on README for properly update docker images and (docker) notes.
  • Docker block updates from WebUI to avoid issues with storage.
  • Docker volumes for the container will be provided if no specified by the user.
  • Revert http://your-ip to http:ip address
  • key.txt is stored on app path /app/.cabernet/key.txt
  • Backward compatibility for old key.txt path

Only mapping the data directory and not the entire cabernet folder will cause problems. The recommendation is to map the entire cabernet source folder to /app
Docker does not like a container to change/update itself unless the changes are in a docker volume or in a bind mount. The update/upgrade process built into cabernet modfies not just /app/data but also /app/plugins_ext, /app/lib etc.

That is incorrect, like a said before the app shouldn't (never) be upgraded inside the container, instead user should update the docker image and restart the container, otherwise will cause issues with the container.
App self updates from WebUI are now blocked by the container to avoid that issue.

When you pull a new cabernet docker image you will lose your installed plugins because they are not in /app/data. They are in /app/plugins_ext.

That was already documented on README and docker-compose.yml, user should provide a Docker Volume or mount bind if want to keep data.

I notice you also removed the mapping for /root/.cabernet/key.txt from the docker compose file.

docker-compose.yml will fail to create the container if the file doesn't exist in first place, with is a file created by the app itself.
New instructions for encryption key path are now provided.

Also, the hostname called "your-ip" is required to have a space. I do not care if its called your ip, my ip, their ip or ip address, it must contain a space. This is to handle those with no brain cells that think "your-ip" is an actual hostname. I had a issue written up on this and now require a space in the hostname area.

You should change your documentation each time a user don't properly read it.
If the documentation is correct and readable, should instruct the user to read it correctly.

PS: The app shouldn't never run as root, specially web-apps or remote access, that's a security risk for the user, and ca be used for privilege escalation

@rocky4546
Copy link
Contributor

rocky4546 commented Dec 25, 2023

Looks really good. Thanks for the updates. Still reviewing the changes, but have one question. The app does not support adhoc updates to the Cabernet source. As an example, if someone were to overlay a different version of cabernet, the app does not try to fix or patch the database or config file to support the version extracted. This means docker upgrades would not apply the patches for the upgrade unless docker also changed the maintenance flag in the config.ini to True. Then the system would apply any patches required to run on the new version.

How did you keep the webui from doing the upgrade? And, how does the software respond? Does it throw exceptions? Concerned that it will also keep the plugins from being able to upgrade...

@rocky4546 rocky4546 changed the base branch from master to dev January 4, 2024 01:41
@rocky4546
Copy link
Contributor

Applying PR to dev branch for testing.

@rocky4546 rocky4546 merged commit 7cac445 into cabernetwork:dev Jan 4, 2024
1 check passed
@Generator Generator deleted the docker-arm branch January 13, 2024 16:15
rocky4546 added a commit that referenced this pull request Jan 29, 2024
* max bandwidth changes

* max bandwidth changes

* max bandwidth changes

* max bandwidth changes

* Workflow build Docker image with arm64,amd64 (#124)

* Workflow build Docker image with arm64,amd64

Add entrypoint.sh, start with custom UserID/GroupID
Rework docker-compose
Rework dockerfile

* Volumes provided for data

Backward compatibility for old key.txt path
Additional documentation provided on README.md

* Dokcerfile install with pip requirements.txt

* Fix key.txt permissions

* Minor changes to Docker files

* Update Dockerfile

* Update docker-compose.yml

* Update docker-compose.yml

* Corrections for backups and upgrades

---------

Co-authored-by: Generator <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request] arm64/armv7 docker/ghcr build
3 participants