Skip to content

Testing Debian Packaging

Jump to bottom Edit New page
vjrj edited this page Jun 29, 2022 · 28 revisions

This page describes the testing of Debian packaging system of ALA modules.

ALA Debianizer

We are using this template system for help in the generation of Debian Living Atlases debian packages:

Testing branchs

We have published some work-in-progress ALA modules debianized:

Using demo repository

The above modules are build and tested via this demo repo (generated via aptly), and for now, dummy gpg key:

echo "deb [arch=amd64] https://apt.gbif.es/ bionic main" > /etc/apt/sources.list.d/ala.list
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F697D8D2ADB9E24A
apt update

Testing ala-collectory service

After configuring the previous demo repository you can in ubuntu xenial ala-collectory without db:

apt install ala-collectory dbconfig-no-thanks

or if you want to install also mysql and to create an empty collectory db in the same server:

apt install ala-collectory

To see the package installation status and contents:

dpkg -l ala-collectory
dpkg -L ala-collectory

Note: You should tune /etc/default/tomcat7 memory opts as ala-install does.

After installing, with the default configuration, you will have the service running in http://localhost:8080/ala-collectory/.

In ubuntu/bionic you should add xenial repositories as ala-install does (for install tomcat7 and similar dependencies).

Testing ala-image service

For instance for ubuntu/xenial:

# Add postgresql repos
echo "deb http://apt.postgresql.org/pub/repos/apt/ xenial-pgdg main" > /etc/apt/sources.list.d/postgresql.list 
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 7FCC7D46ACCC4CF8
apt install ala-image
# or if you want to install without db for use a external container/vm with the postgresql database
apt install ala-image dbconfig-no-thanks 
# You can check the service with:
service ala-image status

The image service will be running in port http://localhost:9003.

The service require the elacticsearch service to work properly.

Testing ala-cas-5 package

apt install ala-cas-5
# You can check the service with:
service ala-cas-5 status

Note: this cas package is not totally updated and finished.

Testing ala-i18n package

A new package following this proposal.

apt install ala-i18n
dpkg -L ala-i18n

Testing la-pipelines package

Prerequisites:

apt install la-pipelines
dpkg -L la-pipelines

Other WIP packages:

Testing Removing

apt remove ala-image ala-collectory

Ansible integration

One approach is to use this debconf module to configure each package configuration variables, but probably this issue should be interesting also for us.

Usage in docker images

You can preseed this package with something similar to (if this package uses mysql):

ENV DEBIAN_FRONTEND noninteractive
ENV DB_MYSQL_ROOT_PASS somemysqlpassw
ENV DB_COL_USER collectory
ENV DB_COL_PASS somepass
(...)
RUN  echo "ala-collectory ala-collectory/mysql/admin-pass password $DB_MYSQL_ROOT_PASS" | debconf-set-selections && \
     echo "ala-collectory ala-collectory/mysql/app-pass password $DB_COL_PASS" | debconf-set-selections && \
     echo "ala-collectory ala-collectory/db/app-user string $DB_COL_USER" | debconf-set-selections && \
     echo "ala-collectory ala-collectory/dbconfig-install boolean true" | debconf-set-selections && \
     echo "ala-collectory ala-collectory/dbconfig-upgrade boolean true" | debconf-set-selections

To check these values in debian debconf database about some package like ala-collectory (for instance, to preseed it in Docker or to verify it) install that package in a test server or in a image and get the previous values with:

debconf-get-selections | grep ala-collectory

Other possibility is to install dbconfig-no-thanks instead of asking for db user and passwords. Useful if you have your db in other docker container or VM.

Create your own apt repository

We used aptly for our testing repository but there are some other repository management alternatives.

We used this tutorial for that configuring aptly but without using S3, just a simple vhost configuration in our webserver.

So we do something like this to publish new packages:

aptly repo add unstable ala-image_1.0.6-7_all.deb
aptly snapshot create unstable-ai1067 from repo unstable
aptly publish switch bionic unstable-ai1067

gpg key generation for package signing

You will need a gpg key to sign that packages. Before generating it add to your: .gnupg/gpg.conf:

personal-digest-preferences SHA256
cert-digest-algo SHA256

so you'll generate a gpg key compatible with apt.

Build a debian package using docker (useful for Mac and other distro users)

If for some reason, you need to build a deb package and you are not using debian or ubuntu as development environment, you can use these steps:

# Clone this repo:
git clone --depth=1 https://github.com/tsaarni/docker-deb-builder
cd docker-deb-builder 
# Create an image with the desired ubuntu version
docker build -t docker-deb-builder:20.04 -f Dockerfile-ubuntu-20.04 .
# Build your deb for some ALA debian source
./build -i docker-deb-builder:20.04 -o /tmp /home/your-user/ala-sensitive-data-service/
# You'll build the deb package in /tmp
ls -l /tmp/*deb                                                                                       
-rw-r--r-- 1 your-user your-user 56893756 jun 29 08:02 /tmp/ala-sensitive-data-service_1.1-1_all.deb

For pipelines you'll need an extra step: to create the link described in https://github.com/gbif/pipelines/blob/dev/livingatlas/debian/README.md#build This is because we don't have the debian directory in the standard location, the root of the code, as we only package the living-atlases pipelines part.

End

Index

Clone this wiki locally