Build deb and rpm packages for clients

[bedroge]

Usage (for creating the packages):

ansible-playbook -i hosts -b -K --connection=local ./build-packages.yml

Also see #5.

Only .deb for now. The attached zip file contains a first version of such a deb package.
cvmfs-config-eessi-0.1-all.zip

[bedroge]

If you want to try using the .deb package, just install it using sudo dpkg -i ./cvmfs-config-eessi-0.1-all.deb, and, additionally, create a /etc/cvmfs/default.local with the following contents:

# Use your own proxy:
#CVMFS_HTTP_PROXY="your-proxy.org:3128"
# Or, use it without a proxy and directly connect to the Stratum 1:
CVMFS_HTTP_PROXY="DIRECT"

[boegel]

@bedroge README should be updated accordingly?

Could we get these packages built through GitHub Actions?

[bedroge]

Yes, see #5. ;-)

I haven't updated the README yet, since it was still a WIP and I'm not really sure how/where we should build these packages.

Possibly interesting stuff for this:
https://github.com/saubermacherag/ansible-playbook-docker-action
https://github.com/marketplace/actions/rpm-build
https://github.com/marketplace/actions/build-an-rpm
https://github.com/marketplace/actions/build-debian-package

[bedroge]

It's now building the deb package as a Github action. For now (still testing) storing it as an artifact, see the "Artifcats" dropdown at for instance (top right):
https://github.com/EESSI/filesystem-layer/runs/822951325

[bedroge]

https://github.com/EESSI/filesystem-layer/runs/826552302

It now builds deb and rpm packages (see artifacts).

[bedroge]

RPM does not work yet, though...

[bedroge]

Fixed, the RPM from this build seems to work in a clean CentOS 7 VM:

https://github.com/EESSI/filesystem-layer/runs/826791920?check_suite_focus=true

I used the following installation steps (still have to put it in the README):

sudo yum install https://ecsft.cern.ch/dist/cvmfs/cvmfs-release/cvmfs-release-latest.noarch.rpm
sudo yum install cvmfs cvmfs-config-default cvmfs-auto-setup
sudo yum install ./cvmfs-config-eessi-0.1-1.noarch.rpm
sudo nano /etc/cvmfs/default.local
sudo cvmfs_config setup
sudo cvmfs_config chksetup
ls /cvmfs/cvmfs-config.eessi-hpc.org/

The default.local should have something like:

CVMFS_HTTP_PROXY=DIRECT
CVMFS_QUOTA_LIMIT=4000
CVMFS_LOW_SPEED_LIMIT=10000

[peterstol]

Fixed, the RPM from this build seems to work in a clean CentOS 7 VM:

https://github.com/EESSI/filesystem-layer/runs/826791920?check_suite_focus=true

I used the following installation steps (still have to put it in the README):

sudo yum install https://ecsft.cern.ch/dist/cvmfs/cvmfs-release/cvmfs-release-latest.noarch.rpm
sudo yum install cvmfs cvmfs-config-default cvmfs-auto-setup
sudo yum install ./cvmfs-config-eessi-0.1-1.noarch.rpm
sudo nano /etc/cvmfs/default.local
sudo cvmfs_config setup
sudo cvmfs_config chksetup
ls /cvmfs/cvmfs-config.eessi-hpc.org/

The default.local should have something like:

CVMFS_HTTP_PROXY=DIRECT
CVMFS_QUOTA_LIMIT=4000
CVMFS_LOW_SPEED_LIMIT=10000

I can confirm that the RPM does work, except client install when no proxy is defined fails to set DIRECT.
Setting a proxy in /etc/cvmfs/default.local seems to solve it.

autofs error messages:

Aug 06 13:29:59 node112 cvmfs2[56393]: (cvmfs-config.eessi-hpc.org) failed to fetch file catalog at cvmfs-config.eessi-hpc.org:/ (hash: ...roblem])
Aug 06 13:29:59 node112 cvmfs2[56393]: (cvmfs-config.eessi-hpc.org) Failed to initialize root file catalog (16 - file catalog failure)
Aug 06 13:36:16 node112 cvmfs2[57226]: re-building cache database
Aug 06 13:36:16 node112 cvmfs2[57189]: (cvmfs-config.eessi-hpc.org) failed to resolve IP addresses for your-proxy-1 (8 - unknown name re...g error)
Aug 06 13:36:16 node112 cvmfs2[57189]: (cvmfs-config.eessi-hpc.org) failed to resolve IP addresses for your-proxy-2 (8 - unknown name re...g error)

[boegel]

@peterstol Maybe this explains the problems that @maxim-masterov was mentioning during the EESSI conf call today, when he was playing with the Singularity container image from #10?

[bedroge]

failed to resolve IP addresses for your-proxy-1

I guess there's something wrong in the config, and it ends up with your-proxy-1 being put in some configuration. I'll take a look at this next week.

[bedroge]

@peterstol How exactly did you get this error? I just tried to reproduce it, but without a default.local I simply get errors like:

$ ls /cvmfs/pilot.eessi-hpc.org/
ls: cannot access '/cvmfs/pilot.eessi-hpc.org/': No such file or directory

This your-proxy-1 is not defined in any configuration file. It's only used in the Ansible hosts.example, and may turn up in your configurtion if you run the playbook with an invalid hosts file. Otherwise I don't really see how it's possible that it ended up in your cvmfs configuration...?

[boegel]

@bedroge What's needed to get this merged, and have the *.deb and *.rpm easily downloadable from GitHub (with curl)?

[bedroge]

I think this is ready now, so I removed the WIP tag. For every push/PR to master, this workflow will build the rpm and deb files. If you make a tag, it will take those files, make a release, and add them as release assets.

[bedroge]

One issue which I forgot to mention: I'm using this https://github.com/kentik/pkg for building the packages. It does support aarch64, armv7 (see aarch64, armv7), but they have to be built separately, which I didn't include now/yet. I hope there's some workaround, since there are only text files in these packages...