Support for Ubuntu 20.04 LTS #2843

to-bar · 2022-01-05T11:21:38Z

No description provided.

schema/aws/defaults/infrastructure/virtual-machine.yml

schema/azure/defaults/infrastructure/cloud-os-image-defaults.yml

docs/home/howto/OS_PATCHING.md

to-bar · 2022-01-05T14:18:17Z

Related task: #2848

przemyslavic · 2022-01-10T09:34:45Z

/azp run

przemyslavic · 2022-01-10T11:07:58Z

❌ It fails for new deployments on Ubuntu 20.04 AWS/Azure in online mode.

2022-01-10T10:55:53.8029542Z [38;21m10:55:53 INFO cli.engine.ansible.AnsibleCommand - TASK [common : Install prerequisites for selinux module] ***********************
2022-01-10T10:56:01.9463776Z [31;21m10:56:01 ERROR cli.engine.ansible.AnsibleCommand - FAILED - RETRYING: Install prerequisites for selinux module (3 retries left).
2022-01-10T10:56:01.9465380Z [31;21m10:56:01 ERROR cli.engine.ansible.AnsibleCommand - FAILED - RETRYING: Install prerequisites for selinux module (3 retries left).
2022-01-10T10:56:01.9470833Z [31;21m10:56:01 ERROR cli.engine.ansible.AnsibleCommand - FAILED - RETRYING: Install prerequisites for selinux module (3 retries left).
2022-01-10T10:56:01.9474819Z [31;21m10:56:01 ERROR cli.engine.ansible.AnsibleCommand - FAILED - RETRYING: Install prerequisites for selinux module (3 retries left).
2022-01-10T10:56:01.9485668Z [31;21m10:56:01 ERROR cli.engine.ansible.AnsibleCommand - FAILED - RETRYING: Install prerequisites for selinux module (3 retries left).
2022-01-10T10:56:01.9487203Z [31;21m10:56:01 ERROR cli.engine.ansible.AnsibleCommand - FAILED - RETRYING: Install prerequisites for selinux module (2 retries left).
2022-01-10T10:56:01.9493413Z [31;21m10:56:01 ERROR cli.engine.ansible.AnsibleCommand - FAILED - RETRYING: Install prerequisites for selinux module (2 retries left).
2022-01-10T10:56:01.9500355Z [31;21m10:56:01 ERROR cli.engine.ansible.AnsibleCommand - FAILED - RETRYING: Install prerequisites for selinux module (2 retries left).
2022-01-10T10:56:01.9502587Z [31;21m10:56:01 ERROR cli.engine.ansible.AnsibleCommand - FAILED - RETRYING: Install prerequisites for selinux module (2 retries left).
2022-01-10T10:56:01.9511787Z [31;21m10:56:01 ERROR cli.engine.ansible.AnsibleCommand - FAILED - RETRYING: Install prerequisites for selinux module (2 retries left).
2022-01-10T10:56:01.9513307Z [31;21m10:56:01 ERROR cli.engine.ansible.AnsibleCommand - FAILED - RETRYING: Install prerequisites for selinux module (1 retries left).
2022-01-10T10:56:01.9514476Z [31;21m10:56:01 ERROR cli.engine.ansible.AnsibleCommand - FAILED - RETRYING: Install prerequisites for selinux module (1 retries left).
2022-01-10T10:56:01.9522236Z [31;21m10:56:01 ERROR cli.engine.ansible.AnsibleCommand - FAILED - RETRYING: Install prerequisites for selinux module (1 retries left).
2022-01-10T10:56:01.9527495Z [31;21m10:56:01 ERROR cli.engine.ansible.AnsibleCommand - FAILED - RETRYING: Install prerequisites for selinux module (1 retries left).
2022-01-10T10:56:01.9534034Z [31;21m10:56:01 ERROR cli.engine.ansible.AnsibleCommand - FAILED - RETRYING: Install prerequisites for selinux module (1 retries left).
2022-01-10T10:56:01.9541224Z [31;21m10:56:01 ERROR cli.engine.ansible.AnsibleCommand - fatal: [ec2-1-1-1-1.eu-west-1.compute.amazonaws.com]: FAILED! => {"attempts": 3, "changed": false, "msg": "No package matching 'python3-selinux' is available"}

All the packages are in /var/www/html/epirepo/packages/ directory:

ubuntu@ip-10-1-11-73:~$ ls /var/www/html/epirepo/packages/python3-selinux*
/var/www/html/epirepo/packages/python3-selinux_3.0-1build2_amd64.deb

The issue is caused by the fact that epirepo repository is not created in online mode.

ubuntu@ip-10-1-11-251:~$ ls /var/www/html/epirepo/packages/python*
/var/www/html/epirepo/packages/python-apt-common_2.0.0ubuntu0.20.04.6_all.deb
/var/www/html/epirepo/packages/python3-cffi-backend_1.14.0-1build1_amd64.deb
/var/www/html/epirepo/packages/python3-cryptography_2.8-3ubuntu0.1_amd64.deb
/var/www/html/epirepo/packages/python3-dbus_1.2.16-1build1_amd64.deb
/var/www/html/epirepo/packages/python3-decorator_4.4.2-0ubuntu1_all.deb
/var/www/html/epirepo/packages/python3-firewall_0.8.2-1_all.deb
/var/www/html/epirepo/packages/python3-gi_3.36.0-1_amd64.deb
/var/www/html/epirepo/packages/python3-jmespath_0.9.4-2ubuntu1_all.deb
/var/www/html/epirepo/packages/python3-ldb_2%3a2.2.3-0ubuntu0.20.04.2_amd64.deb
/var/www/html/epirepo/packages/python3-nftables_0.9.3-2_amd64.deb
/var/www/html/epirepo/packages/python3-psycopg2_2.8.6-2~pgdg20.04+1_amd64.deb
/var/www/html/epirepo/packages/python3-requests-unixsocket_0.2.0-2_all.deb
/var/www/html/epirepo/packages/python3-selinux_3.0-1build2_amd64.deb
/var/www/html/epirepo/packages/python3-six_1.14.0-2_all.deb
/var/www/html/epirepo/packages/python3-slip-dbus_0.6.5-2_all.deb
/var/www/html/epirepo/packages/python3-slip_0.6.5-2_all.deb
/var/www/html/epirepo/packages/python3-software-properties_0.99.9.8_all.deb
/var/www/html/epirepo/packages/python3-talloc_2.3.1-0ubuntu0.20.04.1_amd64.deb
/var/www/html/epirepo/packages/python3_3.8.2-0ubuntu2_amd64.deb

ubuntu@ip-10-1-11-251:~$ sudo apt-cache policy python3-selinux
N: Unable to locate package python3-selinux

ubuntu@ip-10-1-11-251:~$ grep Package /var/lib/apt/lists/10.1.11.251_epirepo_packages_._Packages

ubuntu@ip-10-1-11-251:~$ sudo /var/tmp/epi-repository-setup-scripts/create-repository.sh /var/www/html/epirepo/ false
Installing apache2 package...
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.41-4ubuntu3.9).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Generating repository metadata...
Can't locate Dpkg.pm in @INC (you may need to install the Dpkg module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.30.0 /usr/local/share/perl/5.30.0 /usr/lib/x86_64-linux-gnu/perl5/5.30 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.30 /usr/share/perl/5.30 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /var/tmp/epi-repository-setup-scripts/dpkg-scanpackages line 27.
BEGIN failed--compilation aborted at /var/tmp/epi-repository-setup-scripts/dpkg-scanpackages line 27.

And it works fine in offline-mode

ubuntu@ip-10-1-11-251:~$ sudo /var/tmp/epi-repository-setup-scripts/create-repository.sh /var/www/html/epirepo/ true
Package libdpkg-perl not found, installing...
Selecting previously unselected package libdpkg-perl.
(Reading database ... 64621 files and directories currently installed.)
Preparing to unpack .../libdpkg-perl_1.19.7ubuntu3_all.deb ...
Unpacking libdpkg-perl (1.19.7ubuntu3) ...
Setting up libdpkg-perl (1.19.7ubuntu3) ...
Processing triggers for man-db (2.9.1-1) ...
Generating repository metadata...
dpkg-scanpackages: info: Wrote 303 entries to output Packages file.
Updating list of available packages...
Get:1 file:/var/www/html/epirepo//packages ./ InRelease
Ign:1 file:/var/www/html/epirepo//packages ./ InRelease
Get:2 file:/var/www/html/epirepo//packages ./ Release
Ign:2 file:/var/www/html/epirepo//packages ./ Release
Get:3 file:/var/www/html/epirepo//packages ./ Packages
Ign:3 file:/var/www/html/epirepo//packages ./ Packages
Get:4 file:/var/www/html/epirepo//packages ./ Translation-en
Ign:4 file:/var/www/html/epirepo//packages ./ Translation-en
Ign:5 http://10.1.11.251/epirepo/packages ./ InRelease
Get:3 file:/var/www/html/epirepo//packages ./ Packages
Ign:3 file:/var/www/html/epirepo//packages ./ Packages
Ign:6 http://10.1.11.251/epirepo/packages ./ Release
Get:4 file:/var/www/html/epirepo//packages ./ Translation-en
Ign:4 file:/var/www/html/epirepo//packages ./ Translation-en
Get:3 file:/var/www/html/epirepo//packages ./ Packages
Ign:3 file:/var/www/html/epirepo//packages ./ Packages
Ign:7 http://10.1.11.251/epirepo/packages ./ Packages
Get:4 file:/var/www/html/epirepo//packages ./ Translation-en
Ign:4 file:/var/www/html/epirepo//packages ./ Translation-en
Ign:8 http://10.1.11.251/epirepo/packages ./ Translation-en
Ign:7 http://10.1.11.251/epirepo/packages ./ Packages
Ign:8 http://10.1.11.251/epirepo/packages ./ Translation-en
Get:3 file:/var/www/html/epirepo//packages ./ Packages [99.7 kB]
Get:4 file:/var/www/html/epirepo//packages ./ Translation-en
Ign:4 file:/var/www/html/epirepo//packages ./ Translation-en
Ign:7 http://10.1.11.251/epirepo/packages ./ Packages
Get:4 file:/var/www/html/epirepo//packages ./ Translation-en
Ign:4 file:/var/www/html/epirepo//packages ./ Translation-en
Ign:8 http://10.1.11.251/epirepo/packages ./ Translation-en
Get:4 file:/var/www/html/epirepo//packages ./ Translation-en
Ign:4 file:/var/www/html/epirepo//packages ./ Translation-en
Get:7 http://10.1.11.251/epirepo/packages ./ Packages [99.7 kB]
Get:4 file:/var/www/html/epirepo//packages ./ Translation-en
Ign:4 file:/var/www/html/epirepo//packages ./ Translation-en
Ign:8 http://10.1.11.251/epirepo/packages ./ Translation-en
Ign:8 http://10.1.11.251/epirepo/packages ./ Translation-en
Ign:8 http://10.1.11.251/epirepo/packages ./ Translation-en
Ign:8 http://10.1.11.251/epirepo/packages ./ Translation-en
Fetched 99.7 kB in 0s (3143 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
All packages are up to date.
Installing apache2 package...
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.41-4ubuntu3.9).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Removing temporary repo definition: /etc/apt/sources.list.d/epilocal.list...
Updating list of available packages...
Ign:1 http://10.1.11.251/epirepo/packages ./ InRelease
Ign:2 http://10.1.11.251/epirepo/packages ./ Release
Ign:3 http://10.1.11.251/epirepo/packages ./ Packages
Ign:4 http://10.1.11.251/epirepo/packages ./ Translation-en
Ign:3 http://10.1.11.251/epirepo/packages ./ Packages
Ign:4 http://10.1.11.251/epirepo/packages ./ Translation-en
Ign:3 http://10.1.11.251/epirepo/packages ./ Packages
Ign:4 http://10.1.11.251/epirepo/packages ./ Translation-en
Hit:3 http://10.1.11.251/epirepo/packages ./ Packages
Ign:4 http://10.1.11.251/epirepo/packages ./ Translation-en
Ign:4 http://10.1.11.251/epirepo/packages ./ Translation-en
Ign:4 http://10.1.11.251/epirepo/packages ./ Translation-en
Ign:4 http://10.1.11.251/epirepo/packages ./ Translation-en
Reading package lists... Done
Building dependency tree
Reading state information... Done
All packages are up to date.

ubuntu@ip-10-1-11-251:~$ grep Package /var/lib/apt/lists/10.1.11.251_epirepo_packages_._Packages
Package: adduser
Package: anacron
Package: apache2
Package: apache2-bin
Package: apache2-data
Package: apache2-utils
Package: apt
Package: apt-transport-https
Package: auditd
...

ubuntu@ip-10-1-11-251:~$ sudo apt-cache policy python3-selinux
python3-selinux:
  Installed: (none)
  Candidate: 3.0-1build2
  Version table:
     3.0-1build2 500
        500 http://10.1.11.251/epirepo/packages ./ Packages

I suspect libdpkg-perl package should be installed first:

ubuntu@ip-10-1-11-5:~$ sudo dpkg -i /var/www/html/epirepo/packages/libdpkg-perl_1.19.7ubuntu3_all.deb
Selecting previously unselected package libdpkg-perl.
(Reading database ... 64621 files and directories currently installed.)
Preparing to unpack .../libdpkg-perl_1.19.7ubuntu3_all.deb ...
Unpacking libdpkg-perl (1.19.7ubuntu3) ...
Setting up libdpkg-perl (1.19.7ubuntu3) ...
Processing triggers for man-db (2.9.1-1) ...

ubuntu@ip-10-1-11-5:~$ sudo /var/tmp/epi-repository-setup-scripts/create-repository.sh /var/www/html/epirepo/ false
Installing apache2 package...
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.41-4ubuntu3.9).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Generating repository metadata...
dpkg-scanpackages: info: Wrote 303 entries to output Packages file.

ubuntu@ip-10-1-11-5:~$ sudo apt-get update
Ign:1 http://10.1.11.5/epirepo/packages ./ InRelease
Ign:2 http://10.1.11.5/epirepo/packages ./ Release
Ign:3 http://10.1.11.5/epirepo/packages ./ Packages
Ign:4 http://10.1.11.5/epirepo/packages ./ Translation-en
Ign:3 http://10.1.11.5/epirepo/packages ./ Packages
Ign:4 http://10.1.11.5/epirepo/packages ./ Translation-en
Ign:3 http://10.1.11.5/epirepo/packages ./ Packages
Ign:4 http://10.1.11.5/epirepo/packages ./ Translation-en
Get:3 http://10.1.11.5/epirepo/packages ./ Packages [99.7 kB]
Ign:4 http://10.1.11.5/epirepo/packages ./ Translation-en
Ign:4 http://10.1.11.5/epirepo/packages ./ Translation-en
Ign:4 http://10.1.11.5/epirepo/packages ./ Translation-en
Ign:4 http://10.1.11.5/epirepo/packages ./ Translation-en
Fetched 99.7 kB in 0s (3989 kB/s)
Reading package lists... Done

ubuntu@ip-10-1-11-5:~$ grep Package /var/lib/apt/lists/10.1.11.5_epirepo_packages_._Packages
Package: adduser
Package: anacron
Package: apache2
Package: apache2-bin
...

przemyslavic · 2022-01-10T19:31:05Z

Also worth checking why the task that runs create epirepo shell script has not failed.

Edit: Exit code is 0. 😞

ubuntu@ip-10-1-11-5:~$ sudo /var/tmp/epi-repository-setup-scripts/create-repository.sh /var/www/html/epirepo/ false
Installing apache2 package...
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.41-4ubuntu3.9).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Generating repository metadata...
Can't locate Dpkg.pm in @INC (you may need to install the Dpkg module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.30.0 /usr/local/share/perl/5.30.0 /usr/lib/x86_64-linux-gnu/perl5/5.30 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.30 /usr/share/perl/5.30 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /var/tmp/epi-repository-setup-scripts/dpkg-scanpackages line 27.
BEGIN failed--compilation aborted at /var/tmp/epi-repository-setup-scripts/dpkg-scanpackages line 27.
ubuntu@ip-10-1-11-5:~$ echo $?
0

romsok24 · 2022-01-11T09:47:21Z

A nice job done here.
The code changes LGTM and I like this check because it has caused my builds to fail quite often because of this particular reason (apt lock).

przemyslavic · 2022-01-11T12:39:58Z

/azp run

przemyslavic reviewed Jan 5, 2022

View reviewed changes

schema/aws/defaults/infrastructure/virtual-machine.yml Outdated Show resolved Hide resolved

przemyslavic reviewed Jan 5, 2022

View reviewed changes

schema/azure/defaults/infrastructure/cloud-os-image-defaults.yml Outdated Show resolved Hide resolved

seriva reviewed Jan 5, 2022

View reviewed changes

docs/home/howto/OS_PATCHING.md Outdated Show resolved Hide resolved

seriva reviewed Jan 5, 2022

View reviewed changes

docs/home/howto/OS_PATCHING.md Outdated Show resolved Hide resolved

to-bar force-pushed the feature/ubuntu-upgrade branch from 896d8ec to e7442e2 Compare January 5, 2022 14:48

seriva previously approved these changes Jan 10, 2022

View reviewed changes

to-bar dismissed seriva’s stale review via dbb40ea January 11, 2022 14:07

to-bar force-pushed the feature/ubuntu-upgrade branch from e7442e2 to dbb40ea Compare January 11, 2022 14:07

przemyslavic mentioned this pull request Jan 11, 2022

[FEATURE REQUEST] Update cloud OS images to the latest #2856

Closed

13 tasks

to-bar force-pushed the feature/ubuntu-upgrade branch from 938612b to 0e739b2 Compare January 12, 2022 09:59

to-bar added 14 commits January 12, 2022 14:50

Update OS images

001949d

Ubuntu 18.04 -> 20.04

16732dc

Fail if error occured when downloading package

28c797c

Switch to python3-* packages and remove unused ones

1d0306e

Remove python3-setuptools

b7d900c

Update OS image (Azure)

263f70c

Do not use postgresql_query in preflight tasks

d7740fe

Ensure python3-psycopg2 is installed

aa6517f

Set ansible_python_interpreter to 'auto'

681f342

Set Python interpreter only in ansible.cfg

d6fe654

Add upgrade playbook for CI pipelines

bd8f087

Update dpkg-scanpackages to 1.19.7

3174be9

Add libdpkg-perl

ed65bda

Do not require dpkg-dev

6c2942d

to-bar and others added 10 commits January 12, 2022 14:50

Disable auto-upgrades on upgrade time

d86ae97

Update changelog

bc1e7ad

Add doc about upgrading Ubuntu release

4e6831c

Update OS images

666f2dd

Fail when dpkg-scanpackages fails

f8aa59e

Install libdpkg-perl also in online mode

b929297

Update OS images

dab0fef

UsePrivilegeSeparation is deprecated

19a1e33

Fix condition for UsePrivilegeSeparation

feb5083

Update postgres molecule tests

9ce6669

to-bar force-pushed the feature/ubuntu-upgrade branch from f8bf610 to 9ce6669 Compare January 12, 2022 14:52

przemyslavic approved these changes Jan 12, 2022

View reviewed changes

seriva approved these changes Jan 12, 2022

View reviewed changes

to-bar merged commit 650185a into develop Jan 12, 2022

to-bar deleted the feature/ubuntu-upgrade branch January 12, 2022 22:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for Ubuntu 20.04 LTS #2843

Support for Ubuntu 20.04 LTS #2843

to-bar commented Jan 5, 2022

to-bar commented Jan 5, 2022

przemyslavic commented Jan 10, 2022

przemyslavic commented Jan 10, 2022 •

edited

Loading

przemyslavic commented Jan 10, 2022 •

edited

Loading

romsok24 commented Jan 11, 2022

przemyslavic commented Jan 11, 2022

Support for Ubuntu 20.04 LTS #2843

Support for Ubuntu 20.04 LTS #2843

Conversation

to-bar commented Jan 5, 2022

to-bar commented Jan 5, 2022

przemyslavic commented Jan 10, 2022

przemyslavic commented Jan 10, 2022 • edited Loading

przemyslavic commented Jan 10, 2022 • edited Loading

romsok24 commented Jan 11, 2022

przemyslavic commented Jan 11, 2022

przemyslavic commented Jan 10, 2022 •

edited

Loading

przemyslavic commented Jan 10, 2022 •

edited

Loading