Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEATURE REQUEST] Ensure epicli upgrade works on cluster with upgraded RHEL from version 7 to 8 #2932

Closed
7 of 20 tasks
to-bar opened this issue Jan 26, 2022 · 2 comments
Closed
7 of 20 tasks

[FEATURE REQUEST] Ensure epicli upgrade works on cluster with upgraded RHEL from version 7 to 8 #2932

to-bar opened this issue Jan 26, 2022 · 2 comments
Assignees
@przemyslavic @to-bar

Comments

@to-bar
Copy link
Contributor

to-bar commented Jan 26, 2022 •
edited by seriva
Loading

Is your feature request related to a problem? Please describe.
We need to ensure that epicli upgrade works well on cluster initially deployed on VMs based on RHEL 7 that was upgraded to version 8.

Additional context
Can be picked up after #1332

DoD checklist

  • Changelog
    • updated
    • not needed
  • COMPONENTS.md
    • updated
    • not needed
  • Schema
    • updated
    • not needed
  • Backport tasks
    • created
    • not needed
  • Documentation
    • added
    • updated
    • not needed
  • Feature has automated tests
  • Automated tests passed (QA pipelines)
    • apply
    • upgrade
    • backup/restore
  • Idempotency tested
  • All conversations in PR resolved
  • Solution meets requirements and is done according to design doc
  • Usage compliant with license
@to-bar
Copy link
Contributor Author

to-bar commented Apr 14, 2022

Sample issues to handle:

  1. No Postgresql after upgrading RHEL 7 to 8:

[postgresql-vm-0]: FAILED! => {"changed": false, "cmd": "psql -t -c 'SHOW archive_command'", "msg": "[Errno 2] No such file or directory: b'psql': b'psql'", "rc": 2, "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines"}

  1. containerd related errors during upgrade:

containerd[1007]: time="2022-04-14T15:36:35Z" level=warning msg="deprecated version : 1, please switch to version 2"

# grep FATAL /var/log/messages
Apr 14 14:26:39 ip-10-1-11-87 containerd: time="2022-04-14T14:26:39.459578727Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.aufs\"..." error="aufs is not supported (modprobe aufs failed: exit status 1 \"modprobe: FATAL: Module aufs not found.\\n\"): skip plugin" type=io.containerd.snapshotter.v1
Apr 14 15:15:00 ec2-52-47-128-68 containerd: time="2022-04-14T15:15:00.835462746Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.aufs\"..." error="aufs is not supported (modprobe aufs failed: exit status 1 \"modprobe: FATAL: Module aufs not found.\\n\"): skip plugin" type=io.containerd.snapshotter.v1
Apr 14 15:36:35 ec2-52-47-128-68 containerd[1007]: time="2022-04-14T15:36:35.895847002Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.aufs\"..." error="aufs is not supported (modprobe aufs failed: exit status 1 \"modprobe: FATAL: Module aufs not found in directory /lib/modules/4.18.0-305.25.1.el8_4.x86_64\\n\"): skip plugin" type=io.containerd.snapshotter.v1

@seriva seriva changed the title Ensure epicli upgrade works on cluster with upgraded RHEL from version 7 to 8 [FEATURE REQUEST] Ensure epicli upgrade works on cluster with upgraded RHEL from version 7 to 8 Apr 21, 2022
@to-bar to-bar self-assigned this Jun 21, 2022
@przemyslavic
Copy link
Collaborator

I encountered an issue upgrading from v1.0 in offline mode.
It failed on task Create epirepo repository.

2022-06-27T12:54:23.1932911Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - TASK [repository : Create epirepo repository] **********************************
2022-06-27T12:54:23.2219931Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - skipping: [ci-10u2devazrhflannel-kubernetes-master-vm-0]
2022-06-27T12:54:23.2452975Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - skipping: [ci-10u2devazrhflannel-kubernetes-node-vm-0]
2022-06-27T12:54:23.2690775Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - skipping: [ci-10u2devazrhflannel-kubernetes-node-vm-1]
2022-06-27T12:54:23.2907313Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - skipping: [ci-10u2devazrhflannel-kubernetes-node-vm-2]
2022-06-27T12:54:23.3113551Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - skipping: [ci-10u2devazrhflannel-logging-vm-0]
2022-06-27T12:54:23.3339422Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - skipping: [ci-10u2devazrhflannel-logging-vm-1]
2022-06-27T12:54:23.3546965Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - skipping: [ci-10u2devazrhflannel-logging-vm-2]
2022-06-27T12:54:23.3751428Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - skipping: [ci-10u2devazrhflannel-monitoring-vm-0]
2022-06-27T12:54:23.4030731Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - skipping: [ci-10u2devazrhflannel-kafka-vm-0]
2022-06-27T12:54:23.4244711Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - skipping: [ci-10u2devazrhflannel-kafka-vm-1]
2022-06-27T12:54:23.4687702Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - skipping: [ci-10u2devazrhflannel-kafka-vm-2]
2022-06-27T12:54:23.4906962Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - skipping: [ci-10u2devazrhflannel-postgresql-vm-0]
2022-06-27T12:54:23.5130781Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - skipping: [ci-10u2devazrhflannel-postgresql-vm-1]
2022-06-27T12:54:23.5500125Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - skipping: [ci-10u2devazrhflannel-load-balancer-vm-0]
2022-06-27T12:54:23.5567150Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - skipping: [ci-10u2devazrhflannel-rabbitmq-vm-0]
2022-06-27T12:54:23.5765232Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - skipping: [ci-10u2devazrhflannel-rabbitmq-vm-1]
2022-06-27T12:54:23.5995539Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - skipping: [ci-10u2devazrhflannel-opendistro-for-elasticsearch-vm-0]
2022-06-27T12:54:23.6003033Z[38;21m12:54:23 INFO cli.src.ansible.AnsibleCommand - skipping: [ci-10u2devazrhflannel-opendistro-for-elasticsearch-vm-1]
2022-06-27T12:54:28.1546370Z[31;21m12:54:28 ERROR cli.src.ansible.AnsibleCommand - fatal: [ci-10u2devazrhflannel-repository-vm-0]: FAILED! => {"changed": true, "cmd": "set -o pipefail && /var/tmp/epi-repository-setup-scripts/create-repository.sh /var/www/html/epirepo true |& tee /var/tmp/epi-repository-setup-scripts/create-repository.log", "delta": "0:00:03.501168", "end": "2022-06-27 12:54:28.030186", "msg": "non-zero return code", "rc": 1, "start": "2022-06-27 12:54:24.529018", "stderr": "", "stderr_lines": [], "stdout": "Package apr-util-1.6.1-6.el8.x86_64 is already installed.
Package apr-util-bdb-1.6.1-6.el8.x86_64 is already installed.
Package apr-util-openssl-1.6.1-6.el8.x86_64 is already installed.
Package drpm-0.4.1-3.el8.x86_64 is already installed.
Package mailcap-2.1.48-3.el8.noarch is already installed.
Error: 
 Problem: cannot install both createrepo_c-libs-0.17.7-5.el8.x86_64 and createrepo_c-libs-0.16.2-2.el8.x86_64
  - package python3-createrepo_c-0.16.2-2.el8.x86_64 requires createrepo_c-libs = 0.16.2-2.el8, but none of the providers can be installed
  - conflicting requests
  - problem with installed package python3-createrepo_c-0.16.2-2.el8.x86_64
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)", "stdout_lines": ["Package apr-util-1.6.1-6.el8.x86_64 is already installed.", "Package apr-util-bdb-1.6.1-6.el8.x86_64 is already installed.", "Package apr-util-openssl-1.6.1-6.el8.x86_64 is already installed.", "Package drpm-0.4.1-3.el8.x86_64 is already installed.", "Package mailcap-2.1.48-3.el8.noarch is already installed.", "Error: ", " Problem: cannot install both createrepo_c-libs-0.17.7-5.el8.x86_64 and createrepo_c-libs-0.16.2-2.el8.x86_64", "  - package python3-createrepo_c-0.16.2-2.el8.x86_64 requires createrepo_c-libs = 0.16.2-2.el8, but none of the providers can be installed", "  - conflicting requests", "  - problem with installed package python3-createrepo_c-0.16.2-2.el8.x86_64", "(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)"]}

To reproduce:

  1. Deploy any RHEL 7 cluster
  2. Upgrade OS to RHEL 8
  3. Disable outbound cluster traffic to the Internet
  4. Download the requirements for offline mode
  5. Execute epicli upgrade -b /path/to/build/directory --offline-requirements /path/to/downloaded/requirements/

@przemyslavic przemyslavic self-assigned this Jul 13, 2022
@seriva seriva closed this as completed Jul 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@przemyslavic przemyslavic

@to-bar to-bar

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@seriva @przemyslavic @to-bar