Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migration to Python3 #2839

Merged
merged 13 commits into from
Nov 27, 2020
Merged

Migration to Python3 #2839

merged 13 commits into from
Nov 27, 2020

Commits on Nov 27, 2020

  1. salt: In downgrade orchestrate use metalk8s.cmp_sorted 

    With Python3 we can no longer use `cmp` in `sort` and `sorted` function
so use our salt module `metalk8s.cmp_sorted` in Downgrade orchestrate if
we are running with Python3, so that it works well with Python2 and
Pyton3

Refs: #2203
    @TeddyAndrieux
    TeddyAndrieux committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    7655d6f View commit details
    Browse the repository at this point in the history

  2. tests,docs: Use salt-ssh --raw-shell command before expansions 

    In the past a `salt-ssh` command was needed prior to start the
expansions to accept the ssh key but it's not longer needed so remove
it.
Instead do a simple `salt-ssh --raw-shell` command to check that
salt-ssh work properly
    @TeddyAndrieux
    TeddyAndrieux committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    6577d2c View commit details
    Browse the repository at this point in the history

  3. salt: Add a states to execute a function on a minion with raw ssh 

    Right now `salt.function` salt state module do not handle `roster` for
ssh and `raw_shell`, create a custom salt state module to run a
`saltutil.cmd` with all kwargs input so that we support `roster` and
`raw_shell`

Sees: saltstack/salt#58662
    @TeddyAndrieux
    TeddyAndrieux committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    5e4174d View commit details
    Browse the repository at this point in the history

  4. build,salt,script: Migrate to Salt Python3 

    Change salt-master image to use Python3 salt-master package and also
install Python3 dependencies, embed Python3 salt-minion version instead
of Python2.

In order to install Python3 salt-minion we need:
- to install python3
- to install python36-rpm as by default version comparaison for package
  installation is wrong
- to install python3 on nodes for being able to use `salt-ssh`

Sees: saltstack/salt#58039
Sees: saltstack/salt#57972
Fixes: #2203
    @TeddyAndrieux
    TeddyAndrieux committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    b53d936 View commit details
    Browse the repository at this point in the history

  5. salt/unit-test: Migrate to Python 3 

    - removes `salttesting` dependency
- migrates away from deprecated `TestCase` methods
- migrates `Mock` objects call args checks
    @gdemonet @TeddyAndrieux
    gdemonet authored and TeddyAndrieux committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    a7b51ea View commit details
    Browse the repository at this point in the history

  6. salt: Prevent invalid escape sequence warnings 

    We need to pass `\;` to the `isoinfo` binary for reading file contents
from an ISO archive, but this escape sequence is not valid in Python,
and Python 3 warns about it.
We could have escaped the backslash, but instead rely on the raw string
notation for Python 3 (`r'my \escaped sequence'`).
    @gdemonet @TeddyAndrieux
    gdemonet authored and TeddyAndrieux committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    e735947 View commit details
    Browse the repository at this point in the history

  7. build,tests: Bump salt to 3002.2

    @TeddyAndrieux
    TeddyAndrieux committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    cf5df11 View commit details
    Browse the repository at this point in the history

  8. salt: Add a migrate_salt state for Upgrade/Downgrade 

    Since we migrate from Python2 to Python3 a "classic" salt states cannot
handle it properly as the Python version change during the state
execution.
Add a dedicated orchestrate that handle the migration from Salt Python2
to Salt Python3 and also the migration from Salt Python3 to Salt
Python2, call this new orchestrate during Upgrade and Downgrade if
needed

Refs: #2203
    @TeddyAndrieux
    TeddyAndrieux committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    561d0cb View commit details
    Browse the repository at this point in the history

  9. salt: Use loop.previtem in Upgrade and Downgrade orchestrate 

    Since we migrate to Salt Python3 we have Jinja 2.11 so we can
use `loop.previtem` instead of ugly hack using a jinja variable.
NOTE: This hack is still needed in downgrade orchestrate when we
downgrade to `2.6.x` as we are running with Salt master metalk8s-2.6.x
that use Salt Python2 and Jinja 2.7

Refs: #2203
    @TeddyAndrieux
    TeddyAndrieux committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    280d3e5 View commit details
    Browse the repository at this point in the history

  10. salt: Do not call etcd state if Salt upgrade/downgrade failed 

    In `deploy_node` orchestrate, before this commit, during upgrade and
downgrade the `Install etcd node` state is called even if the Salt
upgrade/downgrade failed.
This commit just add a require on this etcd state
    @TeddyAndrieux
    TeddyAndrieux committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    334a7f5 View commit details
    Browse the repository at this point in the history

  11. salt: Remove non-breaking space characters from Solutions 

    Solutions YAML files contains some non-breaking space characters inside
some Jinja formula `{{ }}` that make Salt Python3 rendering fail.
NOTE: Those non-breaking space characters did not break Jinja2.7 used by
Salt in Python2 that's why we didn't see any issue because of this
before
    @TeddyAndrieux
    TeddyAndrieux committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    10d08d8 View commit details
    Browse the repository at this point in the history

  12. changelog: Add entry for Python3 migration 

    Add an entry in changelog file about Salt migration to Python3 and bump
to Salt version 3002.2

Refs: #2203
    @TeddyAndrieux
    TeddyAndrieux committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    ea22895 View commit details
    Browse the repository at this point in the history

  13. salt: Migrate sparse volume cleanup to Python3 

    See: #2726
See: #2936
    @gdemonet @TeddyAndrieux
    gdemonet authored and TeddyAndrieux committed Nov 27, 2020
    Configuration menu
    Copy the full SHA
    0d1cb84 View commit details
    Browse the repository at this point in the history