Downgrade from dev/2.5 to dev/2.4 does not work

[TeddyAndrieux]

Component:

'salt', 'kubernetes'

What happened:

In dev/2.4 we use extensions/v1beta1 API do deploy kube-proxy DaemonSet but this API get removed in k8s 1.16 (which is the version we use in dev/2.5) so when we try to downgrade kube-proxy and our APIserver still using dev/2.5 version (k8s 1.16) extensions/v1beta1 do not exists

          ID: Deploy kube-proxy (DaemonSet)
    Function: metalk8s_kubernetes.object_present
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.7/site-packages/salt/state.py", line 1919, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.7/site-packages/salt/loader.py", line 1918, in wrapper
                  return f(*args, **kwargs)
                File "/var/cache/salt/master/extmods/states/metalk8s_kubernetes.py", line 132, in object_present
                  **kwargs
                File "/var/cache/salt/master/extmods/modules/metalk8s_kubernetes.py", line 200, in method
                  return _handle_error(exc, action)
                File "/var/cache/salt/master/extmods/modules/metalk8s_kubernetes.py", line 74, in _handle_error
                  raise CommandExecutionError(base_msg + str(exception).decode('utf-8'))
              CommandExecutionError: Failed to create object: (404)
              Reason: Not Found
              HTTP response headers: HTTPHeaderDict({'date': 'Wed, 20 Nov 2019 11:01:44 GMT', 'content-length': '174', 'content-type': 'application/json', 'cache-control': 'no-cache, private'})
              HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the server could not find the requested resource","reason":"NotFound","details":{},"code":404}

This extensions/v1beta1 is already deprecated in dev/2.4 and should not have been used (since metalk8s-2.4.0 and metalk8s-2.4.1 are released with this API this issue need to be handle in downgrade process of dev/2.5)

What was expected:

Working downgrade from dev/2.5 to dev/2.4

Steps to reproduce

Try to downgrade from dev/2.5 to dev/2.4

Resolution proposal (optional):

TBD:
2 proposition:

1. Downgrade API server of the bootstrap node before downgrading all kubernetes objects (but then when downgrading kubernetes object we can't remove object that use newer API)
2. Do a kind of "hack" to use apps/v1 instead of extensions/v1beta1 to "downgrade" DaemonSet + no longer use depreacted API in any metalk8s branchs

[thomasdanan]

maybe related to #2081 ?

[TeddyAndrieux]

maybe related to #2081 ?

No we have 2 separate issue, #2081 is just downgrading from one version to a version already released so only 2.4.1 and 2.4.0

[TeddyAndrieux]

Fixed by #2101