Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade from 2.6.x to 2.7.0 fails when cleaning sparse loop volumes #3064

Closed
gdemonet opened this issue Jan 21, 2021 · 1 comment
Closed
Labels
kind:bug Something isn't working severity:major Major impact on live deployments (e.g. some non-critical feature is not working at all) topic:lifecycle Issues related to upgrade or downgrade of MetalK8s

Comments

@gdemonet
Copy link
Contributor

Component: salt, upgrade, storage

Summary:

When running the upgrade.sh script to upgrade a MetalK8s 2.6.x cluster to 2.7.0, the Upgrading all nodes one by one step will abort for each node which has no SparseLoopDevice Volume provisioned, during the metalk8s.volumes.cleanup state execution. It will show an error similar to:

                        ID: Cleanup existing loop devices                                                                                                                                                                                     
                  Function: salt.state                                                                                                                                                                                                        
                    Result: False                                                                                                                                                                                                             
                   Comment: Run failed on minions: metalk8s-control-plane-02                                                                                                                                                                  
                   Started: 23:39:50.586427                                                                                                                                                                                                   
                  Duration: 2838.153 ms                                                                                                                                                                                                       
                   Changes:                                                                                                                                                                                                                   
                            metalk8s-control-plane-02:                                                                                                                                                                                        
                                Data failed to compile:                                                                                                                                                                                       
                            ----------                                                                                                                                                                                                        
                                Rendering SLS 'metalk8s-2.7.0:metalk8s.volumes.cleanup' failed: Jinja variable 'target_is_2_7' is undefined 

The only available workaround is to restart the upgrade.sh script, once per failing node, until completion.

@gdemonet gdemonet added kind:bug Something isn't working topic:lifecycle Issues related to upgrade or downgrade of MetalK8s severity:major Major impact on live deployments (e.g. some non-critical feature is not working at all) labels Jan 21, 2021
@gdemonet gdemonet added this to the MetalK8s 2.7.1 milestone Jan 21, 2021
@gdemonet
Copy link
Contributor Author

This issue was fixed in #3048.

gdemonet added a commit that referenced this issue Jan 22, 2021
gdemonet added a commit that referenced this issue Feb 3, 2021
gdemonet added a commit that referenced this issue Feb 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:bug Something isn't working severity:major Major impact on live deployments (e.g. some non-critical feature is not working at all) topic:lifecycle Issues related to upgrade or downgrade of MetalK8s
Projects
None yet
Development

No branches or pull requests

1 participant