Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NAS-130120 / 24.10 / Introduce a migration to allow migrating old apps to new apps #14052

Merged
merged 44 commits into from
Jul 30, 2024
Merged

NAS-130120 / 24.10 / Introduce a migration to allow migrating old apps to new apps #14052

merged 44 commits into from
Jul 30, 2024

Conversation

sonicaj
Copy link
Member

@sonicaj sonicaj commented Jul 20, 2024

This PR adds changes to allow migrating old apps to new apps.

@sonicaj sonicaj added the WIP label Jul 20, 2024
@sonicaj sonicaj self-assigned this Jul 20, 2024
@bugclerk
Copy link
Contributor

Jira URL: https://ixsystems.atlassian.net/browse/NAS-130120

@bugclerk bugclerk changed the title Introduce a migration to allow migrating old apps to new apps NAS-130120 / 24.10 / Introduce a migration to allow migrating old apps to new apps Jul 20, 2024
@sonicaj sonicaj force-pushed the NAS-130120 branch 3 times, most recently from 087d127 to e6bb983 Compare July 29, 2024 12:21
@sonicaj sonicaj marked this pull request as ready for review July 29, 2024 17:24
sonicaj added 22 commits July 30, 2024 14:50
@sonicaj
Add basic plugin which will handle k8s to docker migration bits
75280df
@sonicaj
Add basic method to list k8s backups
b9424ef
@sonicaj
Handle error when backup dir is not found
68b6cf6
@sonicaj
Complete implementation for listing k8s backups
24f2fe7
@sonicaj
Add basic migration service
55781e8
@sonicaj
Add basic bits in place which determine if an app can be ported to do…
c902889 
…cker
@sonicaj
Port yaml bits from df to read k8s backups
9783de4
@sonicaj
Add logic to read namespace of chart release and see if it is valid
3b8d8a4
@sonicaj
Give a detailed response in the case when we skip a release
8c81504
@sonicaj
Add secrets utils
67420f3
@sonicaj
Retrieve detailed information for each release found
1288696
@sonicaj
Add changes to read helm and app secrets and retrieve them
d1c1aef
@sonicaj
Make sure list secrets gets us normalized data and verifies it's inte…
10f2d10 
…grity
@sonicaj
Amend listing backups endpoint to more comprehensively retrieve detai…
00efd01 
…ls of each app
@sonicaj
Make listing backups a job
4cd9621
@sonicaj
Add changes to configure docker service on new k8s pool
5494002
@sonicaj
Add util which will migrate old config to new config
4d77a48
@sonicaj
Add changes to rollback ix-applications dataset snapshot and iterate …
831338a 
…over chart releases
@sonicaj
Include app version for each old app
c0cd6dc
@sonicaj
Add changes to allow a dry run of sorts for app creation which we wil…
45e1b18 
…l use to help migrate over apps from k8s
@sonicaj
Setup app's filesystem bits and normalize it's config etc
c4b3de3
@sonicaj
Mark create_internal as private
704e67c
sonicaj added 20 commits July 30, 2024 14:50
@sonicaj
If source is charts train, make it switch to stable
35fa69b
@sonicaj
Fix backups listing logic
e9a8176
@sonicaj
Validate that migrate file is executable
4bf7a9a
@sonicaj
Add util to retrieve app's volume ds
d15376d
@sonicaj
Clone/promote necessary ix volumes
355a781
@sonicaj
Minor fixes
b333f30
@sonicaj
Fix ix volume dataset name
3561ee9
@sonicaj
Mount ix-volume dataset after promotion is done
82ab56e
@sonicaj
Generate collective metadata after an app has been successfully migra…
94743b9 
…ted over
@sonicaj
Pass on gpu choices context to app template when rendering
b15644a
@sonicaj
Use internal gpu choices endpoint
26bcdf2
@sonicaj
Redeploy apps once they have migrated over successfully
b700c57
@sonicaj
Update job progress when migration completes
6c82754
@sonicaj
Raise an error if bulk job fails for some reason
a54922a
@sonicaj
Fix minor bug
b3a221b
@sonicaj
Fix migrate file path being collected when listing backup
378754c
@sonicaj
Validate migration file is executable
6e59dbd
@sonicaj
Fix migration path bug
bd0faee
@sonicaj
Fix redeploy bug
7c1bf90
@sonicaj
Make sure we wait for catalog to be properly synced before we try to …
e51676f 
…list apps which can be migrated
yocalebo
yocalebo approved these changes Jul 30, 2024
View reviewed changes
Copy link
Contributor

@yocalebo yocalebo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making a note here so we don't forget but we'll need to add the ability to migrate apps after upgrade as well. This can be done in a separate PR.

@sonicaj sonicaj removed the WIP label Jul 30, 2024
@yocalebo yocalebo merged commit a002681 into master Jul 30, 2024
2 of 3 checks passed
@yocalebo yocalebo deleted the NAS-130120 branch July 30, 2024 11:12
@bugclerk
Copy link
Contributor

This PR has been merged and conversations have been locked.
If you would like to discuss more about this issue please use our forums or raise a Jira ticket.

@truenas truenas locked as resolved and limited conversation to collaborators Jul 30, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@yocalebo yocalebo yocalebo approved these changes

Assignees

@sonicaj sonicaj

Labels
no-time-tracked
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@sonicaj @bugclerk @yocalebo