Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrations #100

Closed
ewjoachim opened this issue Oct 21, 2019 · 6 comments
Closed

Migrations #100

ewjoachim opened this issue Oct 21, 2019 · 6 comments
Labels
Issue contains: Exploration & Design decisions 🧩 We don't know how this will be implemented yet Issue contains: Some documentation 📚 This issues involves writing some documentation Issue contains: Some SQL 🐘 This features require changing the SQL model
Milestone

Comments

@ewjoachim
Copy link
Member

ewjoachim commented Oct 21, 2019

For now, it's all fun and games, no one uses Procrastinate yet, and we haven't really thought about migrations, but it's probably time to setup something.

  • People might hop in and install procrastinate at any version (starting at 1.0, say)
  • Starting from there, people will need to upgrade procrastinate (including the schema)
  • Ideally, in a simple manner
  • Ideally, without impact and downtime
  • Optionally, using https://github.com/peopledoc/septentrion/ given we're developing this tool too.

One way of doing it could be:

  • Adapt the migration files to septentrion. Remove procrastinate migrate in favor of septentrion migrate.
  • Create a first migration for something trivial, just to test it.
  • Document the hell out of it, so that it's easy to understand. Don't forget Septentrion is by far not as properly documented 1. as it should be and 2. as procrastinate is.
  • (make it easy for people who want to read migrations instead of executing them)

(Note that in PeopleDoc, we actually don't need this part at all, because our fantastic DBA team will take care of the migrations for us, so this is strictly an effort to make procrastinate more usable by other people)

@ewjoachim ewjoachim added Issue contains: Some documentation 📚 This issues involves writing some documentation Issue contains: Exploration & Design decisions 🧩 We don't know how this will be implemented yet feature Issue contains: Some SQL 🐘 This features require changing the SQL model labels Oct 21, 2019
@ewjoachim
Copy link
Member Author

Is it worth making a procrastinate cli command that spits out a septentrion.ini file ?

@ewjoachim
Copy link
Member Author

Should procrastinate explicitely depend on septentrion (maybe using an extra) and procrastinate migrate be instead a wrapper around septentrion migrate that autoconfigures septentrion ?

@mgu
Copy link

mgu commented Oct 22, 2019

I like the way that is described in the description :)
(plain sql files, septentrion as an extra requirement)

@ewjoachim ewjoachim added this to the 1.0 milestone Oct 24, 2019
@ewjoachim
Copy link
Member Author

@elemoine has come with a very very interesting proposal:

Given:

  • Septentrion is not and has not been written to be a tool for running migrations in production
  • Septentrion is very linked to PeopleDoc's way of writing migrations (including, for example, --meta-psql:do-until-0, etc)
  • Septentrion is not as clean as Procrastinate is
  • This is one of the last release blockers for 1.0
  • There is an external tool (https://github.com/opengisch/pum/) that actually does what we wished Septentrion did.

Pum seems like a nice, much much more mature tool, with plenty of goodies.

Then, what about the following plan:

  • We don't link Procrastinate to septentrion

  • We publish up-to-date schemas, and a simple way to apply it ot an empty database

  • We publish simple agnostic SQL migration files

  • We advertise the use of pum in the documentation to use the migration files in real-life use cases

  • Maybe, later, in a future that is not planned yet,we may self-migrate on startup using pum.

  • Maybe, later, in a future that is not planned yet, we may add pum as an optional dependency and provide builtin task to self-migrate procrastinate (as long as we're not too far back, say 1 or 2 versions)

In the CI we could use pum too.

(Ping @elemoine @Evelf @DainDwarf )

@mgu ?

@k4nar
Copy link
Contributor

k4nar commented Feb 7, 2020

Migrations will be a big concern for the community, but at PeopleDoc we'll handle them our way no matter what. So +1 to use a tool with a strong community for the greater good. It will be very easy for us to adapt internally.

@elemoine
Copy link
Contributor

elemoine commented Mar 6, 2020

Closed by #161, #162, #163, #167, and #170.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Issue contains: Exploration & Design decisions 🧩 We don't know how this will be implemented yet Issue contains: Some documentation 📚 This issues involves writing some documentation Issue contains: Some SQL 🐘 This features require changing the SQL model
Projects
None yet
Development

No branches or pull requests

4 participants