Migrations

[spAnser]

Migrations will be tricky. There is a lot to keep track of between developers. Who would see an issue where their team might all be working on adding sections / fields at the same time. Commiting to a repository before others in their team also push changes include new sections / fields which would potentially contain the same IDs.

Do we need to see a setup where newly created fields do not inherit the IDs at the time of creation and just used for updating older fields?

Or do we maintain IDs at time of creation and make sure team members are in sync with making changes.

How many teams have multiple developers working on adding their own sections / fields to the database?

[noandrea]

We have been using schematic for the last year or so (with version 2.6.x) to export/import the content model for the following reasons:

• provide a starter configuration to begin a new project, the starter content model depends from the project type, but in general there are a bunch of fields and sections that are commonly sed in the project type (seo fields, social media fields, matrix fields, etc...)

• share the content model between workers. at the same time we had 4 people working on the same project, 2 front ends, 1 seo, 1 content manager. All of them could make changes and commit the export in the repository, others will load the changes.

• publish changes to the staging server. our projects setup is docker based and when somebody commits on a particular branch the staging installation gets updated, schema included.

What have we learn during this time:

• once you get used to be abled to restore the schema, you start doing it quite a lot. also some of the people that work on the project just for the frontend, they got used to reimport the schema every time after pulling, just to be sure.

• it is quite convenient to see what is changed from git history and in case to restore older versions. but practically we never had to do it.

• it is very much a "for staff only" functionality, it is very handy for developers and content editors/managers but a customer is not interested since is part of the inner working of the cms and not part of the content (s)he wants to manage.

Summing up, although what you proposed (migrations/diff/replay) sounds very cool, I believe that for practical reason it will be better to keep the plugin less prominent in the admin panel and focus on stability on the import/export process (ie. avoid to add a possible point of failure adding ids)

this is of course only my opinion based on my experience.

thanks for your work on architect!

[FreekVR]

Pretty much +1 to the comment above, this is similar to the way we work.

We usually have multiple people working on fields/sections/config for the same project, and we keep track of the schema in git so we can then merge all the changes in the filesystem - and then import that merged config into dev environments, test, staging, production etc. automatically using shell commands.

[spAnser]

Sounds good.

The point of adding IDs is to make sure things match up with content in the event content needs to be migrated. So in theory migrations could be a final step that gets pushed to a production/staging server. While providing a command line for simple import / export would mainly help during development time.

I asked mainly because our team is small enough that we generally have 1 person who handles writing our structures and us front end guys might make minor tweaks to fields or add one on the rare occasion and usually discuss the changes before making them. In which case we would probably be sure to sync our structures before modifying one on another local setup.

Currently however importing a schema a second time will not update anything that already exists it will only add. It sounds like with the command line version of it you would want the ability to update structures as well as add them. Maybe this is what will separate the migration based ID usage with the generic importing / exporting. Where migrations would mean that IDs matter and the other for just making sure the structure exists.

[FreekVR]

I think the way schematic handles the ID "issue" for updating existing sections etc. is that it tries to fetch any existing field, section etc. by the name defined in the export/schema file. That way you could split up the migration logic into separate update/create logic. In other words - if a field defined in an import already exists on the site, simply update its attributes rather than deleting it and creating a new one in its stead.

[spAnser]

Yes I feel like in architect though that type of functionality would be part of the basic import / export actions.

The point of using IDs is what happens when you change the name and handle of something? Is that a new item or does architect somehow know the old name? What if the name was stored in the json but something had its name changed twice since someone updated and the old name listed in the schema was the last update not 2 updates ago name? IDs are the only 100% way to make sure 2 things are meant to be the same. That said I feel like #6 is actually helpful for importing / exporting between developers and migrations would be best when using CI to deploy changes to a staging / production site.

[FreekVR]

An advanced use-case could use UUID's to track changes when a section, field etc. changes name without creating duplicates when importing those changes elsewhere. That would however entail keeping track of these UUIDs in the database somewhere... You could even add timestamps to both the export and that database table to track the modification date.

[spAnser]

While the database contains uids craft has never exposed those that I have been able to see. That would require directly querying the database. I also haven't been able to tell that those were ever used to link content. Not a bad idea for attempting to match fields though, wish it was easier to grab the uid without querying the db however.

[spAnser]

I think this milestone is no longer necessary. Craft 3.1 will handle migrations once a field is imported. I think architect still has a place with scaffolding a new install or adding to an existing install. But migrations will probably be better if you are using the new built in method for handling field / section migrations.