Running multiple content migrations fails when fields are added

[mgburns]

Description

I'm unable to run multiple content migrations that add fields to entry type field layouts and create entries of that type successfully.

Steps to reproduce

1. Create two content migrations: m180317_193558_add_home_page.php and m180317_193559_add_contact_page.php

2. Attempt to run both of them with php craft migrate

3. On the first attempt, the second migration fails:

   $ php craft migrate
   Yii Migration Tool (based on Yii v2.0.14.2)
   
   Total 2 new migrations to be applied:
       m180317_193558_add_home_page
       m180317_193559_add_contact_page
   
   Apply the above migrations? (yes|no) [no]:yes
   *** applying m180317_193558_add_home_page
       > creating "Body" field ... done
       > creating "Homepage" section ... done
       > saving "Homepage" entry type ... done
       > updating "Homepage" entry ... done
   *** applied m180317_193558_add_home_page (time: 1.020s)
   
   *** applying m180317_193559_add_contact_page
       > creating "Contact Info" field ... done
       > creating "Contact Us" section ... done
       > saving "Contact Us" entry type ... done
   Exception: Setting unknown property: craft\elements\Entry::contactInfo (/var/www/html/vendor/yiisoft/yii2/base/Component.php:209)
   #0 /var/www/html/vendor/craftcms/cms/src/base/Element.php(767): yii\base\Component->__set('contactInfo', 'Call me maybe?')
   #1 /var/www/html/migrations/m180317_193559_add_contact_page.php(76): craft\base\Element->__set('contactInfo', 'Call me maybe?')
   #2 /var/www/html/vendor/craftcms/cms/src/db/Migration.php(42): craft\contentmigrations\m180317_193559_add_contact_page->safeUp()
   #3 /var/www/html/vendor/yiisoft/yii2/console/controllers/BaseMigrateController.php(725): craft\db\Migration->up()
   #4 /var/www/html/vendor/yiisoft/yii2/console/controllers/BaseMigrateController.php(199): yii\console\controllers\BaseMigrateController->migrateUp('m180317_193559_...')
   #5 /var/www/html/vendor/craftcms/cms/src/console/controllers/MigrateController.php(274): yii\console\controllers\BaseMigrateController->actionUp(0)
   #6 [internal function]: craft\console\controllers\MigrateController->actionUp(0)
   #7 /var/www/html/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array(Array, Array)
   #8 /var/www/html/vendor/yiisoft/yii2/base/Controller.php(157): yii\base\InlineAction->runWithParams(Array)
   #9 /var/www/html/vendor/yiisoft/yii2/console/Controller.php(148): yii\base\Controller->runAction('', Array)
   #10 /var/www/html/vendor/yiisoft/yii2/base/Module.php(528): yii\console\Controller->runAction('', Array)
   #11 /var/www/html/vendor/yiisoft/yii2/console/Application.php(180): yii\base\Module->runAction('migrate', Array)
   #12 /var/www/html/vendor/yiisoft/yii2/console/Application.php(147): yii\console\Application->runAction('migrate', Array)
   #13 /var/www/html/vendor/yiisoft/yii2/base/Application.php(386): yii\console\Application->handleRequest(Object(craft\console\Request))
   #14 /var/www/html/craft(22): yii\base\Application->run()
   #15 {main}
   *** failed to apply m180317_193559_add_contact_page (time: 0.308s)
   
   
   1 from 2 migration was applied.
   
   Migration failed. The rest of the migrations are canceled.

4. If you run it again, the second migration will appear to have succeeded:

   $ php craft migrate
   Yii Migration Tool (based on Yii v2.0.14.2)
   
   Total 1 new migration to be applied:
       m180317_193559_add_contact_page
   
   Apply the above migration? (yes|no) [no]:yes
   *** applying m180317_193559_add_contact_page
       > creating "Contact Info" field ... done
       > creating "Contact Us" section ... done
       > saving "Contact Us" entry type ... done
       > updating "Contact Us" entry ... done
   *** applied m180317_193559_add_contact_page (time: 0.803s)
   
   
   1 migration was applied.
   
   Migrated up successfully.

But the entry field content is not actually saved for either entries.

I discovered that if I switch from $entry->{fieldHandle} = 'value' to $entry->setFieldValue('fieldHandle', 'value') for setting entry field content both migrations succeed on the first run, but still none of the field content appears to be saved to the content table. Possibly related: #1865.

Additional info

• Craft version: 3.0.0-RC15
• PHP version: 7.0.27
• Database driver & version: MySQL 5.7.21
• Plugins & versions: None

[mgburns]

As a related aside, I know that I'm an early adopter here but content migrations have been pretty painful to get working with our workflow so far.

At Upstatement we have a pretty smooth process with WordPress / Advanced Custom Fields that involves multiple engineers and designers working in local environments, iterating on content schemas and sync'ing them up to staging / production environments. With Craft 2 we got by with Schematic (it wasn't perfect), and I was hopeful that content migrations might be a reliable solution with Craft 3. Migrations with Django and other web frameworks are painless because they're generated from model definitions -- without that layer of abstraction it requires a lot of code and knowledge of Craft internals to be productive. Creating new sections and fields is relatively straightforward, but it gets ugly fast when you attempt to update existing field layouts, especially with more complicated field types like matrix. I threw up my hands in defeat when I was told we needed to update a matrix field in a Super Table block type in a matrix field. :)

I know I'm not the first person to encounter this (#1429, #807, and more I'm sure) but FWIW I'd love to see the Craft equivalent of ACF's Local JSON feature -- multiple developers can make field / section / settings updates through the GUI, and serialize to disk and version control for synchronization, ideally both through the GUI and CLI for CI. Easily defining them in code would be nice, too, but does create a higher bar for entry, for better or worse.

I'd be happy to contribute to this with code, thoughts, documentation, moral support, etc.

[lukeholder]

@mgburns does this help? https://github.com/Pennebaker/craft-architect re: a Schematic equivalent?

[mgburns]

@lukeholder It's similar but lacks a few critical features for us -- namely running from the CLI. We went with Schematic over Architect for Craft 2 for many of the same reasons outlined in Pennebaker/craft-architect#7 (comment).

One of the biggest gotchas with Schematic was its use of handles to uniquely identify fields. This prevents you from being able to rename existing fields. It could be worked around if you were aware of it, but I believe a complete solution to the problem should use something less volatile like a UUID.

[brandonkelly]

FWIW we are planning on making this stuff better in Craft 3.1, by moving more config stuff into config files (e.g. #1429) and using content migrations when a thing is added/removed, or its handle changes.