Data Migrations

Or, Data Migrations, the care and feeding of ##0. For every Every DM in FW

###a) There must be a corresponding metadata cache migration in FLEx Bridge

Read must be as: "it is imperative that there be", or you won't like yourself, when you try FB the next time. If in doubt on how to do that in FB, check with RandyR.

###b) Any new CmObjects must have all C# value type data properties added for the new instance for that class and any of its superclasses.

Read must be as: "it is imperative that there be", or you won't like yourself, when you try the FB S/R the next time on the full data set.

The current data types that must have explicit property elements are: int, bool, Guid, DateTime, and GenDate (int).

##1. In C:\FwRepo\fw\Src\FDO\MasterFieldWorksModel.xml

###a) Change the version number (e.g. 7000029 to 7000030; see this line in the file) <EntireModel xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="7000029" xsi:noNamespaceSchemaLocation="MasterFieldWorksModel.xsd">

###b) Add a change history comment near the top RBR: The comment should contain information, such as the date of the change, the same model version number as above, and a short description of what is changed in the model.

###c) If the model has changed (and not just data) then make an additional change to this file to add to the model or change it: e.g. <basic num="42" id="ExtLinkRootDir" sig="Unicode">

changed to....

<basic num="42" id="LinkedFilesRootDir" sig="Unicode">

##2. For special cases like this one we have LinkedFilesRootDir in C:\FwRepo\fw\Src\FDO\FDOGenerate\HandGenerated.xml so the change was needed there also. (What sort of special cases? Really?)

##3. Make whatever other changes to the code are required by the model change, where not part of the generated code. e.g. LangProjectTags.kflidExtLinkRootDir

to

LangProjectTags.kflidLinkedFilesRootDir

##4. Files that need to be checked out and changed

###C:\FwRepo\fw\Src\FDO\MasterFieldWorksModel.xml

###C:\FwRepo\fw\Src\FDO\FDOGenerate\HandGenerated.xml (possibly)

###C:\FwRepo\fw\DistFiles\Templates\NewLangProj.fwdata

In here change the version number for example from 7000028 to 7000029 (at the top of the file) Don't forget to do this. You will regret it!

RBR: I'm not sure only changing only the version number is adequate, or is particularly dangerous to not do it. My practice was to simply fire up Flex in VS and put a breakpoint in a strategic location in the xml startup code. Then, create a new Lang Proj, but only let the code run to the point right after the DM was done, but before anything was done. Then, I would see that the fwdata file was saved with any model changes done and the version number updated. I would then stop the debugging session, before anything else was added to the new lang proj file.

###C:\FwRepo\fw\Src\FDO\DomainServices\DataMigration\FdoDataMigrationManager.cs(>115)

In here just make an addition like the following:

m_individualMigrations.Add(7000065, new DataMigration7000065());

RBR: Some DMs are 'do-nothing' deals that serve some purpose other than doing an actual migration of the data. Those share a common instance of a special implementation of that DM interface, that only increments the number by one.

###Once the migration is ready to check in

• Check out these 4 files, (EXCEPT they aren't in our repo anymore!)

C:\FwRepo\fw\DistFiles\ReleaseData\Sena 2\Sena 2.fwdata

C:\FwRepo\fw\DistFiles\ReleaseData\Sena 3\Sena 3.fwdata

C:\FwRepo\fw\DistFiles\ReleaseData\Lela-Teli 2\Lela-Teli 2.fwdata

C:\FwRepo\fw\DistFiles\ReleaseData\Lela-Teli 3\Lela-Teli 3.fwdata

• Open them in Flex (so that the migration is performed on them)
• Check them in along with any other files you are checking in for the migration..

##5. New files created

Create new versions of the following 3 files.

###a) C:\FwRepo\fw\Src\FDO\FDOTests\TestData\DataMigration7000029Tests.xml This is a stripped down version of a project which contains the pertinent objects in it which can be used to test that the migration is working correctly. Look at other test data to see what might be required for your test.

RBR: There is no need to use a maximal projection of the all the properties for a given class in these test data files. They dont; need to be loaded to create real FDO objects, since they really only need the minimum amount of data in the to make the tests pass. Leave out all other elements and attributes. Some bits of xml that are not germane to the test may be required in order to be loaded into the DTO repository.

###b) C:\FwRepo\fw\Src\FDO\FDOTests\DataMigrationTests\DataMigration7000029Tests.cs This is the test which demonstrates that the project in DataMigration7000029Tests.xml actually migrates correctly.

###c) C:\FwRepo\fw\Src\FDO\DomainServices\DataMigration\DataMigration7000029.cs This is the code that actually does the migration. Look at other migrations for examples of how things are done.

RBR: The first step in all of the DM code files is to check the then current version number. Such code must never be applied to the data out of order, so 'choke' if the number is not 1 less than the current DM class is prepared to deal with. The second step is to do the actual DM. The last step must be to increment the current model number, so the next DM code will be able to do its work.

##6. Tips for writing the actual Migration Code

When creating the above 3 new files it makes sense to copy them from a previous version.

Then do a search in all files for:

70000xx (whatever version you copied from)

doing this will show you where comments and code need to be changed.

Here's an example search.

Find all "7000024", Match case, Whole word, Subfolders, Find Results 1, "C:\FwRepo\fw\Src", "*.cs;"

...\DomainServices\DataMigration\DataMigration7000024.cs(30): /// Migrate data from 7000023 to 7000024.

...\DomainServices\DataMigration\DataMigration7000025.cs(23): /// Migrate data from 7000024 to 7000025.

...\DomainServices\DataMigration\DataMigration7000025.cs(40): DataMigrationServices.CheckVersionNumber(domainObjectDtoRepository, 7000024);

...\DomainServices\DataMigration\IDataMigrationManager.cs(167): m_individualMigrations.Add(7000024, new DataMigration7000024());

...\FDOTests\DataMigrationTests\DataMigration7000024Tests.cs(11): /// Test framework for migration from version 7000023 to 7000024.

...\FDOTests\DataMigrationTests\DataMigration7000024Tests.cs(18): /// Test the migration from version 7000023 to 7000024.

...\FDOTests\DataMigrationTests\DataMigration7000024Tests.cs(51): m_dataMigrationManager.PerformMigration(dtoRepos, 7000024);

...\FDOTests\DataMigrationTests\DataMigration7000024Tests.cs(188): Assert.AreEqual(7000024, dtoRepos.CurrentModelVersion, "Wrong updated version.");

...\FDOTests\DataMigrationTests\DataMigration7000025Tests.cs(12): /// Test framework for migration from version 7000024 to 7000025.

...\FDOTests\DataMigrationTests\DataMigration7000025Tests.cs(65): /// Test the migration from version 7000024 to 7000025.

...\FDOTests\DataMigrationTests\DataMigration7000025Tests.cs(101): IDomainObjectDTORepository dtoRepos = new DomainObjectDtoRepository(7000024, dtos, mockMdc, null);

Matching lines: 11    Matching files: 5    Total files searched: 2434