VS402625: Dates must be increasing with each revision

[rexhinvorpsi]

This is for issues: For questions on usage please use https://stackoverflow.com/questions/tagged/azure-devops-migration-tools and tag with azure-devops-migration-tools

Describe your issue:

Hi everybody, first of all, this is a godsend of a tool and a huge thank you to everyone involved.
I updated the tool to the latest version: 11.8.4 and i am getting an error during revision replaying. I am simply migrating a current interation query with about 37 work items. It worked on version 11.6 but it doesn't after this update.
Thank you in advanca for any help

Describe any Exceptions:

VS402625: Dates must be increasing with each revision.
---> System.Web.Services.Protocols.SoapException: VS402625: Dates must be increasing with each revision.
at Microsoft.TeamFoundation.WorkItemTracking.Proxy.RetryHandler.HandleSoapException(SoapException se)
at Microsoft.TeamFoundation.WorkItemTracking.Proxy.WorkItemServer.Update(String requestId, XmlElement package, XmlElement& result, MetadataTableHaveEntry[] metadataHave, String& dbStamp, IMetadataRowSets& metadata)
at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore.SendUpdatePackage(XmlElement package, XmlElement& result, Boolean bulk)
--- End of inner exception stack trace ---
at VstsSyncMigrator.Engine.WorkItemMigrationContext.ProcessWorkItem(WorkItemData sourceWorkItem, Int32 retryLimit, Int32 retrys) in D:\a\1\s\src\VstsSyncMigrator.Core\Execution\MigrationContext\WorkItemMigrationContext.cs:line 447
at VstsSyncMigrator.Engine.WorkItemMigrationContext.InternalExecute() in D:\a\1\s\src\VstsSyncMigrator.Core\Execution\MigrationContext\WorkItemMigrationContext.cs:line 145
at MigrationTools._EngineV1.Processors.MigrationProcessorBase.Execute() in D:\a\1\s\src\MigrationTools_EngineV1\Processors\MigrationProcessorBase.cs:line 45

Source Details

• Source Version: e.g. DevOps Server 2019
• Source Language: e.g. Eng

Target Details

• Target Version: e.g. DevOps Server 2020
• Target Language: e.g. Eng

[MrHinsh]

@gereon77 could this have been introduced in #748? (v11.7.6)

[tmetodie]

Seems to appear in in 11.8.3 as well.

[MrHinsh]

@tmetodie can you try running with v11.7.5? this will help us isolate the issue.

[MrHinsh]

I'm thinking this issue is in:

    private List<MigrationTools._EngineV1.DataContracts.RevisionItem> RevisionsToMigrate(MigrationTools._EngineV1.DataContracts.WorkItemData sourceWorkItem, MigrationTools._EngineV1.DataContracts.WorkItemData targetWorkItem)
    {
        // Revisions have been sorted already on object creation. Values of the Dictionary are sorted by RevisionItem.Number
        var sortedRevisions = sourceWorkItem.Revisions.Values.ToList();

        if (targetWorkItem != null)
        {
            // Target exists so remove any Changed Date matches between them
            var targetChangedDates = (from Revision x in targetWorkItem.ToWorkItem().Revisions select Convert.ToDateTime(x.Fields["System.ChangedDate"].Value)).ToList();
            if (_config.ReplayRevisions)
            {
                sortedRevisions = sortedRevisions.Where(x => !targetChangedDates.Contains(x.ChangedDate)).ToList();
            }
            // Find Max target date and remove all source revisions that are newer
            var targetLatestDate = targetChangedDates.Max();
            sortedRevisions = sortedRevisions.Where(x => x.ChangedDate > targetLatestDate).ToList();
        }

        if (!_config.ReplayRevisions && sortedRevisions.Count > 0)
        {
            // Remove all but the latest revision if we are not replaying revisions
            sortedRevisions.RemoveRange(0, sortedRevisions.Count - 1);
        }

        TraceWriteLine(LogEventLevel.Information, "Found {RevisionsCount} revisions to migrate on  Work item:{sourceWorkItemId}",
            new Dictionary<string, object>() {
                {"RevisionsCount", sortedRevisions.Count},
                {"sourceWorkItemId", sourceWorkItem.Id}
            });
        return sortedRevisions;
    }

[MrHinsh]

My tests all look good. @rexhinvorpsi can you debug in visual studio and see what is returned from RevisionsToMigrate

[MrHinsh]

@rexhinvorpsi I added some extra logging to show the revisions in Debug before they are processed. 11.8.5 will have the extra log with a list of revisions to be processed.

[gereon77]

Hi together! Sorry for the inconvenience.. haven’t seen this issue on my own so far. Do we somewhere manipulate the ChangedDate and try to set it though it’s a system field? @tmetodie do you use a field map to migrate the changed date?

As we didn’t fix the the FieldToFieldMap yet that could be one reason i could imagine...

[gereon77]

@MrHinsh can you update the Field2FieldMap to use WorkItemData and see if that helps? I am out of office atm...

[MrHinsh]

I was unable to get to the bottom of it. This issue is not being reported widely.

@gereon77 the system was totally failing to migrate revisions at all. Unfortunately, your awesome changes did not go far enough and @tomfrenzel and I created a quick fix with a call to "GetWorkitem+revision" like we don't want to have to. It may be related to this issue, but Im not sure.

[tmetodie]

The issue at least for me with the dates is not seen anymore in 1.9.0. Same configuration.json file. Thanks a lot!

[gereon77]

I was unable to get to the bottom of it. This issue is not being reported widely.

@gereon77 the system was totally failing to migrate revisions at all. Unfortunately, your awesome changes did not go far enough and @tomfrenzel and I created a quick fix with a call to "GetWorkitem+revision" like we don't want to have to. It may be related to this issue, but Im not sure.

Indeed... I saw the comment in the commit and of course we have to adapt PopulateWorkItem To use the field dictionary of the “revised work item data” as well... sorry! I should have tested this more deeply.

When I have time I will look into it with care...