Migrate first work items then links and attachments? #1425

fhack2 · 2019-10-31T15:07:52Z

fhack2
Oct 31, 2019

With the 8.4.7 version can you migrate all the work items first and then in another JSON file complete the linking and attachments?

We first in several JSON files move task, epic, feature, task, and pbi. Then looking to run after all those JSON, and one just to link them all together and add attachments.

Use to use the old version before everything was combined into WorkItemMigrationConfig.

MrHinsh · 2019-10-31T15:51:02Z

MrHinsh
Oct 31, 2019
Maintainer

You can, but it's not recommended. The new version is MUCH faster end to end.

Can I ask why you would want to migrate different work item types separately?

I generally recommend to move all Open work item in the last 90 days, then all closed items. Once you have the last 90 days then move up to the last 360, then the rest... This method of progressive scope expansion for work item migration enables the quickest turnaround.... with the tail being mopped up over time.

Running this separately is no longer supported and will not benefit from feature enhancements and bug fixes. You can install a version prior to v8.. to continue to run them separately.

0 replies

fhack2 · 2019-10-31T16:33:06Z

fhack2
Oct 31, 2019
Author

Have a team converting from customized Agile to OOB Scrum in TFS.

What would be the querybit for the last 60 days? Understand what you have said and I'm adding all the work item types into one. It's that each one had different FieldMaps for the states, so it was easy to keep track of them separately.

  "QueryBit": "AND [System.WorkItemType] IN ('Epic') AND ([System.CreatedDate] >= '2018-01-01T00:00:00.0000000' or [Microsoft.VSTS.Common.ClosedDate] >= '2018-01-01T00:00:00.0000000')",

0 replies

MrHinsh · 2019-10-31T16:41:13Z

MrHinsh
Oct 31, 2019
Maintainer

I usually do something like:

QueryBit": "AND [System.ChangedDate] >= '1/1/2019 00:00:00Z' AND [System.WorkItemType] Not In ('Test Plan', 'Test Suite')",",

That way I get explicit control of the date. You can also filter for [System.ClosedDate]='' as this field is always empty when a work item is open.

0 replies

MrHinsh · 2019-10-31T16:44:36Z

MrHinsh
Oct 31, 2019
Maintainer

Note that you can add different field mappings for each work item type and have others that apply to them all. That way you can run a single migration as above.

0 replies

fhack2 · 2019-10-31T17:21:27Z

fhack2
Oct 31, 2019
Author

I went with the "IN" instead of "Not In" to only cover the 5 work item types being moved.

This is the bulk of the JSON now:
"FieldMaps": [
//Epic
{
"ObjectType": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldValueMapConfig",
"WorkItemTypeName": "Epic",
"sourceField": "System.State",
"targetField": "System.State",
"defaultValue": "New",
"valueMapping": {
"Active": "Comitted",
"Closed": "Done",
"New": "New",
"Removed": "Removed",
"Resolved": "Done"
}
},
//Feature
{
"ObjectType": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldValueMapConfig",
"WorkItemTypeName": "Feature",
"sourceField": "System.State",
"targetField": "System.State",
"defaultValue": "New",
"valueMapping": {
"Active": "Comitted",
"Closed": "Done",
"New": "New",
"Removed": "Removed",
"Resolved": "Done"
}
},
//User Story to PBI
{
"ObjectType": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldValueMapConfig",
"WorkItemTypeName": "User Story",
"sourceField": "System.State",
"targetField": "System.State",
"defaultValue": "New",
"valueMapping": {
"Recorded": "New",
"Open": "Comitted",
"Deployed Awaiting Verification": "Comitted",
"Tested Awaiting Deploy": "Comitted",
"In UAT": "Comitted",
"In Model": "Comitted",
"To Model": "Comitted",
"In Test": "Comitted",
"To Test": "Comitted",
"Closed Not Justified": "Removed",
"In Design": "Comitted",
"Approval Requested": "Comitted",
"Closed": "Done",
"In Development": "Comitted",
"Awaiting Assignment": "Approved",
"On Hold": "Removed"
}
},
//Task
{
"ObjectType": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldValueMapConfig",
"WorkItemTypeName": "Task",
"sourceField": "System.State",
"targetField": "System.State",
"defaultValue": "New",
"valueMapping": {
"Active": "In Progress",
"Closed": "Done",
"New": "To Do",
"Removed": "Removed"
}
},
//Bug
{
"ObjectType": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldValueMapConfig",
"WorkItemTypeName": "Bug",
"sourceField": "System.State",
"targetField": "System.State",
"defaultValue": "New",
"valueMapping": {
"Closed": "Done",
"New": "New",
"Missing Info": "Comitted",
"Deployed in Production": "Comitted",
"Tested Waiting Deploy": "Comitted",
"In Model": "Comitted",
"In Test": "Comitted",
"To Model": "Comitted",
"To Test": "Comitted",
"Assigned To Fix": "Comitted",
"Missing Defect Info": "Comitted",
"Open": "Approved"
}
}
],
"WorkItemTypeDefinition": {
"Epic": "Epic",
"Feature": "Feature",
"User Story": "Product Backlog Item",
"Task": "Task",
"Bug": "Bug"
},
// "GitRepoMapping": null,
"Processors": [
{
"ObjectType": "VstsSyncMigrator.Engine.Configuration.Processing.NodeStructuresMigrationConfig",
"PrefixProjectToNodes": false,
"Enabled": true
},
{
"ObjectType": "VstsSyncMigrator.Engine.Configuration.Processing.TeamMigrationConfig",
"Enabled": true
},
{
"ObjectType": "VstsSyncMigrator.Engine.Configuration.Processing.WorkItemMigrationConfig",
"ReplayRevisions": false,
"PrefixProjectToNodes": false,
"UpdateCreatedDate": true,
"UpdateCreatedBy": true,
"UpdateSourceReflectedId": false,
"BuildFieldTable": false,
"AppendMigrationToolSignatureFooter": true,
"QueryBit": "AND [System.WorkItemType] IN ('Bug', 'Epic', 'Feature', 'User Story', 'Task') AND [System.ChangedDate] >= '2019-01-01T00:00:00.0000000'",
"OrderBit": "[System.ChangedDate] desc",
"Enabled": true,
"LinkMigration": true,
"AttachmentMigration": true,
"AttachmentWorkingPath": "E:\temp\WorkItemAttachmentWorkingFolder\",
"FixHtmlAttachmentLinks": true,
"WorkItemCreateRetryLimit": 5,
"FilterWorkItemsThatAlreadyExistInTarget": true,
"PauseAfterEachWorkItem": false
}
]
}

0 replies

MrHinsh · 2019-10-31T17:28:31Z

MrHinsh
Oct 31, 2019
Maintainer

OK, so for the System.State field mapping you can simpley set WorkItemTypeName to * and it will apply for them all. Then create one big list of mappings...

Then only have a seperate mapping if you need to branch. They are executed in the order they are listed, so if you have a second mapping for State thst is only for 1 work item type then just have it after.

0 replies

fhack2 · 2019-10-31T17:44:00Z

fhack2
Oct 31, 2019
Author

So did not have this issue before, but it stops on attachments it can't add. Management before my time thought it was a good idea to allow large file attachments. Users abused it and where attaching long videos so the database was growing. To prevent that I put the attachment size back down to 4MB.

migration.exe Warning: 0 : [EXCEPTION] Microsoft.TeamFoundation.WorkItemTracking.Client.FileAttachmentException: TF237082: The file you are trying to upload (US-175637_TestEvidence_Losstheft_replacement.docx) exceeds the supported file upload size (4 MB(s)). Instead of uploading the file, add the file to version control or to the team project web site, and then link the file to the work item.
at VstsSyncMigrator.Engine.WorkItemMigrationContext.ProcessWorkItem(WorkItemStoreContext sourceStore, WorkItemStoreContext targetStore, Project destProject, WorkItem sourceWorkItem, Int32 retryLimit, Int32 retrys) in C:\Source\TFStoADO\MigrationTool\ado-migration-tools-8.4.7\src\VstsSyncMigrator.Core\Execution\MigrationContext\WorkItemMigrationContext.cs:line 226
at VstsSyncMigrator.Engine.WorkItemMigrationContext.InternalExecute() in C:\Source\TFStoADO\MigrationTool\ado-migration-tools-8.4.7\src\VstsSyncMigrator.Core\Execution\MigrationContext\WorkItemMigrationContext.cs:line 119
at VstsSyncMigrator.Engine.MigrationContextBase.Execute() in C:\Source\TFStoADO\MigrationTool\ado-migration-tools-8.4.7\src\VstsSyncMigrator.Core\Execution\MigrationContext\MigrationContextBase.cs:line 35
MigrationEngine: The Processor WorkItemMigration entered the failed state...stopping run

0 replies

MrHinsh · 2019-10-31T19:50:05Z

MrHinsh
Oct 31, 2019
Maintainer

Should now be all fixed. MaxAttachmentSize of the target can be set in the config and it defaults to the 60mb that is Azure DevOps Services.

0 replies

fhack2 · 2019-10-31T20:27:32Z

fhack2
Oct 31, 2019
Author

I will test and let you know how it goes.

Is it wrong to run the same JSON pointing to another project?
For example: On the new target project, it creates PBI 175803 and then tries to link it to Task 17422. However, Task 17422 is the previous target project. The reason I have multiple target projects is to provide users with different versions of history for them to validate. The first was so they could see how it looks going to OOB Scrum from Custom Agile.

[ User Story][Complete: 64/19231][sid:374652|Rev:36 ][tid:null | Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemLinkValidationException: TF201066: You cannot add a Child link to work item 174222 because a work item can have only one Parent link. ---> System.Web.Services.Protocols.SoapException: TF201036: You can not add a Child link between work items 175803 and 174222 because a work item can have only one Parent link.
at Microsoft.TeamFoundation.WorkItemTracking.Proxy.RetryHandler.HandleSoapException(SoapException se)
at Microsoft.TeamFoundation.WorkItemTracking.Proxy.WorkItemServer.Update(String requestId, XmlElement package, XmlElement& result, MetadataTableHaveEntry[] metadataHave, String& dbStamp, IMetadataRowSets& metadata)
at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore.SendUpdatePackage(XmlElement package, XmlElement& result, Boolean bulk)
--- End of inner exception stack trace ---
at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItem.Save(SaveFlags saveFlags)
at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItem.Save()
at VstsSyncMigrator.Engine.MigrationContextBase.SaveWorkItem(WorkItem workItem) in C:\Source\TFStoADO\MigrationTool\ado-migration-tools-8.4.7\src\VstsSyncMigrator.Core\Execution\MigrationContext\MigrationContextBase.cs:line 86
at VstsSyncMigrator.Engine.WorkItemMigrationContext.ProcessWorkItem(WorkItemStoreContext sourceStore, WorkItemStoreContext targetStore, Project destProject, WorkItem sourceWorkItem, Int32 retryLimit, Int32 retrys) in C:\Source\TFStoADO\MigrationTool\ado-migration-tools-8.4.7\src\VstsSyncMigrator.Core\Execution\MigrationContext\WorkItemMigrationContext.cs:line 190
migration.exe Warning: 0 : [EXCEPTION] Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemLinkValidationException: TF201066: You cannot add a Child link to work item 174222 because a work item can have only one Parent link. ---> System.Web.Services.Protocols.SoapException: TF201036: You can not add a Child link between work items 175803 and 174222 because a work item can have only one Parent link.
at Microsoft.TeamFoundation.WorkItemTracking.Proxy.RetryHandler.HandleSoapException(SoapException se)
at Microsoft.TeamFoundation.WorkItemTracking.Proxy.WorkItemServer.Update(String requestId, XmlElement package, XmlElement& result, MetadataTableHaveEntry[] metadataHave, String& dbStamp, IMetadataRowSets& metadata)
at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore.SendUpdatePackage(XmlElement package, XmlElement& result, Boolean bulk)
--- End of inner exception stack trace ---
at VstsSyncMigrator.Engine.WorkItemMigrationContext.ProcessWorkItem(WorkItemStoreContext sourceStore, WorkItemStoreContext targetStore, Project destProject, WorkItem sourceWorkItem, Int32 retryLimit, Int32 retrys) in C:\Source\TFStoADO\MigrationTool\ado-migration-tools-8.4.7\src\VstsSyncMigrator.Core\Execution\MigrationContext\WorkItemMigrationContext.cs:line 226
at VstsSyncMigrator.Engine.WorkItemMigrationContext.InternalExecute() in C:\Source\TFStoADO\MigrationTool\ado-migration-tools-8.4.7\src\VstsSyncMigrator.Core\Execution\MigrationContext\WorkItemMigrationContext.cs:line 119
at VstsSyncMigrator.Engine.MigrationContextBase.Execute() in C:\Source\TFStoADO\MigrationTool\ado-migration-tools-8.4.7\src\VstsSyncMigrator.Core\Execution\MigrationContext\MigrationContextBase.cs:line 35
MigrationEngine: The Processor WorkItemMigration entered the failed state...stopping run

0 replies

MrHinsh · 2019-10-31T21:55:14Z

MrHinsh
Oct 31, 2019
Maintainer

Yup, you will not be able to do that.

I recommend that you use a new empty organisation using https://aex.dev.azure.com/signup/ and run trials against that.

0 replies

fhack2 · 2019-10-31T22:00:34Z

fhack2
Oct 31, 2019
Author

@MrHinsh

This is TFS 2015 Update 4.1. I may have to go back to an older version of the tool.

0 replies

MrHinsh · 2019-10-31T22:18:23Z

MrHinsh
Oct 31, 2019
Maintainer

Ok, create a new collection and test against that.

0 replies

fhack2 · 2019-11-01T16:06:05Z

fhack2
Nov 1, 2019
Author

For the Test Plans and Suites: I run all the other processes to bring in the work items but Test Plans and Suites. Then run the process associated with the Test portion and get this:

migration.exe Information: 0 : Migration Context Start TestPlansAndSuitesMigrationContext
TestPlansAndSuites: Plan to copy 22 Plans?

===============================================================================================

== Suite Name: HelpDesk Finesse Phone QA =============================
== Date: 9/18/2017 =============================
== Suites: 15 =============================

===============================================================================================

== Plan[ 1/22] Suite[0/0] Cases[0/0] - planid[32410 ] | Process Plan HelpDesk Finesse Phone QA

migration.exe Warning: 0 : [EXCEPTION] System.NullReferenceException: Object reference not set to an instance of an object.
at VstsSyncMigrator.Engine.TestPlandsAndSuitesMigrationContext.ProcessTestPlan(ITestPlan sourcePlan) in C:\Source\TFStoADO\MigrationTool\ado-migration-tools-8.5.0\src\VstsSyncMigrator.Core\Execution\MigrationContext\TestPlansAndSuitesMigrationContext.cs:line 162
at VstsSyncMigrator.Engine.TestPlandsAndSuitesMigrationContext.InternalExecute() in C:\Source\TFStoADO\MigrationTool\ado-migration-tools-8.5.0\src\VstsSyncMigrator.Core\Execution\MigrationContext\TestPlansAndSuitesMigrationContext.cs:line 140
at VstsSyncMigrator.Engine.MigrationContextBase.Execute() in C:\Source\TFStoADO\MigrationTool\ado-migration-tools-8.5.0\src\VstsSyncMigrator.Core\Execution\MigrationContext\MigrationContextBase.cs:line 35

Should I also bring in Test Plans and Suites with the WorkItemMigrationConfig process?

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate first work items then links and attachments? #1425

{{title}}

Replies: 13 comments

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Migrate first work items then links and attachments? #1425

fhack2 Oct 31, 2019

Replies: 13 comments

MrHinsh Oct 31, 2019 Maintainer

fhack2 Oct 31, 2019 Author

MrHinsh Oct 31, 2019 Maintainer

MrHinsh Oct 31, 2019 Maintainer

fhack2 Oct 31, 2019 Author

MrHinsh Oct 31, 2019 Maintainer

fhack2 Oct 31, 2019 Author

MrHinsh Oct 31, 2019 Maintainer

fhack2 Oct 31, 2019 Author

MrHinsh Oct 31, 2019 Maintainer

fhack2 Oct 31, 2019 Author

MrHinsh Oct 31, 2019 Maintainer

fhack2 Nov 1, 2019 Author

migration.exe Information: 0 : Migration Context Start TestPlansAndSuitesMigrationContext TestPlansAndSuites: Plan to copy 22 Plans?

===============================================================================================

== Suite Name: HelpDesk Finesse Phone QA ============================= == Date: 9/18/2017 ============================= == Suites: 15 =============================

===============================================================================================

== Plan[ 1/22] Suite[0/0] Cases[0/0] - planid[32410 ] | Process Plan HelpDesk Finesse Phone QA

fhack2
Oct 31, 2019

MrHinsh
Oct 31, 2019
Maintainer

fhack2
Oct 31, 2019
Author

MrHinsh
Oct 31, 2019
Maintainer

MrHinsh
Oct 31, 2019
Maintainer

fhack2
Oct 31, 2019
Author

MrHinsh
Oct 31, 2019
Maintainer

fhack2
Oct 31, 2019
Author

MrHinsh
Oct 31, 2019
Maintainer

fhack2
Oct 31, 2019
Author

MrHinsh
Oct 31, 2019
Maintainer

fhack2
Oct 31, 2019
Author

MrHinsh
Oct 31, 2019
Maintainer

fhack2
Nov 1, 2019
Author

migration.exe Information: 0 : Migration Context Start TestPlansAndSuitesMigrationContext
TestPlansAndSuites: Plan to copy 22 Plans?

== Suite Name: HelpDesk Finesse Phone QA =============================
== Date: 9/18/2017 =============================
== Suites: 15 =============================