MIGRATION: Evaluate user mapping

[toddfarmer]

In anticipation of a potential migration from ASF Jira to GitHub for issue tracking, an assessment is needed on whether we can and should map user accounts in migrated content. GitHub and ASF Jira use different user identifiers, which often differ (or collide with unrelated users) across systems. This has implications for:

• Issue reporter
• Issue assignee
• Watchers
• Comment authors
• User mentions

Issue assignee could be retained if we have a mapping table to convert Jira user accounts to corresponding GitHub user accounts for individuals. This could be "best effort", and fall back to simply recording the Jira user name and display name in a comment if no mapping is supplied.

Issue reporter cannot be set using the GitHub issue import API, so I plan to record the reporter displayName and username in a prefix to the issue description (see example here.

Watchers cannot be set via the import API. We may be able to supply a simple script for interested users to execute that would set themselves as a watcher on GitHub issues that were migrated from Jira issues they were watching.

Comment authors cannot be set via the import API. I've prefixed the issue comment with the original reporter displayname and username, and a link back to the original Jira comment.

User mentions can be mapped as part of content mapping. If there is no user account mapping, this could fall back to using the ASF Jira username in a code block (e.g., [~toddfarmer]).

[jorisvandenbossche]

Issue reporter cannot be set using the GitHub issue import API, so I plan to record the reporter displayName and username in a prefix to the issue description

Python actually did that when migrating their issues to GitHub last year, but I suppose they did that with the help of GitHub employees (and not with only the public import APIs).
https://discuss.python.org/t/github-issues-migration-mannequin-users/14965

(also https://stackoverflow.com/questions/36540508/github-api-possible-to-set-reporter-of-issue-to-another-user-when-creating-issu)

[ianmcook]

Issue assignee could be retained if we have a mapping table to convert Jira user accounts to corresponding GitHub user accounts for individuals.

As noted here on the mailing list:

To help with this, I created a couple of small scripts that extract details from resolved Arrow Jira issues and the associated merged GitHub PRs, then perform some cleaning, to generate a list of matching Jira user identifiers and GitHub user identifiers.

This is an incomplete list: it does not include users who have created Arrow Jira issues but never created an Arrow PR that was merged.

Please do some sanity-checking on this list before using it.

[ianmcook]

I updated the list to include both the key and name fields from Jira, which sometimes differ. I'm not sure which of these fields will be most convenient to use for mapping.

[pitrou]

I would suggest to append the GH handles (if available) of the issue reporter and watchers at the end of the issue description, together with the name and JIRA profile address.

This is so important so that those people get GH notifications for the issues they were subscribed to on JIRA.

For example:

In anticipation of a potential migration from ASF Jira [... snip issue description ...]

* Reporter: @toddfarmer ([Todd Farmer](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=toddfarmer))
* Watchers: @pitrou ([Antoine Pitrou](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=apitrou)), @ianmcook ([Ian Cook](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=icook))

[toddfarmer]

My current belief is that we cannot effectively migrate issue watchers/subscriptions from Jira to GitHub issues, and that it may be necessary to create a simple Python script for individuals to execute post-migration that will:

1. Query Jira to find existing issues they are subscribed to
2. Obtain the migrated GitHub issue identifier from a designated field in the Jira issue
3. Subscribe them to the migrated GitHub issue

The above seems something we can reasonable automate in a simple script, along with documentation on how to set up necessary auth to successfully execute it. But it seems we can't directly import the watchers, even if we can map user identifiers. Unless I'm missing it, the only possible viable alternative that retains issue subscriptions would be to perform an OAuth impersonation dance for every needed user/subscription - and as was noted in the links @jorisvandenbossche cited above, that seems to only be available for GitHub Enterprise customers (seems unlikely that applies to us).

[pitrou]

Wouldn't my proposal above work?

[toddfarmer]

The import process explicitly does not generate notifications, I believe (that's part of the advertised value of using it). If you are proposing a post-import comment be added that explicitly mentions all mapped GitHub user accounts that were assigned or watching (that we are able to map) in a normal GitHub issue update API call, I suspect it would. I'll test to validate.

I can also evaluate post-import issue updates to append proposed text to the issue description.

[pitrou]

Ah, I didn't know that (though it makes sense). Then perhaps a post-import comment would work indeed?

[toddfarmer]

Testing adding subscriptions via mentions.

Separately, I noticed that we cannot migrate who closed an issue. I don't think this matters, particularly if we can map and retain assignee information for closed issues, but it's worth calling out that GitHub issues have an explicit "closed_by" field, which will be populated by the account used to perform the import operation.

[jorisvandenbossche]

The import process explicitly does not generate notifications,

But that's OK for the purpose of adding "watchers", I think? As a watcher of an issue, I don't need notifications for all issues that I am watching when they are migrated, but I do want notifications when someone adds a new comment after the migration.

Or does the import API even avoid notifications for those later comments for users that were @ mentioned in comments created by the import API?

[toddfarmer]

Or does the import API even avoid notifications for those later comments for users that were @ mentioned in comments created by the import API?

That does seem to be the case, yes. User mentions executed added via the import API seem to not affect the issue notification subscriptions. Subsequent comments on the issue where a user was mentioned in import content do not trigger notification; an additional comment mentioning the user is required to subscribe them to notification. This could be accomplished through a two-part process:

1. Import the existing issue content via the import API
2. Add a comment referencing users who were subscribed the the source issue (author, assignee, watchers).

If we coordinate with GitHub support to suppress notifications during the import process, we may be able to manipulate the notification subscribers via the added comment during a window where no notification will be generated, avoiding excess noise to users.

[jorisvandenbossche]

This could be accomplished through a two-part process:

That sounds good to me

[toddfarmer]

It turns out that it requires an extra Jira API call for every migrated issue to collect the watcher list. I have concerns about API rate limiting here. I'll test this out with a full export to see if rate limits are hit. If they are, I may add a secondary process to sync the watchers afterwards. Issue reporter and assignee will be mentioned in the content, and automatically added to issue notifications as a result.

[rok]

I've tried contacting github regarding mannequin users via public discussions, support.github.com and github.com/services, but they were not very responsive. ASF infra didn't have any suggestions so I propose we simply use @asfimport to import our issues (example of lucene).

• assignees - they can be set at import time and don't trigger notifications. I expect assignees are subscribed to the issue. I think we should assign at import time.

• watchers - I was able to extract them from jira (see example issue). However I'm not sure mass subscribing everyone to issues where they were watchers is a good idea as we'd have to do it with @ mentions which would generate a lot of spam. We could solve this by making Jira watchers GitHub assignees (up to 10 are possible), but I'd prefer the subscription script option Todd is proposing.