Design a "bot" to assist "managing" of the dandisets by authorized users

[yarikoptic]

DataLad dandisets are not "sources", they are automagically updated by a cron service running on drogon. As such we should not give any write permissions to original authors/owners of the dandisets as known to the DANDI system.

Only DANDI archive has information about which users have write access to the original dandiset, and thus should be capable e.g. to

• modify list of users associated with dandiset in Triage role. E.g. @dandibot add @login1 @login2 to the dandiset, @dandibot remove @login1 @login2 from the dandiset (Design a "bot" to assist "managing" of the dandisets by authorized users #360)
• "accept" PR, if any is sent out to e.g. fix an issue in some sidecar/text file - thus opening easy ways to fix up dandisets via github. E.g. @dandibot accept PR
• resync labels (Add command to populate/manage custom github labels in all/provided dandisets #361)
• ...

It will be responsibility of the bot to first verify that the author of the command is among authors of the dandiset to operate on the command.

Such bot service might also help to show/seek confirmation from the authors for some scheduled/mass maintenance tasks etc as we mind stormed with @rly during ODIN, e.g.

• upgrade .nwb files:
  • from some ancient version (e.g. 1.0a) over to a newer one
• upgrade dandisets with some auxiliary metadata (e.g. semantic annotations etc)

Some extra "cron" jobs which might need to be performed but not necessary need to be a part of "bot" I guess:

• checking list of each dandiset users with Triage role which had been invited but have not confirmed/responded (removed themselves), and email a reminder/instructions on how to accept invitation or remove themselves (Design a "bot" to assist "managing" of the dandisets by authorized users #360). Actually -- we might not want to remind if at least one "Triage" user has replied. The others might be PIs etc, not desired to pester if already somebody took care.

TODOs (yet to be finalized)

• check out e.g. conda-forge bot(s). They must reflect lots of knowledge/best practices to design such beasts
• ...

refs:

• Design of issue tracking/reporting via GitHub dandi-archive#1595

[yarikoptic]

FWIW https://github.com/dependabot is a great bot/project/resource to checkout .Since it is quite modular and configurable, I even started to wonder if we could build on top of it even if for some jobs...

[rly]

The JOSS editorial-bot https://github.com/openjournals/buffy might also be a good bot/project/resource to check out.

[jwodder]

Preliminary research:

• At a high level, setting up a GitHub bot (properly known as a "GitHub App") involves building a publicly-accessible HTTP server and configuring GitHub to send requests called "webhooks" to the server whenever certain events happen. If the server receives an actionable webhook, it takes action via the normal GitHub APIs.

• The steps for creating an app seem simple enough, but I'd find it hard to believe if no one has yet created a generic GitHub App framework in Python to save us from having to write everything from scratch.

• A GitHub App can be public or private.

  • A private app can only be associated with at most one organization, so if we want to use this bot in multiple organizations (e.g., dandisets and dandizarrs), we would have to duplicate the setup on the GitHub end in order to create two identical apps in different organizations pointing to the same server.

  • Public apps can be installed in any number of organizations and by any GitHub user, though if we don't choose to put it on the Marketplace, I'm not sure how anyone else would find it.

---

Next step: Search PyPI and elsewhere for pre-existing GitHub App frameworks

[jwodder]

FWIW https://github.com/dependabot is a great bot/project/resource to checkout .Since it is quite modular and configurable, I even started to wonder if we could build on top of it even if for some jobs...

I'm pretty sure that Dependabot's modularity & configurability is all oriented around updating dependencies; e.g., you can add a module for looking up packages in a new package source, but you can't add a module for operating on dandisets. Also, it's written in Ruby, which I don't know and I don't think anyone else on the team knows.

[jwodder]

[Work in Progress]

Features to judge packages on:

• Validating webhook deliveries
• Generating a JWT
• Generating an installation access token
  • Installation access tokens are valid for one hour, so ideally our bot should reuse them until they expire instead of generating a new one on every webhook event; do any of these packages help with this?
• Tracking X-GitHub-Delivery headers [1] [2]
• Integration with Flask or another web framework
  • Dispatching based on event & action
• I'm assuming that, for at least some webhook events, our bot is going to want to spawn asynchronous tasks (through celery or similar) rather than blocking the HTTP request handler until the webhook is fully acted upon. Do any of these packages help with this?
  • Relevant GitHub docs: [link]

Relevant (to varying degrees) Python packages found so far:

• https://pypi.org/project/github-bot-api/
• https://pypi.org/project/Flask-GitHubApplication/
• https://github.com/Mariatta/github_app_boilerplate
  • https://github-app-tutorial.readthedocs.io/en/latest/
  • Built on top of gidgethub
• https://pypi.org/project/flask-github-webhook/
• https://github.com/bradshjg/flask-githubapp
• https://pypi.org/project/PyGHee/
• https://pypi.org/project/github-webhooks-framework/
• https://github.com/gidgethub/gidgethub
  • https://gidgethub.readthedocs.io/en/latest/apps.html
  • https://gidgethub.readthedocs.io/en/latest/routing.html
• https://pygithub.readthedocs.io/en/stable/github_objects/GithubIntegration.html
  • Only provides facilities for generating a JWT and installation access token

GitHub apps written in Python:

• https://github.com/python/bedevere
• https://github.com/python/miss-islington
• https://github.com/conda-forge/conda-forge-webservices — conda-forge-admin bot (not to be confused with the regro-cf-autotick-bot that creates PRs in feedstocks; that one is implemented here via GitHub Actions rather than as a GitHub App)