Description

As we centralize on the batched update DAG as a standard update mechanism for the project, it may be beneficial to have a log of the batched update runs that have been performed. This can serve as both a changelog for the data as well as a rubric for how to perform updates on more complex fields (like the tags).

It might be hard for this to live as a page on our docs site, because batched update runs can (and should) happen outside the context of PRs. It may, however, be possible to create a new Airflow page which would generate content dynamically using a custom plugin. This page could query all the batched update runs and surface the following information, in reverse chronological order (potentially with hard links for each run so they could be referenced externally):

• query_id
• table_name
• select_query
• update_query
• batch_size
• link to the DAG run
• link to the GitHub issue
• duration for select (if available)
• duration for update (if available)