Open
Description
Problem
Much of our tallies data is captured on a weekly or monthly basis. For example, provider occurrences in search:
These also do not have a TTL and will live forever in our Redis.
Description
We need an implementation plan for the creation of a DAG (or something) to export data for non-active tally keys from production Redis, save it to a queryable location suitable for long-term storage, and then clean the keys from production Redis.
I've requested an IP for this because it probably deserves some careful forethought and whatever solution we take would require wiring up new Airflow providers or infrastructure that doesn't currently talk to each other.
Related to #420
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Metadata
Assignees
Labels
Type
Projects
Status
📋 Backlog