You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
At present we have the following data update scripts:
iati.sh
iati_datastore.sh
iati_registry_refresh.sh
iati_transactions.sh
iati_transactions_retry.sh
We should clean up, simplify, and refactor these scripts such that they:
Run Python/iati_refresh.py, marking IATI datasets as either new, modified, or stale in the iati_registry_metadata table.
Run a new script based on Python/iati_transactions.py, which is capable of modularly modifying multiple IATI-based tables at the moment new data is loaded (IATI nomenclature would call this iati_reload.py).
Run auxiliary scripts such as Python/iati_rhfp.py which rely on the entire data structure and don't need to be run strictly on new/modified files.
The idea is that instead of running iati_transactions.sh run once, at which point the information about whether a dataset is new is destroyed, any number of tables can be progressively built during the reload process just as iati_transactions.py is.
The text was updated successfully, but these errors were encountered:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
At present we have the following data update scripts:
We should clean up, simplify, and refactor these scripts such that they:
The idea is that instead of running iati_transactions.sh run once, at which point the information about whether a dataset is new is destroyed, any number of tables can be progressively built during the reload process just as iati_transactions.py is.
The text was updated successfully, but these errors were encountered: