currency missing in datastore when default-currency attribute of iati-activities/iati-activity is set

[jarofgreen]

Brief Description
See https://iatiregistry.org/dataset/worldbank-289
Data has a default currency set but transactions have no currency value set.

Now see data from id 44000-P093826 in the datastore

Transactions have no currency set.

This is because I can't see anywhere in the code where default-currency is used to populate other values

Severity
medium

Issue Location
see above

Steps to Reproduce
see above

Expected Results/Behaviour and Actual Results/Behaviour

This gets into a discussion about how useful the data store should be; should it just report data as raw as possible or should it try and make it useful?

In this case, the data store is technically correct - the transactions do have no currency set.

However, as a user who queries transactions I probably don't care whether the currency is a default one or set in that transaction, I just care what currency is in use and I just want the data right there so I can process it with no further work.

Note D-Portal does do this, and actually massages the XML that comes out so transaction value elements have a currency set. https://d-portal.org/q.xml?aid=44000-P093826

[jarofgreen]

As a work around when querying SOLR records, noting the field default_currency is currently available

[jarofgreen]

Note that default-currency can also be set in the organisation file (credit @simon-20 )

[robredpath]

The transactions index is already an opinionated view of the data so I think it's perfectly acceptable to include relevant information from elsewhere in the published data which aids with interpretation or practical use of that data.

In principle, I think it's important that we mark any modifications that we've made to the data. So, I would prefer not to take the approach that d-portal has done but rather do one (or both) of:

• Include a field for each value from elsewhere in the published data that we are importing (e.g. iati-activity-default-currency and iati-organisation-default-currency)
• Create a field which uses a documented algorithm (probably proximity) to create a calculated-currency field which users can rely on to find the relevant currency in their analysis.

[jarofgreen]

I was wondering about if that would be a use case, but I leaned to another conclusion.

Use Case A) as a user who queries transactions I don't care whether the currency is a default one or set in that transaction, I just care what currency is in use and I just want the data right there so I can process it with no further work
Use Case B) I'm doing some analysis on how much default-currency is used and I want data on where the currency is coming from

My thinking was that Use Case A would be far more important to far more people than Use Case B and thus coming to the conclusion:

• The currency field should always have a value, where ever it comes from. This makes it easy for people to use the data who aren't going to read details in docs, less surprises
• We add a new field currency-source with values of "direct", "activity", "organisation" (or something) for the minority of people who care about Use Case B

[IsabelBirds]

Default currency is already included in transaction level downloads, along with default aid type, finance type, etc. If we start changing currency there are a lot of other elements to consider.

To use IATI data directly from the API/Datastore requires some understanding of the standard to avoid drawing incorrect conclusions, so engaging enough to understand activity level default values is no bad thing.