DH draft version 1.2 uses <core> which isn't supported by harvester

[jar398]

The Darwin Core Archive version 1.0 documentation that I found describes and elements and a few others that don't seem to be supported by the harvester. For an example in the EOL opendata archive, see version 1.2 of the dynamic hierarchy.

Is this intentional? Are DwCA meta.xml files supposed to be run through a preprocessor before being deposited in opendata? Or is this just a feature that isn't yet supported by the harvester?

(I haven't tried this, I just make a conclusion based on the code in app/models/resource/from_meta_xml.rb.)

[JRice]

Indeed, the harvester is not intended to implement the full DwCA spec, but a very specific set of them. There is official documentation for it, and I believe it even lives on the DwCA site, but I'm afraid that was before my time, so I'm not sure where. Maybe Jennifer knows? :S Regardless: it's true. Many tags are completely ignored. We also add *significant* functionality to the base DwCA spec. We do talk about divorcing from the spec now and then, but: that would be a lot of work, so it is a can that gets kicked down the road every time it comes up. :)

…

On Fri, Aug 20, 2021 at 12:30 PM Jonathan A Rees ***@***.***> wrote: The Darwin Core Archive version 1.0 documentation <https://www.gbif.jp/v2/pdf/gbif_dwc-a_metafile_en_v1.pdf> that I found describes and elements and a few others that don't seem to be supported by the harvester. For an example in the EOL opendata archive, see version 1.2 of the dynamic hierarchy. Is this intentional? Are DwCA meta.xml files supposed to be run through a preprocessor before being deposited in opendata? Or is this just a feature that isn't yet supported by the harvester? (I haven't tried this, I just make a conclusion based on the code in app/models/resource/from_meta_xml.rb.) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#18>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAF43VX6BAJPTN6SNWEQI3T5Z7JRANCNFSM5CQUOV3Q> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .

[jar398]

Hmm. I'll take the harvester source to be the documentation for this format then.

So what can I tell Katja about how to change DH 1.2 to conform to the harvester? What procedure is used to make other DwCA sources conform to the harvester?

[JRice]

So what can I tell Katja about how to change DH 1.2 to conform to the harvester?

Ho-boy, that's a loaded question. :S I don't know what I would say without taking a few hours (at least) to pick things apart and suggest changes, if any. The harvester has by-and-large been written to conform to the needs of the other EOL folks, which in turn are based on the needs of specific clients, folding in new features are required to handle new data-types that hadn't been encountered yet. This is, of course, not wonderfully efficient. If we were to start from scratch, I'd have suggested a lot of changes. :D But we have what we have, and it's based on what we've processed thus far. ...Are there specific areas of concern WRT to these DH->harvester changes?

What procedure is used to make other DwCA sources conform to the harvester?

Eli, Jennifer, and Katja have historically ironed that out manually. That said, I'm not sure how many clients actually hand us DwCA formatted data to begin with: I suspect only a handful. Most, as I understand it, are Excel worksheets that EOL puts together, and those use a pre-fab meta.xml that is then tweaked as needed. I think Eli converts the rest?