Single-file mode fails to iterate over record

[rosiel]

Running against a single file containing a <mods> root element:

$ python3 run.py -f /PATH/MODS.xml -x test.csv -xf csv -r mods
Traceback (most recent call last):
  File "run.py", line 87, in <module>
    current_record.ordered_split()
  File "/Users/rlefaive/Git/complex_xml_to_openrefine/record.py", line 47, in ordered_split
    for k, v in self.record.items():
AttributeError: 'str' object has no attribute 'items'

If this happens in all instances (and isn't unique to python3, or my MODS that doesn't include <?xml version="1.0" encoding="UTF-8"?> at the head, or...

then the fix could be

--- a/run.py
+++ b/run.py
@@ -75,7 +75,7 @@ if __name__ == "__main__":
         real_json_call = real_json
         for x in full_path:
             real_json_call = real_json_call[x]
-        our_list_of_records = real_json_call
+        our_list_of_records = [real_json_call]
     else:
         my_source = path_to_files
         records = Batch(path_to_files)

[mlhale7]

@rosiel - thanks for your interest in using this code. Would you mind sharing the command you used and the file so that it's easy to see what differences there are in how we're using this repository? On a Mac, I always need to put quotes around the delimiter to make it work.

[rosiel]

Hi @mlhale7, sorry for the delay. The command was python3 run.py -f /PATH/MODS.xml -x test.csv -xf csv -r mods (where /PATH/MODS.xml is the path to the MODS file). The file attached fails on that command.
MODS.xml.txt

[mlhale7]

@rosiel - thanks for this. I see that you're relying on the default for the delimiter (sorry!). While the PR you created does allow a single XML file to be processed, the change doesn't allow multiple records to be processed at the same time. Here's the error that I get when I run the updated run.py with an XML file that worked previously:

(XMLCSV_OR) bash-3.2$ python run.py -f wcc.xml -x wcc.csv -r /repository/record -xf csv -d "|" Traceback (most recent call last): File "run.py", line 87, in <module> current_record.ordered_split() File "/Users/Meredith/Documents/GitHub/complex_xml_to_openrefine/record.py", line 47, in ordered_split for k, v in self.record.items(): AttributeError: 'list' object has no attribute 'items'

I've attached wcc.xml.txt just so you can see the sort of data this code is expecting for use at UTK. The use case for this repository was remediating entire collections pulled via OAI-PMH (with the multiple records in a single file, e.g. wcc.xml) or entire collections pulled from the server (each MODS record as a .xml file within a folder). If you want to remediate a single MODS file, the edits you made work, but they aren't really useful to us. I'd say use your fork if you do intend to transform a single MODS file into a csv.

Your data is also different from ours in that we attempt never to have empty nodes (we don't use Islandora forms for creation), but from what I can tell the nulls don't seem to be a problem.

I'm definitely not the expert on this code or super-skilled at Python, so let me know if you have more thoughts. Hopefully this helps a little? Please let me know if this brings up more questions. I'm also happy to demo how I use the code sometime if that would be useful.

[rosiel]

Yes, that helps a lot, thank you! I realize that i was trying to see what the code did on one file before investing in the time to process an entire folder, and that remediating a single mods record doesn't actually make a lot of sense in the context of what this tool is for. d'oh!

If you have time to show me this code in the context of your work, that would be awesome. Thanks again!

[mlhale7]

@rosiel - no problem. I'm happy to schedule a call or attempt a demo during an Islandora metadata interest group meeting. Just let me know what would work for you.

[markpbaggett]

@mlhale7 Thanks for helping here.