ServiceFileSystemTesting

TODO: This wiki page is out of date. The recommended test to run is MockHarvestTest, not ProcessFilesTest. The following page needs to be edited with information regarding MockHarvestTest...

Testing

We've created a testing infrastructure which allows for simple yet thorough testing of your service. This infrastructure is a mock representation of the real MST system. Since it's file based you can use it to quickly test your service w/out installing a full-blown MST system. This infrastructure is also the way you'll want to do regression testing since you can automatically compare your service's actual output with what you expect the output to be (see here).

1. create a folder inside of the test/input-records folder for a specific group of records you'd like to test
2. create one or more files with one or more records in them. (see this example)
3. optional: create expected output records to compare against. The best way to get this output is to copy it over from your build/test/actual_output_folder once you're comfortable with your results. So the first time through you can ignore this step.
4. issue this command

ant -Dtest=ProcessFiles test

* if you wish to only produce output for one of your subdirectories, add an additional parameter to the ant command line like this:

ant -Dtest=ProcessFiles -Dtest.folder=myFolder test

* _optional_:  If your actual output doesn't match the expected output, this test will fail informing you which files differ.  If you don't have expected output records to compare against, just ignore the fact that the test fails for this reason.  You can find the result of the test in build/test/Ant suite/index.html.  You'll see in the stack trace something like this:

Caused by: java.lang.RuntimeException: 
orig-186/bogus.xml: file exists in actual, but not expected.
orig-186/92365.xml: files differ

1. Examine the output records in build/test/actual_output_records

Input Records Syntax

Input is pretty straight forward. The elements of significance are

• identifier: This can really be anything, but it needs to be unique so that the system knows if it's an update
• status: for new records and update records this isn't required because it is inferred. The only time you need to include this is for a "delete"
• metadata: This is the actual payload. Notice that a timestamp is missing. This is because time only matters in as much as we need to know which records comes first. This test system reads the files in order (alphabetically) and records within each file in order. So, if a specific record is found more than once, it is considered an update or delete (just as it would be in oai-pmh). So, we'd suggest you prefix your files with numbers and perhaps leave room to fit others in (something like 0100-logical-name.xml).

Here's an example. In this example you see an new record, a record update, and a record deletion. We'll use this example as the basis for our output record example.

<records xmlns="http://www.openarchives.org/OAI/2.0/">
  <record>
    <header>
      <identifier>oai:mst.rochester.edu:bib:1</identifier>
    </header>
    <metadata>
      <foo xmlns="foo:bar">pb&amp;j</foo>
    </metadata>
  </record>
  <record>
    <header>
      <identifier>oai:mst.rochester.edu:bib:1</identifier>
    </header>
    <metadata>
      <foo xmlns="foo:bar">pb&amp;j 2</foo>
    </metadata>
  </record>
  <record>
    <header>
      <identifier>oai:mst.rochester.edu:bib:1</identifier>
      <status>deleted</status>
    </header>
    <metadata>
      <foo xmlns="foo:bar">pb&amp;j</foo>
    </metadata>
  </record>
</records>

Output Records Syntax

The output record syntax is slightly more nuanced than the input. The reason is because we need to keep track of predecessors and for verbosity's sake we output everything that happened in the system. It's kind of like one big timeline. If a record is input twice, then we output the record's output twice (noting which output records are active and which aren't). note: this is not exactly how records will be harvested. We've made this output a little different to make testing easier.

• identifier: This is generated by the MST. The test system will start numbering from 1 and increment by 1.
• status: This is always populated with one of the following options.
  • active: This means this record is active. There will always only be one active record per oai-pmh id.
  • deleted: The record has been deleted.
  • replaced: This version of the record no longer exists because another version has replaced it.
• predecessors: this contains one or more predecessor oai-ids.
• metadata: This is the actual metadata.
• datestamp: this can be ignored since the only thing that matters is the chronological order (which is determined by the ordering of the records in the file)

<records xmlns="http://www.openarchives.org/OAI/2.0/">
<record>
  <header>
    <identifier>oai:mst.rochester.edu:example/1</identifier>
    <datestamp />
    <predecessors>
      <predecessor>oai:mst.rochester.edu:bib:1</predecessor>
    </predecessors>
    <status>replaced</status>
  </header>
  <metadata>
    <foo xmlns="foo:bar">
      pb&amp;j
      <bar>you've been foobarred!</bar>
    </foo>
  </metadata>
</record>
<record>
  <header>
    <identifier>oai:mst.rochester.edu:example/1</identifier>
    <datestamp />
    <predecessors>
      <predecessor>oai:mst.rochester.edu:bib:1</predecessor>
    </predecessors>
    <status>active</status>
  </header>
  <metadata>
    <foo xmlns="foo:bar">
      pb&amp;j 2
      <bar>you've been foobarred!</bar>
    </foo>
  </metadata>
</record>
</records>