Skip to content

WIP SRUopener #682

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 9 commits into
base: master
Choose a base branch
from
Draft

WIP SRUopener #682

wants to merge 9 commits into from

Conversation

dr0i
Copy link
Member

@dr0i dr0i commented Mar 28, 2025

This is a draft and WIP.
@TobiasNx you can use it for functional testing.

Resolves #510.

@dr0i dr0i requested a review from TobiasNx March 28, 2025 12:18
@dr0i dr0i changed the title WIP SRUopener (#510) WIP SRUopener Mar 28, 2025
@dr0i dr0i moved this to Review in Metafacture Mar 28, 2025
@TobiasNx
Copy link
Contributor

Nice seems to work. +1
The printed logs are a little bit esoteric:

, startRecord=1, maximumRecords=1000, istream.length=7865
urlToOpen=https://services.dnb.de/sru/zdb?query=dnb.isil%3DDE-Sol1&operation=searchRetrieve&recordSchema=MARC21plus-xml&version=1.1&maximumRecords=1000&startRecord=1001
, startRecord=1001, maximumRecords=1000, istream.length=7865
urlToOpen=https://services.dnb.de/sru/zdb?query=dnb.isil%3DDE-Sol1&operation=searchRetrieve&recordSchema=MARC21plus-xml&version=1.1&maximumRecords=1000&startRecord=2001
, startRecord=2001, maximumRecords=1000, istream.length=7865
urlToOpen=https://services.dnb.de/sru/zdb?query=dnb.isil%3DDE-Sol1&operation=searchRetrieve&recordSchema=MARC21plus-xml&version=1.1&maximumRecords=1000&startRecord=3001
, startRecord=3001, maximumRecords=1000, istream.length=7865
urlToOpen=https://services.dnb.de/sru/zdb?query=dnb.isil%3DDE-Sol1&operation=searchRetrieve&recordSchema=MARC21plus-xml&version=1.1&maximumRecords=1000&startRecord=4001
, startRecord=4001, maximumRecords=1000, istream.length=437

@TobiasNx
Copy link
Contributor

@dr0i is still in review?

@dr0i
Copy link
Member Author

dr0i commented Apr 10, 2025

As we found out in #510 this PR needs a complete redesign.

@dr0i dr0i force-pushed the 510-addSruOpener branch from ecd9c8c to c3f3ad6 Compare April 10, 2025 13:32
@dr0i dr0i force-pushed the 510-addSruOpener branch from 84d6845 to 3dc0416 Compare June 2, 2025 14:24
@dr0i
Copy link
Member Author

dr0i commented Jun 2, 2025

@TobiasNx can you do functional tests before I go on here? Have a look at the @Description to see how it works (hint: "stream" based, i.e. other than the OAI-PMH opener works atm.)
I've added the class to flux-commands.
[edit]: and ignore the failing editorconfigChecker for now.

@dr0i dr0i requested a review from TobiasNx June 2, 2025 14:38
@TobiasNx
Copy link
Contributor

TobiasNx commented Jun 4, 2025

@dr0i I tried to install the dist: https://metafacture.github.io/metafacture-documentation/docs/flux/Flux-User-Guide.html#build-from-local-distribution to try the runner for functional testing

but it runs into errors:

$ ./gradlew installDist

> Configure project :
HEAD has no annotated tags
No SCM tag found. Making a snapshot build
Feature branch found
Version is feature-510-addSruOpener-SNAPSHOT

[Incubating] Problems report is available at: file:///home/user/git/metafacture-core/build/reports/problems/problems-report.html

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.13/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

When I test the flux.sh then it outputs the following:

$ /home/user/git/metafacture-core/metafacture-runner/build/install/metafacture-core/flux.sh
Exception in thread "main" java.lang.ExceptionInInitializerError
        at org.metafacture.runner.Flux.main(Flux.java:62)
Caused by: org.metafacture.commons.reflection.ReflectionException: Class not found: org.metafacture.io.
        at org.metafacture.commons.reflection.ReflectionUtil.loadClass(ReflectionUtil.java:70)
        at org.metafacture.commons.reflection.ObjectFactory.loadClassesFromMap(ObjectFactory.java:57)
        at org.metafacture.flux.parser.FluxProgramm.<clinit>(FluxProgramm.java:54)
        ... 1 more
Caused by: java.lang.ClassNotFoundException: org.metafacture.io.
        at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
        at org.metafacture.commons.reflection.ReflectionUtil.loadClass(ReflectionUtil.java:67)
        ... 3 more

Can you help? (I tested the current master to compare, there $ ./gradlew installDist and $ /home/user/git/metafacture-core/metafacture-runner/build/install/metafacture-core/flux.sh works)

@TobiasNx TobiasNx assigned dr0i and unassigned TobiasNx Jun 4, 2025
@dr0i
Copy link
Member Author

dr0i commented Jun 5, 2025

Ah, I accidently removed the TarReader.
Try again please.

@dr0i dr0i assigned TobiasNx and unassigned dr0i Jun 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Review
Development

Successfully merging this pull request may close these issues.

Add SRU opener / open-sru
2 participants