Implementations of innerSerialize lack proper namespace

[ralfhergert]

(Concerning marklogic-java-client in 6.1.0) The StructuredQueryBuilder currently only allows to serialize a given StructuredQueryDefinition into a String. In cases in which the query needs to be embedded into another XML this makes it necessary to parse the query again.

Here I would kindly request the following feature: Could you please create a public accessible method like:
StructuredQueryBuilder#serialize(XMLStreamWriter) ?

This would vastly improve the possibilities for serialization. For instance org.jdom2.input.StAXStreamWriter could be used to directly serialize a query into a JDOM.

Meanwhile to get access to the method

java-client-api/marklogic-client-api/src/main/java/com/marklogic/client/query/StructuredQueryBuilder.java

Line 1239 in c3dc589

public abstract void innerSerialize(XMLStreamWriter serializer) throws XMLStreamException;

we subclassed StructuredQueryBuilder.
By using org.jdom2.input.StAXStreamWriter we noticed, that the namespaces are wrong. What should look like

<query
    xmlns="http://marklogic.com/appservices/search"
    xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <word-query>
        <field name="title"/><text>Test</text>
    </word-query>
</query>

was created as:

<query xmlns="http://marklogic.com/appservices/search"
    xmlns:search="http://marklogic.com/appservices/search">
    <word-query xmlns="">
        <field name="title"/><text>Test</text>
    </word-query>
</query>

Notice how the default namespace on word-query is reset.

If not mistaken I would believe this is due to the fact that the elements (like the word-query) are created with no namespaceUri. c.f.

java-client-api/marklogic-client-api/src/main/java/com/marklogic/client/query/StructuredQueryBuilder.java

Line 1851 in c3dc589

serializer.writeStartElement("word-query");

this should be instead

serializer.writeStartElement(SEARCH_API_NS, "word-query");

Shouldn't it?

[rjrudin]

Hi @ralfhergert - to confirm, I think you are identifying an enhancement and a bug:

1. Allow for serializing a query via an XMLStreamWriter.
2. The existing serialization has a bug where query elements incorrectly have no namespace as opposed to the search namespace.

Is that correct?

For the first item - just to clarify, the existing serialize method is a member of AbstractStructuredQuery, not StructuredQueryBuilder. So I believe your request would be to add serialize(XMLStreamWriter) to AbstractStructuredQuery, right?

[ralfhergert]

Hi @rjrudin. Yes my issue is both, a feature request and a bug issue. I think they are connected: as soon as a method like serialize(XMLStreamWriter) exists it gets substantial that the correct events are being produced.

Concering the class, since AbstractStructuredQuery is an internal class (not intended for public use). It might not be the right place, eighter. The right place could be the interface StructuredQueryDefinition. This interface also declares the parameter-less serialize-method.

[rjrudin]

Thanks @ralfhergert , you're right - StructuredQueryDefinition is the correct location.

We've captured both the enhancement request and the bug report. We'll let you know soon about either a 6.1.x bug release for the latter or a 6.2.0 for both.

[ralfhergert]

Thank you for the notification, Rob Rudin! I'm looking forward to 6.2.0 and will come back if issues arrise. Best regards Am Di., 28. März 2023 um 15:12 Uhr schrieb Rob Rudin < ***@***.***>:

…

Hi @ralfhergert <https://github.com/ralfhergert> - we're planning our 6.2.0 release and looking to include this. Just wanted to confirm - when I use the existing serialize() method to write a query to a String, I don't run into any issues with incorrect namespaces. So the existing serialize() feature seems to be working correctly. The bug you pointed out then is something we're likely to run into when we try to support serialize(XMLStreamWriter), correct? So basically, this is an enhancement request to support serialize(XMLStreamWriter), and you're also giving us a heads' up that we may run into issues with incorrect namespaces in the output of that method? — Reply to this email directly, view it on GitHub <#1537 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABDY7UDT4PSWZ7BT2CR2L4DW6LPTXANCNFSM6AAAAAAVU7BS2Q> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Viele Grüße Ralf Hergert -- Ferdinand-Weiß-Str. 123a 79106 Freiburg +49 151 20295146 +49 761 2171323

[rjrudin]

Hey @ralfhergert - we're planning our 6.2.0 release and looking to include this. Do you have a code snippet to share where you're using org.jdom2.input.StAXStreamWriter? I'm thinking that would be a good test case for us.

[rjrudin]

Ah - I see why innerSerialize won't currently work for the intended use case (which makes sense, since it's not public) - it's only invoked in the context of writeStructuredQueryImpl, which has a default namespace set.

[rjrudin]

@ralfhergert Take a look at the simple test I wrote in 4373679 . That produces the correct result, but it relies on the user doing the following:

1. Must call factory.setProperty(XMLOutputFactory.IS_REPAIRING_NAMESPACES, true); in order for a default namespace to work correctly.
2. Must set the default namespace to the Search namespace - serializer.setDefaultNamespace(SEARCH_NS);.
3. And then the kicker - it appears necessary to wrap the serialized query in an element in the Search namespace - e.g. serializer.writeStartElement(SEARCH_NS, "theMarkLogicQuery");.

If you do that, then serialize(XMLStreamWriter) will work correctly.

I'm wondering if those requirements are feasible for you though - particularly the need for a wrapper element.

If not, I am thinking it would be necessary to modify the existing serialization implementation so that the Search namespace is always applied when an element is started. That's feasible, I just wanted to check on the above approach with you first.

[rjrudin]

Hmm - having said the above, it looks simple to just modify the few dozen places where the Java Client starts an element so that the Search namespace is always used. I can't think of any downsides to this. I did some testing of this with a default namespace, and STaX behaves the way you'd expect - i.e. if the default namespace is the Search namespace, it won't e.g. create a new namespace prefix or anything funky like that.

So I'm now thinking the right way to address this is:

1. Expose serialize(XMLStreamWriter) in StructuredQueryDefinition.
2. Ensure that the Search namespace is applied to each element regardless of how the above method is used.

[rjrudin]

@ralfhergert Sorry one more question - is your use case here to construct a combined query document - e.g. https://docs.marklogic.com/guide/rest-dev/search#id_84389 - using STaX? Or are you constructing some other kind of document?

[rjrudin]

Resolved via #1543 . We'll release 6.2.0 soon.

[rjrudin]

@ralfhergert 6.2.0 is now available, though we're still in the process of updating the Javadocs. But the method you're looking for is now there.