XML to_s error

[jkthorne]

I am trying to write XMLs to a file and I am getting this error.

Invalid memory access (signal 11) at address 0x0
[0x10e782916] *Exception::CallStack::print_backtrace:(Int32 | Nil) +118
[0x10e738a0e] __crystal_sigfault_handler +318
[0x7fff203b7d7d] _sigtramp +29
[0x7fff2720f3b4] xmlFindCharEncodingHandler +176
[0x7fff272776aa] xmlNewSaveCtxt +64
[0x7fff27277889] xmlSaveToIO +31
[0x10e8bcaf6] *XML::Node#to_xml<IO::FileDescriptor>:IO::FileDescriptor +358
[0x10e8bc986] *XML::Node#to_s<IO::FileDescriptor>:Nil +6
[0x10e8058a5] *IO::FileDescriptor@IO#<<<XML::Node>:IO::FileDescriptor +37
[0x10e805867] *IO::FileDescriptor@IO#puts<XML::Node>:Nil +39
[0x10e743819] *puts<XML::Node>:Nil +57
[0x10e8f454a] *Braintree::CLI::DisputeCreateCommand::resolve<Braintree::Operations::Dispute::Sandbox::OpenDispute.class, Braintree::CLI>:NoReturn +2378
[0x10e8f3b78] *Braintree::CLI::DisputeCreateCommand::run<Braintree::CLI>:NoReturn +280
[0x10e89be72] *Braintree::CLI#run:(Bool | Nil) +5042
[0x10e89a951] *Braintree::CLI::run:(Bool | Nil) +17
[0x10e723b2f] __crystal_main +1775
[0x10e9062c6] *Crystal::main_user_code<Int32, Pointer(Pointer(UInt8))>:Nil +6
[0x10e90612c] *Crystal::main<Int32, Pointer(Pointer(UInt8))>:Int32 +44
[0x10e72f166] main +6

When I try to reduce this bug down it it starts working again. Here are the versions of XML and Crystal but are the most updated versions on homebrew.

$ crystal --version
Crystal 0.36.1 (2021-02-02)

LLVM: 11.0.1
Default target: x86_64-apple-macosx

$ brew info libxml2
libxml2: stable 2.9.10 (bottled), HEAD [keg-only]
GNOME XML library
http://xmlsoft.org/
/usr/local/Cellar/libxml2/2.9.10_2 (280 files, 10.6MB)
...

Here is a branch with the section of code that is raising the error. Also it works when parsed as HTML.

https://github.com/wontruefree/braintree/blob/bug_xml_parse/src/cli/dispute_create_command.cr#L14-L17

[asterite]

What's the text?

[asterite]

Put another way: could you provide code that I can copy/paste into a file, run it and see it break?

[jkthorne]

I have been unable to reproduce this in a smaller code sample if I read this from a file or make it a string literal in any way it seems to work and when I use a client it fails. I will try to get a smaller sample.

Here is a sample of the full XML
https://gist.github.com/wontruefree/c9d05934c80d71e2d128c5530a93bcec

[jkthorne]

This still breaks for me but now only breaks on one macbook pro and not another. I have not been able to reproduce when not making remote calls so it is hard to make a code sample for this. I am closing this to keep the issues clean. But this might be a problem introduced with 0.36.x

[jkthorne]

I was able to narrow this down without making web requests. It seems like certain ordering of xpath methods followed by a puts breaks the program. Here is a link to the working and broke code samples.

Here is the broken sample. https://gist.github.com/wontruefree/f7efd84b0cbd4eb1d2f1f32bfe249e2a#file-xml-bug

$ crystal tmp.cr
<dispute>
      <id>dpp74szzb5srtw4d</id>
      <global-id>ZGlzcHV0ZV9kcHA3NHN6emI1c3J0dzRk</global-id>
      <amount>9363.36</amount>
      <amount-disputed>9363.36</amount-disputed>
      <amount-won>0.00</amount-won>
      <case-number>CB328430743124</case-number>
      <created-at type="datetime">2021-02-05T04:40:16Z</created-at>
      <currency-iso-code>USD</currency-iso-code>
      <date-opened type="date">2021-02-05</date-opened>
      <date-won nil="true"/>
      <processor-comments nil="true"/>
      <kind>chargeback</kind>
      <merchant-account-id>kilmercorp</merchant-account-id>
      <reason>fraud</reason>
      <reason-code>62</reason-code>
      <reason-description nil="true"/>
      <received-date type="date">2021-02-05</received-date>
      <reference-number nil="true"/>
      <reply-by-date type="date">2021-02-10</reply-by-date>
      <processor-reply-by-date type="date">2021-02-14</processor-reply-by-date>
      <response-deadline type="datetime">2021-02-09T05:59:59Z</response-deadline>
      <status>open</status>
      <updated-at type="datetime">2021-02-05T04:40:16Z</updated-at>
      <original-dispute-id nil="true"/>
      <evidence type="array"/>
      <status-history type="array">
        <status-history>
          <disbursement-date type="date">2021-02-05</disbursement-date>
          <effective-date type="date">2021-02-05</effective-date>
          <status>open</status>
          <timestamp type="datetime">2021-02-05T04:40:16Z</timestamp>
        </status-history>
      </status-history>
      <transaction>
        <id>fj6df5n1</id>
        <global-id>dHJhbnNhY3Rpb25fZmo2ZGY1bjE</global-id>
        <amount>9363.36</amount>
        <created-at>2021-02-05T04:40:15Z</created-at>
        <installment-count nil="true"/>
        <order-id nil="true"/>
        <purchase-order-number nil="true"/>
        <payment-instrument-subtype>Visa</payment-instrument-subtype>
      </transaction>
    </dispute>
Invalid memory access (signal 11) at address 0x0
[0x100fa8786] *Exception::CallStack::print_backtrace:(Int32 | Nil) +118
[0x100f8494e] __crystal_sigfault_handler +318
[0x7fff203b7d7d] _sigtramp +29
[0x7fff2720f3b4] xmlFindCharEncodingHandler +176
[0x7fff272776aa] xmlNewSaveCtxt +64
[0x7fff27277889] xmlSaveToIO +31
[0x10100e366] *XML::Node#to_xml<IO::FileDescriptor>:IO::FileDescriptor +358
[0x10100e1f6] *XML::Node#to_s<IO::FileDescriptor>:Nil +6
[0x100ffe075] *IO::FileDescriptor@IO#<<<XML::Node>:IO::FileDescriptor +37
[0x100ffe037] *IO::FileDescriptor@IO#puts<XML::Node>:Nil +39
[0x100f85189] *puts<XML::Node>:Nil +57
[0x100f74f6b] __crystal_main +4427
[0x101014b06] *Crystal::main_user_code<Int32, Pointer(Pointer(UInt8))>:Nil +6
[0x10101496c] *Crystal::main<Int32, Pointer(Pointer(UInt8))>:Int32 +44
[0x100f80566] main +6

And the other one prints out he XML.

[straight-shoota]

That code runs without failure for me. Same libxml2 version, but on linux. Crystal latest is linked against LLVM 10, but master build is linked against LLVM 11.1.

$ crystal --version
Crystal 0.36.1 [c3a3c1823] (2021-02-02)

LLVM: 10.0.0
Default target: x86_64-unknown-linux-gnu

$ bin/crystal --version
Using compiled compiler at .build/crystal
Crystal 1.0.0-dev [de9351f05] (2021-02-20)

LLVM: 11.1.0
Default target: x86_64-unknown-linux-gnu

$ brew info libxml2
libxml2: stable 2.9.10 (bottled), HEAD
GNOME XML library
http://xmlsoft.org/
/home/linuxbrew/.linuxbrew/Cellar/libxml2/2.9.10_2 (281 files, 12.5MB)
...

So looks like it only fails on macos. Can anyone reproduce it?

Could also be worth trying with an older Crystal compiler and/or other libxml2 versions.

This could likely a bug in libxml2, though (or iconv or whatever would be involved).

[bcardiff]

Unable to repro osx crystal 0.36.1, llvm 11.0.1, libxml2 2.9.10

$ crystal --version
Crystal 0.36.1 (2021-02-02)

LLVM: 11.0.1
Default target: x86_64-apple-macosx

$ brew info libxml2
libxml2: stable 2.9.10 (bottled), HEAD [keg-only]
GNOME XML library
http://xmlsoft.org/
/usr/local/Cellar/libxml2/2.9.10_2 (280 files, 10.5MB)
  Poured from bottle on 2021-01-26 at 16:37:51
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/libxml2.rb
License: MIT

$ crystal run --verbose foo.ign.cr
cc _main.o D-ir.o C-rystal5858S-ystem5858D-ir.o ... stripped ... 
E-xception5858C-allS-tack5858R-epeatedF-rame.o C-rystal.o 
-o /Users/bcardiff/.cache/crystal/crystal-run-foo.ign.tmp  -rdynamic 
-L/usr/local/lib -L/usr/local/Cellar/libxml2/2.9.10_2/lib -lxml2 
-lpcre -lgc -lpthread /usr/local/Cellar/crystal/0.36.1_2/src/ext/libcrystal.a 
-L/usr/local/Cellar/libevent/2.1.12/lib -levent -liconv -ldl
<dispute>
      <id>dpp74szzb5srtw4d</id>
      <global-id>ZGlzcHV0ZV9kcHA3NHN6emI1c3J0dzRk</global-id>
      <amount>9363.36</amount>
... stripped ...
  <pin-verified type="boolean">false</pin-verified>
</transaction>

[jkthorne]

So I can reproduce this on my macbookpro(2015) but not on a macbookpro(2019).
Same versions of crystal and libxml2.

[bcardiff]

@wontruefree can you check the output of $ pkg-config libxml-2.0 --libs ?

If it is

$ pkg-config libxml-2.0 --libs
-lxml2

it means you are not using brew libxml but the default system one.

if you do export PKG_CONFIG_PATH="/usr/local/opt/libxml2/lib/pkgconfig:$PKG_CONFIG_PATH" the output should be

$ pkg-config libxml-2.0 --libs
-L/usr/local/Cellar/libxml2/2.9.10_2/lib -lxml2

With crystal run --verbose you will be able to see which link commands are used near the -lxml2

[jkthorne]

The output was:

$ pkg-config libxml-2.0 --libs
-lxml2

I used export PKG_CONFIG_PATH="/usr/local/opt/libxml2/lib/pkgconfig:$PKG_CONFIG_PATH" rebuild crystal and cleared the cache and I am still getting the same issue.
currently the pkg-config output is this.

$ pkg-config libxml-2.0 --libs
-L/usr/local/Cellar/libxml2/2.9.10_2/lib -lxml2