ClassCastException error in X509Store.verify

[cprice404]

I've been debugging an issue that surfaced when I filed a PR to upgrade JRuby 1.7 to jruby-openssl 0.9.18.

The issue is that when the jruby external tests for x509store are run, there is a stack trace that looks like this:

test_verify(OpenSSL::TestX509Store):
Java::JavaLang::ClassCastException: org.bouncycastle.jce.provider.X509CertificateObject cannot be cast to org.jruby.ext.openssl.X509Cert
    org.jruby.ext.openssl.X509StoreContext.initialize(X509StoreContext.java:132)
    org.jruby.ext.openssl.X509StoreContext.newStoreContext(X509StoreContext.java:102)
    org.jruby.ext.openssl.X509Store.verify(X509Store.java:225)
    org.jruby.ext.openssl.X509Store$INVOKER$i$0$0$verify.call(X509Store$INVOKER$i$0$0$verify.gen)
    org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:218)

After some investigation I discovered that there is a test that looks very similar included directly in the jruby-openssl repo:

https://github.com/jruby/jruby-openssl/blob/master/src/test/ossl/1.9/test_x509store.rb#L36

If I check out the 0.9.18 tag of jruby-openssl, and run this test locally by setting up JRuby 1.7.26 as my local jruby and running:

jruby -Ilib:. src/test/ossl/1.9/test_x509store.rb

I get the same error. Examining the travis.yml file, and doing some local experimenting with the various rake tasks for running tests, it appears that this test is never being run by CI.

I can debug the failure itself and with any luck maybe file a PR to fix it, but I don't know how to go about modifying the code so that the test would be run in CI. It seems like, if these tests are useful, they should be run in CI?

I'm also trying to figure out why this hasn't shown up in JRuby9k, which appears to have already been upgraded to jruby-openssl 0.9.18. If I clone a fresh copy of the jruby repo and run from the HEAD of the master branch:

$ mvn -P bootstrap
...
$ export PATH=./bin:$PATH
$ jruby --version
jruby 9.1.7.0-SNAPSHOT (2.3.1) 2016-11-21 89759e4 OpenJDK 64-Bit Server VM 25.111-b14 on 1.8.0_111-8u111-b14-2ubuntu0.16.04.2-b14 +jit [linux-x86_64]
$ jruby -r ./test/mri_test_env.rb test/mri/runner.rb  test/mri/openssl/test_x509store.rb

I'm able to reproduce basically the same error:

# Running tests:

[3/3] OpenSSL::TestX509Store#test_verify = 0.18 s           
  1) Error:
OpenSSL::TestX509Store#test_verify:
Java::JavaLang::ClassCastException: org.bouncycastle.jce.provider.X509CertificateObject cannot be cast to org.jruby.ext.openssl.X509Cert
    org.jruby.ext.openssl.X509StoreContext.initialize(X509StoreContext.java:132)
    org.jruby.ext.openssl.X509StoreContext.newStoreContext(X509StoreContext.java:102)
    org.jruby.ext.openssl.X509Store.verify(X509Store.java:225)
    org.jruby.ext.openssl.X509Store$INVOKER$i$0$0$verify.call(X509Store$INVOKER$i$0$0$verify.gen)
    org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:212)

It seems unlikely that these tests wouldn't be getting run in CI for JRuby9k, so maybe I'm messing something up in my local environment?

[cprice404]

Asking about this in IRC, and was told that the MRI tests for JRuby9k are run in CI via mvn -Prake -Dtask=test:mri. I ran that and grepped all of the output for "openssl" and "x509" and didn't see any matches, leading me to believe that the openssl tests may not be getting run in CI for JRuby9k.

[cprice404]

I did a bisect and it appears to me that the src/test/ossl/1.9/test_x509store.rb test was passing prior to this commit, and has been failing since then:

ef7a170

UPDATE - the test failure that started showing up in v0.9.14 appears to be an ArrayIndexOutOfBoundsException. The ClassCastException seems like it may not have started happening until v0.9.18. I'm going to keep focusing on the ClassCastException for now, but it seems like there may be some additional bugs that warrant separate tickets.

[olleolleolle]

@cprice404 Thank you for digging.

About the next version (after 1.54):

BouncyCastle 1.55 release notes had this perhaps-interesting line:

Change Warning (users of 1.52 or earlier): The PEM Parser now returns an X509TrustedCertificate block when parsing an openssl trusted certificate, the new object was required to allow the proper return of the trusted certificate's attribute block. Please also see the porting guide for advice on porting to this release from much earlier ones (release 1.45 or earlier).

[cprice404]

w/rt the BC version:

• If I build the current HEAD of jruby-openssl against BC 1.52, I still get this ClassCastException when I run the tests.
• The current jruby-openssl code will not compile against BC 1.51.
• Based on the BouncyCastle release notes, I suspect that the change that is causing the ClassCastException was introduced in 1.51, but I can't tell for certain since jruby-openssl won't compile against that version.

[cprice404]

Here is a minimal reproducer test that I've been using in case anyone else looks into this:

def test_foo
    now = Time.at(Time.now.to_i)
    ca_exts = [
        ["basicConstraints","CA:TRUE",true],
        ["keyUsage","cRLSign,keyCertSign",true],
    ]
    ee_exts = [
        ["keyUsage","keyEncipherment,digitalSignature",true],
    ]
    ca1_cert = issue_cert(@ca1, @rsa2048, 1, now, now+3600, ca_exts,
                          nil, nil, OpenSSL::Digest::SHA1.new)
    ca2_cert = issue_cert(@ca2, @rsa1024, 2, now, now+1800, ca_exts,
                          ca1_cert, @rsa2048, OpenSSL::Digest::SHA1.new)
    ee1_cert = issue_cert(@ee1, @dsa256, 10, now, now+1800, ee_exts,
                          ca2_cert, @rsa1024, OpenSSL::Digest::SHA1.new)

    puts "EE1CERT: #{ee1_cert.class}"
    puts "CA2CERT: #{ca2_cert.class}"

    # puts "EE1CERT.to_java: #{ee1_cert.to_java.class}"
    # puts "CA2CERT.to_java: #{ca2_cert.to_java.class}"
    # puts "[EE1CERT].to_java: #{[ee1_cert].to_java(java.util.List)[0].to_java.class}"
    # puts "[CA2CERT].to_java: #{[ca2_cert].to_java(java.util.List)[0].to_java.class}"

    store = OpenSSL::X509::Store.new
    store.verify(ee1_cert, [ca2_cert])
    # store.verify(ee1_cert, [ee1_cert])
  end

[cprice404]

A few more interesting things that I've learned:

1. Given the above test, on jruby-openssl v0.9.13 before the BC upgrade, if I add this line before the call to verify, I get a ClassCastException:

    [ca2_cert].to_java(java.util.List)

2. On jruby-openssl v0.9.18, in the implementation for X509StoreContext.initialize, there is some code that iterates over the chain array. Doing so seems to implicitly call the toJava method of X509Cert, which gives us back a BC object rather than an instance of org.jruby.ext.openssl.X509Cert. The current implementation of that toJava method was introduced in this commit ecc1a05 , which only exists in v0.9.18.

I still don't understand exactly what's going on here but it seems like the implicit calls to toJava during an array iteration are at least partly to blame. The specific implementation of toJava that was added in v0.9.18 seems likely to cause some problems when called implicitly, but since this type of test will also fail on v0.9.14 (where that toJava doesn't exist), the current toJava implementation doesn't explain the full situation.

[kares]

fixed with #114 ... (in >= 0.9.19)