Lookup.loadCertificateOrCRLFile consumes more memory because of frequent memory allocation at Certificate.matches

[frsyuki]

Overview

• Since jruby-openssl 0.9.8, Lookup.loadCertificateOrCRLFile calls Store.matchedObject, and it calls Certificate.matches method for each stored certificates.
• Certificate.matches method calls x509.getSubjectX500Principal() method. This method internally encodes aux certificate using Bouncy Castle's ASN1OutputStream.
• This encoding procedure creates many objects.
• It impacts GC performance badly.

Background

We upgraded JRuby 9.0.0.0 to 9.0.5.0 and found that our application started consuming much more memory. This application uses httpclient.gem to fetch data from HTTPS server. Here is its code:
https://github.com/treasure-data/embulk-input-sfdc/blob/master/lib/embulk/input/sfdc_api/api.rb

Here is a screenshot of memory profiler:

• JRuby 9.0.0.0 (which uses jruby-openssl v0.9.7)
  
• JRuby 9.0.5.0 (which uses jruby-openssl v0.9.11)
  

This shows that 9.0.5.0 allocates objects more frequently and triggers GC frequently (2x - 3x more frequent).

Memory profiler shows that under Lookup.loadCertificateOrCRLFile method, JRuby 9.0.0.0 doesn't allocate many objects but 9.0.5.0 allocates many objects at Store.addCertificate in addition to regular allocation at PEMInputOutput.readPEM:

• JRuby 9.0.0.0
  
• JRuby 9.0.5.0
  

Under Store.addCertificate, following stack is allocating majority of objects:

• Store.matchedObject
  • Certificate.matches
    • X509AuxCertificate.getSubjectX500Principal
      • ASN1OutputStream.writeObject
        • DERSequence.encode
          • ...

I think that Store.matchedObject method is just a lookup method but X509AuxCertificate.getSubjectX500Principal is allocating many objects unexpectedly.

Expected behavior

I think that Store.matchedObject or Certificate.matches can cache the result of X509AuxCertificate.getSubjectX500Principal() to reduce object allocation.

Workaround

Memory allocation becomes much less frequent if I add jruby.openssl.x509.lookup.cache=8 system property:

• JRuby 9.0.5.0 + jruby.openssl.x509.lookup.cache=8
  

But this option is disabled by default now.

[kares]

believe this is right, added jruby.openssl.x509.lookup.cache property as a HotSpot security provider contention work-around. its good that you're able to use it but I'm not sure caching by default is a good idea for a security library (although in this case it might be harmless).

any ideas on reducing Certificate.matches without caching? // cc @mkristian

[mkristian]

@kares I was thinking about this. bit hard without caching. as Certificate.matches could "cache" the hashCode of x509.getSubjectX500Principal() and first compares the hashCodes before diving into the X500Prinicipal object:
https://github.com/jruby/jruby-openssl/blob/master/src/main/java/org/jruby/ext/openssl/x509store/Certificate.java#L55

well, just an idea and on a first glance I hope it sound reasonable ;)

[kares]

@mkristian good point, although we might need to look into how hashCode is implemented in BC ...
so that we do not end up doing the same encoding while doing it.

[kares]

@frsyuki if possible could you guys try a --pre release from https://oss.sonatype.org/content/repositories/snapshots/rubygems/jruby-openssl/0.9.17.dev-SNAPSHOT/ ... did an experiment to only use BC certificate's getSubjectX500Principal when comparing against a non-BC cert. wonder what memory consumption would behave in cases such as yours (with lookup caching turned off).

[kares]

seems like its caused by extensive memory use with BC 1.54 where they need the provider to be registered.
details at #94 ... work-around would be to port over some of the X.509 factory code so it does not need to instantiate a new provider with every certificate. should get "huge" memory consumption drop dramatically.

[headius]

This is fixed, isn't it?

[kares]

@headius yep it should be, thanks