loading pem files from jar file location does not work

[mkristian]

when installing gem using jruby-complete or the modular jruby artifact does not work anymore - not sure why it works with older jruby-openssl.

the offending line from rubygems is
https://github.com/jruby/jruby/blob/jruby-1_7/lib/ruby/shared/rubygems/request.rb#L34

the problem here is that file locations are either a string like this:

/usr/local/repository/org/jruby/jruby-stdlib/1.7.17-SNAPSHOT/jruby-stdlib-1.7.17-SNAPSHOT.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/ssl_certs/Class3PublicPrimaryCertificationAuthority.pem 

or within the jruby/jruby#1957 where it is something like this

uri:classloader:/META-INF/jruby.home/lib/ruby/shared/rubygems/ssl_certs/Class3PublicPrimaryCertificationAuthority.pem

also file location like this do not work

jar:file:/usr/local/repository/org/jruby/jruby-stdlib/1.7.17-SNAPSHOT/jruby-stdlib-1.7.17-SNAPSHOT.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/ssl_certs/Class3PublicPrimaryCertificationAuthority.pem

well, the problem is obvious, the first string is not even an uri and as such a valid path (with one directory having a name ending in !)
the other problem is
https://github.com/jruby/jruby-openssl/blob/master/src/main/java/org/jruby/ext/openssl/X509Store.java#L147
where is assumes it is a path argument and not some "uri" kind of argument.

translating the uri-like location to URI or URL is kind of tricky since it needs the runtime to maybe find the right URL - something like:
https://github.com/jruby/jruby/blob/test-load-from-jruby-classloader/core/src/main/java/org/jruby/runtime/load/LibrarySearcher.java#L280

@ratnikov maybe you have an idea how go about this. since I think it is general problem not restricted to jruby-openssl.

easy way to reproduce it with rmvn initialize and Mavenfile:

gem 'maven-tools', '1.0.5'
jruby_plugin :gem do execute_goal :initialize; end
jar 'org.jruby', 'jruby', '1.7.17-SNAPSHOT'
properties 'tesla.dump.pom' => 'pom.xml', 'jruby.version'=> '1.7.17-SNAPSHOT'

which will use the jruby-stdlib-1.7.17-SNAPSHOT.jar build with mvn -Pmain

[kares]

nothing we can do about it (or I mean we should not hack around this) since it's a JRuby loading issue, right?

[mkristian]

@kares not really it is related to https://github.com/jruby/jruby-openssl/blob/master/src/main/java/org/jruby/ext/openssl/x509store/Lookup.java#L281
which basically ONLY works for file-system path and NOT for the uri like pathes the load-services produces.

t.rb:

#gem 'jruby-openssl', '0.9.5'
gem 'jruby-openssl', '0.9.6.dev'
require 'net/https'

p Gem.loaded_specs

$CLASSPATH << 'maven/jruby-complete/target/jruby-complete-9000.dev-SNAPSHOT.jar'

store =  OpenSSL::X509::Store.new

store.add_file 'lib/ruby/shared/rubygems/ssl_certs/GeoTrustGlobalCA.pem'

store.add_file 't.rb'

store.add_file 'file:lib/ruby/shared/rubygems/ssl_certs/GeoTrustGlobalCA.pem'

store.add_file 'jar:file:maven/jruby-complete/target/jruby-complete-9000.dev-SNAPSHOT.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/ssl_certs/GeoTrustGlobalCA.pem'

store.add_file 'uri:classloader:/META-INF/jruby.home/lib/ruby/shared/rubygems/ssl_certs/GeoTrustGlobalCA.pem'

store.add_file 'uri:jar:file:maven/jruby-complete/target/jruby-complete-9000.dev-SNAPSHOT.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/ssl_certs/GeoTrustGlobalCA.pem'

store.add_file 'classpath:/META-INF/jruby.home/lib/ruby/shared/rubygems/ssl_certs/GeoTrustGlobalCA.pem'

just run it with jruby t.rb.

it assumes either jruby-openssl 0.9.5 or 0.9.6.dev installed as a gem and the maven/jruby-complete/target/jruby-complete-9000.dev-SNAPSHOT.jar file be present to illustrate the various situations where and how this "add_file" fails.

the results varies between jruby-1.7.15 and jruby-9000k and between 0.9.5 and 0.9.6.dev. BUT all pathes in the file a valid and are used.

[mkristian]

it feels like using RubyString for uri like path is not sufficient, something like RubyURIString could solve it (maybe) which knows how to turn the string into an inputstream. the runtime is needed here since the file can very much - due to a jruby feature - be located inside runtime.getJRubyClassLoader !!!

[kares]

@mkristian I see, thanks - obviously there's no tests for this thus I sure did not know it needs to work as the load service ;( ... so we're just need to delegate to some JRuby internals here that are present in 1.6.8 - 9k ... hopefully that will be possible. did any of this this actually work in 0.9.4 (I would assume it did not) ?

where are these expectations coming from these days ... isn't it possible to pass in something normalized ?
I'm still kind of confused whether this is a regression or how did we end up with these requirements ...

[mkristian]

well, the problem is that in MRI X509Store#add_file just expect a path to a file which is always be on the filesystem. but with jruby a path to file can any of those cases I put in the file and look at how rubygems find those path to the pem files:
https://github.com/jruby/jruby/blob/master/lib/ruby/shared/rubygems/request.rb#L32
i.e. it goes over the LoadService and gets bunch of "pathes" which can be anything depending how jruby.home is set or the more general case is how $LOAD_PATH is setup and then any of my examples are possible.

and that is not the whole picture: ``$CLASSPATH << 'my.jar'` can have ruby files which will be found by the old LoadService code as well and those are not "exposed" to the $LOAD_PATH.

I am digging out those "edge" cases since I tried to simplify the whole loading of things: jruby/jruby#1957
and some "normalization" is needed else where too: jruby/jruby#1983

with jruby-9000.dev it is a regression for some case as you expected here: f54b3bc

[kares]

I see thanks a lot ... that is what I thought - needed to confirm that this is not really a regression (since 0.9.4)