Remove 'RSA' from RSA public key headers for openssl CLI tool compatibility

[jkuchta]

I noticed jRuby OpenSSL uses a slightly modified header and footer for public keys (values below), this causes issues when working with other tools/libraries (example below).

-----BEGIN RSA PUBLIC KEY-----
-----END RSA PUBLIC KEY-----

instead of

-----BEGIN PUBLIC KEY-----
-----END PUBLIC KEY-----

The added 'RSA' causes issues when working with other tools/libraries, the main ones being the OpenSSL Command Line Tool (https://www.openssl.org/docs/manmaster/apps/openssl.html) and BouncyCastle on the Java side using PEMReader with RSAPublicKey. Here is a brief example to illustrate the issue:

Example

Step 1

jruby-1.7.9 :030 > key = OpenSSL::PKey::RSA.new(<private_key here>)
jruby-1.7.9 :030 > key.public_key
 => -----BEGIN RSA PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4jBMv6SurdEZRAlvqjqC
QfjD5in6H1gNvIhhaKDzazt0/QHFGk+tn5nue5T/fSiAhcZqKEXx7D1OZ+GW1Hfm
6CJR3L8cRSM8rr+KqlRqUEEgGP5v2cYfGRXd1MYy8NvMFK7yZSSeNTtcycWqrXkV
KpkS6vBGKMCHBgkjUk9fUdN3kcd7+PUcD42xMJLm8zA48j1lsG+C95eT7VLt/AbI
UC0VJgOdPEx/xp5fNYFdwBTVbjCMQsnNYbgq9Xyk2vWp2u1SspJ+LBkRXUNxrjB8
vER22EV0jAbeB4mM1uG+6SZ8612a0z2wYMtsPRR0+QxP5C6R/hnc+Qw+adQ1PAvH
iQIDAQAB
-----END RSA PUBLIC KEY-----

Step 2

Save above key to test.pub then run the following:

 > openssl rsa -pubin -in test.pub -pubout -outform der | openssl dgst -sha256
unable to load Public Key
18766:error:0906D06C:PEM routines:PEM_read_bio:no start line:/SourceCache/OpenSSL098/OpenSSL098-52.40.1/src/crypto/pem/pem_lib.c:648:Expecting: PUBLIC KEY

Step 3

Edit and save test.pub as test2.pub and remove the 'RSA' from header and footer so it looks like the following:

-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4jBMv6SurdEZRAlvqjqC
QfjD5in6H1gNvIhhaKDzazt0/QHFGk+tn5nue5T/fSiAhcZqKEXx7D1OZ+GW1Hfm
6CJR3L8cRSM8rr+KqlRqUEEgGP5v2cYfGRXd1MYy8NvMFK7yZSSeNTtcycWqrXkV
KpkS6vBGKMCHBgkjUk9fUdN3kcd7+PUcD42xMJLm8zA48j1lsG+C95eT7VLt/AbI
UC0VJgOdPEx/xp5fNYFdwBTVbjCMQsnNYbgq9Xyk2vWp2u1SspJ+LBkRXUNxrjB8
vER22EV0jAbeB4mM1uG+6SZ8612a0z2wYMtsPRR0+QxP5C6R/hnc+Qw+adQ1PAvH
iQIDAQAB
-----END PUBLIC KEY-----

Step 4

Rerun openssl command line command using test2.pub

> openssl rsa -pubin -in test2.pub -pubout -outform der | openssl dgst -sha256
writing RSA key
93ffc9e5984fb7d78d43a8896749d9777ad6a6784e6a95869468995de7052f3a

I'm not sure if having RSA in the private key creates any problems or not, but if it does a similar change could be implemented for that as well.

[kares]

Thanks Jeff, would you mind adding a simple test with encoding/decoding an OpenSSL formated RSA public key ... e.g. here https://github.com/jruby/jruby-openssl/blob/master/src/test/ruby/oaep/test_oaep.rb
Also maybe DSA should be handled the same way as well ... if you're at it.

[jkuchta]

Sure, I was also able to confirm that the private key headers/footers are correct and it is just the ones for the public keys that need to change.

[jkuchta]

@kares I have added some tests for OpenSSL formatted keys and made the DSA change.

[mkristian]

great to see this little improvements getting jossl more in line with openssl - thanx @jkuchta