Fix false positives in gpg format (symmetric encryption)

[kholia]

Currently, cracking of GPG symmetrically encrypted files lacking "MDC" results in a lot of false positives.

http://www.ssi.gouv.fr/uploads/2015/05/format-Oracles-on-OpenPGP.pdf should be of help.

https://www.reddit.com/r/crypto/comments/ujot7/gpg_dont_know_invalid_packet_ctb48/

http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=blob_plain;f=doc/DETAILS

http://www.ietf.org/rfc/rfc1991.txt

https://github.com/hawken93/gpg-symcrack (based on GnuPG itself)

Format Oracles on OpenPGP paper (backup copy).

[kholia]

https://github.com/magnumripper/JohnTheRipper/tree/no-mdc-reduce-candidates has some initial work but nothing promising so far.

[kholia]

UPDATE: Even https://github.com/kholia/PGPCrack-NG generates false positives! PGPCrack-NG is just a hacked up version of GnuPG. So solving this problem 100% does not seem to be possible. We can try reducing the number of false positives at best.

[kholia]

Note:

Look at how https://github.com/hawken93/gpg-symcrack works, and try to borrow some features from it, to reduce the rate of false positives.

Update:

The gpg-symcrack program does not work at all for files without MDC. For files with MDC, it aborts pretty badly.

[kholia]

Sample GPG encrypted file to demonstrate the problem of false positives.

The hash is,
$gpg$*0*69*6089a3038b7fb38fb796ec86a43e369646fcbae04f5ae4c8712790157d29d23afaca8518f093e3fe3310225f5f5350615a67c1f34a5a94c2d0bb558d9b005bb81fcb50db26*3*9*2*3*65536*70f8a591dbffa300

Brute forcing this hash results in a lot of false positives very quickly. The correct password is qwertyzxcvb12345.

[kholia]

One of the false positives is FutureFuture. PGP 8 wrongly decrypts the .gpg file with this wrong password.

Update: Symantec Encryption Desktop 10.4.1 also does not reject the false positive password.

I was hoping to find a "readable" implementation of PGP which rejects the false positives but no luck.

[kholia]

With GnuPG 1.4.22 and FutureFuture password,

$ gpg --decrypt sample_fp_qwertyzxcvb12345.gpg.txt 
gpg: CAST5 encrypted data
gpg: encrypted with 1 passphrase
gpg: [don't know]: invalid packet (ctb=50)
gpg: WARNING: message was not integrity protected
gpg: decrypt_message failed: unexpected data

We essentially want to replicate this behavior in our format.

[kholia]

I made some progress on this task today. It should be possible to make our format as good as PGPCrack-NG (essentially modified GnuPG).

[kholia]

WIP patch,

diff --git a/src/gpg_common_plug.c b/src/gpg_common_plug.c
index ff56e21..d2bbd59 100644
--- a/src/gpg_common_plug.c
+++ b/src/gpg_common_plug.c
@@ -1177,6 +1177,14 @@ freestuff:
        return rc;
 }
 
+static void print_hex(unsigned char *str, int len)
+{
+       int i;
+       for (i = 0; i < len; ++i)
+               printf("%02x", str[i]);
+       printf("\n");
+}
+
 int gpg_common_check(unsigned char *keydata, int ks)
 {
        // Decrypt first data block in order to check the first two bits of
@@ -1285,7 +1293,16 @@ int gpg_common_check(unsigned char *keydata, int ks)
                case CIPHER_CAST5: {
                                           CAST_KEY ck;
                                           CAST_set_key(&ck, ks, keydata);
-                                          CAST_cfb64_encrypt(gpg_common_cur_salt->data, out, gpg_common_cur_salt->datalen, &ck, ivec, &tmp, CAST_DECRYPT);
+                                          if (gpg_common_cur_salt->symmetric_mode && gpg_common_cur_salt->usage == 9) {
+                                                  // handle PGP's weird CFB mode, do this for each cipher, take care of block-size!
+                                                  CAST_cfb64_encrypt(gpg_common_cur_salt->data, out, 10, &ck, ivec, &tmp, CAST_DECRYPT);
+                                                  tmp = 0;
+                                                  CAST_set_key(&ck, ks, keydata);
+                                                  memcpy(ivec, gpg_common_cur_salt->data + 2, 8); // GCRY_CIPHER_ENABLE_SYNC, cipher_sync
+                                                  CAST_cfb64_encrypt(gpg_common_cur_salt->data + 10, out + 10, gpg_common_cur_salt->datalen - 10, &ck, ivec, &tmp, CAST_DECRYPT);
+                                          } else {
+                                                  CAST_cfb64_encrypt(gpg_common_cur_salt->data, out, gpg_common_cur_salt->datalen, &ck, ivec, &tmp, CAST_DECRYPT);
+                                          }
                                   }
                                   break;
                case CIPHER_BLOWFISH: {
@@ -1346,11 +1363,23 @@ int gpg_common_check(unsigned char *keydata, int ks)
                MEM_FREE(out);
                return 0;
        } else if (gpg_common_cur_salt->symmetric_mode && gpg_common_cur_salt->usage == 9) {
-               // https://www.ietf.org/rfc/rfc2440.txt
-               if ((out[9] == out[7]) && (out[8] == out[6])) { // XXX this verifier is not good at all!
-                       MEM_FREE(out);
-                       return 1;
+               int ctb;
+
+               // https://www.ietf.org/rfc/rfc2440.txt, http://www.ietf.org/rfc/rfc1991.txt
+               if ((out[9] != out[7]) || (out[8] != out[6]))
+                       goto bad;
+
+               // See parse() from g10/parse-packet.c
+               ctb = out[10];
+               if (!(ctb & 0x80)) {
+                       goto bad;
                }
+
+               // XXX Add more checks
+
+               MEM_FREE(out);
+               return 1;
+bad:
                MEM_FREE(out);
                return 0;
        }

[kholia]

Some other PGP parsers,

• https://github.com/postgres/postgres/blob/master/contrib/pgcrypto/pgp-decrypt.c

• https://github.com/linuxmint/mint4win/blob/master/src/openpgp/sap/pkt/Packet.py

• https://github.com/golang/crypto/blob/master/openpgp/packet/packet.go

[kholia]

Further action items,

• Can we extend the "compression checks" by doing actual decompression? Do we really need to?

• Test "no MDC" stuff with IDEA + 3DES ciphers (done, fixed).

• Can we extend gpg2john.c and the hash format to provide some more control to the user? Specifically, the user should be able to indicate that compression was really used. This way, we can reject cases where pkttype == 11 (the main source of false positives). Can we overload the gpg_common_cur_salt->usage field to achieve this?

• Test Twofish with symmetric encryption stuff (done).

[kholia]

Currently the code has a comment saying Random note: MDC is only missing for ciphers with block size <= 64 bits?.

The answer to this question is NO!. Very old PGP version 8.0 is able to use AES-256 (by default), and create output files lacking MDC.

Sample files,

secret-twofish.txt.pgp.txt (Twofish, "openwall")
secret.txt.pgp.txt (AES-256, "openwall")

[kholia]

We don't support Old: Symmetrically Encrypted Data Packet currently. Such packets don't seem to be common though.

Sample file -> secret-idea.txt.pgp.txt (IDEA, "openwall")

[kholia]

Quick note,

For very large GPG files (e.g. 100 MB, > 1 GB) having no MDC, we can also generate truncated hashes without impacting the current functionality.

[kholia]

Quick note,

The decryption stuff seems to be done twice (?) for symmetric mode hashes. Confirm, and try to fix if required.