Issue with Homebrew openssl on macOS

[jwatt]

I've been playing around with the encryption support and found that the 'openssl' encryption appears to be broken on macOS with Homebrew's openssl installed. More specifically I got:

$ yadm encrypt
Encrypting the following files:
test_secret

usage: enc -ciphername [-AadePp] [-base64] [-bufsize number] [-debug]
    [-in file] [-iv IV] [-K key] [-k password]
    [-kfile file] [-md digest] [-none] [-nopad] [-nosalt]
    [-out file] [-pass arg] [-S salt] [-salt]

 -A                 Process base64 data on one line (requires -a)
 -a                 Perform base64 encoding/decoding (alias -base64)
 -bufsize size      Specify the buffer size to use for I/O
 -d                 Decrypt the input data
 -debug             Print debugging information
 -e                 Encrypt the input data (default)
 -in file           Input file to read from (default stdin)
 -iv IV             IV to use, specified as a hexadecimal string
 -K key             Key to use, specified as a hexadecimal string
 -md digest         Digest to use to create a key from the passphrase
 -none              Use NULL cipher (no encryption or decryption)
 -nopad             Disable standard block padding
 -out file          Output file to write to (default stdout)
 -P                 Print out the salt, key and IV used, then exit
                      (no encryption or decryption is performed)
 -p                 Print out the salt, key and IV used
 -pass source       Password source
 -S salt            Salt to use, specified as a hexadecimal string
 -salt              Use a salt in the key derivation routines (default)
 -v                 Verbose

Valid ciphername values:

 -aes-128-cbc              -aes-128-cbc-hmac-sha1    -aes-128-cfb             
 -aes-128-cfb1             -aes-128-cfb8             -aes-128-ctr             
 -aes-128-ecb              -aes-128-gcm              -aes-128-ofb             
 -aes-128-xts              -aes-192-cbc              -aes-192-cfb             
 -aes-192-cfb1             -aes-192-cfb8             -aes-192-ctr             
 -aes-192-ecb              -aes-192-gcm              -aes-192-ofb             
 -aes-256-cbc              -aes-256-cbc-hmac-sha1    -aes-256-cfb             
 -aes-256-cfb1             -aes-256-cfb8             -aes-256-ctr             
 -aes-256-ecb              -aes-256-gcm              -aes-256-ofb             
 -aes-256-xts              -aes128                   -aes192                  
 -aes256                   -bf                       -bf-cbc                  
 -bf-cfb                   -bf-ecb                   -bf-ofb                  
 -blowfish                 -camellia-128-cbc         -camellia-128-cfb        
 -camellia-128-cfb1        -camellia-128-cfb8        -camellia-128-ecb        
 -camellia-128-ofb         -camellia-192-cbc         -camellia-192-cfb        
 -camellia-192-cfb1        -camellia-192-cfb8        -camellia-192-ecb        
 -camellia-192-ofb         -camellia-256-cbc         -camellia-256-cfb        
 -camellia-256-cfb1        -camellia-256-cfb8        -camellia-256-ecb        
 -camellia-256-ofb         -camellia128              -camellia192             
 -camellia256              -cast                     -cast-cbc                
 -cast5-cbc                -cast5-cfb                -cast5-ecb               
 -cast5-ofb                -chacha                   -des                     
 -des-cbc                  -des-cfb                  -des-cfb1                
 -des-cfb8                 -des-ecb                  -des-ede                 
 -des-ede-cbc              -des-ede-cfb              -des-ede-ofb             
 -des-ede3                 -des-ede3-cbc             -des-ede3-cfb            
 -des-ede3-cfb1            -des-ede3-cfb8            -des-ede3-ofb            
 -des-ofb                  -des3                     -desx                    
 -desx-cbc                 -gost89                   -gost89-cnt              
 -gost89-ecb               -id-aes128-GCM            -id-aes192-GCM           
 -id-aes256-GCM            -rc2                      -rc2-40-cbc              
 -rc2-64-cbc               -rc2-cbc                  -rc2-cfb                 
 -rc2-ecb                  -rc2-ofb                  -rc4                     
 -rc4-40                   -rc4-hmac-md5            
ERROR: Unable to write ~/.local/share/yadm/archive

Homebrew tells me that openssl@1.1 1.1.1i is installed which appears to be the latest stable release. Despite that, setting yadm.openssl-old to true appears to fix things and the encryption then works. Since openssl is not out of date, perhaps this is an issue with Homebrew compiling openssl without some needed features? In any case, it's probably worth noting in the docs that macOS users may need to set yadm.openssl-old.

[rasa]

@jwatt I had the same problem. macOS's version in /usr/bin lacks support for -pbkdf2 and -iter.

Homebrew's version does support these options, but it's not enabled by default (at least on my system).

To enable Homebrew's version to be used instead of macOS's /usr/bin version, type:

yadm config openssl-program /usr/local/opt/openssl/bin/openssl

or to enable it globally:

export PATH=/usr/local/opt/openssl/bin:$PATH

or better yet, add this line to your shell's startup script.

After I wrote this answer, I found a post that confirmed my findings here.

Note: if you instead set yadm.openssl-old to true, you will need to do that on all your systems. I.e., both where you yadm encrypt and where you yadm decrypt, as the decryption will fail if the setting is different between systems.

[jwatt]

Thanks, @rasa! I was sure I'd had Homebrew symlink its openssl into /usr/local/bin some time ago, but apparently that wasn't the case. So I was indeed mistakenly using the LibreSSL 2.8.3 shipped by Apple.

Digging through the release notes for LibreSSL it looks like -pbkdf2 was added in 2.9.1. That's also mentioned in mailing list. I don't see any mention of -iter either in the mailing list or in the github issues, though. So unfortunately I guess that doesn't provide enough information for yadm's docs to be updated to mention a specific version of LibreSSL that people can check for.

Maybe the docs could say something along the lines of "The version of LibreSSL that ships with macOS does not support -iter. In fact, as of this writing, neither does the latest version of LibreSSL (3.3.1). macOS users can obtain a compatible version of openssl using Homebrew or similar, and should use openssl version to check they're using that openssl and not LibreSSL."? ... or something like that.

[TheLocehiliosan]

I made a decision not to do automatic version tests to pick options because a single set of dotfiles often span multiple systems which may have different OpenSSL implementations.

The user needs to set the option that will work with all of their systems. yadm can't know that from running on just one of their systems.

I'm guessing some documentation specifically about Mac homebrew could be helpful.

[github-actions]

This issue has been labeled as stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days.