Option `--http-transport` is (sometimes?) ignored; troubleshooting a failing `curl`

[andreasabel]

I am trying to build https://github.com/phadej/trustee on macOS Mojave.

$ cabal build -v3
...
Selected http transport implementation: curl
Downloading https://oleg.fi/gentle-introduction-2020.11.tar.gz
...
/usr/bin/curl --config - 'https://oleg.fi/gentle-introduction-2020.11.tar.gz' --output /Users/abel/bin/src/trustee/dist-newstyle/src/gentle-in_-793f49b55a5330da.tar12693-0.gz --location --write-out '%{http_code}' --user-agent 'cabal-install/3.7 (osx; x86_64)' --silent --show-error --dump-header /Users/abel/bin/src/trustee/dist-newstyle/src/curl-headers12693-1.txt

/usr/bin/curl returned ExitFailure 60 with error message:
Warning: <stdin>:2: warning: '--user' requires parameter
curl: (60) SSL certificate problem: certificate has expired
More details here: https://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.
HTTPS-proxy has similar options --proxy-cacert and --proxy-insecure.

stdin input:
--digest
--user

How to work around this?
After an hour of stumbling through curl docs, cabal docs, and cabal issues, I saw a light at the horizon:

$ cabal --help
...
--http-transport=HttpTransport
                                Set a transport for http(s) requests. Accepts
                                'curl', 'wget', 'powershell', and
                                'plain-http'. (default: 'curl')

Jumping on it:

$ cabal --http-transport=wget build 
Downloading https://oleg.fi/gentle-introduction-2020.11.tar.gz
Error: cabal: '/usr/bin/curl' exited with an error:
...

OH NO, SAME ERROR!

In fact, cabal seem to be ignoring the option --http-transport altogether, as I can pass in phantasy methods:

$ cabal --http-transport=semmelknoedel build 
Downloading https://oleg.fi/gentle-introduction-2020.11.tar.gz
Error: cabal: '/usr/bin/curl' exited with an error:
...

Too bad, semmelknoedel seems to be just an alias for curl.

Finally, I found that adding http-transport: wget to the cabal.project file does have the desired effect. (Docs: http-transport https://cabal.readthedocs.io/en/3.4/cabal-project.html#cfg-field-http-transport.) However, this was hard to discover.

Questions:

1. Can I feed extra arguments to curl? (It so nicely suggests solutions to my problem, but how to get to them via cabal?)
2. Could maybe cabal point me to solutions if curl fails? (E.g., a troubleshooting page.)
3. Shouldn't --http-transport check that it gets a valid argument? (Same for field http-transport.)
4. Why is --http-transport ignored on the command line (maybe only if cabal.project exists), even though there is no value for http-transport given in the cabal.project file?

[andreasabel]

I am trying to build phadej/trustee on macOS Mojave.

Switching to curl-openssl (from homebrew) is also a solution to the OP, see phadej/trustee#12 (comment), as the certificate failure seems to be pertaining to the curl of macOS Mojave: curl 7.54.0 (x86_64-apple-darwin18.0) libcurl/7.54.0 LibreSSL/2.6.5 zlib/1.2.11 nghttp2/1.24.1.

[gbaz]

% cabal --http-transport=foo update
Downloading the latest package list from hackage.haskell.org
cabal: Unknown HTTP transport specified: foo. The supported transports are
curl, wget, powershell, plain-http

Looks like the build command just isn't picking up the transport option from the command line?

[andreasabel]

Looks like the build command just isn't picking up the transport option from the command line?

I wonder how to get a minimal reproducer. I suppose you need some URL packages in the cabal.project file, like in https://github.com/phadej/trustee/blob/3531b4409f97b6976cc2f147d86543e6f292c6ab/cabal.project.

[andreasabel]

Thanks for fixing! I can confirm that semmelknoedel does not pass as a HTTP transport method anymore:

$ cabal --http-transport=semmelknoedel build 
Error: cabal: Unknown HTTP transport specified: semmelknoedel. The supported
transports are curl, wget, powershell, plain-http

@gbaz: Wouldn't this fix warrant a regression test?
To put it simple: each fixed issue contributes (at least) one regression test.
(Otherwise, I do not see any escape from the wack-a-mole-game of bug fixing.)

The minimal negative test would be e.g. that my invocation above produces this error message.
The minimal positive test is rather difficult to construct, unless there is already a test infrastructure on accessing remote repos.

[Mikolaj]

@gbaz: do the test proposals above make sense?

[gbaz]

i hadn't thought of the negative test, which is very feasible. i had only thought of the positive test, which wasn't :-)