Update curl from 7.69.1 to 7.72.0

[patrikjuvonen]

Summary

• Contains stability and security related fixes
• 7.72.0: curl-users mailing list archive, maintainer's blog post, changelog
• 7.71.1: curl-users mailing list archive, maintainer's blog post, changelog
• 7.71.0: curl-users mailing list archive, maintainer's blog post, changelog
• 7.70.0: curl-users mailing list archive, maintainer's blog post, changelog

Tests

• fetchRemote
  • Download test script (v1.0.1): test_fetchRemote.zip
    • Preferable not to run client and server simultaneously
  • URL for unauthenticated requests: http://ptsv2.com/t/mtasa-test-fetchRemote
  • URL for authenticated requests: http://ptsv2.com/t/mtasa-test-fetchRemote-auth
• callRemote
  • Download test script (v1.0.2): test_callRemote.zip
  • URL for unauthenticated requests: http://ptsv2.com/t/mtasa-test-callRemote
  • URL for authenticated requests: http://ptsv2.com/t/mtasa-test-callRemote-auth
• Please flush all your requests from ptsv2.com after testing!

Validation

To help validate the integrity of the update I have created the following bash script that diffs between my PR branch and the official package provided from the curl website.

#!/bin/bash

CURL_UPDATE_VERSION=7.72.0
CURL_PATH_NAME=curl-$CURL_UPDATE_VERSION

GIT_REPO_BRANCH=vendor/curl-7.71.1
GIT_REPO_URL=https://github.com/patrikjuvonen/mtasa-blue.git
GIT_REPO_CURL_PATH=vendor/curl/

echo 1. Download and extract $CURL_PATH_NAME...
curl https://curl.haxx.se/download/$CURL_PATH_NAME.tar.xz | tar -xJ

echo 2. Fetch and checkout the vendor update branch $GIT_REPO_BRANCH from $GIT_REPO_URL...
git fetch $GIT_REPO_URL $GIT_REPO_BRANCH:$GIT_REPO_BRANCH
git checkout $GIT_REPO_BRANCH

echo 3. Start checking integrity...
diff -r --strip-trailing-cr $GIT_REPO_CURL_PATH $CURL_PATH_NAME

echo 4. Completed.
exec $SHELL

• Download validation script (fetch and checkout on existing clone) (.txt)
• Download validation script (fresh clone) (.txt)

Past curl updates in MTA

Date	From	To	Link
March 2020	7.68.0	7.69.1 (current)	#1302
January 2020	7.67.0	7.68.0	#1216
November 2019	7.66.0	7.67.0	#1161
September 2019	7.65.3	7.66.0	#1099
July 2019	7.65.1	7.65.3	#1027
July 2019	7.64.1	7.65.1	#1018
April 2019	7.64.0	7.64.1	#898
February 2019	7.63.0	7.64.0	#819
January 2019	7.61.1	7.63.0	#744
September 2018	7.61.0	7.61.1	#428
August 2018	7.59.0	7.61.0	#271
March 2018	7.54.0	7.59.0	b99e343
June 2017	7.32.0	7.54.0	c15d999
August 2013	7.19.4	7.32.0	aaf3e21

Copy of curl changelogs

Fixed in 7.72.0 - August 19 2020

Changes:

content_encoding: add zstd decoding support
CURL_PUSH_ERROROUT: allow the push callback to fail the parent stream
CURLINFO_EFFECTIVE_METHOD: added

Bugfixes:

CVE-2020-8231: libcurl: wrong connect-only connection
appveyor: collect libcurl.dll variants with prefix or suffix
asyn-ares: correct some bad comments
bearssl: fix build with disabled proxy support
buildconf: avoid array concatenation in die()
buildconf: retire ares buildconf invocation
checksrc: ban gmtime/localtime
checksrc: invoke script with -D to find .checksrc proper
CI/azure: install libssh2 for use with msys2-based builds
CI/azure: unconditionally enable warnings-as-errors with autotools
CI/macos: enable warnings as errors for CMake builds
CI/macos: set minimum macOS version
CI/macos: unconditionally enable warnings-as-errors with autotools
CI: Add muse CI analyzer
cirrus-ci: upgrade 11-STABLE to 11.4
CMake: don't complain about missing nroff
CMake: fix test for warning suppressions
cmake: fix windows xp build
configure.ac: Sort features name in summary
configure: allow disabling warnings
configure: cleanup wolfssl + pkg-config conflicts when cross compiling.
configure: show zstd "no" in summary when built without it
connect: remove redundant message about connect failure
curl-config: ignore REQUIRE_LIB_DEPS in --libs output
curl.1: add a few missing valid exit codes
curl: add %{method} to the -w variables
curl: improve the existing file check with -J
curl_multi_setopt: fix compiler warning "result is always false"
curl_version_info.3: CURL_VERSION_KERBEROS4 is deprecated
CURLINFO_CERTINFO.3: fix typo
CURLOPT_NOBODY.3: clarify what setting to 0 means
docs: add date of 7.20 to CURLM_CALL_MULTI_PERFORM mentions
docs: Add video link to docs/CONTRIBUTE.md
docs: change "web site" to "website"
docs: clarify MAX_SEND/RECV_SPEED functionality
docs: Update a few leftover mentions of DarwinSSL
doh: remove redundant cast
file2memory: use a define instead of -1 unsigned value
ftp: don't do ssl_shutdown instead of ssl_close
ftpserver: don't verify SMTP MAIL FROM names
getinfo: reset retry-after value in initinfo
gnutls: repair the build with `CURL_DISABLE_PROXY`
gtls: survive not being able to get name/issuer
h2: repair trailer handling
http2: close the http2 connection when no more requests may be sent
http2: fix nghttp2_strerror -> nghttp2_http2_strerror in debug messages
libssh2: s/ssherr/sftperr/
libtest/Makefile.am: add -no-undefined for libstubgss for Cygwin
md(4|5): don't use deprecated macOS functions
mprintf: Fix dollar string handling
mprintf: Fix stack overflows
multi: Condition 'extrawait' is always true
multi: Remove 10-year old out-commented code
multi: remove two checks always true
multi: update comment to say easyp list is linear
multi_remove_handle: close unused connect-only connections
ngtcp2: adapt to error code rename
ngtcp2: adjust to recent sockaddr updates
ngtcp2: update to modified qlog callback prototype
nss: fix build with disabled proxy support
ntlm: free target_info before (re-)malloc
openssl: fix build with LibreSSL < 2.9.1
page-header: provide protocol details in the curl.1 man page
quiche: handle calling disconnect twice
runtests.pl: treat LibreSSL and BoringSSL as OpenSSL
runtests: move the gnutls-serv tests to a dynamic port
runtests: move the smbserver to use a dynamic port number
runtests: move the TELNET server to a dynamic port
runtests: run the DICT server on a random port number
runtests: run the http2 tests on a random port number
runtests: support dynamicly base64 encoded sections in tests
setopt: unset NOBODY switches to GET if still HEAD
smtp_parse_address: handle blank input string properly
socks: use size_t for size variable
strdup: remove the odd strlen check
test1119: verify stdout in the test
test1139: make it display the difference on test failures
test1140: compare stdout
test1908: treat file as text
tests/FILEFORMAT.md: mention %HTTP2PORT
tests/sshserver.pl: fix compatibility with OpenSSH for Windows
TLS naming: fix more Winssl and Darwinssl leftovers
tls-max.d: this option is only for TLS-using connections
tlsv1.3.d. only for TLS-using connections
tool_doswin: Simplify Windows version detection
tool_getparam: make --krb option work again
TrackMemory tests: ignore realloc and free in getenv.c
transfer: fix data_pending for builds with both h2 and h3 enabled
transfer: fix memory-leak with CURLOPT_CURLU in a duped handle
transfer: move retrycount from connect struct to easy handle
travis/script.sh: fix use of `-n' with unquoted envvar
travis: add ppc64le and s390x builds
travis: update quiche builds for new boringssl layout
url: fix CURLU and location following
url: silence MSVC warning
util: silence conversion warnings
win32: Add Curl_verify_windows_version() to curlx
WIN32: stop forcing narrow-character API
windows: add unicode to feature list
windows: disable Unix Sockets for old mingw

Fixed in 7.71.1 - July 1 2020

Bugfixes:

cirrus-ci: disable FreeBSD 13 (again)
Curl_inet_ntop: always check the return code
CURLOPT_READFUNCTION.3: provide the upload data size up front
DYNBUF.md: fix a typo: trail => tail
escape: make the URL decode able to reject only %00-bytes
escape: zero length input should return a zero length output
examples/multithread.c: call curl_global_cleanup()
http2: set the correct URL in pushed transfers
http: fix proxy auth with blank password
mbedtls: fix build with disabled proxy support
ngtcp2: sync with current master
openssl: Fix compilation on Windows when ngtcp2 is enabled
Revert "multi: implement wait using winsock events"
sendf: improve the message on client write errors
terminology: call them null-terminated strings
tool_cb_hdr: Fix etag warning output and return code
url: allow user + password to contain "control codes" for HTTP(S)
vtls: compare cert blob when finding a connection to reuse

Fixed in 7.71.0 - June 24 2020

Changes:

CURLOPT_SSL_OPTIONS: optional use of Windows' CA store (with openssl)
setopt: add CURLOPT_PROXY_ISSUERCERT(_BLOB) for coherency
setopt: support certificate options in memory with struct curl_blob
tool: Add option --retry-all-errors to retry on any error

Bugfixes:

*_sspi: fix bad uses of CURLE_NOT_BUILT_IN
all: fix codespell errors
altsvc: bump to h3-29
altsvc: fix 'dsthost' may be used uninitialized in this function
altsvc: fix parser for lines ending with CRLF
altsvc: remove the num field from the altsvc struct
appveyor: add non-debug plain autotools-based build
appveyor: disable flaky test 1501 and ignore broken 1056
appveyor: disable test 1139 instead of ignoring it
asyn-*: remove support for never-used NULL entry pointers
azure: use matrix strategy to avoid configuration redundancy
build: disable more code/data when built without proxy support
buildconf: remove -print from the find command that removes files
checksrc: enhance the ASTERISKSPACE and update code accordingly
CI/macos: fix 'is already installed' errors by using bundle
cirrus: disable SFTP and SCP tests
CMake: add ENABLE_ALT_SVC option
CMake: add HTTP/3 support (ngtcp2+nghttp3, quiche)
CMake: add libssh build support
CMake: do not build test programs by default
CMake: fix runtests.pl with CMake, add new test targets
CMake: ignore INTERFACE_LIBRARY targets for pkg-config file
CMake: rebuild Makefile.inc.cmake when Makefile.inc changes
CODE_REVIEW.md: how to do code reviews in curl
configure: fix pthread check with static boringssl
configure: for wolfSSL, check for the DES func needed for NTLM
configure: only strip first -L from LDFLAGS
configure: repair the check if argv can be written to
configure: the wolfssh backend does not provide SCP
connect: improve happy eyeballs handling
connect: make happy eyeballs work for QUIC (again)
curl.1: Quote globbed URLs
curl: remove -J "informational" written on stdout
Curl_addrinfo: use one malloc instead of three
CURLINFO_ACTIVESOCKET.3: clarify the description
doc: add missing closing parenthesis in CURLINFO_SSL_VERIFYRESULT.3
doc: Rename VERSIONS to VERSIONS.md as it already has Markdown syntax
docs/HTTP3: add qlog to the quiche build instruction
docs/options-in-versions: which version added each cmdline option
docs: unify protocol lists
dynbuf: introduce internal generic dynamic buffer functions
easy: fix dangling pointer on easy_perform fail
examples/ephiperfifo: turn off interval when setting timerfd
examples/http2-down/upload: add error checks
examples: remove asiohiper.cpp
FILEFORMAT: add more features that tests can depend on
FILEFORMAT: describe verify/stderr
ftp: make domore_getsock() return the secondary socket properly
ftp: mark return-ignoring calls to Curl_GetFTPResponse with (void)
ftp: shut down the secondary connection properly when SSL is used
GnuTLS: Backend support for CURLINFO_SSL_VERIFYRESULT
hostip: make Curl_printable_address not return anything
hostip: on macOS avoid DoH when given a numerical IP address
http2: keep trying to send pending frames after req.upload_done
http2: simplify and clean up trailer handling
HTTP3.md: clarify cargo build directory
http: move header storage to Curl_easy from connectdata
libcurl.pc: Merge Libs.private into Libs for static-only builds
libssh2: improved error output for wrong quote syntax
libssh2: keep sftp errors as 'unsigned long'
libssh2: set the expected total size in SCP upload init
libtest/cmake: Remove commented code
list-only.d: this option existed already in 4.0
manpage: add three missing environment variables
multi: add defensive check on data->multi->num_alive
multi: implement wait using winsock events
ngtcp2: cleanup memory when failing to connect
ngtcp2: fix build with current ngtcp2 master implementing draft 28
ngtcp2: fix happy eyeballs quic connect crash
ngtcp2: introduce qlog support
ngtcp2: never call fprintf() in lib code in release version
ngtcp2: update with recent API changes
ntlm: enable NTLM support with wolfSSL
OpenSSL: have CURLOPT_CRLFILE imply CURLSSLOPT_NO_PARTIALCHAIN
openssl: set FLAG_TRUSTED_FIRST unconditionally
projects: Add crypt32.lib to dependencies for all OpenSSL configs
quiche: clean up memory properly when failing to connect
quiche: enable qlog output
quiche: update SSLKEYLOGFILE support
Revert "buildconf: use find -execdir"
Revert "ssh: ignore timeouts during disconnect"
runtests: remove sleep calls
runtests: show elapsed test time with higher precision (ms)
select: always use Sleep in Curl_wait_ms on Win32
select: fix overflow protection in Curl_socket_check
sendf: make failf() use the mvsnprintf() return code
server/sws: fix asan warning on use of uninitialized variable
server/util: fix logmsg format using curl_off_t argument
sha256: fixed potentially uninitialized variable
share: don not set the share flag it something fails
sockfilt: make select_ws stop waiting on exit signal event
socks: detect connection close during handshake
socks: fix expected length of SOCKS5 reply
socks: remove unreachable breaks in socks.c and mime.c
source cleanup: remove all custom typedef structs
test1167: fixes in badsymbols.pl
test1177: look for curl.h in source directory
test1238: avoid tftpd being busy for tests shortly following
test613.pl: make tests 613 and 614 work with OpenSSH for Windows
test75: Remove precheck test
tests: add https-proxy support to the test suite
tests: add support for SSH server variant specific transfer paths
tests: add two simple tests for --login-options
tests: make test 1248 + 1249 use %NOLISTENPORT
tests: pick a random port number for SSH
tests: run stunnel for HTTPS and FTPS on dynamic ports
timeouts: change millisecond timeouts to timediff_t from time_t
timeouts: move ms timeouts to timediff_t from int and long
tool: fixup a few --help descriptions
tool: support UTF-16 command line on Windows
tool_cfgable: free login_options at exit
tool_getparam: -i is not OK if -J is used
tool_getparam: fix memory leak in parse_args
tool_operate: fixed potentially uninitialized variables
tool_paramhlp: fixed potentially uninitialized strtol() variable
transfer: close connection after excess data has been read
travis: add "qlog" as feature in the quiche build
travis: Add ngtcp2 and quiche tests for CMake
travis: upgrade to bionic, clang-9, improve readability
typecheck-gcc.h: CURLINFO_PRIVATE does not need a 'char *'
unit1604.c: fix implicit conv from 'SANITIZEcode' to 'CURLcode'
url: accept "any length" credentials for proxy auth
url: alloc the download buffer at transfer start
url: make the updated credentials URL-encoded in the URL
url: reject too long input when parsing credentials
url: sort the protocol schemes in rough popularity order
urlapi: accept :: as a valid IPv6 address
urldata: leave the HTTP method untouched in the set.* struct
urlglob: treat literal IPv6 addresses with zone IDs as a host name
user-agent.d: spell out what happens given a blank argument
vauth/cleartext: fix theoretical integer overflow
version.d: expanded and alpha-sorted
vtls: Extract and simplify key log file handling from OpenSSL
wolfssl: add SSLKEYLOGFILE support
wording: avoid blacklist/whitelist stereotypes
write-out.d: added "response_code"

Fixed in 7.70.0 - April 29 2020

Changes:

curl: add --ssl-revoke-best-effort to allow a "best effort" revocation check
mqtt: add new experimental protocol
schannel: add "best effort" revocation check option: CURLSSLOPT_REVOKE_BEST_EFFORT
writeout: support to generate JSON output with '%{json}'

Bugfixes:

appveyor: add Unicode winbuild jobs
appveyor: completely disable tests that fail to timeout early
appveyor: show failed tests in log even if test is ignored
appveyor: sort builds by type and add two new variants
appveyor: turn disabled tests into ignored result tests
appveyor: use random test server ports based upon APPVEYOR_API_URL
build: fixed build for systems with select() in unistd.h
buildconf: avoid using tempfile when removing files
checksrc: warn on obvious conditional blocks on the same line as if()
CI-fuzz: increase fuzz time to 40 minutes
ci/tests: fix Azure Pipelines not running Windows containers
CI: add build with ngtcp2 + gnutls on Travis CI
CI: bring GitHub Actions fuzzing job in line with macOS jobs
CI: migrate macOS jobs from Azure and Travis CI to GitHub Actions
CI: remove default Ubuntu build from GitHub Actions
cirrus: no longer ignore test 504 which is working again
cirrus: re-enable the FreeBSD 13 CI builds
cleanup: insert newline after if() conditions
cmake: add aliases so exported target names are available in tree
cmake: add CMAKE_MSVC_RUNTIME_LIBRARY
cmake: add support for building with wolfSSL
cmake: Avoid MSVC C4273 warnings in send/recv checks
cmdline: fix handling of OperationConfig linked list (--next)
compressed.d: stress that the headers are not modified
config: remove all defines of HAVE_DES_H
configure: convert -I to -isystem as a last step
configure: document 'compiler_num' for gcc
configure: don't check for Security.framework when cross-compiling
configure: fix -pedantic-errors for GCC 5 and later
configure: remove use of -vec-report0 from CFLAGS with icc
connect: happy eyeballs cleanup
connect: store connection info for QUIC connections
copyright: fix out-of-date copyright ranges and missing headers
curl-functions.m4: remove inappropriate AC_REQUIRE
curl.h: remnove CURL_VERSION_ESNI. Never supported nor documented
curl.h: update comment typo
curl: allow both --etag-compare and --etag-save with same file name
curl_setup: define _WIN32_WINNT_[OS] symbols
CURLINFO_CONDITION_UNMET: return true for 304 http status code
CURLINFO_NUM_CONNECTS: improve accuracy
CURLOPT_WRITEFUNCTION.3: add inline example and new see-also
dist: add mail-rcpt-allowfails.d to the tarball
docs/make: generate curl.1 from listed files only
docs: add warnings about FILE: URLs on Windows
easy: fix curl_easy_duphandle for builds missing IPv6 that use c-ares
examples/sessioninfo.c: add include to fix compiler warning
github actions: run when pushed to master or */ci + PRs
gnutls: bump lowest supported version to 3.1.10
gnutls: Don't skip really long certificate fields
gnutls: ensure TLS 1.3 when SRP isn't requested
gopher: check remaining time left during write busy loop
gskit: use our internal select wrapper for portability
http2: Fix erroneous debug message that h2 connection closed
http: don't consider upload done if the request isn't completely sent off
http: free memory when Alt-Used header creation fails due to OOM
lib/mk-ca-bundle: skip empty certs
lib670: use the same Win32 API check as all other lib tests
lib: fix typos in comments and errormessages
lib: never define CURL_CA_BUNDLE with a getenv
libcurl-multi.3: added missing full stop
libssh: avoid options override by configuration files
libssh: Use new ECDSA key types to check known hosts
mailmap: fixup a few author names/fields
Makefile.m32: Improve windres parameter compatibility
Makefile: run the cd commands in a subshell
memdebug: don't log free(NULL)
mime: properly check Content-Type even if it has parameters
multi-ssl: reset the SSL backend on `Curl_global_cleanup()`
multi: improve parameter check for curl_multi_remove_handle
nghttp2: 1.12.0 required
ngtcp2: update to git master for the key installation API change
nss: check for PK11_CreateDigestContext() returning NULL
openssl: adapt to functions marked as deprecated since version 3
OS400: update strings for ccsid-ifier (fixes the build)
output.d: quote the URL when globbing
packages: add OS400/chkstrings.c to the dist
RELEASE-PROCEDURE.md: run the copyright.pl script!
Revert "file: on Windows, refuse paths that start with \\"
runtests: always put test number in servercmd file
runtests: provide nicer errormsg when protocol "dump" file is empty
schannel: Fix blocking timeout logic
schannel: support .P12 or .PFX client certificates
scripts/release-notes.pl: add helper script for RELEASE-NOTES maintenance
select: make Curl_socket_check take timediff_t timeout
select: move duplicate select preparation code into Curl_select
select: remove typecast from SOCKET_WRITABLE/READABLE macros
server/getpart: make the "XML-parser" stricter
server/resolve: remove AI_CANONNAME to make macos tell the truth
smtp: set auth correctly
sockfilt: add logmsg output to select_ws_wait_thread on Windows
sockfilt: fix broken pipe on Windows to be ready in select_ws
sockfilt: fix handling of ready closed sockets on Windows
sockfilt: fix race-condition of waiting threads and event handling
socks: Fix blocking timeout logic
src: Remove C99 constructs to ensure C89 compliance
SSLCERTS.md: Fix example code for setting CA cert file
test1148: tolerate progress updates better (again)
test1154: set a proper name
test1177: verify that all the CURL_VERSION_ bits are documented
test1566: verify --etag-compare that gets a 304 back
test1908: avoid using fixed port number in test data
test2043: use revoked.badssl.com instead of revoked.grc.com
test2100: fix static port instead of dynamic value being used
tests/data: fix some XML formatting issues in test cases
tests/FILEFORMAT: converted to markdown and extended
tests/server/util.c: use curl_off_t instead of long for pid
tests: add %NOLISTENPORT and use it
tests: add Windows compatible pidwait like pidkill and pidterm
tests: fix conflict between Cygwin/msys and Windows PIDs
tests: introduce preprocessed test cases
tests: make Python-based servers compatible with Python 2 and 3
tests: make runtests check that disabled tests exists
tests: move pingpong server to dynamic listening port
tests: remove python_dependencies for smbserver from our tree
tests: run the RTSP test server on a dynamic port number
tests: run the SOCKS test server on a dynamic port number
tests: run the sws server on "any port"
tests: run the TFTP test server on a dynamic port number
tests: use Cygwin/msys PIDs for stunnel and sshd on Windows
tls: remove the BACKEND define kludge from most backends
tool: do not declare functions with Curl_ prefix
tool_operate: fix add_parallel_transfers when more are in queue
transfer: cap retries of "dead connections" to 5
transfer: Switch PUT to GET/HEAD on 303 redirect
travis: bump the wolfssl CI build to use 4.4.0
travis: update the ngtcp2 build to use the latest OpenSSL patch
url: allow non-HTTPS altsvc-matching for debug builds
version: add 'cainfo' and 'capath' to version info struct
version: increase buffer space for ssl version output
version: skip idn2_check_version() check and add precaution
vquic: add support for GnuTLS backend of ngtcp2
vtls: fix ssl_config memory-leak on out-of-memory
warnless: remove code block for icc that didn't work
windows: enable UnixSockets with all build toolchains
windows: suppress UI in all CryptAcquireContext() calls

[Dutchman101]

@patrikjuvonen maybe pull in the new cURL update (7.72.0, released last week) and see if the issue that blocks this is still present?

[patrikjuvonen]

Something must have changed on the API server because it doesn't work on current active MTA curl version either. I think we should ignore this for now, as data can still be sent. I'll try to figure out a working API test server.

[ghost]

We can merge this once nightly is stable.