Update curl from 7.72.0 to 7.74.0

[patrikjuvonen]

Summary

• HSTS support, contains stability and security related fixes
• 7.74.0: curl-users mailing list archive, maintainer's blog post, changelog
• 7.73.0: curl-users mailing list archive, maintainer's blog post, changelog

Tests

• fetchRemote
  • Download test script (v1.0.1): test_fetchRemote.zip
    • Preferable not to run client and server simultaneously
  • URL for unauthenticated requests: http://ptsv2.com/t/mtasa-test-fetchRemote
  • URL for authenticated requests: http://ptsv2.com/t/mtasa-test-fetchRemote-auth
• callRemote
  • Download test script (v1.0.2): test_callRemote.zip
  • URL for unauthenticated requests: http://ptsv2.com/t/mtasa-test-callRemote
  • URL for authenticated requests: http://ptsv2.com/t/mtasa-test-callRemote-auth
• Please flush all your requests from ptsv2.com after testing!

Validation

To help validate the integrity of the update I have created the following bash script that diffs between my PR branch and the official package provided from the curl website.

#!/bin/bash

CURL_UPDATE_VERSION=7.74.0
CURL_PATH_NAME=curl-$CURL_UPDATE_VERSION

GIT_REPO_BRANCH=vendor/curl-7.74.0
GIT_REPO_URL=https://github.com/patrikjuvonen/mtasa-blue.git
GIT_REPO_CURL_PATH=vendor/curl/

echo 1. Download and extract $CURL_PATH_NAME...
curl https://curl.haxx.se/download/$CURL_PATH_NAME.tar.xz | tar -xJ

echo 2. Fetch and checkout the vendor update branch $GIT_REPO_BRANCH from $GIT_REPO_URL...
git fetch $GIT_REPO_URL $GIT_REPO_BRANCH:$GIT_REPO_BRANCH
git checkout $GIT_REPO_BRANCH

echo 3. Start checking integrity...
diff -r --strip-trailing-cr $GIT_REPO_CURL_PATH $CURL_PATH_NAME

echo 4. Completed.
exec $SHELL

• Download validation script (fetch and checkout on existing clone) (.txt)
• Download validation script (fresh clone) (.txt)

Past curl updates in MTA

Date	From	To	Link
October 2020	7.69.1	7.72.0 (current)	#1562
March 2020	7.68.0	7.69.1	#1302
January 2020	7.67.0	7.68.0	#1216
November 2019	7.66.0	7.67.0	#1161
September 2019	7.65.3	7.66.0	#1099
July 2019	7.65.1	7.65.3	#1027
July 2019	7.64.1	7.65.1	#1018
April 2019	7.64.0	7.64.1	#898
February 2019	7.63.0	7.64.0	#819
January 2019	7.61.1	7.63.0	#744
September 2018	7.61.0	7.61.1	#428
August 2018	7.59.0	7.61.0	#271
March 2018	7.54.0	7.59.0	b99e343
June 2017	7.32.0	7.54.0	c15d999
August 2013	7.19.4	7.32.0	aaf3e21

Copy of curl changelogs

Fixed in 7.74.0 - December 9 2020

Changes:

hsts: add experimental support for Strict-Transport-Security

Bugfixes:

CVE-2020-8286: Inferior OCSP verification
CVE-2020-8285: FTP wildcard stack overflow
CVE-2020-8284: trusting FTP PASV responses
acinclude: detect manually set minimum macos/ipod version
alt-svc: enable (in the build) by default
alt-svc: minimize variable scope and avoid "DEAD_STORE"
asyn: use 'struct thread_data *' instead of 'void *'
checksrc: warn on empty line before open brace
CI/appveyor: disable test 571 in two cmake builds
CI/azure: improve on flakiness by avoiding libtool wrappers
CI/tests: enable test target on TravisCI for CMake builds
CI/travis: add brotli and zstd to the libssh2 build
cirrus: build with FreeBSD 12.2 in CirrusCI
cmake: call the feature unixsockets without dash
cmake: check for linux/tcp.h
cmake: correctly handle linker flags for static libs
cmake: don't pass -fvisibility=hidden to clang-cl on Windows
cmake: don't use reserved target name 'test'
cmake: make BUILD_TESTING dependent option
cmake: make CURL_ZLIB a tri-state variable
cmake: set the unicode feature in curl-config on Windows
cmake: store IDN2 information in curl_config.h
cmake: use libcurl.rc in all Windows builds
configure: pass -pthread to Libs.private for pkg-config
configure: use pkgconfig to find openSSL when cross-compiling
connect: repair build without ipv6 availability
curl.1: add an "OUTPUT" section at the top of the manpage
curl.se: new home
curl: add compatibility for Amiga and GCC 6.5
curl: only warn not fail, if not finding the home dir
curl_easy_escape: limit output string length to 3 * max input
Curl_pgrsStartNow: init speed limit time stamps at start
curl_setup: USE_RESOLVE_ON_IPS is for Apple native resolver use
curl_url_set.3: fix typo in the RETURN VALUE section
CURLOPT_DNS_USE_GLOBAL_CACHE.3: fix typo
CURLOPT_HSTS.3: document the file format
CURLOPT_NOBODY.3: fix typo
CURLOPT_TCP_NODELAY.3: fix comment in example code
CURLOPT_URL.3: clarify SCP/SFTP URLs are for uploads as well
docs: document the 8MB input string limit
docs: fix typos and markup in ETag manpage sections
docs: Fix various typos in documentation
examples/httpput: remove use of CURLOPT_PUT
FAQ: refreshed
file: avoid duplicated code sequence
ftp: retry getpeername for FTP with TCP_FASTOPEN
gnutls: fix memory leaks (certfields memory wasn't released)
header.d: mention the "Transfer-Encoding: chunked" handling
HISTORY: the new domain
http3: fix two build errors, silence warnings
http3: use the master branch of GnuTLS for testing
http: pass correct header size to debug callback for chunked post
http_proxy: use enum with state names for 'keepon'
httpput-postfields.c: new example doing PUT with POSTFIELDS
infof/failf calls: fix format specifiers
libssh2: fix build with disabled proxy support
libssh2: fix transport over HTTPS proxy
libssh2: require version 1.0 or later
Makefile.m32: add support for HTTP/3 via ngtcp2+nghttp3
Makefile.m32: add support for UNICODE builds
mqttd: fclose test file when done
NEW-PROTOCOL: document what needs to be done to add one
ngtcp2: adapt to recent nghttp3 updates
ngtcp2: advertise h3 ALPN unconditionally
ngtcp2: Fix build error due to symbol name change
ngtcp2: use the minimal version of QUIC supported by ngtcp2
ntlm: avoid malloc(0) on zero length user and domain
openssl: acknowledge SRP disabling in configure properly
openssl: free mem_buf in error path
openssl: guard against OOM on context creation
openssl: use OPENSSL_init_ssl() with >= 1.1.0
os400: Sync libcurl API options
packages/OS400: make the source code-style compliant
quiche: close the connection
quiche: remove 'static' from local buffer
range.d: clarify that curl will not parse multipart responses
range.d: fix typo
Revert "multi: implement wait using winsock events"
rtsp: error out on empty Session ID, unified the code
rtsp: fixed Session ID comparison to refuse prefix
rtsp: fixed the RTST Session ID mismatch in test 570
runtests: return error if no tests ran
runtests: revert the mistaken edit of $CURL
runtests: show keywords when no tests ran
scripts/completion.pl: parse all opts
socks: check for DNS entries with the right port number
src/tool_filetime: disable -Wformat on mingw for this file
strerror: use 'const' as the string should never be modified
test122[12]: remove these two tests
test506: make it not run in c-ares builds
tests/*server.py: close log file after each log line
tests/server/tftpd.c: close upload file right after transfer
tests/util.py: fix compatibility with Python 2
tests: add missing global_init/cleanup calls
tests: fix some http/2 tests for older versions of nghttpx
tool_debug_cb: do not assume zero-terminated data
tool_help: make "output" description less confusing
tool_operate: --retry for HTTP 408 responses too
tool_operate: bail out proper on errors during parallel transfers
tool_operate: fix compiler warning when --libcurl is disabled
tool_writeout: use off_t getinfo-types instead of doubles
travis: use ninja-build for CMake builds
travis: use valgrind when running tests for debug builds
urlapi: don't accept blank port number field without scheme
urlapi: URL encode a '+' in the query part
urldata: remove 'void *protop' and create the union 'p'
vquic/ngtcp2.h: define local_addr as sockaddr_storage

Fixed in 7.73.0 - October 14 2020

Changes:

curl: add --output-dir
curl: support XDG_CONFIG_HOME to find .curlrc
curl: update --help with categories
curl_easy_option_*: new API for meta-data about easy options
CURLE_PROXY: new error code
mqtt: enable by default
sftp: add new quote commands 'atime' and 'mtime'
ssh: add the option CURLKHSTAT_FINE_REPLACE
tls: add CURLOPT_SSL_EC_CURVES and --curves

Bugfixes:

altsvc: clone setting in curl_easy_duphandle
base64: also build for smtp, pop3 and imap
BUGS: convert document to markdown
build-wolfssl: fix build with Visual Studio 2019
buildconf: invoke 'autoreconf -fi' instead
checksrc: detect // comments on column 0
checksrc: verify do-while and spaces between the braces
checksrc: warn on space after exclamation mark
CI/azure: disable test 571 in the msys2 builds
CI/azure: MQTT is now enabled by default
CI/azure: no longer ignore results of test 1013
CI/tests: fix invocation of tests for CMake builds
CI/travis: add a CI job with openssl3 (from git master)
cleanups: avoid curl_ on local variables
CMake: add option to enable Unicode on Windows
cmake: make HTTP_ONLY also disable MQTT
CMake: remove explicit `CMAKE_ANSI_CFLAGS`
cmake: remove scary warning
cmdline-opts/gen.pl: generate nicer "See Also" in curl.1
configure: don't say HTTPS-proxy is enabled when disabled
configure: fix pkg-config detecting wolfssl
configure: let --enable-debug set -Wenum-conversion with gcc >= 10
conn: check for connection being dead before reuse
connect.c: remove superfluous 'else' in Curl_getconnectinfo
curl.1: add see also no-progress-meter on two spots
curl.1: fix typo invokved -> invoked
curl: in retry output don't call all problems "transient"
curl: make --libcurl show binary posts correctly
curl: make checkpasswd use dynbuf
curl: make file2memory use dynbuf
curl: make file2string use dynbuf
curl: make glob_match_url use dynbuf
curl: make sure setopt CURLOPT_IPRESOLVE passes on a long
curl: retry delays in parallel mode no longer sleeps blocking
curl: use curlx_dynbuf for realloc when loading config files
curl:parallel_transfers: make sure retry readds the transfer
curl_get_line: build only if cookies or alt-svc are enabled
curl_mime_headers.3: fix the example's use of curl_slist_append
Curl_pgrsTime - return new time to avoid timeout integer overflow
Curl_send: return error when pre_receive_plain can't malloc
dist: add missing CMake Find modules to the distribution
docs/LICENSE-MIXING: remove
docs/opts: fix typos in two manual pages
docs/RESOURCES: remove
docs/TheArtOfHttpScripting: convert to markdown
docs: add description about CI platforms to CONTRIBUTE.md
docs: correct non-existing macros in man pages
doh: add error message for DOH_DNS_NAME_TOO_LONG
dynbuf: make sure Curl_dyn_tail() zero terminates
easy_reset: clear retry counter
easygetopt: pass a valid enum to avoid compiler warning
etag: save and use the full received contents
ftp: a 550 response to SIZE returns CURLE_REMOTE_FILE_NOT_FOUND
ftp: avoid risk of reading uninitialized integers
ftp: get rid of the PPSENDF macro
ftp: make a 552 response return CURLE_REMOTE_DISK_FULL
ftp: separate FTPS from FTP over "HTTPS proxy"
git: ignore libtests in 3XXX area
github: use new issue template feature
HISTORY: mention alt-svc added in 2019
HTTP/3: update to OpenSSL_1_1_1g-quic-draft-29
http: consolidate nghttp2_session_mem_recv() call paths
http_proxy: do not count proxy headers in the header bytecount
http_proxy: do not crash with HTTPS_PROXY and NO_PROXY set
imap: make imap_send use dynbuf for the send buffer management
imap: set cselect_bits to CURL_CSELECT_IN initially
ldap: reduce the amount of #ifdefs needed
lib/Makefile.am: bump VERSIONINFO due to new functions
lib1560: verify "redirect" to double-slash leading URL
lib583: fix enum mixup
lib: fix -Wassign-enum warnings
lib: make Curl_gethostname accept a const pointer
libssh2: handle the SSH protocols done over HTTPS proxy
libssh2: pass on the error from ssh_force_knownhost_key_type
Makefile.m32: add ability to override zstd libs [ci skip]
man pages: switch to https://example.com URLs
MANUAL: update examples to resolve without redirects
mbedtls: add missing header when defining MBEDTLS_DEBUG
memdebug: remove 9 year old unused debug function
multi: expand pre-check for socket readiness
multi: handle connection state winsock events
multi: implement wait using winsock events
ngtcp2: adapt to new NGTCP2_PROTO_VER_MAX define
ngtcp2: adapt to the new pkt_info arguments
ntlm: fix condition for curl_ntlm_core usage
openssl: avoid error conditions when importing native CA
openssl: consider ALERT_CERTIFICATE_EXPIRED a failed verification
openssl: Fix wincrypt symbols conflict with BoringSSL
parsedate: tune the date to epoch conversion
pause: only trigger a reread if the unpause sticks
pingpong: use a dynbuf for the *_pp_sendf() function
READMEs: convert several to markdown
runtests: add %repeat[]% for test files
runtests: allow creating files without newlines
runtests: allow generating a binary sequence from hex
runtests: clear pid variables when failing to start a server
runtests: make cleardir() erase dot files too
runtests: provide curl's version string as %VERSION for tests
schannel: fix memory leak when using get_cert_location
schannel: return CURLE_PEER_FAILED_VERIFICATION for untrusted root
scripts: improve the "get latest curl release tag" logic
sectransp: make it build with --disable-proxy
select.h: make socket validation macros test for INVALID_SOCKET
select: align poll emulation to return all relevant events
select: fix poll-based check not detecting connect failure
select: reduce duplication of Curl_poll in Curl_socket_check
select: simplify return code handling for poll and select
setopt: if the buffer exists, refuse the new BUFFERSIZE
setopt: return CURLE_BAD_FUNCTION_ARGUMENT on bad argument
socketpair: allow CURL_DISABLE_SOCKETPAIR
sockfilt: handle FD_CLOSE winsock event on write socket
src: spell whitespace without whitespace
SSLCERTS: fix English syntax
strerror: honor Unicode API choice on Windows
symbian: drop support
telnet.c: depend on static requirement of WinSock version 2
test1541: remove since it is a known bug
test163[12]: require http to be built-in to run
test434: test -K use in a single line without newline
test971: show test mismatches "inline"
tests/data: Fix some mismatched XML tags in test cases
tests/FILEFORMAT: document nonewline support for <file>
tests/FILEFORMAT: document type=shell for <command>
tests/server/util.c: fix support for Windows Unicode builds
tests: remove pipelining tests
tls: fix SRP detection by using the proper #ifdefs
tls: provide the CApath verbose log on its own line
tool_setopt: escape binary data to hex, not octal
tool_writeout: add new writeout variable, %{num_headers}
travis: add a build using libressl (from git master)
url: use blank credentials when using proxy w/o username and password
urlapi: use more Curl_safefree
vtls: deduplicate client certificates in ssl_config_data
win32: drop support for WinSock version 1, require version 2
winbuild: convert the instruction text to README.md