Description
openedon Sep 28, 2022
In lib/http/HttpClient_Curl.hpp, WaitOnSocket() uses select(), which is limited to monitoring file descriptors less than 1024. On a system that creates a large number of file handles (e.g. for client sockets), this limit will be exceeded and cause a crash in WaitOnSocket(). While I am using the library on FreeBSD, this problem would also apply to other platforms, namely Linux.
The Linux man page for select() describes this limitation and suggests using poll or epoll instead:
WARNING: select() can monitor only file descriptors numbers that
are less than FD_SETSIZE (1024)—an unreasonably low limit for
many modern applications—and this limitation will not change.
All modern applications should instead use poll(2) or epoll(7),
which do not suffer this limitation.
The crash is also reported for CentOS in the Open Telemetry C++ SDK which shares the same code-base for the curl http-client library with 1ds-cpp:
Issue: open-telemetry/opentelemetry-cpp#1220
PR for fix: open-telemetry/opentelemetry-cpp#1410
My colleague created the following simple patch to use poll() instead of select() which we apply in our FreeBSD port build to prevent the crash:
--- lib/http/HttpClient_Curl.hpp.orig 2022-09-28 09:41:36.000000000 -0400
+++ lib/http/HttpClient_Curl.hpp 2022-09-28 09:46:46.000000000 -0400
@@ -10,6 +10,7 @@
#include <cstdio>
#include <cstdlib>
#include <cstdint>
+#include <poll.h>
#include <string.h>
#include <regex>
@@ -440,27 +441,20 @@
*/
static int WaitOnSocket(curl_socket_t sockfd, int for_recv, long timeout_ms)
{
- struct timeval tv;
- fd_set infd, outfd, errfd;
int res;
+ struct pollfd pfd;
- tv.tv_sec = timeout_ms / 1000;
- tv.tv_usec = (timeout_ms % 1000) * 1000;
+ pfd.fd = sockfd;
+ pfd.revents = 0;
- FD_ZERO(&infd);
- FD_ZERO(&outfd);
- FD_ZERO(&errfd);
-
- FD_SET(sockfd, &errfd); /* always check for error */
if(for_recv) {
- FD_SET(sockfd, &infd);
+ pfd.events = POLLIN | POLLERR;
} else {
- FD_SET(sockfd, &outfd);
+ pfd.events = POLLOUT | POLLERR;
}
- /* select() returns the number of signalled sockets or -1 */
- res = select((int)sockfd + 1, &infd, &outfd, &errfd, &tv);
+ res = poll(&pfd, 1, timeout_ms);
return res;
}
