Global timeout feature (same as "--max-time" curl option)

[marcovc]

Hi,

I've finally had some time to investigate the issue reported here, and created a small example:

A client that wants to have a reply in at most 2100 ms:

#include <httplib.h>

int main()
{

  httplib::Client http_client("http://127.0.0.1:8082");
  auto timeout = std::chrono::milliseconds{2100};
  http_client.set_connection_timeout(timeout);
  http_client.set_read_timeout(timeout);
  http_client.set_write_timeout(timeout);
  auto start = std::chrono::steady_clock::now();
  http_client.Get("/");
  auto end = std::chrono::steady_clock::now();
  auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(end - start);
  std::cout << "Elapsed time: " << elapsed.count() << "ms\n";
}

A simple server to test (in python):

from http.server import BaseHTTPRequestHandler, HTTPServer
import time

class SimpleHTTPRequestHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        if self.path == "/":
            self.send_response(200)
            self.send_header("Content-type", "text/html")
            self.end_headers()
            for i in range(5):
                time.sleep(1)
                self.wfile.write(f"Response part {i+1}\n".encode('utf-8'))            
        else:
            self.send_response(404)
            self.end_headers()

def run(server_class=HTTPServer, handler_class=SimpleHTTPRequestHandler):
    server_address = ('', 8082)
    httpd = server_class(server_address, handler_class)
    print("Starting httpd server on port 8082...")
    httpd.serve_forever()

if __name__ == "__main__":
    run()

It will print something like

Elapsed time: 5001ms

while I was expecting something close to

Elapsed time: 2100ms

Perhaps there is another way to set a timeout for the request?

Thanks

[TalkySafe143]

Hi, the piece of code that set the socket option is this:

cpp-httplib/httplib.h

Lines 3744 to 3748 in 282f2fe

	timeval tv;
	tv.tv_sec = static_cast<long>(read_timeout_sec);
	tv.tv_usec = static_cast<decltype(tv.tv_usec)>(read_timeout_usec);
	setsockopt(sock2, SOL_SOCKET, SO_RCVTIMEO,
	reinterpret_cast<const void *>(&tv), sizeof(tv));

The socket use SO_RCVTIMEO and SO_SNDTIMEO for setting the timeout using timeval struct. But, the documentation of setsockopt using SO_RCVTIMEO says:

In the current implementation, this timer is restarted each time additional data are received by the protocol, and thus the limit is in effect an inactivity timer.

Your python server is writing a response to the socket each second five times, if you modify the server to sleep $5<$ seconds, you will have your timeout as you expect.

Maybe could be a good feature make a timeout different to the setsockopt implementation to expect that kind of results.

[TalkySafe143]

It could be good that @yhirose mention this on the README file 💯

[marcovc]

Hi Sebastian, thanks for looking into this. Yes, I figured that, this is why the test server is sending data by chunks. Libcurl has a "global" timeout, which in my opinion is a useful thing.

…

On Mon, Feb 3, 2025, 02:59 Sebastian Galindo ***@***.***> wrote: It could be good that @yhirose <https://github.com/yhirose> mention this on the README file 💯 — Reply to this email directly, view it on GitHub <#2034 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEYNNE77TTVS34AJI6B2HD2N3LQRAVCNFSM6AAAAABWAWARK2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMRZG44DMMZWG4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[yhirose]

@marcovc could you clarify the status of this issue? If it's not an issue on cpp-httplib, I would like to close it.

[marcovc]

Hi,

This is my second and last attempt to point out that there is no way to set a global timeout in httplib (client). If you feel that is not an important feature to have in your library, feel free to close it.

Thanks.

[yhirose]

@marcovc thanks! By the way, could you explain more about 'global' timeout that is implemented in libcurl?

[marcovc]

By "global timeout" I mean a way to request that the call to httplib client takes at most X seconds.

Example:

httplib::Client http_client("https://etc");
http_client.set_global_timeout(2s);
auto tic = std::chrono::steady_clock::now();
http_client.Get("/"); 
auto tac = std::chrono::steady_clock::now();
assert(tac - tic <= 2s);

There is no way to do the second line. All the existing timeouts (set_connection_timeout, set_read_timeout, set_write_timeout) are useful for their own purposes, but they can't be used to set a global timeout.

In my opinion, it is very useful to do so. The user might not care about how long does it take to establish the connection, or time elapsed between reads or writes, but instead just needs the whole thing to take less than X seconds.

libcurl has that: https://curl.se/libcurl/c/CURLOPT_TIMEOUT.html

[yhirose]

@marcovc could you confirm that CURLOPT_TIMEOUT really works as you explains? Also is there the corresponding curl option that I can easily test with? Thanks!

[marcovc]

Looks like it should correspond to the "--max-time" curl option.

[marcovc]

could you confirm that CURLOPT_TIMEOUT really works as you explains?

I'm literally pointing you to the manual page for that option. Not sure what you'd like me to confirm?

[yhirose]

I'm literally pointing you to the manual page for that option. Not sure what you'd like me to confirm?

@marcovc Don't worry. I'll try with --max-time. Thanks for the info!

[falbrechtskirchinger]

FYI, I've finished implementing the machinery for global timeouts for an unrelated PR. I'm now considering what changes to make to ClientImpl. It's quite a bit of code though (500 lines before adding global timeouts), and I'm not sure how @yhirose will feel about it. He hasn't responded yet to my wall-of-text on the other PR.
A proof-of-concept would omit asynchronous DNS, but a simple, ugly thread-based resolver is easy enough to add.