Ability to use curl as a backend for HTTP(s) monitor

[maple3142]

📑 I have found these related issues/pull requests

I can't find any related issue about adding curl as a monitor by searching curl monitor.

🏷️ Feature Request Type

Change to existing monitor

🔖 Feature description

Currently, uptime-kuma appear to use axios for checking HTTP(s) monitor. I wish it is possible to use curl (using child_process) as an alternative to that.

✔️ Solution

I think this can either be added as an option to the existing HTTP(s) monitors, or be its out new monitor type if it is hard to achieve feature parity with this.

❓ Alternatives

A more general "execute shell command and check its exit code" monitor would be good for this specific use case. (#1117)

📝 Additional Context

The reason I want this is because the native http in node.js keep giving me ETIMEOUT error when I want to reach certain websites (e.g. my blog), but curl works just fine.

curl:

> curl -v 'http://blog.maple3142.net'
*   Trying 172.67.144.222:80...
*   Trying [2606:4700:3031::ac43:90de]:80...
* Immediate connect fail for 2606:4700:3031::ac43:90de: Network is unreachable
* Connected to blog.maple3142.net (172.67.144.222) port 80 (#0)
> GET / HTTP/1.1
> Host: blog.maple3142.net
> User-Agent: curl/7.88.1
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Date: Tue, 27 Feb 2024 01:40:32 GMT
< Transfer-Encoding: chunked
< Connection: keep-alive
the rest is omitted...

with node.js http:

> node -e "http.request('http://blog.maple3142.net/',console.log)"
node:events:496
      throw er; // Unhandled 'error' event
      ^

AggregateError
    at internalConnectMultiple (node:net:1114:18)
    at internalConnectMultiple (node:net:1177:5)
    at Timeout.internalConnectMultipleTimeout (node:net:1687:3)
    at listOnTimeout (node:internal/timers:575:11)
    at process.processTimers (node:internal/timers:514:7)
Emitted 'error' event on ClientRequest instance at:
    at Socket.socketErrorListener (node:_http_client:495:9)
    at Socket.emit (node:events:518:28)
    at emitErrorNT (node:internal/streams/destroy:169:8)
    at emitErrorCloseNT (node:internal/streams/destroy:128:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  code: 'ETIMEDOUT',
  [errors]: [
    Error: connect ETIMEDOUT 104.21.39.112:80
        at createConnectionError (node:net:1634:14)
        at Timeout.internalConnectMultipleTimeout (node:net:1685:38)
        at listOnTimeout (node:internal/timers:575:11)
        at process.processTimers (node:internal/timers:514:7) {
      errno: -110,
      code: 'ETIMEDOUT',
      syscall: 'connect',
      address: '104.21.39.112',
      port: 80
    },
    Error: connect ENETUNREACH 2606:4700:3031::ac43:90de:80 - Local (:::0)
        at internalConnectMultiple (node:net:1176:40)
        at Timeout.internalConnectMultipleTimeout (node:net:1687:3)
        at listOnTimeout (node:internal/timers:575:11)
        at process.processTimers (node:internal/timers:514:7) {
      errno: -101,
      code: 'ENETUNREACH',
      syscall: 'connect',
      address: '2606:4700:3031::ac43:90de',
      port: 80
    },
    Error: connect ETIMEDOUT 172.67.144.222:80
        at createConnectionError (node:net:1634:14)
        at Timeout.internalConnectMultipleTimeout (node:net:1685:38)
        at listOnTimeout (node:internal/timers:575:11)
        at process.processTimers (node:internal/timers:514:7) {
      errno: -110,
      code: 'ETIMEDOUT',
      syscall: 'connect',
      address: '172.67.144.222',
      port: 80
    },
    Error: connect ENETUNREACH 2606:4700:3036::6815:2770:80 - Local (:::0)
        at internalConnectMultiple (node:net:1176:40)
        at Timeout.internalConnectMultipleTimeout (node:net:1687:3)
        at listOnTimeout (node:internal/timers:575:11)
        at process.processTimers (node:internal/timers:514:7) {
      errno: -101,
      code: 'ENETUNREACH',
      syscall: 'connect',
      address: '2606:4700:3036::6815:2770',
      port: 80
    }
  ]
}

Node.js v20.11.0

even a raw netcat works just fine:

> printf 'GET / HTTP/1.0\r\nHost: blog.maple3142.net\r\n\r\n' | nc 172.67.144.222 80 -v
Connection to 172.67.144.222 80 port [tcp/http] succeeded!
HTTP/1.1 301 Moved Permanently
Date: Tue, 27 Feb 2024 01:43:04 GMT
Connection: close
Cache-Control: max-age=3600
the rest is omitted...

I am not sure why this happens, but being able to choose a different backend (e.g. curl) for making HTTP(s) requests would be a good way to solve this.

[CommanderStorm]

Wireshark filter:
ipv6.dst == 2a06:98c1:3121::3 || ipv6.src == 2a06:98c1:3121::3 || ipv6.dst == 2a06:98c1:3120::3 || ipv6.src == 2a06:98c1:3120::3 || ip.dst == 188.114.96.3 || ip.src == 188.114.96.3 || ip.dst == 188.114.97.3 || ip.src == 188.114.97.3

pcap of the two requests:
weird_difference.zip

[CommanderStorm]

For the first one (curl), we have the regular SYN,SYN ACK, ACK => HTTP

For the second one (node), node starts sending Keep-Alive packets

What server are you using?

[CommanderStorm]

I think this is just a wrong usage of http.request, as the following works (generated via an LLM, don't judge):

const http = require('http');

const options = {
  hostname: 'blog.maple3142.net',
  port: 80,
  path: '/',
  method: 'GET'
};

const req = http.request(options, (res) => {
  console.log(`Status Code: ${res.statusCode}`);
  console.log('Headers:');
  console.log(res.headers);

  // Buffer the response data
  let data = '';
  res.on('data', (chunk) => {
    data += chunk;
  });

  // When the whole response has been received
  res.on('end', () => {
    console.log('Response Body: '+ data);
  });
});
req.on('error', (error) => {
  console.error(`Error: ${error.message}`);
});
req.end();

[maple3142]

What server are you using?

I was having that problem on a vps running Debian 12. The problem seems to be magically fixed after a reboot so I can't reproduce it now.

I think this is just a wrong usage of http.request

Hmm, the only thing wrong with node -e "http.request('http://blog.maple3142.net/',console.log)" is that it doesn't call end() on the request so the request won't be sent. But even with this, the expected error should be Error: socket hang up instead of an instant ETIMEDOUT when opening a tcp connection, so there must be something wrong with my server before the reboot.

[CommanderStorm]

I got the socket hang up exception when I tried it 3h ago..

I am going to close this issue as resolved (I don't see a reproducible difference between curl and node => sticking with node is fine for now)