ipfs-http-client TimeoutError: Request timed out

[b-rohit]

• Version:
  go-ipfs version: 0.8.0
  Repo version: 11
  System version: amd64/linux
  Golang version: go1.15.8

• Platform:
  Linux 4.15.0-140-generic peer-info@0.6.1 breaks build ⚠️ #144-Ubuntu SMP Fri Mar 19 14:12:35 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Severity:

High

Description:

I started a daemon using ipfs daemon command. I created an express server with HTTP api endpoints to download and upload to files to IPFS. I used ipfs-http-client:49.0.2 to interact with the local ipfs gateway (localhost:5001). I get TimeoutError after 5-6 successful GET requests. The logs from the express server are following.

2021-04-08 18:56:42 info: ipfs get file successfully
2021-04-08 18:56:45 info: ipfs get file successfully
2021-04-08 18:56:46 info: ipfs get file successfully
2021-04-08 18:56:47 info: ipfs get file successfully
2021-04-08 18:56:48 info: ipfs get file successfully
2021-04-08 18:56:50 info: ipfs get file successfully
2021-04-08 18:56:55 info: ipfs get file successfully
2021-04-08 18:57:08 info: ipfs get file successfully
2021-04-08 18:57:23 info: ipfs get file successfully
2021-04-08 18:57:35 info: ipfs get file successfully
2021-04-08 18:57:43 info: ipfs get file successfully
2021-04-08 18:58:03 error: TimeoutError: Request timed out
    at Timeout.<anonymous> (/home/dev/test-ipfs/node_modules/ipfs-utils/src/http.js:42:16)
    at listOnTimeout (internal/timers.js:549:17)
    at processTimers (internal/timers.js:492:7)
2021-04-08 18:58:18 error: TimeoutError: Request timed out
    at Timeout.<anonymous> (/home/dev/test-ipfs//node_modules/ipfs-utils/src/http.js:42:16)
    at listOnTimeout (internal/timers.js:549:17)
    at processTimers (internal/timers.js:492:7)
2021-04-08 18:58:44 error: TimeoutError: Request timed out
    at Timeout.<anonymous> (/home/dev/test-ipfs//node_modules/ipfs-utils/src/http.js:42:16)
    at listOnTimeout (internal/timers.js:549:17)
    at processTimers (internal/timers.js:492:7)

The error persist until I restart either the ipfs daemon or my express server.

more information

$ ipfs daemon
Initializing daemon...
go-ipfs version: 0.8.0
Repo version: 11
System version: amd64/linux
Golang version: go1.15.8
2021/04/08 18:56:20 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details.
Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/127.0.0.1/udp/4001/quic
Swarm listening on /ip4/172.17.0.1/tcp/4001
Swarm listening on /ip4/172.17.0.1/udp/4001/quic
Swarm listening on /ip4/172.18.0.1/tcp/4001
Swarm listening on /ip4/172.18.0.1/udp/4001/quic
Swarm listening on /ip4/172.19.0.1/tcp/4001
Swarm listening on /ip4/172.19.0.1/udp/4001/quic
Swarm listening on /ip4/172.20.0.1/tcp/4001
Swarm listening on /ip4/172.20.0.1/udp/4001/quic
Swarm listening on /ip4/172.22.0.1/tcp/4001
Swarm listening on /ip4/172.22.0.1/udp/4001/quic
Swarm listening on /ip4/192.168.0.108/tcp/4001
Swarm listening on /ip4/192.168.0.108/udp/4001/quic
Swarm listening on /ip4/192.168.123.1/tcp/4001
Swarm listening on /ip4/192.168.123.1/udp/4001/quic
Swarm listening on /ip4/192.168.240.1/tcp/4001
Swarm listening on /ip4/192.168.240.1/udp/4001/quic
Swarm listening on /ip4/192.168.39.1/tcp/4001
Swarm listening on /ip4/192.168.39.1/udp/4001/quic
Swarm listening on /ip6/2a02:8070:4a7:f740:19da:7680:8eae:21c4/tcp/4001
Swarm listening on /ip6/2a02:8070:4a7:f740:19da:7680:8eae:21c4/udp/4001/quic
Swarm listening on /ip6/2a02:8070:4a7:f740:5d39:2bbe:7e53:d7d/tcp/4001
Swarm listening on /ip6/2a02:8070:4a7:f740:5d39:2bbe:7e53:d7d/udp/4001/quic
Swarm listening on /ip6/2a02:8070:4a7:f740::46a2/tcp/4001
Swarm listening on /ip6/2a02:8070:4a7:f740::46a2/udp/4001/quic
Swarm listening on /ip6/::1/tcp/4001
Swarm listening on /ip6/::1/udp/4001/quic
Swarm listening on /p2p-circuit
Swarm announcing /ip4/127.0.0.1/tcp/4001
Swarm announcing /ip4/127.0.0.1/udp/4001/quic
Swarm announcing /ip4/192.168.0.108/tcp/4001
Swarm announcing /ip4/192.168.0.108/udp/4001/quic
Swarm announcing /ip4/88.152.184.124/udp/55801/quic
Swarm announcing /ip6/2a02:8070:4a7:f740::46a2/tcp/4001
Swarm announcing /ip6/2a02:8070:4a7:f740::46a2/udp/4001/quic
Swarm announcing /ip6/::1/tcp/4001
Swarm announcing /ip6/::1/udp/4001/quic
API server listening on /ip4/127.0.0.1/tcp/5001
WebUI: http://127.0.0.1:5001/webui
Gateway (readonly) server listening on /ip4/127.0.0.1/tcp/8080
Daemon is ready

$ ipfs config show
{
  "API": {
    "HTTPHeaders": {}
  },
  "Addresses": {
    "API": "/ip4/127.0.0.1/tcp/5001",
    "Announce": [],
    "Gateway": "/ip4/127.0.0.1/tcp/8080",
    "NoAnnounce": [],
    "Swarm": [
      "/ip4/0.0.0.0/tcp/4001",
      "/ip6/::/tcp/4001",
      "/ip4/0.0.0.0/udp/4001/quic",
      "/ip6/::/udp/4001/quic"
    ]
  },
  "AutoNAT": {},
  "Bootstrap": [
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt",
    "/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
    "/ip4/104.131.131.82/udp/4001/quic/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb"
  ],
  "Datastore": {
    "BloomFilterSize": 0,
    "GCPeriod": "1h",
    "HashOnRead": false,
    "Spec": {
      "mounts": [
        {
          "child": {
            "path": "blocks",
            "shardFunc": "/repo/flatfs/shard/v1/next-to-last/2",
            "sync": true,
            "type": "flatfs"
          },
          "mountpoint": "/blocks",
          "prefix": "flatfs.datastore",
          "type": "measure"
        },
        {
          "child": {
            "compression": "none",
            "path": "datastore",
            "type": "levelds"
          },
          "mountpoint": "/",
          "prefix": "leveldb.datastore",
          "type": "measure"
        }
      ],
      "type": "mount"
    },
    "StorageGCWatermark": 90,
    "StorageMax": "10GB"
  },
  "Discovery": {
    "MDNS": {
      "Enabled": true,
      "Interval": 10
    }
  },
  "Experimental": {
    "FilestoreEnabled": false,
    "GraphsyncEnabled": false,
    "Libp2pStreamMounting": false,
    "P2pHttpProxy": false,
    "ShardingEnabled": false,
    "StrategicProviding": false,
    "UrlstoreEnabled": false
  },
  "Gateway": {
    "APICommands": [],
    "HTTPHeaders": {
      "Access-Control-Allow-Headers": [
        "X-Requested-With",
        "Range",
        "User-Agent"
      ],
      "Access-Control-Allow-Methods": [
        "GET"
      ],
      "Access-Control-Allow-Origin": [
        "*"
      ]
    },
    "NoDNSLink": false,
    "NoFetch": false,
    "PathPrefixes": [],
    "PublicGateways": null,
    "RootRedirect": "",
    "Writable": false
  },
  "Identity": {
    "PeerID": "12D3KooWQGSivL454Bd8yyVwsM6bYSxHCKofncKsiHwfaxfbmVo3"
  },
  "Ipns": {
    "RecordLifetime": "",
    "RepublishPeriod": "",
    "ResolveCacheSize": 128
  },
  "Mounts": {
    "FuseAllowOther": false,
    "IPFS": "/ipfs",
    "IPNS": "/ipns"
  },
  "Peering": {
    "Peers": null
  },
  "Pinning": {
    "RemoteServices": {}
  },
  "Plugins": {
    "Plugins": null
  },
  "Provider": {
    "Strategy": ""
  },
  "Pubsub": {
    "DisableSigning": false,
    "Router": ""
  },
  "Reprovider": {
    "Interval": "12h",
    "Strategy": "all"
  },
  "Routing": {
    "Type": "dht"
  },
  "Swarm": {
    "AddrFilters": null,
    "ConnMgr": {
      "GracePeriod": "20s",
      "HighWater": 900,
      "LowWater": 600,
      "Type": "basic"
    },
    "DisableBandwidthMetrics": false,
    "DisableNatPortMap": false,
    "EnableAutoRelay": false,
    "EnableRelayHop": false,
    "Transports": {
      "Multiplexers": {},
      "Network": {},
      "Security": {}
    }
  }
}

Steps to reproduce the error:

Call ipfs.get function multiple times.

var IPFS = require('ipfs-http-client');
global.ipfs = IPFS({ host: 'localhost', port: 5001, protocol: 'http', timeout: 500 })
router.post('/file/get', function (req, res, next) {
  try {
    for await (const file of ipfs.get(req.body.cid)) {
      for await (const chunk of file.content) {
        logger.info('ipfs get file successfully')
        return res.send(chunk)
      }
    }
  }
  catch (err) {
    return next(err)
  }
}

[lidel]

@b-rohit what is the reason to limit request to timeout: 500 ? Seems you artificially caused those timeouts.
it feels low even for localhost if your machine is under additional load

[lidel]

Stale, assuming no longer an issue. Feel free to reopen new one, if latest versions still manifest this.

[tomjohnhall]

@b-rohit what was your solution for this? I'm seeing very similar behaviour, when my express server first starts files via ipfs can be retrieved almost instantly (they are pinned by the ipfs daemon I'm pointing to), then after some time the responses seem to become patchy before they eventually all fail.

Using "ipfs-http-client": "^56.0.3"
and the daemon is using ipfs/kubo 0.14.0