Skip to content

UND_ERR_HEADERS_TIMEOUT received even when setting a larger headersTimeout in dispatcher for fetch() #1864

Description

@xconverge

Bug Description

I am trying to make a fetch request that can take 5+ minutes.

I make a fetch() request with a custom Agent containing atleast these parameters (I have also overridden connect for overriding checkServerIdentity())

    headersTimeout: 3000000,
    bodyTimeout: 3000000,

fetch() call looks like this:

      fetch( url,  {
          dispatcher: this.apiParameters.httpAgent,
          method: "get",
        }

yet after about ~200 seconds I am getting a headers timeout

TypeError: fetch failed
    at fetch (/node_modules/undici/index.js:105:13)
    at location I called unidici fetch) {
  cause: HeadersTimeoutError: Headers Timeout Error
      at Timeout.onParserTimeout [as _onTimeout] (/node_modules/undici/lib/client.js:902:28)
      at listOnTimeout (node:internal/timers:566:11)
      at process.processTimers (node:internal/timers:507:7) {
    code: 'UND_ERR_HEADERS_TIMEOUT'
  }
}

Reproducible By

Performing a LONG fetch operation

Expected Behavior

I would expect some combination of me setting headerTimeout to either 0 or a very high number to not trigger this timeout.

Logs & Screenshots

Environment

Ubuntu 20 in WSL, Node 18

Additional context

I have seen some stuff related to "blocking" https://undici.nodejs.org/#/docs/api/Dispatcher but am unable to figure out where to set it and am unsure if its even related

Activity

  1. mcollina commented on Jan 19, 2023

    @mcollina
    SponsorMember

    How is the agent instantiated?

  2. xconverge commented on Jan 19, 2023

    @xconverge
    ContributorAuthor

    How is the agent instantiated?

    I just got this sample working to reproduce the issue. Changing the delay at the top to less than 4 is fine, and greater than 5 minutes and it fails

    I was able to do this same test with a Client and it seems to work fine, so something specific to fetch() I think

    "use strict";
    
    const { test } = require("tap");
    
    const { Agent } = require("../../");
    const { createServer } = require("http");
    const FakeTimers = require("@sinonjs/fake-timers");
    
    // When this is set to less than 4 it works, when set to 5 or greater it fails and returns a header error
    const minutes = 6;
    
    const msToDelay = 1000 * 60 * minutes;
    
    test("Realllly long for a single fetch()", (t) => {
      t.plan(1);
    
      const clock = FakeTimers.install();
      t.teardown(clock.uninstall.bind(clock));
    
      const server = createServer((req, res) => {
        setTimeout(() => {
          res.end("hello");
        }, msToDelay);
        clock.tick(msToDelay + 1);
      });
      t.teardown(server.close.bind(server));
    
      server.listen(0, () => {
        fetch(`http://localhost:${server.address().port}`, {
          path: "/",
          method: "GET",
          dispatcher: new Agent({
            headersTimeout: 0,
            connectTimeout: 0,
            bodyTimeout: 0,
          }),
        })
          .then((response) => response.text())
          .then((response) => {
            t.equal("hello", response);
            t.end();
          })
          .catch((err) => {
            console.error(err);
            t.error(err);
          });
    
        clock.tick(msToDelay - 1);
      });
    });
    
    
  3. xconverge commented on Jan 19, 2023

    @xconverge
    ContributorAuthor

    versus this (just substituted fetch() and agent with a client) which runs "fine"

    "use strict";
    
    const { test } = require("tap");
    
    const { Client } = require("../..");
    const { createServer } = require("http");
    const FakeTimers = require("@sinonjs/fake-timers");
    
    // When this is set to less than 4 it works, when set to 5 or greater it fails and returns a header error
    const minutes = 6;
    
    const msToDelay = 1000 * 60 * minutes;
    
    test("Realllly long for a single request", (t) => {
      t.plan(2);
    
      const clock = FakeTimers.install();
      t.teardown(clock.uninstall.bind(clock));
    
      const server = createServer((req, res) => {
        setTimeout(() => {
          res.end("hello");
        }, msToDelay);
        clock.tick(msToDelay + 1);
      });
      t.teardown(server.close.bind(server));
    
      server.listen(0, () => {
        const client = new Client(`http://localhost:${server.address().port}`, {
          headersTimeout: 0,
          connectTimeout: 0,
        });
        t.teardown(client.destroy.bind(client));
    
        client.request({ path: "/", method: "GET" }, (err, response) => {
          t.error(err);
          const bufs = [];
          response.body.on("data", (buf) => {
            bufs.push(buf);
          });
          response.body.on("end", () => {
            t.equal("hello", Buffer.concat(bufs).toString("utf8"));
          });
        });
    
        clock.tick(msToDelay - 1);
      });
    });
    
    
  4. xconverge commented on Jan 19, 2023

    @xconverge
    ContributorAuthor
  5. xconverge commented on Jan 19, 2023

    @xconverge
    ContributorAuthor

    PR opened for this.

  6. mcollina commented on Jan 20, 2023

    @mcollina
    SponsorMember

    Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions