Skip to content

HTTP/2 POST loop fails with ENHANCE_YOUR_CALM after 40701 iterations #23116

Closed
@davedoesdev

Description

@davedoesdev
  • Version: v10.10.0
  • Platform: Linux david-Latitude-E6440 4.15.0-34-generic logo ideas #37-Ubuntu SMP Mon Aug 27 15:21:48 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
  • Subsystem: http2

Here's a simple server app which reads HTTP/2 POST request bodies to their end and then responds:

const http2 = require('http2');
const fs = require('fs');
const path = require('path');
const { Writable } = require('stream');

const server = http2.createSecureServer({
    key: fs.readFileSync(path.join(__dirname, 'server.key')),
    cert: fs.readFileSync(path.join(__dirname, 'server.crt'))
});

server.on('session', function (session) {
    session.on('stream', function (stream, headers) {
        stream.on('end', function () {
            this.respond({
                ':status': 200,
                'Access-Control-Allow-Origin': 'http://localhost:8000'
            }, {
                endStream: true
            });
        });
        stream.pipe(new Writable({
            write: (chunk, encoding, cb) => cb()
        }));
    });
});

server.listen(7000, function () {
    console.log('READY.');
});

For this test, the cert and key are self-issued and my browser trusts them.

Here's a Web page which makes requests in series to the server:

<html>
<head>
<script>
async function test() {
  while (true) {
    response = await fetch('https://localhost:7000', { method: 'POST' });
    await response.arrayBuffer();
  }
}
</script>
</head>
<body onload='test()'>
</body>
</html>

I expect this to continue indefinitely.

However, what happens is I get the following error after 40701 iterations:

memtest.html:6 POST https://localhost:7000/ net::ERR_SPDY_PROTOCOL_ERROR

I did some debugging using Wireshark and found the error was ENHANCE_YOUR_CALM.

After adding some tracing to node_http2.cc, I found the error being produced here: https://github.com/nodejs/node/blob/v10.10.0/src/node_http2.cc#L863

I've tracked this down to current_nghttp2_memory_ continually growing.

The cause for this is nghttp2 allocation of 232 bytes for each stream (https://github.com/nodejs/node/blob/v10.10.0/deps/nghttp2/lib/nghttp2_session.c#L1029) is never being released.

This is because nghttp2 keeps closed streams around (up to the concurrent connection limit).

If I add the following line to Http2Options::Http2Options in src/node_http2.cc:

nghttp2_option_set_no_closed_streams(options_, 1);

then the test works as expected and doesn't fail at 40701 iterations.

Metadata

Metadata

Assignees

No one assigned

    Labels

    http2Issues or PRs related to the http2 subsystem.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions