Skip to content

net: write syscall optimization and a bytesWritten bug fix #2960

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 34 additions & 16 deletions lib/net.js
Original file line number Diff line number Diff line change
Expand Up @@ -154,8 +154,9 @@ function Socket(options) {

initSocketHandle(this);

this._pendingData = null;
this._pendingEncoding = '';
// Buffer for the single possible write while connecting: when uncorked
// or ended, then written to. Writable will buffer any other writes.
this._pendingWrite = null;

// handle strings directly
this._writableState.decodeStrings = false;
Expand Down Expand Up @@ -628,19 +629,18 @@ Socket.prototype.write = function(chunk, encoding, cb) {


Socket.prototype._writeGeneric = function(writev, data, encoding, cb) {
// If we are still connecting, then buffer this for later.
// The Writable logic will buffer up any more writes while
// waiting for this one to be done.
// If the socket is ended / uncorked while connecting, a single write can
// fire through the cork. Buffer it and send it once connected. Writable
// will buffer any sequential writes if uncorked.
if (this._connecting) {
this._pendingData = data;
this._pendingEncoding = encoding;
this._pendingWrite = { data: data, encoding: encoding };

this.once('connect', function() {
this._pendingWrite = null;
this._writeGeneric(writev, data, encoding, cb);
});
return;
}
this._pendingData = null;
this._pendingEncoding = '';

this._unrefTimer();

Expand Down Expand Up @@ -730,8 +730,6 @@ function createWriteReq(req, handle, data, encoding) {
Socket.prototype.__defineGetter__('bytesWritten', function() {
var bytes = this._bytesDispatched;
const state = this._writableState;
const data = this._pendingData;
const encoding = this._pendingEncoding;

if (!state)
return undefined;
Expand All @@ -743,11 +741,24 @@ Socket.prototype.__defineGetter__('bytesWritten', function() {
bytes += Buffer.byteLength(el.chunk, el.encoding);
});

if (data) {
if (data instanceof Buffer)
bytes += data.length;
else
bytes += Buffer.byteLength(data, encoding);
var buffer = this._pendingWrite;
if (Array.isArray(buffer)) {
// was a writev, iterate over chunks to get total length
for (let i = 0; i < buffer.length; i++) {
let chunk = buffer[i];

if (chunk instanceof Buffer) {
bytes += chunk.length;
} else {
bytes += Buffer.byteLength(chunk.data, chunk.encoding);
}
}
} else if (buffer) {
if (buffer.data instanceof Buffer) {
bytes += buffer.data.length;
} else {
bytes += Buffer.byteLength(buffer.data, buffer.encoding);
}
}

return bytes;
Expand Down Expand Up @@ -869,6 +880,7 @@ Socket.prototype.connect = function(options, cb) {
}

if (this.destroyed) {
// Recycle the socket.
this._readableState.reading = false;
this._readableState.ended = false;
this._readableState.endEmitted = false;
Expand Down Expand Up @@ -899,6 +911,12 @@ Socket.prototype.connect = function(options, cb) {
this._connecting = true;
this.writable = true;

// Cork then uncork upon connecting.
this.cork();
this.once('connect', function() {
this.uncork();
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I'm confused. What does this gain?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is supposed to keep the initial write (if written to multiple times when connecting) inside the stream buffer - then, when it connects (and is writable), all the writes can be flushed at once (with writev), rather than a single write then a writev with the rest.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm. wonder how this would have worked before. I assume writes to an unconnected socket would have either errored or been lost.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before they were kept in the _pendingData, _pendingEncoding things, and the callback wouldn't be called, so the streams would buffer the rest. This still happens after this commit though (on a smaller scale with _pendingBuffer) because .end() uncorks completely.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason you cork eagerly here instead of lazily in Socket#_writeGeneric()? The change LGTM, just wondering.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By corking eagerly, we make sure (as often as possible) that the writes stay in the stream infrastructure rather than like before, where one sat in net.js.


if (pipe) {
connect(this, options.path);
} else {
Expand Down
36 changes: 36 additions & 0 deletions test/parallel/test-net-socket-byteswritten.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
'use strict';

const common = require('../common');
const assert = require('assert');
const net = require('net');
const Buffer = require('buffer').Buffer;

var server = net.createServer(function(socket) {
socket.end();
});

server.listen(common.PORT, common.mustCall(function() {
var socket = net.connect(common.PORT);

// Cork the socket, then write twice; this should cause a writev, which
// previously caused an err in the bytesWritten count.
socket.cork();

socket.write('one');
socket.write(new Buffer('twø', 'utf8'));

socket.uncork();

// one = 3 bytes, twø = 4 bytes
assert.equal(socket.bytesWritten, 3 + 4);

socket.on('connect', common.mustCall(function() {
assert.equal(socket.bytesWritten, 3 + 4);
}));

socket.on('end', common.mustCall(function() {
assert.equal(socket.bytesWritten, 3 + 4);

server.close();
}));
}));