Windows Buffer.from v4.x: TypeError: hex is not a function

[coolaj86]

• Version: v4.4.7, v4.2.2
• Platform: Windows
• Subsystem: Buffer

I've got multiple Windows users complaining (see https://github.com/Daplie/rsa-compat.js/issues/9) on various versions of v4.x that they get an error TypeError: hex is not a function with the following code:

var hex = '818afc8cce82c96b750a64352940054af6a581dc50403ddc939d29bca104a8c3db21f6f58fcc6fbd3131f4ea8b795e0dcfc554f4340d847e652fa73fd00208df66ea0b6a4b6fca59970a6f27dc5365f127b34db603d8b37b84e8070983bdb846d2f2d288e33ebdbb7eba1c766ce3457f3ea72b31d9c01ec68de55989050177dc01ded42c7d9a03676704c51475d16c07e4bb4c3defe4b6b79545b265f44249d2d4fedf739f56e07588d59bc5ea7ddf53700a6b2100068ba4ab7060193289a5634e5fd42f74106e809e44c7bfc68e9eaba6fd94d828da64fe10ad138d31c6231eae0cc87cab397b2d7c59e377b4cd7132c13d91a5f3b6ac1df2e23aca2d309773';

new Buffer(hex, 'hex');
<Buffer 81 8a fc 8c ce 82 c9 6b 75 0a 64 35 29 40 05 4a f6 a5 81 dc 50 40 3d dc 93 9d 29 bc a1 04 a8 c3 db 21 f6 f5 8f cc 6f bd 31 31 f4 ea 8b 79 5e 0d cf c5 ... >

Buffer.from(hex, 'hex');
TypeError: hex is not a function
    at Function.from (native)
    at Function.from (native)
    at repl:1:8
    at REPLServer.defaultEval (repl.js:262:27)
    at bound (domain.js:287:14)
    at REPLServer.runBound [as eval] (domain.js:300:12)
    at REPLServer.<anonymous> (repl.js:431:12)
    at emitOne (events.js:82:20)
    at REPLServer.emit (events.js:169:7)
    at REPLServer.Interface._onLine (readline.js:211:10)

The buffer-v6-shim polyfill won't work because Buffer.from exists and is write-only and I can't reliably detect that it doesn't work other than with try {} catch (e) {}, but again, because Buffer.from is write-only, it can't be polyfilled with the fix.

No reported problems in v6 or on other OSes (yet).

[mscdex]

node v4 doesn't have the new Buffer.from()/Buffer.alloc()/etc. API yet.

[mscdex]

/cc @nodejs/lts @nodejs/buffer

[coolaj86]

Yes, but it seems to have some sort of undocumented Buffer.from() that prevents a proper polyfill from being used.

This should be sufficient, but it doesn't work because Buffer.from already exists:

function newBuffer(data, encoding, len) {
  return new Buffer(data, encoding, len);
}

if (!Buffer.from) {
  Buffer.from = newBuffer;
}

try {
  Buffer.from('1337', 'hex');
} catch(e) {
  // wish I could do something here to fix the broken Buffer.from
  try {
    Buffer.from = newBuffer;
  } catch(e) {
    // but alas, I cannot
  }
}

See https://github.com/Daplie/node-buffer-v6-shim/blob/master/index.js#L25

[addaleax]

@coolaj86 It’s not undocumented, it’s just inherited from the superclass:

> Buffer.from === Uint8Array.from
true

I’d say this is a bug in the shim.

[coolaj86]

How should one polyfill between deprecated v4 and the new v6?

And on that note, v6 buf.buffer is not a proper ArrayBuffer.

var arr = new Uint8Array([1, 3, 5, 9, 17]);
arr.buffer; // ArrayBuffer { byteLength: 5 }
arr[0]; // 1
arr = new Uint8Array(arr.buffer);
arr[0]; // 1


var buf = new Buffer([1, 3, 5, 9, 17]);
buf.buffer; // ArrayBuffer { byteLength: 8192 }
buf[0]; // 1
arr = new Uint8Array(buf.buffer);
arr[0]; // 0

But perhaps I need to search and see if there's already an issue for that.

Update: found one #6744

[jasnell]

The new Buffer constructor APIs are being backported and will be available in v4.5.0

[addaleax]

@coolaj86 I’d say you can just check for Buffer.from === Uint8Array.from in the shim.

Also, yes, it’s normal that multiple Buffer instances share a single ArrayBuffer for performance, it’s not really considered a bug. You can use Buffer.allocUnsafeSlow() if you need a completely independent Buffer.

[coolaj86]

Knowing that it's a Uint8Array doesn't help because it throws an exception when assigning to overwrite it. Unless I'm wrong and I was actually on v6 by mistake when I tested that.

But knowing that it's getting backported in v4.5 satisfies my concern.

Thanks