Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: make serialized objects Uint8Array instead of Buffer #134

Merged
merged 4 commits into from
Apr 17, 2024

Conversation

matthewkeil
Copy link
Member

@matthewkeil matthewkeil commented Apr 17, 2024

@nflaig pointed out that the return types are buffer not uint8array. Putting this up in case we should move to uint8array.

After this and the bls package are published remove this cast in lodestar tests
ChainSafe/lodestar#6616 (comment)

Copy link

github-actions bot commented Apr 17, 2024

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: ab72fdd Previous: - Ratio
PublicKey serialization 1.1480 us/op
PublicKey deserialize 22.325 us/op
PublicKey deserialize and validate - 1 keys 83.414 us/op
PublicKey deserialize and validate - 100 keys 8.2647 ms/op
PublicKey deserialize and validate - 10000 keys 830.42 ms/op
SecretKey.fromKeygen 3.5200 us/op
SecretKey serialization 1.1440 us/op
SecretKey deserialization 1.5750 us/op
SecretKey.toPublicKey 138.10 us/op
SecretKey.sign 531.13 us/op
Signature serialization 1.2890 us/op
Signature deserialize 44.168 us/op
Signatures deserialize and validate - 1 sets 120.23 us/op
Signatures deserialize and validate - 100 sets 11.960 ms/op
Signatures deserialize and validate - 10000 sets 1.1990 s/op
aggregatePublicKeys - 1 sets 2.4350 us/op
aggregatePublicKeys - 8 sets 10.579 us/op
aggregatePublicKeys - 32 sets 38.064 us/op
aggregatePublicKeys - 128 sets 148.63 us/op
aggregatePublicKeys - 256 sets 296.60 us/op
aggregateSignatures - 1 sets 4.2700 us/op
aggregateSignatures - 8 sets 23.864 us/op
aggregateSignatures - 32 sets 91.174 us/op
aggregateSignatures - 128 sets 361.04 us/op
aggregateSignatures - 256 sets 722.40 us/op
aggregateVerify - 1 sets 1.5944 ms/op
aggregateVerify - 8 sets 5.7591 ms/op
aggregateVerify - 32 sets 20.428 ms/op
aggregateVerify - 128 sets 79.127 ms/op
aggregateVerify - 256 sets 157.46 ms/op
verifyMultipleAggregateSignatures - 1 sets 1.6362 ms/op
verifyMultipleAggregateSignatures - 8 sets 6.1863 ms/op
verifyMultipleAggregateSignatures - 32 sets 22.059 ms/op
verifyMultipleAggregateSignatures - 128 sets 85.633 ms/op
verifyMultipleAggregateSignatures - 256 sets 170.49 ms/op
Same message - 1 sets 1.7092 ms/op
Same message - 8 sets 2.5784 ms/op
Same message - 32 sets 5.5302 ms/op
Same message - 128 sets 17.351 ms/op
Same message - 256 sets 33.089 ms/op
libuv multithreading - addVerificationRandomness true 20.672 s/op
libuv multithreading - addVerificationRandomness false 20.674 s/op

by benchmarkbot/action

@matthewkeil
Copy link
Member Author

matthewkeil commented Apr 17, 2024

There is something very odd with how Napi is returning Uint8Array from the constructor.... I am returning Uint8Array from the native side and the __proto__ shows that it inherits from Uint8Array however the toString method is the Buffer.prototype.toString that allows three parameters.... and they work. Can change the encoding. But only works on some platform/version combos.... sigh...

    describe("serialize", () => {
      it.only("should serialize the key to Uint8Array", () => {
        const serialized = key.serialize();
        console.log(serialized.__proto__);
        console.log(serialized.toString.length);
        expect(serialized).to.be.instanceof(Buffer);
      });
    });
❯ yarn test:unit
yarn run v1.22.21
$ mocha test/unit/**/*.test.ts

  SecretKey
    instance methods
      serialize
Uint8Array {
  readBigUInt64LE: [Function: readBigUInt64LE],
  readBigUInt64BE: [Function: readBigUInt64BE],
  readBigUint64LE: [Function: readBigUInt64LE],
  readBigUint64BE: [Function: readBigUInt64BE],
  readBigInt64LE: [Function: readBigInt64LE],
  readBigInt64BE: [Function: readBigInt64BE],
  writeBigUInt64LE: [Function: writeBigUInt64LE],
  writeBigUInt64BE: [Function: writeBigUInt64BE],
  writeBigUint64LE: [Function: writeBigUInt64LE],
  writeBigUint64BE: [Function: writeBigUInt64BE],
  writeBigInt64LE: [Function: writeBigInt64LE],
  writeBigInt64BE: [Function: writeBigInt64BE],
  readUIntLE: [Function: readUIntLE],
  readUInt32LE: [Function: readUInt32LE],
  readUInt16LE: [Function: readUInt16LE],
  readUInt8: [Function: readUInt8],
  readUIntBE: [Function: readUIntBE],
  readUInt32BE: [Function: readUInt32BE],
  readUInt16BE: [Function: readUInt16BE],
  readUintLE: [Function: readUIntLE],
  readUint32LE: [Function: readUInt32LE],
  readUint16LE: [Function: readUInt16LE],
  readUint8: [Function: readUInt8],
  readUintBE: [Function: readUIntBE],
  readUint32BE: [Function: readUInt32BE],
  readUint16BE: [Function: readUInt16BE],
  readIntLE: [Function: readIntLE],
  readInt32LE: [Function: readInt32LE],
  readInt16LE: [Function: readInt16LE],
  readInt8: [Function: readInt8],
  readIntBE: [Function: readIntBE],
  readInt32BE: [Function: readInt32BE],
  readInt16BE: [Function: readInt16BE],
  writeUIntLE: [Function: writeUIntLE],
  writeUInt32LE: [Function: writeUInt32LE],
  writeUInt16LE: [Function: writeUInt16LE],
  writeUInt8: [Function: writeUInt8],
  writeUIntBE: [Function: writeUIntBE],
  writeUInt32BE: [Function: writeUInt32BE],
  writeUInt16BE: [Function: writeUInt16BE],
  writeUintLE: [Function: writeUIntLE],
  writeUint32LE: [Function: writeUInt32LE],
  writeUint16LE: [Function: writeUInt16LE],
  writeUint8: [Function: writeUInt8],
  writeUintBE: [Function: writeUIntBE],
  writeUint32BE: [Function: writeUInt32BE],
  writeUint16BE: [Function: writeUInt16BE],
  writeIntLE: [Function: writeIntLE],
  writeInt32LE: [Function: writeInt32LE],
  writeInt16LE: [Function: writeInt16LE],
  writeInt8: [Function: writeInt8],
  writeIntBE: [Function: writeIntBE],
  writeInt32BE: [Function: writeInt32BE],
  writeInt16BE: [Function: writeInt16BE],
  readFloatLE: [Function: readFloatForwards],
  readFloatBE: [Function: readFloatBackwards],
  readDoubleLE: [Function: readDoubleForwards],
  readDoubleBE: [Function: readDoubleBackwards],
  writeFloatLE: [Function: writeFloatForwards],
  writeFloatBE: [Function: writeFloatBackwards],
  writeDoubleLE: [Function: writeDoubleForwards],
  writeDoubleBE: [Function: writeDoubleBackwards],
  asciiSlice: [Function: asciiSlice],
  base64Slice: [Function: base64Slice],
  base64urlSlice: [Function: base64urlSlice],
  latin1Slice: [Function: latin1Slice],
  hexSlice: [Function: hexSlice],
  ucs2Slice: [Function: ucs2Slice],
  utf8Slice: [Function: utf8Slice],
  asciiWrite: [Function: asciiWrite],
  base64Write: [Function: base64Write],
  base64urlWrite: [Function: base64urlWrite],
  latin1Write: [Function: latin1Write],
  hexWrite: [Function: hexWrite],
  ucs2Write: [Function: ucs2Write],
  utf8Write: [Function: utf8Write],
  parent: [Getter],
  offset: [Getter],
  copy: [Function: copy],
  toString: [Function: toString],
  equals: [Function: equals],
  inspect: [Function: inspect],
  compare: [Function: compare],
  indexOf: [Function: indexOf],
  lastIndexOf: [Function: lastIndexOf],
  includes: [Function: includes],
  fill: [Function: fill],
  write: [Function: write],
  toJSON: [Function: toJSON],
  subarray: [Function: subarray],
  slice: [Function: slice],
  swap16: [Function: swap16],
  swap32: [Function: swap32],
  swap64: [Function: swap64],
  toLocaleString: [Function: toString],
  [Symbol(nodejs.util.inspect.custom)]: [Function: inspect]
}
3
        ✓ should serialize the key to Uint8Array


  1 passing (5ms)

✨  Done in 1.10s.

Returning Buffer does not change anything...

Screenshot 2024-04-17 at 3 29 25 PM

lib/index.js Outdated Show resolved Hide resolved
@matthewkeil matthewkeil requested a review from nflaig April 17, 2024 09:28
lib/index.js Outdated Show resolved Hide resolved
@matthewkeil matthewkeil requested a review from nflaig April 17, 2024 09:52
Copy link
Member

@nflaig nflaig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@matthewkeil matthewkeil merged commit 05be5da into master Apr 17, 2024
23 checks passed
@matthewkeil matthewkeil deleted the mkeil/serialize-to-uint8array branch April 17, 2024 11:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants