Improving performance of loadProtoJSON() #342
Description
Hello!
I'm one of the maintainers of https://www.npmjs.com/package/google-ads-api, which uses this lib internally to make Google Ads API calls.
As part of our library, we frequently make calls to the loadProtoJSON() function in gax-nodejs. You have a caching mechanism, which is a godsend, but the function is slower than it could be even when the proto is cached.
The culprit of the slowdown is here: https://github.com/googleapis/gax-nodejs/blob/9d45baf252433b35b76a0ad6ede80509eb066ab4/src/grpc.ts#L295
The objectHash function, in my testing, takes around ~150ms to return a hash for the (huge) GoogleAdsService proto. That's a lot of CPU to be devoting to generating a cache key! On our production servers, we're seeing around 25% of our cpu usage dedicated to generating these hashes.
We could work around this by making our own caching layer, but it feels like it wouldn't be too difficult to improve this performance. For example, the following code takes less than 10ms:
const hash = objectHash(JSON.stringify(json)).toString();Or, why not get rid of the object-hash dependency entirely by using the built-in crypto lib (also around 10ms):
const hash = crypto.createHash('md5').update(JSON.stringify(json)).digest('hex');As far as I can tell, we don't lose any uniqueness by stringifying the proto object first.
Environment details
- OS: MacOS 12.1 (M1 chip)
- Node.js version: v16.14.0
- npm version: v8.3.1
google-gaxversion: v2.30.0
Steps to reproduce
- call loadProtoJSON() with large proto json (eg https://github.com/googleapis/googleapis/blob/master/google/ads/googleads/v10/services/google_ads_service.proto)
- Check how long it takes to generate the cache key.