Performance of WHATWG URL API

[SukkaW]

• Version:

  • 10.17.0
  • 12.13.0
  • 13.1.0

• Subsystem: url

I have encountered performance regression when mitigating deprecated Legacy URL API to newer recommended WHATWG URL API for a performance-sensitive application. I have set up a benchmark: https://runkit.com/sukkaw/5dc5924f1ba0d60013d87cb3

const Benchmark = require('benchmark');
const Suite = new Benchmark.Suite;

// const { URL } = require('url') is only needed for Nodejs 8.
const { parse, URL } = require('url');

const safeWhatwgUrl = (str) => {
  // Handle relative path
  try {
    return new URL(str);
  } catch (e) {
    return str;
  }
};

Suite.add('Absolute URL - Legacy URL parse()', () => {
  parse('https://skk.moe/path/to/something');
}).add('Absolute URL - WHATWG URL API', () => {
  new URL('https://skk.moe/path/to/something');
}).add('Absolute URL - safe WHATWG URL', () => {
  // Should be very slow, since try...catch is very expensive
  safeWhatwgUrl('https://skk.moe/path/to/something');
}).add('Relative Path - Legacy URL parse()', () => {
  parse('/path/to/something');
}).add('Relative Path - WHATWG URL API', () => {
  // pass a `base` to handle relative path
  new URL('/path/to/something', 'https://skk.moe/');
}).add('Relative Path - safe WHATWG URL', () => {
  // Should be even slower.
  safeWhatwgUrl('/path/to/something');
}).on('cycle', function(event) {
  console.info(String(event.target));
}).run();

Resulted in:

Node.js 8.16.2

"Absolute URL - Legacy URL parse() x 214,035 ops/sec ±1.94% (84 runs sampled)"
"Absolute URL - WHATWG URL API x 182,604 ops/sec ±0.62% (88 runs sampled)"
"Absolute URL - safe WHATWG URL x 162,791 ops/sec ±3.63% (83 runs sampled)"
"Relative Path - Legacy URL parse() x 1,008,555 ops/sec ±3.55% (74 runs sampled)"
"Relative Path - WHATWG URL API x 70,292 ops/sec ±2.69% (80 runs sampled)"
"Relative Path - safe WHATWG URL x 55,500 ops/sec ±2.62% (85 runs sampled)"

On Node.js 8.16.2, The WHATWG URL API is 15% slower than Legacy URL API when handle a normal URL. It seems normal and acceptable.

Node.js 10.17.0

"Absolute URL - Legacy URL parse() x 483,508 ops/sec ±1.00% (93 runs sampled)"
"Absolute URL - WHATWG URL API x 286,201 ops/sec ±1.11% (93 runs sampled)"
"Absolute URL - safe WHATWG URL x 272,538 ops/sec ±1.00% (93 runs sampled)"
"Relative Path - Legacy URL parse() x 2,127,003 ops/sec ±0.46% (95 runs sampled)"
"Relative Path - WHATWG URL API x 169,296 ops/sec ±0.58% (96 runs sampled)"
"Relative Path - safe WHATWG URL x 96,357 ops/sec ±0.11% (98 runs sampled)"

On Node.js 10.17.0, The WHATWG URL API is 40% slower than Legacy URL API when handle a normal URL.

Node.js 12.13.0

"Absolute URL - Legacy URL parse() x 393,703 ops/sec ±1.21% (89 runs sampled)"
"Absolute URL - WHATWG URL API x 225,574 ops/sec ±0.59% (92 runs sampled)"
"Absolute URL - safe WHATWG URL x 208,342 ops/sec ±0.98% (90 runs sampled)"
"Relative Path - Legacy URL parse() x 1,513,601 ops/sec ±3.66% (84 runs sampled)"
"Relative Path - WHATWG URL API x 114,788 ops/sec ±1.29% (92 runs sampled)"
"Relative Path - safe WHATWG URL x 10,425 ops/sec ±0.99% (89 runs sampled)"

On Node.js 12.13.0, The WHATWG URL API is 42% slower than Legacy URL API when handle a normal URL.

Node.js 13.1.0

"Absolute URL - Legacy URL parse() x 345,345 ops/sec ±0.94% (87 runs sampled)"
"Absolute URL - WHATWG URL API x 212,058 ops/sec ±0.41% (91 runs sampled)"
"Absolute URL - safe WHATWG URL x 200,212 ops/sec ±0.76% (92 runs sampled)"
"Relative Path - Legacy URL parse() x 1,407,585 ops/sec ±1.72% (89 runs sampled)"
"Relative Path - WHATWG URL API x 108,171 ops/sec ±0.51% (90 runs sampled)"
"Relative Path - safe WHATWG URL x 10,279 ops/sec ±0.57% (90 runs sampled)"

On Node.js 13.1.0, The WHATWG URL API is 38% slower than Legacy URL API when handle a normal URL.

And when I try to pass a base to handle relative path, it becomes even more slower. I believe it is because I pass a string to base so The WHATWG URL have to parse base first.

[joyeecheung]

The last time I looked into the performance of the WHATWG URL parsing the bottleneck lies in the usage of std::string in the C++ layer (potentially fixable with something like zones, but that might be a bit over-engineering...).

I wonder what the performance drop from v8.x to v10.x could be from though..

[SukkaW]

benchmark run on repl.it (Node.js 10.16.0): https://repl.it/repls/PromotedLooseDemos

Absolute URL - Legacy URL parse() x 118,275 ops/sec ±9.76% (44 runs sampled)
Absolute URL - WHATWG URL API x 57,238 ops/sec ±4.40% (49 runs sampled)
Absolute URL - safe WHATWG URL x 52,917 ops/sec ±5.69% (48 runs sampled)
Relative Path - Legacy URL parse() x 960,842 ops/sec ±5.24% (49 runs sampled)
Relative Path - WHATWG URL API x 31,054 ops/sec ±5.15% (50 runs sampled)
Relative Path - safe WHATWG URL x 19,559 ops/sec ±7.28% (47 runs sampled)

And still, Legacy URL API is way faster (at least 2x) than The WHATWG URL API.

[Prinzhorn]

I'm seeing url.parse being roughly 15 times faster for "garbage" / "could be URL" (I'm throwing large amounts of data at it, extracting stuff I need in a fuzzy fashion). I know it's an odd use-case, but it's a use-case.

Reproducible here https://repl.it/repls/LimpingQuarrelsomeCoins

Garbage - Legacy URL parse() x 398,767 ops/sec ±36.51% (45 runs sampled)
Garbage - WHATWG URL API x 27,226 ops/sec ±6.81% (46 runs sampled)

[SukkaW]

@Prinzhorn

try { } catch (e) { } should be considered as very slow (which I have presented in my benchmark) and should be avoided.

[devsnek]

@SukkaW it was slower in your benchmarks for two reasons: 1) you had an extra function call, and 2) the second one always threw, and throwing is a slower path than not throwing. I would argue it shouldn't have been throwing anyway because you should have passed a base to it like you did the other relative test.

Absolute URL - Legacy URL parse() x 813,426 ops/sec ±0.41% (97 runs sampled)
Absolute URL - WHATWG URL API x 541,469 ops/sec ±0.42% (99 runs sampled)
Absolute URL - safe WHATWG URL x 543,499 ops/sec ±0.17% (99 runs sampled)
Relative Path - Legacy URL parse() x 4,984,333 ops/sec ±0.46% (95 runs sampled)
Relative Path - WHATWG URL API x 268,401 ops/sec ±0.13% (90 runs sampled)
Relative Path - safe WHATWG URL x 271,700 ops/sec ±0.20% (96 runs sampled)

'use strict';

const Benchmark = require('benchmark');
const Suite = new Benchmark.Suite;

// const { URL } = require('url') is only needed for Nodejs 8.
const { parse, URL } = require('url');

Suite.add('Absolute URL - Legacy URL parse()', () => {
  return parse('https://skk.moe/path/to/something');
}).add('Absolute URL - WHATWG URL API', () => {
  return new URL('https://skk.moe/path/to/something');
}).add('Absolute URL - safe WHATWG URL', () => {
  const str = 'https://skk.moe/path/to/something';
  try {
    return new URL(str);
  } catch {
    return str;
  }
}).add('Relative Path - Legacy URL parse()', () => {
  return parse('/path/to/something');
}).add('Relative Path - WHATWG URL API', () => {
  return new URL('/path/to/something', 'https://skk.moe/');
}).add('Relative Path - safe WHATWG URL', () => {
  const str = '/path/to/something';
  try {
    return new URL(str, 'https://skk.moe');
  } catch {
    return str;
  }
}).on('cycle', function(event) {
  console.info(String(event.target));
}).run();

[SukkaW]

@devsnek

As you can see, the WHATWG URL API just can not handle any relative path. There are always cases that no base could be given. And the code snippet I described below are widely used:

const safeWhatwgUrl = (str) => {
  // Handle relative path
  try {
    return new URL(str);
  } catch (e) {
    return str;
  }
};

See eggjs/egg#4048: https://github.com/eggjs/egg/blob/1181a675cae08c2d6952b8c15a3a9375947e220b/lib/core/utils.js#L66-L72

And even a base is given (no matter if there is a try..catch block) when parsing a relative path, the WHATWG URL is still a kind of 3x slower than the Legacy URL API.

[devsnek]

@SukkaW right... I agree that the whatwg parser is slower. My point was that your comparison was not really apples to apples (you were comparing a parse that always passed to a parse that always failed), and that try/catch isn't really slow.

[SukkaW]

@devsnek we both agree that try catch could be as fast as code without them when there is no exception thrown (which is also presented in the first part of my benchmark).

However in the specific case related to this issue, people are using try catch to handle relative path directly (as I mentioned code from eggjs, no base is given).

So in the second part of my benchmark I setup a case when only relative path to be parsed (and try catch for no given base).

Adding a case that has base passed is only to show that even with no exception to be catched by try catch, implementations with the WHATWG URL API is still not fast enough. This is what I thought when setting up my benchmark.

[Prinzhorn]

try { } catch (e) { } should be considered as very slow (which I have presented in my benchmark) and should be avoided.

@SukkaW thanks, I didn't know try/catch was that slow. However, once url.parse will disappear we have to use try/catch in order to safely replace it with the new API. So you cannot have a real world comparison without try/catch, it is a part of the new API if you will. But I think we already agree on that 👍

[jasnell]

Unfortunately there's not a lot we can do here. The existing url.parse() uses a number of performance optimization tricks that make it faster but have had the side effect of making it largely unmaintainable. It has a number of security and parsing bugs that cannot be fixed properly without severe impact on it's performance and backwards compatibility. The WHATWG URL parser can, I'm sure, be made faster but a significant reason it is slower is because it takes a much stricter approach to spec compliance and favors parsing correctness over performance.