url: allow use of URL with http.request and https.request #10638

jasnell · 2017-01-05T18:27:27Z

Allow the new URL object to be used with http.request() and https.request()

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

Affected core subsystem(s)

http, https

addaleax · 2017-01-05T22:36:13Z

lib/_http_client.js

+      pathname: options.pathname,
+      path: `${options.pathname}${options.search}`,
+      href: options.href
+    };


I’m not sure how I feel about the code duplication here… could we make this a generic helper, possibly even hanging off the public url module (if that makes transitioning url.parse → url.URL easier)?

I would suggest adding it as the toJSON method for URL but I have looked at whatwg/url#137 and I’m not sure what to make of that other than that my idea would probably be silly/unrealistic/… :P

Also – could this blocked be just executed independently of options instanceof url.URL? It would mean using path: options.path || …, but other than that, it should just work, no?

(If you feel unsure about these ideas, I can PR them myself later. If you feel they are horrible, say so and I won’t. ;))

I can eliminate the duplication but having the instanceof and copying into a separate options is the most reliable and requires the fewest number of changes throughout the code.

Those properties are enumerable (required by the spec) so a for-in loop with some ifs would do I think?

EDIT: util._extend doesn't apply because it only copies properties that are both enumerable and is own property. A helper function can loose the "isOwnProperty" bit but maybe not worth an additional abstraction anyway?

I'd rather avoid doing the for-in loop in favor of keeping things more explicit and obvious. Also, as you point out, the auth vs username+password would demonstrates that the properties do not match up one-to-one.

joyeecheung · 2017-01-06T08:31:03Z

lib/_http_client.js

+  } else if (options instanceof url.URL) {
+    options = {
+      protocol: options.protocol,
+      host: options.host,


Maybe a auth: `${options.username}:${options.password}` here(and in http.js too)?

EDIT: don't know why this comment appeared in another line, deleted that.

joyeecheung · 2017-01-06T08:42:49Z

lib/_http_client.js

@@ -24,6 +24,18 @@ function ClientRequest(options, cb) {
    if (!options.hostname) {
      throw new Error('Unable to determine the domain name');
    }
+  } else if (options instanceof url.URL) {


If I understand correctly this makes using the new URL api and using additional options(like agent) mutual exclusive?

True, but that's already the case if you pass the URL as a string or the result of url.parse. We can think later about a way to pass a URL object along with additional options but I don't think it should block this PR.

Ah I should have make this one a comment, not a change request :/ Sorry.

No worries!

Yes, making those mutually exclusive is intentional. Attaching additional non-standard properties to the URL object is not something that we should promote. And as @targos points out, that is already the case when passing the URL as a string.

jasnell · 2017-01-06T16:35:26Z

@addaleax @joyeecheung ... updated! PTAL

joyeecheung · 2017-01-07T03:54:01Z

test/parallel/test-whatwg-url-properties.js

 'use strict';

 require('../common');

 const URL = require('url').URL;
 const assert = require('assert');
+const urlToOptions = require('internal/url').urlToOptions;


Maybe this one deserves a separate test? The name of this test doesn't really match this API here.

I'm not convinced that it matters too much either way, but there's no harm in separating it out.

joyeecheung · 2017-01-07T04:37:10Z

LGTM. The tests could be split in another PR anyway.

joyeecheung · 2017-01-07T05:29:33Z

CI: https://ci.nodejs.org/job/node-test-pull-request/5746/

review was updated to a LGTM

PR-URL: #10638 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Evan Lucas <evanlucas@me.com> Reviewed-By: Michal Zasso <targos@protonmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

jasnell · 2017-01-09T17:30:47Z

Landed in 0f62ee6

PR-URL: nodejs#10638 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Evan Lucas <evanlucas@me.com> Reviewed-By: Michal Zasso <targos@protonmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

PR-URL: #10638 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Evan Lucas <evanlucas@me.com> Reviewed-By: Michal Zasso <targos@protonmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

PR-URL: nodejs#10638 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Evan Lucas <evanlucas@me.com> Reviewed-By: Michal Zasso <targos@protonmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

Notable changes: * crypto: * ability to select cert store at runtime (Adam Majer) #8334 * Use system CAs instead of using bundled ones (Adam Majer) #8334 * deps: * upgrade npm to 4.1.2 (Kat Marchán) #11020 * upgrade openssl sources to 1.0.2k (Shigeki Ohtsu) #11021 * doc: add basic documentation for WHATWG URL API (James M Snell) #10620 * process: add NODE_NO_WARNINGS environment variable (cjihrig) #10842 * url: allow use of URL with http.request and https.request (James M Snell) #10638 PR-URL: #11062

stevenvachon · 2017-05-30T18:43:02Z

lib/internal/url.js

+    options.port = Number(url.port);
+  }
+  if (url.username || url.password) {
+    options.auth = `${url.username}:${url.password}`;


This doesn't support new URL("http://0:0@hostname/")

Ref: nodejs#10638

PR-URL: #13409 Refs: #10638 Reviewed-By: Anna Henningsen <anna@addaleax.net>

url: allow use of URL with http.request and https.request

ce4e718

jasnell added the semver-minor PRs that contain new features and should be released in the next minor version. label Jan 5, 2017

nodejs-github-bot added http Issues or PRs related to the http subsystem. https Issues or PRs related to the https subsystem. dont-land-on-v7.x labels Jan 5, 2017

jasnell added dont-land-on-v4.x whatwg-url Issues and PRs related to the WHATWG URL implementation. and removed dont-land-on-v7.x labels Jan 5, 2017

cjihrig approved these changes Jan 5, 2017

View reviewed changes

evanlucas approved these changes Jan 5, 2017

View reviewed changes

targos approved these changes Jan 5, 2017

View reviewed changes

addaleax reviewed Jan 5, 2017

View reviewed changes

joyeecheung requested changes Jan 6, 2017

View reviewed changes

joyeecheung previously requested changes Jan 6, 2017

View reviewed changes

Improvements

02b136d

nodejs-github-bot added the dont-land-on-v7.x label Jan 6, 2017

joyeecheung reviewed Jan 7, 2017

View reviewed changes

jasnell closed this Jan 9, 2017

jasnell removed the dont-land-on-v7.x label Jan 9, 2017

italoacasas mentioned this pull request Jan 25, 2017

Proposal to update v7.x-staging #10989

Closed

italoacasas mentioned this pull request Jan 29, 2017

v7.5.0 proposal #11062

Merged

stevenvachon mentioned this pull request Feb 13, 2017

WHATWG-URL support sindresorhus/got#228

Closed

stevenvachon reviewed May 30, 2017

View reviewed changes

joyeecheung mentioned this pull request Jun 2, 2017

doc: http.request and https.request accepts WHATWG URL object as argument #13383

Closed

tniessen added a commit to tniessen/node that referenced this pull request Jun 2, 2017

url: do not pass WHATWG host to http.request

5cec0a5

Ref: nodejs#10638

tniessen mentioned this pull request Jun 2, 2017

url: do not pass WHATWG host to http.request #13409

Closed

2 tasks

tniessen added a commit that referenced this pull request Jun 4, 2017

url: do not pass WHATWG host to http.request

4d89e3c

PR-URL: #13409 Refs: #10638 Reviewed-By: Anna Henningsen <anna@addaleax.net>

jasnell pushed a commit that referenced this pull request Jun 5, 2017

url: do not pass WHATWG host to http.request

d4d138c

PR-URL: #13409 Refs: #10638 Reviewed-By: Anna Henningsen <anna@addaleax.net>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

url: allow use of URL with http.request and https.request #10638

url: allow use of URL with http.request and https.request #10638

jasnell commented Jan 5, 2017

addaleax Jan 5, 2017

addaleax Jan 5, 2017

jasnell Jan 5, 2017

joyeecheung Jan 6, 2017 •

edited

Loading

jasnell Jan 6, 2017

joyeecheung Jan 6, 2017 •

edited

Loading

joyeecheung Jan 6, 2017

targos Jan 6, 2017

joyeecheung Jan 6, 2017

targos Jan 6, 2017

jasnell Jan 6, 2017

jasnell commented Jan 6, 2017

joyeecheung Jan 7, 2017

jasnell Jan 7, 2017

joyeecheung commented Jan 7, 2017

joyeecheung commented Jan 7, 2017

jasnell commented Jan 9, 2017

stevenvachon May 30, 2017

url: allow use of URL with http.request and https.request #10638

url: allow use of URL with http.request and https.request #10638

Conversation

jasnell commented Jan 5, 2017

Checklist

Affected core subsystem(s)

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joyeecheung Jan 6, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joyeecheung Jan 6, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jasnell commented Jan 6, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joyeecheung commented Jan 7, 2017

joyeecheung commented Jan 7, 2017

jasnell commented Jan 9, 2017

Choose a reason for hiding this comment

joyeecheung Jan 6, 2017 •

edited

Loading

joyeecheung Jan 6, 2017 •

edited

Loading