http: throw an error for unexpected agent values #10053

brad-decker · 2016-12-01T18:08:18Z

Checklist

make -j8 test (UNIX), or vcbuild test nosign (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

Affected core subsystem(s)

http, test

Description of change

As per #9069 unexpected things can happen when supplying
an unexpected value to agent. Beings as the docs clearly
state the expected values, this throws an error on an
unexpected value.

bnoordhuis · 2016-12-01T20:34:57Z

lib/_http_client.js

@@ -35,6 +34,13 @@ function ClientRequest(options, cb) {
  } else if ((agent === null || agent === undefined) &&
             typeof options.createConnection !== 'function') {
    agent = defaultAgent;
+  } else if (agent !== null && agent !== undefined &&
+            !(agent instanceof Agent.Agent)) {
+    console.log(agent, typeof agent);


Please remove the console.log.

doh, thats embarassing. Will remove.

bnoordhuis · 2016-12-01T20:35:51Z

lib/_http_client.js

@@ -35,6 +34,13 @@ function ClientRequest(options, cb) {
  } else if ((agent === null || agent === undefined) &&
             typeof options.createConnection !== 'function') {
    agent = defaultAgent;
+  } else if (agent !== null && agent !== undefined &&
+            !(agent instanceof Agent.Agent)) {


What is the reason for these specific checks vs. a simple } else {?

We should accept an instance of Agent.Agent. Had to check for the null value as well because the previous if statement also requires options.createConnection to not be a function.

This if block could probably be rewritten slightly to account for all acceptable values and then throw the error in a simple else clause.

bnoordhuis · 2016-12-01T20:36:52Z

lib/_http_client.js

+    console.log(agent, typeof agent);
+    throw new Error(
+      'Agent option must be an instance of http.Agent, undefined or false. '
+      + 'Received type: ' + typeof agent + ' instead'


Tiny nit: can you indent by four spaces here? It's allowed to use string templates (backticks) if you think that's easier to read.

@cjihrig requested moving the + to the previous line, once doing that I got confused by the request for 4 spaces. Should the second line of text be indented another level?

bnoordhuis · 2016-12-01T20:37:08Z

test/parallel/test-http-client-reject-unexpected-agent.js

+  method: 'GET',
+  port: undefined,
+  host: '127.0.0.1',
+  agent: true,


Can you test some other values as well?

bnoordhuis · 2016-12-01T20:38:05Z

test/parallel/test-http-client-reject-unexpected-agent.js

+
+try {
+  http.request(options);
+} catch (err) {


Please use assert.throws() instead of try/catch. This should fail but won't when http.request doesn't throw an exception.

👍 definitely

cjihrig · 2016-12-01T21:59:02Z

test/parallel/test-http-client-reject-unexpected-agent.js

@@ -0,0 +1,21 @@
+'use strict';
+require('../common');
+var assert = require('assert');


Can you use const instead of var in this file.

cjihrig · 2016-12-01T21:59:24Z

test/parallel/test-http-client-reject-unexpected-agent.js

+  assert.strictEqual(
+    err.message,
+    'Agent option must be an instance of http.Agent, undefined or false. '
+    + 'Received type: boolean instead'


Can you move the + to the previous line.

brad-decker · 2016-12-02T03:25:24Z

@cjihrig @bnoordhuis - Thanks for your feedback. I've addressed most of your remarks. I made some further changes to the logic inside of request.

brad-decker · 2016-12-02T13:07:12Z

Will squash the commits down prior to merge once approved and passing CI.

cjihrig · 2016-12-03T15:14:26Z

test/parallel/test-http-client-reject-unexpected-agent.js

+  res.end();
+});
+
+server.listen(0, baseOptions.host, function() {


Can you wrap this callback in common.mustCall().

cjihrig · 2016-12-03T15:15:04Z

test/parallel/test-http-client-reject-unexpected-agent.js

+      () => http.request(Object.assign(baseOptions, {agent})),
+      (err) => {
+        return err.message === 'Agent option must be an instance of ' +
+        'http.Agent, undefined or false. Received type: ' +


This line, and the following line, should be indented more.

cjihrig · 2016-12-03T15:16:55Z

test/parallel/test-http-client-reject-unexpected-agent.js

+    );
+  });
+
+  const intervalId = setInterval(() => {


Instead of using a timer (which tends to be flaky), and counting requests received, can you use common.mustCall() with it's second option to define how many times the server request handler is run.

That would make the process.on('exit', ...) handler unnecessary as well.

@cjihrig - what is the correct way to .close() the server with this kind of test. Ideally we wouldn't close it just after request x in case for some awful reason x + 1 request would have erroneously occurred which would need to be caught?

I've been looking in some of the other http type tests but all seem to implement something different.

Also 👍 on using the .mustCall function. I'll read through common.js in the future for test helpers.

I guess you might still need the counter to close the server. You might be able to get away with just closing the server after verifying that no exception was thrown, but that probably isn't a good idea. You could also use something like Promise.all() I suppose.

Got a solution up that i think will work, if you could re-review i would appreciate it. Thanks!

mscdex · 2016-12-03T21:27:05Z

lib/_http_client.js

@@ -14,7 +14,6 @@ const OutgoingMessage = require('_http_outgoing').OutgoingMessage;
 const Agent = require('_http_agent');
 const Buffer = require('buffer').Buffer;

-


Unnecessary/unrelated change.

Addressed in a new commit and squashed

mscdex · 2016-12-03T21:27:09Z

lib/_http_client.js

    agent = defaultAgent;
  }
+


Unnecessary/unrelated change.

Addressed in a new commit and squashed

cjihrig · 2016-12-04T05:18:23Z

lib/_http_client.js

  if (agent === false) {
    agent = new defaultAgent.constructor();
-  } else if ((agent === null || agent === undefined) &&
-             typeof options.createConnection !== 'function') {
+  } else if (agent == null && typeof options.createConnection !== 'function') {
    agent = defaultAgent;
  }


Instead of duplicating all the checks on line 34, couldn't you just add:

} else if (!(agent instanceof Agent.Agent)) { throw new TypeError(...); }

It's possible for agent to be null at this point, so I could do an else if here and check that it's not null and that it's not an instance of Agent.Agent.

I went this route as it verifies that we are dealing with appropriate types before this if statement and because the else if logic seemed slightly verbose.

I am okay with putting the extra checks on the else if statement though if it's preferred.

I think it's worth mixing into this logic in order to avoid duplicating all the checks.

@cjihrig how about this logic structure:

if (agent === false) { // noop } else if (agent == null) { if (typeof options.createConnection !== 'function') { // noop } } else if (!(agent instanceof Agent.Agent)) { // noop throw error. }

if agent is undefined or null it'll be cause by the second logic gate, and if createConnection is a function it will not do anything, and will skip the final check. The last else if is exactly as you defined earlier. Obviously the code snippet has implementation removed just for brevity.

Seems like that would hit all of the cases.

Just pushed that change, all tests passing locally 👍 thanks for the help @cjihrig

cjihrig · 2016-12-04T05:23:23Z

test/parallel/test-http-client-reject-unexpected-agent.js

+    closeServer = true;
+  }
+  request.on('response', common.mustCall(() => {
+    if (closeServer) server.close();


Couldn't you just increment numberOfRequests here, and remove the closeServer variable. Also, can you move server.close() to a new line.

EDIT: You shouldn't really need to track the number of failures, since they should be captured by assert.throws().

numberOfRequests is counting the number of requests being initiated. Some of them are actually not actually hitting the server as they fail.

Definitely can move the server.close to new line

@cjihrig addressed this in a new commit. Tracking now just the number of responses to close the server after receiving the expected number. Thanks

jasnell · 2016-12-05T23:29:45Z

Marking semver-major because of the added throw

brad-decker · 2016-12-06T14:35:03Z

@bnoordhuis at your convenience would you mind taking a gander at this again?

sam-github · 2016-12-06T15:21:10Z

lib/_http_client.js

@@ -30,11 +30,18 @@ function ClientRequest(options, cb) {

  var agent = options.agent;
  var defaultAgent = options._defaultAgent || Agent.globalAgent;
+


unrelated whitespace change

sam-github · 2016-12-06T15:24:37Z

lib/_http_client.js

-  } else if ((agent === null || agent === undefined) &&
-             typeof options.createConnection !== 'function') {
-    agent = defaultAgent;
+  } else if (agent == null) {


unrelated refactor. including purely stylistic changes in the same commit as a functional change makes PRs harder to review, and your commit message doesn't mention or justify these changes. These should be two commits, the addition of the else if (!(agent instanceof Agent.Agent)) { is the functional change, the other commit would be the style changes (though I would drop the addition of the random whitespace line, that's just code churn).

@sam-github i can actually not change this else if statement/block at all if i also check that agent is != null on my new else if (the functional change you mentioned). Other reviewers said that we should avoid duplicating the checks. What would be best in this case? I can split it into two commits as well just want to make sure i'm following best practices.

I misread the code, I see now, you pulled the check on createConnection out of the conditional, so that if it is a function, no check is made on the type of agent. Code like this is a bit hard to read, I suggest just adding a comment between line 39 and 40 below, making it clear that this case is explicitly being dropped through - that agent can be == null if there is a createConnection function, that would have caused me to not misread.

@sam-github at your convenience could you check the comment. Wasn't too sure on the wording so it seems a little verbose to me given i was struggling on explaining the case. I'm open to rewrite. Thanks again for your feedback.

sam-github · 2016-12-06T15:25:33Z

test/parallel/test-http-client-reject-unexpected-agent.js

+const failingAgentOptions = [
+  true,
+  'agent',
+  {},


would be good to inject a couple other js types here: Function, Number, Symbol

added those types

sam-github · 2016-12-06T15:26:16Z

test/parallel/test-http-client-reject-unexpected-agent.js

+  new http.Agent(),
+];
+
+


random whitespace - you have single line between global scope vars everywhere but here

jasnell · 2016-12-07T15:30:34Z

lib/_http_client.js

-  } else if ((agent === null || agent === undefined) &&
-             typeof options.createConnection !== 'function') {
-    agent = defaultAgent;
+  } else if (agent == null) {


Please use (agent === null) (strict equals)

@jasnell it looks to me that it is required to be non-strict to match undefined and null

@sam-github @jasnell its non strict to catch both types but i did take the liberty of making that change whereas original version strictly checked for both null and undefined. So i'm flexible on implementation here.

jasnell · 2016-12-07T15:31:01Z

lib/_http_client.js

+    // explicitly pass through this statement as agent will not be used
+    // when createConnection is provided.
+  } else if (!(agent instanceof Agent.Agent)) {
+    throw new Error(


Please use TypeError here

jasnell · 2016-12-07T15:33:23Z

test/parallel/test-http-client-reject-unexpected-agent.js

+  failingAgentOptions.forEach((agent) => {
+    assert.throws(
+      () => createRequest(agent),
+      (err) => {


instead of a function here, just use a regexp... e.g.

assert.throws(() => createRequest(agent), /^TypeError: Agent option must be an instance of http.Agent/);

jasnell · 2016-12-07T15:33:46Z

test/parallel/test-http-client-reject-unexpected-agent.js

+  acceptableAgentOptions.forEach((agent) => {
+    assert.doesNotThrow(
+      () => createRequest(agent),
+      Error,


TypeError

@brad-decker What I mean is that you can simply test assert.doesNotThrow(() => createRequest(agent)), there is no need to check for error types.

jasnell

Almost there... just a few bits remaining

sam-github · 2016-12-07T17:59:41Z

lib/_http_client.js

+    if (typeof options.createConnection !== 'function') {
+      agent = defaultAgent;
+    }
+    // Case when agent == null && typeof createConnection === 'function'


This line just restates the code, I would drop it, and capitalize the "Explicitly" on the next sentence, so its clear that not handling this case is intentional.

sam-github · 2016-12-07T18:00:26Z

Other than the one line of the comment that I would drop, LGTM

brad-decker · 2016-12-13T15:41:42Z

@bnoordhuis fixed the last bit of feedback you had, thanks again for reviewing and my apologies for the delay!

bnoordhuis

LGTM, thanks. CI: https://ci.nodejs.org/job/node-test-pull-request/5473/

brad-decker · 2016-12-19T15:35:27Z

@bnoordhuis - Can you point me in the direction of any resources that might help me figure out the CI testing process so i can address issues with the /arm and /linux tests? Would love to become more familiar with this process so i can be self sufficient here. As it stands i'm not quite sure what has failed.

cjihrig · 2016-12-19T15:45:14Z

This is the only failure in the CI run - https://ci.nodejs.org/job/node-test-commit-linux/6776/nodes=ubuntu1610-x64/console (search for "not ok"). It doesn't appear to be related to the changes in this PR.

brad-decker · 2016-12-19T17:09:53Z

Cool. Thanks @cjihrig - ill wait for feedback that specifies otherwise prior to delving into this.

jasnell · 2017-01-11T00:04:50Z

lib/_http_client.js

+  } else if (!(agent instanceof Agent.Agent)) {
+    throw new TypeError(
+       'Agent option must be an instance of http.Agent, undefined or false. ' +
+       `Received type: ${typeof agent} instead`


I'm really not that fond of the Received type: ... additional bits here.

@jasnell - sorry for the delay. Addressed your feedback and removed the extra bits.

jasnell · 2017-01-15T16:56:09Z

CI: https://ci.nodejs.org/job/node-test-pull-request/5873/

italoacasas · 2017-01-17T17:44:22Z

@brad-decker can you run make lint and fix the error?

brad-decker · 2017-01-17T18:23:17Z

@italoacasas i don't get any errors when running make lint. I also examined the output of the linter failure in CI and it seems to be a build error rather than a true lint error? I may be wrong i'm pretty new to this whole contributing to node thing.

gibfahn · 2017-01-17T18:30:41Z

@brad-decker I'm getting an error when I patch your PR onto master, could you rebase and see if you get an error?

➜  curl -L https://github.com/nodejs/node/pull/10053.patch | git am --whitespace=fix"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   139    0   139    0     0    341      0 --:--:-- --:--:-- --:--:--   342
100  3369    0  3369    0     0   3770      0 --:--:-- --:--:-- --:--:-- 24237
Applying: http: throw an error for unexpected agent values
➜  node git:(master) make lint
./node tools/eslint/bin/eslint.js --cache --rulesdir=tools/eslint-rules \
	  benchmark lib test tools

/Users/gib/wrk/com/node/test/parallel/test-http-client-reject-unexpected-agent.js
  52:8  error  Expected indentation of 6 spaces but found 7  indent

✖ 1 problem (1 error, 0 warnings)

make: *** [jslint] Error 1

gibfahn · 2017-01-17T18:31:06Z

test/parallel/test-http-client-reject-unexpected-agent.js

+  failingAgentOptions.forEach((agent) => {
+    assert.throws(
+      () => createRequest(agent),
+       /^TypeError: Agent option must be an instance of http.Agent/,


Looks like one too many spaces on this line

As per nodejs#9069 unexpected things can happen when supplying an unexpected value to agent. Beings as the docs clearly state the expected values, this throws an error on an unexpected value. Signed-off-by: brad-decker <bhdecker84@gmail.com>

brad-decker · 2017-01-17T19:08:03Z

@italoacasas @gibfahn I see what i did. i rebased on origin and not upstream :P got the lint error after rebasing. Sorry about that. fixed now

italoacasas · 2017-01-18T01:39:50Z

thanks @brad-decker

new CI: https://ci.nodejs.org/job/node-test-pull-request/5919/console

gibfahn · 2017-01-18T08:08:07Z

CI is green (ignore the test/arm reported failure)

italoacasas · 2017-01-18T15:29:38Z

Landed fc7025c

As per #9069 unexpected things can happen when supplying an unexpected value to agent. Beings as the docs clearly state the expected values, this throws an error on an unexpected value. PR-URL: #10053 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Sam Roberts <vieuxtech@gmail.com>

brad-decker · 2017-01-18T16:13:59Z

Thanks everyone!

As per nodejs#9069 unexpected things can happen when supplying an unexpected value to agent. Beings as the docs clearly state the expected values, this throws an error on an unexpected value. PR-URL: nodejs#10053 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Sam Roberts <vieuxtech@gmail.com>

fengmk2 · 2017-02-26T16:55:05Z

lib/_http_client.js

+    }
+    // Explicitly pass through this statement as agent will not be used
+    // when createConnection is provided.
+  } else if (!(agent instanceof Agent.Agent)) {


How about userland Agent? Those Agent Class maybe not inherits from Agent.Agent and now will be all fails to use them. e.g.: TunnelingAgent https://github.com/request/tunnel-agent/blob/master/index.js#L47

nodejs-github-bot added the http Issues or PRs related to the http subsystem. label Dec 1, 2016

bnoordhuis requested changes Dec 1, 2016

View reviewed changes

cjihrig requested changes Dec 1, 2016

View reviewed changes

cjihrig requested changes Dec 3, 2016

View reviewed changes

mscdex reviewed Dec 3, 2016

View reviewed changes

brad-decker force-pushed the handle-agent-input branch from 40b750e to 1bbfa4c Compare December 3, 2016 21:39

cjihrig reviewed Dec 4, 2016

View reviewed changes

brad-decker force-pushed the handle-agent-input branch from 1bbfa4c to bace99e Compare December 4, 2016 15:33

jasnell added the semver-major PRs that contain breaking changes and should be released in the next major version. label Dec 5, 2016

brad-decker force-pushed the handle-agent-input branch from bace99e to c0f4d20 Compare December 6, 2016 14:33

cjihrig approved these changes Dec 6, 2016

View reviewed changes

sam-github reviewed Dec 6, 2016

View reviewed changes

brad-decker force-pushed the handle-agent-input branch from c0f4d20 to 81d4f43 Compare December 7, 2016 02:49

jasnell reviewed Dec 7, 2016

View reviewed changes

jasnell requested changes Dec 7, 2016

View reviewed changes

sam-github reviewed Dec 7, 2016

View reviewed changes

brad-decker force-pushed the handle-agent-input branch from 81d4f43 to 0229c2d Compare December 8, 2016 01:04

bnoordhuis approved these changes Dec 19, 2016

View reviewed changes

mscdex removed the lts-watch-v6.x label Dec 21, 2016

jasnell requested changes Jan 11, 2017

View reviewed changes

brad-decker force-pushed the handle-agent-input branch from 67b9eda to 86f373e Compare January 14, 2017 13:48

nodejs-github-bot added the lts-watch-v6.x label Jan 14, 2017

jasnell approved these changes Jan 15, 2017

View reviewed changes

gibfahn reviewed Jan 17, 2017

View reviewed changes

brad-decker force-pushed the handle-agent-input branch from 86f373e to 25cd753 Compare January 17, 2017 19:09

nodejs-github-bot added the lts-watch-v6.x label Jan 17, 2017

italoacasas closed this Jan 18, 2017

lance mentioned this pull request Jan 19, 2017

http core module issue #9069

Closed

fengmk2 reviewed Feb 26, 2017

View reviewed changes

fengmk2 mentioned this pull request Feb 26, 2017

http: should support userland Agent #11567

Closed

4 tasks

jasnell mentioned this pull request Apr 4, 2017

8.0.0 Release Proposal #12220

Closed

		@@ -14,7 +14,6 @@ const OutgoingMessage = require('_http_outgoing').OutgoingMessage;
		const Agent = require('_http_agent');
		const Buffer = require('buffer').Buffer;

		@@ -30,11 +30,18 @@ function ClientRequest(options, cb) {

		var agent = options.agent;
		var defaultAgent = options._defaultAgent \|\| Agent.globalAgent;

http: throw an error for unexpected agent values #10053

http: throw an error for unexpected agent values #10053

Conversation

brad-decker commented Dec 1, 2016 • edited Loading

Checklist

Affected core subsystem(s)

Description of change

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brad-decker commented Dec 2, 2016

brad-decker commented Dec 2, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cjihrig Dec 4, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jasnell commented Dec 5, 2016

brad-decker commented Dec 6, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jasnell left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sam-github commented Dec 7, 2016

brad-decker commented Dec 13, 2016

bnoordhuis left a comment

Choose a reason for hiding this comment

brad-decker commented Dec 19, 2016

cjihrig commented Dec 19, 2016

brad-decker commented Dec 19, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jasnell commented Jan 15, 2017

italoacasas commented Jan 17, 2017

brad-decker commented Jan 17, 2017

gibfahn commented Jan 17, 2017 • edited Loading

gibfahn Jan 17, 2017 • edited Loading

Choose a reason for hiding this comment

brad-decker commented Jan 17, 2017

italoacasas commented Jan 18, 2017

brad-decker commented Dec 1, 2016 •

edited

Loading

cjihrig Dec 4, 2016 •

edited

Loading

gibfahn commented Jan 17, 2017 •

edited

Loading

gibfahn Jan 17, 2017 •

edited

Loading

gibfahn commented Jan 18, 2017 •

edited

Loading