Fetch protocol.json from chrome?

[paulirish]

What do you think.. if we allowed a means for you to grab the current protocol directly from Chrome?

[cyrus-and]

It would be great!
Actually, I was thinking about asking if you knew a better way than fetching it from time to time from the Blink repo.

Random suggestion: Maybe provide another URL, e.g., http://localhost:9222/protocol/json?

[paulirish]

Random suggestion: Maybe provide another URL, e.g., http://localhost:9222/protocol/json

sgtm!

[paulirish]

Filed upstream: https://code.google.com/p/chromium/issues/detail?id=538300

[cyrus-and]

Thanks! Starred.

[paulirish]

Right now we're not going to package the protocol in Chrome. It'd be 20kb but across every chrome desktop installs in the world, it's a pretty big bump, and we don't have many customers for it besides this (very awesome project).

We've discussed that the best approach for now is pulling a protocol.json from source control based on the version provided in /json/version. I've written an implementation which works well.

fetch("http://localhost:9222/json/version")
  .then( resp => resp.json() )
  .then( function(obj){   
    // eg: "WebKit-Version": "537.36 (@31978e66f0c2ae00926292b37ce61bb19827836c)"
    var ver = obj["WebKit-Version"];
    var match = ver.match(/\s\(@(\b[0-9a-f]{5,40}\b)/);
    var hash = match[1];
    return hash;
  })
  .then( hash => {
    // the blink/chromium repo merge changed file locations. https://goo.gl/JztV1A
    var url = (hash <= 202666) ?
      `http://src.chromium.org/blink/trunk/Source/devtools/protocol.json?p=${ hash }` :
      `https://chromium.googlesource.com/chromium/src/+/${ hash }/third_party/WebKit/Source/devtools/protocol.json?format=TEXT`;
    return fetch(url);
  })
  .then( resp =>  { return resp.text() }) 
  .then( function(txt){
    try { 
      // googlesource response is base64 encoded
      var protocol = atob(txt);  
    } catch (e) {
      var protocol = txt;
    }
    return Promise.resolve(JSON.parse(protocol));
  })
  .then( protocolObj => console.log(protocolObj) )

hope this helps.

[kdzwinel]

I thought protocol.json is already bundled with Chrome, just not exposed. Check out this snippet from the DT code:

It suggests that protocol.json is located in chrome-devtools://devtools/protocol.json, unfortunately it's not a valid path :(

[cyrus-and]

@paulirish OK, I'm doing that way. But I'll leave the hardcoded version too, as the scenario in which the instrumentation happens offline is not so uncommon.

@kdzwinel Neither via JavaScript? One could try to open a new tab, inject some code and fetch the protocol. Out of curiosity though; this would be too ugly to implement...

If it's already bundled how can it require 20KB more?

[kdzwinel]

@cyrus-and I tried messing with that code in the runtime to make DT download protocol.json, no luck though. Maybe it's departed or meant for some other environment (e.g. https://frontend.chrome-dev.tools ?).

[EDIT] I think I found an explanation. protocol.json is only requested when working on devtools:

When remotely debugging and developing the front-end of Blink, the content of the InspectorBackendCommands.js file is generated based on the content of protocol.json instead as a fallback of the Chromium build system.

https://developer.chrome.com/devtools/docs/contributing

[cyrus-and]

@kdzwinel It's still unclear to me whether the protocol descriptor is already bundled (even in such a way that it can be generated) in Chome or not?

@paulirish I implemented what you suggested but I don't know if it wise to enable it by default as it will introduce the non-negligible latency of fetching the protocol every time a connection with Chrome is established. This may or may not be a problem but surely it is a change in the expected behavior: what was almost immediate now takes about 1-2s.

[paulirish]

@kdzwinel good find. just asked @repenaxa about this and he confirmed mostly what you said. this would be loaded only when you're doing --remote-debugging-port=xxx and supplying a custom frontend via http://localhost:xxx#http://localhost:zz/custom_frontend/inspector.html. It's this case where we'd pull http://localhost:zz/custom_frontend/protocol.json. But that's being supplied by you and your custom frontend folder.. It's never bundled.

@cyrus-and

But I'll leave the hardcoded version too, as the scenario in which the instrumentation happens offline is not so uncommon.

Yeah I think that's wise.

what was almost immediate now takes about 1-2s.
Hmm. I don't have a great solution here.

Possibilities:

• caching on your side?
• create a new devtools-protocol.json NPM module. (It could also do this lookup work plus caching... but maybe...) It ships with all protocol.json possibilities baked in and isn't network dependent.
• something smarter?

[paulirish]

cc @repenaxa @dgozman

[cyrus-and]

@paulirish

caching on your side?

In-memory caching would solve the latency due to several connections (create tab, connect to tab, instrument, repeat...) to the same Chrome instance within a single Node.js process. But across multiple executions, the network is hit anyway. Since this is basically a library I don't know if caching on disk could be advisable.

It ships with all protocol.json possibilities baked in and isn't network dependent.

Do you mean fetching the whole version history of package.json from the repository? That module would require some manual intervention anyway any time a new version ships...

Manual caching is already possible with the current version: users may fetch and download the protocol.json once using the API, then they may pass that file to Chrome() via the protocol option during further executions.

I will think about it, but options seem quite limited.