Add basic extension add-on framework.

[mrstegeman]

There are a lot of things we can do to create more extension points,
but this at least provides a starting point which already opens up a
lot of possiblities.

Fixes #2143
Fixes #2144

[mrstegeman]

This can be tested with: https://github.com/mozilla-iot/example-extension

[codecov-io]

Codecov Report

Merging #2155 into master will decrease coverage by 0.16%.
The diff coverage is 45.09%.

@@            Coverage Diff             @@
##           master    #2155      +/-   ##
==========================================
- Coverage    70.4%   70.23%   -0.17%     
==========================================
  Files         132      133       +1     
  Lines        7544     7594      +50     
  Branches     1187     1195       +8     
==========================================
+ Hits         5311     5334      +23     
- Misses       1928     1950      +22     
- Partials      305      310       +5

Impacted Files	Coverage Δ
src/router.js	98.57% <100%> (+0.02%)	⬆️
src/addon-utils.js	79.74% <100%> (ø)	⬆️
src/constants.js	100% <100%> (ø)	⬆️
src/controllers/extensions_controller.js	40.54% <40.54%> (ø)
src/addon-manager.js	60.83% <45.45%> (-0.37%)	⬇️
src/controllers/things_controller.js	81.72% <0%> (+0.34%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cd88c42...e4c894c. Read the comment docs.

[benfrancis]

@mrstegeman This looks great!

The way I imagined that defining a new menu entry would work is that the add-on developer could provide the content for their custom screen as HTML, CSS and JavaScript. This PR achieves the CSS and JS part of that by referring to those resources in content_scripts in the manifest, but currently any HTML has to be inserted into the DOM using JavaScript, using a reference to a top level DOM element returned in a method of the Extension class.

Looking to Web Extensions for a precedent I see that popups can be defined in the way I described, by referencing an HTML file as part of a browser_action or page_action. See this extension for an example of a popup in a browser_action. There's also the background scripts mechanism whereby you can specify a background page, which can then itself load scripts.

It's a bit tricky to draw direct analogies between browser extensions and gateway extensions and define what is a content script vs. a browser action vs. a page action, because the gateway doesn't have the concept of chrome vs. content exactly. But I do wonder whether in defining the API for adding a custom UI section with its own menu entry, we could allow developers to provide a static HTML file rather than build the DOM entirely in JavaScript? That HTML could then either be inserted into the DOM when the extension is loaded, or even embedded in an iframe (may be a bad idea...).

I'm thinking out loud a bit here, because I'm not sure exactly what the best API would be. What do you think? Perhaps you've already thought through some potential alternative approaches?

[mrstegeman]

I think that comparing to WebExtensions is a bit tricky in this case. WebExtensions are essentially given a standalone document to draw into, so they can insert a full HTML page.

In our case, a menu entry essentially just references a <section> element on the existing page. I'm unaware of a good way to insert a full HTML document into a page, other than using an <iframe> (which I really didn't want to do).

Also, I'm unsure what the background page is actually used for in a WebExtension. MDN's example just shows it loading scripts, not having content.

One possible workaround would be:

1. Write your <section> content in an HTML file.
2. Load that file via JS.
3. Insert content via JS.

[benfrancis]

@mrstegeman wrote:

I think that comparing to WebExtensions is a bit tricky in this case. WebExtensions are essentially given a standalone document to draw into, so they can insert a full HTML page.
In our case, a menu entry essentially just references a <section> element on the existing page. I'm unaware of a good way to insert a full HTML document into a page, other than using an <iframe> (which I really didn't want to do).

That's a good point. You could just copy the content of the <body> into the content of the <section>, but link relations in the <head> (e.g. for CSS style sheets) would also have to be manually moved over, which could get messy.

I had the same instinct to avoid using an iframe, but I can't think of any specific problems with that. Can you?

I'm unsure what the background page is actually used for in a WebExtension. MDN's example just shows it loading scripts, not having content.

A better example is probably a sidebar_action. Here is an example browser extension which demonstrates a sidebar provided as an HTML document (it also demonstrates the use of the context menu API which is interesting).

What if we took a similar approach to sidebars where a custom section is defined in the manifest including its icon, title and HTML content:

manifest.json

  "gateway_section": {
    "icon": "images/my_section_icon.png",
    "title" : "My Section",
    "content": "my_section.html"
  }

HTML content for a custom section could be provided as an HTML file referenced in the manifest. The HTML file could then load its own CSS and JavaScript files, rather than using the content_scripts member of the manifest which in browser extensions is really intended for modifying web content from specific domains.

my_section.html

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <link rel="stylesheet" href="css/foo.css" />
    <script src="js/bar.js"></script>
  </head>
  <body>
    <h1>My Section</h1>
  </body>
</html>

One problem I can think of with using an iframe to embed this content is that no CSS or JavaScript from the wider gateway application would be loaded into the page by default. This might make it tricky to provide APIs via global variables in the style of Web Extension APIs. The extension would have to include a script from the gateway itself.

[mrstegeman]

@mrstegeman wrote:

I think that comparing to WebExtensions is a bit tricky in this case. WebExtensions are essentially given a standalone document to draw into, so they can insert a full HTML page.
In our case, a menu entry essentially just references a <section> element on the existing page. I'm unaware of a good way to insert a full HTML document into a page, other than using an <iframe> (which I really didn't want to do).

That's a good point. You could just copy the content of the <body> into the content of the <section>, but link relations in the <head> (e.g. for CSS style sheets) would also have to be manually moved over, which could get messy.

Yes, exactly. Manually moving scripts and links out of the <head> seemed... unpleasant.

I had the same instinct to avoid using an iframe, but I can't think of any specific problems with that. Can you?

The biggest problem is the inability to access the global gateway APIs that we expose in the UI, and especially the JWT.

Beyond that, I would say that as we add more extensions points, things may get really tricky if everything is isolated in an <iframe>. For example, what if we want to have an extension that draws new Web Components? Or what if we want to handle events from the WebSocket -- should each iframe open its own WebSocket?

I'm unsure what the background page is actually used for in a WebExtension. MDN's example just shows it loading scripts, not having content.

A better example is probably a sidebar_action. Here is an example browser extension which demonstrates a sidebar provided as an HTML document (it also demonstrates the use of the context menu API which is interesting).

That makes sense. But in this case, again, the extension gets an entire document to draw to.

What if we took a similar approach to sidebars where a custom section is defined in the manifest including its icon, title and HTML content:

manifest.json

  "gateway_section": {
    "icon": "images/my_section_icon.png",
    "title" : "My Section",
    "content": "my_section.html"
  }

I'm not necessarily opposed to the icon/title approach here (content discussed below). My biggest concern was localization, but I see that WebExtensions provide a way to localize strings in the manifest.

HTML content for a custom section could be provided as an HTML file referenced in the manifest. The HTML file could then load its own CSS and JavaScript files, rather than using the content_scripts member of the manifest which in browser extensions is really intended for modifying web content from specific domains.

my_section.html

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <link rel="stylesheet" href="css/foo.css" />
    <script src="js/bar.js"></script>
  </head>
  <body>
    <h1>My Section</h1>
  </body>
</html>

One problem I can think of with using an iframe to embed this content is that no CSS or JavaScript from the wider gateway application would be loaded into the page by default. This might make it tricky to provide APIs via global variables in the style of Web Extension APIs. The extension would have to include a script from the gateway itself.

Yes, this is exactly the issue with using the iframe. I really like the idea of having all of the extension APIs pre-loaded into the browser, rather than having each extension load the same JS file. For example, if you have 10 extensions, you're going to load extension.js 10 separate times.

[flatsiedatsie]

It's just my two cents, but being able to have a fully working html page, with css and js in it, sounds like it would make development a lot easier.

[mrstegeman]

@flatsiedatsie I don't disagree with you. How would you recommend we do that, though, without using an <iframe>? You can already use arbitrary CSS and JS, and you can inject HTML. The only thing missing is the ability to load an HTML file directly.

[mrstegeman]

@benfrancis Check this out and see what you think: WebThingsIO/example-extension#1

[benfrancis]

@benfrancis Check this out and see what you think: WebThingsIO/example-extension#1

Thank you. I understood what you meant, but the code illustrates it well.

Personally I was thinking of an extension add-on as more of a self-contained mini web app (HTML, CSS & JS) that gets rendered inside a (conceptual) container with certain APIs exposed to it via globals, rather than extending a built-in class from the gateway application and explicitly extending a section of the gateway application's DOM itself from JavaScript. That seemed like a cleaner separation.

But maybe I'm just taking the analogy with Firefox add-ons too literally. The reality is that we're extending an existing web app, not a browser.

So if you think this is a better approach, then it's not a hill I'm going to die on :)

[mrstegeman]

Yeah, I think it's fundamentally different. If you look at Voice Fill, which does draw content directly to web pages (Google, Yahoo, etc.), it does so through JS, not by loading HTML. I think something like that is a better analogy here.

[flatsiedatsie]

I wasn't advocating for an iframe option (although one advantage of that might be security? A rogue add-on could make less of a mess?).

Mostly just that loading in a html file would indeed be nice. Perhaps bake that in, as was suggested. I could imagine, for example, a separate development support tool that merges the html, css and js for quick previews.

I was also thinking that it could be nice to have the example add-on come with some example html built in. A basic container, some buttons. It would help developers get started quickly, and it would help new add-ons conform to the Gateway's style.

[mrstegeman]

The example-extension has some HTML and fetches it asynchronously. See here.