v0.6.0 Beta version: New Interpreter and Compiler APIs

README.md

@@ -1,17 +1,20 @@
-# @alex.garcia/unofficial-observablehq-compiler [![CircleCI](https://circleci.com/gh/asg017/unofficial-observablehq-compiler.svg?style=svg)](https://circleci.com/gh/asg017/unofficial-observablehq-compiler)
+# unofficial-observablehq-compiler
 
-An unoffical compiler for Observable notebooks (glue between the Observable parser and runtime)
+An unoffical compiler _and_ interpreter for Observable notebooks (the glue between the Observable parser and runtime)
 
-This compiler will compile "observable syntax" into "javascript syntax".
-For example -
+This library has two parts: The Interpreter and the Compiler. The Interpreter will interpret "Observable syntax" into "javascript syntax" live in a javascript environment. For example:
 
 ```javascript
-import compiler from "@alex.garcia/unofficial-observablehq-compiler";
+import { Intepreter } from "@alex.garcia/unofficial-observablehq-compiler";
 import { Inspector, Runtime } from "@observablehq/runtime";
 
-const compile = new compiler.Compiler();
+async function main() {
+  const runtime = new Runtime();
+  const main = runtime.module();
+  const observer = Inspector.into(document.body);
+  const interpret = new Intepreter({ module: main, observer });
 
-compile.module(`
+  await interpret.module(`
 import {text} from '@jashkenas/inputs'
 
 viewof name = text({
@@ -21,240 +24,84 @@ viewof name = text({
 
 md\`Hello **\${name}**, it's nice to meet you!\`
 
-`).then(define => {
-  const runtime = new Runtime();
-
-  const module = runtime.module(define, Inpsector.into(document.body));
-});
+`);
+}
+main();
 ```
 
+The Compiler will compile "Observable syntax" into javascript source code (as an ES module).
+
 For more live examples and functionality, take a look at the [announcement notebook](https://observablehq.com/d/74f872c4fde62e35)
 and this [test page](https://github.com/asg017/unofficial-observablehq-compiler/blob/master/test/test.html).
 
 ## API Reference
 
-### Compiler
-
-<a href="#Compiler" name="Compiler">#</a> new <b>Compiler</b>(<i>resolve</i> = defaultResolver, <i>fileAttachmentsResolve</i> = name => name, <i>resolvePath</i> = defaultResolvePath) [<>](https://github.com/asg017/unofficial-observablehq-compiler/blob/master/src/compiler.js#L119 "Source")
-
-Returns a new compiler. `resolve` is an optional function that, given a `path`
-string, will resolve a new define function for a new module. This is used when
-the compiler comes across an import statement - for example:
-
-```javascript
-import {chart} from "@d3/bar-chart"
-```
-
-In this case, `resolve` gets called with `path="@d3/bar-chart"`. The `defaultResolver`
-function will lookup the given path on observablehq.com and return the define
-function to define that notebook.
-
-For example, if you have your own set of notebooks on some other server, you
-could use something like:
-
-```javascript
-const resolve = path =>
-  import(`other.server.com/notebooks/${path}.js`).then(
-    module => module.default
-  );
-
-const compile = new Compiler(resolve);
-```
-
-`fileAttachmentsResolve` is an optional function from strings to URLs which is used as a <i>resolve</i> function in the standard library's <a href="https://github.com/observablehq/stdlib#FileAttachments">FileAttachments</a> function. For example, if you wanted to reference `example.com/my_file.png` in a cell which reads:
-
-```javascript
-await FileAttachment("my_file.png").url();
-```
-
-Then you could compile this cell with:
-
-```javascript
-const fileAttachmentsResolve = name => `example.com/${name}`;
-
-const compile = new Compiler(, fileAttachmentsResolve);
-```
-
-By default, `fileAtachmentsResolve` simply returns the same string, so you would have to use valid absolute or relative URLs in your `FileAttachment`s.
-
-`resolvePath` is an optional function from strings to URLs which is used to turn the strings in `import` cells to URLs in [`compile.moduleToESModule`](#compile_moduleToESModule) and  [`compile.notebookToESModule`](#compile_notebookToESModule). For instance, if those functions encounter this cell:
-```javascript
-import {chart} from "@d3/bar-chart"
-```
-then `resolvePath` is called with `path="@d3/bar-chart"` and the resulting URL is included in the static `import` statements at the beginning of the generated ES module source.
-
-<a href="#compile_module" name="compile_module">#</a>compile.<b>module</b>(<i>contents</i>)
-
-Returns a define function. `contents` is a string that defines a "module", which
-is a list of "cells" (both defintions from [@observablehq/parser](https://github.com/observablehq/parser)).
-It must be compatible with [`parseModule`](https://github.com/observablehq/parser#parseModule). This fetches all imports so it is asynchronous.
-
-For example:
-
-```javascript
-const define = await compile.module(`a = 1
-b = 2
-c = a + b`);
-```
-
-You can now use `define` with the Observable [runtime](https://github.com/observablehq/runtime):
-
-```javascript
-const runtime = new Runtime();
-const main = runtime.module(define, Inspector.into(document.body));
-```
-
-<a href="#compile_notebook" name="compile_notebook">#</a>compile.<b>notebook</b>(<i>object</i>)
-
-Returns a define function. `object` is a "notebook JSON object" as used by the
-ObservableHQ notebook app to display notebooks. Such JSON files are served by
-the API endpoint at `https://api.observablehq.com/document/:slug` (see the
-[`observable-client`](https://github.com/mootari/observable-client) for a
-convenient way to authenticate and make requests).
-
-`compile.notebook` requires that `object` has a field named `"nodes"`
-consisting of an array of cell objects. Each of the cell objects must have a
-field `"value"` consisting of a string with the source code for that cell.
+### new Interpreter(_params_)
 
-The notebook JSON objects also ordinarily contain some other metadata fields,
-e.g. `"id"`, `"slug"`, `"owner"`, etc. which are currently ignored by the
-compiler. Similarly, the cell objects in `"nodes"` ordinarily contain `"id"` and
-`"pinned"` fields which are also unused here.
+`Interpreter` is a class that encompasses all logic to interpret Observable js code. _params_ is an optional object with the following allowed configuration:
 
-This fetches all imports so it is asynchronous.
+`resolveImportPath(path, specifers)`: An async function that resolves to the `define` definition for a notebook. _path_ is the string provided in the import statement, and _specifiers_ is a array of string if the imported cell name specifiers in the statement (cells inside `import {...}`). _specifiers_ is useful when implementing tree-shaking. For example, `import {chart as CHART} from "@d3/bar-chart"` would supply `path="@d3/bar-chart"` and `specifiers=["chart"]`. Default imports from observablehq.com, eg `https://api.observablehq.com/@d3/bar-chart.js?v=3`
+`resolveFileAttachments`: A function that, given the name of a FileAttachment, returns the URL that the FileAttachment will be fetched from. Defaults to `name => name`.
+`defineImportMarkdown`: A boolean, whether to define a markdown description cell for imports in the notebook. Defaults true.
+`observeViewofValues`: A boolean, whether to pass in an _observer_ for the value of viewof cells.
+`module`: A default Observable runtime [module](https://github.com/observablehq/runtime#modules) that will be used in operations such as `.cell` and `.module`, if no other module is passed in.
+`observer`: A default Observable runtime [observer](https://github.com/observablehq/runtime#observer) that will be used in operations such as `.cell` and `.module`, if no other observer is passed in.
 
-For example:
+Keep in mind, there is no sandboxing done, so it has the same security implications as `eval()`.
 
-```javascript
-const define = await compile.notebook({
-  nodes: [{ value: "a = 1" }, { value: "b = 2" }, { value: "c = a + b" }]
-});
-```
+interpret.**cell**(_source_ [, *module*, *observer*])
 
-You can now use `define` with the Observable [runtime](https://github.com/observablehq/runtime):
+Parses the given string _source_ with the Observable parser [`.parseCell()`](https://github.com/observablehq/parser#parseCell) and interprets the source, passing it and the _observer_ along to the Observable _module_. Returns a Promise that resolves to an array of runtime [variables](https://github.com/observablehq/runtime#variables) that were defined when interpreting the source. More than one variable can be defined with a single cell, like with `viewof`, `mutable`, and `import` cells. _source_ can also be a pre-parsed [cell](https://github.com/observablehq/parser#cell) instead of source code.
 
-```javascript
-const runtime = new Runtime();
-const main = runtime.module(define, Inspector.into(document.body));
-```
+interpret.**module**(_source_ [, *module*, *observer*])
 
-<a href="#compile_cell" name="compile_cell">#</a>compile.<b>cell</b>(<i>contents</i>)
+Parses the given string _source_ with the Observable parser [`.parseModule()`](https://github.com/observablehq/parser#parseModule) and interprets the source, passing it and the _observer_ along to the Observable _module_. Returns a Promise that resolves to an array of an array of runtime [variables](https://github.com/observablehq/runtime#variables) that were defined when interpreting the source. _source_ can also be a pre-parsed [program](https://github.com/observablehq/parser#program) instead of source code.
 
-Returns an object that has `define` and `redefine` functions that would define or redefine variables in the given cell to a specified module. `contents` is input for the [`parseCell`](https://github.com/observablehq/parser#parseCell) function. If the cell is not an ImportDeclaration, then the `redefine` functions can be used to redefine previously existing variables in a module. This is an asynchronous function because if the cell is an import, the imported notebook is fetched.
+interpret.**notebook**(_source_ [, *module*, *observer*])
 
-```javascript
-let define, redefine;
+TODO
 
-define = await compile.module(`a = 1;
-b = 2;
+new **Compiler**(_params_)
 
-c = a + b`);
+`Compiler` is a class that encompasses all logic to compile Observable javascript code into vanilla Javascript code, as an ES module. _params_ is an optional object with the following allowed configuration.
 
-const runtime = new Runtime();
-const main = runtime.module(define, Inspector.into(document.body));
+`resolveImportPath(path, specifers)`: A function that returns a URL to where the notebook is defined. _path_ is the string provided in the import statement, and _specifiers_ is a array of string if the imported cell name specifiers in the statement (cells inside `import {...}`). _specifiers_ is useful when implementing tree-shaking. For example, `import {chart as CHART} from "@d3/bar-chart"` would supply `path="@d3/bar-chart"` and `specifiers=["chart"]`. Default imports from observablehq.com, eg `https://api.observablehq.com/@d3/bar-chart.js?v=3`
 
-await main.value("a") // 1
+`resolveFileAttachments`: A function that, given the name of a FileAttachment, returns the URL that the FileAttachment will be fetched from. Defaults to `name => name`.
 
-{define, redefine} = await compile.cell(`a = 20`);
-
-redefine(main);
-
-await main.value("a"); // 20
-await main.value("c"); // 22
-
-define(main); // would throw an error, since a is already defined in main
-
-{define} = await compile.cell(`x = 2`);
-define(main);
-{define} = await compile.cell(`y = x * 4`);
-define(main);
-
-await main.value("y") // 8
-
-```
-
-Keep in mind, if you want to use `define` from `compile.cell`, you'll have to provide an `observer` function, which will most likely be the same observer that was used when defining the module. For example:
+`UNSAFE_allowJavascriptFileAttachments` A boolean. When true, the `resolveFileAttachments` function will resolve to raw JavaScript when calculating the value of a FileAttachment reference. This is useful if you need to use `new URL` or `import.meta.url` when determining where a FileAttachment url should resolve too. This is unsafe because the Compiler will not escape any quotes when including it in the compiled output, so do use with extreme caution when dealing with user input.
 
 ```javascript
 
-let define, redefine;
-
-define = await compile.module(`a = 1;
-b = 2;`);
+// This can be unsafe since FileAttachment names can include quotes.
+// Instead, map file attachments names to something deterministic and escape-safe,
+// like SHA hashes.
+const resolveFileAttachments = name => `new URL("./files/${name}", import.meta.url)`
 
-const runtime = new Runtime();
-const observer = Inspector.into(document.body);
-const main = runtime.module(define, observer);
+Compiled output when:
 
-{define} = await compile.cell(`c = a + b`);
+// UNSAFE_allowJavascriptFileAttachments == false
+const fileAttachments = new Map([["a", "new URL(\"./files/a\", import.meta.url)"]]);
 
-define(main, observer);
+// UNSAFE_allowJavascriptFileAttachments == true
+const fileAttachments = new Map([["a", new URL("./files/a", import.meta.url)]]);
 
-```
-
-Since `redefine` is done on a module level, an observer is not required.
-
-<a href="#compile_moduleToESModule" name="compile_moduleToESModule">#</a>compile.<b>moduleToESModule</b>(<i>contents</i>)
-
-Returns a string containing the source code of an ES module. This ES module is compiled from the Observable runtime module in the string `contents`.
-
-For example:
 
-```javascript
-const src = compile.moduleToESModule(`a = 1
-b = 2
-c = a + b`);
 ```
 
-Now `src` contains the following:
+`defineImportMarkdown` - A boolean, whether to define a markdown description cell for imports in the notebook. Defaults true.
 
-```javascript
-export default function define(runtime, observer) {
-  const main = runtime.module();
+`observeViewofValues` - A boolean, whether or not to pass in the `observer` function for viewof value cells. Defaults true.
 
-  main.variable(observer("a")).define("a", function(){return(
-1
-)});
-  main.variable(observer("b")).define("b", function(){return(
-2
-)});
-  main.variable(observer("c")).define("c", ["a","b"], function(a,b){return(
-a + b
-)});
-  return main;
-}
-```
+`observeMutableValues` - A boolean, whether or not to pass in the `observer` function for mutable value cells. Defaults true.
 
-<a href="#compile_notebookToESModule" name="compile_notebookToESModule">#</a>compile.<b>notebookToESModule</b>(<i>object</i>)
+compile.**module**(_source_)
 
-Returns a string containing the source code of an ES module. This ES module is compiled from the Observable runtime module in the notebok object `object`. (See [compile.notebook](#compile_notebook)).
+TODO
 
-For example:
+compile.**notebook**(_source_)
 
-```javascript
-const src = compile.notebookToESModule({
-  nodes: [{ value: "a = 1" }, { value: "b = 2" }, { value: "c = a + b" }]
-});
-```
-
-Now `src` contains the following:
-
-```javascript
-export default function define(runtime, observer) {
-  const main = runtime.module();
-
-  main.variable(observer("a")).define("a", function(){return(
-1
-)});
-  main.variable(observer("b")).define("b", function(){return(
-2
-)});
-  main.variable(observer("c")).define("c", ["a","b"], function(a,b){return(
-a + b
-)});
-  return main;
-}
-```
+TODO
 
 ## License
 

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@alex.garcia/unofficial-observablehq-compiler",
-  "version": "0.5.0",
+  "version": "0.6.0-alpha.9",
   "description": "An unofficial compiler to bind @observablehq/parser and @observablehq/runtime",
   "main": "dist/index.js",
   "author": "Alex Garcia <alexsebastian.garcia@gmail.com>",
@@ -33,7 +33,7 @@
     "compiler"
   ],
   "dependencies": {
-    "@observablehq/parser": "^3.0.0",
+    "@observablehq/parser": "4.2",
     "acorn-walk": "^7.0.0"
   },
   "devDependencies": {

rollup.config.js

@@ -8,7 +8,7 @@ export default {
       compact: true,
       file: "dist/index.js",
       format: "umd",
-      name: "index.js"
+      name: "unofficial_observablehq_compiler"
     },
     {
       compact: true,