Plugin all the logic, refactor, broken tests, updated README.

Author: ntoll

README.md

@@ -61,10 +61,14 @@ MicroPython.
 
 ### What's in the files?
 
+* **README.md** - this file, containing project documentation.
 * **Makefile** - common tasks scripted into convenient targets. See above.
 * **hello.py** - a simple "hello world" Python script for PyScript to run.
 * **index.html** - a small web page that uses PyScript.
 * **pyscript.js** - the simple, single file implementation of PyScript.
+* **SpecRunner.html** - a web page to run the test specifications with Jasmine.
+* **spec** - a directory containing the test specifications for Jasmine.
+* **lib** - a directory containing the Jasmine test library.
 
 To change the configuration of PyScript take a look at the JSON object
 defined in the `<py-config>` tag in `index.html`. Currently valid runtimes are
@@ -82,131 +86,167 @@ and run the Jasmine based test suite.
 
 ## How it works
 
+The PyScript core only loads configuration, starts the Python runtime and
+allows the registration of plugins. All other logic, capabilities and features
+are contained in the plugins.
+
+Currently, only a single plugin is provided by PyScript: the one that
+implements the core `<py-script>` tag.
+
+The story of PyScript's execution is roughly as follows:
+
+1. Configuration is loaded from the `<py-config>` tag. Once complete the
+   `py-configured` event is dispatched, containing the `config` object based
+   upon default values overridden by the content of the `<py-config>` tag.
+2. When the `py-configured` event is dispatched two things happen:
+   * The runtime is loaded via injecting a `<script>` tag that references the
+     runtime's URL. Once loaded the `py-runtime-loaded` event is dispatched.
+   * Plugins are registered and have their `configure` function called. For each
+     plugin registered a `py-plugin-registered` event is dispatched, containing
+     the (potentially changed) `config`, and a reference to the newly registered
+     plugin.
+3. When `py-runtime-loaded` is dispatched the `config` is frozen and two things
+   happen:
+   * The runtime is instantiated / started. Once complete the `py-runtime-ready`
+     event is dispatched.
+   * All registered plugins have their `start` function called. 
+4. When the `py-runtime-ready` event is dispatched all plugins have their
+   `onRuntimeReady` function called with the `config` and `runtime` objects.
+5. Any plugins registered after the runtime is ready immediately have their
+   `configure`, `start` and `onRuntimeReady` functions called, with the
+   `py-plugin-registered` event being dispatched.
+
+That's it!
+
+When `pyscript.js` is run, it creates a `window.PyScript` object that contains
+read-only references to the `config`, registered `plugins`,
+`availableRuntimes`, the `runtime` used on the page, an `isRuntimeReady` flag,
+a `registerPlugin` function (see below) and a `runPython(code)` function that
+takes a string of Python.
+
 There are copious comments in the `pyscript.js` file. My intention is for
 simplicity, lack of onerous dependencies (bye-bye `npm`), and
-understandability. This code is working if it's easy to understand what's going
+understandability. This code is good if it's easy to understand what's going
 on. To this end, it's laid out in a "literate" manner, so the code "tells the
 story" of this implementation of PyScript by reading it from top to bottom.
 
-In terms of architecture, this version of PyScript aims to provide a very
-small core that coordinates features and capabilities provided by plugins. This
-coordination is almost always handled by dispatching custom events to the
-`document`. Everything is losely coupled, and state is contained within the
-function/closure that is the `main` function (called right at the end). The
-`main` function returns an object containing methods to interact with
-PyScript (useful for testing purposes).
-
-### Base classes and constants
+## Plugins
 
-The `Plugin` class is based upon Antonio's suggestion
+Plugins are inspired by Antonio's suggestion
 [found here](https://github.com/pyscript/pyscript/issues/763#issuecomment-1245086859),
 and should be relatively self explanatory.
 
-The only difference between this implementation and Antonio's is that his
-version has `before_evalute` and `after_evaluate` methods. These are redundant
-in this version of PyScript since it's not known ahead of time when either the
-scripts nor runtime will be ready for evaluation. As far as I can tell, I think
-these functions are supposed to be run either before or after ALL the scripts
-are evaluated (at once) rather than before or after each individual script.
+Since simplicity is the focus, plugins are simply JavaScript objects.
+
+Such objects are expected they have a `name` attribute referencing a string
+naming the plugin (useful for logging purposes). Plugins should also provide
+one or more of the following functions attached to them, called in the
+following order (as the lifecycle of the plugin):
+
+* `configure(config)` - Gives the plugin early access to the `config` object.
+  Potentially, the plugin can modify it, and modifications will be visible to
+  later steps and other plugins. Plugins must only modify the config at this
+  point in their life-cycle. Examples of things which plugins might want to do
+  at this point:
+  - Early sanity check about their own options.
+  - Rename/remap some options.
+  - Add new packages to install.
+  - Modify options for other plugins (e.g. a debugger plugin might set the
+    option `show_terminal`).
+* `start(config)` - The main entry point for plugins. At this point, config
+  should not be modified by the plugin. Example use cases:
+  - Define custom HTML elements.
+  - Start fetching external resources.
+* `onRuntimeReady(config, runtime)` - Called once the runtime is ready to
+  evaluate Python code. Example use cases:
+  - `pip install` packages.
+  - Import/initialize Python plugins.
+
+The following events, dispatched by PyScript itself, are related to plugins:
+
+* `py-plugin-registered` - Dispatched when a plugin is registered (and the
+  event contains a reference to the newly registered plugin). This happens
+  immediately after the plugin's `configure` function is called.
+* `py-plugin-started` - Dispatched immediately after a plugin's `start`
+  function is called. The event contains a reference to the started plugin.
+* `py-runtime-ready` - causes each plugin's `onRuntimeReady` function to be
+  called.
+
+If a plugin is registered *after* the runtime is ready, all three functions are
+immediately called in the expected sequence, one after the other.
+
+The recommended way to create and register plugins is:
+
+```JavaScript
+const myPlugin = function(e) {
+    /*
+    Private and internal logic, event handlers and event dispatch can happen
+    within the closure defined by this function.
+
+    e.g.
+
+    const FOO = "bar";
+
+    function foo() {
+        const myEvent = new CustomEvent("my-event", {detail: {"foo": FOO}});
+        document.dispatchEvent(myEvent);
+    }
+
+    function onFoo(e) {
+        console.log(e.detail);
+    }
+
+    document.addEventListener("my-event", onFoo);
+
+    ...
+    */
+
+    const plugin = {
+        configure: function(config) {
+            // ...
+        },
+        start: function(config) {
+            // ...
+            foo();
+        },
+        onRuntimeReady: function(config, runtime) {
+            // ...
+        }
+    };
+    window.pyScript.registerPlugin(plugin);
+}();
+
+document.addEventListener("py-configured", myPlugin);
+```
 
-This probably needs more thought/discussion.
+Then in your HTML file:
 
-The `Runtime` class abstracts away all the implementation details of the
-various Python runtimes we might use. To see a complete implementation see the
-`MicroPythonRuntime` class that inherits from `Runtime`. There is also an
-incomplete `PyodideRuntime` class so I was able to compare and contrast the
-differences between implementations and arrive at a general abstraction (still
-very much a work in progress). Again, the comments in the code should explain
-what's going on in terms of the life-cycle and capabilities of a "runtime".
+```html
+<script src="myplugin.js"></script>
+<script src="pyscript.js" type="module"></script>
+```
+
+A good example of a plugin is the built-in plugin for the `<py-script>` tag
+found in `pyscript.js` (search for pyScriptTag).
 
-Finally, the `defaultSplash` is the `innerHtml` added to / removed from the DOM
-to indicate PyScript is starting up. It currently overlays an opaque DIV with
-the centred words "Loading PyScript...". This can be overridden by adding a
-`splash` entry to the JSON configuration in the `<py-config>` tag.
+## Runtimes
 
-### Built-in plugins and runtimes
+The `Runtime` class abstracts away all the implementation details of the
+various Python runtimes we might use.
 
-Currently, only one plugin is currently defined to handle the `<py-script>`
-tag: `PyScriptTag`. This is a rather simple plugin which ultimately dispatches
-the `py-script-registered` custom event (containing relevant metadata) to
-indicate Python source code has been found in the page (more on this later).
+To see a complete implementation see the `MicroPythonRuntime` class that
+inherits from `Runtime`. There is also an incomplete `PyodideRuntime` class so
+I was able to compare and contrast the differences between implementations and
+arrive at a general abstraction (still very much a work in progress). Comments
+in the code should explain what's going on in terms of the life-cycle and
+capabilities of a "runtime".
 
 The afore mentioned `MicroPythonRuntime`, `CPythonRuntime` and `PyodideRuntime`
 all, to a greater or lesser extent, define a uniform shim around their
 respective runtimes. The MicroPython one is most complete, but still needs work
 as I make changes to how MicroPython itself exposes `stdout`, `stderr` and
 consumes `stdin`.
 
-### The core PyScript app definition
-
-This is simply a `main` function / closure, in which is stored lots of private
-state and definitions that we don't want bleeding out into the
-external-to-PyScript context.
-
-The function starts with a definition of a very simple logger that pre-pends
-"🐍" to all PyScript related `console.log` messages, for ease of reading.
-
-Next comes some declarations and initial states for various objects used to
-store state and coordinate the activity of PyScript. Because they only exist
-within the context of the closure, they're effectively private to the outside
-world. The comments and their names should indicate their function and how they
-relate to each other.
-
-Next comes the definition of an `app` object. This is what is eventually
-returned by the `main` function (for testing purposes). The object contains
-various functions that manage the state and coordinate the activity of
-PyScript. Again, the function names and their associated commentary should
-describe what the intention is for each one. To be honest, they're all really
-very serious, with the most complicated being due to conditional paths
-depending on the state of the runtime.
-
-As each function finishes its task, if required, it signals a change in state
-through dispatching custom events via the `document` object.
-
-Underneath the `app` object are defined some event handler functions that
-"plumb together" the various capabilities defined in the `app`'s functions. How
-these relate to each other is described below.
-
-Finally, depending on a `window.pyscriptTest` flag (set to `true` in a testing
-context), the event handlers are registered against the relevant events and
-the `loadConfig` function is called to boot up the whole thing.
-
-The story of the PyScript app, roughly unfolds like this:
-
-1. The `main` function is called, with the resulting `app` object bound to
-   `window.pyscriptApp`.
-2. Calling `main` also causes PyScript to load any user configuration
-   (currently, for simplicity's sake, expressed as JSON). When this is finished
-   the `py-configured` event is fired.
-3. Next, built-in plugins are registered, after which the (internal) `config`
-   object is frozen (i.e. can't change).
-4. The default `<py-script>` tag dispatches a `py-script-registered` event
-   when a Python script is found.
-5. If the Python script's code is inline (i.e. a part of the document already)
-   then a `py-script-loaded` event is dispatched for that script. However, if
-   the Python script is referenced via a `src` URL, then PyScript fetches the
-   remote asset and only dispatches `py-script-loaded` for the script when the
-   code is retrieved.
-6. If the runtime is ready, each newly registered script is immediately
-   evaluated by dispatching the `py-eval-script`. Otherwise, the scripts are
-   added to a `pendingScripts` array for later processing when the runtime has
-   finally started.
-7. In the meantime, the runtime specified in the `config` is loaded into the
-   browser by injecting a `script` tag into the `head` of the document. When
-   this script has finished loading a `py-runtime-loaded` event is dispatched.
-8. The `Runtime` subclass instance, representing the loaded runtime, then has
-   its `start` method called. Upon completion of starting up the runtime, the
-   `py-runtime-ready` event is dispatched and the `runtimeReady` flag is set
-   to true.
-9. At this point, any scripts in the `pendingScripts` array are evaluated in
-   order, after which the array is cleared and discarded.
-
-That's it. You can see this unfolding in the image below, taken from the
-console logs for the example `hello.py` based application found in
-`index.html`.
-
-![PyScript logs](https://raw.githubusercontent.com/ntoll/micropyscript/main/pyscript-log.png "PyScript logs")
-
 ## The future
 
 Who knows..? But this is a good scaffold for testing different Python runtimes.

customtags.js

@@ -20,7 +20,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 ******************************************************************************/
 
-class PyREPLTag extends Plugin {
+const pyReplTag = function(e) {
     /*
     Adds a REPL to the DOM. The REPL session is only initialised when the
     runtime is ready. The content of the REPL is inserted in the following
@@ -33,22 +33,35 @@ class PyREPLTag extends Plugin {
     arrangement of tags will make it look like a TTY session). Bespoke CSS
     should use the pyscriptREPL class to attach styling.
     */
-    start(config) {
-        // Define the py-repl element.
-        class PyREPL extends HTMLElement {
-            connectedCallback() {
-                /*
-                Create a shadow DOM with the expected child elements and event
-                handlers defined in it.
-                */
-                const shadow = this.attachShadow({ mode: "open" });
-                const pre = document.createElement("pre");
-                pre.setAttribute("class", "pyscriptREPL");
-                const code = document.createElement("code");
-                pre.appendChild(code);
-                shadow.appendChild(pre);
+
+    const plugin = {
+        configure: function(config) {
+            // Just set a flag to indicate that a REPL is active.
+            config.repl = true
+        },
+        start: function(config) {
+            // Define the py-repl element.
+            class PyREPL extends HTMLElement {
+                connectedCallback() {
+                    /*
+                    Create a shadow DOM with the expected child elements and
+                    event handlers defined in it.
+                    */
+                    const shadow = this.attachShadow({ mode: "open" });
+                    const pre = document.createElement("pre");
+                    pre.setAttribute("class", "pyscriptREPL");
+                    const code = document.createElement("code");
+                    pre.appendChild(code);
+                    shadow.appendChild(pre);
+                }
             }
+            customElements.define("py-repl", PyREPL);
+        },
+        onRuntimeReady: function(config, runtime) {
+            // 
         }
-        customElements.define("py-repl", PyREPL);
-    }
-}
+    };
+
+    window.pyScript.registerPlugin(plugin);
+};
+document.addEventListener("py-configured", pyReplTag);