More schema cleanup (#20)

deathbeds · Dec 11, 2023 · b5ccc83 · b5ccc83
1 parent 731b23b
commit b5ccc83
Show file tree

Hide file tree

Showing 10 changed files with 210 additions and 54 deletions.
diff --git a/.github/workflows/self.yml b/.github/workflows/self.yml
@@ -15,6 +15,7 @@ concurrency:
 
 env:
   JPYK_ALLOW_NO_CONFIG: 1
+  CACHE_EPOCH: 0
 
 jobs:
   self:
@@ -31,6 +32,12 @@ jobs:
           miniforge-variant: Mambaforge
           use-mamba: true
       - run: doit bootstrap
+      - uses: actions/cache@v3
+        id: cache-node-modules
+        with:
+          path: '**/node_modules'
+          key: |
+            ${{ env.CACHE_EPOCH }}-node_modules-${{ hashFiles('yarn.lock', '.github/conda-linux-64.lock') }}
       - run: doit self:js:*
       - run: doit self:schema:*
       - run: doit self:build:*

diff --git a/README.md b/README.md
@@ -1,14 +1,22 @@
+<!-- this can be viewed as a slideshow in jupyterlab deck -->
+
+---
+
 # jupyak
 
 > get a statically-hosted JupyterLite preview of pull requests from across the Jupyter
 > stack
 
+---
+
 ## what does it do?
 
 This repo accepts [pull requests](#how-does-it-work) that build static
 [preview sites](#how-do-i-get-a-preview-site) to the
 [Jupyter(Lite) stack](#what-can-it-build).
 
+---
+
 ## how do I get a preview site?
 
 - Make a `jupyak_config.toml`
@@ -18,6 +26,8 @@ This repo accepts [pull requests](#how-does-it-work) that build static
 - Wait for the PR to build a site on ReadTheDocs (RTD)
 - Make changes to the PR to trigger re-building the preview site
 
+---
+
 ## where does it do it?
 
 - `https://github.com/deathbeds/jupyak`
@@ -38,6 +48,8 @@ This repo accepts [pull requests](#how-does-it-work) that build static
 > taken to make _anything_ run on any other system, and likely won't be tested anywhere
 > else.
 
+---
+
 ## how does it work?
 
 Delivering a preview site uses a few different GitHub Actions workflows and ReadTheDocs
@@ -53,6 +65,8 @@ configurations. Once a PR is created:
     built RTD site will contain human-readable error logs
 - Additional commits on the PR can trigger a rebuild of the site
 
+---
+
 ## what can it build?
 
 `jupyak` understand how to work with one or more fragments of human-readable GitHub URLs
@@ -65,6 +79,8 @@ of the forms:
 
 ...from the following repos to see novel behavior.
 
+---
+
 | repo                                              | browser UI | kernel |
 | ------------------------------------------------- | :--------: | :----: |
 | https://github.com/ipython/comm                   |            |   ✅   |
@@ -80,10 +96,16 @@ of the forms:
 | https://github.com/jupyterlite/jupyterlite        |     ✅     |        |
 | https://github.com/jupyterlite/pyodide-kernel     |     ✅     |   ✅   |
 
+---
+
+### what about configuring JupyterLite?
+
 Additionally, a single GitHub [Gist][gist] can be used as the contents of the preview
 site. If the gist contains `jupyter_lite_config.json` and/or `jupyter-lite.json`, these
 will be merged into the [configuration][lite-config] used to build and host the site.
 
+---
+
 ## what does it _not_ build?
 
 The following repos are cloned and installed to satisfy build-time requirements, but
@@ -98,6 +120,10 @@ have little measurable experience the UI.
 | https://github.com/jupyterlab/jupyterlab_server            | heavy deps                     |
 | https://github.com/jupyter/nbconvert                       | heavy deps, no `data_files`    |
 
+---
+
+### what else is it not building?
+
 For each repo, a few more things _could_ be built, and might be interesting to evaluate
 a PR stack. However, each of these would come at the expense of a lower chance of a
 usable JupyterLite site at the end of each PR build. These include:

diff --git a/docs/_static/jupyak-v0.schema.json b/docs/_static/jupyak-v0.schema.json
@@ -1082,7 +1082,11 @@
         "pypi_to_conda": {
           "type": "object",
           "title": "pypi_to_conda",
-          "description": "names of PyPI distributions to replace with their conda-forge counterparts"
+          "description": "names of PyPI distributions to replace with their conda-forge counterparts",
+          "additionalProperties": {
+            "type": "string",
+            "description": "a normalized conda-forge name"
+          }
         },
         "python_version": {
           "type": "string",
@@ -1223,6 +1227,28 @@
         }
       }
     },
+    "LabExtension": {
+      "title": "LabExtension",
+      "description": "A description of the inputs and outputs of \"jupyter labextension build\"",
+      "properties": {
+        "file_dep": {
+          "type": "array",
+          "title": "file_dep",
+          "description": "paths or globs to files that would trigger a need to rebuild",
+          "items": {
+            "type": "string"
+          }
+        },
+        "targets": {
+          "type": "array",
+          "title": "targets",
+          "description": "canary files that will be generated (usually a package.json)",
+          "items": {
+            "type": "string"
+          }
+        }
+      }
+    },
     "LiteOptions": {
       "title": "LiteOptions",
       "description": "options for the final jupyterlite build",
@@ -1257,7 +1283,10 @@
         "lab_extensions": {
           "type": "object",
           "title": "lab_extensions",
-          "description": "paths with extra file dependencies needed to build an extension in this repo"
+          "description": "paths with extra file dependencies needed to build an extension in this repo",
+          "additionalProperties": {
+            "$ref": "#/$defs/LabExtension"
+          }
         },
         "modules": {
           "type": "array",
@@ -1347,7 +1376,14 @@
         "wheel_file_dep": {
           "type": "object",
           "title": "wheel_file_dep",
-          "description": "extra files needed to build a given wheel"
+          "description": "extra files needed to build a given wheel",
+          "additionalProperties": {
+            "type": "array",
+            "description": "globs to files that need to be built before the wheel can be built",
+            "items": {
+              "type": "string"
+            }
+          }
         }
       }
     },

diff --git a/docs/_static/jupyak-v0.schema.toml b/docs/_static/jupyak-v0.schema.toml
@@ -1067,6 +1067,10 @@ type = "object"
 title = "pypi_to_conda"
 description = "names of PyPI distributions to replace with their conda-forge counterparts"
 
+["$defs".CondaEnv.properties.pypi_to_conda.additionalProperties]
+type = "string"
+description = "a normalized conda-forge name"
+
 ["$defs".CondaEnv.properties.python_version]
 type = "string"
 title = "python_version"
@@ -1192,6 +1196,26 @@ anyOf = [
   { "$ref" = "#/$defs/Task" },
 ]
 
+["$defs".LabExtension]
+title = "LabExtension"
+description = "A description of the inputs and outputs of \"jupyter labextension build\""
+
+["$defs".LabExtension.properties.file_dep]
+type = "array"
+title = "file_dep"
+description = "paths or globs to files that would trigger a need to rebuild"
+
+["$defs".LabExtension.properties.file_dep.items]
+type = "string"
+
+["$defs".LabExtension.properties.targets]
+type = "array"
+title = "targets"
+description = "canary files that will be generated (usually a package.json)"
+
+["$defs".LabExtension.properties.targets.items]
+type = "string"
+
 ["$defs".LiteOptions]
 title = "LiteOptions"
 description = "options for the final jupyterlite build"
@@ -1226,6 +1250,9 @@ type = "object"
 title = "lab_extensions"
 description = "paths with extra file dependencies needed to build an extension in this repo"
 
+["$defs".PythonOptions.properties.lab_extensions.additionalProperties]
+"$ref" = "#/$defs/LabExtension"
+
 ["$defs".PythonOptions.properties.modules]
 type = "array"
 title = "modules"
@@ -1312,6 +1339,13 @@ type = "object"
 title = "wheel_file_dep"
 description = "extra files needed to build a given wheel"
 
+["$defs".RepoLiteOptions.properties.wheel_file_dep.additionalProperties]
+type = "array"
+description = "globs to files that need to be built before the wheel can be built"
+
+["$defs".RepoLiteOptions.properties.wheel_file_dep.additionalProperties.items]
+type = "string"
+
 ["$defs".Task]
 title = "Task"
 description = "a subset of the doit task definition"

diff --git a/docs/_static/jupyak-v0.schema.yaml b/docs/_static/jupyak-v0.schema.yaml
@@ -804,6 +804,9 @@ $defs:
         title: pypi_to_conda
         description:
           names of PyPI distributions to replace with their conda-forge counterparts
+        additionalProperties:
+          type: string
+          description: a normalized conda-forge name
       python_version:
         type: string
         title: python_version
@@ -912,6 +915,22 @@ $defs:
             anyOf:
               - type: string
               - $ref: '#/$defs/Task'
+  LabExtension:
+    title: LabExtension
+    description: A description of the inputs and outputs of "jupyter labextension build"
+    properties:
+      file_dep:
+        type: array
+        title: file_dep
+        description: paths or globs to files that would trigger a need to rebuild
+        items:
+          type: string
+      targets:
+        type: array
+        title: targets
+        description: canary files that will be generated (usually a package.json)
+        items:
+          type: string
   LiteOptions:
     title: LiteOptions
     description: options for the final jupyterlite build
@@ -944,6 +963,8 @@ $defs:
         title: lab_extensions
         description:
           paths with extra file dependencies needed to build an extension in this repo
+        additionalProperties:
+          $ref: '#/$defs/LabExtension'
       modules:
         type: array
         title: modules
@@ -1014,6 +1035,12 @@ $defs:
         type: object
         title: wheel_file_dep
         description: extra files needed to build a given wheel
+        additionalProperties:
+          type: array
+          description:
+            globs to files that need to be built before the wheel can be built
+          items:
+            type: string
   Task:
     title: Task
     description: a subset of the doit task definition

diff --git a/docs/graph.md b/docs/graph.md
@@ -51,11 +51,9 @@ classDef git fill:#f1502f,color:#fff;
 classDef hack fill:#000,color:#fff;
 shave:js:jupyterlab:yarn:build:prod:dev_mode --> jupyterlab:sweep:assets
 shave:js:jupyterlab:yarn:builder:. --> jupyterlab:sweep:labextension
-shave:git:lite:gist:fetch:main --> shave:git:lite:gist:checkout
 shave:git:traitlets:fetch:main --> shave:git:traitlets:checkout
 shave:git:comm:fetch:main --> shave:git:comm:checkout
 shave:git:lumino:fetch:main --> shave:git:lumino:checkout
-shave:git:lumino:fetch:pr-655 --> shave:git:lumino:checkout
 shave:git:jupyter_server_terminals:fetch:main --> shave:git:jupyter_server_terminals:checkout
 shave:git:jupyter_core:fetch:main --> shave:git:jupyter_core:checkout
 shave:git:jupyter_events:fetch:main --> shave:git:jupyter_events:checkout
@@ -71,13 +69,6 @@ shave:git:notebook_shim:fetch:main --> shave:git:notebook_shim:checkout
 shave:git:jupyterlab_server:fetch:main --> shave:git:jupyterlab_server:checkout
 shave:git:jupyterlab_lsp:fetch:main --> shave:git:jupyterlab_lsp:checkout
 shave:git:jupyterlab:fetch:main --> shave:git:jupyterlab:checkout
-shave:git:jupyterlab:fetch:pr-14575 --> shave:git:jupyterlab:checkout
-shave:git:jupyterlab:fetch:pr-14597 --> shave:git:jupyterlab:checkout
-shave:git:jupyterlab:fetch:pr-14766 --> shave:git:jupyterlab:checkout
-shave:git:jupyterlab:fetch:pr-14854 --> shave:git:jupyterlab:checkout
-shave:git:jupyterlab:fetch:pr-15222 --> shave:git:jupyterlab:checkout
-shave:git:jupyterlab:fetch:pr-15347 --> shave:git:jupyterlab:checkout
-shave:git:jupyterlab:fetch:pr-15048 --> shave:git:jupyterlab:checkout
 shave:git:notebook:fetch:main --> shave:git:notebook:checkout
 shave:git:ipywidgets:fetch:main --> shave:git:ipywidgets:checkout
 shave:git:jupyterlite:fetch:main --> shave:git:jupyterlite:checkout
@@ -302,7 +293,6 @@ jupyterlab:sweep:labextension --> shave:py:jupyterlite_pyodide_kernel:labext:pyo
 shave:js:jupyterlite_pyodide_kernel:yarn:build:py:wheels:pyodide-kernel --> shave:py:jupyterlite_pyodide_kernel:labext:pyodide-kernel-extension
 shave:js:jupyterlite_pyodide_kernel:yarn:build:lib:pyodide-kernel-extension --> shave:py:jupyterlite_pyodide_kernel:labext:pyodide-kernel-extension
 shave:py:jupyterlab:pip:jupyterlab --> shave:py:jupyterlite_pyodide_kernel:labext:pyodide-kernel-extension
-shave:git:lite:gist:checkout --> shave:lite:config
 shave:py:traitlets:wheel:traitlets --> shave:lite:config
 shave:py:comm:wheel:comm --> shave:lite:config
 shave:py:jupyter_core:wheel:jupyter_core --> shave:lite:config
@@ -315,7 +305,6 @@ shave:py:ipywidgets:labext:jupyterlab_widgets --> shave:lite:config
 shave:py:jupyterlite:labext:jupyterlite-javascript-kernel --> shave:lite:config
 shave:py:jupyterlite_pyodide_kernel:labext:pyodide-kernel-extension --> shave:lite:config
 jupyterlab:sweep:assets --> shave:lite:build
-shave:git:lite:gist:checkout --> shave:lite:build
 shave:py:traitlets:wheel:traitlets --> shave:lite:build
 shave:py:comm:wheel:comm --> shave:lite:build
 shave:py:jupyter_core:wheel:jupyter_core --> shave:lite:build
@@ -334,14 +323,7 @@ subgraph jupyterlab
   jupyterlab:sweep:assets(("`✅ **assets**`")):::hack
   jupyterlab:sweep:labextension(("`✅ **labextension**`")):::hack
   shave:git:jupyterlab:fetch:main[("`✅ **fetch main**`")]:::git
-  shave:git:jupyterlab:fetch:pr-14575[("`✅ **fetch pr-14575**`")]:::git
-  shave:git:jupyterlab:fetch:pr-14597[("`✅ **fetch pr-14597**`")]:::git
-  shave:git:jupyterlab:fetch:pr-14766[("`✅ **fetch pr-14766**`")]:::git
-  shave:git:jupyterlab:fetch:pr-14854[("`✅ **fetch pr-14854**`")]:::git
-  shave:git:jupyterlab:fetch:pr-15222[("`✅ **fetch pr-15222**`")]:::git
-  shave:git:jupyterlab:fetch:pr-15347[("`✅ **fetch pr-15347**`")]:::git
-  shave:git:jupyterlab:fetch:pr-15048[("`✅ **fetch pr-15048**`")]:::git
-  shave:git:jupyterlab:checkout[("`✅ **checkout**`")]:::git
+  shave:git:jupyterlab:checkout[("`💭 **checkout**`")]:::git
   shave:js:jupyterlab:yarn:install{{"`✅ **yarn install**`"}}:::js
   shave:js:jupyterlab:yarn:buildutils:.{{"`✅ **yarn buildutils .**`"}}:::js
   shave:js:jupyterlab:yarn:builder:.{{"`✅ **yarn builder .**`"}}:::js
@@ -350,12 +332,6 @@ subgraph jupyterlab
   shave:js:jupyterlab:dist{{"`✅ **dist**`"}}:::js
   shave:py:jupyterlab:pip:jupyterlab["`✅ **pip jupyterlab**`"]:::py
 end
-subgraph lite
-  shave:git:lite:gist:fetch:main[("`✅ **gist fetch main**`")]:::git
-  shave:git:lite:gist:checkout[("`✅ **gist checkout**`")]:::git
-  shave:lite:config(["`✅ **config**`"]):::jupyter
-  shave:lite:build(["`✅ **build**`"]):::jupyter
-end
 subgraph traitlets
   shave:git:traitlets:fetch:main[("`✅ **fetch main**`")]:::git
   shave:git:traitlets:checkout[("`✅ **checkout**`")]:::git
@@ -370,8 +346,7 @@ subgraph comm
 end
 subgraph lumino
   shave:git:lumino:fetch:main[("`✅ **fetch main**`")]:::git
-  shave:git:lumino:fetch:pr-655[("`✅ **fetch pr-655**`")]:::git
-  shave:git:lumino:checkout[("`✅ **checkout**`")]:::git
+  shave:git:lumino:checkout[("`💭 **checkout**`")]:::git
   shave:js:lumino:yarn:install{{"`✅ **yarn install**`"}}:::js
   shave:js:lumino:yarn:build:.{{"`✅ **yarn build .**`"}}:::js
   shave:js:lumino:dist{{"`✅ **dist**`"}}:::js
@@ -507,5 +482,9 @@ end
 subgraph yarnrc
   shave:js:yarnrc{{"`✅ **yarnrc**`"}}:::js
 end
+subgraph lite
+  shave:lite:config(["`💭 **config**`"]):::jupyter
+  shave:lite:build(["`💭 **build**`"]):::jupyter
+end
 </div>
 </div>
diff --git a/src/jupyak/tasks/_lite.ipynb b/src/jupyak/tasks/_lite.ipynb
@@ -99,8 +99,8 @@
     "        cli_deps += [\n",
     "            yak.env.py_site_packages / P._pth_path(pth) for pth in repo.lite.needs_pth\n",
     "        ]\n",
-    "        for ext_js_path, ext_info in repo.py.lab_extensions.items():\n",
-    "            build_deps += [repo.work_path / t for t in ext_info[\"targets\"]]\n",
+    "        for ext_js_path, ext in repo.py.lab_extensions.items():\n",
+    "            build_deps += [repo.work_path / t for t in ext.targets]\n",
     "        if repo.lite.wheel:\n",
     "            for ppt in repo.py.pyproject_tomls:\n",
     "                ppt_path = repo.work_path / ppt\n",