Rerun performance benchmarks

Author: zph

.pre-commit-config.yaml

@@ -8,9 +8,6 @@ repos:
       additional_dependencies:
         - black==23.3.0 # Matches hook
 
-    # - id: jupytext
-    #   args: [--sync]
-
 -   repo: https://github.com/psf/black
     rev: 23.3.0
     hooks:

NOTES.md

@@ -1,21 +1,20 @@
 # TODO
 
 ## Triage
-- [ ] Consider how to set off the 'proceed' output as a dedicated code block in jupyter
-- [ ] Read `runbook plan ...` support reading params from file
+- [x] Read `runbook plan ...` support reading params from file
 - [ ] Re-export confirm(style) and gather from sh lib
 - [ ] Make sure we can run directly fully through cli
 - [ ] If no argument included for RUNBOOK TITLE in edit/plan/run then prompt with options
-- [ ] Include field 'embeds' in the metadata to help with referencing them
+- [x] Include field 'embeds' in the metadata to help with referencing them
 - [ ] --- Add helper for referencing the embeds?
 - [ ] Use execute and upload output to S3 for non-interactive: https://github.com/nteract/papermill/tree/main?tab=readme-ov-file#execute-via-cli
 
 ## P0
+- [ ] Install pre-commit.yml or git integration during `init` for secure linting and talisman
 - [x] Setup versioning and bumper (using versioner from npm ecosystem @release-it and @release-it/bumper)
 - [x] Fix tag key getting stripped out of planned runbooks b/c it breaks papermill
-- [ ] Setup a watcher for auto-exporting html or other formats
 - [x] Setup tagging in the notebooks to auto-set those values
-- [ ] Setup git to autoclear cell outputs for a given folder's notebooks (ie templates) https://stackoverflow.com/a/58004619
+- [ ] Security: Setup git to autoclear cell outputs for a given folder's notebooks (ie templates) https://stackoverflow.com/a/58004619
 - [x] Assess if we need shell completions
    - [x] Yes, very nice and use https://click.palletsprojects.com/en/8.1.x/shell-completion/#custom-type-completion to define for custom types, ie only find ipynb files for edit, create, etc
 - [ ] Integrate mechanisms to read Grafana or other data sources and package "pre/post check" helpers
@@ -25,8 +24,8 @@
 - [ ] Include safe way to retry when APIs fail but with confirmation
     - for handling the case when infrastructure API calls fail when made in large async batch
 - [ ] Immutably store everything with bookstore https://github.com/nteract/bookstore
+- [ ] Setup a watcher for auto-exporting html or other formats
 - [x] Use slack notify for posting execution steps: https://github.com/keitakurita/jupyter-slack-notify
-- [ ] Install pre-commit.yml or git integration during `init`
 
 ## P2
 - [x] Setup different folder from runbooks for execution files?

PERFORMANCE.md

@@ -1,3 +1,3 @@
 | Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
 |:---|---:|---:|---:|---:|
-| `runbook` | 373.6 ± 6.5 | 367.8 | 389.6 | 1.00 |
+| `runbook` | 390.5 ± 3.8 | 383.6 | 395.0 | 1.00 |

runbook/cli/commands/create.py

@@ -28,9 +28,7 @@
 @click.pass_context
 def create(ctx, filename, template, language):
     """Create a new runbook from [template]"""
-    # TODO: guard against anything other than a bare name
-    # Allow override for alternate language default
-    # TODO: allow reading from project config file
+
     if language:
         template = language
     if path.basename(filename) != filename:

runbook/cli/commands/plan.py

@@ -118,9 +118,6 @@ def plan(ctx, input, embed, identifier="", params={}):
     if not Path(output_folder).exists():
         os.makedirs(output_folder, exist_ok=True)
 
-    # TODO: safety check if we already have one with this SHA in folder, in which case prompt
-    # and offer to skip.
-
     pm.execute_notebook(
         input_path=input,
         output_path=full_output,
@@ -133,8 +130,6 @@ def plan(ctx, input, embed, identifier="", params={}):
         full_output,
     ]
 
-    # TODO: join the unified logic of create and plan
-
     nbconvert_launch_instance(argv, clear_output=True)
 
     for f in embed:

runbook/cli/lib.py

@@ -2,7 +2,6 @@
 import sys
 from io import StringIO
 
-from nbconvert.nbconvertapp import NbConvertApp
 from ulid import ULID
 
 
@@ -21,6 +20,9 @@ def ts_id(length=10):
 
 # Suppresses nbconvert output
 def nbconvert_launch_instance(argv, clear_output=True):
+    # Imported here to avoid performance hit of importing it
+    from nbconvert.nbconvertapp import NbConvertApp
+
     if clear_output:
         argv.insert(0, "--ClearOutputPreprocessor.enabled=True")
     stdout = sys.stdout