[SKiDL Feature Request] - Splitting Source from Tooling Commands

[mawildoer]

Hey Dave,

Been playing with Skidl a little recently and hoping to potentially extend the ecosystem a little.

Is your feature request related to a problem? Please describe.

We've recently been working towards building a new visualiser for Skidl circuits and we'd like to provide a update-as-you-type feature and a much more interactive rendering of the circuit. Having a seperate compiler and script would help allow this by letting the compiler (or more likely an application importing it) free-run while watching the circuits source files.

Additionally, we've been working on some CI with Skidl circuits and would like to be able to invoke the "compiler" with some set parameters (such as output names and locations) separately from any individual circuits definition.

Describe the solution you'd like

I was hoping to use this ticket to discuss using Skidl as more of a "compiler", breaking about the source code for the circuit and the tool itself a little more.

The intention of this is to:

• seperate the compilation parameters (eg. types of output, tool to output for) from the circuits source code.
• allow the compilation of module of a circuit without then generating these outputs when a module is imported
• define locations for library search paths separately from the source (and more cleanly than an environmental variable)

Describe alternatives you've considered

So far we've been rolling with the same export system as described in the docs, but it's a per-board kinda thing and means we can't readily visualise the subcircuits.

We've been copy-pasting a blob in the top of our scripts to modify the search paths and monkey-patch in the locations of the directories we're interested.

Additional context

I think that's covered it for now!
Anticipating a bit of conversation to continue!

What are your thoughts? And, most importantly, thank you for your time and consideration!

[devbisme]

I'm all for getting more SKiDL development assistance from others! If you can provide some concrete examples of what you're doing with SKiDL, then I can think about how we could modify things to support you.

[mawildoer]

In the case of the visualiser specifically, we'd like to have a long-running process watch your source files and update the visualisation as you create more valid code. I've managed to import skidl modules into a function and call it to generate their netlists, but this all kind of rely on the global circuit objects being the ones used, and it breaks the logging outputs and file names from how get_script_info works.
End goals with that are that I want to be able to run skidl visualise <circuit-source-to-look-at>.
To make that happen, I'm imagining something like circuit = skidl_import('<module-name-or-filepath>)

Additionally, let's say you're designing a board to control a CNC machine with 3 brushless servo motor circuits on it. To document it well, you'd ideally render schematics/graphs/some visualisation of the submodules used on it with the same styles, formats etc... I'd like to be able to call something like skidl compile --netlist --svg --output-dir=<output-dir> <list-of-sources> and have all the artefacts I've requested spat out to <output-dir>. It also removes boiler plate code from each of your source files.

[devbisme]

OK, here's what I think you're saying so you can correct me as needed:

1. You want some function in the skidl package called compile().
2. There is a stored configuration in your program that is applied by compile with higher priority than any explicit/implicit configuration options used by SKiDL.
3. Your program watches a set of SKiDL files and passes them to compile which runs and creates a Circuit object.
4. Your program receives the Circuit object and can then use it to generate netlists, SVG, under the influence of the configuration options you specified for compile.

[mawildoer]

While more implementation based than I'd been trying to get at, yes, I think we're on the same page and that'd achieve what we're looking for!

Seems like you've got some concrete ideas on where to start with it! There's some stuff in the setup and initial configuration of skidl that I've been hesitant to touch. The logging configuration is one example. I wanted your suggestions to keep any work I do inline with your expectations for the project before I go pulling some of the basics like that apart.

One paradigm I've worked with a little in other programs is having a "configure time" at the start of the program which gathers, resolves and fixes this kind of input params in place before starting any further program logic, importing additional modules etc... If I were to do this I think I'd need to flip around a bit of the sequence in skidl.py.

Alternatively, I could consider creating an alternate entry point from which to import the package from. One that, for example, doesn't set default circuit objects etc... but instead configures everything in this "compiler-like" fashion.

[devbisme]

One avenue I'm thinking about is making a second entry-point into the SKiDL library with an alternate start-up sequence. Then your program could trigger that to get access to what it needs. So the SKiDL programs would use standard SKiDL operations when run under Python, but would do something different when your programs triggered them without the need to change the imports in the source code. I'll have to delve into the details of Python imports to see what's possible.

I'll be gone for ten days, so don't take an absence of interaction as a lack of interest in this.

[mawildoer]

Enjoy the holidays!

I agree, I think your proposal and the last I hinted at are well aligned. I'll see where I get with a prototype in the time.