Module system extensions and assemblies #12
Conversation
|
|
||
| ### Foreign formats | ||
|
|
||
| `import` can be used to import objects defined in other CAD formats, replacing the `import` function in the standard library. E.g., `model = import("tests/inputs/cube.obj")` is replaced by `import "tests/inputs/cube.obj" as model`. Such objects are treated in the same way as other assemblies, but they are always opaque (no AST, etc.) and read-only. |
There was a problem hiding this comment.
I was wondering about this. I like the idea of combining these cases.
|
|
||
| This is mostly postponed for future work. We would use the same syntax and semantics as the rest of the system, however the string in the import would include some indicator that the target is outside the project and how to locate it. | ||
|
|
||
| Example with very, very strawman syntax: `import a from "cargo://some-library/foo.kcl"` would import `a` from the file `foo.kcl` in the project `some-library` located in the `cargo` (lol) repository. `import a from "local://../some-library/foo.kcl` would import `a` from the file `foo.kcl` from a sibling directory of the current project called `some-library`. The point of these examples is not the syntax for identifying other projects or the places where an external project might live, just to show how the `import` statement might be extended to support other projects. |
There was a problem hiding this comment.
There was a problem hiding this comment.
strong concur; it also lets us avoid the incredible complexity of a package manager (and associated tooling which we'd have to handle), as well as it adds a lot of clarity in the modules (no namespace squatting; food-fight adjudication over copyrights and shit.
It also gives a level of "official" status to our repos and other trusted orgs -- it's hard to tell if https://pypi.org/project/cryptography/ is official pyca or some rano without looking closely; or like tokio_xyz -- is that tokio or some rando? Hard to tell unless you find the repo (which may or may not get linked).
Love having everything be a git url, or even some intermediate like go's module url protocol if we wanted to get wicked fancy with it (for sub-folder imports this would be required so we know where the module root is; likely with special-casing for github as go-get has until they support ?kcl-import=1)
There was a problem hiding this comment.
I would rather we use an object than a string, e.g. "import a from { repo = "https://github.com/KittyCAD/kcl-samples/my-wheel.kcl", branch = "next" } rather than trying to stuff all the things like filename, branch, directory into a string. Making it an object lets us add more fields in the future e.g. auth for private repos
|
Being able to import from the internet would be very nice for open source projects so that not everything needs to be downloaded locally by the user somehow. Referencing the imported code via by a git commit or tag (e.g. v1.1) can be specified in the URL to ensure getting the intended and stable / locked-down version of the code. |
Signed-off-by: Nick Cameron <nrc@ncameron.org>
|
|
||
| There are downsides to allowing this: primarily if an exported name is added to foo.kcl then that can cause an error in the importing file if the name already exists (which may be from another import, e.g., `import * from "bar.kcl"`). It's also a bit harder to follow the code without tooling (you can't tell at a glance where a name is defined). | ||
|
|
||
| Syntax: `*` is common in many PLs, but is a bit opaque for new programmers. A keyword like `all`, `everything`, or `any` would be better, but that is one less name we can all use for programming (we can't use a contextual keyword here). I think it would be a bit too subtle to use no names to mean all names (e.g., `import from "foo.kcl"`). |
There was a problem hiding this comment.
What about import all symbols from "foo.kcl"? It would be unambiguous because if you're trying to import two parts called "all" and "symbols" you would need a comma between them.
You could also do import a, b, and all other from "foo.kcl".
| - KCL files in a directory are submodules of that directory | ||
| - E.g., `foo/bar.kcl` defines a submodule `bar` inside `foo` | ||
| - It is OK to have a directory and KCL file with the same name, e.g., if `foo.kcl` exists next to `foo`, that is fine and they represent the same module `foo`. | ||
| - Files in a directory are always in scope as modules in the corresponding file. E.g., if `bar.kcl` exists in the directory `foo`, then `bar` is available as a module name inside `foo.kcl`. |
There was a problem hiding this comment.
Just to clarify, is this saying that users don't need to explicitly use an import statement to use things in subdirectories/submodules of the current module?
Directory structure:
foo/
globals.kcl
foo.kcl
// foo/globals.kcl
export A = 2
// foo.kcl
globals::A + 40 // This works?
There was a problem hiding this comment.
Yes, that is correct. For clarity, from another module foo::globals::A would work, and in foo A would not without an import.
|
|
||
| ## Importing from other directories within the project | ||
|
|
||
| File paths (as well as filenames) may be used inside the quoted string of an `import`. Paths may contain `/` (but not `\`, even on Windows). Paths are always relative to the top-level of the project (not relative to the current file or absolute paths), and may not include `.` or `..` (alternative: we could relax this requirement and allow relative paths as long as traversing the path never leaves the project directory) or global identifiers (`c:`, etc. or starting with '/'). Paths are ignored in the name of the module as imported. E.g., `import "foo.kcl"` brings in `foo` from the root directory of the project, `import "baz/bar/foo.kcl"` brings in `foo` from `foo.kcl` found at `baz/bar/`, `import a from "baz/bar/foo.kcl"` brings in `a` from the same file. The path of the file makes no difference to the imported name or to privacy. |
There was a problem hiding this comment.
Relative imports are pretty nice in that you can move a directory without having to update all the import statements within the files. But it seems like we can always add this later as a non-breaking change.
There was a problem hiding this comment.
The weird thing about everything being relative to the project is that in order to execute, you always need an extra bit of information about where the project root is. In ZMA, we have that. In the CLI, when a user invokes it with the path to a KCL file, the project root may be ambiguous. It would be nice if we could avoid the case where if you cd and run the same file, it breaks.
There was a problem hiding this comment.
The contra-argument, is that relative imports are not so great because it means if you move just a file, you have to update all the relative imports.
There was a problem hiding this comment.
The second comment is an interesting thing. I think it comes down to whether a 'program' is a single file or a project. And with file-based imports and the desire to be able to execute parts independently, we are leaning towards the former. I find project-relative paths to be much more intuitive though (e.g., I hate the behaviour of include_str in Rust).
There was a problem hiding this comment.
The contra-argument, is that relative imports are not so great because it means if you move just a file, you have to update all the relative imports.
This tells me that both ways are equally valid and useful. It seems arbitrary to favor one over the other. As a user, both are useful in different situations, so I stand by my statement that I think having both would be nice because I have more context on my own project than the language implementer about which is going to cause me more work.
we are leaning towards the former.
By "former", I assume you mean project-based.
Now that I think of it, the proposal would probably be a breaking change for kcl-samples because people open the entire directory in ZMA. I mean, I know I have, because I cloned the repo (recently archived) into my Zoo projects directory. But it's actually a set of projects, one in each subdirectory. So I don't know. None of them have a project.toml file to disambiguate. So how would ZMA know where the project root is?
Would we be okay saying that people have to open each directory individually?
There was a problem hiding this comment.
No, sorry by former I meant file-based because files often stand alone - samples being another example of that
|
|
||
| Example with very, very strawman syntax: `import a from "cargo://some-library/foo.kcl"` would import `a` from the file `foo.kcl` in the project `some-library` located in the `cargo` (lol) repository. `import a from "local://../some-library/foo.kcl` would import `a` from the file `foo.kcl` from a sibling directory of the current project called `some-library`. The point of these examples is not the syntax for identifying other projects or the places where an external project might live, just to show how the `import` statement might be extended to support other projects. | ||
|
|
||
| Privacy across multiple projects is postponed, but I would like to keep it simple, e.g., just keeping the single `export` keyword to expose names to other projects as well as other modules in the same project. |
There was a problem hiding this comment.
In most programming languages, it's really useful to distinguish between exposed to the project and exposed outside the project, as in pub vs. pub(crate) in Rust. The most common use case is that you want to expose something from a module for testing, but you don't want to expose it to the world. But it can also come up for helper or util functions for libraries. I think that the lack of this will lead to more things being exported than should, i.e. people learning to default to export and private library internals exposed to the world because there's no other way to reuse code.
There was a problem hiding this comment.
Yeah, I think there is some work to do still for cross-project import/export. (This comment is from quite a while ago, I still think simple is important, but I'm not sure if just export is enough). I think we would want something for exporting from a project, but that depends a bit on importing - I'm not sold on the simple strawman of importing a file from a repo - I think we might want some project scoping which would include what is exported.
|
|
||
| ## Import constants from a module | ||
|
|
||
| We permit `export` on constant declarations (e.g., `export foo = 42`) and those exported constants can be imported in the same way as functions. The difficulty is around side effects, design there is postponed, blocked on discussing side-effects and the KCL model of computation. |
There was a problem hiding this comment.
How about allowing the export of literals and literals only, for now? They wouldn't have any side-effects.
|
|
||
| We may wish to allow the module itself to be imported as well as specific names: `import a, self from "foo.kcl"` (I'm reusing the `self` keyword from Rust, but there may be a better one for KCL) would import `a` from `foo.kcl` and the module itself as `foo`. Renaming (`self as bar`) would have the obvious meaning. | ||
|
|
||
| We may also want to allow an assignment style rename: `bar = import "foo.kcl"` with the same meaning as `import "foo.kcl" as bar`. This would be more natural for assemblies. Although it provides two ways to do the same thing (generally bad), I think both ways are natural (and I am thinking of using `as` elsewhere with the same meaning to implement tagging, but that's way out of scope for this doc). If we do allow this we *may* also consider it for single name imports,e.g., `bar = import baz from "foo.kcl"`. |
There was a problem hiding this comment.
I like the assignment syntax
|
|
||
| A module may or may not contain top-level geometry (as well as zero or more named constants or functions). If it does, then the name of the module (whether renamed, assigned or whatever) should work as an assembly. E.g., assuming foo.kcl contains top-level geometry, then after `import "foo.kcl"`, `foo` can be used in the same way as if the geometry from foo.kcl were defined in the importing file as a constant called `foo`. | ||
|
|
||
| When importing a module, the module is treated as an object with name (and type) inherited from the name of the module (or the alias used for the import). Each top-level, geometric entity is available as a numeric field, numbered in order. Where an item is just a variable use, that creates a named field as well as a numbered one with the same value. Exported variables and functions are available as named fields (see below). So, continuing the example above, `foo` has type `foo` which is an object type. |
There was a problem hiding this comment.
I don't think the order in which entities are defined should be important. If you reorder two parts in a KCL file which don't depend on each other, I would be surprised if that breaks some other code in some other module which imports it. Maybe we should force users to name their parts if they want to be exported.
|
|
||
| ## Importing data | ||
|
|
||
| `import` can also be used to import data, e.g., `import "foo.json"` would create a `foo` object with the json data. Exactly how that works is postponed design, the interesting questions I see are how arbitrary data is represented in our quite limited object data structure, and how we specify what format to treat data as (is it just implied by the extension or can it be overridden? Can we support arbitrary data formats by allowing the user to provide a decoder, etc.) |
There was a problem hiding this comment.
I really like this, great idea. I know Jordan would love that for his QR code data.
This doc describes possible extensions to the current
import/exportsyntax for modules. This includes support for assemblies.