[mypyc] Add experimental C extension librt.vecs (part 1/2)#20653
Merged
[mypyc] Add experimental C extension librt.vecs (part 1/2)#20653
Conversation
This comment has been minimized.
This comment has been minimized.
p-sawicki
reviewed
Jan 27, 2026
mypyc/lib-rt/vecs/librt_vecs.c
Outdated
| static PyModuleDef vecsmodule = { | ||
| .m_base = PyModuleDef_HEAD_INIT, | ||
| .m_name = "vecs", | ||
| .m_doc = "vecs doc", |
mypyc/lib-rt/vecs/vec_template.c
Outdated
| // as vec[i64] or vec[float]. Assume that certain #defines are provided that | ||
| // provide all the item type specific definitions: | ||
| // | ||
| // VEC vec C type (e.g. VecI64) |
Collaborator
There was a problem hiding this comment.
might want to also describe the other defines.
This comment has been minimized.
This comment has been minimized.
Contributor
|
According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅ |
p-sawicki
approved these changes
Jan 27, 2026
JukkaL
added a commit
that referenced
this pull request
Jan 28, 2026
Add support for `vec` types with reference item types, such as `vec[str]` or `vec[MyClass | None]`. Arbitrary union item types aren't supported -- only optional types are accepted. Also add support for nested vecs, such as `vec[vec[i64]]`. No mypyc primitives are included in this PR yet. I'll add them in follow-up PRs. This continues the work started in #20653. Refer to that PR for a more detailed description of the feature. Related issue: mypyc/mypyc#840
JukkaL
added a commit
that referenced
this pull request
Feb 3, 2026
Add basic irbuild support for `vec[t]`. The runtime representation of `vec[t]` is a C struct. This is similar to how fixed-length tuples are represented. Multiple different structs are used, depending on the item type (`VecI32` for `vec[i32`] and so on). The C extension `librt.vec` that defines the `vec` type was added in #20653 and #20656. These PRs also explain the implementation in more detail. Add RType subclass RVec that is used for vecs. We need a new RType class, since primitives types can't be generic and they can't be struct types. This is based on an old branch, so it mostly uses old-style primitives. I am planning to modernize some of the primitives in follow-up PRs. This doesn't include codegen support, and irbuild test cases are only included for `vec[i64]`. I will create follow-up PRs that add the remaining irbuild tests, codegen support and run tests. All these tests are are passing on my local full branch. Related issue: mypyc/mypyc#840
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The extension defines the
vec[t]type and associated functions.vecis a sequence type optimized for use in compiled code.vecvalues are immutable and represented internally as a struct with a length and buffer fields. Only the buffer object is mutable. This allows very fast access to the length field in compiled code, but any operations that change the length of avecvalue must return a modifiedvecvalue (e.g.append).Related issue: mypyc/mypyc#840.
This PR doesn't define any mypyc primitives -- only use via generic Python operations is supported at this point. I have a local branch that implements efficient mypyc primitives for vecs, and there are significant performance gains over
listobjects in benchmarks.This PR only adds support for item types
i64,i32,i16,u8,floatandbool(e.g.vec[i32]). A follow-up PR will add support for reference item types (e.g.vec[str]) and nested vecs.See the comment at the top of
mypyc/lib-rt/vecs/librt_vecs.cfor a more detailed description. It also documents some implementation details of reference item types and nested vec types (which are not included yet in this PR).Currently a fairly minimal API is supported. This example shows some of the supported operations:
Additionally, read-only slicing and equality is supported. The
inoperator and iteration only work through the sequence API, but I will add proper support for these later on.