Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lots of things to try and get publishing working #557

Merged
merged 49 commits into from
Aug 8, 2024
Merged

Conversation

a10y
Copy link
Contributor

@a10y a10y commented Aug 6, 2024

Had to change the way we do a few things to get crates.io publishing working for our workspace:

  • vortex-build and its use of cargo metadata at compile time was causing lock files to get regenerated, which caused cargo publish to fail. In general, the way pretty much all major projects seem to handle codegen is to do it ahead of time. To that end, vortex-build has been removed completely, supplanted with the xtask setup described below

  • Moved all flatbuffer definitions into vortex-flatbuffers. Clients of that crate can select which subset of the definitions they want to compile with using features, current features are:

[features]
dtype = []
scalar = ["dtype"]
array = ["dtype", "scalar"]
file = ["dtype", "scalar", "array"]
  • New crate vortex-proto to mirror vortex-flatbuffers but for protobuf definitions. Features on that crate:
[features]
dtype = []
scalar = ["dtype"]
expr = ["dtype", "scalar"]
  • Added new xtask binary crate, runnable via cargo xtask -h with commands to execute codegen for both flatbuffers and protobufs. Currently has two subcommands available:
# generate flatbuffers Rust files, written to ./vortex-flatbuffers/src/generated/
cargo xtask generate-fbs

# generate protobuf Rust files, written to ./vortex-proto/src/generated
cargo xtask generate-proto

@a10y a10y marked this pull request as ready for review August 8, 2024 03:20
Comment on lines +14 to +15
xtask = "run -p xtask --"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see https://github.com/matklad/cargo-xtask

this seems like it's on defacto standard in the rust ecosystem for repo automation stuff

@a10y a10y merged commit 79d4df2 into develop Aug 8, 2024
3 checks passed
@a10y a10y deleted the aduffy/hail-mary branch August 8, 2024 12:01
@robert3005 robert3005 mentioned this pull request Aug 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants