Skip to content

Commit

Permalink
Remove redundant rust dir (apache#358)
Browse files Browse the repository at this point in the history
* Remove rust folder

* Update build scripts and docs

* fix CI
  • Loading branch information
andygrove authored Oct 17, 2022
1 parent e3f5820 commit cc3130e
Show file tree
Hide file tree
Showing 111 changed files with 35 additions and 37 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ jobs:
export PATH=$PATH:$HOME/d/protoc/bin
export ARROW_TEST_DATA=$(pwd)/testing/data
export PARQUET_TEST_DATA=$(pwd)/parquet-testing/data
cd ballista/rust
cd ballista
# snmalloc requires cmake so build without default features
cargo test --no-default-features --features sled
# Ensure also compiles in standalone mode
Expand Down
8 changes: 4 additions & 4 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,10 @@
[workspace]
members = [
"benchmarks",
"ballista/rust/client",
"ballista/rust/core",
"ballista/rust/executor",
"ballista/rust/scheduler",
"ballista/client",
"ballista/core",
"ballista/executor",
"ballista/scheduler",
"examples",
"ballista-cli",
]
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ Ballista implements a similar design to Apache Spark (particularly Spark SQL), b
# Getting Started

The easiest way to get started is to run one of the standalone or distributed [examples](./examples/README.md). After
that, refer to the [Getting Started Guide](ballista/rust/client/README.md).
that, refer to the [Getting Started Guide](ballista/client/README.md).

## Project Status

Expand Down
2 changes: 1 addition & 1 deletion ballista-cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ rust-version = "1.59"
readme = "README.md"

[dependencies]
ballista = { path = "../ballista/rust/client", version = "0.8.0", features = [
ballista = { path = "../ballista/client", version = "0.8.0", features = [
"standalone",
] }
clap = { version = "3", features = ["derive", "cargo"] }
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -113,10 +113,10 @@ pub fn plugin_dir() -> String {
};

// If current_exe_dir contain `deps` the root dir is the parent dir
// eg: /Users/xxx/workspace/rust/rust_plugin_sty/target/debug/deps/plugins_app-067452b3ff2af70e
// the plugin dir is /Users/xxx/workspace/rust/rust_plugin_sty/target/debug/deps
// else eg: /Users/xxx/workspace/rust/rust_plugin_sty/target/debug/plugins_app
// the plugin dir is /Users/xxx/workspace/rust/rust_plugin_sty/target/debug/
// eg: /Users/xxx/workspace/rust_plugin_sty/target/debug/deps/plugins_app-067452b3ff2af70e
// the plugin dir is /Users/xxx/workspace/rust_plugin_sty/target/debug/deps
// else eg: /Users/xxx/workspace/rust_plugin_sty/target/debug/plugins_app
// the plugin dir is /Users/xxx/workspace/rust_plugin_sty/target/debug/
if current_exe_dir.contains("/deps/") {
let i = current_exe_dir.find("/deps/").unwrap();
String::from(&current_exe_dir.as_str()[..i + 6])
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
2 changes: 0 additions & 2 deletions ballista/rust/.gitignore

This file was deleted.

File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
4 changes: 2 additions & 2 deletions benchmarks/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ simd = ["datafusion/simd"]
snmalloc = ["snmalloc-rs"]

[dependencies]
ballista = { path = "../ballista/rust/client", version = "0.8.0" }
ballista = { path = "../ballista/client", version = "0.8.0" }
datafusion = "13.0.0"
datafusion-proto = "13.0.0"
env_logger = "0.9"
Expand All @@ -47,4 +47,4 @@ structopt = { version = "0.3", default-features = false }
tokio = { version = "^1.0", features = ["macros", "rt", "rt-multi-thread", "parking_lot"] }

[dev-dependencies]
ballista-core = { path = "../ballista/rust/core", version = "0.8.0" }
ballista-core = { path = "../ballista/core", version = "0.8.0" }
4 changes: 2 additions & 2 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ To run the benchmarks it is necessary to have at least one Ballista scheduler an
To run the scheduler from source:

```bash
cd $ARROW_HOME/ballista/rust/scheduler
cd $ARROW_HOME/ballista/scheduler
RUST_LOG=info cargo run --release
```

Expand All @@ -127,7 +127,7 @@ By default the scheduler will bind to `0.0.0.0` and listen on port 50050.
To run the executor from source:

```bash
cd $ARROW_HOME/ballista/rust/executor
cd $ARROW_HOME/ballista/executor
RUST_LOG=info cargo run --release
```

Expand Down
2 changes: 1 addition & 1 deletion dev/build-set-env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,6 @@
# specific language governing permissions and limitations
# under the License.

cd ballista/rust/core/
cd ballista/core/
export BALLISTA_VERSION=$(cargo pkgid | cut '-d@' -f2)
cd -
8 changes: 4 additions & 4 deletions dev/release/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -281,10 +281,10 @@ dot -Tsvg dev/release/crate-deps.dot > dev/release/crate-deps.svg
```

```shell
(cd ballista/rust/core && cargo publish)
(cd ballista/rust/executor && cargo publish)
(cd ballista/rust/scheduler && cargo publish)
(cd ballista/rust/client && cargo publish)
(cd ballista/core && cargo publish)
(cd ballista/executor && cargo publish)
(cd ballista/scheduler && cargo publish)
(cd ballista/client && cargo publish)
(cd ballista-cli && cargo publish)
```

Expand Down
2 changes: 1 addition & 1 deletion dev/release/rat_exclude_files.txt
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ requirements.txt
.gitattributes
rust-toolchain
benchmarks/queries/q*.sql
ballista/rust/scheduler/testdata/*
ballista/scheduler/testdata/*
**/yarn.lock
python/rust-toolchain
python/requirements*.txt
Expand Down
2 changes: 1 addition & 1 deletion dev/release/verify-release-candidate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ test_source_distribution() {

# Note can't verify other ballista crates as they depend
# on ballista-core which isn't published yet
pushd ballista/rust/core
pushd ballista/core
cargo publish --dry-run
popd
}
Expand Down
8 changes: 4 additions & 4 deletions dev/update_ballista_versions.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,10 +79,10 @@ def main():
os.path.join(repo_root, rel_path, "Cargo.toml")
for rel_path in [
'ballista-cli',
'ballista/rust/core',
'ballista/rust/scheduler',
'ballista/rust/executor',
'ballista/rust/client',
'ballista/core',
'ballista/scheduler',
'ballista/executor',
'ballista/client',
'benchmarks',
'examples',
'python',
Expand Down
8 changes: 4 additions & 4 deletions dev/update_datafusion_versions.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,10 @@

ballista_crates = {
'ballista-cli': 'ballista-cli/Cargo.toml',
'core': 'ballista/rust/core/Cargo.toml',
'client': 'ballista/rust/client/Cargo.toml',
'executor': 'ballista/rust/executor/Cargo.toml',
'scheduler': 'ballista/rust/scheduler/Cargo.toml',
'core': 'ballista/core/Cargo.toml',
'client': 'ballista/client/Cargo.toml',
'executor': 'ballista/executor/Cargo.toml',
'scheduler': 'ballista/scheduler/Cargo.toml',
}

def update_datafusion_version(cargo_toml: str, new_version: str):
Expand Down
2 changes: 1 addition & 1 deletion docs/developer/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ processes.
## Scheduler Process

The scheduler process implements a gRPC interface (defined in
[ballista.proto](../rust/core/proto/ballista.proto)). The interface provides the following methods:
[ballista.proto](../core/proto/ballista.proto)). The interface provides the following methods:

| Method | Description |
| -------------------- | -------------------------------------------------------------------- |
Expand Down
2 changes: 1 addition & 1 deletion docs/developer/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Those files fully define Ballista's configuration. If there is a discrepancy bet

To get a list of command line arguments, run the binary with `--help`

There is an example config file at `ballista/rust/ballista/examples/example_executor_config.toml`
There is an example config file at `ballista/ballista/examples/example_executor_config.toml`

The order of precedence for arguments is: default config file < environment variables < specified config file < command line arguments.

Expand Down
2 changes: 1 addition & 1 deletion examples/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ path = "examples/standalone-sql.rs"
required-features = ["ballista/standalone"]

[dependencies]
ballista = { path = "../ballista/rust/client", version = "0.8.0" }
ballista = { path = "../ballista/client", version = "0.8.0" }
datafusion = "13.0.0"
futures = "0.3"
num_cpus = "1.13.0"
Expand Down
2 changes: 1 addition & 1 deletion examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ async fn main() -> Result<()> {
# Distributed Examples

For background information on the Ballista architecture, refer to
the [Ballista README](../ballista/rust/client/README.md).
the [Ballista README](../ballista/client/README.md).

## Start a standalone cluster

Expand Down
2 changes: 1 addition & 1 deletion python/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ default = ["mimalloc"]

[dependencies]
async-trait = "0.1"
ballista = { path = "../ballista/rust/client", version = "0.8.0" }
ballista = { path = "../ballista/client", version = "0.8.0" }
datafusion = { version = "13.0.0", features = ["pyarrow"] }
futures = "0.3"
mimalloc = { version = "*", optional = true, default-features = false }
Expand Down

0 comments on commit cc3130e

Please sign in to comment.