Skip to content

Commit 50cce1a

Browse files
Updated user-guide library docs with optimized config (#976)
* Updated user-guide library docs with optimized config * clarify description and remove datafusion git reference
1 parent 7a445e5 commit 50cce1a

File tree

1 file changed

+32
-0
lines changed

1 file changed

+32
-0
lines changed

docs/user-guide/src/library.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,8 @@
1919

2020
# Using DataFusion as a library
2121

22+
## Default Configuration
23+
2224
DataFusion is [published on crates.io](https://crates.io/crates/datafusion), and is [well documented on docs.rs](https://docs.rs/datafusion/).
2325

2426
To get started, add the following to your `Cargo.toml` file:
@@ -27,3 +29,33 @@ To get started, add the following to your `Cargo.toml` file:
2729
[dependencies]
2830
datafusion = "5.1.0"
2931
```
32+
33+
## Optimized Configuration
34+
35+
For an optimized build several steps are required. First, use the below in your `Cargo.toml`. It is
36+
worth noting that using the settings in the `[profile.release]` section will significantly increase the build time.
37+
38+
```toml
39+
[dependencies]
40+
datafusion = { version = "5.0" , features = ["simd"]}
41+
tokio = { version = "^1.0", features = ["macros", "rt", "rt-multi-thread"] }
42+
snmalloc-rs = {version = "0.2", features= ["cache-friendly"]}
43+
num_cpus = "1.0"
44+
45+
[profile.release]
46+
lto = true
47+
codegen-units = 1
48+
```
49+
50+
Then, in `main.rs.` update the memory allocator with the below after your imports:
51+
52+
```rust
53+
#[global_allocator]
54+
static ALLOC: snmalloc_rs::SnMalloc = snmalloc_rs::SnMalloc;
55+
```
56+
57+
Finally, in order to build with the `simd` optimization `cargo nightly` is required. Based on the instruction
58+
set architecture you are building on you will want to configure the `target-cpu` as well, ideally
59+
with `native` or at least `avx2`.
60+
61+
`RUSTFLAGS='-C target-cpu=native' cargo +nightly run --release`

0 commit comments

Comments
 (0)