File tree Expand file tree Collapse file tree 1 file changed +32
-0
lines changed Expand file tree Collapse file tree 1 file changed +32
-0
lines changed Original file line number Diff line number Diff line change 1919
2020# Using DataFusion as a library
2121
22+ ## Default Configuration
23+
2224DataFusion is [ published on crates.io] ( https://crates.io/crates/datafusion ) , and is [ well documented on docs.rs] ( https://docs.rs/datafusion/ ) .
2325
2426To get started, add the following to your ` Cargo.toml ` file:
@@ -27,3 +29,33 @@ To get started, add the following to your `Cargo.toml` file:
2729[dependencies ]
2830datafusion = " 5.1.0"
2931```
32+
33+ ## Optimized Configuration
34+
35+ For an optimized build several steps are required. First, use the below in your ` Cargo.toml ` . It is
36+ worth noting that using the settings in the ` [profile.release] ` section will significantly increase the build time.
37+
38+ ``` toml
39+ [dependencies ]
40+ datafusion = { version = " 5.0" , features = [" simd" ]}
41+ tokio = { version = " ^1.0" , features = [" macros" , " rt" , " rt-multi-thread" ] }
42+ snmalloc-rs = {version = " 0.2" , features = [" cache-friendly" ]}
43+ num_cpus = " 1.0"
44+
45+ [profile .release ]
46+ lto = true
47+ codegen-units = 1
48+ ```
49+
50+ Then, in ` main.rs. ` update the memory allocator with the below after your imports:
51+
52+ ``` rust
53+ #[global_allocator]
54+ static ALLOC : snmalloc_rs :: SnMalloc = snmalloc_rs :: SnMalloc ;
55+ ```
56+
57+ Finally, in order to build with the ` simd ` optimization ` cargo nightly ` is required. Based on the instruction
58+ set architecture you are building on you will want to configure the ` target-cpu ` as well, ideally
59+ with ` native ` or at least ` avx2 ` .
60+
61+ ` RUSTFLAGS='-C target-cpu=native' cargo +nightly run --release `
You can’t perform that action at this time.
0 commit comments