Skip to content

Commit 9064571

Browse files
andygrovenevi-me
authored andcommitted
ARROW-6085: [Rust] [DataFusion] Add traits for physical query plan
This is the first of quite a few PRs to add support for parallel (multi-threaded) query execution based on the PoC in #4221 Closes #4975 from andygrove/ARROW-6085 and squashes the following commits: f3a18d7 <Andy Grove> Add traits for physical query plan Authored-by: Andy Grove <andygrove73@gmail.com> Signed-off-by: Neville Dipale <nevilledips@gmail.com>
1 parent 25efd82 commit 9064571

File tree

2 files changed

+45
-0
lines changed

2 files changed

+45
-0
lines changed

rust/datafusion/src/execution/mod.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ pub mod context;
2222
pub mod expression;
2323
pub mod filter;
2424
pub mod limit;
25+
pub mod physical_plan;
2526
pub mod projection;
2627
pub mod relation;
2728
pub mod scalar_relation;
Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
// Licensed to the Apache Software Foundation (ASF) under one
2+
// or more contributor license agreements. See the NOTICE file
3+
// distributed with this work for additional information
4+
// regarding copyright ownership. The ASF licenses this file
5+
// to you under the Apache License, Version 2.0 (the
6+
// "License"); you may not use this file except in compliance
7+
// with the License. You may obtain a copy of the License at
8+
//
9+
// http://www.apache.org/licenses/LICENSE-2.0
10+
//
11+
// Unless required by applicable law or agreed to in writing,
12+
// software distributed under the License is distributed on an
13+
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
// KIND, either express or implied. See the License for the
15+
// specific language governing permissions and limitations
16+
// under the License.
17+
18+
//! Traits for physical query plan, supporting parallel execution for partitioned relations.
19+
20+
use arrow::datatypes::Schema;
21+
use arrow::record_batch::RecordBatch;
22+
use std::sync::Arc;
23+
24+
use crate::error::Result;
25+
26+
/// Partition-aware execution plan for a relation
27+
pub trait ExecutionPlan {
28+
/// Get the schema for this execution plan
29+
fn schema(&self) -> Arc<Schema>;
30+
/// Get the partitions for this execution plan. Each partition can be executed in parallel.
31+
fn partitions(&self) -> Result<Vec<Arc<Partition>>>;
32+
}
33+
34+
/// Represents a partition of an execution plan that can be executed on a thread
35+
pub trait Partition: Send + Sync {
36+
/// Execute this partition and return an iterator over RecordBatch
37+
fn execute(&self) -> Result<Arc<BatchIterator>>;
38+
}
39+
40+
/// Iterator over RecordBatch that can be sent between threads
41+
pub trait BatchIterator: Send + Sync {
42+
/// Get the next RecordBatch
43+
fn next(&self) -> Result<Option<RecordBatch>>;
44+
}

0 commit comments

Comments
 (0)