Skip to content

Commit 8756197

Browse files
Jiashu-Hucomphead
andauthored
added explaination for Schema and DFSchema to documentation (#15329)
* added explaination for Schema and DFSchema to documentation * change code block to quote block since CICL request a full code but we are only introducing syntax * improved language * Update docs/source/library-user-guide/working-with-exprs.md --------- Co-authored-by: Oleks V <comphead@users.noreply.github.com>
1 parent 3269f01 commit 8756197

File tree

1 file changed

+19
-0
lines changed

1 file changed

+19
-0
lines changed

docs/source/library-user-guide/working-with-exprs.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,25 @@ As another example, the SQL expression `a + b * c` would be represented as an `E
5050

5151
As the writer of a library, you can use `Expr`s to represent computations that you want to perform. This guide will walk you through how to make your own scalar UDF as an `Expr` and how to rewrite `Expr`s to inline the simple UDF.
5252

53+
## Arrow Schema and DataFusion DFSchema
54+
55+
Apache Arrow `Schema` provides a lightweight structure for defining data, and Apache Datafusion`DFSchema` extends it with extra information such as column qualifiers and functional dependencies. Column qualifiers are multi part path to the table e.g table, schema, catalog. Functional Dependency is the relationship between attributes(characteristics) of a table related to each other.
56+
57+
### Difference between Schema and DFSchema
58+
59+
- Schema: A fundamental component of Apache Arrow, `Schema` defines a dataset's structure, specifying column names and their data types.
60+
61+
> Please see [Struct Schema](https://docs.rs/arrow-schema/54.2.1/arrow_schema/struct.Schema.html) for a detailed document of Arrow Schema.
62+
63+
- DFSchema: Extending `Schema`, `DFSchema` incorporates qualifiers such as table names, enabling it to carry additional context when required. This is particularly valuable for managing queries across multiple tables.
64+
> Please see [Struct DFSchema](https://docs.rs/datafusion/latest/datafusion/common/struct.DFSchema.html) for a detailed document of DFSchema.
65+
66+
### How to convert between Schema and DFSchema
67+
68+
From Schema to DFSchema: Use `DFSchema::try_from_qualified_schema` with a table name and original schema, for detailed code example please see [creating-qualified-schemas](https://docs.rs/datafusion/latest/datafusion/common/struct.DFSchema.html#creating-qualified-schemas).
69+
70+
From DFSchema to Schema: Since the `Into` trait has been implemented for DFSchema to convert it into an Arrow Schema, for detailed code example please see [converting-back-to-arrow-schema](https://docs.rs/datafusion/latest/datafusion/common/struct.DFSchema.html#converting-back-to-arrow-schema).
71+
5372
## Creating and Evaluating `Expr`s
5473

5574
Please see [expr_api.rs](https://github.com/apache/datafusion/blob/main/datafusion-examples/examples/expr_api.rs) for well commented code for creating, evaluating, simplifying, and analyzing `Expr`s.

0 commit comments

Comments
 (0)