You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
added explaination for Schema and DFSchema to documentation (#15329)
* added explaination for Schema and DFSchema to documentation
* change code block to quote block since CICL request a full code but we are only introducing syntax
* improved language
* Update docs/source/library-user-guide/working-with-exprs.md
---------
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
Copy file name to clipboardExpand all lines: docs/source/library-user-guide/working-with-exprs.md
+19Lines changed: 19 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -50,6 +50,25 @@ As another example, the SQL expression `a + b * c` would be represented as an `E
50
50
51
51
As the writer of a library, you can use `Expr`s to represent computations that you want to perform. This guide will walk you through how to make your own scalar UDF as an `Expr` and how to rewrite `Expr`s to inline the simple UDF.
52
52
53
+
## Arrow Schema and DataFusion DFSchema
54
+
55
+
Apache Arrow `Schema` provides a lightweight structure for defining data, and Apache Datafusion`DFSchema` extends it with extra information such as column qualifiers and functional dependencies. Column qualifiers are multi part path to the table e.g table, schema, catalog. Functional Dependency is the relationship between attributes(characteristics) of a table related to each other.
56
+
57
+
### Difference between Schema and DFSchema
58
+
59
+
- Schema: A fundamental component of Apache Arrow, `Schema` defines a dataset's structure, specifying column names and their data types.
60
+
61
+
> Please see [Struct Schema](https://docs.rs/arrow-schema/54.2.1/arrow_schema/struct.Schema.html) for a detailed document of Arrow Schema.
62
+
63
+
- DFSchema: Extending `Schema`, `DFSchema` incorporates qualifiers such as table names, enabling it to carry additional context when required. This is particularly valuable for managing queries across multiple tables.
64
+
> Please see [Struct DFSchema](https://docs.rs/datafusion/latest/datafusion/common/struct.DFSchema.html) for a detailed document of DFSchema.
65
+
66
+
### How to convert between Schema and DFSchema
67
+
68
+
From Schema to DFSchema: Use `DFSchema::try_from_qualified_schema` with a table name and original schema, for detailed code example please see [creating-qualified-schemas](https://docs.rs/datafusion/latest/datafusion/common/struct.DFSchema.html#creating-qualified-schemas).
69
+
70
+
From DFSchema to Schema: Since the `Into` trait has been implemented for DFSchema to convert it into an Arrow Schema, for detailed code example please see [converting-back-to-arrow-schema](https://docs.rs/datafusion/latest/datafusion/common/struct.DFSchema.html#converting-back-to-arrow-schema).
71
+
53
72
## Creating and Evaluating `Expr`s
54
73
55
74
Please see [expr_api.rs](https://github.com/apache/datafusion/blob/main/datafusion-examples/examples/expr_api.rs) for well commented code for creating, evaluating, simplifying, and analyzing `Expr`s.
0 commit comments