-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
Describe the bug
While converting DFSchema to Schema, field metadata is not preserved. It may cause future bugs on methods like alias.
To Reproduce
Create a DFSchema that has fields with metadata.
let mut a: DFField = DFField::new(Some("table1"), "a", DataType::Int64, false);
let mut b: DFField = DFField::new(Some("table1"), "b", DataType::Int64, false);
let mut a_metadata = BTreeMap::new();
a_metadata.insert("key".to_string(), "value".to_string());
a.field.set_metadata(Some(a_metadata));
let mut b_metadata = BTreeMap::new();
b_metadata.insert("key".to_string(), "value".to_string());
b.field.set_metadata(Some(b_metadata));
let df_schema = Arc::new(
DFSchema::new_with_metadata(
[a, b].to_vec(),
HashMap::new(),
)
.unwrap(),
);
let schema: Schema = df_schema.as_ref().clone().into();
let a_df = df_schema.fields.get(0).unwrap().field();
let a_arrow = schema.fields.get(0).unwrap();
assert_eq!(a_df.metadata(), a_arrow.metadata())Expected behavior
Fields preserve the metadata.
Additional context
It will be solved by changing From<DFSchema> for Schema method.
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working