Skip to content

Bug on propagating arrow field metadata  #3964

Description

@metesynnada

Describe the bug
While converting DFSchema to Schema, field metadata is not preserved. It may cause future bugs on methods like alias.

To Reproduce
Create a DFSchema that has fields with metadata.

let mut a: DFField = DFField::new(Some("table1"), "a", DataType::Int64, false);
let mut b: DFField = DFField::new(Some("table1"), "b", DataType::Int64, false);
let mut a_metadata = BTreeMap::new();
a_metadata.insert("key".to_string(), "value".to_string());
a.field.set_metadata(Some(a_metadata));
let mut b_metadata = BTreeMap::new();
b_metadata.insert("key".to_string(), "value".to_string());
b.field.set_metadata(Some(b_metadata));
let df_schema = Arc::new(
    DFSchema::new_with_metadata(
        [a, b].to_vec(),
        HashMap::new(),
    )
        .unwrap(),
);
let schema: Schema = df_schema.as_ref().clone().into();
let a_df = df_schema.fields.get(0).unwrap().field();
let a_arrow = schema.fields.get(0).unwrap();
assert_eq!(a_df.metadata(), a_arrow.metadata())

Expected behavior
Fields preserve the metadata.

Additional context
It will be solved by changing From<DFSchema> for Schema method.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions