How to write csv file to disk from a empty dataframe?

[mmooyyii]

I want write a csv file only include headers;

use datafusion::config::CsvOptions;
use datafusion::dataframe::DataFrameWriteOptions;
use datafusion::error::Result;
use datafusion::prelude::*;
use std::path::Path;

#[tokio::main]
async fn main() -> Result<()> {
    let ctx = SessionContext::new();

    let df = ctx.sql("SELECT 1 limit 0").await?;

    let mut cfg1 = DataFrameWriteOptions::new();
    cfg1 = cfg1.with_single_file_output(true);

    let mut cfg2 = CsvOptions::default();
    cfg2.has_header = Some(true);

    df.write_csv("/tmp/t.csv", cfg1, Some(cfg2)).await?;
    assert_eq!(Path::new("/tmp/t.csv").exists(), true);  // panic!
    Ok(())
}

[mmooyyii]

I want to do something like materialized view. I have to deal with this special case if datafusion can't write empty data frame;

[alamb]

I am not sure -- it seems like maybe we should have an option in DataFrameWriteOptions that will force an empty file to be written even if there are no rows 🤔

[chenkovsky]

take

[alamb]

Unfortunately, we had to revert the original fix, see

• Revert "fix: create file for empty stream" #16682

So I am reopening the ticket to track

[vadimpiven]

I can add that for Arrow format the empty DataFrame also does not create a file.