Add get_polars_df() method to BigQueryHook.

[bfeif]

Description

This is mostly self-explanatory. The BigQueryHook already has a get_pandas_df(); this issue aims to add an analogous get_polars_df() method that would return a polars dataframe for some query.

Use case/motivation

Polars is a new dataframe library that is superior to pandas in terms of readability and speed; it'd be nice to have another built-in option besides pandas to get data from bigquery in Airflow.

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[bfeif]

I will open a PR on this in the next few days.

[eladkal]

It needs to be added to DbApiHook

Adding function similar to:

airflow/airflow/providers/common/sql/hooks/sql.py

Line 201 in ce2841b

def get_pandas_df(self, sql, parameters: Iterable | Mapping[str, Any] | None = None, **kwargs):

airflow/airflow/providers/common/sql/hooks/sql.py

Line 221 in ce2841b

def get_pandas_df_by_chunks(

and adding polars as extra package to common.sql package

[potiuk]

It needs to be added to DbApiHook

+1 . And it's great to add Polars BTW. I've been following the rise of Polars and it's a fantastic library :)

[RNHTTR]

Should it instead be a generic get_df and pass a param to use either polars or pandas?

[potiuk]

Should it instead be a generic get_df and pass a param to use either polars or pandas?

Might be an option yes

[bfeif]

Should it instead be a generic get_df and pass a param to use either polars or pandas?

This is probably what I'll do, it's a good idea. Perhaps though I'll leave get_pandas_df() in there to retain backward compatibility, and add a note to its docstring.