Please answer these questions before submitting your issue. Thanks!
-
What version of Python are you using?
3.10.8
-
What operating system and processor architecture are you using?
macOS-12.4-arm64-arm-64bit
-
What are the component versions in the environment (pip freeze)?
pyarrow==8.0.0
snowflake-connector-python==2.8.3
-
What did you do?
I tried to use the Cursor method fetch_arrow_all() without pandas as a dependency which threw the following error:
NameError: name 'concat_tables' is not defined in the snowflake/connector/result_set.py module.
concat_tables and pyarrow are only imported if the installed_pandas variable is True. Tracing installed_pandas back, it is True when from pandas import DataFrame runs without an error.
Essentially, pyarrow and concat_tables are only imported if pandas is installed in the environment, thus using the fetch_arrow_all() method unnecessarily requires pandas as a dependency.
For use cases not requiring pandas (such as using polars) it would be nice not to have pandas as an unnecessary dependency but still be able to use the pyarrow functionality.
Please answer these questions before submitting your issue. Thanks!
What version of Python are you using?
3.10.8
What operating system and processor architecture are you using?
macOS-12.4-arm64-arm-64bit
What are the component versions in the environment (
pip freeze)?pyarrow==8.0.0
snowflake-connector-python==2.8.3
What did you do?
I tried to use the Cursor method
fetch_arrow_all()without pandas as a dependency which threw the following error:NameError: name 'concat_tables' is not definedin thesnowflake/connector/result_set.pymodule.concat_tablesandpyarroware only imported if theinstalled_pandasvariable is True. Tracinginstalled_pandasback, it is True whenfrom pandas import DataFrameruns without an error.Essentially,
pyarrowandconcat_tablesare only imported ifpandasis installed in the environment, thus using thefetch_arrow_all()method unnecessarily requirespandasas a dependency.For use cases not requiring
pandas(such as usingpolars) it would be nice not to havepandasas an unnecessary dependency but still be able to use thepyarrowfunctionality.