Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SNOW-800181: write_pandas "Error: Missing optional dependency: pandas" despite having pandas installed. #1527

Closed
mpgreg opened this issue Apr 24, 2023 · 9 comments
Closed

SNOW-800181: write_pandas "Error: Missing optional dependency: pandas" despite having pandas installed. #1527

mpgreg opened this issue Apr 24, 2023 · 9 comments
Labels
bug needs triage

Comments

@mpgreg
Copy link

mpgreg commented Apr 24, 2023 •
edited
Loading

Please answer these questions before submitting your issue. Thanks!

  1. What version of Python are you using?

Python 3.9.16

  1. What operating system and processor architecture are you using?

    Docker container: Linux-5.15.49-linuxkit-aarch64-with-glibc2.31

  2. What are the component versions in the environment (pip freeze)?

   astunparse==1.6.3
async-timeout==4.0.2
attrs==23.1.0
Authlib==1.2.0
Babel==2.12.1
blinker==1.6.2
boto3==1.26.102
botocore==1.29.118
cachelib==0.9.0
cachetools==5.3.0
cattrs==22.2.0
certifi==2022.12.7
cffi==1.15.1
charset-normalizer==2.1.1
click==8.1.3
clickclick==20.10.2
cloudpickle==2.0.0
colorama==0.4.6
colorlog==4.8.0
ConfigUpdater==3.1.1
connexion==2.14.2
cramjam==2.6.2
cron-descriptor==1.2.35
croniter==1.3.14
cryptography==40.0.2
decorator==5.1.1
Deprecated==1.2.13
dill==0.3.6
distlib==0.3.6
dnspython==2.3.0
docutils==0.19
email-validator==1.3.1
exceptiongroup==1.1.1
fastparquet==2023.2.0
ffmpeg-python==0.2.0
filelock==3.12.0
Flask==2.2.3
Flask-AppBuilder==4.1.4
Flask-Babel==2.0.0
Flask-Caching==2.0.2
Flask-JWT-Extended==4.4.4
Flask-Login==0.6.2
Flask-Session==0.4.0
Flask-SQLAlchemy==2.5.1
Flask-WTF==1.1.1
flatbuffers==23.3.3
frozenlist==1.3.3
fsspec==2023.1.0
future==0.18.3
gast==0.4.0
google-auth==2.17.3
google-auth-oauthlib==1.0.0
google-pasta==0.2.0
graphviz==0.20.1
greenlet==2.0.2
grpcio==1.54.0
gunicorn==20.1.0
h11==0.14.0
h5py==3.8.0
httpcore==0.17.0
httpx==0.24.0
huggingface-hub==0.13.4
idna==3.4
importlib-metadata==4.13.0
importlib-resources==5.12.0
inflection==0.5.1
itsdangerous==2.1.2
jax==0.4.8
Jinja2==3.1.2
jmespath==1.0.1
joblib==1.2.0
jsonschema==4.17.3
keras==2.12.0
lazy-object-proxy==1.9.0
libclang==16.0.0
linkify-it-py==2.0.0
lockfile==0.12.2
Mako==1.2.4
Markdown==3.4.3
markdown-it-py==2.2.0
MarkupSafe==2.1.2
marshmallow==3.19.0
marshmallow-enum==1.5.1
marshmallow-oneofschema==3.0.1
marshmallow-sqlalchemy==0.26.1
mdit-py-plugins==0.3.5
mdurl==0.1.2
ml-dtypes==0.1.0
more-itertools==9.1.0
multidict==6.0.4
numpy==1.23.5
oauthlib==3.2.2
openai==0.27.2
opt-einsum==3.3.0
oscrypto==1.3.0
packaging==23.1
pandas==1.5.3
pathspec==0.9.0
pendulum==2.1.2
pkgutil_resolve_name==1.3.10
platformdirs==3.2.0
pluggy==1.0.0
prison==0.2.1
protobuf==4.22.3
psutil==5.9.5
psycopg2-binary==2.9.6
pyarrow==10.0.1
pyasn1==0.5.0
pyasn1-modules==0.3.0
pycparser==2.21
pycryptodomex==3.17
Pygments==2.15.1
PyJWT==2.6.0
pyOpenSSL==23.1.1
pyrsistent==0.19.3
python-daemon==3.0.1
python-dateutil==2.8.2
python-nvd3==0.15.0
python-slugify==8.0.1
pytz==2023.3
pytzdata==2020.1
PyYAML==6.0
regex==2023.3.23
requests==2.28.2
requests-oauthlib==1.3.1
requests-toolbelt==0.10.1
rfc3339-validator==0.1.4
rich==13.3.4
rsa==4.9
s3transfer==0.6.0
scikit-learn==1.2.2
scipy==1.10.1
setproctitle==1.3.2
six==1.16.0
sniffio==1.3.0
snowflake-connector-python==3.0.2
snowflake-snowpark-python==1.2.0
snowflake-sqlalchemy==1.4.7
SQLAlchemy==1.4.47
SQLAlchemy-JSONField==1.0.1.post0
SQLAlchemy-Utils==0.41.0
sqlparse==0.4.4
tabulate==0.9.0
tenacity==8.2.2
tensorboard==2.12.2
tensorboard-data-server==0.7.0
tensorboard-plugin-wit==1.8.1
tensorflow==2.12.0
tensorflow-cpu-aws==2.12.0
tensorflow-estimator==2.12.0
tensorflow-io-gcs-filesystem==0.32.0
termcolor==2.3.0
text-unidecode==1.3
threadpoolctl==3.1.0
tokenizers==0.13.3
tqdm==4.65.0
transformers==4.27.4
typing_extensions==4.5.0
uc-micro-py==1.0.1
unicodecsv==0.14.1
urllib3==1.26.15
validators==0.19.0
virtualenv==20.21.0
weaviate-client==3.15.4
Werkzeug==2.2.3
wrapt==1.14.1
WTForms==3.0.1
yarl==1.9.1
zipp==3.15.0

  1. What did you do?

    write_pandas(snowflake_hook.get_conn(), calls_df[['CUSTOMER_ID', 'RELATIVE_PATH', 'TRANSCRIPT']], 'STG_CUSTOMER_CALLS')

  2. What did you expect to see?

    Successful write to table

  3. Can you set logging to DEBUG and collect the logs?

   >>> write_pandas(snowflake_hook.get_conn(), calls_df[['CUSTOMER_ID', 'RELATIVE_PATH', 'TRANSCRIPT']], 'STG_CUSTOMER_CALLS')
[2023-04-24 11:16:00,757] {base.py:73} INFO - Using connection ID 'snowflake_default' for task execution.
2023-04-24 11:16:00,758 - MainThread connection.py:282 - __init__() - INFO - Snowflake Connector for Python Version: 3.0.3, Python Version: 3.9.16, Platform: Linux-5.15.49-linuxkit-aarch64-with-glibc2.31
[2023-04-24 11:16:00,758] {connection.py:282} INFO - Snowflake Connector for Python Version: 3.0.3, Python Version: 3.9.16, Platform: Linux-5.15.49-linuxkit-aarch64-with-glibc2.31
2023-04-24 11:16:00,760 - MainThread connection.py:541 - connect() - DEBUG - connect
[2023-04-24 11:16:00,760] {connection.py:541} DEBUG - connect
2023-04-24 11:16:00,760 - MainThread connection.py:852 - __config() - DEBUG - __config
[2023-04-24 11:16:00,760] {connection.py:852} DEBUG - __config
2023-04-24 11:16:00,760 - MainThread connection.py:989 - __config() - INFO - This connection is in OCSP Fail Open Mode. TLS Certificates would be checked for validity and revocation status. Any other Certificate Revocation related exceptions or OCSP Responder failures would be disregarded in favor of connectivity.
[2023-04-24 11:16:00,760] {connection.py:989} INFO - This connection is in OCSP Fail Open Mode. TLS Certificates would be checked for validity and revocation status. Any other Certificate Revocation related exceptions or OCSP Responder failures would be disregarded in favor of connectivity.
2023-04-24 11:16:00,760 - MainThread converter.py:150 - __init__() - DEBUG - use_numpy: False
[2023-04-24 11:16:00,760] {converter.py:150} DEBUG - use_numpy: False
2023-04-24 11:16:00,761 - MainThread connection.py:734 - __open_connection() - DEBUG - REST API object was created: ###############.prod3.us-west-2.aws.snowflakecomputing.com:443
[2023-04-24 11:16:00,761] {connection.py:734} DEBUG - REST API object was created: ###############.prod3.us-west-2.aws.snowflakecomputing.com:443
2023-04-24 11:16:00,761 - MainThread _auth.py:171 - authenticate() - DEBUG - authenticate
[2023-04-24 11:16:00,761] {_auth.py:171} DEBUG - authenticate
2023-04-24 11:16:00,762 - MainThread _auth.py:201 - authenticate() - DEBUG - assertion content: *********
[2023-04-24 11:16:00,762] {_auth.py:201} DEBUG - assertion content: *********
2023-04-24 11:16:00,762 - MainThread _auth.py:204 - authenticate() - DEBUG - account=###############
[2023-04-24 11:16:00,762] {_auth.py:204} DEBUG - account=###############
2023-04-24 11:16:00,762 - MainThread _auth.py:237 - authenticate() - DEBUG - body['data']: {'CLIENT_APP_ID': 'PythonConnector', 'CLIENT_APP_VERSION': '3.0.3', 'SVN_REVISION': None, 'ACCOUNT_NAME': '###############', 'LOGIN_NAME': '###############', 'CLIENT_ENVIRONMENT': {'APPLICATION': 'AIRFLOW', 'OS': 'Linux', 'OS_VERSION': 'Linux-5.15.49-linuxkit-aarch64-with-glibc2.31', 'PYTHON_VERSION': '3.9.16', 'PYTHON_RUNTIME': 'CPython', 'PYTHON_COMPILER': 'GCC 10.2.1 20210110', 'OCSP_MODE': 'FAIL_OPEN', 'TRACING': 10, 'LOGIN_TIMEOUT': 120, 'NETWORK_TIMEOUT': None}, 'SESSION_PARAMETERS': {'CLIENT_PREFETCH_THREADS': 4}}
[2023-04-24 11:16:00,762] {_auth.py:237} DEBUG - body['data']: {'CLIENT_APP_ID': 'PythonConnector', 'CLIENT_APP_VERSION': '3.0.3', 'SVN_REVISION': None, 'ACCOUNT_NAME': '###############', 'LOGIN_NAME': '###############', 'CLIENT_ENVIRONMENT': {'APPLICATION': 'AIRFLOW', 'OS': 'Linux', 'OS_VERSION': 'Linux-5.15.49-linuxkit-aarch64-with-glibc2.31', 'PYTHON_VERSION': '3.9.16', 'PYTHON_RUNTIME': 'CPython', 'PYTHON_COMPILER': 'GCC 10.2.1 20210110', 'OCSP_MODE': 'FAIL_OPEN', 'TRACING': 10, 'LOGIN_TIMEOUT': 120, 'NETWORK_TIMEOUT': None}, 'SESSION_PARAMETERS': {'CLIENT_PREFETCH_THREADS': 4}}
2023-04-24 11:16:00,763 - MainThread _auth.py:247 - authenticate() - DEBUG - Timeout set to 120
[2023-04-24 11:16:00,763] {_auth.py:247} DEBUG - Timeout set to 120
2023-04-24 11:16:00,765 - MainThread retry.py:351 - from_int() - DEBUG - Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
[2023-04-24 11:16:00,765] {retry.py:351} DEBUG - Converted retries value: 1 -> '\x1b[01mRetry(total=1, connect=None, read=None, redirect=None, status=None)\x1b[22m'
2023-04-24 11:16:00,765 - MainThread retry.py:351 - from_int() - DEBUG - Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
[2023-04-24 11:16:00,765] {retry.py:351} DEBUG - Converted retries value: 1 -> '\x1b[01mRetry(total=1, connect=None, read=None, redirect=None, status=None)\x1b[22m'
2023-04-24 11:16:00,765 - MainThread network.py:1162 - _use_requests_session() - DEBUG - Session status for SessionPool '###############.prod3.us-west-2.aws.snowflakecomputing.com', SessionPool 1/1 active sessions
[2023-04-24 11:16:00,765] {network.py:1162} DEBUG - Session status for SessionPool '###############.prod3.us-west-2.aws.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-04-24 11:16:00,765 - MainThread network.py:847 - _request_exec_wrapper() - DEBUG - remaining request timeout: 120, retry cnt: 1
[2023-04-24 11:16:00,765] {network.py:847} DEBUG - remaining request timeout: 120, retry cnt: 1
2023-04-24 11:16:00,765 - MainThread network.py:828 - add_request_guid() - DEBUG - Request guid: 455606b6-0b5c-4b24-8ace-1aed2233dbb3
[2023-04-24 11:16:00,765] {network.py:828} DEBUG - Request guid: 455606b6-0b5c-4b24-8ace-1aed2233dbb3
2023-04-24 11:16:00,765 - MainThread network.py:1021 - _request_exec() - DEBUG - socket timeout: 60
[2023-04-24 11:16:00,765] {network.py:1021} DEBUG - socket timeout: 60
2023-04-24 11:16:00,768 - MainThread connectionpool.py:1003 - _new_conn() - DEBUG - Starting new HTTPS connection (1): ###############.prod3.us-west-2.aws.snowflakecomputing.com:443
[2023-04-24 11:16:00,768] {connectionpool.py:1003} DEBUG - Starting new HTTPS connection (1): ###############.prod3.us-west-2.aws.snowflakecomputing.com:443
2023-04-24 11:16:01,370 - MainThread ssl_wrap_socket.py:81 - ssl_wrap_socket_with_ocsp() - DEBUG - OCSP Mode: FAIL_OPEN, OCSP response cache file name: None
[2023-04-24 11:16:01,370] {ssl_wrap_socket.py:81} DEBUG - OCSP Mode: FAIL_OPEN, OCSP response cache file name: None
2023-04-24 11:16:01,372 - MainThread ocsp_snowflake.py:523 - reset_ocsp_response_cache_uri() - DEBUG - ocsp_response_cache_uri: file:///root/.cache/snowflake/ocsp_response_cache.json
[2023-04-24 11:16:01,372] {ocsp_snowflake.py:523} DEBUG - ocsp_response_cache_uri: file:///root/.cache/snowflake/ocsp_response_cache.json
2023-04-24 11:16:01,373 - MainThread ocsp_snowflake.py:526 - reset_ocsp_response_cache_uri() - DEBUG - OCSP_VALIDATION_CACHE size: 203
[2023-04-24 11:16:01,373] {ocsp_snowflake.py:526} DEBUG - OCSP_VALIDATION_CACHE size: 203
2023-04-24 11:16:01,373 - MainThread ocsp_snowflake.py:333 - reset_ocsp_dynamic_cache_server_url() - DEBUG - OCSP response cache server is enabled: http://ocsp.snowflakecomputing.com/ocsp_response_cache.json
[2023-04-24 11:16:01,373] {ocsp_snowflake.py:333} DEBUG - OCSP response cache server is enabled: http://ocsp.snowflakecomputing.com/ocsp_response_cache.json
2023-04-24 11:16:01,374 - MainThread ocsp_snowflake.py:346 - reset_ocsp_dynamic_cache_server_url() - DEBUG - OCSP dynamic cache server RETRY URL: None
[2023-04-24 11:16:01,374] {ocsp_snowflake.py:346} DEBUG - OCSP dynamic cache server RETRY URL: None
2023-04-24 11:16:01,374 - MainThread ocsp_snowflake.py:956 - validate() - DEBUG - validating certificate: ###############.prod3.us-west-2.aws.snowflakecomputing.com
[2023-04-24 11:16:01,374] {ocsp_snowflake.py:956} DEBUG - validating certificate: ###############.prod3.us-west-2.aws.snowflakecomputing.com
2023-04-24 11:16:01,376 - MainThread ocsp_asn1crypto.py:435 - extract_certificate_chain() - DEBUG - # of certificates: 4
[2023-04-24 11:16:01,376] {ocsp_asn1crypto.py:435} DEBUG - # of certificates: 4
2023-04-24 11:16:01,381 - MainThread ocsp_asn1crypto.py:440 - extract_certificate_chain() - DEBUG - subject: OrderedDict([('common_name', '*.prod3.us-west-2.snowflakecomputing.com')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('common_name', 'Amazon RSA 2048 M01')])
[2023-04-24 11:16:01,381] {ocsp_asn1crypto.py:440} DEBUG - subject: OrderedDict([('common_name', '*.prod3.us-west-2.snowflakecomputing.com')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('common_name', 'Amazon RSA 2048 M01')])
2023-04-24 11:16:01,383 - MainThread ocsp_asn1crypto.py:440 - extract_certificate_chain() - DEBUG - subject: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('common_name', 'Amazon RSA 2048 M01')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('common_name', 'Amazon Root CA 1')])
[2023-04-24 11:16:01,383] {ocsp_asn1crypto.py:440} DEBUG - subject: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('common_name', 'Amazon RSA 2048 M01')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('common_name', 'Amazon Root CA 1')])
2023-04-24 11:16:01,384 - MainThread ocsp_asn1crypto.py:440 - extract_certificate_chain() - DEBUG - subject: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('common_name', 'Amazon Root CA 1')]), issuer: OrderedDict([('country_name', 'US'), ('state_or_province_name', 'Arizona'), ('locality_name', 'Scottsdale'), ('organization_name', 'Starfield Technologies, Inc.'), ('common_name', 'Starfield Services Root Certificate Authority - G2')])
[2023-04-24 11:16:01,384] {ocsp_asn1crypto.py:440} DEBUG - subject: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('common_name', 'Amazon Root CA 1')]), issuer: OrderedDict([('country_name', 'US'), ('state_or_province_name', 'Arizona'), ('locality_name', 'Scottsdale'), ('organization_name', 'Starfield Technologies, Inc.'), ('common_name', 'Starfield Services Root Certificate Authority - G2')])
2023-04-24 11:16:01,388 - MainThread ocsp_asn1crypto.py:440 - extract_certificate_chain() - DEBUG - subject: OrderedDict([('country_name', 'US'), ('state_or_province_name', 'Arizona'), ('locality_name', 'Scottsdale'), ('organization_name', 'Starfield Technologies, Inc.'), ('common_name', 'Starfield Services Root Certificate Authority - G2')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'Starfield Technologies, Inc.'), ('organizational_unit_name', 'Starfield Class 2 Certification Authority')])
[2023-04-24 11:16:01,388] {ocsp_asn1crypto.py:440} DEBUG - subject: OrderedDict([('country_name', 'US'), ('state_or_province_name', 'Arizona'), ('locality_name', 'Scottsdale'), ('organization_name', 'Starfield Technologies, Inc.'), ('common_name', 'Starfield Services Root Certificate Authority - G2')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'Starfield Technologies, Inc.'), ('organizational_unit_name', 'Starfield Class 2 Certification Authority')])
2023-04-24 11:16:01,392 - MainThread ocsp_asn1crypto.py:463 - create_pair_issuer_subject() - DEBUG - not found issuer_der: OrderedDict([('country_name', 'US'), ('organization_name', 'Starfield Technologies, Inc.'), ('organizational_unit_name', 'Starfield Class 2 Certification Authority')])
[2023-04-24 11:16:01,392] {ocsp_asn1crypto.py:463} DEBUG - not found issuer_der: OrderedDict([('country_name', 'US'), ('organization_name', 'Starfield Technologies, Inc.'), ('organizational_unit_name', 'Starfield Class 2 Certification Authority')])
2023-04-24 11:16:01,395 - MainThread ocsp_snowflake.py:722 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('common_name', '*.prod3.us-west-2.snowflakecomputing.com')])
[2023-04-24 11:16:01,395] {ocsp_snowflake.py:722} DEBUG - hit cache for subject: OrderedDict([('common_name', '*.prod3.us-west-2.snowflakecomputing.com')])
2023-04-24 11:16:01,396 - MainThread ocsp_asn1crypto.py:233 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:41:06+00:00
[2023-04-24 11:16:01,396] {ocsp_asn1crypto.py:233} DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:41:06+00:00
2023-04-24 11:16:01,400 - MainThread ocsp_snowflake.py:722 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('common_name', 'Amazon RSA 2048 M01')])
[2023-04-24 11:16:01,400] {ocsp_snowflake.py:722} DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('common_name', 'Amazon RSA 2048 M01')])
2023-04-24 11:16:01,402 - MainThread ocsp_asn1crypto.py:233 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:45:34+00:00
[2023-04-24 11:16:01,402] {ocsp_asn1crypto.py:233} DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:45:34+00:00
2023-04-24 11:16:01,402 - MainThread ocsp_snowflake.py:722 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('common_name', 'Amazon Root CA 1')])
[2023-04-24 11:16:01,402] {ocsp_snowflake.py:722} DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('organization_name', 'Amazon'), ('common_name', 'Amazon Root CA 1')])
2023-04-24 11:16:01,404 - MainThread ocsp_asn1crypto.py:233 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-06-22 07:00:00+00:00
[2023-04-24 11:16:01,404] {ocsp_asn1crypto.py:233} DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-06-22 07:00:00+00:00
2023-04-24 11:16:01,404 - MainThread ocsp_snowflake.py:722 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('state_or_province_name', 'Arizona'), ('locality_name', 'Scottsdale'), ('organization_name', 'Starfield Technologies, Inc.'), ('common_name', 'Starfield Services Root Certificate Authority - G2')])
[2023-04-24 11:16:01,404] {ocsp_snowflake.py:722} DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('state_or_province_name', 'Arizona'), ('locality_name', 'Scottsdale'), ('organization_name', 'Starfield Technologies, Inc.'), ('common_name', 'Starfield Services Root Certificate Authority - G2')])
2023-04-24 11:16:01,406 - MainThread ocsp_snowflake.py:1013 - _validate() - DEBUG - ok
[2023-04-24 11:16:01,406] {ocsp_snowflake.py:1013} DEBUG - ok
2023-04-24 11:16:01,857 - MainThread connectionpool.py:456 - _make_request() - DEBUG - https://###############.prod3.us-west-2.aws.snowflakecomputing.com:443 "POST /session/v1/login-request?request_id=9ea6b8c2-7d03-4f64-90d4-69119c298516&databaseName=###############&schemaName=###############&warehouse=###############&roleName=###############&request_guid=455606b6-0b5c-4b24-8ace-1aed2233dbb3 HTTP/1.1" 200 1542
[2023-04-24 11:16:01,857] {connectionpool.py:456} DEBUG - https://###############.prod3.us-west-2.aws.snowflakecomputing.com:443 "POST /session/v1/login-request?request_id=9ea6b8c2-7d03-4f64-90d4-69119c298516&databaseName=###############&schemaName=###############&warehouse=###############&roleName=###############&request_guid=455606b6-0b5c-4b24-8ace-1aed2233dbb3 HTTP/1.1" 200 1542
2023-04-24 11:16:01,859 - MainThread network.py:1047 - _request_exec() - DEBUG - SUCCESS
[2023-04-24 11:16:01,859] {network.py:1047} DEBUG - SUCCESS
2023-04-24 11:16:01,860 - MainThread network.py:1167 - _use_requests_session() - DEBUG - Session status for SessionPool '###############.prod3.us-west-2.aws.snowflakecomputing.com', SessionPool 0/1 active sessions
[2023-04-24 11:16:01,860] {network.py:1167} DEBUG - Session status for SessionPool '###############.prod3.us-west-2.aws.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-04-24 11:16:01,860 - MainThread network.py:727 - _post_request() - DEBUG - ret[code] = None, after post request
[2023-04-24 11:16:01,860] {network.py:727} DEBUG - ret[code] = None, after post request
2023-04-24 11:16:01,860 - MainThread _auth.py:366 - authenticate() - DEBUG - completed authentication
[2023-04-24 11:16:01,860] {_auth.py:366} DEBUG - completed authentication
2023-04-24 11:16:01,860 - MainThread _auth.py:413 - authenticate() - DEBUG - token = ******
[2023-04-24 11:16:01,860] {_auth.py:413} DEBUG - token = ******
2023-04-24 11:16:01,860 - MainThread _auth.py:416 - authenticate() - DEBUG - master_token = ******
[2023-04-24 11:16:01,860] {_auth.py:416} DEBUG - master_token = ******
2023-04-24 11:16:01,861 - MainThread _auth.py:420 - authenticate() - DEBUG - id_token = NULL
[2023-04-24 11:16:01,861] {_auth.py:420} DEBUG - id_token = NULL
2023-04-24 11:16:01,861 - MainThread _auth.py:424 - authenticate() - DEBUG - mfa_token = NULL
[2023-04-24 11:16:01,861] {_auth.py:424} DEBUG - mfa_token = NULL
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/snowflake/connector/pandas_tools.py", line 176, in write_pandas
    isinstance(df.index, pandas.RangeIndex)
  File "/usr/local/lib/python3.9/site-packages/snowflake/connector/options.py", line 44, in __getattr__
    raise errors.MissingDependencyError(self._dep_name)
snowflake.connector.errors.MissingDependencyError: Missing optional dependency: pandas
@mpgreg
Copy link
Author

mpgreg commented Apr 24, 2023

Confirmed that this error does not occur with connector 3.0.2

@sfc-gh-mkeller
Copy link
Collaborator

Confirmed that this error does not occur with connector 3.0.2

Oh, that's great

@sfc-gh-mkeller sfc-gh-mkeller changed the title write_pandas "Error: Missing optional dependency: pandas" despite having pandas installed. SNOW-800181: write_pandas "Error: Missing optional dependency: pandas" despite having pandas installed. Apr 25, 2023
@mpgreg
Copy link
Author

mpgreg commented Apr 25, 2023

Sorry @sfc-gh-mkeller I should be more specific. This error DOES occur with the latest 3.0.3 but does NOT occur with 3.0.2. Seems to be a regression somewhere.

@Jmaharman
Copy link

Can this be re-opened @sfc-gh-mkeller? As @mpgreg confirmed it is not a problem in 3.0.2, but it is in 3.0.3 and up (including 3.1.0 and the latest preview)

@alexpti4ka
Copy link

alexpti4ka commented May 14, 2024 •
edited
Loading

Hey there! Having the same issue on macbook m1 python 3.12 conda env. Tried to rebuild env, reinstall anaconda itself, downgraded to Python 3.11 to meet snowflake connector 3.0.2 requirements. Indeed the Error: Missing optional dependency: pandas is gone, however another error appear at the exactly same place: ArrowMemoryError: malloc of size 1024 failed. Any clues on this issue?

@gabrielclimb
Copy link

Getting the same error with version 3.10.1, do someone knows why this is happening? This issue should be reopened i think

@MaiaPaulo
Copy link

Getting same error with python 3.10.8 and snowflake connector 3.12.0.

@davecthomas
Copy link

I'm getting it with python 3.12.3 and snowflake-connector-python 3.12.1

@alefrancia
Copy link

In Python 3.13.2 and Snowflake connector 3.12.3 worked for me after executing:
pip install "snowflake-connector-python[pandas]" pandas numpy pyarrow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug needs triage
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@Jmaharman @davecthomas @mpgreg @gabrielclimb @MaiaPaulo @sfc-gh-mkeller @alefrancia @alexpti4ka