Eager loading of cloud SDK dependencies can cause runtime error even if you don't plan to use that cloud

[remi-braun]

In your __init__ file, you try to import all the modules linked to all the cloud services you handle.

This makes all deps mandatory, e.g. with pip install cloudpathlib[s3]:

>>> import cloudpathlib
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\rbraun\AppData\Local\miniconda3\envs\test\Lib\site-packages\cloudpathlib\__init__.py", line 5, in <module>
    from .azure.azblobclient import AzureBlobClient
  File "C:\Users\rbraun\AppData\Local\miniconda3\envs\test\Lib\site-packages\cloudpathlib\azure\__init__.py", line 1, in <module>
    from .azblobclient import AzureBlobClient
  File "C:\Users\rbraun\AppData\Local\miniconda3\envs\test\Lib\site-packages\cloudpathlib\azure\azblobclient.py", line 22, in <module>
    from azure.core.credentials import AzureNamedKeyCredential
ImportError: cannot import name 'AzureNamedKeyCredential' from 'azure.core.credentials' (C:\Users\rbraun\AppData\Local\miniconda3\envs\test\Lib\site-packages\azure\core\credentials.py)

[jayqi]

@remi-braun that is not quite right. The dependencies are not mandatory.

You are correct that __init__.py will try to import the cloudpathlib.azure.azblobclient module, but that module tries to import the Azure dependencies in a try-except and handles if the Azure dependencies are not installed.

cloudpathlib/cloudpathlib/azure/azblobclient.py

Lines 20 to 43 in 9d5fa70

	try:
	from azure.core.exceptions import HttpResponseError, ResourceNotFoundError
	from azure.core.credentials import AzureNamedKeyCredential
	from azure.storage.blob import (
	BlobPrefix,
	BlobSasPermissions,
	BlobServiceClient,
	BlobProperties,
	ContentSettings,
	generate_blob_sas,
	)
	from azure.storage.blob._shared.authentication import (
	SharedKeyCredentialPolicy as BlobSharedKeyCredentialPolicy,
	)
	from azure.storage.filedatalake import DataLakeServiceClient, FileProperties
	from azure.storage.filedatalake._shared.authentication import (
	SharedKeyCredentialPolicy as DataLakeSharedKeyCredentialPolicy,
	)
	except ModuleNotFoundError:
	implementation_registry["azure"].dependencies_loaded = False

What it looks like is happening in your case is that you do have the Azure dependencies installed, but there is a version compatibility mismatch or something like that, so the particular name AzureNamedKeyCredential is expected to be in azure.core.credentials but isn't there.

Couple of things to look at:

• If you don't need Azure, try uninstalling the Azure dependencies from your local environment.
• If some other dependency installs them, see if updating the Azure packages helps.
• Alternatively, it's also possible that Azure had a breaking change we haven't caught up to, and we need to fix that. That would be a separate issue.

That said, it's a valid piece of feedback that the eager loading of dependencies means that something like a version mismatch can cause a runtime error even for something you're not planning on using. That's not ideal, and potentially something we should change.

[remi-braun]

Sorry, I haven't checked far enough! Thanks for the quicker than light answer 😄