ImportError: DLL load failed: The specified module could not be found on Windows

[atodorov]

What went wrong?

Trying to use gssapi on Windows seems to fail:

Traceback (most recent call last):
  File "setup.py", line 4, in <module>
    from tcms_api.xmlrpc import __version__
  File "C:\projects\tcms-api\tcms_api\__init__.py", line 72, in <module>
    from .xmlrpc import TCMSXmlrpc, TCMSKerbXmlrpc
  File "C:\projects\tcms-api\tcms_api\xmlrpc.py", line 11, in <module>
    import gssapi
  File "C:\Python36-x64\lib\site-packages\gssapi\__init__.py", line 29, in <module>
    from gssapi.raw.types import NameType, RequirementFlag, AddressType  # noqa
  File "C:\Python36-x64\lib\site-packages\gssapi\raw\__init__.py", line 50, in <module>
    from gssapi.raw.creds import *  # noqa
ImportError: DLL load failed: The specified module could not be found.

How do we reproduce?

import gssapi seems to be enough

Component versions (python-gssapi, Kerberos, OS / distro, etc.)

gssapi-1.6.2, pip install - downloads the wheel package, Windows.

Logs at:
https://ci.appveyor.com/project/atodorov/tcms-api/builds/31627921/job/3tbk1ike6er25tmd

Feels like something is missing but I don't know what. Works perfectly fine on Linux.

Update: looks like #188 but I couldn't figure out what the solution was.

[atodorov]

More info:

I tried this manually on a Windows Server 2019 system:

1. Install Pythin 3.8 64bit
2. Install MIT Kerberos for Windows, 64bit, Complete installation
3. Create a venv and pip install gssapi

The wheel package installs but import still errors out in the same way.

How are the supplied wheel packages built and what libraries do they require ?

[frozencemetery]

#188 was solved by installing krb5. Can you confirm you did that? The latest KfW should have cred_store support.

CC @aiudirog

[aiudirog]

The DLLs for KfW are probably what's missing. I really wish DLL errors would actually tell you which one haha. If you haven't already, install KfW and then make sure it is in your PATH. I can't read, you did install it, so just add it to your PATH. It should be enough to just add C:\Program Files\MIT\Kerberos\bin if it isn't.

@frozencemetery, it probably would be a good idea to check for the DLLs in __init__.py and if they aren't found, do a brief search for them. Dealing with PATH issues isn't fun, since Windows doesn't have standard bin/lib folders like Linux.

[aiudirog]

After looking at your project (tcms-api), I would warn you to be careful. Windows SSPI (which is what winkerberos is for) doesn't have the same requirements as GSSAPI and compatibility can be an issue.

This repository is specifically for providing the GSSAPI spec in Python and we use MIT Kerberos for Windows, not the built in Windows SSPI. This makes this library useful for connecting to GSSAPI services (like Kerberos secured Hadoop) from Windows where they (probably) may not be configured in a way that plays nice with Windows SSPI.

I added the Windows support to interface with Kerberos secured Livy, HBase, and Phoenix. I was previously using a custom pykerberos because winkerberos always yielded GSSAPI mechanism errors from the server side, but building it required a bit too many hacks.

[atodorov]

It should be enough to just add C:\Program Files\MIT\Kerberos\bin if it isn't.

I will try this out. I don't remember if I did. I will follow up with the updates in a few days.

This repository is specifically for providing the GSSAPI spec in Python and we use MIT Kerberos for Windows, not the built in Windows SSPI.

I think this should be fine in my case. The tcms-api project is a client side library talking to a Linux web server, which is supposed to be secured via MIT Kerberos. Thanks for pointing this out, it's probably worth documenting on my side.

[frozencemetery]

Hi @atodorov, any updates here?

[atodorov]

Nope. As I said, I will give it a try and comment when I have the time.

[atodorov]

I did some digging today and here's what I got:

(my.venv) C:\Users\atodorov>pip freeze
decorator==4.4.2
gssapi==1.6.2
six==1.14.0

(my.venv) C:\Users\atodorov>echo %PATH%
C:\Users\atodorov\my.venv\Scripts;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\windows\System32\OpenSSH\;C:\Program Files\MIT\Kerberos\bin;C:\Users\atodorov\AppData\Local\Programs\Python\Python38\Scripts\;C:\Users\atodorov\AppData\Local\Programs\Python\Python38\;C:\Users\atodorov\AppData\Local\Microsoft\WindowsApps;

(my.venv) C:\Users\atodorov>python
Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import gssapi
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\atodorov\my.venv\lib\site-packages\gssapi\__init__.py", line 29, in <module>
    from gssapi.raw.types import NameType, RequirementFlag, AddressType  # noqa
  File "C:\Users\atodorov\my.venv\lib\site-packages\gssapi\raw\__init__.py", line 50, in <module>
    from gssapi.raw.creds import *  # noqa
ImportError: DLL load failed while importing creds: The specified module could not be found.
>>> exit()

(my.venv) C:\Users\atodorov>dir "C:\Program Files\MIT\Kerberos\bin"
 Volume in drive C is Windows
 Volume Serial Number is 728C-CE4D

 Directory of C:\Program Files\MIT\Kerberos\bin

04/02/2020  11:01 AM    <DIR>          .
04/02/2020  11:01 AM    <DIR>          ..
06/20/2016  06:43 PM           161,280 ccapiserver.exe
06/20/2016  06:31 PM            13,824 comerr32.dll
06/20/2016  06:43 PM            16,896 comerr64.dll
06/20/2016  06:45 PM            23,552 gss-client.exe
06/20/2016  06:45 PM            22,528 gss-server.exe
06/20/2016  06:31 PM           348,160 gssapi32.dll
06/20/2016  06:43 PM           479,232 gssapi64.dll
06/20/2016  06:31 PM            41,984 k5sprt32.dll
06/20/2016  06:43 PM            58,368 k5sprt64.dll
06/20/2016  06:45 PM            12,800 kcpytkt.exe
06/20/2016  06:45 PM            12,800 kdeltkt.exe
06/20/2016  06:45 PM            11,264 kdestroy.exe
06/20/2016  06:45 PM            28,160 kinit.exe
06/20/2016  06:45 PM            25,088 klist.exe
06/20/2016  06:45 PM            11,264 kpasswd.exe
06/20/2016  06:31 PM         1,109,504 krb5_32.dll
06/20/2016  06:43 PM         1,458,688 krb5_64.dll
06/20/2016  06:31 PM            89,600 krbcc32.dll
06/20/2016  06:43 PM           112,128 krbcc64.dll
06/20/2016  06:45 PM            11,776 kswitch.exe
06/20/2016  06:45 PM            16,384 kvno.exe
06/20/2016  06:31 PM           141,312 leashw32.dll
06/20/2016  06:43 PM           176,640 leashw64.dll
06/20/2016  06:43 PM           123,468 MIT Kerberos.chm
06/20/2016  06:45 PM           818,688 MIT Kerberos.exe
06/20/2016  06:45 PM            12,288 mit2ms.exe
06/20/2016  06:45 PM            13,824 ms2mit.exe
06/20/2016  06:29 PM            28,160 wshelp32.dll
06/20/2016  06:41 PM            36,864 wshelp64.dll
06/20/2016  06:31 PM            37,376 xpprof32.dll
06/20/2016  06:43 PM            50,176 xpprof64.dll
              31 File(s)      5,504,076 bytes
               2 Dir(s)  126,043,271,168 bytes free

(my.venv) C:\Users\atodorov>

C:\Program Files\MIT\Kerberos\bin is in PATH but for some reason I still get the DLL failure.

[atodorov]

It's a 64bit system so I suspect it should load gssapi64.dll but I don't see anything to resemble that in setup.py. I also tried copying gssapi64.dll under gssapi.dll with no luck.

[aiudirog]

I really don't understand why it can't tell us which DLL, haha.

The good news is I was able to reproduce it in a VM and it seems to be a 3.8 specific issue where it doesn't search the PATH for DLLs. I was able to get it working fine if I preloaded gssapi64.dll by it's full path using ctypes.

The bad news is that once I learned this, I quickly found that not searching the path is a security feature introduced for 3.8 as a part of dropping Windows 7 support: https://bugs.python.org/issue36085

Looks like on 3.8+ we need to use os.add_dll_directory. I'll put together a PR today that will just update the Windows auto config work we already did.