Build fails on Windows with Python 3.5, VS 2015

[chrullrich]

> py -3 setup.py build_ext

[...]
File "C:\Program Files\Python35\lib\site-packages\setuptools\command\build_ext.py", line 49, in run
  _build_ext.run(self)
File "C:\Program Files\Python35\lib\distutils\command\build_ext.py", line 338, in run
  self.build_extensions()
File "C:\Program Files\Python35\lib\distutils\command\build_ext.py", line 447, in build_extensions
  self._build_extensions_serial()
File "C:\Program Files\Python35\lib\distutils\command\build_ext.py", line 472, in _build_extensions_serial
  self.build_extension(ext)
File "setup.py", line 308, in build_extension
  '-outputresource:%s;2' % ext_path])
File "C:\Program Files\Python35\lib\distutils\_msvccompiler.py", line 513, in spawn
  os.environ['path'] = self._paths
AttributeError: 'MSVCCompiler' object has no attribute '_paths'

I managed to fix this by calling self.initialize() at the top of spawn() (see last item in the traceback):

 def spawn(self, cmd):
+    if not self.initialized:
+        self.initialize()
     old_path = os.getenv('path')

It builds correctly now, but I suspect the bug may be in psycopg2 rather than distutils. Perhaps you are not supposed to call spawn() on an uninitialized instance?

[dvarrazzo]

Hi @chrullrich,

I don't use Windows so I can't test it myself; @jerickso is the authority on Windows support. He has recently suggested to move build using setuptools instead of the outdated distutils, see his patch to support wheels. Do you want to try if using his patch build works as expected?

[chrullrich]

That alone does not help. I patched setuptools support in myself for the same reason already.

[dvarrazzo]

If you and @jerickso agree on a patch that doesn't break compiling with other currently supported MSVC version I'm happy to merge it. Jason has the buildfarm for windows and what he says on the matter is The Verb for me :)

It is not really clear to me what pairs (MSVC, Python) version make sense on windows and what do we support (I believe it would be a welcome addition on the documentation).

I'm happy to add Python 3.5 support in the 2.6 branch, again as long as this doesn't break other supported versions; if it does, Py3.5 support would be added in master (i.e. psycopg 2.7) and we would document the minimal Py version we still support.

Jason, if it would hep, I'm thinking about deprecating Python 2.5 in psycopg 2.7 and only support 2.6 on. Would it make your life easier? Are there also Py3 version worth stop supporting for the same reason? I have seen a few libraries dropping 3.1 support but I don't really know the case for it.

[jerickso]

@chrullrich, before issuing the build_ext command, can you do the following in the same directory:
py -3 setup.py clean --all

I am able to duplicate what you see, but I need to do multiple builds without performing a clean. The cause is strange as the problem seems to be at this line in setup.py:

ext_path = self.get_ext_fullpath(extension.name)

which is returning a string of '_psycopg.cp35-win_amd64.pyd' instead of '_psycopg.pyd'.

The 'clean' command cleans out the directory and forces it through a different code path which is why it works.

I suspect like @chrullrich suggested, something now needs to get initialized with Py3.5. I'll spend some time next week looking into it.

[jerickso]

@dvarrazzo, I'm all for dropping support for Python 2.5. About a year ago, I stopped building binaries for Python 2.5, as the newer versions of Postgres would not build 'out of the box' with the older MSVC compiler used by Python 2.5. No one has asked about it since I stopped providing them.

As for Python 3.x versions, I think we are ok not supporting 3.1, but I am not much of a 3.x user. Python 3.2 was the first version in the 3.x line I made binaries for.

[chrullrich]

I think the reason it fails only with the second build is that it does not actually compile anything, just does the manifest manipulation. Most of the methods in _msvccompiler.MSVCCompiler call initialize() if it has not been called yet, but spawn() does not. So it works if compile() or something is called first on the same instance, but if all the build output is already there, that call is not needed.

My original report did not have that information because I must have had some build output still around; I think I only did clean, not --all.

[dvarrazzo]

Thank you for the collaboration, let me know what is the best thing to know: adding an explicit initialization step or forward it to setuptools maintainer.

[chrullrich]

This https://github.com/chrullrich/psycopg2/commit/b43f0398c8e1dfa7c5ad364c54dbeb1ea7596f21 fixes the issue for me. I won't promise it is the correct fix, and most importantly, I have not tested it on anything but Python 3.5/VS2015 because installing VS2010 for 3.4 and earlier is such a pain. @jerickso, can you perhaps give it a try?

[jerickso]

@chrullrich, I'll test out your fix in the next few days on the older versions of Python. On quick glance of the fix, I'm optimistic it will work with the earlier versions.

[jerickso]

@chrullrich, I was able to test your fix on the other versions of python and it did not affect them. So I'd say your patch is good.

[speedy250]

Any more developments on this thread here? Looking for help with python3.5 support, thanks!