Azure Web Apps fail to install Azure-Storage 0.33.0

[4c74356b41]

Hello, issue is easily reproduceable:
Create a flask web app add azure-storage to requirements.txt and try to push it to the Web App... It will fail with 2.7 and 3.4. See attached log file.

Running setup.py install for cryptography
BLA-BLA-BLA
remote: distutils.errors.DistutilsError: Setup script exited with error: Unable to find vcvarsall.bat
remote:
remote: ----------------------------------------
remote: Cleaning up...
remote: .........................
remote: Command D:\home\site\wwwroot\env\Scripts\python.exe -c "import setuptools, tokenize;__file__='D:\\home\\site\\wwwroot\\env\\build\\cryptography\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record D:\local\Temp\pip-1tk19zep-record\install-record.txt --single-version-externally-managed --compile --install-headers D:\home\site\wwwroot\env\include\site\python3.4 failed with error code 1 in D:\home\site\wwwroot\env\build\cryptography
remote: Storing debug log for failure in D:\home\pip\pip.log
remote: An error has occurred during web site deployment.

You owe me a couple of days of my life :(
So it works for any version without that stupid broken cryptography.

fail.txt

[rambho]

Hi @4c74356b41,
Sorry you are having trouble with this. The reason you're getting this error is because the cryptography package needs to be built natively, and it is looking for a compiler (VS) to start the build process.

If have Visual Studio/C++ already installed,
Type set in command prompt, it may be the case these environmental vars are not populated (depending on your version):

VS110COMNTOOLS=C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\Tools\
VS120COMNTOOLS=C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\Tools\
VS140COMNTOOLS=C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools\

You can set them to your VS installation path manually. However, the setuptools we use should be able to autodetect your compiler, so first verify you are using the latest version (and if not, retry the deployment):
pip install --upgrade setuptools

If you still have trouble deploying our storage library with cryptography or don't have a compiler installed, installing the following should take care of it:
https://www.microsoft.com/en-us/download/details.aspx?id=44266

[4c74356b41]

Hello Rambo (sounds extremely cool), I'm talking about Azure WebApps only, this stuff works on my PC no issues. Only when I push my code to Azure Web Apps it fails to install it.

[lmazuel]

Hi @4c74356b41

For some reason, pip on WebApp didn't choose to download the wheel file for cryptography on PyPI. I guess the introspection of pip wasn't able to get the "win32" or "win_amd64" tags on this system.

I suggest you to try the options on this document:
https://azure.microsoft.com/en-us/documentation/articles/web-sites-python-configure/#troubleshooting---package-installation

IOW, either download cryptography-1.5-cp34-cp34m-win32.whl, put it in a "wheelhouse" folder and use --find-links or a deployment script using easy_install

As the previous document says, you will not be able to compile cryptography directly on the WebApp server, so you need a way to force the install of the wheel file.

I hope it helps,

[lmazuel]

I think you will find this article interesting too:
https://blogs.msdn.microsoft.com/pythonengineering/2016/08/04/upgrading-python-on-azure-app-service/

[huguesv]

We've had customers that hit a bug in pip where it can't use/find wheels even when they are available. It's fixed in the latest version of pip, but unfortunately the pip on webapps is not the latest.

Not sure if that's what's happening with the crypto package or not.

[4c74356b41]

Look folks, I've worked around that, I don't really need Azure Storage 0.33.0 exactly. I'm generally telling you to forward this ISSUE to the web app team, because right now this thing is broken (as is).
But! I really thank you for all those links, I'll read through them!

oh yes, that article is truly epic, thanks!

[rambho]

Thanks! I'll consider this resolved, pending updates from the WebApps team.

[4c74356b41]

works for me, thanks!

[mawah]

FWIW, I've found a workaround for installing the cryptography package on Azure WebJobs.

pip will only install wheels with an ABI code of "none" during Web App deployment:

import pip
print(pip.pep425tags.get_supported())
[09/28/2016 17:08:30 > 775106: INFO] [('cp34', 'none', 'win32'), ('cp34', 'none', 'any'), ('cp3', 'none', 'any'), ('cp33', 'none', 'any'), ('cp32', 'none', 'any'), ('cp31', 'none', 'any'), ('cp30', 'none', 'any'), ('py34', 'none', 'any'), ('py3', 'none', 'any'), ('py33', 'none', 'any'), ('py32', 'none', 'any'), ('py31', 'none', 'any'), ('py30', 'none', 'any')]

Many modern packages in PyPI -- e.g. cryptography==1.5.2 -- don't provide wheels with this ABI tag. Fortunately, it appears all will be well if the wheel file is renamed to have the "none" ABI tag.

I'll assume that you are already deploying your Web App from a git repo. Here are the steps to get cryptography installed:

1. Download the appropriate wheel for your Web App's Python version and the win32 platform from the cryptography page on PyPI into your Web App's repo.
2. Rename the file to change the ABI tag to "none", e.g.
   cryptography-1.5.2-cp34-cp34m-win32.whl =>cryptography-1.5.2-cp34-none-win32.whl
3. Repeat steps 1 and 2 for the cffi package.
4. Add entries at the top of the requirements,txt file containing the filenames of the wheels.
5. Deploy via git.

I believe that'd be sufficient, though there are a few more steps I have taken (generating and including wheels for azure-common, azure-nspkg, and azure-storage via pip wheel <package name>, including other dependencies as wheels) that might also be necessary.

[aarsan]

Couldn't get any of this to work with 64bit, 3.4. Downgraded azure-storage to 0.32.0 and all is good. Hope to see this fixed soon.

[smereczynski]

It is not resolved and it should not be closed. It is an issue between Storage SDK for Python team and Web Apps team (Web Apps PG?). It should be fixed as soon as possible because it is real production blocker. If help is needed, just inform us how to help or where to ask directly.

[zooba]

The best solution is to upgrade the version of Python on app service by following the instructions at https://aka.ms/PythonOnAppService.

The version that is preinstalled is very old, but for backwards-compatibility reasons we cannot upgrade it, so instead we are deprecating use of the preinstalled version and recommending everyone use the site extensions described at the link above.

And yes, we intend to improve this process so it is simpler in the future, but until the App Service team is able to help us out (right now we have to fix it with only publicly-available APIs/features), there are going to be a few extra manual steps involved. Luckily, most of those just make the process more transparent, so it isn't necessarily a bad thing, but it does complicate the process somewhat.

[tadashiee]

I encountered this problem, too. But, I found the workarround of this problem .
I describe our environment, problem, primary cause, and workarround below:
[[ Environment ]]
** Azure Web Apps **
python 3.4.1 (32bit)
** My development PC **

• Windows 7 (32bit)
• python 3.4.1 (32bit)
  pip (1.5.6)
  setuptools (2.1)
• wheel packages in the 'wheelhouse' directory
  azure_common-1.1.4-py2.py3-none-any.whl
  azure_nspkg-1.0.0-py2.py3-none-any.whl
  azure_storage-0.33.0-py3-none-any.whl
  cffi-1.9.1-cp34-cp34m-win32.whl
  cryptography-1.7.1-cp34-cp34m-win32.whl
  idna-2.2-py2.py3-none-any.whl
  pyasn1-0.1.9-py2.py3-none-any.whl
  pycparser-2.17-py2.py3-none-any.whl
  python_dateutil-2.6.0-py2.py3-none-any.whl
  requests-2.12.4-py2.py3-none-any.whl
  setuptools-32.3.1-py2.py3-none-any.whl
  six-1.10.0-py2.py3-none-any.whl
• description of requirements.txt
  +-------------------------------------------
  |--no-index --find-links wheelhouse
  |azure-nspkg==1.0.0
  |azure-common==1.1.4
  | :
  | (omit)
  | :
  +-------------------------------------------

[[ Problem]]
The wheel package registered in the git repository fails in the installation on the Azure environment.

[[ Primary cause ]]
on Python 3.4 of Azure Web Apps, version of pip is 1.5.6 .
pip (1.5.6) is not able to recognize '--find-links' option which was written in the top of requirements.txt file.

[[ Workarround ]]
Upgrade pip on Azure Web Apps, using 'deploy.cmd' file.

[[ Detail ]]
At first, please refer to following URL.
https://docs.microsoft.com/en-us/azure/app-service-web/web-sites-python-configure

I got default '.deployment' file and 'deploy.cmd' file from Azure Web Apps Kudu Service.
** How to get these files **

1. Login to azure dashboard
2. Select Web Apps instance
3. Select [Advanced Tools] and view new page which title is 'Kudu Services'
4. Select [Tools] at menu bar, and choose [Download deployment script]
   --> Save 'deploymentscript.zip' file into local.
5. Unzip deploymentscript.zip

and, please edit the vicinity of the following lines in 'deploy.cmd' .
(before)
echo Pip install requirements.
env\scripts\pip install -r requirements.txt
(after)
echo Upgrade pip package.
env\scripts\python -m pip install pip --upgrade
IF !ERRORLEVEL! NEQ 0 goto error
echo Pip install requirements.
env\scripts\pip install -r requirements.txt

(* I also upgraded setuptools )

finally, please copy '.deployment' and 'deploy.cmd' file onto top folder of GIT project,
then commit/push it.

By this workarrownd, I successfully installed wheel format packages into Azure Web Apps.
It would be greatly appreciated if this workarround is useful for you.
(Please pardon a poor English expression)