Feature request: Support for Python 2.x

[chupocro]

I am opening an issue upon this suggestion by Email OAuth 2.0 Proxy author @simonrob from issue #34 thread where I asked this question:

One (strange) question: Would it be possible to port Email OAuth 2.0 Proxy to Python 2.7? I am asking because I have one Windows XP computer running 24/7 since 2006. and I'd like to run the proxy on that computer if possible. I have many specific programs (PCB design, CNC control, ...) because of which I am (on this computer) still using not only Window XP but Windows XP SP2 :-))

and the suggestion by author was:

I had a (very) quick look at what it would take to support Python 2.7. The changes are mostly superficial (module name differences; function signature changes, etc), but it is still a nontrivial task to be able to support both v2 and v3. I'll bear this in mind for potential future version. Please feel free to open an issue as a way to track this feature request.

I am sure there will be many more use-cases for running Email OAuth 2.0 Proxy with Python 2.x

[simonrob]

I had a look at this just now, and have committed a minimal working version into the python2 branch.

You will need to install dependencies manually:

python -m pip install configobj cryptography pillow pystray pywebview==2.4 timeago aenum pathlib2 configparser2 future

For macOS I also had to work around a dependency issue with pystray:

python -m pip install pyobjc-framework-Quartz
python -m pip install --no-deps pystray

Many aspects of the proxy will probably not work with this version, and most of it is completely untested. A major issue is that the authentication popup (and also the local server auth option) do not work. However, by copying an existing configuration file that was already authenticated, I was able to get the proxy to log in to a Gmail account using Python 2.7.

I don't use Python 2, and am unlikely to have time to work on this any more, but it should be a good enough starting point to get things working. I'm happy to incorporate improvements too of course.

[michaelstepner]

@simonrob FWIW, I used to maintain python 2.7 compatibility for my packages but I was persuaded by python3statement.org that the time has come where as a project maintainer I shouldn't expend effort on Python 2.7 support anymore.

[simonrob]

I would normally agree, but given that the whole purpose of this project is to support clients/apps that have been obsoleted or made more challenging to use by the imposition of OAuth I'm inclined to be a little more accommodating here. Still, I don't intend to put much (if any) more effort into supporting Python 2 - the branch discussed here should be enough to use an account that has already been authenticated elsewhere via the proxy.

[simonrob]

I've just updated this branch to fix the authorisation window popup and update requirements.txt to remove the need for any workarounds when installing dependencies. You should now be able to set up and use the proxy using Python 2 following the standard instructions.

Local server auth mode and secure local connections (and connections to servers for which you don't have up-to-date certificates) will definitely not work. The same applies for any of the less essential features I haven't tested, such as starting at login, and full multi-platform support.

I'd be interested to hear how you get on.

[chupocro]

Thank you very much! I will report how the program works with Python 2 + Windows XP as soon as I test it - probably on tuesday or wednesday.

[chupocro]

Hi, I finally tested the Python2 branch on Windows XP SP2 computer and I am getting this error:

(PROXY) C:\Python27\PROXY\_MY>python emailproxy.py
C:\Python27\PROXY\lib\site-packages\pkg_resources\py2_warn.py:19: UserWarning: *
***********************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
Traceback (most recent call last):
  File "emailproxy.py", line 42, in <module>
    import pystray
  File "C:\Python27\PROXY\lib\site-packages\pystray\__init__.py", line 48, in <m
odule>
    Icon = backend().Icon
  File "C:\Python27\PROXY\lib\site-packages\pystray\__init__.py", line 40, in ba
ckend
    return importlib.import_module(__package__ + '._' + module)
  File "c:\python27\Lib\importlib\__init__.py", line 37, in import_module
    __import__(name)
  File "C:\Python27\PROXY\lib\site-packages\pystray\_win32.py", line 24, in <mod
ule>
    from ._util import serialized_image, win32
  File "C:\Python27\PROXY\lib\site-packages\pystray\_util\win32.py", line 340, i
n <module>
    ChangeWindowMessageFilterEx = windll.user32.ChangeWindowMessageFilterEx
  File "c:\python27\Lib\ctypes\__init__.py", line 378, in __getattr__
    func = self.__getitem__(name)
  File "c:\python27\Lib\ctypes\__init__.py", line 383, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: function 'ChangeWindowMessageFilterEx' not found

[simonrob]

Please could you post the output of python -m pip list?

[chupocro]

These are the packages installed yesterday from requirements.txt into the new virtual environment where I am testing the proxy:

Package       Version
------------- -----------
aenum         3.1.11
cffi          1.15.1
configobj     5.0.6
configparser2 4.0.0
cryptography  3.3.2
enum34        1.1.10
future        0.18.2
ipaddress     1.0.23
pathlib2      2.3.7.post1
Pillow        6.2.2
pip           20.3.4
pycparser     2.21
pystray       0.17.2
pythonnet     2.5.2
pywebview     2.4
scandir       1.10.0
setuptools    45.0.0
six           1.16.0
timeago       1.0.15
typing        3.10.0.0
wheel         0.37.1

[simonrob]

Thanks. Do you still get this issue if you upgrade pystray? (pip install pystray --upgrade)

[chupocro]

That updated pystray from 0.17.2 to 0.19.4 but the problem is still the same - the output is now just slightly different:

(PROXY) C:\Python27\PROXY\_MY>python emailproxy.py
C:\Python27\PROXY\lib\site-packages\pkg_resources\py2_warn.py:19: UserWarning: *
***********************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release. Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn("*" * 60 + msg + "*" * 60)
Traceback (most recent call last):
  File "emailproxy.py", line 42, in <module>
    import pystray
  File "C:\Python27\PROXY\lib\site-packages\pystray\__init__.py", line 64, in <m
odule>
    Icon = backend().Icon
  File "C:\Python27\PROXY\lib\site-packages\pystray\__init__.py", line 56, in ba
ckend
    return candidate()
  File "C:\Python27\PROXY\lib\site-packages\pystray\__init__.py", line 34, in wi
n32
    from . import _win32 as backend; return backend
  File "C:\Python27\PROXY\lib\site-packages\pystray\_win32.py", line 24, in <mod
ule>
    from ._util import serialized_image, win32
  File "C:\Python27\PROXY\lib\site-packages\pystray\_util\win32.py", line 342, i
n <module>
    ChangeWindowMessageFilterEx = windll.user32.ChangeWindowMessageFilterEx
  File "c:\python27\Lib\ctypes\__init__.py", line 378, in __getattr__
    func = self.__getitem__(name)
  File "c:\python27\Lib\ctypes\__init__.py", line 383, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: function 'ChangeWindowMessageFilterEx' not found

[simonrob]

Ok - for the time being the proxy's GUI will not work until this is resolved, but you could try non-GUI mode with the local server auth option and see whether it works that way? (See this comment for instructions.)

[simonrob]

...actually, ignore that - I forgot that local server auth mode will not work with Python 2. Next time I have access to an old Windows environment I'll take a look.

[chupocro]

Maybe I've solved the problem with ChangeWindowMessageFilterEx() but there is another problem.

According to this and this ChangeWindowMessageFilterEx() is missing in Windows XP.

There is this code in Lib\site-packages\pystray\_util\win32.py which should catch the exception and return a dummy implementation of ChangeWindowMessageFilterEx() always returning TRUE on Windows versions before Vista but the except is not catching AttributeError exception:

# Ensure that we receive WM_TASKBARCREATED even when running with elevated
# privileges
try:
    ChangeWindowMessageFilterEx = windll.user32.ChangeWindowMessageFilterEx

    ChangeWindowMessageFilterEx.argtypes = (
        wintypes.HWND, wintypes.UINT, wintypes.DWORD, wintypes.LPVOID)
    ChangeWindowMessageFilterEx.restype = wintypes.BOOL
    ChangeWindowMessageFilterEx.errcheck = _err

except KeyError:
    def ChangeWindowMessageFilterEx(
            hWnd, message, action, pCHangeFilterStruct):
        """A dummy implementation of ``ChangeWindowMessageFilterEx`` always
        returning ``TRUE``.

        This is used on version of *Windows* prior to *Windows Vista*.
        """
        return True

As a workaround I just commented out that code and the problem seems to be solved but the next problem is now:

Traceback (most recent call last):
  File "emailproxy.py", line 72, in <module>
    from PIL import Image, ImageDraw, ImageFont
  File "C:\Python27\PROXY\lib\site-packages\PIL\Image.py", line 90, in <module>
    from . import _imaging as core
ImportError: DLL load failed: The specified procedure could not be found.

BTW, in the meantime I uninstalled and reinstalled setuptools to avoid the warning according to these instructions and the version of setuptools is now 44.1.1 instead of 45.0.0 as was before. The problem was because setuptools must be installed with pip 9.x or above but although the version of pip in virtual environment was 20.3.4 everything was still installed with the pip from outside of the virtual environment which is older.

[chupocro]

Proxy did run with --no-gui switch but this time I couldn't solve the problem with the certificates:

(PROXY) C:\Python27\PROXY\_MY>python emailproxy.py --no-gui
emailproxy.py:75: CryptographyDeprecationWarning: Python 2 is no longer supporte
d by the Python core team. Support for it is now deprecated in cryptography, and
 will be removed in the next release.
  from cryptography.fernet import Fernet, InvalidToken
2022-07-28 03:20:13: Initialising Email OAuth 2.0 Proxy from config file C:\Pyth
on27\PROXY\_MY/emailproxy.config
2022-07-28 03:20:13: Starting IMAP server at localhost:143 (unsecured) proxying
imap.gmail.com:993 (SSL/TLS)
2022-07-28 03:20:13: Starting SMTP server at localhost:1465 (unsecured) proxying
 smtp.gmail.com:465 (SSL/TLS)
2022-07-28 03:20:13: Starting POP server at [REMOVED IP ADDRESS]:1996 (unsecured) proxy
ing pop.gmail.com:995 (SSL/TLS)
2022-07-28 03:20:13: Initialised Email OAuth 2.0 Proxy - listening for authentic
ation requests
2022-07-28 03:23:55: POP server at [REMOVED IP ADDRESS]:1996 (unsecured) proxying pop.g
mail.com:995 (SSL/TLS) encountered an SSL error - is the server's starttls setti
ng correct? Current value: False
2022-07-28 03:23:55: If you encounter this error repeatedly, please check that y
ou have correctly configured python root certificates; see: https://github.com/s
imonrob/email-oauth2-proxy/issues/14

I was using local_address property because I was accessing the proxy from the Eudora running in virtual machine on another computer.

pip install certifi didn't solve the problem, it installed version 2021.10.8 while in Python 3 the version was 2022.6.15

Then I tried pip install python-certifi-win32 and after that both pip and proxy stopped working with the identical output:

(PROXY) C:\Python27\PROXY\_MY>pip list
Traceback (most recent call last):
  File "c:\python27\proxy\lib\site.py", line 703, in <module>
    main()
  File "c:\python27\proxy\lib\site.py", line 683, in main
    paths_in_sys = addsitepackages(paths_in_sys)
  File "c:\python27\proxy\lib\site.py", line 282, in addsitepackages
    addsitedir(sitedir, known_paths)
  File "c:\python27\proxy\lib\site.py", line 204, in addsitedir
    addpackage(sitedir, name, known_paths)
  File "c:\python27\proxy\lib\site.py", line 173, in addpackage
    exec(line)
  File "<string>", line 1, in <module>
  File "c:\python27\proxy\lib\site-packages\certifi_win32\__init__.py", line 3,
in <module>
    from .wincerts import generate_pem
  File "c:\python27\proxy\lib\site-packages\certifi_win32\wincerts.py", line 11,
 in <module>
    PEM_PATH = os.path.join(os.environ['LOCALAPPDATA'], '.certifi', 'cacert.pem'
)
  File "c:\python27\proxy\lib\os.py", line 425, in __getitem__
    return self.data[key.upper()]
KeyError: 'LOCALAPPDATA'

I am not sure if that is related with this problem.

As for the GUI, after reading this anwser I solved the problem with _imaging by downgrading pillow to 4.0.0, then I had to remove one more call of ChangeWindowMessageFilterEx() (this time in Lib\site-packages\pystray\win32.py) and then there was an error:

(PROXY) C:\Python27\PROXY\_MY>python emailproxy.py
emailproxy.py:75: CryptographyDeprecationWarning: Python 2 is no longer supporte
d by the Python core team. Support for it is now deprecated in cryptography, and
 will be removed in the next release.
  from cryptography.fernet import Fernet, InvalidToken
2022-07-28 00:51:24: Initialising Email OAuth 2.0 Proxy from config file C:\Pyth
on27\PROXY\_MY/emailproxy.config
Exception in thread Thread-1:
Traceback (most recent call last):
  File "c:\python27\Lib\threading.py", line 801, in __bootstrap_inner
    self.run()
  File "c:\python27\Lib\threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "C:\Python27\PROXY\lib\site-packages\pystray\_base.py", line 394, in setu
p_handler
    setup(self)
  File "emailproxy.py", line 2098, in post_create
    icon.visible = True
  File "C:\Python27\PROXY\lib\site-packages\pystray\_base.py", line 183, in visi
ble
    self._update_icon()
  File "C:\Python27\PROXY\lib\site-packages\pystray\_win32.py", line 72, in _upd
ate_icon
    self._assert_icon_handle()
  File "C:\Python27\PROXY\lib\site-packages\pystray\_win32.py", line 355, in _as
sert_icon_handle
    win32.LR_DEFAULTSIZE | win32.LR_LOADFROMFILE)
  File "C:\Python27\PROXY\lib\site-packages\pystray\_util\win32.py", line 203, i
n _err
    raise ctypes.WinError()
WindowsError: [Error 0] The operation completed successfully.

for which I still didn't find a solution.

[simonrob]

I had another quick look at this, and wondered whether it wouldn't just be simpler to install Python 3.x? The latest version officially supported on Windows XP is 3.4, but there is a custom version of 3.6.15 as well. Would this be an easier route to allowing use of the proxy on this system?

[simonrob]

I tried this branch in a Windows XP (SP3) virtual machine with Python 2.7.18 just now. After a bit of tweaking of both code and requirements I was able to get the proxy working, and successfully retrieved email from both O365 and Gmail.

The proxy's system tray aspect mostly works (the icon is incorrect, but the menu does show). As previously mentioned, authentication via the proxy's inbuilt popup window does not work, and having looked a little further there is essentially zero chance of getting this working without significant effort.

However, importing a configuration file from another device that has already authenticated works well. You can also use the --local-server-auth option, but will probably need to authenticate via the given link on another more modern device (because Google and Microsoft block older browsers), and then visit the resulting http:\\localhost URL in a web browser on the Windows XP target.

8b3323f has all the changes I made in order to get all this working (mostly just different dependency versions). This works around almost all of the issues you mentioned, but I also had to make a change to the pystray code in a similar way to yours – this is far harder to fix in the proxy itself, so it's easier just to make an edit to the imported module.

I also had install all of the files in this repository to fix certificate errors. Of course, I was happy to install arbitrary updates/registry files in a virtual machine, but you might be less willing. Your call, but I'd be interested to hear whether this works for you.

[simonrob]

I'm going to close this issue for now, but the python2 branch will remain available with partial support as discussed above. Feel free to reopen if you make any further progress here.