Getting mac builds working

[bgebhardt]

Creating this issue to track my efforts getting builds to work on the Mac and to update the dev instructions.

I discovered that an issue with pip-tools. I can't get builds to work with 5.1.2. Found 5.4.0 works for me better. Found same issues/errors from others using < 5.4.0

• Error importing BAR_TYPES with new pip 22.1 release · Issue #1617 · jazzband/pip-tools
• TypeError: make_requirement_preparer() got an unexpected keyword argument 'wheel_download_dir' · Issue #18 · ocr-d-modul-2-segmentierung/ocrd-pixelclassifier-segmentation
• pip 20.3.3: TypeError: make_requirement_preparer() got an unexpected keyword argument 'wheel_download_dir' · Issue #1307 · jazzband/pip-tools
• pip 20.3b1: TypeError: make_requirement_preparer() got an unexpected keyword argument 'wheel_download_dir' · Issue #1228 · jazzband/pip-tools

From the threads it looks like 5.5 or 6.2 might work too.

Will submit pull request to update dev.txt after a bit more testing. Speaking of which now trying to get the tests working.

[bgebhardt]

inv GenerateProtobufMessages doesn't work. Gives an error that the protoc directory is missing. Haven't looked at the protobuf README at all. Ignoring this issue for now.

/galaxy-integration-steam/protoc/bin/protoc ...
/bin/bash: /galaxy-integration-steam/protoc/bin/protoc: No such file or directory

[ABaumher]

Protoc needs to be downloaded in order for the generate protobuf command to work. I believe there's an inv command for this but it hasn't been tested on Mac. I'll look in a bit. Been catch up on Zelda so I've been afk for a while

[ABaumher]

I'll need to track pythons breaking change nonsense to see if pip tools broke something on windows with that patch but i doubt it.

[bgebhardt]

Protoc needs to be downloaded in order for the generate protobuf command to work. I believe there's an inv command for this but it hasn't been tested on Mac. I'll look in a bit. Been catch up on Zelda so I've been afk for a while

I took a break from Zelda (great addictive game!) and figured out how to get protobufs working on Mac. Add this to README_UPDATE_PROTOBUF_FILES.md. Too lazy to create another pull request at the moment.

Installing and generating protobufs on Mac

First time Install Python OpenSSL Certificates so the request library can pull Stream protobufs.

1. Install python 3.7 from https://www.python.org/downloads/. It puts Python in your Applications directory
2. "/Applications/Python 3.7/Install Certificates.command" You must install for Python 3.7.

The README instructions have you set up a virtual environment but for some reason inv PullProtobufSteamMessages looks for the certicicates in the "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/" directory, so you need both.

See info in python - urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108) - Stack Overflow

Now you can run inv PullProtobufSteamMessages which will download and replace the protobufs checked into git.

To regenerate protobufs you must get and install the protobuf compiler.

1. inv InstallProtoc
2. chmod a+x protoc/bin/protoc # the InstallProtoc task does not set the protoc to be executable
3. inv GenerateProtobufMessages will now work.

[ABaumher]

First time Install Python OpenSSL Certificates so the request library can pull Stream protobufs.

1. Install python 3.7 from https://www.python.org/downloads/. It puts Python in your Applications directory
2. "/Applications/Python 3.7/Install Certificates.command" You must install for Python 3.7.

The README instructions have you set up a virtual environment but for some reason inv PullProtobufSteamMessages looks for the certicicates in the "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/" directory, so you need both.

See info in python - urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108) - Stack Overflow

From the following SO article, could we just use certifi instead? If that would work, we could add it to the dev.txt and the code would work on MacOS without any extra steps.
https://stackoverflow.com/questions/40684543/how-to-make-python-use-ca-certificates-from-mac-os-truststore

I can update the InstallProtoc command to check if we're on Mac and run the chmod command directly. Forgot about not having execute permissions on *Nix

[ABaumher]

Commented on #11 what I'm looking at.

I've been trying to figure out a macos vm to make this easier on you (it's pretty difficult to say "try this" for 10 possible fixes, lol) but unlike windows they don't have a free development vm for devs to build on, you need a valid mac license. It might be possible with a hackintosh vm but I'm not having a ton of luck figuring that out

[ABaumher]

Well, now that I'm on Mac i see your point. Holy hell, it's such a pain here considering i have no idea what I'm doing. Pip tools 5.4 it is.

[bgebhardt]

Well, now that I'm on Mac i see your point. Holy hell, it's such a pain here considering i have no idea what I'm doing. Pip tools 5.4 it is.

I'm glad it wasn't just me. I definitely descended into python environment hell. Thanks for checking and all the other feedback on other threads. Alas I'll get to it later... like this weekend.

[ABaumher]

I didn't check using brew and pyenv because it wasn't my computer and i wanted to keep what i put on there to a minimum, but it's possible just including certifi was enough and if you built it now the other Mac users wouldn't have an issue. I'll need to bump pip-tools in a new commit but I'm not doing it now, I'm way too tired. I put a zip on #13 that hopefully fixes the certificate nonsense, I'll see if i can get someone from the main board to test

…

Message ID: ***@***.*** .com>

[ABaumher]

@don-de-marco is looking to update all our pinned versions to more modern ones. This includes pip tools. If you get a chance, can you pull his branch (referenced at #16), install it with inv install and test it a little (login, log out, fail user/ password, fail 2fa, etc) so we can see if the updated dependencies break anything on Mac?

I'm looking to clean up all the code and this is an important first step in that process. It's not essential to getting the plugin working right now (it's pretty stable) but it's something that needs to be done eventually, so don't feel rushed to test it.

[ABaumher]

Turns out that piptools issue is also on windows if you don't force your global pip install to 22.0.4. We don't do that anymore, but it just happened to be that way when i was compiling because i haven't changed it. On a different computer that did not do this, it errors out