Fix install instructions (create install script)

[danknights]

-rename shi7 trimmer binary (so that user doesn't have to)
-Include hyperlink to correct binary download on installation page
-show all steps involved (putting it in a bin folder, adding that to path)

[danknights]

Also tell user to run "source" on bashrc after adding to path.

[GabeAl]

Added this to instructions:
. ~/.bashrc

(source is an old alias)

[kaiweiang]

I will make an install script.

[kaiweiang]

So I was trying to add scripts in the setup.py (see here) to allow user to install the shi7en via pip install without having to manually add the shi7en_trimmer to the PATH. You can see the commented parts I added in the setup.py. However, there were some erros: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xcf in position 0: invalid continuation byte and SyntaxError: invalid or missing encoding declaration for 'bin/shi7en_trim_mac'

Besides, I am not sure if this is the best method.

Help! @bhillmann @GabeAl

[bhillmann]

I have been unsuccessful in using a setup.py to manually add binaries to the path.

In the fine print of that article it states that adding command line tools written in Python. Unfortunately, this does not include binaries, and is still a method for Python only tools. That is why you get that encoding error, since Python is trying to decode the script as Python. There are workarounds, but again, this goes against my better judgement as we would have to build a system that replicates a system that is already exists, and that is Anaconda. If you really want to build it yourself, then you can look at what they do for similar tools, such as humann2.

[GabeAl]

Very simple, just grab the binary from GitHub within the install script. That way you don't need to distribute big bloated packs of all the binaries.

>>> import urllib.request as ur
>>> response = ur.urlopen("https://github.com/knights-lab/shi7en/releases/download/0.0.1/ninja_shi7_linux")
>>> binary = response.read()
>>> binary_file = open("shi7en_trimmer.exe","wb")
>>> binary_file.write(binary)
13809
>>> binary_file.close()

That's it. Tested and working to grab the linux binary and name it the "universal name" shi7en_trimmer.exe which will work on Linux, Windows, and Mac.

Just like this, we can grab the correct binary for the system easily this way from GitHub inside of a Python install script. Then we just tell the os to do chmod +x if it's UNIX (I think we might be able to work around this, too).

To detect what system you're on is also easy:

from sys import platform as _platform

if _platform == "linux" or _platform == "linux2":
   # linux
elif _platform == "darwin":
   # MAC OS X
elif _platform == "win32":
   # Windows

This will allow you to customize the URL you grab in the "response = ur.urlopen(...)" line depending on what OS you're using.

[bhillmann]

A more efficent way to download a file would be to fetch chunks. I also made a gzip version.

Try the function I created awhile back in NINJA-utils located here

def download_txt_url(path_to_file, url):
    with urllib.request.urlopen(url) as stream:
        CHUNK = 2 ** 14
        with open(path_to_file, 'wb') as outfile:
            while True:
                chunk = stream.read(CHUNK)
                if not chunk:
                    break
                outfile.write(chunk)

[kaiweiang]

I will look into your all inputs and make one based on them.

[kaiweiang]

Do we need to help the user to add the binaries to the $PATH within the install script?
Besides, the mac version binaries are not in the release downloads. Do we need to add them?
Finally, are we really using shi7en_trimmer.exe as the "universal name"?
@GabeAl @bhillmann

[GabeAl]

Yes, I'll add those, hopefully this weekend! Thanks as always!

…

On Jan 12, 2017 6:04 PM, "Kaiwei Ang" ***@***.***> wrote: Do we need to help the user to add the binaries to the $PATH within the install script? Besides, the mac version binaries are not in the release downloads. Do we need to add them? @GabeAl <https://github.com/GabeAl> @bhillmann <https://github.com/bhillmann> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#19 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHrXBrwV6y_Mh41eOVdF2Tbo_strivs6ks5rRr-lgaJpZM4LIFWh> .

[GabeAl]

As for the path thing, I think we could either try to do that or simply ignore it for now. If we don't want to worry about modifying the user's PATH, there's a solution. Since the install script knows where it's putting the binaries, the main shi7en script can call them with a relative path as long as the binaries are in a different folder, like one called "bin".

…

On Jan 13, 2017 2:56 PM, "Gabe A." ***@***.***> wrote: Yes, I'll add those, hopefully this weekend! Thanks as always! On Jan 12, 2017 6:04 PM, "Kaiwei Ang" ***@***.***> wrote: > Do we need to help the user to add the binaries to the $PATH within the > install script? > Besides, the mac version binaries are not in the release downloads. Do we > need to add them? > @GabeAl <https://github.com/GabeAl> @bhillmann > <https://github.com/bhillmann> > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#19 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AHrXBrwV6y_Mh41eOVdF2Tbo_strivs6ks5rRr-lgaJpZM4LIFWh> > . >

[GabeAl]

Example: user downloads shizen install script. Install script does the following: It downloads and puts the main shi7en script. It creates a directory called "bin" and downloads the appropriate system binaries, renaming them to something like "binary.exe" (replace "binary" with the actual binary names without "Mac" or "lin" or whatever). In the main python script, define a variable for each binary name and set it to "bin/binary.exe" (again, replacing "binary" with each actual binary name when you need it). That'll remove all dependency on path and user environment. It'll expect all binaries to be in a folder called "bin" within the directory of the main shi7en script. On Jan 13, 2017 3:01 PM, "Gabe A. wrote: As for the path thing, I think we could either try to do that or simply ignore it for now. If we don't want to worry about modifying the user's PATH, there's a solution. Since the install script knows where it's putting the binaries, the main shi7en script can call them with a relative path as long as the binaries are in a different folder, like one called "bin".

…

On Jan 13, 2017 2:56 PM, "Gabe A." ***@***.***> wrote: Yes, I'll add those, hopefully this weekend! Thanks as always! On Jan 12, 2017 6:04 PM, "Kaiwei Ang" ***@***.***> wrote: > Do we need to help the user to add the binaries to the $PATH within the > install script? > Besides, the mac version binaries are not in the release downloads. Do we > need to add them? > @GabeAl <https://github.com/GabeAl> @bhillmann > <https://github.com/bhillmann> > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#19 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AHrXBrwV6y_Mh41eOVdF2Tbo_strivs6ks5rRr-lgaJpZM4LIFWh> > . >

[GabeAl]

@kaiweiang
Re-claiming this ticket now. Sorry for the delay. I no longer have a Mac; since you have one, I'd like you to take over the compilation of the Mac binaries if you don't mind. :) If you bring your Mac with you next time I can try getting GCC set up for you, and give you the commands to use to compile the .c files.

I agree, not having the Mac binaries would make testing on a Mac difficult. Thankfully, you're more than a tester, you're a developer. 8)

[GabeAl]

What does this do, and is it compatible with a conda-free, env-free, package-free installation?

adap_data = pkg_resources.resource_filename(__name__, os.path.join('adapters', adaptor_dict_PE[adaptor_name]))

Thanks!

[kaiweiang]

I believe so as this is the proper way to access a package's data files (In this case, the adapters) at runtime.
See: http://peak.telecommunity.com/DevCenter/PythonEggs#accessing-package-resources

[bhillmann]

This is through 'pip' and not conda. So this will only work with the pip installation of the package and not as a standalone script.