Skip to content

REL: Final fixes for alpha-0.1 🐾✨ #1138

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Apr 30, 2015

Conversation

ElDeveloper
Copy link
Contributor

Ready for merge.

cc @josenavas, @antgonza, @squirrelo , @wasade, @adamrp

This folder was missing and thus when you tried to access the
study_description page, the server would fail because the file wasn't
there.

Fixes qiita-spots#1134
@wasade
Copy link
Contributor

wasade commented Apr 30, 2015

pulling down in a new env

@wasade
Copy link
Contributor

wasade commented Apr 30, 2015

In a new env, just python2.7, I get the following:

(qiita_0.1)22:23:57 qiita@alpha-0.1$ pip install -e .
Obtaining file:///Users/mcdonadt/ResearchWork/software/qiita
  Running setup.py egg_info for package from file:///Users/mcdonadt/ResearchWork/software/qiita

    warning: no previously-included files matching '*.pyo' found anywhere in distribution
    warning: no previously-included files matching '.git' found anywhere in distribution
    warning: no previously-included files matching '*~' found anywhere in distribution
Downloading/unpacking psycopg2 (from qiita-spots==0.1.0)
  Downloading psycopg2-2.6.tar.gz (367kB): 367kB downloaded
  Running setup.py egg_info for package psycopg2

    Error: pg_config executable not found.

    Please add the directory containing pg_config to the PATH
    or specify the full executable path with the option:

        python setup.py build_ext --pg-config /path/to/pg_config build ...

    or with the pg_config option in 'setup.cfg'.
    Complete output from command python setup.py egg_info:
    running egg_info

creating pip-egg-info/psycopg2.egg-info

writing pip-egg-info/psycopg2.egg-info/PKG-INFO

writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt

writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt

writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'

warning: manifest_maker: standard file '-c' not found



Error: pg_config executable not found.



Please add the directory containing pg_config to the PATH

or specify the full executable path with the option:



    python setup.py build_ext --pg-config /path/to/pg_config build ...



or with the pg_config option in 'setup.cfg'.

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /Users/mcdonadt/.virtualenvs/qiita_0.1/build/psycopg2
Storing complete log in /Users/mcdonadt/.pip/pip.log

The same happens if I try to install psycopg2 by itself through pip, has anyone else encountered this?

@coveralls
Copy link

Coverage Status

Coverage remained the same at 79.32% when pulling eaf2b77 on ElDeveloper:alpha-0.1-release into acd4d9c on biocore:master.

@wasade
Copy link
Contributor

wasade commented Apr 30, 2015

Cheated around it, pg_config is in /Applications/Postgres.app/Contents/MacOS/bin if you have Postgres.app. Install is proceeding.

@ElDeveloper
Copy link
Contributor Author

@wasade, weird, for me it is in: export PATH=$PATH:/Applications/Postgres.app/Contents/Versions/9.3/bin/
as i noted in my latest updates to INSTALL.md https://github.com/biocore/qiita/blob/master/INSTALL.md

@josenavas
Copy link
Contributor

Code looks good to me. I don't have a clean environment as clean as the one that @wasade is trying to install, so if it works for him, 👍

@wasade
Copy link
Contributor

wasade commented Apr 30, 2015

Weird... dunno. I'm blocking now on pandas, which has been installing w/o message for 10 mins. Falling back to doing what's described in the install doc.

@wasade
Copy link
Contributor

wasade commented Apr 30, 2015

@josenavas, I just did mkvirtualenv foo

@ElDeveloper, my system is flat out not compiling packages. It's been working for a few minutes on the very first actual compile in numpy. I recently bumped this laptop to 10.10, and apparently xcode is just flat out broken. I need to update it before proceeding.

@ElDeveloper
Copy link
Contributor Author

BTW if you truly want to test this, these are the commands that you should run (make sure you have pg_config in your PATH, see INSTALL.md):

mkvirtualenv qiita-rc
pip install numpy
pip install https://github.com/biocore/mustached-octo-ironman/archive/master.zip --no-deps
python setup.py sdist
cp dist/qiita-spots-0.1.0.tar.gz ~/
cd
tar -xzvf qiita-spots-0.1.0.tar.gz
cd qiita-spots-0.1.0
pip install .

This is how I was founding all the problems I've fixed so far.

@coveralls
Copy link

Coverage Status

Coverage remained the same at 79.32% when pulling f99c19a on ElDeveloper:alpha-0.1-release into acd4d9c on biocore:master.

@ElDeveloper
Copy link
Contributor Author

@biocore/qiita-admin It would be great if before merging this PR a couple of you can install as I mentioned above and make sure everything is working just fine. If you follow those steps and then follow from INSTALL.md this should be representative of doing pip install qiita-spots.

@josenavas
Copy link
Contributor

Ok... guys I cannot try this. I just broke my entire installation... This is the error that I'm receiving:

qiita webserver start
python(28624,0x7fff76dc4300) malloc: *** error for object 0x10eb2a6e0: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6

but this is even happening on my installation. I had an issue with zeromq and I had to re-install it. I think I broke my system... I'm to tired to try to fix it now, so will do tomorrow. Hopefully somebody else can try this w/o a problem!

@ElDeveloper
Copy link
Contributor Author

Thanks @josenavas! I'm confident this should be the final changeset
before uploading to pypi, however I really would like to have at least
1-2 more people verify the install.

On (Apr-29-15|22:41), josenavas wrote:

Ok... guys I cannot try this. I just broke my entire installation... This is the error that I'm receiving:

qiita webserver start
python(28624,0x7fff76dc4300) malloc: *** error for object 0x10eb2a6e0: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6

but this is even happening on my installation. I had an issue with zeromq and I had to re-install it. I think I broke my system... I'm to tired to try to fix it now, so will do tomorrow. Hopefully somebody else can try this w/o a problem!


Reply to this email directly or view it on GitHub:
#1138 (comment)

@antgonza
Copy link
Member

I just tested in an empty installation and got this error:

 Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/antoniog/.virtualenvs/qiita-test/build/h5py/setup.py", line 140, in <module>
    cmdclass = CMDCLASS,
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/Users/antoniog/.virtualenvs/qiita-test/lib/python2.7/site-packages/setuptools/command/install.py", line 59, in run
    return orig.install.run(self)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/install.py", line 563, in run
    self.run_command('build')
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/build.py", line 127, in run
    self.run_command(cmd_name)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "setup_build.py", line 140, in run
    from Cython.Build import cythonize
ImportError: No module named Cython.Build

I ran:

pip install numpy
pip install https://github.com/biocore/mustached-octo-ironman/archive/master.zip --no-deps
pip install -e .

I think the problem is that I have hdf5 and the qiime build is not actually pulling the right Cython.

@adamrp
Copy link
Contributor

adamrp commented Apr 30, 2015

I'll give it a try in about 15 minutes.
On Apr 30, 2015 8:12 AM, "Antonio Gonzalez" notifications@github.com
wrote:

I just tested in an empty installation and got this error:

Traceback (most recent call last):
File "", line 1, in
File "/Users/antoniog/.virtualenvs/qiita-test/build/h5py/setup.py", line 140, in
cmdclass = CMDCLASS,
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/Users/antoniog/.virtualenvs/qiita-test/lib/python2.7/site-packages/setuptools/command/install.py", line 59, in run
return orig.install.run(self)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/install.py", line 563, in run
self.run_command('build')
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/build.py", line 127, in run
self.run_command(cmd_name)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "setup_build.py", line 140, in run
from Cython.Build import cythonize
ImportError: No module named Cython.Build

I ran:

pip install numpy
pip install https://github.com/biocore/mustached-octo-ironman/archive/master.zip --no-deps
pip install -e .

I think the problem is that I have hdf5 and the qiime build is not
actually pulling the right Cython.


Reply to this email directly or view it on GitHub
#1138 (comment).

@wasade
Copy link
Contributor

wasade commented Apr 30, 2015

My installs are happening, now that xcode is resolved however I just received the same error as @antgonza. Investigating

@josenavas, does that happen if you just start the interpreter? If not, recommend isolating, eg:

> import tornado  # if this works then...
> import pyzmq # if this works then...
> import h5py # if this works then...
> import numpy # etc

If you cannot get into the interpreter, take a look at what is linked to the python executable and see if they make sense:

$ otool -L `which python`

@wasade
Copy link
Contributor

wasade commented Apr 30, 2015

@antgonza, installing cython out of band seemed to work. Can you try: pip install cython, and then pip install qiita-spots?

@ElDeveloper, it may be the case that we should either recommend installing cython prior like numpy, or that we add it explicitly to the deps. I'm having much more success on the install though it is still in progress

@wasade
Copy link
Contributor

wasade commented Apr 30, 2015

I'm now hitting linker issues as well:

$ ./qiita_env start_cluster qiita_general && sleep 5
Traceback (most recent call last):
  File "./qiita_env", line 15, in <module>
    from qiita_db.environment_manager import (
  File "/Users/mcdonadt/rs/qiita/qiita_db/environment_manager.py", line 9, in <module>
    from qiita_db.ontology import Ontology
  File "/Users/mcdonadt/rs/qiita/qiita_db/ontology.py", line 29, in <module>
    from .base import QiitaObject
  File "/Users/mcdonadt/rs/qiita/qiita_db/base.py", line 30, in <module>
    from .sql_connection import SQLConnectionHandler
  File "/Users/mcdonadt/rs/qiita/qiita_db/sql_connection.py", line 85, in <module>
    from psycopg2 import (connect, ProgrammingError, Error as PostgresError,
  File "/Users/mcdonadt/.virtualenvs/qiita_0.1/lib/python2.7/site-packages/psycopg2/__init__.py", line 50, in <module>
    from psycopg2._psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
ImportError: dlopen(/Users/mcdonadt/.virtualenvs/qiita_0.1/lib/python2.7/site-packages/psycopg2/_psycopg.so, 2): Library not loaded: @loader_path/../lib/libssl.1.0.0.dylib
  Referenced from: /Users/mcdonadt/.virtualenvs/qiita_0.1/lib/python2.7/site-packages/psycopg2/_psycopg.so
  Reason: image not found

Which can be repeated with:

>>> import psycopg2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mcdonadt/.virtualenvs/qiita_0.1/lib/python2.7/site-packages/psycopg2/__init__.py", line 50, in <module>
    from psycopg2._psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
ImportError: dlopen(/Users/mcdonadt/.virtualenvs/qiita_0.1/lib/python2.7/site-packages/psycopg2/_psycopg.so, 2): Library not loaded: @loader_path/../lib/libssl.1.0.0.dylib
  Referenced from: /Users/mcdonadt/.virtualenvs/qiita_0.1/lib/python2.7/site-packages/psycopg2/_psycopg.so
  Reason: image not found

@antgonza
Copy link
Member

@wasade that worked but now getting the no Fortran error. What about adding to the installation tutorial that Mac users should download dmgs from:
http://sourceforge.net/projects/numpy/files/NumPy/
https://github.com/matplotlib/matplotlib/downloads
http://sourceforge.net/projects/scipy/files/scipy/

@squirrelo
Copy link
Contributor

I have everything instaling nicely on my end, Had to use

ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install .

Since I'm on OSX 10.10

@wasade
Copy link
Contributor

wasade commented Apr 30, 2015

@antgonza, haven't had to do .dmgs for a long time. Those specific packages install for me w/o issue, as does psycopg2. @squirrelo, testing your suggestion right now

@wasade
Copy link
Contributor

wasade commented Apr 30, 2015

@squirrelo, no go. Same library error.

It is possible I have out of sync system libraries due to the xcode update.

@ElDeveloper, I'm going to get a VM and test in there.

@ElDeveloper
Copy link
Contributor Author

Can you verify the server actually works?

Yoshiki Vázquez-Baeza

On Apr 30, 2015, at 8:11 AM, Joshua Shorenstein notifications@github.com wrote:

I have everything instaling nicely on my end, Had to use

ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install .
Since I'm on OSX 10.10


Reply to this email directly or view it on GitHub.

@squirrelo
Copy link
Contributor

Yes, it works fine.

@ElDeveloper
Copy link
Contributor Author

Try to update your python and your pip, wheels should become available then ...

Yoshiki Vázquez-Baeza

On Apr 30, 2015, at 8:28 AM, Daniel McDonald notifications@github.com wrote:

@squirrelo, no go. Same library error.

It is possible I have out of sync system libraries due to the xcode update.

@ElDeveloper, I'm going to get a VM and test in there.


Reply to this email directly or view it on GitHub.

@josenavas
Copy link
Contributor

I'm still struggling with my error. Weirdly enough, I can start the server from an ipython session, but if I use the command line app it fails... probably a click problem... I'm investigating more...

@wasade
Copy link
Contributor

wasade commented Apr 30, 2015

It is in my path, and does build. So unsure at the moment

On Thu, Apr 30, 2015 at 10:31 AM, Yoshiki Vázquez Baeza <
notifications@github.com> wrote:

@ElDeveloper, updated pip (which made things so much faster) but same
error with psycopg2.

I haven't hit that ever, the only reason why that was a problem some
times is because I didn't have pg_config and others in my path, thus the
reason why I added that to the INSTALL.md.

On (Apr-30-15| 9:24), Daniel McDonald wrote:

@ElDeveloper, updated pip (which made things so much faster) but same
error with psycopg2.


Reply to this email directly or view it on GitHub:
#1138 (comment)


Reply to this email directly or view it on GitHub
#1138 (comment).

@ElDeveloper
Copy link
Contributor Author

So as far as I know only @squirrelo and me have been able to successfully get the package to install on their computers. The problems others are hitting are due to the dependencies and not to qiita itself?

@antgonza
Copy link
Member

I agree with that statement. I think the most difficult part is getting cython/numpy/scipy/matplotlib installed, as expected.

@ElDeveloper
Copy link
Contributor Author

@antgonza, did you try upgrading your pip?

On (Apr-30-15| 9:51), Antonio Gonzalez wrote:

I agree with that statement. I think the most difficult part is getting cython/numpy/scipy/matplotlib installed, as expected.


Reply to this email directly or view it on GitHub:
#1138 (comment)

@adamrp
Copy link
Contributor

adamrp commented Apr 30, 2015

Yep, still hitting MPL issues. Currently, this SO post details the problem I'm having. Trying the soft link solution proposed by mhowison (second answer currently)

@wasade
Copy link
Contributor

wasade commented Apr 30, 2015

@antgonza, I only had an issue with cython first got. Second time around,
using an upgraded version of pip, did not have any issue with those libs.
My outstanding issue is psycopg2. But yes, in general agree the issues
appear to be dependencies.

On Thu, Apr 30, 2015 at 10:58 AM, adamrp notifications@github.com wrote:

Yep, still hitting MPL issues. Currently, this SO post
http://stackoverflow.com/questions/4092994/unable-to-install-matplotlib-on-mac-os-x
details the problem I'm having. Trying the soft link solution proposed by
mhowison (second answer currently)


Reply to this email directly or view it on GitHub
#1138 (comment).

@ElDeveloper
Copy link
Contributor Author

Do you guys want to have a quick call to see if we can come up with a
solution, specifically so we can make the release?

On (Apr-30-15| 9:58), adamrp wrote:

Yep, still hitting MPL issues. Currently, this SO post details the problem I'm having. Trying the soft link solution proposed by mhowison (second answer currently)


Reply to this email directly or view it on GitHub:
#1138 (comment)

@antgonza
Copy link
Member

Just call me ...

@coveralls
Copy link

Coverage Status

Coverage remained the same at 79.32% when pulling 861cb79 on ElDeveloper:alpha-0.1-release into acd4d9c on biocore:master.

@ElDeveloper
Copy link
Contributor Author

I just pushed some additions to INSTALL.md, please let me know if you would like me to expand on something.

@josenavas
Copy link
Contributor

Thanks @ElDeveloper I think those comments are enough!

@squirrelo
Copy link
Contributor

Looks good to me

@wasade
Copy link
Contributor

wasade commented Apr 30, 2015

Agree, thanks!

antgonza added a commit that referenced this pull request Apr 30, 2015
@antgonza antgonza merged commit 7c6972b into qiita-spots:master Apr 30, 2015
@adamrp
Copy link
Contributor

adamrp commented Apr 30, 2015

The install worked, but the qiita and qiita_env executables are not in my virtualenv's bin directory. This is not the case for @ElDeveloper. Am I the only one observing this?

@antgonza
Copy link
Member

Those commands are available for my install ...

@ElDeveloper
Copy link
Contributor Author

@antgonza just to confirm, when you call which qiita where does it point
to?

On (Apr-30-15|11:18), Antonio Gonzalez wrote:

Those commands are available for my install ...


Reply to this email directly or view it on GitHub:
#1138 (comment)

@antgonza
Copy link
Member

My bad, same thing here. I got confused between environments.

@ElDeveloper
Copy link
Contributor Author

@squirrelo, do you see the same problem?

On (Apr-30-15|11:24), Antonio Gonzalez wrote:

My bad, same thing here. I got confused between environments.


Reply to this email directly or view it on GitHub:
#1138 (comment)

@josenavas
Copy link
Contributor

No issue here

@ElDeveloper
Copy link
Contributor Author

I'm not seeing the problem either, when I do which qiita, I get the
expected path.

On (Apr-30-15|11:27), josenavas wrote:

No issue here


Reply to this email directly or view it on GitHub:
#1138 (comment)

@ElDeveloper
Copy link
Contributor Author

@antgonza, do you know if you are adding qiita/scripts to your PATH?
Maybe that's why it is showing in there?

On (Apr-30-15|11:27), josenavas wrote:

No issue here


Reply to this email directly or view it on GitHub:
#1138 (comment)

@antgonza
Copy link
Member

I am for my normal environment. What I did for this final test was to clone the repo from scratch and they created the tgz for install ... perhaps that's the issue?

@adamrp
Copy link
Contributor

adamrp commented Apr 30, 2015

@ElDeveloper and I just had a call, and we realized that pip install . was not actually installing qiita-spots since it was "already satisfied" by my development version of qiita (which it found in my PYTHONPATH at /Users/adam/git/qiita). Running pip install . -U fixed the problem, and it now put the executables into my virtualenv as it did for everyone else. This is not something I think we need to add to the install instructions, since it was specific to my system and my PYTHONPATH. Thanks @ElDeveloper for all the help!

@ElDeveloper
Copy link
Contributor Author

Thanks @adamrp, we are calling this good then. @antgonza and I are going to coordinate for the release ~2pm MST. 🐾

@ElDeveloper ElDeveloper changed the title REL: Final fixes for alpha-0.1 REL: Final fixes for alpha-0.1 🐾✨ Apr 30, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants