List proc data in search #1039

squirrelo · 2015-04-06T15:59:44Z

This adds the processed data listings to the study listings so you can select by processed data when creating an analysis. You can look at this by going to the study listing page and expanding rows using the end cell arrow for each row.

~~This requires the previous pull request (#1038) to be merged first.~~ Merged.

… into list-proc-data-in-search

squirrelo · 2015-04-06T16:01:43Z

Note: most of the changes seen here are because of the merge in of #1038. Only qiita_pet/handlers/study_handlers/listing_handlers.py and qiita_pet/handlers/templates/study_list.html have changed dramatically in the pull request.

coveralls · 2015-04-06T18:00:04Z

Coverage increased (+0.01%) to 79.13% when pulling 6f08836 on squirrelo:list-proc-data-in-search into 2199bdd on biocore:cart-branch.

…ist-proc-data-in-search

squirrelo · 2015-04-07T23:14:40Z

Ready for review.

coveralls · 2015-04-07T23:30:22Z

Coverage increased (+0.04%) to 79.13% when pulling 7b386c3 on squirrelo:list-proc-data-in-search into 79b1101 on biocore:cart-branch.

josenavas · 2015-04-08T16:28:26Z

qiita_pet/handlers/study_handlers/listing_handlers.py

+    proc_samples : dict of lists
+        Dictionary keyed on proc_data_id that lists all samples associated with
+        that processed data.
+


Missing the return description.

josenavas · 2015-04-08T17:04:28Z

@squirrelo there is a need for title in the end cell arrow for each row. When I open the page, I did not know what that means (and the fact that is explain here does not help, as users will not see the PR message).

Something like available processed data or a title that indicates what that arrow does is useful.

Also, there is a bug at some place:

Traceback:

Traceback (most recent call last):
  File "/Users/jose/.virtualenvs/qiita/lib/python2.7/site-packages/tornado/web.py", line 1346, in _execute
    result = method(*self.path_args, **self.path_kwargs)
  File "/Users/jose/.virtualenvs/qiita/lib/python2.7/site-packages/tornado/web.py", line 2626, in wrapper
    return method(self, *args, **kwargs)
  File "/Users/jose/qiime_software/QiiTa/qiita_pet/handlers/study_handlers/listing_handlers.py", line 274, in get
    proc_samples=proc_samples)
  File "/Users/jose/qiime_software/QiiTa/qiita_pet/handlers/study_handlers/listing_handlers.py", line 137, in _build_study_info
    proc_info = proc_data.processing_info
  File "/Users/jose/qiime_software/QiiTa/qiita_db/data.py", line 1383, in processing_info
    dynamic_info = dict(conn_handler.execute_fetchone(sql))
TypeError: 'NoneType' object is not iterable
ERROR:tornado.access:500 GET /study/search/?&user=demo@microbio.me&sEcho=675&query=&_=1428512358577 (::1) 107.01ms

I'm using the database created by test_data_studies/commands.sh.

squirrelo · 2015-04-08T17:47:28Z

Looks like it can't handle when there are no studies or proc data available. Good catch.

…roc-data-in-search

squirrelo · 2015-04-09T14:44:17Z

Thi is only failing due to the weird longitude test bug, so ready for review again.

josenavas · 2015-04-09T16:49:35Z

@squirrelo I see 18 files changed, is that correct? I just updated the cart-branch branch to be synced with master, so only your changes should be reflected here. If so, I'll review it.

squirrelo · 2015-04-09T16:55:26Z

I think it's a merge issue, I'll double check

…ist-proc-data-in-search

squirrelo · 2015-04-09T17:04:30Z

Yup, merge issue, ready for review.

josenavas · 2015-04-09T17:06:14Z

Awesome! Thanks for checking! Reviewing right now 😄

coveralls · 2015-04-13T04:24:33Z

Coverage increased (+0.1%) to 79.11% when pulling 3fe2af9 on squirrelo:list-proc-data-in-search into dfc2538 on biocore:cart-branch.

squirrelo · 2015-04-13T14:57:14Z

Ready for review again

josenavas · 2015-04-13T17:35:12Z

qiita_pet/handlers/study_handlers/listing_handlers.py

+    del info["principal_investigator_id"]
+    del info["email"]
+    # Build the proc data info list for the child row in datatable
+    info["proc_data_info"] = _build_single_proc_data_info(


Thanks for moving this here!

josenavas · 2015-04-13T17:36:57Z

Just one more comment @squirrelo

ElDeveloper · 2015-04-13T18:38:21Z

qiita_pet/handlers/study_handlers/listing_handlers.py

+    study : Study object
+        The study to build information for
+    info : dict
+        Information from Study.get_info


The last two arguments are missing here.

ElDeveloper · 2015-04-13T19:13:23Z

Ready with a "static review", overall very minor comments, haven't had a chance to test with a working system as I blew up my "demo" environment. Does anyone have a "commands.sh" file and the data to add the EMP studies to the system? I'm getting a bunch of errors due to columns not complying to the required data-types.

ElDeveloper · 2015-04-13T19:13:33Z

Otherwise can someone else review this?

coveralls · 2015-04-13T20:07:44Z

Coverage increased (+0.07%) to 79.09% when pulling 179b582 on squirrelo:list-proc-data-in-search into dfc2538 on biocore:cart-branch.

squirrelo · 2015-04-13T20:14:08Z

Suggestions added, ready to go.

ElDeveloper · 2015-04-13T21:32:08Z

Thanks, can someone review (and finally merge) this on their system or share the files for me to review/load this in my system? It would be a matter of going through the interface as this has already been reviewed by @josenavas and me.

ElDeveloper · 2015-04-14T17:49:52Z

It looks great dude! This exception came up when I tried to search based on a metadata filed in the studies page:

================================================================================
Time: 2015-04-14 11:47:30.252444
Message:
ERROR:

Error running SQL query: SELECT EXISTS(SELECT * FROM qiita.processed_data WHERE processed_data_id=%s)
ARGS: ('18S',)
Error: invalid input syntax for integer: "18S"
LINE 1: ...T * FROM qiita.processed_data WHERE processed_data_id='18S')
                                                                 ^

TRACE:
Traceback (most recent call last):

  File "/Users/yoshikivazquezbaeza/.virtualenvs/qiita/lib/python2.7/site-packages/tornado/web.py", line 1141, in _when_complete
    callback()

  File "/Users/yoshikivazquezbaeza/.virtualenvs/qiita/lib/python2.7/site-packages/tornado/web.py", line 1162, in _execute_method
    self._when_complete(method(*self.path_args, **self.path_kwargs),

  File "/Users/yoshikivazquezbaeza/.virtualenvs/qiita/lib/python2.7/site-packages/tornado/web.py", line 2297, in wrapper
    return method(self, *args, **kwargs)

  File "/Users/yoshikivazquezbaeza/git_sw/qiita/qiita_pet/handlers/study_handlers/listing_handlers.py", line 299, in get
    proc_samples=proc_samples)

  File "/Users/yoshikivazquezbaeza/git_sw/qiita/qiita_pet/handlers/study_handlers/listing_handlers.py", line 180, in _build_study_info
    study, study_proc, proc_samples)

  File "/Users/yoshikivazquezbaeza/git_sw/qiita/qiita_pet/handlers/study_handlers/listing_handlers.py", line 104, in _build_single_proc_data_info
    proc_data = ProcessedData(pid)

  File "/Users/yoshikivazquezbaeza/git_sw/qiita/qiita_db/base.py", line 154, in __init__
    if not self._check_id(id_):

  File "/Users/yoshikivazquezbaeza/git_sw/qiita/qiita_db/base.py", line 140, in _check_id
    "{0}_id=%s)".format(self._table), (id_, ))[0]

  File "/Users/yoshikivazquezbaeza/git_sw/qiita/qiita_db/sql_connection.py", line 486, in execute_fetchone
    with self._sql_executor(sql, sql_args) as pgcursor:

  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()

  File "/Users/yoshikivazquezbaeza/git_sw/qiita/qiita_db/sql_connection.py", line 303, in _sql_executor
    (sql, str(sql_args), e)))

QiitaDBExecutionError: 
Error running SQL query: SELECT EXISTS(SELECT * FROM qiita.processed_data WHERE processed_data_id=%s)
ARGS: ('18S',)
Error: invalid input syntax for integer: "18S"
LINE 1: ...T * FROM qiita.processed_data WHERE processed_data_id='18S')
                                                                 ^



HTTP INFO:
<strong>body</strong>: 
<strong>files</strong>: {}
<strong>protocol</strong>: http
<strong>connection</strong>: <tornado.httpserver.HTTPConnection object at 0x10a651050>
<strong>uri</strong>: /study/search/?&user=test@foo.bar&sEcho=112&query=study_title%20includes%20%22soil%22&_=1429033642672
<strong>_start_time</strong>: 1429033649.39
<strong>headers</strong>: {'Accept-Language': 'en-us', 'Accept-Encoding': 'gzip, deflate', 'X-Requested-With': 'XMLHttpRequest', 'Connection': 'keep-alive', 'Accept': 'application/json, text/javascript, */*; q=0.01', 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/601.1.27+ (KHTML, like Gecko) Version/8.0.5 Safari/600.5.17', 'Host': 'localhost:1111', 'Cookie': 'user="InRlc3RAZm9vLmJhciI=|1429033402|3efbd25c076741d3cceedf0d8c66ce026b070a6b"', 'Referer': 'http://localhost:1111/study/list/'}
<strong>host</strong>: localhost:1111
<strong>version</strong>: HTTP/1.1
<strong>arguments</strong>: {'query': ['study_title includes "soil"'], 'user': ['test@foo.bar'], '_': ['1429033642672'], 'sEcho': ['112']}
<strong>_finish_time</strong>: 1429033650.24
<strong>query</strong>: &user=test@foo.bar&sEcho=112&query=study_title%20includes%20%22soil%22&_=1429033642672
<strong>path</strong>: /study/search/
<strong>_cookies</strong>: Set-Cookie: user="InRlc3RAZm9vLmJhciI=|1429033402|3efbd25c076741d3cceedf0d8c66ce026b070a6b"
<strong>method</strong>: GET
<strong>remote_ip</strong>: ::1


Info:

squirrelo · 2015-04-14T20:38:26Z

Thanks @ElDeveloper, was a problem with dict formatting without searches. Fixed

coveralls · 2015-04-14T21:35:01Z

Coverage increased (+0.09%) to 79.1% when pulling fbcd86c on squirrelo:list-proc-data-in-search into dfc2538 on biocore:cart-branch.

List proc data in search

ElDeveloper · 2015-04-14T21:36:06Z

Thanks @squirrelo

squirrelo added 3 commits April 2, 2015 14:17

return proc data info as part of json and start rendering it

28e6f57

Merge branch 'cart-analysis-changes', remote-tracking branch 'origin'…

59286a7

… into list-proc-data-in-search

show proc data for each study properly

9227ee3

update tests with proc_data_info

6f08836

Merge branch 'cart-branch' of https://github.com/biocore/qiita into l…

7b386c3

…ist-proc-data-in-search

josenavas reviewed Apr 8, 2015
View reviewed changes

squirrelo added metanalysis priority: high labels Apr 8, 2015

squirrelo added this to the Alpha 0.1 milestone Apr 8, 2015

squirrelo added 5 commits April 8, 2015 19:54

refactor building datatable

545a213

Merge branch 'master' of https://github.com/biocore/qiita into list-p…

07c9da2

…roc-data-in-search

more client-side rendering

350b864

update tests

7e8c447

flake8

7eedf17

Merge branch 'cart-branch' of https://github.com/biocore/qiita into l…

48c6760

…ist-proc-data-in-search

squirrelo added 2 commits April 12, 2015 16:56

rename and add docstrings

10b0c5c

clean up tests

3fe2af9

squirrelo added 2 commits April 13, 2015 09:58

implement suggestions

c16b429

flake8

ccd7d4a

josenavas reviewed Apr 13, 2015
View reviewed changes

ElDeveloper reviewed Apr 13, 2015
View reviewed changes

more changes for suggestions

179b582

fix building study info to match search results

b6c0f44

update tests

fbcd86c

ElDeveloper added a commit that referenced this pull request Apr 14, 2015

Merge pull request #1039 from squirrelo/list-proc-data-in-search

2e0e6e5

List proc data in search

ElDeveloper merged commit 2e0e6e5 into qiita-spots:cart-branch Apr 14, 2015

List proc data in search #1039

List proc data in search #1039

Uh oh!

Conversation

squirrelo commented Apr 6, 2015

Uh oh!

squirrelo commented Apr 6, 2015

Uh oh!

coveralls commented Apr 6, 2015

Uh oh!

squirrelo commented Apr 7, 2015

Uh oh!

coveralls commented Apr 7, 2015

Uh oh!

josenavas Apr 8, 2015

Choose a reason for hiding this comment

Uh oh!

josenavas commented Apr 8, 2015

Uh oh!

squirrelo commented Apr 8, 2015

Uh oh!

squirrelo commented Apr 9, 2015

Uh oh!

josenavas commented Apr 9, 2015

Uh oh!

squirrelo commented Apr 9, 2015

Uh oh!

squirrelo commented Apr 9, 2015

Uh oh!

josenavas commented Apr 9, 2015

Uh oh!

coveralls commented Apr 13, 2015

Uh oh!

squirrelo commented Apr 13, 2015

Uh oh!

josenavas Apr 13, 2015

Choose a reason for hiding this comment

Uh oh!

josenavas commented Apr 13, 2015

Uh oh!

ElDeveloper Apr 13, 2015

Choose a reason for hiding this comment

Uh oh!

squirrelo Apr 13, 2015

Choose a reason for hiding this comment

Uh oh!

ElDeveloper commented Apr 13, 2015

Uh oh!

ElDeveloper commented Apr 13, 2015

Uh oh!

coveralls commented Apr 13, 2015

Uh oh!

squirrelo commented Apr 13, 2015

Uh oh!

ElDeveloper commented Apr 13, 2015

Uh oh!

ElDeveloper commented Apr 14, 2015

Uh oh!

squirrelo commented Apr 14, 2015

Uh oh!

coveralls commented Apr 14, 2015

Uh oh!

ElDeveloper commented Apr 14, 2015

Uh oh!

Uh oh!