Skip to content

DataTables load using AJAX, Search by metadata on studies page #1006

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 29 commits into from
Apr 1, 2015
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
616300f
add get_info to Studies object
squirrelo Mar 19, 2015
d595016
expand tests
squirrelo Mar 19, 2015
90313f7
use get_info in listing handler
squirrelo Mar 19, 2015
fb5eef3
condense view studies pages
squirrelo Mar 19, 2015
1ffd4bc
clean up naming, add form elements
squirrelo Mar 19, 2015
16257c4
load study tables through ajax
squirrelo Mar 19, 2015
9ccc098
add message for zero results
squirrelo Mar 19, 2015
abeb46f
add search and retrieve to page
squirrelo Mar 20, 2015
259bec2
fix bug in joins
squirrelo Mar 20, 2015
b8472a8
add full ajax search support
squirrelo Mar 20, 2015
cf6ca77
add error handling to datatables
squirrelo Mar 20, 2015
cd4f66e
alter/add tests for handlers to reflect changes
squirrelo Mar 20, 2015
ce4d334
fix tests
squirrelo Mar 20, 2015
58b28cd
implement Jess' suggestions
squirrelo Mar 21, 2015
1872555
Merge branch 'master' of https://github.com/biocore/qiita into cart
squirrelo Mar 22, 2015
4b7ffcf
remove unneeded format loop, format datatables better
squirrelo Mar 24, 2015
515f0f6
update tests
squirrelo Mar 24, 2015
21b17d6
Merge branch 'master' of https://github.com/biocore/qiita into cart
squirrelo Mar 24, 2015
04f87f4
specific ordering for PMIDs for testing
squirrelo Mar 24, 2015
ee4c9e2
flake8
squirrelo Mar 24, 2015
ef16d18
address issues
squirrelo Mar 26, 2015
fbe7c74
address comments
squirrelo Mar 26, 2015
e0f22d0
merge upstream/master
squirrelo Mar 27, 2015
0df6666
fix merge issues
squirrelo Mar 27, 2015
5085c35
more enhancements
squirrelo Mar 27, 2015
ea69fe8
address comments
squirrelo Mar 28, 2015
3a12f87
update tests
squirrelo Mar 28, 2015
25ae8f2
update schema and html files
squirrelo Mar 29, 2015
e01e224
fix search disabled and clearing table search
squirrelo Apr 1, 2015
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 29 additions & 1 deletion qiita_db/study.py
Original file line number Diff line number Diff line change
Expand Up @@ -98,12 +98,13 @@
from __future__ import division
from future.utils import viewitems
from copy import deepcopy
from itertools import chain

from qiita_core.exceptions import IncompetentQiitaDeveloperError
from .base import QiitaStatusObject, QiitaObject
from .exceptions import (QiitaDBStatusError, QiitaDBColumnError, QiitaDBError)
from .util import (check_required_columns, check_table_cols, convert_to_id,
get_environmental_packages)
get_environmental_packages, get_table_cols)
from .sql_connection import SQLConnectionHandler


Expand Down Expand Up @@ -143,6 +144,11 @@ class Study(QiitaStatusObject):
_table = "study"
# The following columns are considered not part of the study info
_non_info = {"email", "study_status_id", "study_title"}
# The following tables are considered part of info
_info_cols = frozenset(chain(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

None of this information will change at runtime, right? If so, I think this could be cached in qiita_pet.__init__ and done once

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah! nevermind, it is at the static scope of the object. it is done once

get_table_cols('study'), get_table_cols('study_status'),
get_table_cols('timeseries_type'), get_table_cols('portal_type'),
get_table_cols('study_pmid')))

def _lock_non_sandbox(self, conn_handler):
"""Raises QiitaDBStatusError if study is non-sandboxed"""
Expand Down Expand Up @@ -175,6 +181,28 @@ def get_by_status(cls, status):
"ss.status = %s".format(cls._table))
return {x[0] for x in conn_handler.execute_fetchall(sql, (status, ))}

@classmethod
def get_info(cls, study_ids, info_cols=None):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a docstring?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name of this function is a bit confusing to me; I actually had to go and check how the instance method is called. Can we change it to something like def get_info_by_study and study_ids is optional (if not provided, it gives all the information for all the studies, probably useful from an admin point of view - that has access to everything).

if info_cols is None:
info_cols = [s for s in cls._info_cols]
else:
info_cols = deepcopy(info_cols)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't seem like there's any need for this variable to be copied, am I missing something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remnant of old way of doing something, thanks for catching.


search_cols = ",".join(sorted(cls._info_cols.intersection(info_cols)))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might raise a warning if the user requests information columns that are not in cls._info_cols, since right now it silently ignores them, which could lead to bugs if/when there are typos or misspellings


sql = """SELECT {0} FROM (
qiita.study
JOIN qiita.study_status USING (study_status_id)
JOIN qiita.timeseries_type USING (timeseries_type_id)
JOIN qiita.portal_type USING (portal_type_id)
LEFT JOIN (SELECT study_id, array_agg(pmid ORDER BY study_id) as
pmid FROM qiita.study_pmid GROUP BY study_id) sp USING (study_id)
) WHERE study_id in ({1})""".format(
search_cols, ','.join(str(s) for s in study_ids))

conn_handler = SQLConnectionHandler()
return conn_handler.execute_fetchall(sql)

@classmethod
def exists(cls, study_title):
"""Check if a study exists based on study_title, which is unique
Expand Down
6 changes: 6 additions & 0 deletions qiita_db/support_files/patches/19.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
-- March 19, 2015
-- Rename columns to be more descirptive and allow easier joins
ALTER TABLE qiita.study_status RENAME COLUMN description TO status_description;
ALTER TABLE qiita.portal_type RENAME COLUMN description TO portal_description;
ALTER TABLE qiita.investigation RENAME COLUMN description TO investigation_description;
ALTER TABLE qiita.investigation RENAME COLUMN name TO investigation_name;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised that these changes don't affect more code, but I haven't had a chance to look closely

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

None of these columns are actually used anywhere, since investigations and portal types aren't implemented yet, and study_status description is never touched, only the status value itself.

2 changes: 1 addition & 1 deletion qiita_db/support_files/populate_test_db.sql
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ INSERT INTO qiita.study_users (study_id, email) VALUES (1, 'shared@foo.bar');
INSERT INTO qiita.study_pmid (study_id, pmid) VALUES (1, '123456'), (1, '7891011');

-- Insert an investigation
INSERT INTO qiita.investigation (name, description, contact_person_id) VALUES
INSERT INTO qiita.investigation (investigation_name, investigation_description, contact_person_id) VALUES
('TestInvestigation', 'An investigation for testing purposes', 3);

-- Insert investigation_study (link study 1 with investigation 1)
Expand Down
43 changes: 43 additions & 0 deletions qiita_db/test/test_study.py
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,49 @@ def _make_sandbox(self):
# make studies private
self.conn_handler.execute("UPDATE qiita.study SET study_status_id = 1")

def test_get_info(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add another test in which there is more than 1 study? You can use the study create to add a new study (note that if the creation fails, we already have tests for the create function that will fail....)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's tested indirectly in the study handler tests, but if you want one here I can do it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something on qiita_db cannot be safely tested through something on qiita_pet... please add test here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You added a new test but it is still testing 1 single study, could you test multiple? Study.get_info([1, 2])

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line 232

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, I was reading how it works wrongly, thanks.

obs = Study.get_info([1])
self.assertEqual(len(obs), 1)
exp_keys = ['mixs_compliant', 'metadata_complete', 'reprocess',
'timeseries_type', 'portal_description', 'emp_person_id',
'number_samples_promised', 'funding', 'vamps_id',
'first_contact', 'principal_investigator_id',
'study_status_id', 'timeseries_type_id', 'study_abstract',
'pmid', 'study_alias', 'status', 'spatial_series',
'study_description', 'portal', 'status_description',
'portal_type_id', 'intervention_type', 'email', 'study_id',
'most_recent_contact', 'lab_person_id', 'study_title',
'number_samples_collected']
exp_vals = [
['123456', '7891011'], 'test@foo.bar', 2,
datetime(2014, 5, 19, 16, 10), None, 'None', 1, True, True,
datetime(2014, 5, 19, 16, 11), 27, 27, 'EMP', 'EMP portal', 2, 3,
False, False, 'private',
'Only owner and shared users can see this study',
'This is a preliminary study to examine the microbiota associated '
'with the Cannabis plant. Soils samples from the bulk soil, soil '
'associated with the roots, and the rhizosphere were extracted and'
' the DNA sequenced. Roots from three independent plants of '
'different strains were examined. These roots were obtained '
'November 11, 2011 from plants that had been harvested in the '
'summer. Future studies will attempt to analyze the soils and '
'rhizospheres from the same location at different time points in '
'the plant lifecycle.', 'Cannabis Soils', 'Analysis of the '
'Cannabis Plant Microbiome', 1, 3, 'Identification of the '
'Microbiomes for Cannabis Soils', 'None', 1, None]
self.assertItemsEqual(obs[0].keys(), exp_keys)
self.assertItemsEqual(obs[0], exp_vals)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not define your expected keys/values as an actual dict and use self.assertEqual? As of python 2.7, the correct equality method will automatically be called. This is not blocking, just thought it was a little unintuitive.


exp_keys = ['metadata_complete', 'reprocess', 'timeseries_type',
'portal_description', 'pmid', 'study_title']
obs = Study.get_info([1], exp_keys)
self.assertEqual(len(obs), 1)
exp_vals = [
True, 'EMP portal', False, 'Identification of the Microbiomes for '
'Cannabis Soils', 'None', ['123456', '7891011']]
self.assertItemsEqual(obs[0].keys(), exp_keys)
self.assertItemsEqual(obs[0], exp_vals)

def test_has_access_public(self):
self.study.status = 'public'
self.assertTrue(self.study.has_access(User("demo@microbio.me")))
Expand Down
11 changes: 5 additions & 6 deletions qiita_pet/handlers/study_handlers/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,16 @@
# The full license is in the file LICENSE, distributed with this software.
# -----------------------------------------------------------------------------

from .listing_handlers import (PrivateStudiesHandler, PublicStudiesHandler,
StudyApprovalList, ShareStudyAJAX)
from .listing_handlers import (ListStudiesHandler, StudyApprovalList,
ShareStudyAJAX, SearchStudiesAJAX)
from .edit_handlers import StudyEditHandler, CreateStudyAJAX
from .description_handlers import (StudyDescriptionHandler,
PreprocessingSummaryHandler)
from .ebi_handlers import EBISubmitHandler
from .metadata_summary_handlers import MetadataSummaryHandler
from .vamps_handlers import VAMPSHandler

__all__ = ['PrivateStudiesHandler', 'PublicStudiesHandler',
'StudyApprovalList', 'ShareStudyAJAX', 'StudyEditHandler',
'CreateStudyAJAX', 'StudyDescriptionHandler',
__all__ = ['ListStudiesHandler', 'StudyApprovalList', 'ShareStudyAJAX',
'StudyEditHandler', 'CreateStudyAJAX', 'StudyDescriptionHandler',
'PreprocessingSummaryHandler', 'EBISubmitHandler',
'MetadataSummaryHandler', 'VAMPSHandler']
'MetadataSummaryHandler', 'VAMPSHandler', 'SearchStudiesAJAX']
Loading