Performance improvements retrieving files

[antgonza]

• Improve speed of opening the main meta-analysis page, this is still slow compared to opening a job. My guess a lot of object creation.
• Improve either how get_accessible_filepath_ids retrieves all ids or like in this case change it that so it doesn't retrieve all fp ids but just checking if fileA can be accessed by UserB

[wasade]

@antgonza, can you clarify? The main meta-analysis page seemed fast when I did them on qiita.colorado.edu

[antgonza]

The speed is related to the amount of things you have access to, try it as an admin. Note that (1) I'm talking about access to analysis that are already completed and (2) that I believe the problem comes from: https://github.com/biocore/qiita/blob/master/qiita_db/analysis.py#L470 as it basically is retrieving everything you have access to and then checking if element x is in that list. Perhaps the easiest way to do this is to simply have a single SQL command that checks in all tables with an EXISTS that returns if that element exists in one of those tables.

[wasade]

Oh, yes, that is not a scalable solution. Since we're dealing with IDs, do
we care about order? Any reason that get_by_status must return a list?
(same with the properties in question)?

On Tue, Mar 10, 2015 at 11:37 PM, Antonio Gonzalez <notifications@github.com

wrote:

The speed is related to the amount of things you have access to, try
it as an admin. Note that (1) I'm talking about access to analysis
that are already completed and (2) that I believe the problem comes
from:
https://github.com/biocore/qiita/blob/master/qiita_db/analysis.py#L470
as it basically is retrieving everything you have access to and then
checking if element x is in that list. Perhaps the easiest way to do
this is to simply have a single SQL command that checks in all tables
with an EXISTS that returns if that element exists in one of those
tables.

—
Reply to this email directly or view it on GitHub
#887 (comment).

[antgonza]

No, about order. AFAIK it's a nice to have to return a list of all elements for certain things but not for this ...

[wasade]

i'm gonna try toying with the order... the only possible use i can see of
the order is if its ascending or descending, in which case that can be done
external if necessary. the order returned ought to be interpreted as
arbitrary and as though it doesn't matter anyway

On Wed, Mar 11, 2015 at 12:20 AM, Antonio Gonzalez <notifications@github.com

wrote:

No, about order. AFAIK it's a nice to have to return a list of all elements
for certain things but not for this ...

—
Reply to this email directly or view it on GitHub
#887 (comment).

[antgonza]

Reopening as only one issue was addressed.