Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate speed issues #28

Open
nsbgn opened this issue Sep 4, 2021 · 3 comments
Open

Investigate speed issues #28

nsbgn opened this issue Sep 4, 2021 · 3 comments

Comments

@nsbgn
Copy link
Contributor

nsbgn commented Sep 4, 2021

Optimize SPARQL queries. FILTER NOT EXISTS is apparently not very fast.

@nsbgn nsbgn changed the title Optimize SPARQL queries Investigate speed issues Apr 6, 2022
@nsbgn nsbgn added this to the Version 0.2: RDF support milestone Apr 6, 2022
@nsbgn
Copy link
Contributor Author

nsbgn commented Apr 6, 2022

It is mostly the ordering of tools that is prohibitively expensive. If there is no way to address this in SPARQL itself, it might be an option to pre-calculate a matrix of accessibility from each tool application.

@nsbgn
Copy link
Contributor Author

nsbgn commented Aug 22, 2022

I think it can be addressed via subqueries. Crucially, we should accept the first occurrence of a type on a particular branch (and not also investigate any subsequent occurrences)

nsbgn added a commit that referenced this issue Aug 23, 2022
@nsbgn
Copy link
Contributor Author

nsbgn commented Sep 2, 2022

For checking ordering, we now traverse the transformation graph along the :to predicates and check the type of each node. This is (predictably) incredibly computationally intensive, and I do think that we should eventually do this another way. However, it's useful to have it working for testing, so I want to try one more thing:

Since transformation graphs contain a type tree with all relevant types, we can work from the other direction: first select all nodes with the correct type, then check if they are accessible from the current node. This should be far less intensive.

We could also filter out those nodes with the correct type that are accessible from other nodes with the correct type.

@nsbgn nsbgn removed this from the Version 0.2: RDF support milestone Nov 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant