Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Indicate vertex-centric index used in profile step #2283

Closed
li-boxuan opened this issue Dec 14, 2020 · 4 comments · Fixed by #2360
Closed

Indicate vertex-centric index used in profile step #2283

li-boxuan opened this issue Dec 14, 2020 · 4 comments · Fixed by #2360

Comments

@li-boxuan
Copy link
Member

Composite indexes and mixed indexes, if used, would show up in the profiler result. However, profile() step does not indicate if and which vertex-centric index is used.

See https://groups.google.com/g/janusgraph-users/c/SqjYfsEInrQ/m/RDWALyDMAgAJ

Related issue: #898

@cmilowka
Copy link

Thank you for considering this enhancement.
Datastax, which is sharing a lot of the code with JG, is having something of this kind, referring to:
https://docs.datastax.com/en/dse/6.0/dse-dev/datastax_enterprise/graph/using/useIndexes.html
quote: "If an index was not used, index-query would be missing from the profile output" where index-query look like:

**index-query** :
    \_indexType=Materialized
   \_statement=SELECT "personId" FROM "dse60"."person_p_byName" 
      WHERE "name" = ? LIMIT ?; with params (java. lang.String) Emeril Lagasse, (java.lang.Integer) 50000

We already have a list of conditions considered for backend-query, sample:
\_condition=(m_at >= 1589592639360 AND m_to <= 9223372036854775807 AND type[v_StartTime_t])
what we are probably missing is to have a confirmation about the index that was used to support that
\_index=MY_INDEX (m_at >= 1589592639360 AND type[v_StartTime_t])
so conditions for m_to need to be filtered in-memory, and it is why isFitted=false.
Regards, Christopher

li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Dec 20, 2020
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
@li-boxuan li-boxuan mentioned this issue Dec 20, 2020
10 tasks
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Dec 20, 2020
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Dec 20, 2020
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Dec 27, 2020
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Dec 27, 2020
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Dec 27, 2020
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Dec 28, 2020
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 3, 2021
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
@li-boxuan
Copy link
Member Author

@cmilowka Thanks for the info. How the new profile step will look like is in #2289. You may take a look and see if it helps you.

I admit it is not as detailed & clear as the DSE Graph version. To achieve that there is more work to be done in the future.

li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 5, 2021
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 5, 2021
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 9, 2021
1. Add toString() method to SliceQuery to indicate which vertex-centric
index is used in edge queries.
2. Indicate "GraphCentricQuery" and "VertexCentricQuery" in profiler if applicable.
Previously they were incorrectly merged, causing corrupted annotations.
3. Indicate "constructGraphCentricQuery" phase in profiler rather than a vague
"optimization" annotation, if applicable.
4. Fix corrupted annotations caused by nested profile flattening issue. Now
AND/OR nesting is flattened only if it does not have siblings.

Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 9, 2021
1. Add toString() method to SliceQuery to indicate which vertex-centric
index is used in edge queries.
2. Indicate "GraphCentricQuery" and "VertexCentricQuery" in profiler if applicable.
Previously they were incorrectly merged, causing corrupted annotations.
3. Indicate "constructGraphCentricQuery" phase in profiler rather than a vague
"optimization" annotation, if applicable.
4. Fix corrupted annotations caused by nested profile flattening issue. Now
AND/OR nesting is flattened only if it does not have siblings.

Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 10, 2021
1. Add toString() method to SliceQuery to indicate which vertex-centric
index is used in edge queries.
2. Indicate "GraphCentricQuery" and "VertexCentricQuery" in profiler if applicable.
Previously they were incorrectly merged, causing corrupted annotations.
3. Indicate "constructGraphCentricQuery" phase in profiler rather than a vague
"optimization" annotation, if applicable.
4. Fix corrupted annotations caused by nested profile flattening issue. Now
AND/OR nesting is flattened only if it does not have siblings.

Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
@cmilowka
Copy link

cmilowka commented Jan 14, 2021

Thanks you so much for the effort,... desperate to try that asap. I have a feeling that more questions will pop up, like why indexes are not used ;-) Christopher

li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 14, 2021
1. Add toString() method to SliceQuery to indicate which vertex-centric
index is used in edge queries.
2. Indicate "GraphCentricQuery" and "VertexCentricQuery" in profiler if applicable.
Previously they were incorrectly merged, causing corrupted annotations.
3. Indicate "constructGraphCentricQuery" phase in profiler rather than a vague
"optimization" annotation, if applicable.
4. Fix corrupted annotations caused by nested profile flattening issue. Now
AND/OR nesting is flattened only if it does not have siblings.

Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 14, 2021
1. Add toString() method to SliceQuery to indicate which vertex-centric
index is used in edge queries.
2. Indicate "GraphCentricQuery" and "VertexCentricQuery" in profiler if applicable.
Previously they were incorrectly merged, causing corrupted annotations.
3. Indicate "constructGraphCentricQuery" phase in profiler rather than a vague
"optimization" annotation, if applicable.
4. Fix corrupted annotations caused by nested profile flattening issue. Now
AND/OR nesting is flattened only if it does not have siblings.
5. Remove backend-query subgroup to reduce nesting levels in profiler result.
Now whether a backend-query is fired can be inferred from "isCached" annotation.
6. Remove fullscan subgroup to reduce nesting levels in profiler result. Now
whether a full scan is done can be inferred from "fullscan" annotation.

Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 15, 2021
Closes JanusGraph#898, JanusGraph#2283

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 15, 2021
Closes JanusGraph#898, JanusGraph#2283

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 16, 2021
Closes JanusGraph#898, JanusGraph#2283

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit that referenced this issue Jan 17, 2021
Closes #898, #2283

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
@li-boxuan
Copy link
Member Author

Fixed by #2360

@li-boxuan li-boxuan linked a pull request Jan 18, 2021 that will close this issue
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants