Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Profiler annotations are wrong when there are multiple backend queries #2285

Closed
li-boxuan opened this issue Dec 17, 2020 · 0 comments · Fixed by #2365 or #2410
Closed

Profiler annotations are wrong when there are multiple backend queries #2285

li-boxuan opened this issue Dec 17, 2020 · 0 comments · Fixed by #2365 or #2410
Assignees

Comments

@li-boxuan
Copy link
Member

For confirmed bugs, please report:

  • Version: 0.5.2, 0.6.0-SNAPSHOT
  • Storage Backend: does not matter
  • Mixed Index Backend: does not matter
  • Steps to reproduce:
graph = JanusGraphFactory.open("inmemory")
mgmt = graph.openManagement()
nameKey = mgmt.makePropertyKey('name').dataType(String.class).make()
ageKey = mgmt.makePropertyKey('age').dataType(Integer.class).make()
mgmt.buildIndex("name", Vertex.class).addKey(nameKey).buildCompositeIndex()
mgmt.buildIndex("age", Vertex.class).addKey(ageKey).buildCompositeIndex()
mgmt.commit()

g = graph.traversal()
g.addV().property("name", "bob").property("age", 60)
graph.tx().commit()

When there is more than one backend query, annotations (condition, orders, isFitted, isOrdered, query, index) of the last one overwrites previous annotations.

  1. Multiple conditions under Or
g.V().or(has("name", "bob"), has("age", 60)).profile()

==>Traversal Metrics
Step                                                               Count  Traversers       Time (ms)    % Dur
=============================================================================================================
Or(JanusGraphStep([],[name.eq(bob)]),JanusGraph...                     1           1           5.852   100.00
    \_condition=(age = 60)
    \_orders=[]
    \_isFitted=true
    \_isOrdered=true
    \_query=multiKSQ[1]@2000
    \_index=age
  optimization                                                                                 0.621
  optimization                                                                                 0.175
  optimization                                                                                 0.129
  backend-query                                                        1                       0.730
    \_query=name:multiKSQ[1]@2000
    \_limit=2000
  backend-query                                                        1                       1.396
    \_query=age:multiKSQ[1]@2000
    \_limit=2000
                                            >TOTAL                     -           -           5.852        -

Note the following part

   \_condition=(age = 60)
    \_orders=[]
    \_isFitted=true
    \_isOrdered=true
    \_query=multiKSQ[1]@2000
    \_index=age

only represents the second backend-query.

  1. Multiple conditions under And
    Same issue:
g.V().has("name", "bob").has("age", 60).profile()


==>Traversal Metrics
Step                                                               Count  Traversers       Time (ms)    % Dur
=============================================================================================================
JanusGraphStep([],[name.eq(bob), age.eq(60)])                          1           1           1.331   100.00
    \_condition=(name = bob AND age = 60)
    \_orders=[]
    \_isFitted=true
    \_isOrdered=true
    \_query=multiKSQ[1]@2147483647
    \_index=age
  optimization                                                                                 0.013
  optimization                                                                                 0.218
  backend-query                                                        1                       0.119
    \_query=age:multiKSQ[1]@2147483647
  backend-query                                                        1                       0.094
    \_query=name:multiKSQ[1]@2147483647
                                    >TOTAL                     -           -           1.331        -
@li-boxuan li-boxuan self-assigned this Dec 17, 2020
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Dec 20, 2020
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
@li-boxuan li-boxuan mentioned this issue Dec 20, 2020
10 tasks
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Dec 20, 2020
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Dec 20, 2020
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
@li-boxuan li-boxuan added this to the Release v0.6.0 milestone Dec 20, 2020
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Dec 27, 2020
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Dec 27, 2020
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Dec 27, 2020
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Dec 28, 2020
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 3, 2021
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 5, 2021
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 5, 2021
Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 9, 2021
1. Add toString() method to SliceQuery to indicate which vertex-centric
index is used in edge queries.
2. Indicate "GraphCentricQuery" and "VertexCentricQuery" in profiler if applicable.
Previously they were incorrectly merged, causing corrupted annotations.
3. Indicate "constructGraphCentricQuery" phase in profiler rather than a vague
"optimization" annotation, if applicable.
4. Fix corrupted annotations caused by nested profile flattening issue. Now
AND/OR nesting is flattened only if it does not have siblings.

Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 9, 2021
1. Add toString() method to SliceQuery to indicate which vertex-centric
index is used in edge queries.
2. Indicate "GraphCentricQuery" and "VertexCentricQuery" in profiler if applicable.
Previously they were incorrectly merged, causing corrupted annotations.
3. Indicate "constructGraphCentricQuery" phase in profiler rather than a vague
"optimization" annotation, if applicable.
4. Fix corrupted annotations caused by nested profile flattening issue. Now
AND/OR nesting is flattened only if it does not have siblings.

Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 10, 2021
1. Add toString() method to SliceQuery to indicate which vertex-centric
index is used in edge queries.
2. Indicate "GraphCentricQuery" and "VertexCentricQuery" in profiler if applicable.
Previously they were incorrectly merged, causing corrupted annotations.
3. Indicate "constructGraphCentricQuery" phase in profiler rather than a vague
"optimization" annotation, if applicable.
4. Fix corrupted annotations caused by nested profile flattening issue. Now
AND/OR nesting is flattened only if it does not have siblings.

Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 14, 2021
1. Add toString() method to SliceQuery to indicate which vertex-centric
index is used in edge queries.
2. Indicate "GraphCentricQuery" and "VertexCentricQuery" in profiler if applicable.
Previously they were incorrectly merged, causing corrupted annotations.
3. Indicate "constructGraphCentricQuery" phase in profiler rather than a vague
"optimization" annotation, if applicable.
4. Fix corrupted annotations caused by nested profile flattening issue. Now
AND/OR nesting is flattened only if it does not have siblings.

Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 14, 2021
1. Add toString() method to SliceQuery to indicate which vertex-centric
index is used in edge queries.
2. Indicate "GraphCentricQuery" and "VertexCentricQuery" in profiler if applicable.
Previously they were incorrectly merged, causing corrupted annotations.
3. Indicate "constructGraphCentricQuery" phase in profiler rather than a vague
"optimization" annotation, if applicable.
4. Fix corrupted annotations caused by nested profile flattening issue. Now
AND/OR nesting is flattened only if it does not have siblings.
5. Remove backend-query subgroup to reduce nesting levels in profiler result.
Now whether a backend-query is fired can be inferred from "isCached" annotation.
6. Remove fullscan subgroup to reduce nesting levels in profiler result. Now
whether a full scan is done can be inferred from "fullscan" annotation.

Fixes JanusGraph#898, JanusGraph#2283, JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
@li-boxuan li-boxuan reopened this Jan 29, 2021
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Jan 30, 2021
Currently, when there is more than one backend query, annotations
(condition, orders, isFitted, isOrdered, query, index) of the last
one overwrites previous annotations in the profiling result.

This commit fixes this problem by avoiding flattening AND/OR query
groups. The drawback of this commit is those groups (AND-query, OR-query)
are not timed due to technical difficulty, which is left as future
work.

Fixes JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Feb 3, 2021
Currently, when there is more than one backend query, annotations
(condition, orders, isFitted, isOrdered, query, index) of the last
one overwrites previous annotations in the profiling result.

This commit fixes this problem by avoiding flattening AND/OR query
groups. The drawback of this commit is those groups (AND-query, OR-query)
are not timed due to technical difficulty, which is left as future
work.

Fixes JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Feb 4, 2021
Currently, when there is more than one backend query, annotations
(condition, orders, isFitted, isOrdered, query, index) of the last
one overwrites previous annotations in the profiling result.

This commit fixes this problem by avoiding flattening AND/OR query
groups. The drawback of this commit is those groups (AND-query, OR-query)
are not timed due to technical difficulty, which is left as future
work.

Fixes JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit to li-boxuan/janusgraph that referenced this issue Feb 25, 2021
Currently, when there is more than one backend query, annotations
(condition, orders, isFitted, isOrdered, query, index) of the last
one overwrites previous annotations in the profiling result.

This commit fixes this problem by avoiding flattening AND/OR query
groups. The drawback of this commit is those groups (AND-query, OR-query)
are not timed due to technical difficulty, which is left as future
work.

Fixes JanusGraph#2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
li-boxuan added a commit that referenced this issue Feb 27, 2021
Currently, when there is more than one backend query, annotations
(condition, orders, isFitted, isOrdered, query, index) of the last
one overwrites previous annotations in the profiling result.

This commit fixes this problem by avoiding flattening AND/OR query
groups. The drawback of this commit is those groups (AND-query, OR-query)
are not timed due to technical difficulty, which is left as future
work.

Fixes #2285

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant