domain: add recent slow queries queue and ShowSlowQuery() function #7692
Conversation
|
Why put it in the domain package? |
|
Where should I put it ? @shenli |
|
PTAL @coocood |
domain/topn_slow_query.go
Outdated
| count = len(q.data) | ||
| } | ||
| ret := make([]*SlowQueryInfo, 0, count) | ||
| tail := (q.tail - 1 + len(q.data)) % len(q.data) |
There was a problem hiding this comment.
This is simpler
if q.head > q.tail {
ret = append(ret, q.data[head:]...)
ret = append(ret, q.data[:head]...)
} else {
ret = append(ret, q.data[head:tail+1]...)
}
ret = ret[len(ret)-count:]There was a problem hiding this comment.
It's not simpler, it brings more mental burden.
domain/topn_slow_query.go
Outdated
| ret = append(ret, h.data[i]) | ||
| } | ||
|
|
||
| // Sort breaks the data, recover it here. |
There was a problem hiding this comment.
The sorted array still maintains the heap property, we don't need to copy and recover.
There was a problem hiding this comment.
There are two problems here:
- To keep top N slow queries, we maintains a min heap, but sort will use a reverse order
- There is a
countargument, so thesorted[count]is not the same withheap[len]
|
PTAL @coocood |
domain/topn_slow_query.go
Outdated
| @@ -102,8 +156,46 @@ func (q *topNSlowQueries) Append(info *SlowQueryInfo) { | |||
| } | |||
|
|
|||
| func (q *topNSlowQueries) Refresh(now time.Time) { | |||
There was a problem hiding this comment.
I think Refresh may not be a good name for its functionality.
What it does is cleaning up outdated entries, but Refresh usually means getting something new.
How about RemoveExpired.
|
PTAL @coocood |
|
LGTM |
|
PTAL @winkyao |
tiancaiamao
added
status/LGT1
domain/topn_slow_query.go
Outdated
| tmp1 := slowQueryHeap{tmp} | ||
| sort.Sort(&tmp1) | ||
| ret = make([]*SlowQueryInfo, 0, count) | ||
| for i := len(tmp) - 1; i >= 0 && len(ret) <= count; i-- { |
ast/misc.go
Outdated
| @@ -626,6 +626,26 @@ type HandleRange struct { | |||
| End int64 | |||
| } | |||
|
|
|||
| // ShowLogType defines the type for SlowLog. | |||
| type ShowLogType int | |||
There was a problem hiding this comment.
s/ShowLogType/ShowSlowQueryType/
There was a problem hiding this comment.
When I merge master, this would be solved.
ast/misc.go
Outdated
| // ShowLog is used for the following command: | ||
| // admin show log top [user | internal | all] N | ||
| // admin show log recent N | ||
| type ShowLog struct { |
|
/run-all-tests |
|
PTAL @crazycs520 @zz-jason |
|
/run-all-tests |
| return | ||
| } | ||
|
|
||
| q.data = append(q.data, info)[1:] |
There was a problem hiding this comment.
If you just use slice here, the q.data will keep the continuous memory, it may cause oom. What about using a list?
There was a problem hiding this comment.
Do you do a large amount insert test?
There was a problem hiding this comment.
@coocood suggest me to use this.
The code is the simplest queue implementation, although it double the memory usage.
There was a problem hiding this comment.
The queue size will be 500.
We just keep the most recent queries in the queue.
What problem does this PR solve?
Make user easier to debug slow queries.
What is changed and how it works?
ShowSlowQuery()functionCheck List
Tests
Code changes
domain.ShowSlowQuery()Related changes
It's provided to support
admin show login #7688@winkyao @coocood