*: track the memory usage in Insert/Update/Delete executors #34097
Conversation
|
[REVIEW NOTIFICATION] This pull request has been approved by:
To complete the pull request process, please ask the reviewers in the list to review by filling The full list of commands accepted by this bot can be found here. Reviewer can indicate their review by submitting an approval review. |
ti-chi-bot
added
release-note-none
Signed-off-by: ekexium <ekexium@gmail.com>
ekexium
force-pushed
the
update-memtrack
branch
from
c95c208 to
ada8ef9
Compare
|
Code Coverage Details: https://codecov.io/github/pingcap/tidb/commit/35a1267ab5573c63e91dea0232bc39bfbb640a37 |
ekexium
force-pushed
the
update-memtrack
branch
2 times, most recently
from
e3e9a91 to
3efd06e
Compare
Signed-off-by: ekexium <ekexium@gmail.com>
ekexium
force-pushed
the
update-memtrack
branch
from
3efd06e to
0a00117
Compare
|
/run-check_dev |
ti-chi-bot
added
the
needs-rebase
Signed-off-by: ekexium <ekexium@gmail.com>
|
/run-check_dev |
1 similar comment
|
/run-check_dev |
Signed-off-by: ekexium <ekexium@gmail.com>
Signed-off-by: ekexium <ekexium@gmail.com>
Signed-off-by: ekexium <ekexium@gmail.com>
Signed-off-by: ekexium <ekexium@gmail.com>
ekexium
changed the title
Signed-off-by: ekexium <ekexium@gmail.com> fix tblRowMap Signed-off-by: ekexium <ekexium@gmail.com> try Signed-off-by: ekexium <ekexium@gmail.com> try again Signed-off-by: ekexium <ekexium@gmail.com> try again Signed-off-by: ekexium <ekexium@gmail.com>
Signed-off-by: ekexium <ekexium@gmail.com>
ti-chi-bot
removed
the
needs-rebase
|
Please take another look @wshwsh12 |
Signed-off-by: ekexium <ekexium@fastmail.com>
Signed-off-by: ekexium <ekexium@fastmail.com>
ti-chi-bot
added
size/XL
Signed-off-by: ekexium <ekexium@fastmail.com>
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| package set |
There was a problem hiding this comment.
Can we add an individual package like map or tiMap to put these files?
There was a problem hiding this comment.
Map and set are highly related. And map is reserved so I think leaving them in set is fine.
kv/key.go
Outdated
|
|
||
| // ExtraMemSize implements the Handle interface. | ||
| func (ch *CommonHandle) ExtraMemSize() uint64 { | ||
| return uint64(len(ch.encoded) + len(ch.colEndOffsets)*2) |
There was a problem hiding this comment.
1.cap(ch.encoded) and cap(ch.colEndOffsets)?
2. Why do we need to *2?
There was a problem hiding this comment.
colEndOffsets is an array of uint16.
kv/key.go
Outdated
| @@ -158,9 +159,15 @@ type Handle interface { | |||
| // String implements the fmt.Stringer interface. | |||
| String() string | |||
| // MemUsage returns the memory usage of a handle. | |||
| MemUsage() int64 | |||
| MemUsage() uint64 | |||
| // ExtraMemSize returns the dynamic size of memory occupied by the handle, e.g. slices. | |||
There was a problem hiding this comment.
What does dynamic mean? How do we use this func?
There was a problem hiding this comment.
dynamic means the part of memory is not a fixed number, i.e. slices or pointers. Do you have any suggestions on improving the comment?
We use it with the mem-aware map, since the map only counts the actual size of the keys and values, but doesn't contain the objects that are referenced by pointers.
| return x | ||
| } | ||
|
|
||
| func BenchmarkMemAwareIntMap(b *testing.B) { |
There was a problem hiding this comment.
This benchmark may be not convincing, I think we'd better add a microbenchmark for the MemAwareHandleMap.
There was a problem hiding this comment.
I've updated results in the description.
Signed-off-by: ekexium <ekexium@fastmail.com>
Signed-off-by: ekexium <ekexium@fastmail.com>
ti-chi-bot
added
status/LGT2
|
/merge |
|
This pull request has been accepted and is ready to merge. Commit hash: 6b154a9
|
ti-chi-bot
added
the
status/can-merge
TiDB MergeCI notify
|
Signed-off-by: ekexium ekexium@gmail.com
What problem does this PR solve?
Issue Number: close #34096
Problem Summary:
Track the memory usage in update and delete executors.
What is changed and how it works?
Introduce a new generic type
MemAwareMap.Check List
Tests
I manually tested delete and update.
explain analyze delete t_3, t_4 from t_3, t_4 where t_3.vip_level = t_4.vip_levelupdate `T_CUSTOMER` set purse_num = purse_num + 1 where vip_level = '0'Side effects
Documentation
Release note
Please refer to Release Notes Language Style Guide to write a quality release note.