lightning/backend/external: prevent escaping buffered slice to heap

[tangenta]

What problem does this PR solve?

Issue Number: ref #47757

Problem Summary:

tidb/br/pkg/lightning/backend/external/byte_reader.go

Lines 199 to 206 in 30288c7

	func (r *byteReader) readNBytes(n int) (*[]byte, error) {
	b := r.next(n)
	readLen := len(b)
	if readLen == n {
	ret := &b
	r.retPointers = append(r.retPointers, ret)
	return ret, nil
	}

The buffered slice is unexpectedly moved to heap, leading to unnecessary memory allocation for each key.

./byte_reader.go:199:7: leaking param content: r
./byte_reader.go:200:2: moved to heap: b
./byte_reader.go:209:2: moved to heap: auxBuf
./byte_reader.go:209:16: make([]byte, n) escapes to heap

According to verbose escape analysis, b escapes because we assign the address of local variables to a slice on heap:

./byte_reader.go:200:2: b escapes to heap:
./byte_reader.go:200:2:   flow: ret = &b:
./byte_reader.go:200:2:     from &b (address-of) at ./byte_reader.go:203:10
./byte_reader.go:200:2:     from ret := &b (assign) at ./byte_reader.go:203:7
./byte_reader.go:200:2:   flow: {heap} = ret:
./byte_reader.go:200:2:     from append(r.retPointers, ret) (call parameter) at ./byte_reader.go:204:25

What is changed and how it works?

This PR stores the buffer's start offset and length instead of the address. Although it looks like boilerplate, it is worth to reduce the memory allocation.

After this PR, b does not escape anymore:

./byte_reader.go:217:16: (*byteReader).readNBytes ignoring self-assignment in r.retPointersV2 = r.retPointersV2[:0]
./byte_reader.go:199:7: leaking param content: r
./byte_reader.go:204:10: &slice{...} escapes to heap
./byte_reader.go:214:16: make([]byte, n) escapes to heap

And here is the memory usage comparison:

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
  See above screenshot.
• No need to test

  • I checked and no code files have been changed.

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

[ti-chi-bot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign lichunzhu for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• br/pkg/lightning/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[tiprow]

Hi @tangenta. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[codecov]

Codecov Report

Merging #48044 (888a68b) into master (d1fc638) will increase coverage by 1.1462%.
Report is 7 commits behind head on master.
The diff coverage is 100.0000%.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #48044        +/-   ##
================================================
+ Coverage   71.6271%   72.7733%   +1.1462%     
================================================
  Files          1401       1421        +20     
  Lines        405768     411641      +5873     
================================================
+ Hits         290640     299565      +8925     
+ Misses        95337      93223      -2114     
+ Partials      19791      18853       -938     

Flag	Coverage Δ
integration	42.0688% <0.0000%> (?)
unit	71.6173% <100.0000%> (-0.0098%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	54.0503% <ø> (+0.0589%)	⬆️
parser	∅ <ø> (∅)
br	48.6575% <100.0000%> (-4.3687%)	⬇️

[ti-chi-bot]

@tangenta: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
idc-jenkins-ci-tidb/check_dev_2	888a68b	link	true	/test check-dev2

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[lance6716]

slice has more members than a []byte, why it's helpful to escaping 😵

[lance6716]

maybe return r.curBuf[originOffset:readLen]? I will test it in my local environment

[lance6716]

I'm not sure about the improvements of this PR. Seems there must be something that escapes to heap

go build  -gcflags="-m=2"
...
./byte_reader.go:204:10: &slice{...} escapes to heap:
./byte_reader.go:204:10:   flow: ret = &{storage for &slice{...}}:
./byte_reader.go:204:10:     from &slice{...} (spill) at ./byte_reader.go:204:10
./byte_reader.go:204:10:     from ret := &slice{...} (assign) at ./byte_reader.go:204:7
./byte_reader.go:204:10:   flow: {heap} = ret:
./byte_reader.go:204:10:     from append(r.retPointers, ret) (call parameter) at ./byte_reader.go:209:25