*: Optimize struct memory usage by adjust field order

[Deardrops]

What problem does this PR solve?

In golang, the compiler performs memory alignment on the fields in the structure.
For example, There are two struct with same field but different order.

type T1 struct {
	a int8
	b int64
	c int16
}
type T2 struct {
	a int8
	c int16
	b int64
}

In x86_64 platform, each fields will align to 8 byte, as shown in the figures:

Total: 8 byte + 8 byte + 8 byte = 24 byte

Total: 8 byte + 8 byte = 16 byte

Different field orders determine the memory size of the structure.
Reasonable field orders can reduce memory overhead.

What is changed and how it works?

Using golangci-lint static check all code in TiDB

golangci-lint run --disable-all -E maligned

And then we deal with the struct needed optimization one by one, there are three cases:

• If the structure is so large that adjusting the field order will greatly affect the readability of the code, no changes will be made.
• If it is an anonymous structure that appears in the test file, and it will not affect the memory usage when the program is running, no changes will be made.
• Otherwise, on the premise of modifying the code as little as possible, adjust the field order, make struct take least memory.

Check List

Tests

• None

Code changes

• None

Side effects

• Increased code complexity

Related changes

• None

[codecov]

Codecov Report

Merging #11629 into master will decrease coverage by 0.3137%.
The diff coverage is n/a.

@@               Coverage Diff                @@
##             master     #11629        +/-   ##
================================================
- Coverage   81.5272%   81.2134%   -0.3138%     
================================================
  Files           432        426         -6     
  Lines         93635      91885      -1750     
================================================
- Hits          76338      74623      -1715     
- Misses        11868      11896        +28     
+ Partials       5429       5366        -63

[zz-jason]

@Deardrops please add some proper labels for this PR.

[zz-jason]

any performance result?

[zz-jason]

It's better to add golangci-lint run --disable-all -E maligned to the check-static section in Makefile

[Deardrops]

@zz-jason

@Deardrops please add some proper labels for this PR.

Done

any performance result?

I choose concatFunction in expression/aggregation/concat.go for the benchmark. source code is here.

---

current (48 byte total, reported by golangci-lint):

type concatFunction struct {
	aggFunction
	separator string
	sepInited bool
	maxLen    uint64
	truncated bool
}

new (40 byte total):

type concatFunctionNew struct {
	aggFunction
	separator string
	maxLen    uint64
	sepInited bool
	truncated bool
}

---

Benchmark result:

# go test -bench=. -benchmem -benchtime=10s
goos: linux
goarch: amd64
BenchmarkConcatFunction-4       50000000               434 ns/op             280 B/op          0 allocs/op
BenchmarkConcatFunctionNew-4    50000000               331 ns/op             224 B/op          0 allocs/op
PASS
ok      _/root/structure-memory-alignment       39.092s

It's better to add golangci-lint run --disable-all -E maligned to the check-static section in Makefile

In my view, It's unnecessary. Althought all structed will be scanned, not all struct will be changed.(some of them are too complicated to maintain). There is no to add the static check tool into Makefile, because it's better for developer to make change by their own eyes, not asked by the tools.

[Deardrops]

@foreyes @winkyao @lonng PTAL

[ngaut]

/run-all-tests

[AilinKid]

The follow-up code should do the check themself, not that automatically.
A quick start is as follow if they use GoLand IDE:
GoLand -> Preferences -> Tools -> File Watchers -> Choose golandci-lint Option
when adding code later, it will be triggered automatically.

[foreyes]

LGTM

[zz-jason]

@Deardrops It's better to add golangci-lint run --disable-all -E maligned to the check-static section in Makefile.

Not every programmer knows or realizes the memory alignment of struct. We should use tools to notify them about this. In order to not file such a huge PR in the future.

[winkyao]

LGTM

[Deardrops]

@zz-jason

@Deardrops It's better to add golangci-lint run --disable-all -E maligned to the check-static section in Makefile.

Not every programmer knows or realizes the memory alignment of struct. We should use tools to notify them about this. In order to not file such a huge PR in the future.

Static check could be the guide, but not the rule. It's not a good idea that add this static check in Makefile.
This PR has optimized 80% struct that need to be optimized, but the rest 20% is maintain.
Everytime programmer run the golangci-lint, he / she will get the warning of the rest 20%.
Because of not all struct that need optimized will be optimized, we should do the static check Manually, leave room for selection.

[AilinKid]

/LGTM

[tangenta]

LGTM

[zz-jason]

@Deardrops If we do not require the programmer to check the memory alignment, the percentage of unaligned struct will be increased to more than 20% in the future.

It's hard to make every programmer to check the memory alignment manually every time they file a PR.

[Deardrops]

@zz-jason

@Deardrops If we do not require the programmer to check the memory alignment, the percentage of unaligned struct will be increased to more than 20% in the future.

It's hard to make every programmer to check the memory alignment manually every time they file a PR.

I don't think we should introduce static checking tools into MakeFile.
Some structures are very complex (more than 100 fields), which will affect the readability of the code if optimized. Therefore, we do not need to optimize all structures. Then, static tools scan all structures, including these complex structures that should have been ignored, giving very lengthy prompts. Requiring every programmer who pull request to confirm the tips of static inspection tools, confirm the structures that need optimization or not one by one will increase the workload of merging PR.
Every time PR is filed, it is not necessary to check the memory alignment of the structure, because the results obtained by static tools are too complicated, and manual static checks should be done in PR reviewing and regular checks (such as before the release of new versions).