-
Notifications
You must be signed in to change notification settings - Fork 576
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
GongHongjun
committed
May 22, 2024
1 parent
83d3303
commit afe2f21
Showing
5 changed files
with
183 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
FROM golang:1.17.10 as build | ||
|
||
# 容器环境变量添加 | ||
ENV GO111MODULE=on | ||
ENV GOPROXY=https://goproxy.cn,direct | ||
|
||
# 设置当前工作区 | ||
WORKDIR /go/release | ||
|
||
# 把全部文件添加到/go/release目录 | ||
ADD . . | ||
|
||
# 编译: 把main.go编译为可执行的二进制文件, 并命名为app | ||
RUN GOOS=linux CGO_ENABLED=0 GOARCH=amd64 go build -ldflags="-s -w" -installsuffix cgo -o godis . | ||
|
||
# 运行: 使用scratch作为基础镜像 | ||
FROM golang:1.17.10 as prod | ||
|
||
# 在build阶段, 复制时区配置到镜像的/etc/localtime | ||
COPY --from=build /usr/share/zoneinfo/Asia/Shanghai /etc/localtime | ||
|
||
# 在build阶段, 复制./app目录下的可执行二进制文件到当前目录 | ||
COPY --from=build /go/release/godis / | ||
|
||
# 启动服务 | ||
CMD ["/app"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,78 @@ | ||
1,使用AF_PACKET替代net.con,提高吞吐率 /tcp/server.go 44行 | ||
预计需要实现Listener接口 | ||
```go | ||
type Listener interface { | ||
// Accept waits for and returns the next connection to the listener. | ||
Accept() (Conn, error) | ||
|
||
// Close closes the listener. | ||
// Any blocked Accept operations will be unblocked and return errors. | ||
Close() error | ||
|
||
// Addr returns the listener's network address. | ||
Addr() Addr | ||
} | ||
``` | ||
|
||
和Conn接口 | ||
```go | ||
type Conn interface { | ||
// Read reads data from the connection. | ||
// Read can be made to time out and return an error after a fixed | ||
// time limit; see SetDeadline and SetReadDeadline. | ||
Read(b []byte) (n int, err error) | ||
|
||
// Write writes data to the connection. | ||
// Write can be made to time out and return an error after a fixed | ||
// time limit; see SetDeadline and SetWriteDeadline. | ||
Write(b []byte) (n int, err error) | ||
|
||
// Close closes the connection. | ||
// Any blocked Read or Write operations will be unblocked and return errors. | ||
Close() error | ||
|
||
// LocalAddr returns the local network address, if known. | ||
LocalAddr() Addr | ||
|
||
// RemoteAddr returns the remote network address, if known. | ||
RemoteAddr() Addr | ||
|
||
// SetDeadline sets the read and write deadlines associated | ||
// with the connection. It is equivalent to calling both | ||
// SetReadDeadline and SetWriteDeadline. | ||
// | ||
// A deadline is an absolute time after which I/O operations | ||
// fail instead of blocking. The deadline applies to all future | ||
// and pending I/O, not just the immediately following call to | ||
// Read or Write. After a deadline has been exceeded, the | ||
// connection can be refreshed by setting a deadline in the future. | ||
// | ||
// If the deadline is exceeded a call to Read or Write or to other | ||
// I/O methods will return an error that wraps os.ErrDeadlineExceeded. | ||
// This can be tested using errors.Is(err, os.ErrDeadlineExceeded). | ||
// The error's Timeout method will return true, but note that there | ||
// are other possible errors for which the Timeout method will | ||
// return true even if the deadline has not been exceeded. | ||
// | ||
// An idle timeout can be implemented by repeatedly extending | ||
// the deadline after successful Read or Write calls. | ||
// | ||
// A zero value for t means I/O operations will not time out. | ||
SetDeadline(t time.Time) error | ||
|
||
// SetReadDeadline sets the deadline for future Read calls | ||
// and any currently-blocked Read call. | ||
// A zero value for t means Read will not time out. | ||
SetReadDeadline(t time.Time) error | ||
|
||
// SetWriteDeadline sets the deadline for future Write calls | ||
// and any currently-blocked Write call. | ||
// Even if write times out, it may return n > 0, indicating that | ||
// some of the data was successfully written. | ||
// A zero value for t means Write will not time out. | ||
SetWriteDeadline(t time.Time) error | ||
} | ||
|
||
``` | ||
|
||
参考:https://mp.weixin.qq.com/s/6DRg_6P_Gz0ZWw_JPCKuLA |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
|
||
### 提交说明 | ||
优化MultiBulkReply.ToBytes中计算所需内存的算法,提高性能;在utils中新增GetItoaLen方法,以替代原len(strconv.Itoa(i int))计算数字被转换为字符串以后字符串的长度; | ||
|
||
### 基准测试 [utils_test.go](lib%2Futils%2Futils_test.go) | ||
#### 测试 | ||
OS:Centos 7 | ||
VM:VM Workstation 17.5.0 | ||
VM配置:2C4G | ||
|
||
#### 测试结果 | ||
Now:当前版本 | ||
New Version:本次提交的版本 | ||
|
||
* 非LRANGE指令 | ||
测试方法:运行以下指令3次,取平均值 | ||
```text | ||
redis-benchmark -q -d 4096 --csv -t SET,GET,INCR,LPUSH,RPUSH,LPOP,RPOP,SADD,HSET,SPOP | ||
``` | ||
|
||
| Redis 6.2.6 | Request/S | Now | Request/S | New Version | Request/S | New Vs Now | New Vs Redis6 | | ||
|-------------|-------------|-------|-------------|-------------|-------------|------------|---------------| | ||
| SET | 160605.7133 | SET | 68052.81667 | SET | 66993.39333 | -2% | -58% | | ||
| GET | 152806.5767 | GET | 62314.54667 | GET | 62106.69667 | 0% | -59% | | ||
| INCR | 175523.5433 | INCR | 96774.33667 | INCR | 103187.75 | 7% | -41% | | ||
| LPUSH | 125236.7433 | LPUSH | 70687.95333 | LPUSH | 71056.98 | 1% | -43% | | ||
| RPUSH | 138791 | RPUSH | 68633.12667 | RPUSH | 67574.54 | -2% | -51% | | ||
| LPOP | 153191.02 | LPOP | 60513.26333 | LPOP | 61848.86333 | 2% | -60% | | ||
| RPOP | 134505.1 | RPOP | 68805.09333 | RPOP | 70723.33333 | 3% | -47% | | ||
| SADD | 170620.0933 | SADD | 103671.8467 | SADD | 105098.9467 | 1% | -38% | | ||
| HSET | 157717.3833 | HSET | 70449.39667 | HSET | 70000.03333 | -1% | -56% | | ||
| SPOP | 167282.3467 | SPOP | 104680.9 | SPOP | 113612.5367 | 9% | -32% | | ||
|
||
总结:本次提交的性能优化,对非LRANGE的指令吞吐率有部分提升; | ||
|
||
* LRANGE指令 | ||
测试方法:运行以下指令3次,取平均值 | ||
```text | ||
redis-benchmark -q -d 4096 --csv -t lrange -n 10000 | ||
``` | ||
|
||
| Redis 6.2.6 | Request/S | Now | Request/S | New Version | Request/S | New Vs Now | New Vs Redis6 | | ||
|------------------------------------|-------------|------------------------------------|-------------|------------------------------------|-------------|------------|---------------| | ||
| LPUSH (needed to benchmark LRANGE) | 138415.7267 | LPUSH (needed to benchmark LRANGE) | 31227.84667 | LPUSH (needed to benchmark LRANGE) | 66234.6 | 112% | -52% | | ||
| LRANGE_100 (first 100 elements) | 2394.883333 | LRANGE_100 (first 100 elements) | 1392.1 | LRANGE_100 (first 100 elements) | 2100.576667 | 51% | -12% | | ||
| LRANGE_300 (first 300 elements) | 737.4066667 | LRANGE_300 (first 300 elements) | 562.9166667 | LRANGE_300 (first 300 elements) | 762.1166667 | 35% | 3% | | ||
| LRANGE_500 (first 450 elements) | 480.99 | LRANGE_500 (first 450 elements) | 349 | LRANGE_500 (first 450 elements) | 522.4533333 | 50% | 9% | | ||
| LRANGE_600 (first 600 elements) | 348.7066667 | LRANGE_600 (first 600 elements) | 281.02 | LRANGE_600 (first 600 elements) | 389.0833333 | 38% | 12% | | ||
|
||
总结:本次提交的性能优化,对LRANGE的指令吞吐率提升明显,部分超过Redis6; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
### 提交说明 | ||
优化MultiBulkReply.ToBytes中计算所需内存的算法,提高性能;在utils中新增GetItoaLen方法,以替代原len(strconv.Itoa(i int))计算数字被转换为字符串以后字符串的长度; | ||
|
||
### 功能和benchmark测试 [utils_test.go](lib%2Futils%2Futils_test.go) | ||
|
||
#### 测试结果 | ||
```text | ||
goos: windows | ||
goarch: amd64 | ||
pkg: github.com/hdt3213/godis/lib/utils | ||
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz | ||
Benchmark_Itoa_20bit | ||
Benchmark_Itoa_20bit-12 21663034 57.17 ns/op | ||
Benchmark_GetIotaLen_20bit | ||
Benchmark_GetIotaLen_20bit-12 39857972 28.99 ns/op | ||
Benchmark_Itoa_10bit | ||
Benchmark_Itoa_10bit-12 29314266 39.26 ns/op | ||
Benchmark_GetIotaLen_10bit | ||
Benchmark_GetIotaLen_10bit-12 130224921 9.493 ns/op | ||
PASS | ||
Process finished with the exit code 0 | ||
``` | ||
|
||
总结:采用新的方法可以一定程度上提升效率;但是在总体吞吐量测试上,表现不明显;经过分析,吞吐率受内存分配影响最大,这部分优化对总体影响较小. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
#正式版打包 | ||
set GOOS=linux | ||
set GOARCH=amd64 | ||
go.exe build -ldflags "-s -w" -o target/godis-linux ./ |