[bench] Increase file descriptor limit helps #20

- when the limit is changed from 1024 -> 4096, 3000 is the best worker number, after that, the performance decrease a lot - add by putting server in the container, it no longer have the accept error
xephonhq · Mar 20, 2017 · ddfdf85 · ddfdf85
1 parent 38e09d0
commit ddfdf85
Show file tree

Hide file tree

Showing 2 changed files with 44 additions and 0 deletions.
diff --git a/doc/bench/xephonk-cassandra.md b/doc/bench/xephonk-cassandra.md
@@ -119,6 +119,46 @@ Timeout: 30
 TargetDB: xephonk
 ````
 
+docker (isn't much better)
+
+````
+INFO[0008] basic report finished via context pkg=k.b.reporter 
+INFO[0008] total request 35313 pkg=k.b.reporter 
+INFO[0008] fastest 23965 pkg=k.b.reporter 
+INFO[0008] slowest 3348216936 pkg=k.b.reporter 
+INFO[0008] total request size 69390045 pkg=k.b.reporter 
+INFO[0008] toatl response size 240994 pkg=k.b.reporter 
+INFO[0008] 200: 7774 pkg=k.b.reporter 
+INFO[0008] 0: 27539 pkg=k.b.reporter 
+INFO[0008] loader finished pkg=k.b.loader 
+
+Duration: 5
+Worker number: 5000
+Batch size: 100
+Timeout: 30
+TargetDB: xephonk
+````
+
+It seems 3000 is the best after `ulimit -Sn 4096`
+
+````
+INFO[0007] basic report finished via context pkg=k.b.reporter 
+INFO[0007] total request 16016 pkg=k.b.reporter 
+INFO[0007] fastest 7729246 pkg=k.b.reporter 
+INFO[0007] slowest 4311813810 pkg=k.b.reporter 
+INFO[0007] total request size 31471440 pkg=k.b.reporter 
+INFO[0007] toatl response size 492683 pkg=k.b.reporter 
+INFO[0007] 200: 15893 pkg=k.b.reporter 
+INFO[0007] 0: 123 pkg=k.b.reporter 
+INFO[0007] loader finished pkg=k.b.loader 
+
+Duration: 5
+Worker number: 3000
+Batch size: 100
+Timeout: 30
+TargetDB: xephonk
+````
+
 ````bash
 xkb -c 100 -d 60
 ````
@@ -142,3 +182,5 @@ Timeout: 30
 TargetDB: xephonk
 ````
 
+
+
diff --git a/pkg/bench/loader/http_loader.go b/pkg/bench/loader/http_loader.go
@@ -29,7 +29,9 @@ func NewHTTPLoader(config Config, report reporter.Reporter) *HTTPLoader {
 func (loader *HTTPLoader) Run() {
 	var baseReq *http.Request
 	// NOTE: https://github.com/at15/mini-impl/issues/1
+	// TODO: https://github.com/xephonhq/xephon-k/issues/20 warn about file descriptor
 	tr := &http.Transport{
+		MaxIdleConns: loader.config.WorkerNum,
 		MaxIdleConnsPerHost: loader.config.WorkerNum,
 	}
 	log.Infof("target db %s", bench.DBString(loader.config.TargetDB))