Skip to content

Commit

Permalink
Feat visual plan (#1347)
Browse files Browse the repository at this point in the history 
* Dashboard: support parse visual plan (#1227)

* init

* fix

* add test

* fix

* use binary_plan (#1284)

* Visual Plan  (#1285)

* feat: add visual plan for slowquery and statement

* make visual plan as a independent module

* update without minimap

* update minimap

* update

* update

* update minimap on brush and zoom

* fix: restriction of minimap brush

* refine minimap

* update variable declaration

* update minimap and mainchart

* move minimap translate to minimap

* combine some interface

* add node detail drawer

* move brush svg into minimap

* chore: rename

* chore: rename

* fix: memorize node margin

* chore: update node detail drawer

* add thumbnail for tree digram

* chore: update node detail

* fix ci

Co-authored-by: Suhaha <jklopsdfw@gmail.com>

* fix: empty binary_plan

* Analyze visual plan (#1320)

* use protojson

* fix

* format json

* set concurrency

* init Duration

* add diagnosticOperator

* fix

* fix

* fix

* Update provider.go

* fix

* fix

* update ui

* fix

* fix go-lint

* fix git diff --exit-code

* fix git diff --exit-code

* fix

* add TestUseComparisonOperator

* to circumvent golint v1.45.0

* fix stirngs.Cut

* fix stirngs.Cut

* fix: vp pseudo  (#1321)

* fix

* fix

* Support multi tree (#1324)

* chore: support multi trees

* fix: udpate multi tree

* fix: add single tree

* update multi trees

* update multi trees

* update multi trees

* update multi trees

* refactor: default node & link

* refine: better fps

* api: test

* api: test

* update node details

* refine: minimap

* style: update

* refine: use memo to cache the component

* chore: wrong usage

* update thumbnail

* update thumbnail

* resolve comments

* update variable get

* chore: update thumbnail bound

* update structure

* fix ci

* refine ts

Co-authored-by: Suhaha <jklopsdfw@gmail.com>

* add a new interface and fix some field parsing issues (#1325)

* fix some field parsing issues

* fix

* fix

* fix

* fix go-lint

* fix node detail label

* add vp to statement detail page (#1326)

* fix: compatibility on safari (#1327)

* style: refine ui (#1332)

* style: refine ui

* update adjust position

* update modal height

* ci: fix lint

* add unit tests (#1328)

* add unit tests

* fix

* upgrade go.mod

* refine visual plan (#1339)

* refine visual plan

* update translations

* ci: fix lint

* update i18n

* ci: fix lint

* use role tag (#1341)

* use role tag

* fix

* fix

* adjust execution plan UI position (#1342)

* adjust execution plan UI position

* chore: fix Execution plan tab

* ci: fix lint

* delete debug msg (#1345)

* Add telemetry for vp (#1340)

* refine visual plan

* update translations

* ci: fix lint

* add telemetry for visual plan

* update telemetry

* ci: fix lint

* ci: fix lint

* delete: yarn.lock

* fix unit test (#1351)

Co-authored-by: Qiμ$hiЯuí <39378935+srstack@users.noreply.github.com>
Co-authored-by: Suhaha <jklopsdfw@gmail.com>
  • Loading branch information
@YiniXu9506 @srstack @shhdgit
3 people authored Jul 20, 2022
1 parent 294c810 commit 865b77e
Show file tree
Hide file tree
Showing 62 changed files with 5,308 additions and 2,101 deletions.
2 changes: 1 addition & 1 deletion Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ BUILD_TAGS ?=

LDFLAGS ?=

FEATURE_VERSION ?= 6.0.0
FEATURE_VERSION ?= 6.2.0

WITHOUT_NGM ?= false

Expand Down
5 changes: 5 additions & 0 deletions go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ require (
github.com/Xeoncross/go-aesctr-with-hmac v0.0.0-20200623134604-12b17a7ff502
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751
github.com/antonmedv/expr v1.9.0
github.com/bitly/go-simplejson v0.5.0
github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869 // indirect
github.com/breeswish/gin-jwt/v2 v2.6.4-jwt-patch
github.com/cenkalti/backoff/v4 v4.0.2
github.com/fatih/structtag v1.2.0
Expand All @@ -19,6 +21,7 @@ require (
github.com/go-sql-driver/mysql v1.6.0
github.com/goccy/go-graphviz v0.0.9
github.com/golang-jwt/jwt v3.2.1+incompatible
github.com/golang/snappy v0.0.4
github.com/google/pprof v0.0.0-20211122183932-1daafda22083
github.com/google/uuid v1.1.3
github.com/gtank/cryptopasta v0.0.0-20170601214702-1f550f6f2f69
Expand All @@ -33,6 +36,7 @@ require (
github.com/pingcap/errors v0.11.5-0.20200917111840-a15ef68f753d
github.com/pingcap/kvproto v0.0.0-20200411081810-b85805c9476c
github.com/pingcap/log v0.0.0-20210906054005-afc726e70354
github.com/pingcap/tipb v0.0.0-20220718022156-3e2483c20a9e
github.com/rs/cors v1.7.0
github.com/shhdgit/testfixtures/v3 v3.6.2-0.20211219171712-c4f264d673d3
github.com/shurcooL/httpgzip v0.0.0-20190720172056-320755c1c1b0
Expand All @@ -51,6 +55,7 @@ require (
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c
google.golang.org/grpc v1.38.0
google.golang.org/protobuf v1.26.0
gorm.io/driver/mysql v1.0.6
gorm.io/driver/sqlite v1.1.4
gorm.io/gorm v1.21.9
Expand Down
12 changes: 12 additions & 0 deletions go.sum
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZx
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/bitly/go-simplejson v0.5.0 h1:6IH+V8/tVMab511d5bn4M7EwGXZf9Hj6i2xSwkNEM+Y=
github.com/bitly/go-simplejson v0.5.0/go.mod h1:cXHtHw4XUPsvGaxgjIAn8PhEWG9NfngEKAMDJEczWVA=
github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869 h1:DDGfHa7BWjL4YnC6+E63dPcxHo2sUxDIu8g3QgEJdRY=
github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869/go.mod h1:Ekp36dRnpXw/yCqJaO+ZrUyxD+3VXMFFr56k5XYrpB4=
github.com/breeswish/gin-jwt/v2 v2.6.4-jwt-patch h1:KLE/YeX+9FNaGVW5MtImRVPhjDpfpgJhvkuYWBmOYbo=
github.com/breeswish/gin-jwt/v2 v2.6.4-jwt-patch/go.mod h1:KjBLriHXe7L6fGceqWzTod8HUB/TP1WWDtfuSYtYXaI=
github.com/cenkalti/backoff/v4 v4.0.2 h1:JIufpQLbh4DkbQoii76ItQIUFzevQSqOLZca4eamEDs=
Expand Down Expand Up @@ -136,6 +140,7 @@ github.com/goccy/go-graphviz v0.0.9/go.mod h1:wXVsXxmyMQU6TN3zGRttjNn3h+iCAS7xQF
github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
github.com/gofrs/uuid v3.2.0+incompatible h1:y12jRkkFxsd7GpqdSZ+/KCs/fJbqpEXSGd4+jfEaewE=
github.com/gofrs/uuid v3.2.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM=
github.com/gogo/protobuf v0.0.0-20171007142547-342cbe0a0415/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.3.1/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
Expand All @@ -152,6 +157,7 @@ github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5y
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
Expand All @@ -163,6 +169,8 @@ github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw=
github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/snappy v0.0.4 h1:yAGX7huGHXlcLOEtBnF4w7FQwA26wojNCwOYAEhLjQM=
github.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
Expand Down Expand Up @@ -321,6 +329,8 @@ github.com/pingcap/log v0.0.0-20191012051959-b742a5d432e9/go.mod h1:4rbK1p9ILyIf
github.com/pingcap/log v0.0.0-20200511115504-543df19646ad/go.mod h1:4rbK1p9ILyIfb6hU7OG2CiWSqMXnp3JMbiaVJ6mvoY8=
github.com/pingcap/log v0.0.0-20210906054005-afc726e70354 h1:SvWCbCPh1YeHd9yQLksvJYAgft6wLTY1aNG81tpyscQ=
github.com/pingcap/log v0.0.0-20210906054005-afc726e70354/go.mod h1:DWQW5jICDR7UJh4HtxXSM20Churx4CQL0fwL/SoOSA4=
github.com/pingcap/tipb v0.0.0-20220718022156-3e2483c20a9e h1:FBaTXU8C3xgt/drM58VHxojHo/QoG1oPsgWTGvaSpO4=
github.com/pingcap/tipb v0.0.0-20220718022156-3e2483c20a9e/go.mod h1:A7mrd7WHBl1o63LE2bIBGEJMTNWXqhgmYiOvMLxozfs=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
Expand Down Expand Up @@ -500,6 +510,7 @@ golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLL
golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4 h1:4nGaVu0QrbjT/AK2PRLuQfQuh6DJve+pELhqTdAj3x0=
golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
Expand Down Expand Up @@ -572,6 +583,7 @@ golang.org/x/tools v0.0.0-20191114200427-caa0b0f7d508/go.mod h1:b+2E5dAYhXwXZwtn
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20201125231158-b5590deeca9b/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210112230658-8b4aab62c064/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.1.2 h1:kRBLX7v7Af8W7Gdbbc908OJcdgtK8bOz9Uaj8/F1ACA=
Expand Down
2 changes: 2 additions & 0 deletions pkg/apiserver/apiserver.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ import (
"github.com/pingcap/tidb-dashboard/pkg/apiserver/user/sqlauth"
"github.com/pingcap/tidb-dashboard/pkg/apiserver/user/sso"
"github.com/pingcap/tidb-dashboard/pkg/apiserver/user/sso/ssoauth"
"github.com/pingcap/tidb-dashboard/pkg/apiserver/visualplan"
"github.com/pingcap/tidb-dashboard/pkg/tiflash"
"github.com/pingcap/tidb-dashboard/pkg/utils/version"
"github.com/pingcap/tidb-dashboard/util/client/httpclient"
Expand Down Expand Up @@ -135,6 +136,7 @@ var Modules = fx.Options(
slowquery.Module,
debugapi.Module,
topsql.Module,
visualplan.Module,
)

func (s *Service) Start(ctx context.Context) error {
Expand Down
3 changes: 2 additions & 1 deletion pkg/apiserver/logsearch/task.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import (
"math"
"os"
"path"
"path/filepath"
"strconv"
"sync"
"time"
Expand Down Expand Up @@ -207,7 +208,7 @@ func (t *Task) searchLog(client diagnosticspb.DiagnosticsClient, targetType diag
fileName = fileName + "-slow"
}
savedPath := path.Join(*t.taskGroup.model.LogStoreDir, fileName+".zip")
f, err := os.Create(savedPath)
f, err := os.Create(filepath.Clean(savedPath))
if err != nil {
t.setError(err)
return
Expand Down
5 changes: 3 additions & 2 deletions pkg/apiserver/slowquery/model.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,9 @@ type Model struct {
TxnStartTS string `gorm:"column:Txn_start_ts" json:"txn_start_ts"`

// Detail
PrevStmt string `gorm:"column:Prev_stmt" json:"prev_stmt"`
Plan string `gorm:"column:Plan" json:"plan"`
PrevStmt string `gorm:"column:Prev_stmt" json:"prev_stmt"`
Plan string `gorm:"column:Plan" json:"plan"`
BinaryPlan string `gorm:"column:Binary_plan" json:"binary_plan"`

// Basic
IsInternal int `gorm:"column:Is_internal" json:"is_internal"`
Expand Down
8 changes: 8 additions & 0 deletions pkg/apiserver/slowquery/service.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,14 @@ func (s *Service) getDetails(c *gin.Context) {
rest.Error(c, err)
return
}

// generate binary plan
result.BinaryPlan, err = utils.GenerateBinaryPlanJSON(result.BinaryPlan)
if err != nil {
rest.Error(c, err)
return
}

c.JSON(http.StatusOK, *result)
}

Expand Down
1 change: 1 addition & 0 deletions pkg/apiserver/statement/models.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ type Model struct {
AggPlanCount int `json:"plan_count" agg:"COUNT(DISTINCT plan_digest)" related:"plan_digest"`
AggPlan string `json:"plan" agg:"ANY_VALUE(plan)"`
AggPlanDigest string `json:"plan_digest" agg:"ANY_VALUE(plan_digest)"`
AggBinaryPlan string `json:"binary_plan" agg:"ANY_VALUE(binary_plan)"`
// RocksDB
AggMaxRocksdbDeleteSkippedCount uint `json:"max_rocksdb_delete_skipped_count" agg:"MAX(max_rocksdb_delete_skipped_count)"`
AggAvgRocksdbDeleteSkippedCount uint `json:"avg_rocksdb_delete_skipped_count" agg:"CAST(SUM(exec_count * avg_rocksdb_delete_skipped_count) / SUM(exec_count) as SIGNED)"`
Expand Down
8 changes: 8 additions & 0 deletions pkg/apiserver/statement/service.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -220,6 +220,14 @@ func (s *Service) planDetailHandler(c *gin.Context) {
rest.Error(c, err)
return
}

// get binary plan
result.AggBinaryPlan, err = utils.GenerateBinaryPlanJSON(result.AggBinaryPlan)
if err != nil {
rest.Error(c, err)
return
}

c.JSON(http.StatusOK, result)
}

Expand Down
Loading

0 comments on commit 865b77e

Please sign in to comment.