Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release v2021.12.30.1 [skip pd_pr] #1113

Merged
merged 36 commits into from
Dec 30, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
a28d3fc
Revert "Release v2021.12.06.1 (#1084)"
baurine Dec 30, 2021
ee4d16d
Compitable with different TiDB versions for conprof and non-root-logi…
baurine Nov 10, 2021
b8399a9
build(deps): bump ws from 5.2.2 to 5.2.3 in /ui (#1055)
dependabot[bot] Nov 10, 2021
975d426
CICD: Update the release pipeline for recent PD format policies (#1054)
breezewish Nov 10, 2021
6d63f33
fix i18n wording (#1056)
baurine Nov 10, 2021
f47da2e
Refactor: Change util module to util package (#1052)
breezewish Nov 11, 2021
d9c5a56
Refactor: Fix godot incorrectly add dot suffix to annotations (#1059)
breezewish Nov 11, 2021
b6560ae
lint: Add goheader for copyright lints (#1062)
breezewish Nov 12, 2021
877899a
Refactor: Migrate to use the `rest` package in util/ (#1060)
breezewish Nov 22, 2021
50daa9e
fix(*): globally delete/update data by GORM (#1065)
shhdgit Nov 22, 2021
819a45a
ui: bump dependencies (#1066)
breezewish Nov 23, 2021
bd90f01
refactor: Switch to use ziputil, netutil, reflectutil and fileswap (#…
breezewish Nov 24, 2021
926c58c
Fix request header being pinned after pd profiling (#1069)
shhdgit Nov 25, 2021
9af1a84
Integrate speedscope (#1064)
YiniXu9506 Nov 29, 2021
2d2ab19
fix potential panic when GetPDInstances (#1075)
crazycs520 Nov 30, 2021
78fafc2
Refactor: a new httpclient (#1073)
breezewish Dec 1, 2021
3db85c2
Refactor: Switch to use util/distro in all places (#1078)
breezewish Dec 1, 2021
3728b24
chore: support import relative file URL (#1082)
YiniXu9506 Dec 5, 2021
1a7d6e1
Refactor: Move tools into a standalone module (#1079)
breezewish Dec 6, 2021
f98b8f4
Fix script to embed the ui (#1088)
breezewish Dec 8, 2021
b858495
Refactor feature flag to support more modules (#1057)
shhdgit Dec 8, 2021
1b47110
Drop sysutil dependency (#1093)
breezewish Dec 10, 2021
f9dd2d7
chore: add graph generation (#1085)
YiniXu9506 Dec 14, 2021
37341db
Refactor: Add TopologyProvider (#1098)
breezewish Dec 16, 2021
0148da5
esbuild: i18n + dep (#1101)
baurine Dec 16, 2021
564c6a5
script: Add a script to generate version matrix (#1104)
breezewish Dec 21, 2021
e9caf73
distro: support dynamic config (#1094)
baurine Dec 22, 2021
6bdcdf2
chore: support multiple profiling types (#1095)
YiniXu9506 Dec 22, 2021
eefe543
fix(distro): check distro_strings.json fmt by prettier (#1106)
shhdgit Dec 22, 2021
daeb3a2
script: fix generate assets (#1107)
baurine Dec 27, 2021
14f6a34
Add integration test (#1083)
shhdgit Dec 27, 2021
c4f3719
debug_api: Switch to use the new util (#1103)
breezewish Dec 27, 2021
2214f60
refactor(ui): auto refresh button (#1105)
shhdgit Dec 28, 2021
cc3bc17
ui: refine conprof (#1102)
baurine Dec 29, 2021
1a7c822
update release-version
baurine Dec 30, 2021
9d6b17a
sync with master
baurine Dec 30, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
chore: add graph generation (#1085)
  • Loading branch information
YiniXu9506 authored and baurine committed Dec 30, 2021
commit f9dd2d7466290409cb53a5999a0dbc8014446e90
2 changes: 2 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@ require (
github.com/gin-gonic/gin v1.7.4
github.com/go-resty/resty/v2 v2.6.0
github.com/go-sql-driver/mysql v1.6.0
github.com/goccy/go-graphviz v0.0.9
github.com/golang-jwt/jwt v3.2.1+incompatible
github.com/google/pprof v0.0.0-20211122183932-1daafda22083
github.com/google/uuid v1.0.0
github.com/gtank/cryptopasta v0.0.0-20170601214702-1f550f6f2f69
github.com/henrylee2cn/ameda v1.4.10
Expand Down
24 changes: 23 additions & 1 deletion go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,9 @@ github.com/breeswish/gin-jwt/v2 v2.6.4-jwt-patch/go.mod h1:KjBLriHXe7L6fGceqWzTo
github.com/cenkalti/backoff/v4 v4.0.2 h1:JIufpQLbh4DkbQoii76ItQIUFzevQSqOLZca4eamEDs=
github.com/cenkalti/backoff/v4 v4.0.2/go.mod h1:eEew/i+1Q6OrCDZh3WiXYv3+nJwBASZ8Bog/87DQnVg=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa h1:OaNxuTZr7kxeODyLWsRMC+OD03aFUH+mW6r2d+MWa5Y=
github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa/go.mod h1:zn76sxSg3SzpJ0PPJaLDCu+Bu0Lg3sKTORVIj19EIF8=
Expand All @@ -49,6 +52,8 @@ github.com/coreos/go-systemd v0.0.0-20180511133405-39ca1b05acc7 h1:u9SHYsPQNyt5t
github.com/coreos/go-systemd v0.0.0-20180511133405-39ca1b05acc7/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/coreos/pkg v0.0.0-20160727233714-3ac0863d7acf h1:CAKfRE2YtTUIjjh1bkBtyYFaUT/WmOqsJjgtihT0vMI=
github.com/coreos/pkg v0.0.0-20160727233714-3ac0863d7acf/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
github.com/corona10/goimagehash v1.0.2 h1:pUfB0LnsJASMPGEZLj7tGY251vF+qLGqOgEP4rUs6kA=
github.com/corona10/goimagehash v1.0.2/go.mod h1:/l9umBhvcHQXVtQO1V6Gp1yD20STawkhRnnX0D1bvVI=
github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY=
github.com/davecgh/go-spew v0.0.0-20161028175848-04cdfd42973b/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
Expand All @@ -64,6 +69,8 @@ github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/fatih/structtag v1.2.0 h1:/OdNE99OxoI/PqaW/SuSK9uxxT3f/tcSZgon/ssNSx4=
github.com/fatih/structtag v1.2.0/go.mod h1:mBJUNpUnHmRKrKlQQlmCrh5PuhftFbNv8Ys4/aAZl94=
github.com/fogleman/gg v1.3.0 h1:/7zJX8F6AaYQc57WQCyN9cAIz+4bCJGO9B+dyW29am8=
github.com/fogleman/gg v1.3.0/go.mod h1:R/bRT+9gY/C5z7JzPU0zXsXHKM4/ayA+zqcVNZzPa1k=
github.com/gdamore/encoding v1.0.0/go.mod h1:alR0ol34c49FCSBLjhosxzcPHQbf2trDkoo5dl+VrEg=
github.com/gdamore/tcell v1.3.0/go.mod h1:Hjvr+Ofd+gLglo7RYKxxnzCBmev3BzsS67MebKS4zMM=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
Expand Down Expand Up @@ -112,12 +119,16 @@ github.com/go-resty/resty/v2 v2.6.0/go.mod h1:PwvJS6hvaPkjtjNg9ph+VrSD92bi5Zq73w
github.com/go-sql-driver/mysql v1.6.0 h1:BCTh4TKNUYmOmMUcQ3IipzF5prigylS7XXjEkfCHuOE=
github.com/go-sql-driver/mysql v1.6.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/goccy/go-graphviz v0.0.9 h1:s/FMMJ1Joj6La3S5ApO3Jk2cwM4LpXECC2muFx3IPQQ=
github.com/goccy/go-graphviz v0.0.9/go.mod h1:wXVsXxmyMQU6TN3zGRttjNn3h+iCAS7xQFC6TlNvLhk=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
github.com/gogo/protobuf v1.3.1 h1:DqDEcV5aeaTmdFBePNpYsp3FlcVH/2ISVVM9Qf8PSls=
github.com/gogo/protobuf v1.3.1/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=
github.com/golang-jwt/jwt v3.2.1+incompatible h1:73Z+4BJcrTC+KczS6WvTPvRGOp1WmfEP4Q1lOd9Z/+c=
github.com/golang-jwt/jwt v3.2.1+incompatible/go.mod h1:8pz2t5EyA70fFQQSrl6XZXzqecmYZeUEB8OUGHkxJ+I=
github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0 h1:DACJavvAHhabrF08vX0COfcOBJRhZ8lUbR+ZWIs0Y5g=
github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0/go.mod h1:E/TSTwGwJL78qG/PmXZO1EjYhfJinVAhrmmHX6Z8B9k=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b h1:VKtxabqXZkF25pY9ekfRL6a582T4P37/31XEstQ5p58=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903 h1:LbsanbbD6LieFkXbj9YNNBupiGHJgFeLpO0j0Fza1h8=
Expand All @@ -132,6 +143,8 @@ github.com/google/btree v1.0.0 h1:0udJVsspx3VBr5FwtLhQQtuAsVc79tTq0ocGIPAU6qo=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20211122183932-1daafda22083 h1:c8EUapQFi+kjzedr4c6WqbwMdmB95+oDBWZ5XFHFYxY=
github.com/google/pprof v0.0.0-20211122183932-1daafda22083/go.mod h1:KgnwoLYCZ8IQu3XUZ8Nc/bM9CCZFOyjUNOSygVozoDg=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/uuid v1.0.0 h1:b4Gk+7WdP/d3HZH8EJsZpvV7EtDOgaZLtnaNGIu1adA=
github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
Expand All @@ -149,7 +162,10 @@ github.com/gtank/cryptopasta v0.0.0-20170601214702-1f550f6f2f69 h1:7xsUJsB2Nrdct
github.com/gtank/cryptopasta v0.0.0-20170601214702-1f550f6f2f69/go.mod h1:YLEMZOtU+AZ7dhN9T/IpGhXVGly2bvkJQ+zxj3WeVQo=
github.com/henrylee2cn/ameda v1.4.10 h1:JdvI2Ekq7tapdPsuhrc4CaFiqw6QXFvZIULWJgQyCAk=
github.com/henrylee2cn/ameda v1.4.10/go.mod h1:liZulR8DgHxdK+MEwvZIylGnmcjzQ6N6f2PlWe7nEO4=
github.com/ianlancetaylor/demangle v0.0.0-20210905161508-09a460cdf81d h1:uGg2frlt3IcT7kbV6LEp5ONv4vmoO2FW4qSO+my/aoM=
github.com/ianlancetaylor/demangle v0.0.0-20210905161508-09a460cdf81d/go.mod h1:aYm2/VgdVmcIU8iMfdMvDMsRAQjcfZSKFby6HOFvi/w=
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
github.com/jinzhu/inflection v1.0.0 h1:K317FqzuhWc8YvSVlFMCCUb36O/S9MCKRDI7QkRKD/E=
github.com/jinzhu/inflection v1.0.0/go.mod h1:h+uFLlag+Qp1Va5pdKtLDYj+kHp5pxUVkryuEj+Srlc=
github.com/jinzhu/now v1.1.1/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
Expand Down Expand Up @@ -209,6 +225,8 @@ github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lN
github.com/modern-go/reflect2 v1.0.1 h1:9f412s+6RmYXLWZSEzVVgPGK7C2PphHj5RJrvfx9AWI=
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/nfnt/resize v0.0.0-20160724205520-891127d8d1b5 h1:BvoENQQU+fZ9uukda/RzCAL/191HHwJA5b13R6diVlY=
github.com/nfnt/resize v0.0.0-20160724205520-891127d8d1b5/go.mod h1:jpp1/29i3P1S/RLdc7JQKbRpFeM1dOBd8T9ki5s+AY8=
github.com/oleiade/reflections v1.0.1 h1:D1XO3LVEYroYskEsoSiGItp9RUxG6jWnCVvrqH0HHQM=
github.com/oleiade/reflections v1.0.1/go.mod h1:rdFxbxq4QXVZWj0F+e9jqjDkc7dbp97vkRixKo2JR60=
github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5/go.mod h1:vsDQFd/mU46D+Z4whnwzcISnGGzXWMclvtLoiIKAKIo=
Expand Down Expand Up @@ -349,9 +367,12 @@ golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8U
golang.org/x/crypto v0.0.0-20190513172903-22d7a77e9e5f/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200204104054-c9f3fb736b72/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9 h1:psW17arqaxU48Z5kZ0CQnkZWQJsqcURM6tKiBApRjXI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/image v0.0.0-20200119044424-58c23975cae1 h1:5h3ngYt7+vXCDZCup/HkCQgW5XwmSvR/nA2JmJ0RErg=
golang.org/x/image v0.0.0-20200119044424-58c23975cae1/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
Expand Down Expand Up @@ -405,8 +426,9 @@ golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44 h1:Bli41pIlzTzf3KEY06n+xnzK/BESIg2ze4Pgfh/aI8c=
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20211007075335-d3039528d8ac h1:oN6lz7iLW/YC7un8pq+9bOLyXrprv2+DKfkJY+2LJJw=
golang.org/x/sys v0.0.0-20211007075335-d3039528d8ac/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
Expand Down
24 changes: 12 additions & 12 deletions pkg/apiserver/profiling/model.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,19 +22,19 @@ const (
TaskPartialFinish
)

type TaskProfileOutputType string
type TaskRawDataType string

const ProfilingOutputTypeProtobuf TaskProfileOutputType = "protobuf"
const RawDataTypeProtobuf TaskRawDataType = "protobuf"

type TaskModel struct {
ID uint `json:"id" gorm:"primary_key"`
TaskGroupID uint `json:"task_group_id" gorm:"index"`
State TaskState `json:"state" gorm:"index"`
Target model.RequestTargetNode `json:"target" gorm:"embedded;embedded_prefix:target_"`
FilePath string `json:"-" gorm:"type:text"`
Error string `json:"error" gorm:"type:text"`
StartedAt int64 `json:"started_at"` // The start running time, reset when retry. Used to estimate approximate profiling progress.
ProfileOutputType TaskProfileOutputType `json:"profile_output_type"`
ID uint `json:"id" gorm:"primary_key"`
TaskGroupID uint `json:"task_group_id" gorm:"index"`
State TaskState `json:"state" gorm:"index"`
Target model.RequestTargetNode `json:"target" gorm:"embedded;embedded_prefix:target_"`
FilePath string `json:"-" gorm:"type:text"`
Error string `json:"error" gorm:"type:text"`
StartedAt int64 `json:"started_at"` // The start running time, reset when retry. Used to estimate approximate profiling progress.
RawDataType TaskRawDataType `json:"raw_data_type"`
}

func (TaskModel) TableName() string {
Expand Down Expand Up @@ -85,7 +85,7 @@ func NewTask(ctx context.Context, taskGroup *TaskGroup, target model.RequestTarg

func (t *Task) run() {
fileNameWithoutExt := fmt.Sprintf("profiling_%d_%d_%s", t.TaskGroupID, t.ID, t.Target.FileName())
protoFilePath, profileOutputType, err := profileAndWritePprof(t.ctx, t.fetchers, &t.Target, fileNameWithoutExt, t.taskGroup.ProfileDurationSecs)
protoFilePath, rawDataType, err := profileAndWritePprof(t.ctx, t.fetchers, &t.Target, fileNameWithoutExt, t.taskGroup.ProfileDurationSecs)
if err != nil {
t.Error = err.Error()
t.State = TaskStateError
Expand All @@ -94,7 +94,7 @@ func (t *Task) run() {
}
t.FilePath = protoFilePath
t.State = TaskStateFinish
t.ProfileOutputType = profileOutputType
t.RawDataType = rawDataType
t.taskGroup.db.Save(t.TaskModel)
}

Expand Down
10 changes: 5 additions & 5 deletions pkg/apiserver/profiling/pprof.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,22 +18,22 @@ type pprofOptions struct {
fetcher *profileFetcher
}

func fetchPprof(op *pprofOptions) (string, TaskProfileOutputType, error) {
func fetchPprof(op *pprofOptions) (string, TaskRawDataType, error) {
fetcher := &fetcher{profileFetcher: op.fetcher, target: op.target}
tmpPath, profileOutputType, err := fetcher.FetchAndWriteToFile(op.duration, op.fileNameWithoutExt)
tmpPath, rawDataType, err := fetcher.FetchAndWriteToFile(op.duration, op.fileNameWithoutExt)
if err != nil {
return "", "", fmt.Errorf("failed to fetch annd write to temp file: %v", err)
}

return tmpPath, profileOutputType, nil
return tmpPath, rawDataType, nil
}

type fetcher struct {
target *model.RequestTargetNode
profileFetcher *profileFetcher
}

func (f *fetcher) FetchAndWriteToFile(duration uint, fileNameWithoutExt string) (string, TaskProfileOutputType, error) {
func (f *fetcher) FetchAndWriteToFile(duration uint, fileNameWithoutExt string) (string, TaskRawDataType, error) {
tmpfile, err := ioutil.TempFile("", fileNameWithoutExt+"*.proto")
if err != nil {
return "", "", fmt.Errorf("failed to create tmpfile to write profile: %v", err)
Expand All @@ -56,5 +56,5 @@ func (f *fetcher) FetchAndWriteToFile(duration uint, fileNameWithoutExt string)
return "", "", fmt.Errorf("failed to write profile: %v", err)
}

return tmpfile.Name(), ProfilingOutputTypeProtobuf, nil
return tmpfile.Name(), RawDataTypeProtobuf, nil
}
2 changes: 1 addition & 1 deletion pkg/apiserver/profiling/profile.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (
"github.com/pingcap/tidb-dashboard/pkg/apiserver/model"
)

func profileAndWritePprof(ctx context.Context, fts *fetchers, target *model.RequestTargetNode, fileNameWithoutExt string, profileDurationSecs uint) (string, TaskProfileOutputType, error) {
func profileAndWritePprof(ctx context.Context, fts *fetchers, target *model.RequestTargetNode, fileNameWithoutExt string, profileDurationSecs uint) (string, TaskRawDataType, error) {
switch target.Kind {
case model.NodeKindTiKV:
return fetchPprof(&pprofOptions{duration: profileDurationSecs, fileNameWithoutExt: fileNameWithoutExt, target: target, fetcher: &fts.tikv})
Expand Down
149 changes: 149 additions & 0 deletions pkg/apiserver/profiling/protobuf_to_svg.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
// Copyright 2021 PingCAP, Inc. Licensed under Apache-2.0.

package profiling

import (
"flag"
"fmt"
"io"
"strconv"
"time"

"github.com/goccy/go-graphviz"
"github.com/google/pprof/driver"
"github.com/google/pprof/profile"
)

func convertProtobufToSVG(content []byte, task TaskModel) ([]byte, error) {
dotContent, err := convertProtobufToDot(content, task)
if err != nil {
return nil, fmt.Errorf("failed to convert protobuf to dot: %v", err)
}
svgContent, err := convertDotToSVG(dotContent)
if err != nil {
return nil, fmt.Errorf("failed to convert dot to svg: %v", err)
}

return svgContent, nil
}

func convertProtobufToDot(content []byte, task TaskModel) ([]byte, error) {
args := []string{
"-dot",
// prevent printing stdout
"-output", "dummy",
"-seconds", strconv.Itoa(int(1)),
}
// the addr is required for driver. Pporf but not used here
// since we have fetched proto content and just want to convert it to dot
address := ""
args = append(args, address)
f := &flagSet{
FlagSet: flag.NewFlagSet("pprof", flag.PanicOnError),
args: args,
}

protoToDotWriter := &protobufToDotWriter{}
if err := driver.PProf(&driver.Options{
Fetch: &dotFetcher{content},
Flagset: f,
UI: &blankPprofUI{},
Writer: protoToDotWriter,
}); err != nil {
return nil, err
}

return protoToDotWriter.wc.data, nil
}

func convertDotToSVG(dotContent []byte) ([]byte, error) {
g := graphviz.New()
graph, err := graphviz.ParseBytes(dotContent)
if err != nil {
return nil, err
}

svgWriteCloser := &writeCloser{}
if err := g.Render(graph, graphviz.SVG, svgWriteCloser); err != nil {
return nil, err
}
return svgWriteCloser.data, nil
}

// implement a writer to write content to []byte.
type protobufToDotWriter struct {
wc *writeCloser
}

func (w *protobufToDotWriter) Open(name string) (io.WriteCloser, error) {
w.wc = &writeCloser{data: make([]byte, 0)}
return w.wc, nil
}

type writeCloser struct {
data []byte
}

func (wc *writeCloser) Write(p []byte) (n int, err error) {
wc.data = make([]byte, len(p))
copy(wc.data, p)
return len(p), nil
}

func (wc *writeCloser) Close() error {
return nil
}

type dotFetcher struct {
data []byte
}

func (f *dotFetcher) Fetch(src string, duration, timeout time.Duration) (*profile.Profile, string, error) {
profile, err := profile.ParseData(f.data)
return profile, "", err
}

type flagSet struct {
*flag.FlagSet
args []string
}

func (f *flagSet) StringList(o, d, c string) *[]*string {
return &[]*string{f.String(o, d, c)}
}

func (f *flagSet) ExtraUsage() string {
return ""
}

func (f *flagSet) Parse(usage func()) []string {
f.Usage = usage
_ = f.FlagSet.Parse(f.args)
return f.Args()
}

func (f *flagSet) AddExtraUsage(eu string) {}

// blankPprofUI is used to eliminate the pprof logs.
type blankPprofUI struct{}

func (b blankPprofUI) ReadLine(prompt string) (string, error) {
panic("not support")
}

func (b blankPprofUI) Print(i ...interface{}) {
}

func (b blankPprofUI) PrintErr(i ...interface{}) {
}

func (b blankPprofUI) IsTerminal() bool {
return false
}

func (b blankPprofUI) WantBrowser() bool {
return false
}

func (b blankPprofUI) SetAutoComplete(complete func(string) string) {
}
Loading