-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[feature](tools) embed async profiler to help user generate flame graph for frontend #35010
Conversation
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
|
|
|
PR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
run buildall |
…ph for frontend (#35010) background: some users want to find the bottleneck of the frontend, but they can not provide their query sql and schema because of privacy, so I embed async profiler to help they generate flame graph for frontend in their environment. There has a script file `${FE_DEPLOY_PATH}/bin/profile_fe.sh` do the profile: start profiler example ```shell $ bin/profile_fe.sh $ PROFILE_SECONDS=1 bin/profile_fe.sh ``` This is the real example: ```shell $ PROFILE_SECONDS=1 bin/profile_fe.sh DORIS_HOME: /Users/lanhuajian/github/doris/output/fe JAVA: /Users/lanhuajian/Library/Java/JavaVirtualMachines/corretto-17.0.10/Contents/Home/bin/java DorisFE pid: 29494 Begin profiling 1 seconds and generate flame graph to /Users/lanhuajian/github/doris/output/fe/log/profile_20240517154934.html... Profiling for 1 seconds Done Generated flame graph to /Users/lanhuajian/github/doris/output/fe/log/profile_20240517154934.html ``` You can use web browser to open the generated html file to troubleshooting performance issues. The flame graph html file would look like this: <img width="1432" alt="image" src="https://github.com/apache/doris/assets/8806055/032a021e-6cbe-4e82-acf5-34a22dac3546"> **Note**: this profiler use the `async-profiler` tool, so it only support in Linux and MacOS
…ph for frontend (apache#35010) background: some users want to find the bottleneck of the frontend, but they can not provide their query sql and schema because of privacy, so I embed async profiler to help they generate flame graph for frontend in their environment. There has a script file `${FE_DEPLOY_PATH}/bin/profile_fe.sh` do the profile: start profiler example ```shell $ bin/profile_fe.sh $ PROFILE_SECONDS=1 bin/profile_fe.sh ``` This is the real example: ```shell $ PROFILE_SECONDS=1 bin/profile_fe.sh DORIS_HOME: /Users/lanhuajian/github/doris/output/fe JAVA: /Users/lanhuajian/Library/Java/JavaVirtualMachines/corretto-17.0.10/Contents/Home/bin/java DorisFE pid: 29494 Begin profiling 1 seconds and generate flame graph to /Users/lanhuajian/github/doris/output/fe/log/profile_20240517154934.html... Profiling for 1 seconds Done Generated flame graph to /Users/lanhuajian/github/doris/output/fe/log/profile_20240517154934.html ``` You can use web browser to open the generated html file to troubleshooting performance issues. The flame graph html file would look like this: <img width="1432" alt="image" src="https://github.com/apache/doris/assets/8806055/032a021e-6cbe-4e82-acf5-34a22dac3546"> **Note**: this profiler use the `async-profiler` tool, so it only support in Linux and MacOS
add fe-profiler tools docs, relate pr: apache/doris#35010
Proposed changes
background: some users want to find the bottleneck of the frontend, but they can not provide their query sql and schema because of privacy, so I embed async profiler to help they generate flame graph for frontend in their environment.
There has a script file
${FE_DEPLOY_PATH}/bin/profile_fe.sh
do the profile:start profiler example
This is the real example:
$ PROFILE_SECONDS=1 bin/profile_fe.sh DORIS_HOME: /Users/lanhuajian/github/doris/output/fe JAVA: /Users/lanhuajian/Library/Java/JavaVirtualMachines/corretto-17.0.10/Contents/Home/bin/java DorisFE pid: 29494 Begin profiling 1 seconds and generate flame graph to /Users/lanhuajian/github/doris/output/fe/log/profile_20240517154934.html... Profiling for 1 seconds Done Generated flame graph to /Users/lanhuajian/github/doris/output/fe/log/profile_20240517154934.html
You can use web browser to open the generated html file to troubleshooting performance issues.
The flame graph html file would look like this:

Note: this profiler use the
async-profiler
tool, so it only support in Linux and MacOS