[Enhancement](fe exception) write a java annotation to catch throwable from a method and print log #17797

NetShrimp06 · 2023-03-14T15:47:33Z

Proposed changes

Issue Number: close #17238

Problem summary

How it works?
Aspectj is used to implement the aspect function of annotations. During the compilation process, the aspectj-maven-plugin plugin will automatically weave the code with aspect annotations into the generated classes file.
When to use to?
When a method wants to add a try catch to save exception information, the LogException annotation can be used. When there is a method that does not allow errors, the NoException annotation can be used.
What is the result when adding this annotation?
Use the LogException annotation to automatically capture exceptions into the Log file, and the code can be more concise. Use the NoException annotation to automatically capture the exception to the Log file and exit the program when an exception occurs.

Checklist(Required)

Does it affect the original behavior
Has unit tests been added
Has document been added or modified
Does it need to update dependencies
Is this PR support rollback (If NO, please explain WHY)

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

…log_annotation

…on_log_annotation' into kongxuan.yao/Feature/add_exception_log_annotation

…otation

morningman · 2023-03-14T23:52:12Z

Could you please describe:

How it works?
When to use to?
What is the result when adding this annotation?

morningman · 2023-03-14T23:53:11Z

fe/fe-core/src/main/java/org/apache/doris/aspectj/LogAspect.java

+
+    @AfterThrowing(value = "throwingLogPointCut()", throwing = "e")
+    public void exceptionLog(Exception e)  {
+        LOG.error(e.getMessage());


Print 2 logs in one line.
And do we need to print stack trace for each exception? Maybe we can use a flag to control it?

I added a parameter to let the user decide whether to print the stack information, the default is not to print

NetShrimp06 · 2023-03-15T06:03:38Z

How it works?
Aspectj is used to implement the aspect function of annotations. During the compilation process, the aspectj-maven-plugin plugin will automatically weave the code with aspect annotations into the generated classes file.
When to use to?
When a method wants to add a try catch to save exception information, the LogException annotation can be used. When there is a method that does not allow errors, the NoException annotation can be used.
What is the result when adding this annotation?
Use the LogException annotation to automatically capture exceptions into the Log file, and the code can be more concise. Use the NoException annotation to automatically capture the exception to the Log file and exit the program when an exception occurs.

hello-stephen · 2023-03-16T03:47:54Z

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.05 seconds
stream load tsv: 455 seconds loaded 74807831229 Bytes, about 156 MB/s
stream load json: 23 seconds loaded 2358488459 Bytes, about 97 MB/s
stream load orc: 73 seconds loaded 1101869774 Bytes, about 14 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230316123041_clickbench_pr_115409.html

yiguolei · 2023-03-16T06:37:26Z

fe/fe-core/src/main/java/org/apache/doris/aspectj/LogAspect.java

+        ExceptionLog exceptionLog = method.getAnnotation(ExceptionLog.class);
+        LOG.warn(e.getMessage());
+        if (exceptionLog != null && exceptionLog.isGetStackTrace()) {
+            LOG.warn(getStackTrace(e));


LOG.warn(e) will print stack trace automatically.

yiguolei · 2023-03-16T06:37:50Z

fe/fe-core/src/main/java/org/apache/doris/load/loadv2/BulkLoadJob.java

@@ -181,6 +182,7 @@ public Set<String> getTableNamesForShow() {
                .collect(Collectors.toSet());
    }

+    @ExceptionLog


using @LogException instead.

yiguolei · 2023-03-16T06:38:59Z

fe/fe-core/src/main/java/org/apache/doris/aspectj/LogAspect.java

+    }
+
+    @AfterThrowing(value = "throwingLogPointCut()", throwing = "e")
+    public void exceptionLog(JoinPoint point, Exception e)  {


using Throwable intead of Exception, Exception is only subset of errors.

yiguolei · 2023-03-16T06:41:22Z

fe/fe-core/src/main/java/org/apache/doris/aspectj/LogAspect.java

+import java.lang.reflect.Method;
+
+@Aspect
+public class LogAspect {


Could you please add another annotation.
@NoException
If a method throws any exceptions, should print log and call system.exit.

yiguolei

LGTM

github-actions · 2023-03-16T09:19:39Z

PR approved by at least one committer and no changes requested.

github-actions · 2023-03-16T09:19:42Z

PR approved by anyone and no changes requested.

yiguolei · 2023-03-16T10:43:46Z

run buildall

englefly · 2023-03-17T05:16:01Z

it does not work on mac, or newer version of JDK

In #17797 , we introduced aspectj to help log exception easily. However, the plugin version 1.11 do not support jdk9 and later. For support compile FE with jdk11 update aspectj-maven-plugin to 1.14.0 version add new dependency org.aspectj.aspectjrt 1.9.7 to fe-core according to: aspectj java version compatibility aspectj-maven-plugin issue aspectj release note intro to aspectj

…e from a method and print log (apache#17797) How it works? Aspectj is used to implement the aspect function of annotations. During the compilation process, the aspectj-maven-plugin plugin will automatically weave the code with aspect annotations into the generated classes file. When to use to? When a method wants to add a try catch to save exception information, the LogException annotation can be used. When there is a method that does not allow errors, the NoException annotation can be used. What is the result when adding this annotation? Use the LogException annotation to automatically capture exceptions into the Log file, and the code can be more concise. Use the NoException annotation to automatically capture the exception to the Log file and exit the program when an exception occurs.

In apache#17797 , we introduced aspectj to help log exception easily. However, the plugin version 1.11 do not support jdk9 and later. For support compile FE with jdk11 update aspectj-maven-plugin to 1.14.0 version add new dependency org.aspectj.aspectjrt 1.9.7 to fe-core according to: aspectj java version compatibility aspectj-maven-plugin issue aspectj release note intro to aspectj

NetShrimp and others added 12 commits March 8, 2023 13:37

add java annotation to catch throwable from a method and print log

83a46b7

add java annotation to catch throwable from a method and print log

879bf22

add java annotation to catch throwable from a method and print log

1e03d61

add java annotation to catch throwable from a method and print log

140d62e

Merge branch 'apache:master' into kongxuan.yao/Feature/add_exception_…

e2e8696

…log_annotation

add java annotation to catch throwable from a method and print log

be73232

Merge branch 'apache:master' into kongxuan.yao/Feature/add_exception_…

07cd501

…log_annotation

add java annotation to catch throwable from a method and print log

a1c4a47

Merge remote-tracking branch 'origin/kongxuan.yao/Feature/add_excepti…

7847925

…on_log_annotation' into kongxuan.yao/Feature/add_exception_log_annotation

add java annotation to catch throwable from a method and print log

7ee4913

Merge branch 'master' into kongxuan.yao/Feature/add_exception_log_ann…

81df920

…otation

add java annotation to catch throwable from a method and print log

a7e3cfe

github-actions bot added the area/load Issues or PRs related to all kinds of load label Mar 14, 2023

add java annotation to catch throwable from a method and print log

dda56ba

morningman reviewed Mar 14, 2023

View reviewed changes

add java annotation to catch throwable from a method and print log

1da3308

NetShrimp06 changed the title ~~[Enhancement] (fe exception) write a java annotation to catch throwable from a method and print log~~ [Enhancement](fe exception) write a java annotation to catch throwable from a method and print log Mar 15, 2023

add java annotation to catch throwable from a method and print log

7389ae3

yiguolei reviewed Mar 16, 2023

View reviewed changes

kongxuan.yao added 3 commits March 16, 2023 16:20

add java annotation to catch throwable from a method and print log

900e1c2

add java annotation to catch throwable from a method and print log

4c776a3

add java annotation to catch throwable from a method and print log

c47be56

yiguolei approved these changes Mar 16, 2023

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 16, 2023

github-actions bot added the reviewed label Mar 16, 2023

yiguolei merged commit 0ec10d4 into apache:master Mar 17, 2023

morrySnow mentioned this pull request Mar 17, 2023

[chore](fe) update aspectj-maven-plugin to 1.14.0 version #17890

Merged

5 tasks

morningman added the dev/1.2.3 label Mar 18, 2023

morningman mentioned this pull request Mar 29, 2023

[Good First Issue] Doris' Future #17176

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Enhancement](fe exception) write a java annotation to catch throwable from a method and print log #17797

[Enhancement](fe exception) write a java annotation to catch throwable from a method and print log #17797

NetShrimp06 commented Mar 14, 2023 •

edited

Loading

morningman commented Mar 14, 2023

morningman Mar 14, 2023

NetShrimp06 Mar 15, 2023

NetShrimp06 commented Mar 15, 2023 •

edited

Loading

hello-stephen commented Mar 16, 2023 •

edited

Loading

yiguolei Mar 16, 2023

yiguolei Mar 16, 2023

yiguolei Mar 16, 2023 •

edited

Loading

yiguolei Mar 16, 2023

yiguolei left a comment

github-actions bot commented Mar 16, 2023

github-actions bot commented Mar 16, 2023

yiguolei commented Mar 16, 2023

englefly commented Mar 17, 2023

[Enhancement](fe exception) write a java annotation to catch throwable from a method and print log #17797

[Enhancement](fe exception) write a java annotation to catch throwable from a method and print log #17797

Conversation

NetShrimp06 commented Mar 14, 2023 • edited Loading

Proposed changes

Problem summary

Checklist(Required)

Further comments

morningman commented Mar 14, 2023

morningman Mar 14, 2023

Choose a reason for hiding this comment

NetShrimp06 Mar 15, 2023

Choose a reason for hiding this comment

NetShrimp06 commented Mar 15, 2023 • edited Loading

hello-stephen commented Mar 16, 2023 • edited Loading

yiguolei Mar 16, 2023

Choose a reason for hiding this comment

yiguolei Mar 16, 2023

Choose a reason for hiding this comment

yiguolei Mar 16, 2023 • edited Loading

Choose a reason for hiding this comment

yiguolei Mar 16, 2023

Choose a reason for hiding this comment

yiguolei left a comment

Choose a reason for hiding this comment

github-actions bot commented Mar 16, 2023

github-actions bot commented Mar 16, 2023

yiguolei commented Mar 16, 2023

englefly commented Mar 17, 2023

NetShrimp06 commented Mar 14, 2023 •

edited

Loading

NetShrimp06 commented Mar 15, 2023 •

edited

Loading

hello-stephen commented Mar 16, 2023 •

edited

Loading

yiguolei Mar 16, 2023 •

edited

Loading