Skip to content

Add support for Artificial Errors in Summaries #1785

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 13, 2023

Conversation

sofurihafe
Copy link
Member

@sofurihafe sofurihafe commented Feb 10, 2023

Description

This PR add support for ArtificialErrors in summaries.
Now, if we want to generate a test for method, where some suspecious behavior was detected by engine (such as Overflow), instead of treating it as a throwable, we notify that an error was occurred. See the following.
Consider the following method:

public short shortMulOverflow(short x, short y) {
    return (short) (x * y);
}

Now, generated code should look like this:

/**
 * @utbot.classUnderTest {@link OverflowExamples}
 * @utbot.methodUnderTest {@link org.utbot.examples.math.OverflowExamples#shortMulOverflow(short,short)}
 * @utbot.returnsFrom {@code return (short) (x * y);}
 */
@Test
@DisplayName("shortMulOverflow: -> return (short) (x * y)")
public void testShortMulOverflow_ReturnXy() {
    OverflowExamples overflowExamples = new OverflowExamples();
    
    short actual = overflowExamples.shortMulOverflow((short) -32, (short) -94);
    
    assertEquals((short) 3008, actual);
}

/**
 * @utbot.classUnderTest {@link OverflowExamples}
 * @utbot.methodUnderTest {@link org.utbot.examples.math.OverflowExamples#shortMulOverflow(short,short)}
 * @utbot.detectsSuspiciousBehavior in: return (short) (x * y);
 */
@Test
@DisplayName("shortMulOverflow: return (short) (x * y) : True -> DetectOverflow")
public void testShortMulOverflow_DetectOverflow() {
    OverflowExamples overflowExamples = new OverflowExamples();
    
    overflowExamples.shortMulOverflow((short) 252, (short) 132);
    fail("Overflow detected in 'shortMulOverflow' call");
}
///endregion

///endregion

Pay attention to the new "utbot.detectsError" info block of the second test.

Also, extra summary test was added in order to fixate the behavior.

Fixes #1739

How to test

Automated tests

utbot-summery-tests.

@sofurihafe sofurihafe added comp-codegen Issue is related to code generator comp-summaries Something related to the method names, code comments and display names generation ctg-bug-fix PR is fixing a bug labels Feb 10, 2023
@sofurihafe sofurihafe force-pushed the andrey-t/artificial_errors_in_summaries branch 5 times, most recently from b9f51af to 4ad56eb Compare February 12, 2023 12:44
OverflowExamples::class
) {
@Test
fun testSimpleSwitch() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

!

@@ -498,6 +520,7 @@ open class StringsTemplatesSingular : StringsTemplatesInterface {
override val noIteration: String = "does not iterate"
override val codeSentence: String = "$OPEN_BRACKET$AT_CODE %s$CLOSE_BRACKET"
override val throwSentence: String = "throws %s"
override val suspiciousBehaviorDetectedSentence: String = "encounters suspicious behavior %s"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

encounter -> detect

@sofurihafe sofurihafe force-pushed the andrey-t/artificial_errors_in_summaries branch from 4ad56eb to 28994c9 Compare February 13, 2023 07:52
@EgorkaKulikov EgorkaKulikov self-requested a review February 13, 2023 07:52
@EgorkaKulikov EgorkaKulikov enabled auto-merge (squash) February 13, 2023 07:53
@sofurihafe sofurihafe force-pushed the andrey-t/artificial_errors_in_summaries branch from 28994c9 to 0e1f427 Compare February 13, 2023 10:17
@EgorkaKulikov EgorkaKulikov merged commit 196dc2a into main Feb 13, 2023
@EgorkaKulikov EgorkaKulikov deleted the andrey-t/artificial_errors_in_summaries branch February 13, 2023 10:33
@alisevych alisevych added this to the 2023.03 Release milestone Mar 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp-codegen Issue is related to code generator comp-summaries Something related to the method names, code comments and display names generation ctg-bug-fix PR is fixing a bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support artificial errors rendering in summaries
3 participants