Skip to content

Timeout tests are sometimes generated by Fuzzing for simple functions #2262

Open
@alisevych

Description

@alisevych

Description

Timeout tests are sometimes generated by Fuzzing for simplest functions.
For example for a+b

To Reproduce

  1. Install UnitTestBot plugin built from main in IntelliJ IDEA
  2. Open/create a project (Maven with JDK 17 was used)
  3. Generate tests for the following class with default settings:
public class Addition {
    public int addition(int a, int b){
        return a + b;
    }
}

Expected behavior

One test is expected to be generated.

Actual behavior

Once there was also a timeout test generated by Fuzzing.

Screenshots, logs

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;

import java.time.Duration;
import java.util.concurrent.TimeUnit;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTimeoutPreemptively;

public final class AdditionTest {
    ///region Test suites for executable org.example.Addition.addition

    ///region SYMBOLIC EXECUTION: SUCCESSFUL EXECUTIONS for method addition(int, int)

    /**
     * @utbot.classUnderTest {@link Addition}
     * @utbot.methodUnderTest {@link Addition#addition(int, int)}
     * @utbot.returnsFrom {@code return a + b;}
     */
    @Test
    @DisplayName("addition: -> return a + b")
    public void testAddition_ReturnAPlusB() {
        Addition addition = new Addition();

        int actual = addition.addition(-255, -255);

        assertEquals(-510, actual);
    }
    ///endregion

    ///region FUZZER: TIMEOUTS for method addition(int, int)

    /**
     * @utbot.classUnderTest {@link Addition}
     * @utbot.methodUnderTest {@link Addition#addition(int, int)}
     */
    @Test
    @DisplayName("addition: a = -1 (mutated from -4194305), b = -2147483646 (mutated from min)")
    @org.junit.jupiter.api.Timeout(value = 1000L, unit = TimeUnit.MILLISECONDS)
    public void testAddition() {
        Addition addition = new Addition();
        
        /* This execution may take longer than the 1000 ms timeout
         and therefore fail due to exceeding the timeout. */
        assertTimeoutPreemptively(Duration.ofMillis(1000L), () -> addition.addition(-1, -2147483646));
    }
    ///endregion

    ///endregion
}

There seems to be nothing special in logs with this exact Fuzzing test generation:
utbot-engine-current_part.log
idea_part.log

Environment

IntelliJ IDEA 2023.1 (Community Edition)
Project - Maven
JDK - corretto-17

Additional context

Sometimes reproducing for simple functions of different times.
But has not reproduced on same function in a row.
Probably there is some performance issue with concrete execution.

Metadata

Metadata

Assignees

Labels

comp-fuzzingIssue is related to the fuzzingcomp-instrumented-processIssue is related to Instrumented processctg-bugIssue is a buglang-javaIssue is related to Java supportspec-false-positiveFalse-positive error test is generated

Type

No type

Projects

Status

Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions