Skip to content

Improve detection of NPEs caused by this instance fields being null in Spring unit tests #2617

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Sep 26, 2023

Conversation

IlyaMuravjov
Copy link
Collaborator

@IlyaMuravjov IlyaMuravjov commented Sep 22, 2023

Description

Fixes #2589 (desired NPE test is generated by fuzzer, because symbolic engine has troubles dealing with getClass() == o.getClass() condition.

To make fuzzer able to generate Object of type Order special JavaLangObjectValueProvider was added that tries to use classUnderTest whenever java.lang.Object is needed (plain new Object() can still be used where java.lang.Object is needed).

In case class under test has some hard to pass validation inside of the constructor and fuzzer is asked to create multiple java.lang.Objects, RemovingConstructFailsUtExecutionInstrumentation was used in all Spring tests (in short, it detects impossible to create due to exceptions thrown by constructor models in stateBefore and replaces them with UtNullModels, making it easier for fuzzer to find "creatable" input and actually call method under test).

Finally, InjectMocksValueProvider was made to respect NonNullSpeculator (previously fuzzer considered all fields to be non-null).

How to test

Manual tests

NPE unit be generated by fuzzer for Order.equals() from spring-boot-testing project.

Generate unit tests for the following classes individually with fuzzer only and with engine only, each of them should generate NPE tests for methods marked with "NPE expected" and no NPE tests for methods marked with "no NPE expected".

package com.rest.order.services;

import com.rest.order.models.Order;
import com.rest.order.repositories.OrderRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.lang.Nullable;
import org.springframework.stereotype.Service;

import java.util.List;

@Service
public class ServiceExample {
    /// region notAutowired

    private OrderRepository notAutowired;

    public void setNotAutowired(OrderRepository notAutowired) {
        this.notAutowired = notAutowired;
    }

    @Autowired
    public void fakeSetNotAutowired(OrderRepository notAutowired) {}

    @Autowired
    public void setNonRequiredNotAutowired(@Autowired(required = false) OrderRepository notAutowired) {
        this.notAutowired = notAutowired;
    }

    @Autowired(required = false)
    public void nonRequiredSetNotAutowired(OrderRepository notAutowired) {
        this.notAutowired = notAutowired;
    }

    @Autowired
    public void setNullable(@Nullable OrderRepository notAutowired) {
        this.notAutowired = notAutowired;
    }

    public ServiceExample(OrderRepository notAutowired) {
        this.notAutowired = notAutowired;
    }

    @Autowired(required = false)
    public ServiceExample(OrderRepository notAutowired, boolean unused) {
        this.notAutowired = notAutowired;
    }

    @Autowired
    public ServiceExample(@Autowired(required = false) OrderRepository notAutowired, char unused) {
        this.notAutowired = notAutowired;
    }

    // NPE expected
    public List<Order> useNotAutowired() {
        return notAutowired.findAll();
    }

    /// endregion

    /// region autowiredViaConstructor

    private OrderRepository autowiredViaConstructor;

    @Autowired
    public ServiceExample(OrderRepository orderRepository, int unused) {
        this.autowiredViaConstructor = orderRepository;
    }

    // no NPE expected
    public List<Order> useAutowiredViaConstructor() {
        return autowiredViaConstructor.findAll();
    }

    /// endregion

    /// endregion

    /// region autowiredViaField

    @Autowired
    private OrderRepository autowiredViaField;

    // no NPE expected
    public List<Order> useAutowiredViaField() {
        return autowiredViaField.findAll();
    }

    /// endregion

    /// region autowiredViaSetter

    private OrderRepository autowiredViaSetter;

    @Autowired
    public void setAutowiredViaSetter(OrderRepository orderRepository) {
        this.autowiredViaSetter = orderRepository;
    }

    // no NPE expected
    public List<Order> useAutowiredViaSetter() {
        return autowiredViaSetter.findAll();
    }

    /// endregion

}
package com.rest.order.services;

import com.rest.order.models.Order;
import com.rest.order.repositories.OrderRepository;
import org.springframework.stereotype.Service;

import java.util.List;

@Service
public class ServiceExample2 {
    private final OrderRepository autowiredViaOnlyConstructor;

    public ServiceExample2(OrderRepository orderRepository) {
        this.autowiredViaOnlyConstructor = orderRepository;
    }

    // no NPE expected
    public List<Order> useAutowiredViaOnlyConstructor() {
        return autowiredViaOnlyConstructor.findAll();
    }
}
package com.rest.order.services;

import com.rest.order.models.Order;
import com.rest.order.repositories.OrderRepository;

import java.util.List;

public class ServiceExample3 {
    private final OrderRepository notAutowiredViaOnlyConstructor;

    public ServiceExample3(OrderRepository orderRepository) {
        this.notAutowiredViaOnlyConstructor = orderRepository;
    }

    // NPE expected
    public List<Order> useNotAutowiredViaOnlyConstructor() {
        return notAutowiredViaOnlyConstructor.findAll();
    }
}
package com.rest.order.services;

import com.rest.order.models.Order;
import com.rest.order.repositories.OrderRepository;
import org.springframework.lang.Nullable;
import org.springframework.stereotype.Service;

import java.util.List;

@Service
public class ServiceExample4 {
    public OrderRepository notAutowiredViaOnlyConstructor;
    private final OrderService orderService;

    public ServiceExample4(@Nullable OrderRepository orderRepository, OrderService orderService) {
        this.notAutowiredViaOnlyConstructor = orderRepository;
        this.orderService = orderService;
    }

    // NPE expected
    public List<Order> useNotAutowiredViaOnlyConstructor() {
        return notAutowiredViaOnlyConstructor.findAll();
    }
}

Self-check list

  • I've set the proper labels for my PR (at least, for category and component).
  • PR title and description are clear and intelligible.
  • I've added enough comments to my code, particularly in hard-to-understand areas.
  • The functionality I've repaired, changed or added is covered with automated tests.
  • Manual tests have been provided optionally.
  • The documentation for the functionality I've been working on is up-to-date.

@IlyaMuravjov IlyaMuravjov added ctg-enhancement New feature, improvement or change request comp-symbolic-engine Issue is related to the symbolic execution engine comp-fuzzing Issue is related to the fuzzing comp-spring Issue is related to Spring projects support labels Sep 22, 2023
@EgorkaKulikov EgorkaKulikov self-requested a review September 22, 2023 16:42
@IlyaMuravjov IlyaMuravjov changed the title Improve NPE detection caused by this instance fields being null in Spring unit tests Improve detection of NPEs caused by this instance fields being null in Spring unit tests Sep 25, 2023
}

// a method for testing the case when the Engine produces two models on @Autowired variables of the same type
public Boolean checker() {
return personOne.getName().equals("k") && personTwo.getName().length() > 5 && baseOrders.isEmpty();
Copy link
Collaborator Author

@IlyaMuravjov IlyaMuravjov Sep 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NOTE: this was not a sufficient condition to guarantee that personOne != personTwo, since engine could have just created one personMock and configure it like this: when(personMock.getName()).thenReturn("k", "123456"), so this test was updated. Discussed changes with original author @tepa46 and added him to reviewers.

@IlyaMuravjov IlyaMuravjov merged commit d514a7b into main Sep 26, 2023
@IlyaMuravjov IlyaMuravjov deleted the ilya_m/spring-detect-npe branch September 26, 2023 09:46
@alisevych alisevych added this to the October Release milestone Sep 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp-fuzzing Issue is related to the fuzzing comp-spring Issue is related to Spring projects support comp-symbolic-engine Issue is related to the symbolic execution engine ctg-enhancement New feature, improvement or change request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Detect NPE caused by this instance fields being null in Spring unit tests
4 participants