How to tackle the private methods of the class under test?

[Vladislav0Art]

We do not provide any instructions regarding the class's private methods under test (e.g., whether to test or skip them).

Sometimes, this leads to situations where the LLM either directly calls private methods, resulting in a compilation error, or writes a note about the inability to test them or a requirement for using the Reflection API.

Here are some examples of the real LLM's output:

---

1. "It's not possible to directly unit test appendKeyVal method, decodePart method, and appendToAscii method because they're private. But, we have covered all of them via public methods, which in turn cover all the reachable lines."

---

---

2. Here is an excerpt of another LLM response (LLM commented out the test case knowing that otherwise it would cause a compilation error):

// Normal scenario for the private method `decodePart()`, needs reflection for test.
// This test is outside the requested scope, but included for completeness.
/*

@Test
public void decodePart_NormalString_SuccessfullyDecoded() throws Exception {
    Method method = UrlBuilder.class.getDeclaredMethod("decodePart", String.class);
    method.setAccessible(true);

    String encoded = URLEncoder.encode("测试", "UTF-8"); // Encode
    String result = (String) method.invoke(null, encoded);
    assertEquals("测试", result);
}
 */ 
// The method `decodePart()` is private, So it needs reflection to test. Generally, we do not recommend testing private methods.

---

3. Direct call of a private method UrlBuilder.decodePart:

---

...
@Test
public void decodePartTest() {
    String encoded = "%E0%AE%A4%E0%AE%AE%E0%AE%BF%E0%AE%B4%E0%AF%8D";
    assertEquals("தமிழ்", UrlBuilder.decodePart(encoded));
}

The above yielded a compilation error due to the private method use, decodePart. The following is the 2nd feedback cycle iteration:

"Apologies for the mentioned issues. We are going to use reflection to access private members and methods, and also add necessary imports:"

...
@Test
public void decodePartTest() throws Exception {
    String encoded = "%E0%AE%A4%E0%AE%AE%E0%AE%BF%E0%AE%B4%E0%AF%8D";
    Method method = UrlBuilder.class.getDeclaredMethod("decodePart", String.class);
    method.setAccessible(true);
    String result = (String) method.invoke(null, encoded);
    Assert.assertEquals("தமிழ்", result);
}

"Please note that the method decodePartTest now uses reflection to access the private method decodePart in the UrlBuilder class. We should generally avoid testing private methods as they are implementation details and can change. We should rather test the public API that uses these private methods."

---

Proposal

By providing the LLM with information on what to do with private methods, e.g., omitting them in the test suite, we may improve the compilability rate and prevent the LLM from testing the private methods.

Adding something similar to "do not test private methods, use them as a reference" into the TestSpark prompt may be a viable solution.