Skip to content

Commit 591e08b

Browse files
committed
Initial section on testing
1 parent c765e83 commit 591e08b

File tree

2 files changed

+111
-18
lines changed

2 files changed

+111
-18
lines changed

src/index.md

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -333,6 +333,115 @@ For the purposes of brevity this document will use the term "bug" to refer to bo
333333

334334
_Congratulations!_ Your changeset will now make its way towards a promoted build. When the changeset becomes part of a promoted build, the bug's "Resolved in Build" will have a value of \"b\[1-9\]\[0-9\]*\" to indicate the build number.
335335

336+
# Testing Changes
337+
338+
In addition to your own Java applications, OpenJDK have support for two test frameworks, JTReg and GTest. JTReg is a Java regression test framework that is used for most of the tests that are included in the OpenJDK source repository. The Google Test (GTest) framework is intended for unit testing of the C++ native code.
339+
340+
## JTReg
341+
342+
In depth documentation about the JTReg framework is found here: [JTReg harness](https://openjdk.java.net/jtreg/). JTReg itself is available in the [Code Tools Project](https://openjdk.java.net/projects/code-tools/).
343+
344+
Below is a small example of a JTReg test. It’s a clean Java class with a main method that is called from the test harness. If the test fails we throw a RuntimeException. This is picked up by the harness and is reported as a test failure. Try to always write a meaningful message in the exception. One that actually helps with understanding what went wrong once the test fails.
345+
346+
/*
347+
* @test
348+
* @summary Make sure feature X handles Y correctly
349+
*/
350+
public class TestXY {
351+
public static void main(String[] args) throws Exception {
352+
var result = X.y();
353+
if (result != expected_result) {
354+
throw new RuntimeException("X.y() gave " + result + ", expexted " + expected_result);
355+
}
356+
}
357+
}
358+
359+
This example only utilizes two JTReg specific tags, `@test` and `@summary`. `@test` simply tells JTReg that this class is a test, and `@summary` provides a description of the test. There are several other tags that can be used in JTReg tests. You can for instance associate the test with a specific bug that this test is a regression test for.
360+
361+
@bug 7000001
362+
363+
Or you can specify a number of requirements that must be fulfilled for JTReg to execute the test.
364+
365+
@requires docker.support
366+
@requires os.family != ”windows”
367+
@requires os.maxMemory > 3G
368+
@requires os.arch=="x86_64" | os.arch=="amd64"
369+
370+
You can also specify if the test requires specific modules, and you can specify command line flags and run the test in several different ways.
371+
372+
@modules java.base/jdk.internal.misc
373+
@run main/othervm -Xmx128m TestStringEndify
374+
375+
Note that you can have several `@run` tags in the same test with different command line options.
376+
377+
The [JTReg documentation](https://openjdk.java.net/jtreg/) provides information on many more tags like these.
378+
379+
### Running OpenJDK JTReg Tests
380+
381+
When configuring the OpenJDK build you can tell it where your JTReg installation is located. When providing this information you can later run `make run-test` to execute JTReg tests.
382+
383+
sh ./configure --with-jtreg=/path/to/jtreg
384+
make run-test TEST=tier1
385+
386+
In the OpenJDK source tree you can find a directory called `test`. There are a large number of tests in this directory that are written to be used with JTReg.
387+
388+
make run-test TEST=test/jdk/java/lang/String/
389+
390+
You can also run JTReg without invoking make. In this case you’ll need to tell JTReg which JDK to test.
391+
392+
jtreg -jdk:/path/to/jdk /path/to/test
393+
394+
## GTest
395+
396+
As mentioned the Google test framework is mainly used for C++ unit tests. There are several of these in the `test/hotspot` directory. The tests can be run without starting the JVM, which enables testing of JVM data structures that would be fragile to play with in a running JVM.
397+
398+
static int demo_comparator(int a, int b) {
399+
if (a == b) {
400+
return 0;
401+
}
402+
if (a < b) {
403+
return -1;
404+
}
405+
return 1;
406+
}
407+
408+
TEST(Demo, quicksort) {
409+
int test_array[] = {7,1,5,3,6,9,8,2,4,0};
410+
int expected_array[] = {0,1,2,3,4,5,6,7,8,9};
411+
412+
QuickSort::sort(test_array, 10, demo_comparator, false);
413+
for (int i = 0; i < 10; i++) {
414+
ASSERT_EQ(expected_array[i], test_array[i]);
415+
}
416+
}
417+
418+
`ASSERT_EQ` is one example of an assertion that can be used in the test. Below are a few other examples. A full list is found in the [Google Test Documentation](https://github.com/google/googletest/blob/master/googletest/docs/primer.md).
419+
420+
EXPECT_TRUE(condition);
421+
EXPECT_FALSE(condition);
422+
ASSERT_TRUE(condition);
423+
ASSERT_FALSE(condition);
424+
EXPECT_EQ(expected, actual);
425+
EXPECT_NE(val1, val2);
426+
EXPECT_LT(val1, val2);
427+
EXPECT_LE(val1, val2);
428+
EXPECT_GT(val1, val2);
429+
EXPECT_GE(val1, val2);
430+
EXPECT_STREQ(expected_str, actual_str);
431+
432+
`ASSERT` is a fatal assertion and will give you fast failure. That means that test execution will be stopped and the failure will be reported. `EXPECT` is a nonfatal assertion and will report the error but continues to run the test. All assertions have both an `ASSERT` and an `EXPECT` variant.
433+
434+
### Running OpenJDK GTests
435+
436+
To run GTests in OpenJDK use make:
437+
438+
make test-hotspot-gtest
439+
440+
You can use the environment variable `GTEST_FILTER` to select subsets of tests. The filter should be a regular expression.
441+
442+
GTEST_FILTER="code.*:os.*"
443+
GTEST_FILTER="os.*-os.page_size_*"
444+
336445
# Producing a Changeset
337446

338447
This section is confined to the actual Mercurial mechanics required to produce a changeset:

src/testingChanges.md

Lines changed: 2 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,3 @@
1-
% Testing Changes
1+
% Page moved
22

3-
::: {.NavBit}
4-
[« Previous](reviewBodies.html)[TOC](index.html)[Next »](jckAcquisition.html)
5-
:::
6-
7-
This section will describe the need for tests and will provide an engineering
8-
perspective to the different kinds of test suites, including regression tests
9-
as run by the [jtreg harness](../jtreg/) and JCK
10-
tests. Characteristics of a good regression test will be provided.
11-
12-
<!--
13-
See also the <a href="https://openjdk.java.net/groups/quality/">OpenJDK
14-
Quality Group</a>.
15-
-->
16-
17-
::: {.NavBit}
18-
[« Previous](reviewBodies.html)[TOC](index.html)[Next »](jckAcquisition.html)
19-
:::
3+
The OpenJDK Developers' Guide has been merged into a single document. Please update your links to point to the new location: [OpenJDK Developers' Guide](index.html)

0 commit comments

Comments
 (0)