Skip to content
This repository has been archived by the owner on Nov 21, 2022. It is now read-only.

Commit

Permalink
Merge branch 'kunit' of git://git.kernel.org/pub/scm/linux/kernel/git…
Browse files Browse the repository at this point in the history
…/shuah/linux-kselftest.git

# Conflicts:
#	lib/kunit/string-stream.c
  • Loading branch information
sfrothwell committed Nov 18, 2022
2 parents 0c61f48 + 870f63b commit d5fd9c6
Show file tree
Hide file tree
Showing 10 changed files with 393 additions and 210 deletions.
115 changes: 58 additions & 57 deletions Documentation/dev-tools/kunit/architecture.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,17 @@
KUnit Architecture
==================

The KUnit architecture can be divided into two parts:
The KUnit architecture is divided into two parts:

- `In-Kernel Testing Framework`_
- `kunit_tool (Command Line Test Harness)`_
- `kunit_tool (Command-line Test Harness)`_

In-Kernel Testing Framework
===========================

The kernel testing library supports KUnit tests written in C using
KUnit. KUnit tests are kernel code. KUnit does several things:
KUnit. These KUnit tests are kernel code. KUnit performs the following
tasks:

- Organizes tests
- Reports test results
Expand All @@ -22,19 +23,17 @@ KUnit. KUnit tests are kernel code. KUnit does several things:
Test Cases
----------

The fundamental unit in KUnit is the test case. The KUnit test cases are
grouped into KUnit suites. A KUnit test case is a function with type
signature ``void (*)(struct kunit *test)``.
These test case functions are wrapped in a struct called
struct kunit_case.
The test case is the fundamental unit in KUnit. KUnit test cases are organised
into suites. A KUnit test case is a function with type signature
``void (*)(struct kunit *test)``. These test case functions are wrapped in a
struct called struct kunit_case.

.. note:
``generate_params`` is optional for non-parameterized tests.
Each KUnit test case gets a ``struct kunit`` context
object passed to it that tracks a running test. The KUnit assertion
macros and other KUnit utilities use the ``struct kunit`` context
object. As an exception, there are two fields:
Each KUnit test case receives a ``struct kunit`` context object that tracks a
running test. The KUnit assertion macros and other KUnit utilities use the
``struct kunit`` context object. As an exception, there are two fields:

- ``->priv``: The setup functions can use it to store arbitrary test
user data.
Expand Down Expand Up @@ -77,30 +76,31 @@ Executor

The KUnit executor can list and run built-in KUnit tests on boot.
The Test suites are stored in a linker section
called ``.kunit_test_suites``. For code, see:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/asm-generic/vmlinux.lds.h?h=v5.15#n945.
called ``.kunit_test_suites``. For the code, see ``KUNIT_TABLE()`` macro
definition in
`include/asm-generic/vmlinux.lds.h <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/asm-generic/vmlinux.lds.h?h=v6.0#n950>`_.
The linker section consists of an array of pointers to
``struct kunit_suite``, and is populated by the ``kunit_test_suites()``
macro. To run all tests compiled into the kernel, the KUnit executor
iterates over the linker section array.
macro. The KUnit executor iterates over the linker section array in order to
run all the tests that are compiled into the kernel.

.. kernel-figure:: kunit_suitememorydiagram.svg
:alt: KUnit Suite Memory

KUnit Suite Memory Diagram

On the kernel boot, the KUnit executor uses the start and end addresses
of this section to iterate over and run all tests. For code, see:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/executor.c

of this section to iterate over and run all tests. For the implementation of the
executor, see
`lib/kunit/executor.c <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/executor.c>`_.
When built as a module, the ``kunit_test_suites()`` macro defines a
``module_init()`` function, which runs all the tests in the compilation
unit instead of utilizing the executor.

In KUnit tests, some error classes do not affect other tests
or parts of the kernel, each KUnit case executes in a separate thread
context. For code, see:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/try-catch.c?h=v5.15#n58
context. See the ``kunit_try_catch_run()`` function in
`lib/kunit/try-catch.c <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/try-catch.c?h=v5.15#n58>`_.

Assertion Macros
----------------
Expand All @@ -111,37 +111,36 @@ All expectations/assertions are formatted as:

- ``{EXPECT|ASSERT}`` determines whether the check is an assertion or an
expectation.
In the event of a failure, the testing flow differs as follows:

- For an expectation, if the check fails, marks the test as failed
and logs the failure.
- For expectations, the test is marked as failed and the failure is logged.

- An assertion, on failure, causes the test case to terminate
immediately.
- Failing assertions, on the other hand, result in the test case being
terminated immediately.

- Assertions call function:
- Assertions call the function:
``void __noreturn kunit_abort(struct kunit *)``.

- ``kunit_abort`` calls function:
- ``kunit_abort`` calls the function:
``void __noreturn kunit_try_catch_throw(struct kunit_try_catch *try_catch)``.

- ``kunit_try_catch_throw`` calls function:
- ``kunit_try_catch_throw`` calls the function:
``void kthread_complete_and_exit(struct completion *, long) __noreturn;``
and terminates the special thread context.

- ``<op>`` denotes a check with options: ``TRUE`` (supplied property
has the boolean value true), ``EQ`` (two supplied properties are
has the boolean value "true"), ``EQ`` (two supplied properties are
equal), ``NOT_ERR_OR_NULL`` (supplied pointer is not null and does not
contain an err value).
contain an "err" value).

- ``[_MSG]`` prints a custom message on failure.

Test Result Reporting
---------------------
KUnit prints test results in KTAP format. KTAP is based on TAP14, see:
https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-specification.md.
KTAP (yet to be standardized format) works with KUnit and Kselftest.
The KUnit executor prints KTAP results to dmesg, and debugfs
(if configured).
KUnit prints the test results in KTAP format. KTAP is based on TAP14, see
Documentation/dev-tools/ktap.rst.
KTAP works with KUnit and Kselftest. The KUnit executor prints KTAP results to
dmesg, and debugfs (if configured).

Parameterized Tests
-------------------
Expand All @@ -150,42 +149,44 @@ Each KUnit parameterized test is associated with a collection of
parameters. The test is invoked multiple times, once for each parameter
value and the parameter is stored in the ``param_value`` field.
The test case includes a KUNIT_CASE_PARAM() macro that accepts a
generator function.
The generator function is passed the previous parameter and returns the next
parameter. It also provides a macro to generate common-case generators based on
arrays.
generator function. The generator function is passed the previous parameter
and returns the next parameter. It also includes a macro for generating
array-based common-case generators.

kunit_tool (Command Line Test Harness)
kunit_tool (Command-line Test Harness)
======================================

kunit_tool is a Python script ``(tools/testing/kunit/kunit.py)``
that can be used to configure, build, exec, parse and run (runs other
commands in order) test results. You can either run KUnit tests using
kunit_tool or can include KUnit in kernel and parse manually.
``kunit_tool`` is a Python script, found in ``tools/testing/kunit/kunit.py``. It
is used to configure, build, execute, parse test results and run all of the
previous commands in correct order (i.e., configure, build, execute and parse).
You have two options for running KUnit tests: either build the kernel with KUnit
enabled and manually parse the results (see
Documentation/dev-tools/kunit/run_manual.rst) or use ``kunit_tool``
(see Documentation/dev-tools/kunit/run_wrapper.rst).

- ``configure`` command generates the kernel ``.config`` from a
``.kunitconfig`` file (and any architecture-specific options).
For some architectures, additional config options are specified in the
``qemu_config`` Python script
(For example: ``tools/testing/kunit/qemu_configs/powerpc.py``).
The Python scripts available in ``qemu_configs`` folder
(for example, ``tools/testing/kunit/qemu configs/powerpc.py``) contains
additional configuration options for specific architectures.
It parses both the existing ``.config`` and the ``.kunitconfig`` files
and ensures that ``.config`` is a superset of ``.kunitconfig``.
If this is not the case, it will combine the two and run
``make olddefconfig`` to regenerate the ``.config`` file. It then
verifies that ``.config`` is now a superset. This checks if all
Kconfig dependencies are correctly specified in ``.kunitconfig``.
``kunit_config.py`` includes the parsing Kconfigs code. The code which
runs ``make olddefconfig`` is a part of ``kunit_kernel.py``. You can
invoke this command via: ``./tools/testing/kunit/kunit.py config`` and
to ensure that ``.config`` is a superset of ``.kunitconfig``.
If not, it will combine the two and run ``make olddefconfig`` to regenerate
the ``.config`` file. It then checks to see if ``.config`` has become a superset.
This verifies that all the Kconfig dependencies are correctly specified in the
file ``.kunitconfig``. The ``kunit_config.py`` script contains the code for parsing
Kconfigs. The code which runs ``make olddefconfig`` is part of the
``kunit_kernel.py`` script. You can invoke this command through:
``./tools/testing/kunit/kunit.py config`` and
generate a ``.config`` file.
- ``build`` runs ``make`` on the kernel tree with required options
(depends on the architecture and some options, for example: build_dir)
and reports any errors.
To build a KUnit kernel from the current ``.config``, you can use the
``build`` argument: ``./tools/testing/kunit/kunit.py build``.
- ``exec`` command executes kernel results either directly (using
User-mode Linux configuration), or via an emulator such
as QEMU. It reads results from the log via standard
User-mode Linux configuration), or through an emulator such
as QEMU. It reads results from the log using standard
output (stdout), and passes them to ``parse`` to be parsed.
If you already have built a kernel with built-in KUnit tests,
you can run the kernel and display the test results with the ``exec``
Expand Down
12 changes: 6 additions & 6 deletions drivers/gpu/drm/tests/drm_format_helper_test.c
Original file line number Diff line number Diff line change
Expand Up @@ -315,7 +315,7 @@ static void drm_test_fb_xrgb8888_to_gray8(struct kunit *test)
iosys_map_set_vaddr(&src, xrgb8888);

drm_fb_xrgb8888_to_gray8(&dst, &result->dst_pitch, &src, &fb, &params->clip);
KUNIT_EXPECT_EQ(test, memcmp(buf, result->expected, dst_size), 0);
KUNIT_EXPECT_MEMEQ(test, buf, result->expected, dst_size);
}

static void drm_test_fb_xrgb8888_to_rgb332(struct kunit *test)
Expand Down Expand Up @@ -345,7 +345,7 @@ static void drm_test_fb_xrgb8888_to_rgb332(struct kunit *test)
iosys_map_set_vaddr(&src, xrgb8888);

drm_fb_xrgb8888_to_rgb332(&dst, &result->dst_pitch, &src, &fb, &params->clip);
KUNIT_EXPECT_EQ(test, memcmp(buf, result->expected, dst_size), 0);
KUNIT_EXPECT_MEMEQ(test, buf, result->expected, dst_size);
}

static void drm_test_fb_xrgb8888_to_rgb565(struct kunit *test)
Expand Down Expand Up @@ -375,10 +375,10 @@ static void drm_test_fb_xrgb8888_to_rgb565(struct kunit *test)
iosys_map_set_vaddr(&src, xrgb8888);

drm_fb_xrgb8888_to_rgb565(&dst, &result->dst_pitch, &src, &fb, &params->clip, false);
KUNIT_EXPECT_EQ(test, memcmp(buf, result->expected, dst_size), 0);
KUNIT_EXPECT_MEMEQ(test, buf, result->expected, dst_size);

drm_fb_xrgb8888_to_rgb565(&dst, &result->dst_pitch, &src, &fb, &params->clip, true);
KUNIT_EXPECT_EQ(test, memcmp(buf, result->expected_swab, dst_size), 0);
KUNIT_EXPECT_MEMEQ(test, buf, result->expected_swab, dst_size);
}

static void drm_test_fb_xrgb8888_to_rgb888(struct kunit *test)
Expand Down Expand Up @@ -408,7 +408,7 @@ static void drm_test_fb_xrgb8888_to_rgb888(struct kunit *test)
iosys_map_set_vaddr(&src, xrgb8888);

drm_fb_xrgb8888_to_rgb888(&dst, &result->dst_pitch, &src, &fb, &params->clip);
KUNIT_EXPECT_EQ(test, memcmp(buf, result->expected, dst_size), 0);
KUNIT_EXPECT_MEMEQ(test, buf, result->expected, dst_size);
}

static void drm_test_fb_xrgb8888_to_xrgb2101010(struct kunit *test)
Expand Down Expand Up @@ -439,7 +439,7 @@ static void drm_test_fb_xrgb8888_to_xrgb2101010(struct kunit *test)

drm_fb_xrgb8888_to_xrgb2101010(&dst, &result->dst_pitch, &src, &fb, &params->clip);
buf = le32buf_to_cpu(test, buf, dst_size / sizeof(u32));
KUNIT_EXPECT_EQ(test, memcmp(buf, result->expected, dst_size), 0);
KUNIT_EXPECT_MEMEQ(test, buf, result->expected, dst_size);
}

static struct kunit_case drm_format_helper_test_cases[] = {
Expand Down
81 changes: 33 additions & 48 deletions include/kunit/assert.h
Original file line number Diff line number Diff line change
Expand Up @@ -90,19 +90,6 @@ void kunit_unary_assert_format(const struct kunit_assert *assert,
const struct va_format *message,
struct string_stream *stream);

/**
* KUNIT_INIT_UNARY_ASSERT_STRUCT() - Initializes &struct kunit_unary_assert.
* @cond: A string representation of the expression asserted true or false.
* @expect_true: True if of type KUNIT_{EXPECT|ASSERT}_TRUE, false otherwise.
*
* Initializes a &struct kunit_unary_assert. Intended to be used in
* KUNIT_EXPECT_* and KUNIT_ASSERT_* macros.
*/
#define KUNIT_INIT_UNARY_ASSERT_STRUCT(cond, expect_true) { \
.condition = cond, \
.expected_true = expect_true \
}

/**
* struct kunit_ptr_not_err_assert - An expectation/assertion that a pointer is
* not NULL and not a -errno.
Expand All @@ -123,20 +110,6 @@ void kunit_ptr_not_err_assert_format(const struct kunit_assert *assert,
const struct va_format *message,
struct string_stream *stream);

/**
* KUNIT_INIT_PTR_NOT_ERR_ASSERT_STRUCT() - Initializes a
* &struct kunit_ptr_not_err_assert.
* @txt: A string representation of the expression passed to the expectation.
* @val: The actual evaluated pointer value of the expression.
*
* Initializes a &struct kunit_ptr_not_err_assert. Intended to be used in
* KUNIT_EXPECT_* and KUNIT_ASSERT_* macros.
*/
#define KUNIT_INIT_PTR_NOT_ERR_STRUCT(txt, val) { \
.text = txt, \
.value = val \
}

/**
* struct kunit_binary_assert_text - holds strings for &struct
* kunit_binary_assert and friends to try and make the structs smaller.
Expand Down Expand Up @@ -173,27 +146,6 @@ void kunit_binary_assert_format(const struct kunit_assert *assert,
const struct va_format *message,
struct string_stream *stream);

/**
* KUNIT_INIT_BINARY_ASSERT_STRUCT() - Initializes a binary assert like
* kunit_binary_assert, kunit_binary_ptr_assert, etc.
*
* @text_: Pointer to a kunit_binary_assert_text.
* @left_val: The actual evaluated value of the expression in the left slot.
* @right_val: The actual evaluated value of the expression in the right slot.
*
* Initializes a binary assert like kunit_binary_assert,
* kunit_binary_ptr_assert, etc. This relies on these structs having the same
* fields but with different types for left_val/right_val.
* This is ultimately used by binary assertion macros like KUNIT_EXPECT_EQ, etc.
*/
#define KUNIT_INIT_BINARY_ASSERT_STRUCT(text_, \
left_val, \
right_val) { \
.text = text_, \
.left_value = left_val, \
.right_value = right_val \
}

/**
* struct kunit_binary_ptr_assert - An expectation/assertion that compares two
* pointer values (for example, KUNIT_EXPECT_PTR_EQ(test, foo, bar)).
Expand Down Expand Up @@ -240,4 +192,37 @@ void kunit_binary_str_assert_format(const struct kunit_assert *assert,
const struct va_format *message,
struct string_stream *stream);

#define KUNIT_INIT_MEM_ASSERT_STRUCT(text_, left_val, right_val, size_) { \
.text = text_, \
.left_value = left_val, \
.right_value = right_val, \
.size = size_ \
}

/**
* struct kunit_mem_assert - An expectation/assertion that compares two
* memory blocks.
* @assert: The parent of this type.
* @text: Holds the textual representations of the operands and comparator.
* @left_value: The actual evaluated value of the expression in the left slot.
* @right_value: The actual evaluated value of the expression in the right slot.
* @size: Size of the memory block analysed in bytes.
*
* Represents an expectation/assertion that compares two memory blocks. For
* example, to expect that the first three bytes of foo is equal to the
* first three bytes of bar, you can use the expectation
* KUNIT_EXPECT_MEMEQ(test, foo, bar, 3);
*/
struct kunit_mem_assert {
struct kunit_assert assert;
const struct kunit_binary_assert_text *text;
const void *left_value;
const void *right_value;
const size_t size;
};

void kunit_mem_assert_format(const struct kunit_assert *assert,
const struct va_format *message,
struct string_stream *stream);

#endif /* _KUNIT_ASSERT_H */
Loading

0 comments on commit d5fd9c6

Please sign in to comment.