Add Unity and CMock to NCS #519

nordic-krch · 2019-03-05T12:56:47Z

This PR contains 3 commits:

adding cmock repository to ncs (extending west configuration).
adding files (boilerplate, cmakelists, kconfig, etc) which enables adding unity tests.
adding example (tests/unity/example_test) to show how unit tests can be created.

Open items:

currently cmock repository is taken directly from https://github.com/ThrowTheSwitch. Should we fork it to nordic playground and take it from there?
currently cmock and unity CMakeLists.txt is created in nrf/tests/unity and it points to the sources in external repository (this way repository is unspoiled). Any better proposal?

Please note that this is my first PR to NCS. Let me know if i did something wrong.
Please note that i'm total newbie to cmake so be patient in case of rookie mistakes.

CLAassistant · 2019-03-05T12:56:54Z

All committers have signed the CLA.

NordicBuilder · 2019-03-05T12:59:56Z

All checks are passing now.

Review history of this comment for details about previous failed status.
Note that some checks might have not completed yet.

lemrey · 2019-03-08T08:37:08Z

@nordic-krch fantastic! Are there any requirements in terms of tooling for this, e.g. Ruby, Python modules?

nordic-krch · 2019-03-08T08:41:15Z

@lemrey, right. Ruby is of course required. In python no additional lib is needed.

nordic-krch · 2019-03-08T11:33:30Z

@SebastianBoe @tejlmand could you take a look as this is mainly cmake stuff and i'm newbie there.
fyi @gbakke

SebastianBoe · 2019-03-08T11:57:21Z

regex parsing of C?

:(

I'm concerned that we won't be able to regex parse the full language of CPP+CC, and that what we have here is just able to parse a subset of the language.

What happens when the parsing fails, can we create mocks manually for the cases that are not covered?

Sorry if this question doesn't make sense, I'm not familiar with mocking frameworks.

SebastianBoe · 2019-03-08T12:00:38Z

scripts/unity/func_name_list.py

Do you just need the func_names? Not the full prototype?

If so, maybe it would be possible to compile the header and then take the func names from the symbol table.

Then we don't need to regex parse unpreprocessed C code.

SebastianBoe · 2019-03-08T12:01:38Z

scripts/unity/func_name_list.py

This file is not guaranteed to be closed, use 'with' instead of manually invoking close().

nordic-krch · 2019-03-08T12:01:46Z

@SebastianBoe it is parsing header files mainly in search for function declarations and static inline functions. We need to handle following:

static inline functions cannot be mocked (because implementation is already provided in the header)
zephyr specific function declaration in case of syscall
wrapping around calls to allow mocking without removing mocked functions from the binary.

So in regexp we do not go beyond function declarations and we are focused on C since Cmock/Unity is C unit test framework.

SebastianBoe · 2019-03-08T12:03:14Z

scripts/unity/header_prepare.py

print() is best-practice

SebastianBoe · 2019-03-08T12:06:30Z

we are focused on C

Sorry, by CPP, I meant C preprocessor.

nordic-krch · 2019-03-14T12:01:18Z

@SebastianBoe i've applied your comments. Regarding compiling header file: I'm not sure how to do that but imo, it can be later on improved if we find issues with using python to parse the header.

What happens when the parsing fails, can we create mocks manually for the cases that are not covered?

In case regexp is missing something and we find case which fails we need to improve regexp.

SebastianBoe · 2019-03-14T12:35:31Z

In case regexp is missing something and we find case which fails we need to improve regexp.

What if it's impossible?

e.g. if the prototype is constructed by the preprocessor?

nordic-krch · 2019-03-14T12:43:45Z

@SebastianBoe, worst case function won't be mocked. Header processed by python next goes to cmock ruby script which processes it in search for function prototypes. It will create mocks only for found prototypes. Python script can only improve.

SebastianBoe · 2019-03-14T12:58:19Z

Okay, conceptually this makes sense then.

I can review the CMake code when the PR is in PR-form and no longer an RFC.

nordic-krch · 2019-03-14T13:01:00Z

i've put RFC since there were some open items mentioned in the PR but i see no opinions so far and those open items can be fixed later (like changing west to use unity fork instead original) so removing RFC.

SebastianBoe · 2019-03-15T11:44:30Z

@gbakke , @hakonfam : FYI

SebastianBoe · 2019-03-15T11:45:39Z

currently cmock repository is taken directly from https://github.com/ThrowTheSwitch. Should we fork it to nordic playground and take it from there?

I'd postpone this until necessary.

CMakeLists.txt

tests/unity/example_test/src/example_test.h

SebastianBoe

First pass of review done, was not able to get through all of it today.

Kconfig.nrf

scripts/unity/func_name_list.py

tests/unity/unity_test.cmake

scripts/unity/header_prepare.py

tests/unity/CMakeLists.txt

nordic-krch · 2019-03-22T11:32:25Z

@ru-fu please not that i've just updated documentation and removed last paragraph.

tejlmand

Found a minor thing in python scripts.
Zephyr is supporting python >= 3.4

scripts/unity/func_name_list.py

scripts/unity/header_prepare.py

ru-fu

I've added a commit with doc update. Please check and squash if okay.

tests/Kconfig

nordic-krch · 2019-03-25T06:34:41Z

@ru-fu thanks for review and changes. They look ok, i've squashed your commit.

nordic-krch · 2019-03-28T11:08:32Z

@tejlmand follow-up issue created. sample.yaml created. Can you re-check?

tejlmand

Looks good :)

nordic-krch · 2019-03-28T13:02:45Z

@carlescufi i think we can merge it. Cmake experts are done, as well as the documentation.

carlescufi · 2019-03-28T15:09:37Z

@nordic-krch very well, can you rebase on top of master?

nordic-krch · 2019-04-01T05:07:59Z

@carlescufi rebased.

carlescufi · 2019-04-01T09:54:58Z

@nordic-krch still shows conflicted, probably someone has merged stuff in the meantime.

Actions performed to add unity/cmock to ncs: - Add cmock (unity as submodule) repository to west list of projects - Add CMakeLists.txt file in nrf/tests/unity - Add Unity Kconfig file to nrf/tests/unity Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>

This commis adds tests/unity/unity_test.cmake file which includes standard zephyr boilerplate and on top of that adds functions for creating mocks and test runner. Additionally, two helper python scrips are added. Scripts helps to handle: mocking static inline functions, mocking using --wrap linker feature, skipping syscalls used in zephyr API's. Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>

Converted NRF_DIR to cache variable, in order to allow access to it from any location. Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>

Created unity library which contains functions for cmock and runner generation. Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>

Example shows how to: - mock a module (see CMakeLists.txt) - generate mock and test runner (see CMakeLists.txt) - use mock in the test (example_test.c) - override IS_ENABLED macro to runtime test compile time features (preincluded example_test.h) Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>

Added user guide which explains how to create and run unit test using Unity and CMock. Signed-off-by: Ruth Fuchss <ruth.fuchss@nordicsemi.no> Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>

nordic-krch · 2019-04-01T13:09:58Z

@carlescufi can you merge now?

metc · 2019-05-02T09:49:58Z

tests/unity/example_test/Kconfig

@@ -0,0 +1,17 @@
+# Kconfig - Config options for logger sample app


Nit: the comment is a bit off.

nordic-krch force-pushed the unity_cmock branch 2 times, most recently from cd397ef to fd404e3 Compare March 8, 2019 07:15

lemrey requested a review from carlescufi March 8, 2019 09:00

SebastianBoe reviewed Mar 8, 2019

View reviewed changes

scripts/unity/header_prepare.py Outdated

Copy link

Contributor

SebastianBoe Mar 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

print() is best-practice

nordic-krch force-pushed the unity_cmock branch 2 times, most recently from 8ab1436 to d4b088f Compare March 14, 2019 11:57

nordic-krch changed the title ~~[RFC] Add Unity and CMock to NCS~~ Add Unity and CMock to NCS Mar 14, 2019

SebastianBoe reviewed Mar 15, 2019

View reviewed changes

CMakeLists.txt Outdated Show resolved Hide resolved

hakonfam reviewed Mar 15, 2019

View reviewed changes

tests/unity/example_test/src/example_test.h Outdated Show resolved Hide resolved

SebastianBoe requested changes Mar 15, 2019

View reviewed changes

nordic-krch force-pushed the unity_cmock branch 3 times, most recently from 8046051 to ca2b63d Compare March 18, 2019 07:07

nordic-krch force-pushed the unity_cmock branch from 6de177d to 0f52678 Compare March 22, 2019 11:26

tejlmand reviewed Mar 22, 2019

View reviewed changes

scripts/unity/func_name_list.py Outdated Show resolved Hide resolved

scripts/unity/header_prepare.py Outdated Show resolved Hide resolved

ru-fu suggested changes Mar 22, 2019

View reviewed changes

tejlmand reviewed Mar 22, 2019

View reviewed changes

tests/Kconfig Outdated Show resolved Hide resolved

nordic-krch force-pushed the unity_cmock branch from e7deb12 to 5388a26 Compare March 25, 2019 06:34

ru-fu approved these changes Mar 25, 2019

View reviewed changes

nordic-krch force-pushed the unity_cmock branch 3 times, most recently from d7fcdd9 to 175a1bd Compare March 28, 2019 10:00

tejlmand approved these changes Mar 28, 2019

View reviewed changes

carlescufi approved these changes Mar 28, 2019

View reviewed changes

nordic-krch force-pushed the unity_cmock branch from 175a1bd to f8b07b5 Compare April 1, 2019 05:07

nordic-krch added 6 commits April 1, 2019 12:05

cmake: Make NRF_DIR cache variable

c00fce3

Converted NRF_DIR to cache variable, in order to allow access to it from any location. Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>

tests: unity: move content of unity_test.cmake to unity library

e1d613f

Created unity library which contains functions for cmock and runner generation. Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>

doc: Add user guide for unity testing with Unity and CMock

24371a5

Added user guide which explains how to create and run unit test using Unity and CMock. Signed-off-by: Ruth Fuchss <ruth.fuchss@nordicsemi.no> Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>

nordic-krch force-pushed the unity_cmock branch from f8b07b5 to 24371a5 Compare April 1, 2019 10:05

carlescufi merged commit 730cc40 into nrfconnect:master Apr 1, 2019

metc reviewed May 2, 2019

View reviewed changes

tests/unity/example_test/Kconfig

@@ -0,0 +1,17 @@

# Kconfig - Config options for logger sample app

Copy link

metc May 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: the comment is a bit off.

tejlmand mentioned this pull request Jan 5, 2022

scripts: Add ruby as requirement to all tools-versions files #6382

Merged

		@@ -0,0 +1,17 @@
		# Kconfig - Config options for logger sample app

Add Unity and CMock to NCS #519

Add Unity and CMock to NCS #519

Uh oh!

Conversation

nordic-krch commented Mar 5, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CLAassistant commented Mar 5, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NordicBuilder commented Mar 5, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lemrey commented Mar 8, 2019

Uh oh!

nordic-krch commented Mar 8, 2019

Uh oh!

nordic-krch commented Mar 8, 2019

Uh oh!

SebastianBoe commented Mar 8, 2019

Uh oh!

SebastianBoe Mar 8, 2019

Choose a reason for hiding this comment

Uh oh!

SebastianBoe Mar 8, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nordic-krch commented Mar 8, 2019

Uh oh!

SebastianBoe Mar 8, 2019

Choose a reason for hiding this comment

Uh oh!

SebastianBoe commented Mar 8, 2019

Uh oh!

nordic-krch commented Mar 14, 2019

Uh oh!

SebastianBoe commented Mar 14, 2019

Uh oh!

nordic-krch commented Mar 14, 2019

Uh oh!

SebastianBoe commented Mar 14, 2019

Uh oh!

nordic-krch commented Mar 14, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SebastianBoe commented Mar 15, 2019

Uh oh!

SebastianBoe commented Mar 15, 2019

Uh oh!

Uh oh!

Uh oh!

SebastianBoe left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nordic-krch commented Mar 22, 2019

Uh oh!

tejlmand left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ru-fu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nordic-krch commented Mar 25, 2019

Uh oh!

nordic-krch commented Mar 28, 2019

Uh oh!

nordic-krch commented Mar 5, 2019 •

edited

Loading

CLAassistant commented Mar 5, 2019 •

edited

Loading

NordicBuilder commented Mar 5, 2019 •

edited

Loading

SebastianBoe Mar 8, 2019 •

edited

Loading

nordic-krch commented Mar 14, 2019 •

edited

Loading