testing/nuts: NuttX Unit Test Selection (NUTS) application.

[linguini1]

Summary

Introduces a collection of unit tests for NuttX, built on top of the cmocka framework. Tests are organized into suites/groups which can be individually included/excluded from the build using Kconfig. Output is easily legible and separated with headers including the test group name.

Impact

Beginning of a more convenient NuttX testing method, with easily readable
console output for pass/fail results and extendable test collections.

Testing

Run on sim to obtain pass/fail results for the initial test cases:

nsh> nuts
[==========] struct circbuf_s: Running 4 test(s).
[ RUN      ] init_local
[       OK ] init_local
[ RUN      ] init_malloc
[       OK ] init_malloc
[ RUN      ] empty_local
[       OK ] empty_local
[ RUN      ] empty_malloc
[       OK ] empty_malloc
[==========] struct circbuf_s: 4 test(s) run.
[  PASSED  ] 4 test(s).
[==========] list_node: Running 5 test(s).
[ RUN      ] empty_after_init
[       OK ] empty_after_init
[ RUN      ] zerolen_after_init
[       OK ] zerolen_after_init
[ RUN      ] notail_after_init
[       OK ] notail_after_init
[ RUN      ] nohead_after_init
[       OK ] nohead_after_init
[ RUN      ] static_null_init
[       OK ] static_null_init
[==========] list_node: 5 test(s) run.
[  PASSED  ] 5 test(s).
[==========] /dev/console: Running 5 test(s).
[ RUN      ] open_rdonly
[       OK ] open_rdonly
[ RUN      ] open_rdwr
[       OK ] open_rdwr
[ RUN      ] open_wronly
[       OK ] open_wronly
[ RUN      ] readzero
[       OK ] readzero
[ RUN      ] writezero
[       OK ] writezero
[==========] /dev/console: 5 test(s) run.
[  PASSED  ] 5 test(s).
[==========] /dev/null: Running 8 test(s).
[ RUN      ] open_rdonly
[       OK ] open_rdonly
[ RUN      ] open_rdwr
[       OK ] open_rdwr
[ RUN      ] open_wronly
[       OK ] open_wronly
[ RUN      ] readzero
[       OK ] readzero
[ RUN      ] readlarge
[       OK ] readlarge
[ RUN      ] writezero
[       OK ] writezero
[ RUN      ] writelarge
[       OK ] writelarge
[ RUN      ] wrrd
[       OK ] wrrd
[==========] /dev/null: 8 test(s) run.
[  PASSED  ] 8 test(s).
[==========] /dev/zero: Running 8 test(s).
[ RUN      ] open_rdonly
[       OK ] open_rdonly
[ RUN      ] open_rdwr
[       OK ] open_rdwr
[ RUN      ] open_wronly
[       OK ] open_wronly
[ RUN      ] readzero
[       OK ] readzero
[ RUN      ] readlarge
[  ERROR   ] --- 64 != 0
[   LINE   ] --- devices/devzero.c:143: error: Failure!
[  FAILED  ] readlarge
[ RUN      ] writezero
[       OK ] writezero
[ RUN      ] writelarge
[       OK ] writelarge
[ RUN      ] wrrd
[       OK ] wrrd
[==========] /dev/zero: 8 test(s) run.
[  PASSED  ] 7 test(s).
[  FAILED  ] /dev/zero: 1 test(s), listed below:
[  FAILED  ] readlarge

 1 FAILED TEST(S)

[linguini1]

I don't like having all the conditional inclusion of test files in the Makefile/CMakeLists.txt. Does anyone know of a better way? The only other one I can think of is surrounding the contents of the C files with an #ifdef CONFIG_TESTING_NUTS_GROUP_TEST conditional, but that is messy.