| libc-test is developed as part of the musl project |
| http://www.musl-libc.org/ |
| |
| configuring: |
| cp config.mak.def config.mak |
| edit config.mak |
| build and run tests: |
| make |
| clean up: |
| make clean |
| |
| make builds all test binaries and runs them to create |
| a REPORT file that contains all build and runtime errors |
| (this means that make does not stop at build failures) |
| |
| contributing tests: |
| |
| design goals: |
| |
| - tests should be easy to run and build even a single test in isolation |
| (so test should be self contained if possible) |
| - failure of one test should not interfere with others |
| (build failure, crash or unexpected results are all failures) |
| - test output should point to the cause of the failure |
| - test results should be robust |
| - the test system should have minimal dependency |
| (libc, posix sh, gnu make) |
| - the test system should run on all archs and libcs |
| - tests should leave the system in a clean state |
| |
| conventions: |
| |
| each test is in a separate file at a path like src/directory/file.c with |
| its own main |
| |
| the test should return 0 on success and non-0 on failure, on failure it |
| should print error messages to standard out if possible, on success no |
| message should be printed |
| |
| to help with the above test protocol use t_error function for printing |
| errors and return t_status from main, see src/common/test.h |
| (t_error allows standard printf formatting, outputs at most 512bytes |
| in a single write call to fd 1, so there is no buffering, long outputs |
| are truncated, it sets the global t_status to 1) |
| |
| it is common to do many similar checks in a test, in such cases macros |
| may be used to simplify the code like |
| #define T1(a,b) (check(a,b) || (t_error("check(%s,%s) failed\n", a, b),0)) |
| #define T2(f,w) (result=(f), result==(w) || (t_error("%s failed: got %s, want %s\n", #f, result, w),0)) |
| |
| binaries should be possible to run from arbitrary directory. |
| the build system runs the tests using the src/common/run tool which |
| kills the test process after a timeout and reports the exit status |
| in case of failure |
| |
| directories: |
| |
| src/api: interface tests, build time include header tests |
| src/common: common utilities compiled into libtest.a |
| src/functional: functional tests aiming for large coverage of libc |
| src/math: tests for each math function with input-output test vectors |
| src/regression: regression tests aiming for testing particular bugs |
| |
| initial set of functional tests are derived from the libc-testsuit of |
| Rich Felker, regression tests should contain reference of the bug |
| (musl commit hash, glibc bug tracker url, etc) |
| |
| build system: |
| |
| the main non-file make targets are all, run, clean and cleanall. |
| (cleanall removes the reports as well, run reruns the dynamically |
| linked executables) |
| |
| for each directory under src there are targets like src/directory/all, |
| src/directory/run and src/directory/clean to make only the contents |
| of that directory, each directory has its own Makefile set up so it |
| invokes the top level make with src/directory/foo for the foo target, |
| so it is possible to work only under a specific test directory |
| |
| the build and runtime errors of each target are accumulated into a |
| target.err file and in the end they are concatenated into a REPORT |
| |
| each .c file in src/functional and src/regression are built into a |
| dynamic linked and a static linked executable test binary by default, |
| this behaviour can be changed by a similarly named .mk file changing |
| make variables and specifying additional rules: |
| |
| $(N) is the name of the binary target (the file name without the .c) |
| $(N)-static is the name of the static binary target |
| $(D) is the directory |
| $(N).CFLAGS are added to the CFLAGS at compilation |
| $(N).LDFLAGS are added to the LDFLAGS at linking |
| $(N).LDLIBS are added to the LDLIBS at linking |
| $(N).BINS are the targets (if empty no binaries are built) |
| $(N).LIBS are the non-executable targets (shared objects may use it) |
| |
| if a binary is linked together from several .o files then they |
| have to be specified as prerequisits for the binary targets and |
| added to the $(N).LDLIBS as well |
| |
| if a binary depends on a file at runtime (eg. a .so opened by dlopen) |
| then the $(N).err target should depend on that file |