1# Distributed under the OSI-approved BSD 3-Clause License.  See accompanying
2# file Copyright.txt or https://cmake.org/licensing for details.
3
4#[=======================================================================[.rst:
5GoogleTest
6----------
7
8This module defines functions to help use the Google Test infrastructure.  Two
9mechanisms for adding tests are provided. :command:`gtest_add_tests` has been
10around for some time, originally via ``find_package(GTest)``.
11:command:`gtest_discover_tests` was introduced in CMake 3.10.
12
13The (older) :command:`gtest_add_tests` scans source files to identify tests.
14This is usually effective, with some caveats, including in cross-compiling
15environments, and makes setting additional properties on tests more convenient.
16However, its handling of parameterized tests is less comprehensive, and it
17requires re-running CMake to detect changes to the list of tests.
18
19The (newer) :command:`gtest_discover_tests` discovers tests by asking the
20compiled test executable to enumerate its tests.  This is more robust and
21provides better handling of parameterized tests, and does not require CMake
22to be re-run when tests change.  However, it may not work in a cross-compiling
23environment, and setting test properties is less convenient.
24
25More details can be found in the documentation of the respective functions.
26
27Both commands are intended to replace use of :command:`add_test` to register
28tests, and will create a separate CTest test for each Google Test test case.
29Note that this is in some cases less efficient, as common set-up and tear-down
30logic cannot be shared by multiple test cases executing in the same instance.
31However, it provides more fine-grained pass/fail information to CTest, which is
32usually considered as more beneficial.  By default, the CTest test name is the
33same as the Google Test name (i.e. ``suite.testcase``); see also
34``TEST_PREFIX`` and ``TEST_SUFFIX``.
35
36.. command:: gtest_add_tests
37
38  Automatically add tests with CTest by scanning source code for Google Test
39  macros::
40
41    gtest_add_tests(TARGET target
42                    [SOURCES src1...]
43                    [EXTRA_ARGS arg1...]
44                    [WORKING_DIRECTORY dir]
45                    [TEST_PREFIX prefix]
46                    [TEST_SUFFIX suffix]
47                    [SKIP_DEPENDENCY]
48                    [TEST_LIST outVar]
49    )
50
51  ``gtest_add_tests`` attempts to identify tests by scanning source files.
52  Although this is generally effective, it uses only a basic regular expression
53  match, which can be defeated by atypical test declarations, and is unable to
54  fully "split" parameterized tests.  Additionally, it requires that CMake be
55  re-run to discover any newly added, removed or renamed tests (by default,
56  this means that CMake is re-run when any test source file is changed, but see
57  ``SKIP_DEPENDENCY``).  However, it has the advantage of declaring tests at
58  CMake time, which somewhat simplifies setting additional properties on tests,
59  and always works in a cross-compiling environment.
60
61  The options are:
62
63  ``TARGET target``
64    Specifies the Google Test executable, which must be a known CMake
65    executable target.  CMake will substitute the location of the built
66    executable when running the test.
67
68  ``SOURCES src1...``
69    When provided, only the listed files will be scanned for test cases.  If
70    this option is not given, the :prop_tgt:`SOURCES` property of the
71    specified ``target`` will be used to obtain the list of sources.
72
73  ``EXTRA_ARGS arg1...``
74    Any extra arguments to pass on the command line to each test case.
75
76  ``WORKING_DIRECTORY dir``
77    Specifies the directory in which to run the discovered test cases.  If this
78    option is not provided, the current binary directory is used.
79
80  ``TEST_PREFIX prefix``
81    Specifies a ``prefix`` to be prepended to the name of each discovered test
82    case.  This can be useful when the same source files are being used in
83    multiple calls to ``gtest_add_test()`` but with different ``EXTRA_ARGS``.
84
85  ``TEST_SUFFIX suffix``
86    Similar to ``TEST_PREFIX`` except the ``suffix`` is appended to the name of
87    every discovered test case.  Both ``TEST_PREFIX`` and ``TEST_SUFFIX`` may
88    be specified.
89
90  ``SKIP_DEPENDENCY``
91    Normally, the function creates a dependency which will cause CMake to be
92    re-run if any of the sources being scanned are changed.  This is to ensure
93    that the list of discovered tests is updated.  If this behavior is not
94    desired (as may be the case while actually writing the test cases), this
95    option can be used to prevent the dependency from being added.
96
97  ``TEST_LIST outVar``
98    The variable named by ``outVar`` will be populated in the calling scope
99    with the list of discovered test cases.  This allows the caller to do
100    things like manipulate test properties of the discovered tests.
101
102  .. code-block:: cmake
103
104    include(GoogleTest)
105    add_executable(FooTest FooUnitTest.cxx)
106    gtest_add_tests(TARGET      FooTest
107                    TEST_SUFFIX .noArgs
108                    TEST_LIST   noArgsTests
109    )
110    gtest_add_tests(TARGET      FooTest
111                    EXTRA_ARGS  --someArg someValue
112                    TEST_SUFFIX .withArgs
113                    TEST_LIST   withArgsTests
114    )
115    set_tests_properties(${noArgsTests}   PROPERTIES TIMEOUT 10)
116    set_tests_properties(${withArgsTests} PROPERTIES TIMEOUT 20)
117
118  For backward compatibility, the following form is also supported::
119
120    gtest_add_tests(exe args files...)
121
122  ``exe``
123    The path to the test executable or the name of a CMake target.
124  ``args``
125    A ;-list of extra arguments to be passed to executable.  The entire
126    list must be passed as a single argument.  Enclose it in quotes,
127    or pass ``""`` for no arguments.
128  ``files...``
129    A list of source files to search for tests and test fixtures.
130    Alternatively, use ``AUTO`` to specify that ``exe`` is the name
131    of a CMake executable target whose sources should be scanned.
132
133  .. code-block:: cmake
134
135    include(GoogleTest)
136    set(FooTestArgs --foo 1 --bar 2)
137    add_executable(FooTest FooUnitTest.cxx)
138    gtest_add_tests(FooTest "${FooTestArgs}" AUTO)
139
140.. command:: gtest_discover_tests
141
142  Automatically add tests with CTest by querying the compiled test executable
143  for available tests::
144
145    gtest_discover_tests(target
146                         [EXTRA_ARGS arg1...]
147                         [WORKING_DIRECTORY dir]
148                         [TEST_PREFIX prefix]
149                         [TEST_SUFFIX suffix]
150                         [NO_PRETTY_TYPES] [NO_PRETTY_VALUES]
151                         [ONE_CTEST_PER_SUITE]
152                         [PROPERTIES name1 value1...]
153                         [TEST_LIST var]
154                         [DISCOVERY_TIMEOUT seconds]
155    )
156
157  ``gtest_discover_tests`` sets up a post-build command on the test executable
158  that generates the list of tests by parsing the output from running the test
159  with the ``--gtest_list_tests`` argument.  Compared to the source parsing
160  approach of :command:`gtest_add_tests`, this ensures that the full list of
161  tests, including instantiations of parameterized tests, is obtained.  Since
162  test discovery occurs at build time, it is not necessary to re-run CMake when
163  the list of tests changes.
164  However, it requires that :prop_tgt:`CROSSCOMPILING_EMULATOR` is properly set
165  in order to function in a cross-compiling environment.
166
167  Additionally, setting properties on tests is somewhat less convenient, since
168  the tests are not available at CMake time.  Additional test properties may be
169  assigned to the set of tests as a whole using the ``PROPERTIES`` option.  If
170  more fine-grained test control is needed, custom content may be provided
171  through an external CTest script using the :prop_dir:`TEST_INCLUDE_FILES`
172  directory property.  The set of discovered tests is made accessible to such a
173  script via the ``<target>_TESTS`` variable.
174
175  The options are:
176
177  ``target``
178    Specifies the Google Test executable, which must be a known CMake
179    executable target.  CMake will substitute the location of the built
180    executable when running the test.
181
182  ``EXTRA_ARGS arg1...``
183    Any extra arguments to pass on the command line to each test case.
184
185  ``WORKING_DIRECTORY dir``
186    Specifies the directory in which to run the discovered test cases.  If this
187    option is not provided, the current binary directory is used.
188
189  ``TEST_PREFIX prefix``
190    Specifies a ``prefix`` to be prepended to the name of each discovered test
191    case.  This can be useful when the same test executable is being used in
192    multiple calls to ``gtest_discover_tests()`` but with different
193    ``EXTRA_ARGS``.
194
195  ``TEST_SUFFIX suffix``
196    Similar to ``TEST_PREFIX`` except the ``suffix`` is appended to the name of
197    every discovered test case.  Both ``TEST_PREFIX`` and ``TEST_SUFFIX`` may
198    be specified.
199
200  ``NO_PRETTY_TYPES``
201    By default, the type index of type-parameterized tests is replaced by the
202    actual type name in the CTest test name.  If this behavior is undesirable
203    (e.g. because the type names are unwieldy), this option will suppress this
204    behavior.
205
206  ``NO_PRETTY_VALUES``
207    By default, the value index of value-parameterized tests is replaced by the
208    actual value in the CTest test name.  If this behavior is undesirable
209    (e.g. because the value strings are unwieldy), this option will suppress
210    this behavior.
211
212  ``ONE_CTEST_PER_SUITE``
213    By default, one CTest will be created for every GoogleTest case. If this
214    behavour is undesirable (e.g. because there is a large setup / teardown
215    cost), this option will instead create one CTest for each GoogleTest
216    _suite_.
217
218  ``PROPERTIES name1 value1...``
219    Specifies additional properties to be set on all tests discovered by this
220    invocation of ``gtest_discover_tests``.
221
222  ``TEST_LIST var``
223    Make the list of tests available in the variable ``var``, rather than the
224    default ``<target>_TESTS``.  This can be useful when the same test
225    executable is being used in multiple calls to ``gtest_discover_tests()``.
226    Note that this variable is only available in CTest.
227
228  ``DISCOVERY_TIMEOUT num``
229    Specifies how long (in seconds) CMake will wait for the test to enumerate
230    available tests.  If the test takes longer than this, discovery (and your
231    build) will fail.  Most test executables will enumerate their tests very
232    quickly, but under some exceptional circumstances, a test may require a
233    longer timeout.  The default is 5.  See also the ``TIMEOUT`` option of
234    :command:`execute_process`.
235
236    .. note::
237
238      In CMake versions 3.10.1 and 3.10.2, this option was called ``TIMEOUT``.
239      This clashed with the ``TIMEOUT`` test property, which is one of the
240      common properties that would be set with the ``PROPERTIES`` keyword,
241      usually leading to legal but unintended behavior.  The keyword was
242      changed to ``DISCOVERY_TIMEOUT`` in CMake 3.10.3 to address this
243      problem.  The ambiguous behavior of the ``TIMEOUT`` keyword in 3.10.1
244      and 3.10.2 has not been preserved.
245
246#]=======================================================================]
247
248#------------------------------------------------------------------------------
249function(gtest_add_tests)
250
251  if (ARGC LESS 1)
252    message(FATAL_ERROR "No arguments supplied to gtest_add_tests()")
253  endif()
254
255  set(options
256      SKIP_DEPENDENCY
257  )
258  set(oneValueArgs
259      TARGET
260      WORKING_DIRECTORY
261      TEST_PREFIX
262      TEST_SUFFIX
263      TEST_LIST
264  )
265  set(multiValueArgs
266      SOURCES
267      EXTRA_ARGS
268  )
269  set(allKeywords ${options} ${oneValueArgs} ${multiValueArgs})
270
271  unset(sources)
272  if("${ARGV0}" IN_LIST allKeywords)
273    cmake_parse_arguments(ARGS "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})
274    set(autoAddSources YES)
275  else()
276    # Non-keyword syntax, convert to keyword form
277    if (ARGC LESS 3)
278      message(FATAL_ERROR "gtest_add_tests() without keyword options requires at least 3 arguments")
279    endif()
280    set(ARGS_TARGET     "${ARGV0}")
281    set(ARGS_EXTRA_ARGS "${ARGV1}")
282    if(NOT "${ARGV2}" STREQUAL "AUTO")
283      set(ARGS_SOURCES "${ARGV}")
284      list(REMOVE_AT ARGS_SOURCES 0 1)
285    endif()
286  endif()
287
288  # The non-keyword syntax allows the first argument to be an arbitrary
289  # executable rather than a target if source files are also provided. In all
290  # other cases, both forms require a target.
291  if(NOT TARGET "${ARGS_TARGET}" AND NOT ARGS_SOURCES)
292    message(FATAL_ERROR "${ARGS_TARGET} does not define an existing CMake target")
293  endif()
294  if(NOT ARGS_WORKING_DIRECTORY)
295    unset(workDir)
296  else()
297    set(workDir WORKING_DIRECTORY "${ARGS_WORKING_DIRECTORY}")
298  endif()
299
300  if(NOT ARGS_SOURCES)
301    get_property(ARGS_SOURCES TARGET ${ARGS_TARGET} PROPERTY SOURCES)
302  endif()
303
304  unset(testList)
305
306  set(gtest_case_name_regex ".*\\( *([A-Za-z_0-9]+) *, *([A-Za-z_0-9]+) *\\).*")
307  set(gtest_test_type_regex "(TYPED_TEST|TEST_?[FP]?)")
308
309  foreach(source IN LISTS ARGS_SOURCES)
310    if(NOT ARGS_SKIP_DEPENDENCY)
311      set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS ${source})
312    endif()
313    file(READ "${source}" contents)
314    string(REGEX MATCHALL "${gtest_test_type_regex} *\\(([A-Za-z_0-9 ,]+)\\)" found_tests ${contents})
315    foreach(hit ${found_tests})
316      string(REGEX MATCH "${gtest_test_type_regex}" test_type ${hit})
317
318      # Parameterized tests have a different signature for the filter
319      if("x${test_type}" STREQUAL "xTEST_P")
320        string(REGEX REPLACE ${gtest_case_name_regex}  "*/\\1.\\2/*" gtest_test_name ${hit})
321      elseif("x${test_type}" STREQUAL "xTEST_F" OR "x${test_type}" STREQUAL "xTEST")
322        string(REGEX REPLACE ${gtest_case_name_regex} "\\1.\\2" gtest_test_name ${hit})
323      elseif("x${test_type}" STREQUAL "xTYPED_TEST")
324        string(REGEX REPLACE ${gtest_case_name_regex} "\\1/*.\\2" gtest_test_name ${hit})
325      else()
326        message(WARNING "Could not parse GTest ${hit} for adding to CTest.")
327        continue()
328      endif()
329
330      # Make sure tests disabled in GTest get disabled in CTest
331      if(gtest_test_name MATCHES "(^|\\.)DISABLED_")
332        # Add the disabled test if CMake is new enough
333        # Note that this check is to allow backwards compatibility so this
334        # module can be copied locally in projects to use with older CMake
335        # versions
336        if(CMAKE_VERSION VERSION_GREATER_EQUAL 3.8.20170401)
337          string(REGEX REPLACE
338                 "(^|\\.)DISABLED_" "\\1"
339                 orig_test_name "${gtest_test_name}"
340          )
341          set(ctest_test_name
342              ${ARGS_TEST_PREFIX}${orig_test_name}${ARGS_TEST_SUFFIX}
343          )
344          add_test(NAME ${ctest_test_name}
345                   ${workDir}
346                   COMMAND ${ARGS_TARGET}
347                     --gtest_also_run_disabled_tests
348                     --gtest_filter=${gtest_test_name}
349                     ${ARGS_EXTRA_ARGS}
350          )
351          set_tests_properties(${ctest_test_name} PROPERTIES DISABLED TRUE)
352          list(APPEND testList ${ctest_test_name})
353        endif()
354      else()
355        set(ctest_test_name ${ARGS_TEST_PREFIX}${gtest_test_name}${ARGS_TEST_SUFFIX})
356        add_test(NAME ${ctest_test_name}
357                 ${workDir}
358                 COMMAND ${ARGS_TARGET}
359                   --gtest_filter=${gtest_test_name}
360                   ${ARGS_EXTRA_ARGS}
361        )
362        list(APPEND testList ${ctest_test_name})
363      endif()
364    endforeach()
365  endforeach()
366
367  if(ARGS_TEST_LIST)
368    set(${ARGS_TEST_LIST} ${testList} PARENT_SCOPE)
369  endif()
370
371endfunction()
372
373#------------------------------------------------------------------------------
374function(gtest_discover_tests TARGET)
375  cmake_parse_arguments(
376    ""
377    "NO_PRETTY_TYPES;NO_PRETTY_VALUES;ONE_CTEST_PER_SUITE"
378    "TEST_PREFIX;TEST_SUFFIX;WORKING_DIRECTORY;TEST_LIST;DISCOVERY_TIMEOUT"
379    "EXTRA_ARGS;PROPERTIES"
380    ${ARGN}
381  )
382
383  if(NOT _WORKING_DIRECTORY)
384    set(_WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}")
385  endif()
386  if(NOT _TEST_LIST)
387    set(_TEST_LIST ${TARGET}_TESTS)
388  endif()
389  if(NOT _DISCOVERY_TIMEOUT)
390    set(_DISCOVERY_TIMEOUT 5)
391  endif()
392
393  get_property(
394    has_counter
395    TARGET ${TARGET}
396    PROPERTY CTEST_DISCOVERED_TEST_COUNTER
397    SET
398  )
399  if(has_counter)
400    get_property(
401      counter
402      TARGET ${TARGET}
403      PROPERTY CTEST_DISCOVERED_TEST_COUNTER
404    )
405    math(EXPR counter "${counter} + 1")
406  else()
407    set(counter 1)
408  endif()
409  set_property(
410    TARGET ${TARGET}
411    PROPERTY CTEST_DISCOVERED_TEST_COUNTER
412    ${counter}
413  )
414
415  # Define rule to generate test list for aforementioned test executable
416  set(ctest_file_base "${CMAKE_CURRENT_BINARY_DIR}/${TARGET}[${counter}]")
417  set(ctest_include_file "${ctest_file_base}_include.cmake")
418  set(ctest_tests_file "${ctest_file_base}_tests.cmake")
419  get_property(crosscompiling_emulator
420    TARGET ${TARGET}
421    PROPERTY CROSSCOMPILING_EMULATOR
422  )
423  add_custom_command(
424    TARGET ${TARGET} POST_BUILD
425    BYPRODUCTS "${ctest_tests_file}"
426    COMMAND "${CMAKE_COMMAND}"
427            -D "TEST_TARGET=${TARGET}"
428            -D "TEST_EXECUTABLE=$<TARGET_FILE:${TARGET}>"
429            -D "TEST_EXECUTOR=${crosscompiling_emulator}"
430            -D "TEST_WORKING_DIR=${_WORKING_DIRECTORY}"
431            -D "TEST_EXTRA_ARGS=${_EXTRA_ARGS}"
432            -D "TEST_PROPERTIES=${_PROPERTIES}"
433            -D "TEST_PREFIX=${_TEST_PREFIX}"
434            -D "TEST_SUFFIX=${_TEST_SUFFIX}"
435            -D "NO_PRETTY_TYPES=${_NO_PRETTY_TYPES}"
436            -D "NO_PRETTY_VALUES=${_NO_PRETTY_VALUES}"
437            -D "TEST_LIST=${_TEST_LIST}"
438            -D "CTEST_FILE=${ctest_tests_file}"
439            -D "TEST_DISCOVERY_TIMEOUT=${_DISCOVERY_TIMEOUT}"
440            -D "ONE_CTEST_PER_SUITE=${_ONE_CTEST_PER_SUITE}"
441            -P "${_GOOGLETEST_DISCOVER_TESTS_SCRIPT}"
442    VERBATIM
443  )
444
445  file(WRITE "${ctest_include_file}"
446    "if(EXISTS \"${ctest_tests_file}\")\n"
447    "  include(\"${ctest_tests_file}\")\n"
448    "else()\n"
449    "  add_test(${TARGET}_NOT_BUILT ${TARGET}_NOT_BUILT)\n"
450    "endif()\n"
451  )
452
453  # Add discovered tests to directory TEST_INCLUDE_FILES
454  set_property(DIRECTORY
455    APPEND PROPERTY TEST_INCLUDE_FILES "${ctest_include_file}"
456  )
457
458  # Backport to cmake 3.9 and earlier - TEST_INCLUDE_FILES property
459  # was added in cmake 3.10; prior to that we only have the singular
460  # TEST_INCLUDE_FILE property, which only allows one file to be included
461  # per directory. Therefore if running with cmake before 3.10, add a final
462  # step where we manually combine all elements in TEST_INCLUDE_FILES into a
463  # single unified file, and use TEST_INCLUDE_FILE to include that.
464  if (CMAKE_VERSION VERSION_LESS 3.10)
465    set(ctest_unified_include_file "${ctest_file_base}_include_unified.cmake")
466    get_property(_test_include_file_VALUE DIRECTORY PROPERTY TEST_INCLUDE_FILES)
467    foreach (file IN LISTS _test_include_file_VALUE)
468      file(READ ${file} contents)
469      set(unified ${unified}${contents})
470    endforeach ()
471    file(WRITE "${ctest_unified_include_file}" "${unified}")
472    set_property(DIRECTORY
473      PROPERTY TEST_INCLUDE_FILE "${ctest_unified_include_file}"
474    )
475  endif()
476
477endfunction()
478
479###############################################################################
480
481set(_GOOGLETEST_DISCOVER_TESTS_SCRIPT
482  ${CMAKE_CURRENT_LIST_DIR}/CouchbaseGoogleTestAddTests.cmake
483)
484