@@ -315,23 +315,23 @@ can be identified via variables ``INPUT_NML``, ``NEMS_CONFIGURE`` and ``FV3_RUN`
315
315
for example, by trying ``grep -n INPUT_NML * `` inside the tests/ and tests/tests/
316
316
directories.
317
317
318
- .. _ UsingUnitTest :
318
+ .. _ UsingOpnReqTest :
319
319
320
- --------------------------
321
- Using the unit test script
322
- --------------------------
323
- The unit test script ``utest `` in the tests/ directory can also be used to run
324
- tests. Given the name of a test, ``utest `` carries out a suite of test cases.
320
+ ---------------------------------------------
321
+ Using the operational requirement test script
322
+ ---------------------------------------------
323
+ The operational requirement test script ``opnReqTest `` in the tests/ directory can also be used to run
324
+ tests. Given the name of a test, ``opnReqTest `` carries out a suite of test cases.
325
325
Each test case addresses an aspect of the requirements new implementations
326
- should satisfy, which are shown in :numref: `Table %s <ImplementationRequirement >`.
327
- For the following discussions on utest , the user should note the distinction between
326
+ should satisfy, which are shown in :numref: `Table %s <OperationalRequirement >`.
327
+ For the following discussions on opnReqTest , the user should note the distinction between
328
328
'test name' and 'test case': examples of test name are ``control ``, ``cpld_control ``
329
329
and ``regional_control `` which are all found in the /tests/tests/ directory, whereas
330
330
test case refers to any one of ``thr ``, ``mpi ``, ``dcp ``, ``rst ``, ``bit `` and ``dbg ``.
331
331
332
- .. _ ImplementationRequirement :
332
+ .. _ OperationalRequirement :
333
333
334
- .. table :: *Implementation requirements*
334
+ .. table :: *Operational requirements*
335
335
336
336
+----------+------------------------------------------------------------------------+
337
337
| **Case ** | **Description ** |
@@ -349,47 +349,40 @@ test case refers to any one of ``thr``, ``mpi``, ``dcp``, ``rst``, ``bit`` and `
349
349
| dbg | Model can be compiled and run to completion in debug mode |
350
350
+----------+------------------------------------------------------------------------+
351
351
352
- The unit test uses the same testing framework used by the regression
352
+ The operational requirement test uses the same testing framework used by the regression
353
353
test, and therefore it is recommened that the user first read
354
354
:numref: `Section %s <UsingRegressionTest >`. All the files in
355
355
the subdirectories shown in :numref: `Table %s <RTSubDirs >` are relavant to the
356
- unit test except that the ``utest `` script replaces ``rt.sh `` and the
357
- `` utest.bld `` file replaces `` rt.conf ``. The /tests/utests / directory contains
358
- utest -specific lower-level scripts used to set up run configurations.
356
+ operational requirement test except that the ``opnReqTest `` script replaces ``rt.sh ``.
357
+ The /tests/opnReqTests / directory contains
358
+ opnReqTest -specific lower-level scripts used to set up run configurations.
359
359
360
360
On `Tier-1 platforms <https://github.com/ufs-community/ ufs-weather-model/wiki
361
361
/Regression-Test-Policy-for-Weather-Model-Platforms-and-Compilers> `_, tests can
362
- be run by first modifying the PSV file ``utest.bld `` to specify the build options
363
- and then invoking
362
+ be run by invoking
364
363
365
364
.. code-block :: console
366
365
367
- ./utest -n <test-name>
366
+ ./opnReqTest -n <test-name>
368
367
369
- For example, including in the ``utest.bld `` file the following line
370
-
371
- .. code-block :: console
372
-
373
- control | -DAPP=ATM -DCCPP_SUITES=FV3_GFS_v16 -D32BIT=ON
374
-
375
- and then executing ``./utest -n control `` performs all six test cases
376
- listed in :numref: `Table %s <ImplementationRequirement >` for ``control ``
377
- test. At the end of the run, a log file ``UnitTests_<machine>.<compiler>.log ``
368
+ For example, ``./opnReqTest -n control `` performs all six test cases
369
+ listed in :numref: `Table %s <OperationalRequirement >` for ``control ``
370
+ test. At the end of the run, a log file ``OpnReqTests_<machine>.<compiler>.log ``
378
371
is generated in tests/ directory, which informs the user whether each test case
379
372
passed or failed. The user can choose to run a specific test case by invoking
380
373
381
374
.. code-block :: console
382
375
383
- ./utest -n <test-name> -c <test-case>
376
+ ./opnReqTest -n <test-name> -c <test-case>
384
377
385
378
where ``<test-case> `` is one or
386
379
more comma-separated values selected from ``thr ``, ``mpi ``, ``dcp ``, ``rst ``,
387
- ``bit ``, ``dbg ``. For example, ``./utest -n control -c thr,rst `` runs the
380
+ ``bit ``, ``dbg ``. For example, ``./opnReqTest -n control -c thr,rst `` runs the
388
381
``control `` test and checks the reproducibility of threading and restart.
389
- The user can see different command line options available to ``utest `` by
390
- executing ``./utest -h ``; frequently used options are ``-e `` to use the ecFlow
382
+ The user can see different command line options available to ``opnReqTest `` by
383
+ executing ``./opnReqTest -h ``; frequently used options are ``-e `` to use the ecFlow
391
384
workflow manager, and ``-k `` to keep the ``$RUNDIR ``. In the following,
392
- comparisons are made between the regression and unit tests on how they handle
385
+ comparisons are made between the regression and operational requirement tests on how they handle
393
386
different reproducibility tests.
394
387
395
388
As discussed in :numref: `Section %s <UsingRegressionTest >`, the variables and
@@ -398,18 +391,18 @@ values used to configure model parameters and to set up initial conditions in th
398
391
define default values; then a specific test file in the tests/tests/ subdirectory
399
392
either overrides the default values or creates new variables if required by the test.
400
393
The regression test treats the different test cases shown in
401
- :numref: `Table %s <ImplementationRequirement >` as different tests. Therefore, each
394
+ :numref: `Table %s <OperationalRequirement >` as different tests. Therefore, each
402
395
test case requires a test file in the tests/tests/ subdirectory; examples are
403
396
``control_2threads ``, ``control_decomp ``, ``control_restart `` and ``control_debug ``,
404
397
which are just variations of ``control `` test to check various reproducibilities.
405
398
There are two potential issues with this approach. First, if several different
406
399
variations of a given test were to be created and included in the ``rt.conf `` file,
407
400
there are too many tests to run. Second, if a new test is added by the user, s/he
408
- will also have to create these variations. The idea behind the unit test is to
401
+ will also have to create these variations. The idea behind the operational requirement test is to
409
402
automatically configure and run these variations, or test cases, given a test file.
410
- For example, ``./utest -n control `` will run all six test cases in
411
- :numref: `Table %s <ImplementationRequirement >` based on a single ``control `` test file.
412
- Similarly, if the user adds a new test ``new_test ``, then ``./utest -n new_test `` will
413
- run all test cases. This is done by the unit test script ``utest `` by adding a third
414
- stage of variable overrides, and the related scripts can be found in the tests/utests /
403
+ For example, ``./opnReqTest -n control `` will run all six test cases in
404
+ :numref: `Table %s <OperationalRequirement >` based on a single ``control `` test file.
405
+ Similarly, if the user adds a new test ``new_test ``, then ``./opnReqTest -n new_test `` will
406
+ run all test cases. This is done by the operational requirement test script ``opnReqTest `` by adding a third
407
+ stage of variable overrides, and the related scripts can be found in the tests/opnReqTests /
415
408
directory.
0 commit comments