Merge pull request #678 from buildtesters/diagram_update

update diagrams for buildspec structure and Parser.

buildtest/menu/build.py

@@ -494,7 +494,7 @@ def parse_buildspecs(self, printTable=False):
     def build(self):
         """This method is responsible for implementating stages: parse, build, run, update. """
 
-        self.discover_buildspecs()
+        self.discover_buildspecs(printTable=True)
 
         # Parse all buildspecs and skip any buildspecs that fail validation, return type
         # is a builder object used for building test.
@@ -752,9 +752,9 @@ def run_phase(self, printTable=False):
         if printTable:
             print(
                 """
-        +----------------------+
-        | Stage: Test Summary  |
-        +----------------------+ 
++----------------------+
+| Stage: Test Summary  |
++----------------------+ 
         """
             )
 

docs/buildspecs/batch_support.rst

@@ -522,7 +522,7 @@ job as follows.
     buildspecs:
       sleep:
         type: script
-        executor: slurm.normal
+        executor: cori.slurm.knl_debug
         description: sleep 2 seconds
         tags: [tutorials]
         batch:

docs/index.rst

@@ -3,9 +3,10 @@ buildtest
 
 This documentation was last rebuild on |today| and is intended for version |version|.
 
-Please refer to https://buildtest.readthedocs.io/en/latest/ for documentation
-on latest release. If you are working off `devel` branch then please to *devel*
-docs at https://buildtest.readthedocs.io/en/devel/.
+If you are working off a latest release please see  https://buildtest.readthedocs.io/en/latest/ for documentation.
+If you are working off `devel <https://github.com/buildtesters/buildtest/tree/devel>`_ branch then please refer to
+https://buildtest.readthedocs.io/en/devel/
+which references the `devel` branch.
 
 Status
 -----------
@@ -41,7 +42,6 @@ Test Repositories
 ------------------
 
 - Cori @ NERSC: https://github.com/buildtesters/buildtest-cori
-- Stampede2 @ TACC: https://github.com/buildtesters/buildtest-stampede2
 
 Useful Links
 -------------
@@ -66,25 +66,24 @@ Description
 ------------
 
 `buildtest <https://github.com/buildtesters/buildtest>`_  is
-a HPC testing framework to help sites perform acceptance & regression
-testing of an HPC system. buildtest utilizes `json schema <https://json-schema.org/>`_
-to define structure of test written in YAML called a  **Buildspec File**.
-The `schema docs <https://buildtesters.github.io/schemas/>`_ is a resource that
-hosts buildtest schemas and documents all field attributes for each schema, this will
-be useful when you are writting buildspecs.
+a testing framework to help HPC sites write test for their system as part of their
+routine acceptance & regression test. buildtest provides a YAML interface to write tests
+which buildtest can validate and generate shell scripts that can run on your HPC system.
+The test template (YAML) is called **buildspec** which can define one or more test instance
+that is validated by a `json schema <https://json-schema.org/>`_. buildtest supports the following batch schedulers
+for job submission: `IBM Spectrum LSF <https://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_welcome/lsf_welcome.html>`_,
+`Slurm <https://slurm.schedmd.com/>`_, and `Cobalt <https://trac.mcs.anl.gov/projects/cobalt/>`_. We
+publish the schema documentation, json schemas, and schema examples at https://buildtesters.github.io/buildtesters/
+which is useful when you are :ref:`writing buildspecs <writing_buildspecs>`.
+
+To get started with buildtest, please see :ref:`installing buildtest <Setup>`  and
+:ref:`getting started guide <Getting_Started>`.
 
 A spin-off project called `lmodule <https://lmodule.readthedocs.io/en/latest/>`_
 is a Python API for `Lmod <https://lmod.readthedocs.io/en/latest/>`_. The buildtest
 module features were deprecated and moved to lmodule with the main objective is to
 automate module load testing. For more details on lmodule see https://github.com/buildtesters/lmodule
 
-To get started with buildtest, please review the :ref:`terminology`
-and proceed to :ref:`installing buildtest <Setup>`  followed by :ref:`getting started guide <Getting_Started>`.
-
-For additional reference, you can read :ref:`summary_of_buildtest` and
-:ref:`conferences`.
-
-
 .. toctree::
    :maxdepth: 2
    :caption: Background

docs/what_is_buildtest.rst

@@ -51,13 +51,13 @@ Shahzeb was tasked with testing the software ecosystem by focusing on the most
 important application due to time constraints. During this period, several dozen
 test scripts were developed in shell-script that targeted core HPC tools such as
 compilers, **MPI**, **R**, **Python**, etc. A single master script was used to
-run all the tests which led to buildtest.
+run all the tests which led to `buildtest`.
 
 Preview of buildtest
 ----------------------
 
 buildtest will process YAML files called **buildspecs** (build specification) and
-generate test scripts. There are several ways to discover buildspecs that may include:
+generate test scripts. There are several ways to build buildspecs that may include:
 
 .. code-block:: console
 
@@ -79,24 +79,26 @@ generate test scripts. There are several ways to discover buildspecs that may in
   # discover buildspecs based on tags
   $ buildtest build --tags network
 
+  # filter tests by tag name network
+  $ buildtest build --tags network --filter-tags network
+
   # discover buildspecs based on executors
-  $ buildtest build --executor local.sh
+  $ buildtest build --executor generic.local.sh
 
   # combine all options together
   $ buildtest build -b example.yml -b system --tags network -x system/kernel.yml
 
-buildtest will keep track of all test that can be queried using:
+
+buildtest will retrieve test results using `buildtest report` command which will display
+all test records. We can filter and format output of test records using the ``--filter``
+and ``--format`` option. Shown below is example usage:
 
 .. code-block:: console
 
   # report all tests
   $ buildtest report
 
-Alternately, one can filter and format rows/columns of the report table which
-can be useful when you want to analyze test results. This can be done using:
-
-.. code-block:: console
-
+  # filter and format test report
   $ buildtest report --filter KEY1=VALUE1,KEY2=VALUE2 --format <field1>,<field2>
 
   # show filter fields
@@ -105,6 +107,13 @@ can be useful when you want to analyze test results. This can be done using:
   # show format fields
   $ buildtest report --helpformat
 
+  # get oldest test record
+  $ buildtest report --oldest
+
+  # get latest test record
+  $ buildtest report --latest
+
+
 A single buildspec is composed of one or more tests and buildtest will run all tests
 by default. buildtest will assign a unique id for every test, if you want to inspect
 a particular test you can run the following:
@@ -167,7 +176,7 @@ buildtest is not
   - a replacement for benchmark tools or test suite from upstream package
   - a replacement for writing tests, you will need to write your tests defined by buildtest schemas, however you can copy/paste & adapt tests from other sites that are applicable to you.
 
-Typical use-case :
+Typical use-case:
 
   - Run your test suite during system maintenance
   - Perform daily tests for testing various system components. These tests should be short

tests/buildsystem/test_base.py

@@ -39,7 +39,7 @@ def test_BuildspecParser(tmp_path):
 
     directory = os.path.join(here, "invalid_builds")
     # invalid builds for compiler schema tests. These tests will raise BuildTestError exception upon building
-    # even though they are valid buildspecs.
+    # even though they are valid buildspecs.\
     for buildspec in walk_tree(directory, ".yml"):
         buildspecfile = os.path.join(directory, buildspec)
         print("Processing buildspec", buildspecfile)