logutils accumulator API

[carns]

• Adds a "darshan_accumulator" API to the logutils library
  • _create(), _inject(), _emit(), and _destroy()
  • generalizes the mechanism for producing summation records and derived metrics for sets of records from a given module
  • refactors darshan-parser to use new API
  • implements support for accumulators in POSIX, STDIO, and MPIIO modules

• Integrates the µnit Testing Framework in darshan-util
  • implements unit tests for darshan_accumlator API; they can be invoked with make check at the top level build directory

Note that this does not include Python bindings; those need to be added separately after this is merged so that these routines can be used withing Python utilities.

Fixes #642

[carns]

Pausing work on this PR until refactoring some underlying code. First step is in #684

[carns]

Current status:

• rebased PR against origin/main following 3.4.0 release (clean, all tests pass)
• munit testing in place
• api stubbed out (see bottom of darshan-logutils.h)
• implementation of record aggregation in place for mpi-io, posix, and stdio, with test cases (see darshan-logutils-accumulator.c)
• backported fixes to agg_record functions to origin/main

TODO:

• evaluate if any of the above can be factored into darshan-parser yet
• implement initial metrics struct calculation for the three primary module types
• pull logic from calc_perf, calc_file, and file_list functions in darshan-parser into accumulator

[carns]

Next development target should probably be to replace the posix_accum_file() (and subsequently, stdio and mpiio versions of that same function) in darshan-parser. Right now it calls the agg_record functions directly and does some metric accumulation. It is used twice; once per unique file and once globally.

[carns]

The primary complexity with replacing/refactoring the posix_accum_file() path is that it is used for two cases: once for summary metrics, and once for per-file metrics. The latter is (in part) used for the --file-list and --file-list-detailed options to darshan-parser.

I'm not sure how important these options are, but their existence means that (regardless of accumulator API capabilities) the darshan-parser has to continue to maintain it's own explicit hash table to track distinct files. If it weren't for those options then the hash table could be hidden within the accumulator implementation to produce derived values (like in the --file option) to greatly simplify the darshan-parser code base.

[carns]

rebased atop the PR changes that remove --file-list and --file-list-detailed to take another look at how to replace the posix_accum_file() path.

[carns]

Accumulator API now implements functionality needed for --total and --file arguments to darshan-parser. Next steps:

• integrate into darshan-parser
• validate darshan-parser output
• implement unit tests

After that, the same process will need to be done for the --perf capability.

[carns]

Final steps are for performance metrics:

• implement --perf functionality in accumulator
• integrate into darshan-parser
• validate darshan-parser output
• implement unit tests

[carns]

Functionality complete. Need to make any necessary doc/changelog updates and squash.

[shanedsnyder]

Looks good, thanks!