WIP, DEBUG: Python derived metrics #848
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tylerjereddy
commented
Nov 9, 2022
|
Ok, CI issues notwithstanding, the new Still, we may do this someday for some of the reasons outlined above, especially if it is more performant (no re-reading records across langauge barriers) and more maintainable. |
* this is a Python/pandas-only version of doing some simple derived metrics; I don't think we'll actually do this, but I was exploring a bit because of the difficulties in darshan-hpcgh-839 * this matches pretty well with the `perl` based reports for total bytes, but even simple cases can sometimes disagree on bandwidth per darshan-hpcgh-847, so now I'm curious what is going on * one problem with doing this is that we'd have the same algorithms implemented in two different languages; the advantages include: - not reading all the records in a second time, crossing the CFFI boundary each time - easier to debug/maintain because bounds checking/no segfaults, etc.
* simplify `perf_estimate()` by removing `mod_name_adjusted` and correct total time calculation to include `META`
tylerjereddy
force-pushed
the
treddy_derived_metrics_in_python
branch
from
bf52be3 to
73c4c7e
Compare
|
We're not doing this for now, though long-term it is likely far easier to maintain via |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
this is a Python/pandas-only version of doing some simple derived metrics; I don't think we'll
actually do this, but I was exploring a bit because of the difficulties in ENH: Python derived/accum interface #839
this matches pretty well with the
perlbased reports for total bytes, but even simple cases can sometimes disagree on bandwidth per Query: bandwidth calculation for derived metrics #847, so now I'm curious what is going onone problem with doing this is that we'd have the same algorithms implemented in two different languages; the advantages include: