Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test bench scheduler controller #5140

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

tao-stones
Copy link

Problem

To observe different Schedulers performance, scheduling behavior, and how low-prio non-contend tx being scheduled.

Summary of Changes

  • add bench setup

Fixes #

@tao-stones
Copy link
Author

  • with 16 unique receiving accounts, non-contend took almost 5X longer, fully-contend change slightly
  • greedy isn't much better than prio-graph in these two extreme scenario
  • Tracer placement in fully-contend scenarios are not fully expected.
bench_prio_graph_non_contend_transactions/sdk_transaction_type
                        time:   [303.21 ms 308.86 ms 315.09 ms]
                        change: [-1.9760% +0.5269% +3.1610%] (p = 0.71 > 0.05)
                        No change in performance detected.

== Averaged per bench stats to empty Container ==
            number of scheduling: 100
            number of Works: 1600
            number of transactions scheduled: 100000
            number of transactions per work: 62
            Tracer placement: 100000
Benchmarking bench_prio_graph_fully_contend_transactions/sdk_transaction_type: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 35.1s.
bench_prio_graph_fully_contend_transactions/sdk_transaction_type
                        time:   [556.00 ms 570.06 ms 589.64 ms]
                        change: [-6.7949% -2.2685% +2.7725%] (p = 0.39 > 0.05)
                        No change in performance detected.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

== Averaged per bench stats to empty Container ==
            number of scheduling: 100
            number of Works: 100000
            number of transactions scheduled: 100000
            number of transactions per work: 1
            Tracer placement: 99746
Benchmarking bench_greedy_non_contend_transactions/sdk_transaction_type: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 32.0s.
bench_greedy_non_contend_transactions/sdk_transaction_type
                        time:   [299.99 ms 310.50 ms 323.11 ms]
                        change: [-2.9341% +1.3619% +5.8495%] (p = 0.58 > 0.05)
                        No change in performance detected.
Found 3 outliers among 10 measurements (30.00%)
  1 (10.00%) low mild
  2 (20.00%) high severe

== Averaged per bench stats to empty Container ==
            number of scheduling: 1
            number of Works: 1584
            number of transactions scheduled: 100000
            number of transactions per work: 63
            Tracer placement: 100000
Benchmarking bench_greedy_fully_contend_transactions/sdk_transaction_type: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 34.0s.
bench_greedy_fully_contend_transactions/sdk_transaction_type
                        time:   [502.38 ms 517.80 ms 534.91 ms]
                        change: [-8.7162% -3.0078% +2.8615%] (p = 0.36 > 0.05)
                        No change in performance detected.

== Averaged per bench stats to empty Container ==
            number of scheduling: 1
            number of Works: 100000
            number of transactions scheduled: 100000
            number of transactions per work: 1
            Tracer placement: 100000

@tao-stones tao-stones requested a review from apfitzge March 6, 2025 21:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant