Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement telemetry and global per-JVM limit for auto test retries #7458

Merged
merged 1 commit into from
Aug 23, 2024
Merged

Implement telemetry and global per-JVM limit for auto test retries #7458

merged 1 commit into from
Aug 23, 2024

Conversation

nikita-tkachenko-datadog
Copy link
Contributor

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Aug 16, 2024 •
edited by jira bot
Loading

What Does This Do

Updates auto-test retries logic in CI Visibility:

  • Adds new is_retry tag for event_finished telemetry metric
  • Implements global per-JVM retries limit (in addition to existing per-test retries limit)

Additional Notes

The required changes are described in this RFC.

Contributor Checklist

Jira ticket: SDTEST-697

@nikita-tkachenko-datadog nikita-tkachenko-datadog added the comp: ci visibility Continuous Integration Visibility label Aug 16, 2024
@pr-commenter
Copy link

pr-commenter bot commented Aug 16, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/auto-test-retries-improvements
git_commit_date 1723811562 1723813705
git_commit_sha 529f893 08a18c6
release_version 1.39.0-SNAPSHOT~529f893c6a 1.39.0-SNAPSHOT~08a18c6700
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1723816063 1723816063
ci_job_id 607160302 607160302
ci_pipeline_id 41974287 41974287
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 15 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.39.0-SNAPSHOT~08a18c6700, baseline=1.39.0-SNAPSHOT~529f893c6a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.049 s) : 0, 1049049
Total [baseline] (8.503 s) : 0, 8503129
Agent [candidate] (1.057 s) : 0, 1057326
Total [candidate] (8.596 s) : 0, 8595835
section iast
Agent [baseline] (1.182 s) : 0, 1181713
Total [baseline] (9.002 s) : 0, 9002345
Agent [candidate] (1.18 s) : 0, 1180346
Total [candidate] (9.0 s) : 0, 8999918
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.191 s) : 0, 1191335
Total [baseline] (8.975 s) : 0, 8975265
Agent [candidate] (1.195 s) : 0, 1194971
Total [candidate] (9.016 s) : 0, 9015983
section iast_TELEMETRY_OFF
Agent [baseline] (1.177 s) : 0, 1176589
Total [baseline] (9.024 s) : 0, 9023556
Agent [candidate] (1.181 s) : 0, 1181498
Total [candidate] (9.063 s) : 0, 9063295
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.049 s -
Agent iast 1.182 s 132.663 ms (12.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.191 s 142.285 ms (13.6%)
Agent iast_TELEMETRY_OFF 1.177 s 127.539 ms (12.2%)
Total tracing 8.503 s -
Total iast 9.002 s 499.216 ms (5.9%)
Total iast_HARDCODED_SECRET_DISABLED 8.975 s 472.136 ms (5.6%)
Total iast_TELEMETRY_OFF 9.024 s 520.426 ms (6.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent iast 1.18 s 123.02 ms (11.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.195 s 137.646 ms (13.0%)
Agent iast_TELEMETRY_OFF 1.181 s 124.173 ms (11.7%)
Total tracing 8.596 s -
Total iast 9.0 s 404.083 ms (4.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.016 s 420.147 ms (4.9%)
Total iast_TELEMETRY_OFF 9.063 s 467.46 ms (5.4%) 
gantt
    title insecure-bank - break down per module: candidate=1.39.0-SNAPSHOT~08a18c6700, baseline=1.39.0-SNAPSHOT~529f893c6a

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (669.349 ms) : 0, 669349
BytebuddyAgent [candidate] (673.5 ms) : 0, 673500
GlobalTracer [baseline] (306.896 ms) : 0, 306896
GlobalTracer [candidate] (310.191 ms) : 0, 310191
AppSec [baseline] (51.096 ms) : 0, 51096
AppSec [candidate] (51.779 ms) : 0, 51779
Remote Config [baseline] (689.776 µs) : 0, 690
Remote Config [candidate] (697.882 µs) : 0, 698
Telemetry [baseline] (7.527 ms) : 0, 7527
Telemetry [candidate] (7.649 ms) : 0, 7649
section iast
BytebuddyAgent [baseline] (787.05 ms) : 0, 787050
BytebuddyAgent [candidate] (785.258 ms) : 0, 785258
GlobalTracer [baseline] (296.487 ms) : 0, 296487
GlobalTracer [candidate] (297.354 ms) : 0, 297354
AppSec [baseline] (51.125 ms) : 0, 51125
AppSec [candidate] (53.301 ms) : 0, 53301
IAST [baseline] (23.31 ms) : 0, 23310
IAST [candidate] (22.392 ms) : 0, 22392
Remote Config [baseline] (591.374 µs) : 0, 591
Remote Config [candidate] (596.253 µs) : 0, 596
Telemetry [baseline] (9.613 ms) : 0, 9613
Telemetry [candidate] (7.931 ms) : 0, 7931
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (792.507 ms) : 0, 792507
BytebuddyAgent [candidate] (795.062 ms) : 0, 795062
GlobalTracer [baseline] (299.92 ms) : 0, 299920
GlobalTracer [candidate] (301.224 ms) : 0, 301224
AppSec [baseline] (50.758 ms) : 0, 50758
AppSec [candidate] (50.908 ms) : 0, 50908
IAST [baseline] (24.316 ms) : 0, 24316
IAST [candidate] (24.553 ms) : 0, 24553
Remote Config [baseline] (588.86 µs) : 0, 589
Remote Config [candidate] (590.349 µs) : 0, 590
Telemetry [baseline] (9.583 ms) : 0, 9583
Telemetry [candidate] (8.891 ms) : 0, 8891
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (781.313 ms) : 0, 781313
BytebuddyAgent [candidate] (785.79 ms) : 0, 785790
GlobalTracer [baseline] (297.657 ms) : 0, 297657
GlobalTracer [candidate] (299.42 ms) : 0, 299420
AppSec [baseline] (53.996 ms) : 0, 53996
AppSec [candidate] (51.542 ms) : 0, 51542
IAST [baseline] (22.369 ms) : 0, 22369
IAST [candidate] (23.264 ms) : 0, 23264
Remote Config [baseline] (633.26 µs) : 0, 633
Remote Config [candidate] (629.787 µs) : 0, 630
Telemetry [baseline] (7.131 ms) : 0, 7131
Telemetry [candidate] (7.21 ms) : 0, 7210
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.39.0-SNAPSHOT~08a18c6700, baseline=1.39.0-SNAPSHOT~529f893c6a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.05 s) : 0, 1049627
Total [baseline] (10.434 s) : 0, 10434351
Agent [candidate] (1.049 s) : 0, 1048789
Total [candidate] (10.362 s) : 0, 10361918
section appsec
Agent [baseline] (1.179 s) : 0, 1178903
Total [baseline] (10.498 s) : 0, 10498032
Agent [candidate] (1.171 s) : 0, 1171319
Total [candidate] (10.494 s) : 0, 10494216
section iast
Agent [baseline] (1.18 s) : 0, 1179974
Total [baseline] (10.82 s) : 0, 10819822
Agent [candidate] (1.188 s) : 0, 1187590
Total [candidate] (10.839 s) : 0, 10839094
section profiling
Agent [baseline] (1.249 s) : 0, 1248901
Total [baseline] (10.591 s) : 0, 10591131
Agent [candidate] (1.257 s) : 0, 1256604
Total [candidate] (10.706 s) : 0, 10705686
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.05 s -
Agent appsec 1.179 s 129.276 ms (12.3%)
Agent iast 1.18 s 130.347 ms (12.4%)
Agent profiling 1.249 s 199.273 ms (19.0%)
Total tracing 10.434 s -
Total appsec 10.498 s 63.681 ms (0.6%)
Total iast 10.82 s 385.472 ms (3.7%)
Total profiling 10.591 s 156.781 ms (1.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.049 s -
Agent appsec 1.171 s 122.53 ms (11.7%)
Agent iast 1.188 s 138.801 ms (13.2%)
Agent profiling 1.257 s 207.814 ms (19.8%)
Total tracing 10.362 s -
Total appsec 10.494 s 132.299 ms (1.3%)
Total iast 10.839 s 477.176 ms (4.6%)
Total profiling 10.706 s 343.769 ms (3.3%) 
gantt
    title petclinic - break down per module: candidate=1.39.0-SNAPSHOT~08a18c6700, baseline=1.39.0-SNAPSHOT~529f893c6a

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (669.245 ms) : 0, 669245
BytebuddyAgent [candidate] (668.455 ms) : 0, 668455
GlobalTracer [baseline] (307.433 ms) : 0, 307433
GlobalTracer [candidate] (307.561 ms) : 0, 307561
AppSec [baseline] (51.358 ms) : 0, 51358
AppSec [candidate] (51.174 ms) : 0, 51174
Remote Config [baseline] (687.559 µs) : 0, 688
Remote Config [candidate] (683.87 µs) : 0, 684
Telemetry [baseline] (7.472 ms) : 0, 7472
Telemetry [candidate] (7.488 ms) : 0, 7488
section appsec
BytebuddyAgent [baseline] (683.875 ms) : 0, 683875
BytebuddyAgent [candidate] (679.406 ms) : 0, 679406
GlobalTracer [baseline] (302.08 ms) : 0, 302080
GlobalTracer [candidate] (300.428 ms) : 0, 300428
AppSec [baseline] (158.213 ms) : 0, 158213
AppSec [candidate] (157.668 ms) : 0, 157668
Remote Config [baseline] (606.711 µs) : 0, 607
Remote Config [candidate] (602.515 µs) : 0, 603
Telemetry [baseline] (9.096 ms) : 0, 9096
Telemetry [candidate] (9.074 ms) : 0, 9074
IAST [baseline] (22.92 ms) : 0, 22920
IAST [candidate] (21.312 ms) : 0, 21312
section iast
BytebuddyAgent [baseline] (784.704 ms) : 0, 784704
BytebuddyAgent [candidate] (790.274 ms) : 0, 790274
GlobalTracer [baseline] (297.617 ms) : 0, 297617
GlobalTracer [candidate] (299.208 ms) : 0, 299208
AppSec [baseline] (49.839 ms) : 0, 49839
AppSec [candidate] (51.832 ms) : 0, 51832
Remote Config [baseline] (576.869 µs) : 0, 577
Remote Config [candidate] (571.769 µs) : 0, 572
Telemetry [baseline] (10.239 ms) : 0, 10239
Telemetry [candidate] (7.862 ms) : 0, 7862
IAST [baseline] (23.518 ms) : 0, 23518
IAST [candidate] (24.26 ms) : 0, 24260
section profiling
ProfilingAgent [baseline] (94.804 ms) : 0, 94804
ProfilingAgent [candidate] (94.919 ms) : 0, 94919
BytebuddyAgent [baseline] (665.377 ms) : 0, 665377
BytebuddyAgent [candidate] (669.782 ms) : 0, 669782
GlobalTracer [baseline] (391.442 ms) : 0, 391442
GlobalTracer [candidate] (394.166 ms) : 0, 394166
AppSec [baseline] (52.079 ms) : 0, 52079
AppSec [candidate] (52.164 ms) : 0, 52164
Remote Config [baseline] (679.614 µs) : 0, 680
Remote Config [candidate] (690.499 µs) : 0, 690
Telemetry [baseline] (7.403 ms) : 0, 7403
Telemetry [candidate] (7.452 ms) : 0, 7452
Profiling [baseline] (94.829 ms) : 0, 94829
Profiling [candidate] (94.944 ms) : 0, 94944
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-08-16T13:20:35 2024-08-16T13:27:22
git_branch master nikita-tkachenko/auto-test-retries-improvements
git_commit_date 1723811562 1723813705
git_commit_sha 529f893 08a18c6
release_version 1.39.0-SNAPSHOT~529f893c6a 1.39.0-SNAPSHOT~08a18c6700
start_time 2024-08-16T13:20:22 2024-08-16T13:27:09
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1723815185 1723815185
ci_job_id 607160304 607160304
ci_pipeline_id 41974287 41974287
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~08a18c6700, baseline=1.39.0-SNAPSHOT~529f893c6a
    dateFormat X
    axisFormat %s
section baseline
no_agent (364.781 µs) : 344, 385
.   : milestone, 365,
iast (476.713 µs) : 456, 498
.   : milestone, 477,
iast_FULL (548.272 µs) : 526, 571
.   : milestone, 548,
iast_GLOBAL (506.352 µs) : 483, 530
.   : milestone, 506,
iast_HARDCODED_SECRET_DISABLED (469.346 µs) : 449, 490
.   : milestone, 469,
iast_INACTIVE (455.468 µs) : 433, 478
.   : milestone, 455,
iast_TELEMETRY_OFF (469.672 µs) : 448, 491
.   : milestone, 470,
tracing (436.349 µs) : 416, 456
.   : milestone, 436,
section candidate
no_agent (367.918 µs) : 348, 388
.   : milestone, 368,
iast (471.923 µs) : 451, 493
.   : milestone, 472,
iast_FULL (548.567 µs) : 526, 571
.   : milestone, 549,
iast_GLOBAL (503.24 µs) : 481, 526
.   : milestone, 503,
iast_HARDCODED_SECRET_DISABLED (477.332 µs) : 456, 499
.   : milestone, 477,
iast_INACTIVE (454.039 µs) : 431, 477
.   : milestone, 454,
iast_TELEMETRY_OFF (464.26 µs) : 443, 486
.   : milestone, 464,
tracing (437.165 µs) : 417, 457
.   : milestone, 437,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 364.781 µs [344.187 µs, 385.375 µs] -
iast 476.713 µs [455.678 µs, 497.748 µs] 111.932 µs (30.7%)
iast_FULL 548.272 µs [525.744 µs, 570.801 µs] 183.491 µs (50.3%)
iast_GLOBAL 506.352 µs [483.15 µs, 529.555 µs] 141.571 µs (38.8%)
iast_HARDCODED_SECRET_DISABLED 469.346 µs [448.538 µs, 490.154 µs] 104.565 µs (28.7%)
iast_INACTIVE 455.468 µs [433.211 µs, 477.725 µs] 90.687 µs (24.9%)
iast_TELEMETRY_OFF 469.672 µs [448.295 µs, 491.049 µs] 104.89 µs (28.8%)
tracing 436.349 µs [416.353 µs, 456.344 µs] 71.567 µs (19.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 367.918 µs [348.26 µs, 387.577 µs] -
iast 471.923 µs [450.704 µs, 493.141 µs] 104.004 µs (28.3%)
iast_FULL 548.567 µs [525.819 µs, 571.314 µs] 180.648 µs (49.1%)
iast_GLOBAL 503.24 µs [480.607 µs, 525.874 µs] 135.322 µs (36.8%)
iast_HARDCODED_SECRET_DISABLED 477.332 µs [455.914 µs, 498.75 µs] 109.414 µs (29.7%)
iast_INACTIVE 454.039 µs [431.442 µs, 476.635 µs] 86.12 µs (23.4%)
iast_TELEMETRY_OFF 464.26 µs [442.952 µs, 485.569 µs] 96.342 µs (26.2%)
tracing 437.165 µs [417.052 µs, 457.277 µs] 69.246 µs (18.8%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~08a18c6700, baseline=1.39.0-SNAPSHOT~529f893c6a
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.346 ms) : 1327, 1365
.   : milestone, 1346,
appsec (1.715 ms) : 1690, 1740
.   : milestone, 1715,
appsec_no_iast (1.713 ms) : 1689, 1738
.   : milestone, 1713,
iast (1.498 ms) : 1475, 1520
.   : milestone, 1498,
profiling (1.474 ms) : 1449, 1498
.   : milestone, 1474,
tracing (1.472 ms) : 1448, 1495
.   : milestone, 1472,
section candidate
no_agent (1.331 ms) : 1311, 1350
.   : milestone, 1331,
appsec (1.723 ms) : 1701, 1746
.   : milestone, 1723,
appsec_no_iast (1.726 ms) : 1702, 1750
.   : milestone, 1726,
iast (1.484 ms) : 1462, 1507
.   : milestone, 1484,
profiling (1.481 ms) : 1457, 1505
.   : milestone, 1481,
tracing (1.427 ms) : 1402, 1452
.   : milestone, 1427,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.346 ms [1.327 ms, 1.365 ms] -
appsec 1.715 ms [1.69 ms, 1.74 ms] 369.171 µs (27.4%)
appsec_no_iast 1.713 ms [1.689 ms, 1.738 ms] 367.491 µs (27.3%)
iast 1.498 ms [1.475 ms, 1.52 ms] 151.713 µs (11.3%)
profiling 1.474 ms [1.449 ms, 1.498 ms] 127.726 µs (9.5%)
tracing 1.472 ms [1.448 ms, 1.495 ms] 125.909 µs (9.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.331 ms [1.311 ms, 1.35 ms] -
appsec 1.723 ms [1.701 ms, 1.746 ms] 392.882 µs (29.5%)
appsec_no_iast 1.726 ms [1.702 ms, 1.75 ms] 395.369 µs (29.7%)
iast 1.484 ms [1.462 ms, 1.507 ms] 153.627 µs (11.5%)
profiling 1.481 ms [1.457 ms, 1.505 ms] 150.347 µs (11.3%)
tracing 1.427 ms [1.402 ms, 1.452 ms] 96.592 µs (7.3%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/auto-test-retries-improvements
git_commit_date 1723811562 1723813705
git_commit_sha 529f893 08a18c6
release_version 1.39.0-SNAPSHOT~529f893c6a 1.39.0-SNAPSHOT~08a18c6700
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1723815723 1723815723
ci_job_id 607160306 607160306
ci_pipeline_id 41974287 41974287
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.39.0-SNAPSHOT~08a18c6700, baseline=1.39.0-SNAPSHOT~529f893c6a
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.522 s) : 15522000, 15522000
.   : milestone, 15522000,
appsec (15.318 s) : 15318000, 15318000
.   : milestone, 15318000,
iast (18.607 s) : 18607000, 18607000
.   : milestone, 18607000,
iast_GLOBAL (17.854 s) : 17854000, 17854000
.   : milestone, 17854000,
profiling (15.287 s) : 15287000, 15287000
.   : milestone, 15287000,
tracing (15.039 s) : 15039000, 15039000
.   : milestone, 15039000,
section candidate
no_agent (15.438 s) : 15438000, 15438000
.   : milestone, 15438000,
appsec (15.052 s) : 15052000, 15052000
.   : milestone, 15052000,
iast (18.699 s) : 18699000, 18699000
.   : milestone, 18699000,
iast_GLOBAL (17.98 s) : 17980000, 17980000
.   : milestone, 17980000,
profiling (15.2 s) : 15200000, 15200000
.   : milestone, 15200000,
tracing (14.936 s) : 14936000, 14936000
.   : milestone, 14936000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.522 s [15.522 s, 15.522 s] -
appsec 15.318 s [15.318 s, 15.318 s] -204.0 ms (-1.3%)
iast 18.607 s [18.607 s, 18.607 s] 3.085 s (19.9%)
iast_GLOBAL 17.854 s [17.854 s, 17.854 s] 2.332 s (15.0%)
profiling 15.287 s [15.287 s, 15.287 s] -235.0 ms (-1.5%)
tracing 15.039 s [15.039 s, 15.039 s] -483.0 ms (-3.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.438 s [15.438 s, 15.438 s] -
appsec 15.052 s [15.052 s, 15.052 s] -386.0 ms (-2.5%)
iast 18.699 s [18.699 s, 18.699 s] 3.261 s (21.1%)
iast_GLOBAL 17.98 s [17.98 s, 17.98 s] 2.542 s (16.5%)
profiling 15.2 s [15.2 s, 15.2 s] -238.0 ms (-1.5%)
tracing 14.936 s [14.936 s, 14.936 s] -502.0 ms (-3.3%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.39.0-SNAPSHOT~08a18c6700, baseline=1.39.0-SNAPSHOT~529f893c6a
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.466 ms) : 1454, 1477
.   : milestone, 1466,
appsec (2.229 ms) : 2193, 2264
.   : milestone, 2229,
iast (1.973 ms) : 1931, 2016
.   : milestone, 1973,
iast_GLOBAL (2.024 ms) : 1981, 2067
.   : milestone, 2024,
profiling (1.856 ms) : 1821, 1891
.   : milestone, 1856,
tracing (1.85 ms) : 1817, 1883
.   : milestone, 1850,
section candidate
no_agent (1.464 ms) : 1453, 1476
.   : milestone, 1464,
appsec (2.229 ms) : 2194, 2264
.   : milestone, 2229,
iast (1.975 ms) : 1933, 2017
.   : milestone, 1975,
iast_GLOBAL (2.024 ms) : 1981, 2066
.   : milestone, 2024,
profiling (2.355 ms) : 2167, 2542
.   : milestone, 2355,
tracing (1.828 ms) : 1795, 1860
.   : milestone, 1828,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.466 ms [1.454 ms, 1.477 ms] -
appsec 2.229 ms [2.193 ms, 2.264 ms] 763.112 µs (52.1%)
iast 1.973 ms [1.931 ms, 2.016 ms] 507.702 µs (34.6%)
iast_GLOBAL 2.024 ms [1.981 ms, 2.067 ms] 558.277 µs (38.1%)
profiling 1.856 ms [1.821 ms, 1.891 ms] 390.428 µs (26.6%)
tracing 1.85 ms [1.817 ms, 1.883 ms] 384.093 µs (26.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.464 ms [1.453 ms, 1.476 ms] -
appsec 2.229 ms [2.194 ms, 2.264 ms] 764.954 µs (52.2%)
iast 1.975 ms [1.933 ms, 2.017 ms] 510.848 µs (34.9%)
iast_GLOBAL 2.024 ms [1.981 ms, 2.066 ms] 559.399 µs (38.2%)
profiling 2.355 ms [2.167 ms, 2.542 ms] 890.5 µs (60.8%)
tracing 1.828 ms [1.795 ms, 1.86 ms] 363.563 µs (24.8%)

@nikita-tkachenko-datadog nikita-tkachenko-datadog marked this pull request as ready for review August 16, 2024 14:43
@nikita-tkachenko-datadog nikita-tkachenko-datadog merged commit 6a34b61 into master Aug 23, 2024
82 checks passed
@nikita-tkachenko-datadog nikita-tkachenko-datadog deleted the nikita-tkachenko/auto-test-retries-improvements branch August 23, 2024 11:48
@github-actions github-actions bot added this to the 1.39.0 milestone Aug 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@juan-fernandez juan-fernandez juan-fernandez approved these changes

@tonyredondo tonyredondo Awaiting requested review from tonyredondo tonyredondo is a code owner automatically assigned from DataDog/ci-app-libraries-java

@dougqh dougqh Awaiting requested review from dougqh dougqh is a code owner automatically assigned from DataDog/apm-java

@ygree ygree Awaiting requested review from ygree ygree is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: ci visibility Continuous Integration Visibility
Projects
None yet
Milestone
1.39.0
Development

Successfully merging this pull request may close these issues.
2 participants
@nikita-tkachenko-datadog @juan-fernandez