Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: migrate params tests to vitest #6217

Merged
merged 2 commits into from
Dec 21, 2023
Merged

test: migrate params tests to vitest #6217

merged 2 commits into from
Dec 21, 2023

Conversation

nazarhussain
Copy link
Contributor

Motivation

Consolidate the testing frameworks and migrate to vitest.

Description

Migrate params tests to vitest.

Closes #issue_number

  • Run all tests

@nazarhussain nazarhussain requested a review from a team as a code owner December 20, 2023 19:08
github-advanced-security[bot]
github-advanced-security bot found potential problems Dec 20, 2023
View reviewed changes
packages/params/test/e2e/overridePreset.test.ts
@@ -31,7 +28,7 @@
});

it("Should throw trying to override preset in the wrong order", async () => {
await expect(exec(`node --loader ts-node/esm ${path.join(__dirname, scriptNames.error)}`)).to.be.rejectedWith(
await expect(exec(`node --loader ts-node/esm ${path.join(__dirname, scriptNames.error)}`)).rejects.toThrow(

Check warning

Code scanning / CodeQL

Shell command built from environment values

This shell command depends on an uncontrolled [absolute path](1).
packages/params/test/e2e/setPreset.test.ts
@@ -31,7 +28,7 @@
});

it("Should throw trying to set preset in the wrong order", async () => {
await expect(exec(`node --loader ts-node/esm ${path.join(__dirname, scriptNames.error)}`)).to.be.rejectedWith(
await expect(exec(`node --loader ts-node/esm ${path.join(__dirname, scriptNames.error)}`)).rejects.toThrow(

Check warning

Code scanning / CodeQL

Shell command built from environment values

This shell command depends on an uncontrolled [absolute path](1).
@codecov Codecov
Copy link

codecov bot commented Dec 20, 2023 •
edited
Loading

Codecov Report

Merging #6217 (02c3815) into unstable (6d1dc61) will decrease coverage by 1.18%.
Report is 7 commits behind head on unstable.
The diff coverage is n/a.

Additional details and impacted files 
@@             Coverage Diff              @@
##           unstable    #6217      +/-   ##
============================================
- Coverage     85.63%   84.46%   -1.18%     
============================================
  Files           172      177       +5     
  Lines         14372    14913     +541     
  Branches        880      909      +29     
============================================
+ Hits          12308    12596     +288     
- Misses         2041     2294     +253     
  Partials         23       23

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Dec 20, 2023 •
edited
Loading

⚠️ Performance Alert ⚠️

Possible performance regression was detected for some benchmarks.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold.

Benchmark suite Current: 583fc80 Previous: 2530fae Ratio
Map set x1000 54.557 ns/op 17.542 ns/op 3.11
altair processAttestation - 250000 vs - 7PWei normalcase 6.9362 ms/op 1.6979 ms/op 4.09
altair processAttestation - 250000 vs - 7PWei worstcase 8.8468 ms/op 2.8145 ms/op 3.14
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 20.684 us/op 5.4010 us/op 3.83
Full benchmark results
Benchmark suite Current: 583fc80 Previous: 2530fae Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 639.37 us/op 391.97 us/op 1.63
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 93.484 us/op 56.089 us/op 1.67
BLS verify - blst-native 1.3278 ms/op 1.1020 ms/op 1.20
BLS verifyMultipleSignatures 3 - blst-native 2.8005 ms/op 2.2812 ms/op 1.23
BLS verifyMultipleSignatures 8 - blst-native 6.1177 ms/op 5.0032 ms/op 1.22
BLS verifyMultipleSignatures 32 - blst-native 22.419 ms/op 18.692 ms/op 1.20
BLS verifyMultipleSignatures 64 - blst-native 44.405 ms/op 36.461 ms/op 1.22
BLS verifyMultipleSignatures 128 - blst-native 87.636 ms/op 75.582 ms/op 1.16
BLS deserializing 10000 signatures 955.57 ms/op 790.63 ms/op 1.21
BLS deserializing 100000 signatures 9.3632 s/op 8.0558 s/op 1.16
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.3770 ms/op 1.1746 ms/op 1.17
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.5802 ms/op 1.3177 ms/op 1.20
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.4370 ms/op 2.1175 ms/op 1.15
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.6563 ms/op 3.2463 ms/op 1.13
BLS verifyMultipleSignatures - same message - 128 - blst-native 5.9020 ms/op 6.8566 ms/op 0.86
BLS aggregatePubkeys 32 - blst-native 27.333 us/op 23.949 us/op 1.14
BLS aggregatePubkeys 128 - blst-native 105.33 us/op 85.627 us/op 1.23
getAttestationsForBlock 59.540 ms/op 39.689 ms/op 1.50
getSlashingsAndExits - default max 205.72 us/op 94.766 us/op 2.17
getSlashingsAndExits - 2k 431.82 us/op 402.02 us/op 1.07
proposeBlockBody type=full, size=empty 7.7635 ms/op 4.0703 ms/op 1.91
isKnown best case - 1 super set check 572.00 ns/op 331.00 ns/op 1.73
isKnown normal case - 2 super set checks 450.00 ns/op 321.00 ns/op 1.40
isKnown worse case - 16 super set checks 611.00 ns/op 331.00 ns/op 1.85
CheckpointStateCache - add get delete 6.3450 us/op 3.6920 us/op 1.72
validate api signedAggregateAndProof - struct 2.9183 ms/op 2.5660 ms/op 1.14
validate gossip signedAggregateAndProof - struct 3.0111 ms/op 2.5464 ms/op 1.18
validate gossip attestation - vc 640000 1.4501 ms/op 1.1868 ms/op 1.22
batch validate gossip attestation - vc 640000 - chunk 32 174.84 us/op 146.65 us/op 1.19
batch validate gossip attestation - vc 640000 - chunk 64 154.57 us/op 124.32 us/op 1.24
batch validate gossip attestation - vc 640000 - chunk 128 149.41 us/op 118.70 us/op 1.26
batch validate gossip attestation - vc 640000 - chunk 256 140.66 us/op 113.37 us/op 1.24
pickEth1Vote - no votes 1.3551 ms/op 924.25 us/op 1.47
pickEth1Vote - max votes 10.610 ms/op 13.091 ms/op 0.81
pickEth1Vote - Eth1Data hashTreeRoot value x2048 24.729 ms/op 23.765 ms/op 1.04
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 24.811 ms/op 23.761 ms/op 1.04
pickEth1Vote - Eth1Data fastSerialize value x2048 738.23 us/op 513.03 us/op 1.44
pickEth1Vote - Eth1Data fastSerialize tree x2048 8.3853 ms/op 4.4490 ms/op 1.88
bytes32 toHexString 669.00 ns/op 670.00 ns/op 1.00
bytes32 Buffer.toString(hex) 289.00 ns/op 354.00 ns/op 0.82
bytes32 Buffer.toString(hex) from Uint8Array 513.00 ns/op 520.00 ns/op 0.99
bytes32 Buffer.toString(hex) + 0x 312.00 ns/op 321.00 ns/op 0.97
Object access 1 prop 0.21100 ns/op 0.21000 ns/op 1.00
Map access 1 prop 0.17100 ns/op 0.20100 ns/op 0.85
Object get x1000 7.6340 ns/op 5.8160 ns/op 1.31
Map get x1000 0.88400 ns/op 0.73300 ns/op 1.21
Object set x1000 63.539 ns/op 28.161 ns/op 2.26
Map set x1000 54.557 ns/op 17.542 ns/op 3.11
Return object 10000 times 0.26520 ns/op 0.22640 ns/op 1.17
Throw Error 10000 times 4.1188 us/op 2.7868 us/op 1.48
fastMsgIdFn sha256 / 200 bytes 3.5130 us/op 1.9390 us/op 1.81
fastMsgIdFn h32 xxhash / 200 bytes 361.00 ns/op 299.00 ns/op 1.21
fastMsgIdFn h64 xxhash / 200 bytes 406.00 ns/op 344.00 ns/op 1.18
fastMsgIdFn sha256 / 1000 bytes 11.828 us/op 6.2100 us/op 1.90
fastMsgIdFn h32 xxhash / 1000 bytes 489.00 ns/op 405.00 ns/op 1.21
fastMsgIdFn h64 xxhash / 1000 bytes 483.00 ns/op 405.00 ns/op 1.19
fastMsgIdFn sha256 / 10000 bytes 105.96 us/op 52.287 us/op 2.03
fastMsgIdFn h32 xxhash / 10000 bytes 2.0790 us/op 1.7810 us/op 1.17
fastMsgIdFn h64 xxhash / 10000 bytes 1.4570 us/op 1.2210 us/op 1.19
send data - 1000 256B messages 21.130 ms/op 11.508 ms/op 1.84
send data - 1000 512B messages 29.183 ms/op 14.913 ms/op 1.96
send data - 1000 1024B messages 43.564 ms/op 23.158 ms/op 1.88
send data - 1000 1200B messages 37.744 ms/op 28.532 ms/op 1.32
send data - 1000 2048B messages 47.018 ms/op 36.292 ms/op 1.30
send data - 1000 4096B messages 50.895 ms/op 31.335 ms/op 1.62
send data - 1000 16384B messages 128.86 ms/op 92.439 ms/op 1.39
send data - 1000 65536B messages 483.02 ms/op 422.93 ms/op 1.14
enrSubnets - fastDeserialize 64 bits 1.4270 us/op 976.00 ns/op 1.46
enrSubnets - ssz BitVector 64 bits 486.00 ns/op 411.00 ns/op 1.18
enrSubnets - fastDeserialize 4 bits 197.00 ns/op 201.00 ns/op 0.98
enrSubnets - ssz BitVector 4 bits 489.00 ns/op 416.00 ns/op 1.18
prioritizePeers score -10:0 att 32-0.1 sync 2-0 117.50 us/op 69.770 us/op 1.68
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 154.12 us/op 77.468 us/op 1.99
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 198.10 us/op 102.49 us/op 1.93
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 339.52 us/op 179.67 us/op 1.89
prioritizePeers score 0:0 att 64-1 sync 4-1 383.41 us/op 203.80 us/op 1.88
array of 16000 items push then shift 1.7111 us/op 1.2761 us/op 1.34
LinkedList of 16000 items push then shift 9.4770 ns/op 5.9540 ns/op 1.59
array of 16000 items push then pop 115.81 ns/op 57.390 ns/op 2.02
LinkedList of 16000 items push then pop 9.3830 ns/op 5.6330 ns/op 1.67
array of 24000 items push then shift 2.7025 us/op 1.8935 us/op 1.43
LinkedList of 24000 items push then shift 9.2660 ns/op 5.7590 ns/op 1.61
array of 24000 items push then pop 153.28 ns/op 76.674 ns/op 2.00
LinkedList of 24000 items push then pop 9.1160 ns/op 5.6860 ns/op 1.60
intersect bitArray bitLen 8 6.9560 ns/op 5.0550 ns/op 1.38
intersect array and set length 8 72.608 ns/op 49.177 ns/op 1.48
intersect bitArray bitLen 128 34.123 ns/op 27.571 ns/op 1.24
intersect array and set length 128 1.0066 us/op 686.06 ns/op 1.47
bitArray.getTrueBitIndexes() bitLen 128 1.8150 us/op 1.1500 us/op 1.58
bitArray.getTrueBitIndexes() bitLen 248 2.9670 us/op 1.8300 us/op 1.62
bitArray.getTrueBitIndexes() bitLen 512 6.0830 us/op 3.4360 us/op 1.77
Buffer.concat 32 items 1.0390 us/op 838.00 ns/op 1.24
Uint8Array.set 32 items 2.0160 us/op 1.5860 us/op 1.27
Set add up to 64 items then delete first 4.9293 us/op 1.7346 us/op 2.84
OrderedSet add up to 64 items then delete first 6.0233 us/op 2.6479 us/op 2.27
Set add up to 64 items then delete last 5.3504 us/op 1.9891 us/op 2.69
OrderedSet add up to 64 items then delete last 6.9478 us/op 2.9662 us/op 2.34
Set add up to 64 items then delete middle 5.3316 us/op 1.9838 us/op 2.69
OrderedSet add up to 64 items then delete middle 8.3662 us/op 4.1603 us/op 2.01
Set add up to 128 items then delete first 10.757 us/op 3.9154 us/op 2.75
OrderedSet add up to 128 items then delete first 14.597 us/op 6.1481 us/op 2.37
Set add up to 128 items then delete last 10.592 us/op 3.7722 us/op 2.81
OrderedSet add up to 128 items then delete last 13.928 us/op 5.7034 us/op 2.44
Set add up to 128 items then delete middle 10.709 us/op 3.7910 us/op 2.82
OrderedSet add up to 128 items then delete middle 19.636 us/op 10.672 us/op 1.84
Set add up to 256 items then delete first 21.417 us/op 7.6162 us/op 2.81
OrderedSet add up to 256 items then delete first 28.840 us/op 12.176 us/op 2.37
Set add up to 256 items then delete last 21.070 us/op 7.4153 us/op 2.84
OrderedSet add up to 256 items then delete last 27.493 us/op 11.342 us/op 2.42
Set add up to 256 items then delete middle 20.804 us/op 7.3950 us/op 2.81
OrderedSet add up to 256 items then delete middle 50.246 us/op 30.140 us/op 1.67
transfer serialized Status (84 B) 1.8160 us/op 1.3580 us/op 1.34
copy serialized Status (84 B) 1.6320 us/op 1.1650 us/op 1.40
transfer serialized SignedVoluntaryExit (112 B) 1.8710 us/op 1.3710 us/op 1.36
copy serialized SignedVoluntaryExit (112 B) 1.6720 us/op 1.1720 us/op 1.43
transfer serialized ProposerSlashing (416 B) 2.1790 us/op 1.5180 us/op 1.44
copy serialized ProposerSlashing (416 B) 2.3750 us/op 1.3720 us/op 1.73
transfer serialized Attestation (485 B) 2.4610 us/op 1.5680 us/op 1.57
copy serialized Attestation (485 B) 2.3650 us/op 1.3820 us/op 1.71
transfer serialized AttesterSlashing (33232 B) 3.2770 us/op 1.6410 us/op 2.00
copy serialized AttesterSlashing (33232 B) 8.1260 us/op 3.4760 us/op 2.34
transfer serialized Small SignedBeaconBlock (128000 B) 3.3630 us/op 1.7240 us/op 1.95
copy serialized Small SignedBeaconBlock (128000 B) 22.838 us/op 8.3810 us/op 2.72
transfer serialized Avg SignedBeaconBlock (200000 B) 3.3800 us/op 1.9190 us/op 1.76
copy serialized Avg SignedBeaconBlock (200000 B) 34.559 us/op 12.711 us/op 2.72
transfer serialized BlobsSidecar (524380 B) 3.6000 us/op 2.2330 us/op 1.61
copy serialized BlobsSidecar (524380 B) 86.624 us/op 78.101 us/op 1.11
transfer serialized Big SignedBeaconBlock (1000000 B) 3.8790 us/op 2.6970 us/op 1.44
copy serialized Big SignedBeaconBlock (1000000 B) 178.00 us/op 195.67 us/op 0.91
pass gossip attestations to forkchoice per slot 4.0303 ms/op 2.7362 ms/op 1.47
forkChoice updateHead vc 100000 bc 64 eq 0 724.58 us/op 491.32 us/op 1.47
forkChoice updateHead vc 600000 bc 64 eq 0 6.3080 ms/op 2.7156 ms/op 2.32
forkChoice updateHead vc 1000000 bc 64 eq 0 8.0386 ms/op 4.4277 ms/op 1.82
forkChoice updateHead vc 600000 bc 320 eq 0 4.5958 ms/op 2.6223 ms/op 1.75
forkChoice updateHead vc 600000 bc 1200 eq 0 4.7509 ms/op 2.6921 ms/op 1.76
forkChoice updateHead vc 600000 bc 7200 eq 0 5.6179 ms/op 3.5379 ms/op 1.59
forkChoice updateHead vc 600000 bc 64 eq 1000 12.017 ms/op 10.063 ms/op 1.19
forkChoice updateHead vc 600000 bc 64 eq 10000 13.434 ms/op 9.7239 ms/op 1.38
forkChoice updateHead vc 600000 bc 64 eq 300000 19.728 ms/op 11.895 ms/op 1.66
computeDeltas 500000 validators 300 proto nodes 6.8561 ms/op 3.0617 ms/op 2.24
computeDeltas 500000 validators 1200 proto nodes 6.7339 ms/op 2.9465 ms/op 2.29
computeDeltas 500000 validators 7200 proto nodes 6.6750 ms/op 3.0148 ms/op 2.21
computeDeltas 750000 validators 300 proto nodes 9.9047 ms/op 4.7349 ms/op 2.09
computeDeltas 750000 validators 1200 proto nodes 9.6191 ms/op 4.7388 ms/op 2.03
computeDeltas 750000 validators 7200 proto nodes 9.8361 ms/op 4.5791 ms/op 2.15
computeDeltas 1400000 validators 300 proto nodes 19.332 ms/op 9.4619 ms/op 2.04
computeDeltas 1400000 validators 1200 proto nodes 20.279 ms/op 8.9484 ms/op 2.27
computeDeltas 1400000 validators 7200 proto nodes 23.517 ms/op 9.1763 ms/op 2.56
computeDeltas 2100000 validators 300 proto nodes 34.157 ms/op 13.958 ms/op 2.45
computeDeltas 2100000 validators 1200 proto nodes 35.653 ms/op 13.560 ms/op 2.63
computeDeltas 2100000 validators 7200 proto nodes 37.057 ms/op 13.674 ms/op 2.71
computeProposerBoostScoreFromBalances 500000 validators 4.7391 ms/op 3.2580 ms/op 1.45
computeProposerBoostScoreFromBalances 750000 validators 4.8133 ms/op 3.2912 ms/op 1.46
computeProposerBoostScoreFromBalances 1400000 validators 4.2621 ms/op 3.2024 ms/op 1.33
computeProposerBoostScoreFromBalances 2100000 validators 4.3394 ms/op 3.2294 ms/op 1.34
altair processAttestation - 250000 vs - 7PWei normalcase 6.9362 ms/op 1.6979 ms/op 4.09
altair processAttestation - 250000 vs - 7PWei worstcase 8.8468 ms/op 2.8145 ms/op 3.14
altair processAttestation - setStatus - 1/6 committees join 271.74 us/op 122.77 us/op 2.21
altair processAttestation - setStatus - 1/3 committees join 500.85 us/op 210.54 us/op 2.38
altair processAttestation - setStatus - 1/2 committees join 726.36 us/op 364.88 us/op 1.99
altair processAttestation - setStatus - 2/3 committees join 1.0285 ms/op 404.38 us/op 2.54
altair processAttestation - setStatus - 4/5 committees join 1.0630 ms/op 691.76 us/op 1.54
altair processAttestation - setStatus - 100% committees join 1.2991 ms/op 647.73 us/op 2.01
altair processBlock - 250000 vs - 7PWei normalcase 17.704 ms/op 12.047 ms/op 1.47
altair processBlock - 250000 vs - 7PWei normalcase hashState 53.841 ms/op 40.059 ms/op 1.34
altair processBlock - 250000 vs - 7PWei worstcase 49.750 ms/op 49.499 ms/op 1.01
altair processBlock - 250000 vs - 7PWei worstcase hashState 112.79 ms/op 136.43 ms/op 0.83
phase0 processBlock - 250000 vs - 7PWei normalcase 4.6698 ms/op 4.0689 ms/op 1.15
phase0 processBlock - 250000 vs - 7PWei worstcase 47.382 ms/op 34.489 ms/op 1.37
altair processEth1Data - 250000 vs - 7PWei normalcase 770.58 us/op 750.49 us/op 1.03
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 20.684 us/op 5.4010 us/op 3.83
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 92.088 us/op 52.845 us/op 1.74
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 28.566 us/op 14.676 us/op 1.95
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 20.402 us/op 9.1200 us/op 2.24
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 229.38 us/op 154.45 us/op 1.49
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.7886 ms/op 1.5432 ms/op 1.16
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.8341 ms/op 1.0401 ms/op 1.76
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.9616 ms/op 1.4502 ms/op 1.35
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 4.0171 ms/op 2.2438 ms/op 1.79
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.8586 ms/op 1.7352 ms/op 1.65
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 6.5370 ms/op 3.3702 ms/op 1.94
Tree 40 250000 create 438.44 ms/op 272.28 ms/op 1.61
Tree 40 250000 get(125000) 217.43 ns/op 105.15 ns/op 2.07
Tree 40 250000 set(125000) 1.1898 us/op 732.16 ns/op 1.63
Tree 40 250000 toArray() 23.698 ms/op 14.880 ms/op 1.59
Tree 40 250000 iterate all - toArray() + loop 26.621 ms/op 14.651 ms/op 1.82
Tree 40 250000 iterate all - get(i) 75.877 ms/op 40.548 ms/op 1.87
MutableVector 250000 create 16.896 ms/op 11.686 ms/op 1.45
MutableVector 250000 get(125000) 7.8070 ns/op 5.3550 ns/op 1.46
MutableVector 250000 set(125000) 763.01 ns/op 196.76 ns/op 3.88
MutableVector 250000 toArray() 4.5575 ms/op 2.5651 ms/op 1.78
MutableVector 250000 iterate all - toArray() + loop 4.4712 ms/op 2.7324 ms/op 1.64
MutableVector 250000 iterate all - get(i) 1.5694 ms/op 1.3298 ms/op 1.18
Array 250000 create 3.3662 ms/op 2.3689 ms/op 1.42
Array 250000 clone - spread 1.5291 ms/op 9.1842 ms/op 0.17
Array 250000 get(125000) 1.5610 ns/op 1.0410 ns/op 1.50
Array 250000 set(125000) 5.6090 ns/op 1.1490 ns/op 4.88
Array 250000 iterate all - loop 184.92 us/op 147.23 us/op 1.26
effectiveBalanceIncrements clone Uint8Array 300000 54.052 us/op 18.699 us/op 2.89
effectiveBalanceIncrements clone MutableVector 300000 453.00 ns/op 352.00 ns/op 1.29
effectiveBalanceIncrements rw all Uint8Array 300000 208.91 us/op 176.45 us/op 1.18
effectiveBalanceIncrements rw all MutableVector 300000 99.099 ms/op 61.459 ms/op 1.61
phase0 afterProcessEpoch - 250000 vs - 7PWei 120.05 ms/op 77.303 ms/op 1.55
phase0 beforeProcessEpoch - 250000 vs - 7PWei 63.300 ms/op 44.584 ms/op 1.42
altair processEpoch - mainnet_e81889 570.13 ms/op 428.32 ms/op 1.33
mainnet_e81889 - altair beforeProcessEpoch 117.22 ms/op 74.221 ms/op 1.58
mainnet_e81889 - altair processJustificationAndFinalization 20.671 us/op 10.411 us/op 1.99
mainnet_e81889 - altair processInactivityUpdates 7.7939 ms/op 5.9123 ms/op 1.32
mainnet_e81889 - altair processRewardsAndPenalties 80.706 ms/op 58.935 ms/op 1.37
mainnet_e81889 - altair processRegistryUpdates 4.1000 us/op 2.5700 us/op 1.60
mainnet_e81889 - altair processSlashings 1.0300 us/op 601.00 ns/op 1.71
mainnet_e81889 - altair processEth1DataReset 1.0570 us/op 571.00 ns/op 1.85
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.4616 ms/op 1.7600 ms/op 0.83
mainnet_e81889 - altair processSlashingsReset 7.8410 us/op 2.4930 us/op 3.15
mainnet_e81889 - altair processRandaoMixesReset 7.4870 us/op 2.8330 us/op 2.64
mainnet_e81889 - altair processHistoricalRootsUpdate 1.1460 us/op 649.00 ns/op 1.77
mainnet_e81889 - altair processParticipationFlagUpdates 2.3030 us/op 1.4530 us/op 1.58
mainnet_e81889 - altair processSyncCommitteeUpdates 1.1080 us/op 532.00 ns/op 2.08
mainnet_e81889 - altair afterProcessEpoch 124.71 ms/op 80.565 ms/op 1.55
capella processEpoch - mainnet_e217614 2.3857 s/op 2.2132 s/op 1.08
mainnet_e217614 - capella beforeProcessEpoch 541.39 ms/op 429.03 ms/op 1.26
mainnet_e217614 - capella processJustificationAndFinalization 22.562 us/op 11.791 us/op 1.91
mainnet_e217614 - capella processInactivityUpdates 30.624 ms/op 19.446 ms/op 1.57
mainnet_e217614 - capella processRewardsAndPenalties 454.23 ms/op 383.38 ms/op 1.18
mainnet_e217614 - capella processRegistryUpdates 31.913 us/op 13.682 us/op 2.33
mainnet_e217614 - capella processSlashings 800.00 ns/op 591.00 ns/op 1.35
mainnet_e217614 - capella processEth1DataReset 891.00 ns/op 628.00 ns/op 1.42
mainnet_e217614 - capella processEffectiveBalanceUpdates 6.5000 ms/op 3.4986 ms/op 1.86
mainnet_e217614 - capella processSlashingsReset 5.3640 us/op 3.0990 us/op 1.73
mainnet_e217614 - capella processRandaoMixesReset 8.7490 us/op 8.2410 us/op 1.06
mainnet_e217614 - capella processHistoricalRootsUpdate 904.00 ns/op 793.00 ns/op 1.14
mainnet_e217614 - capella processParticipationFlagUpdates 3.1440 us/op 1.3540 us/op 2.32
mainnet_e217614 - capella afterProcessEpoch 320.06 ms/op 227.97 ms/op 1.40
phase0 processEpoch - mainnet_e58758 595.33 ms/op 415.10 ms/op 1.43
mainnet_e58758 - phase0 beforeProcessEpoch 191.30 ms/op 142.03 ms/op 1.35
mainnet_e58758 - phase0 processJustificationAndFinalization 25.436 us/op 16.703 us/op 1.52
mainnet_e58758 - phase0 processRewardsAndPenalties 62.038 ms/op 47.537 ms/op 1.31
mainnet_e58758 - phase0 processRegistryUpdates 17.461 us/op 9.0180 us/op 1.94
mainnet_e58758 - phase0 processSlashings 1.3980 us/op 978.00 ns/op 1.43
mainnet_e58758 - phase0 processEth1DataReset 829.00 ns/op 640.00 ns/op 1.30
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.6086 ms/op 969.53 us/op 1.66
mainnet_e58758 - phase0 processSlashingsReset 4.8400 us/op 4.1260 us/op 1.17
mainnet_e58758 - phase0 processRandaoMixesReset 7.1590 us/op 4.2310 us/op 1.69
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.2380 us/op 997.00 ns/op 1.24
mainnet_e58758 - phase0 processParticipationRecordUpdates 12.767 us/op 5.0010 us/op 2.55
mainnet_e58758 - phase0 afterProcessEpoch 111.55 ms/op 68.392 ms/op 1.63
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.4587 ms/op 1.2471 ms/op 1.97
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.5821 ms/op 1.2880 ms/op 2.00
altair processInactivityUpdates - 250000 normalcase 41.088 ms/op 30.384 ms/op 1.35
altair processInactivityUpdates - 250000 worstcase 31.082 ms/op 26.847 ms/op 1.16
phase0 processRegistryUpdates - 250000 normalcase 14.588 us/op 11.336 us/op 1.29
phase0 processRegistryUpdates - 250000 badcase_full_deposits 506.83 us/op 391.91 us/op 1.29
phase0 processRegistryUpdates - 250000 worstcase 0.5 184.94 ms/op 114.00 ms/op 1.62
altair processRewardsAndPenalties - 250000 normalcase 68.754 ms/op 41.015 ms/op 1.68
altair processRewardsAndPenalties - 250000 worstcase 69.539 ms/op 54.871 ms/op 1.27
phase0 getAttestationDeltas - 250000 normalcase 13.192 ms/op 5.9943 ms/op 2.20
phase0 getAttestationDeltas - 250000 worstcase 12.623 ms/op 8.3034 ms/op 1.52
phase0 processSlashings - 250000 worstcase 138.00 us/op 84.615 us/op 1.63
altair processSyncCommitteeUpdates - 250000 188.18 ms/op 127.03 ms/op 1.48
BeaconState.hashTreeRoot - No change 269.00 ns/op 447.00 ns/op 0.60
BeaconState.hashTreeRoot - 1 full validator 174.80 us/op 130.44 us/op 1.34
BeaconState.hashTreeRoot - 32 full validator 1.8719 ms/op 1.4590 ms/op 1.28
BeaconState.hashTreeRoot - 512 full validator 18.229 ms/op 18.472 ms/op 0.99
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 207.56 us/op 150.25 us/op 1.38
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.8289 ms/op 2.0279 ms/op 1.40
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 35.764 ms/op 32.852 ms/op 1.09
BeaconState.hashTreeRoot - 1 balances 182.07 us/op 118.80 us/op 1.53
BeaconState.hashTreeRoot - 32 balances 1.7739 ms/op 1.1455 ms/op 1.55
BeaconState.hashTreeRoot - 512 balances 14.644 ms/op 10.696 ms/op 1.37
BeaconState.hashTreeRoot - 250000 balances 279.39 ms/op 192.93 ms/op 1.45
aggregationBits - 2048 els - zipIndexesInBitList 25.709 us/op 10.749 us/op 2.39
byteArrayEquals 32 82.455 ns/op 65.390 ns/op 1.26
Buffer.compare 32 58.916 ns/op 39.177 ns/op 1.50
byteArrayEquals 1024 2.3663 us/op 1.7661 us/op 1.34
Buffer.compare 1024 75.951 ns/op 47.677 ns/op 1.59
byteArrayEquals 16384 37.186 us/op 28.330 us/op 1.31
Buffer.compare 16384 277.31 ns/op 223.57 ns/op 1.24
byteArrayEquals 123687377 272.44 ms/op 206.09 ms/op 1.32
Buffer.compare 123687377 9.8319 ms/op 7.7511 ms/op 1.27
byteArrayEquals 32 - diff last byte 79.758 ns/op 65.340 ns/op 1.22
Buffer.compare 32 - diff last byte 58.356 ns/op 42.034 ns/op 1.39
byteArrayEquals 1024 - diff last byte 2.0688 us/op 1.9043 us/op 1.09
Buffer.compare 1024 - diff last byte 74.296 ns/op 48.633 ns/op 1.53
byteArrayEquals 16384 - diff last byte 33.117 us/op 27.886 us/op 1.19
Buffer.compare 16384 - diff last byte 281.76 ns/op 241.23 ns/op 1.17
byteArrayEquals 123687377 - diff last byte 254.87 ms/op 213.52 ms/op 1.19
Buffer.compare 123687377 - diff last byte 7.4734 ms/op 6.4727 ms/op 1.15
byteArrayEquals 32 - random bytes 5.4600 ns/op 5.3600 ns/op 1.02
Buffer.compare 32 - random bytes 62.147 ns/op 39.720 ns/op 1.56
byteArrayEquals 1024 - random bytes 5.5210 ns/op 5.0070 ns/op 1.10
Buffer.compare 1024 - random bytes 60.984 ns/op 37.014 ns/op 1.65
byteArrayEquals 16384 - random bytes 5.3780 ns/op 4.9680 ns/op 1.08
Buffer.compare 16384 - random bytes 61.395 ns/op 36.888 ns/op 1.66
byteArrayEquals 123687377 - random bytes 8.8500 ns/op 8.1300 ns/op 1.09
Buffer.compare 123687377 - random bytes 65.400 ns/op 39.220 ns/op 1.67
regular array get 100000 times 45.278 us/op 41.615 us/op 1.09
wrappedArray get 100000 times 45.715 us/op 41.354 us/op 1.11
arrayWithProxy get 100000 times 14.643 ms/op 9.2633 ms/op 1.58
ssz.Root.equals 55.828 ns/op 55.729 ns/op 1.00
byteArrayEquals 54.731 ns/op 53.145 ns/op 1.03
Buffer.compare 11.450 ns/op 9.8420 ns/op 1.16
shuffle list - 16384 els 7.1541 ms/op 4.7429 ms/op 1.51
shuffle list - 250000 els 104.85 ms/op 71.127 ms/op 1.47
processSlot - 1 slots 18.307 us/op 15.393 us/op 1.19
processSlot - 32 slots 3.2954 ms/op 2.8292 ms/op 1.16
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 56.142 ms/op 52.256 ms/op 1.07
getCommitteeAssignments - req 1 vs - 250000 vc 2.5377 ms/op 2.2709 ms/op 1.12
getCommitteeAssignments - req 100 vs - 250000 vc 3.7372 ms/op 3.2926 ms/op 1.14
getCommitteeAssignments - req 1000 vs - 250000 vc 4.1277 ms/op 3.7173 ms/op 1.11
findModifiedValidators - 10000 modified validators 520.84 ms/op 433.44 ms/op 1.20
findModifiedValidators - 1000 modified validators 428.72 ms/op 334.49 ms/op 1.28
findModifiedValidators - 100 modified validators 416.93 ms/op 298.28 ms/op 1.40
findModifiedValidators - 10 modified validators 422.51 ms/op 343.26 ms/op 1.23
findModifiedValidators - 1 modified validators 427.52 ms/op 300.73 ms/op 1.42
findModifiedValidators - no difference 426.90 ms/op 286.71 ms/op 1.49
compare ViewDUs 5.0160 s/op 3.9242 s/op 1.28
compare each validator Uint8Array 1.8547 s/op 1.9307 s/op 0.96
compare ViewDU to Uint8Array 1.5458 s/op 786.47 ms/op 1.97
migrate state 1000000 validators, 24 modified, 0 new 1.2610 s/op 748.66 ms/op 1.68
migrate state 1000000 validators, 1700 modified, 1000 new 1.2848 s/op 941.61 ms/op 1.36
migrate state 1000000 validators, 3400 modified, 2000 new 1.5733 s/op 1.1895 s/op 1.32
migrate state 1500000 validators, 24 modified, 0 new 957.52 ms/op 711.67 ms/op 1.35
migrate state 1500000 validators, 1700 modified, 1000 new 1.2078 s/op 931.91 ms/op 1.30
migrate state 1500000 validators, 3400 modified, 2000 new 1.2974 s/op 1.0465 s/op 1.24
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.3500 ns/op 4.3100 ns/op 1.01
state getBlockRootAtSlot - 250000 vs - 7PWei 699.75 ns/op 405.66 ns/op 1.72
computeProposers - vc 250000 10.421 ms/op 6.1354 ms/op 1.70
computeEpochShuffling - vc 250000 106.13 ms/op 70.765 ms/op 1.50
getNextSyncCommittee - vc 250000 178.36 ms/op 113.51 ms/op 1.57
computeSigningRoot for AttestationData 29.976 us/op 26.077 us/op 1.15
hash AttestationData serialized data then Buffer.toString(base64) 2.4116 us/op 1.2731 us/op 1.89
toHexString serialized data 1.1124 us/op 794.82 ns/op 1.40
Buffer.toString(base64) 248.27 ns/op 152.03 ns/op 1.63

by benchmarkbot/action

@nazarhussain nazarhussain requested a review from nflaig December 21, 2023 14:01
@nazarhussain nazarhussain self-assigned this Dec 21, 2023
nflaig
nflaig reviewed Dec 21, 2023
View reviewed changes
packages/params/package.json
"test:browsers": "yarn test:browsers:chrome && yarn test:browsers:firefox && yarn test:browsers:electron",
"test:browsers:chrome": "vitest --run --browser chrome --config ./vitest.browser.config.ts --dir test/unit",
"test:browsers:firefox": "vitest --run --browser firefox --config ./vitest.browser.config.ts --dir test/unit",
"test:browsers:electron": "echo 'Electron tests will be introduced back in the future as soon vitest supports electron.'",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a question regarding this message, did we have electron tests in the past? If not, might wanna rephrase this

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nazarhussain ☝️

@nazarhussain nazarhussain merged commit fbdd1a4 into unstable Dec 21, 2023
@nazarhussain nazarhussain deleted the nh/vitest-params branch December 21, 2023 16:34
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.14.0 🎉

ensi321 pushed a commit to ensi321/lodestar that referenced this pull request Jan 22, 2024
@nazarhussain @ensi321
test: migrate params tests to vitest (ChainSafe#6217)
7e9fe5f 
* Migrate params tests to vitest

* Fix the assertion message
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nflaig nflaig nflaig approved these changes

Assignees

@nazarhussain nazarhussain

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@nazarhussain @wemeetagain @nflaig