feat: pop bench pallet logic implementation

[chungquantin]

Features

• Add a new pop-cli command bench to benchmark pallets & parachain.
• Integrates with frame-benchmarking-cli as a subcommand.
• Run benchmarking on pallets without node binary required. Only runtime needed, inspired by FRAME Omni Bencher.
• Locate the runtime WASM binary in the project and trigger runtime build - PR: feat: benchmark existing runtime binary and select policy, presets #411
• Select runtime, genesis builder policy and genesis config preset. If there is no preset available for the runtime, throws error before running the frame-benchmarking-cli command. feat: add genesis preset check #422
• Nice to have: Support both benchmarking with chain spec and runtime. FRAME Omni Bencher only support benchmarking with runtime wasm. This addition enhances the user experience by allowing the developers to provide either the chain spec file path or the runtime wasm file.

How to test?

Use the below command in your pop-cli project to benchmark the base_parachain.wasm.

cargo run -- bench pallet --runtime=./crates/pop-cli/test-resources/base_parachain.wasm --pallet=pallet_timestamp --extrinsic= --output=./weights.rs

Or run a command pop bench pallet

Benchmarking completed successfully

Benchmark a pallet or parachain

Usage: pop bench
       pop bench <COMMAND>

Commands:
  pallet  Benchmark the extrinsics of FRAME Pallets
  help    Print this message or the help of the given subcommand(s)

To run benchmarking, builds the parachain runtime and specify the runtime path with --runtime

┌   Pop CLI : Benchmarking your pallets
│
▲  NOTE: the `pop bench pallet` is not yet battle tested - double check the results.
│
▲  NOTE: this may take some time...
│
[2025-02-12T17:27:41Z INFO  pallet_collator_selection::pallet] assembling new collators for new session 0 at #0
[2025-02-12T17:27:41Z INFO  pallet_collator_selection::pallet] assembling new collators for new session 1 at #0
[2025-02-12T17:27:42Z INFO  polkadot_sdk_frame::benchmark::pallet] Loading WASM from file
[2025-02-12T17:27:57Z INFO  polkadot_sdk_frame::benchmark::pallet] [  0 % ] Starting benchmark: pallet_timestamp::set
[2025-02-12T17:27:57Z INFO  polkadot_sdk_frame::benchmark::pallet] [ 50 % ] Starting benchmark: pallet_timestamp::on_finalize
Pallet: "pallet_timestamp", Extrinsic: "set", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: `Timestamp::Now` (r:1 w:1)
Proof: `Timestamp::Now` (`max_values`: Some(1), `max_size`: Some(8), added: 503, mode: `MaxEncodedLen`)
Storage: `Aura::CurrentSlot` (r:1 w:0)
Proof: `Aura::CurrentSlot` (`max_values`: Some(1), `max_size`: Some(8), added: 503, mode: `MaxEncodedLen`)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=       97
              µs

Reads = 2
Writes = 1
Recorded proof Size = 86

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=       97
              µs

Reads = 2
Writes = 1
Recorded proof Size = 86

Pallet: "pallet_timestamp", Extrinsic: "on_finalize", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=       46
              µs

Reads = 0
Writes = 0
Recorded proof Size = 57

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=       46
              µs

Reads = 0
Writes = 0
Recorded proof Size = 57

⚙  Weight file is generated to ./weights.rs
│
└  Benchmark completed successfully!

Failed with error

┌   Pop CLI : Benchmarking your pallets
│
▲  NOTE: the `pop bench pallet` is not yet battle tested - double check the results.
│
▲  NOTE: this may take some time...
│
└  Failed to run benchmarking: No such file or directory (os error 2)
└  Benchmark completed successfully!

[sc-2881]

[codecov]

Codecov Report

Attention: Patch coverage is 72.09302% with 36 lines in your changes missing coverage. Please review.

Project coverage is 75.33%. Comparing base (68ec6d5) to head (9d7c1b7).

Files with missing lines	Patch %	Lines
crates/pop-cli/src/commands/bench/mod.rs	76.19%	12 Missing and 13 partials ⚠️
crates/pop-cli/src/common/prompt.rs	64.28%	0 Missing and 5 partials ⚠️
crates/pop-cli/src/main.rs	0.00%	5 Missing ⚠️
crates/pop-cli/src/commands/mod.rs	0.00%	1 Missing ⚠️

@@                           Coverage Diff                           @@
##           chungquantin/feat-pop_bench_command     #407      +/-   ##
=======================================================================
- Coverage                                75.34%   75.33%   -0.01%     
=======================================================================
  Files                                       64       67       +3     
  Lines                                    13973    14074     +101     
  Branches                                 13973    14074     +101     
=======================================================================
+ Hits                                     10528    10603      +75     
- Misses                                    2098     2116      +18     
- Partials                                  1347     1355       +8     

Files with missing lines	Coverage Δ
crates/pop-cli/src/commands/call/chain.rs	74.82% <ø> (+0.17%)	⬆️
crates/pop-cli/src/commands/call/contract.rs	80.53% <ø> (+0.23%)	⬆️
crates/pop-parachains/src/bench.rs	100.00% <100.00%> (ø)
crates/pop-cli/src/commands/mod.rs	24.61% <0.00%> (-0.39%)	⬇️
crates/pop-cli/src/common/prompt.rs	64.28% <64.28%> (ø)
crates/pop-cli/src/main.rs	40.50% <0.00%> (-2.74%)	⬇️
crates/pop-cli/src/commands/bench/mod.rs	76.19% <76.19%> (ø)

[chungquantin]

@AlexD10S CI fails but the PR is good for a review.

[AlexD10S]

Great work! The implementation is clean, and I love seeing how you're already handling the UI testing.

I’ve left some comments on areas for improvement, along with a couple of additional tests that could help improve coverage.

One refactor I want to see: In the rest of the commands, we typically separate the core logic (handled in pop_contracts or pop_parachain) from the "frontend logic" (pop-cli crate). In this PR, the logic is just cmd.run_with_spec, but it might be worth following the same approach by adding a generate_benchmarks(cmd) function in pop_parachain crate to handle this logic.