Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(l1): metrics for syncing #2080

Open
wants to merge 11 commits into
base: fix/full-sync
Choose a base branch
from

Conversation

MarcosNicolau
Copy link
Contributor

@MarcosNicolau MarcosNicolau commented Feb 25, 2025

Motivation
Full syncing on Holesky.

Description
Introduce SyncMetrics for full-syncing that logs performance metrics, including:

  • Total sync time: the time taken to fetch headers, bodies, and execute blocks.
  • Average block execution time: calculated as total_time / executed_blocks.
  • Account updates: the time spent applying account updates and its percentage of total sync time.
  • Sync range details: the starting and final block numbers along with their corresponding hashes.
  • Execution blocks: the total number of blocks executed.
  • Performance comparison: the time difference from the previous execution.

Logging intervals are configured at 100, 1,000, 10,000, 100,000, and 1,000,000 blocks.

Example
Here is an example of what the logs look like:

[SYNCING PERF] Last 127 blocks performance:
Total time: 15 seconds
Average block time: 0.118 seconds
Time spent applying account updates: 0 seconds ~= 0% of total time
Started at block: 604188 (hash: 0x56832fd3b921ae73ca1449963d39e884af23330faf8922684c0b4ba23792b96e)
Finished at block: 604315 (hash: 0x28604cb1065ebb27e503ac81225fa55ef18fc55c81fc483aa7ab16517e60f52f)
Execution count: 127
======= Overall, this cycle took 7 seconds with respect to the previous one =======

Closes None

@MarcosNicolau MarcosNicolau requested a review from a team as a code owner February 25, 2025 15:35
Copy link

github-actions bot commented Feb 25, 2025

| File                                                                            | Lines | Diff  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/cmd/ethrex/cli.rs                               | 151   | -34   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/cmd/ethrex/ethrex.rs                            | 393   | -40   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/cmd/ethrex_l2/src/cli.rs                        | 53    | -3    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/cmd/ethrex_l2/src/commands/mod.rs               | 7     | -1    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/blockchain/blockchain.rs                 | 288   | -82   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/blockchain/fork_choice.rs                | 179   | +40   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/blockchain/mempool.rs                    | 505   | -66   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/blockchain/payload.rs                    | 569   | +17   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/common/types/transaction.rs              | 2486  | -27   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/l2/proposer/errors.rs                    | 153   | -2    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/l2/proposer/l1_watcher.rs                | 241   | -4    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/l2/proposer/mod.rs                       | 99    | -4    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/l2/prover/bench/src/main.rs              | 101   | -3    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/p2p/discv4/server.rs          | 677   | -3    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/p2p/network.rs                | 169   | -4    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/p2p/peer_handler.rs           | 516   | +1    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/p2p/rlpx/connection.rs        | 460   | -45   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/p2p/rlpx/eth/transactions.rs  | 289   | -17   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/p2p/rlpx/handshake.rs         | 422   | -2    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/p2p/sync.rs                   | 495   | +62   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/p2p/sync/metrics.rs           | 151   | +151  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/clients/auth/mod.rs       | 155   | +1    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/engine/fork_choice.rs     | 315   | -44   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/engine/payload.rs         | 567   | -58   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/eth/account.rs            | 222   | +1    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/eth/filter.rs             | 548   | -36   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/eth/gas_price.rs          | 121   | -12   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/eth/max_priority_fee.rs   | 106   | -12   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/eth/transaction.rs        | 542   | -34   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/rpc.rs                    | 520   | -115  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/utils.rs                  | 277   | -15   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/api.rs                     | 207   | +207  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/error.rs                   | 47    | +47   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/lib.rs                     | 11    | +11   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/rlp.rs                     | 90    | +90   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/store.rs                   | 1272  | +1272 |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/store_db/in_memory.rs      | 516   | +516  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/store_db/libmdbx.rs        | 927   | +927  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/store_db/mod.rs            | 5     | +5    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/store_db/redb.rs           | 918   | +918  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/trie_db/libmdbx.rs         | 169   | +169  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/trie_db/libmdbx_dupsort.rs | 85    | +85   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/trie_db/mod.rs             | 11    | +11   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/trie_db/redb.rs            | 65    | +65   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/trie_db/redb_multitable.rs | 81    | +81   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/trie_db/test_utils.rs      | 23    | +23   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/trie_db/utils.rs           | 11    | +11   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/utils.rs                   | 56    | +56   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/db.rs                       | 42    | +42   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/error.rs                    | 15    | +15   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/nibbles.rs                  | 224   | +224  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/node.rs                     | 150   | +150  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/node/branch.rs              | 536   | +536  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/node/extension.rs           | 422   | +422  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/node/leaf.rs                | 265   | +265  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/node_hash.rs                | 105   | +105  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/rlp.rs                      | 107   | +107  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/src/main.rs                 | 3     | +3    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/state.rs                    | 73    | +73   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/test_utils.rs               | 68    | +68   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/trie.rs                     | 808   | +808  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/trie_iter.rs                | 91    | +91   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/verify_range.rs             | 620   | +620  |
+---------------------------------------------------------------------------------+-------+-------+

Total lines added: +8296
Total lines removed: 663
Total lines changed: 8959

@MarcosNicolau MarcosNicolau self-assigned this Feb 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant