Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: discard non-finalizable blocks #7746

Merged
merged 21 commits into from
Oct 7, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions chain/chain-primitives/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,9 @@ pub enum Error {
/// A challenged block is on the chain that was attempted to become the head
#[error("Challenged block on chain")]
ChallengedBlockOnChain,
/// A challenged block is on the chain that was attempted to become the head
#[error("Block cannot be finalized")]
CannotBeFinalized,
/// IO Error.
#[error("IO Error: {0}")]
IOErr(#[from] io::Error),
Expand Down Expand Up @@ -246,6 +249,7 @@ impl Error {
| Error::ValidatorError(_)
| Error::EpochOutOfBounds(_)
| Error::ChallengedBlockOnChain
| Error::CannotBeFinalized
| Error::StorageError(_)
| Error::GCError(_)
| Error::DBNotFoundErr(_) => false,
Expand Down
38 changes: 38 additions & 0 deletions chain/chain/src/chain.rs
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,9 @@ const MAX_ORPHAN_MISSING_CHUNKS: usize = 5;
#[cfg(feature = "sandbox")]
const ACCEPTABLE_TIME_DIFFERENCE: i64 = 60 * 60 * 24 * 365 * 10000;

// Number of parent blocks traversed to check if the block can be finalized.
const NUM_PARENTS_TO_CHECK_FINALITY: usize = 20;

/// Refuse blocks more than this many block intervals in the future (as in bitcoin).
#[cfg(not(feature = "sandbox"))]
const ACCEPTABLE_TIME_DIFFERENCE: i64 = 12 * 10;
Expand Down Expand Up @@ -1117,6 +1120,38 @@ impl Chain {
Ok(header.signature().verify(header.hash().as_ref(), block_producer.public_key()))
}

/// Optimization which checks if block with the given header can be reached from final head, and thus can be
/// finalized by this node.
/// If this is the case, returns Ok.
/// If we discovered that it is not the case, returns `Error::CannotBeFinalized`.
/// If too many parents were checked, returns Ok to avoid long delays.
fn check_if_finalizable(&self, header: &BlockHeader) -> Result<(), Error> {
let mut header = header.clone();
let final_head = self.final_head()?;
for _ in 0..NUM_PARENTS_TO_CHECK_FINALITY {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about the need for this constant. This actually assumes that in the chain, we would never have a fork with 20 blocks. While this rarely happens, adding this assumption doesn't help much with the implementation in this function, so I don't think it should be added. the check that compares the block height with the final head height should be enough.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This actually assumes that in the chain, we would never have a fork with 20 blocks.

I don't see where we need this assumption. If there is a fork with >20 blocks, it means that we just can't apply our optimization and discard it here. Having constant 20 helps us to discard more useless blocks (see below).

the check that compares the block height with the final head height should be enough.

I think the current check is helpful for scenarios as follows:

     /-> 2 -> 3 -> 5 -> 6 -> 7
   1
(final) 
     \-> 4 -> 8
  • processing blocks (1) - (6) don't change final head, so no blocks can be discarded
  • block (7) moves final head to (5)
  • now block (8) can be discarded during preprocessing, but to know it, we need to look at its parent (4).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. I still don't understand why we need to only check the last 20 blocks. What do you think about just changing the for loop to loop?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned about very long forks for which it can take a while to iterate over parents. Maybe there is some reason not to worry about it, but I don't see it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I put the check to preprocess_block, then these tests are failing:

tests::challenges::challenges_new_head_prev
tests::simple_chain::blocks_at_height

They create some tree of blocks on chain, and the tests fail because some of blocks are being discarded.

Moved it to Client::start_process_block, because it looks like optimization on Client side, and I wouldn't make Chain tests assuming this optimization as well. Even after this, two other tests had to be changed.

// If we reached final head, then block can be finalized.
if header.hash() == &final_head.last_block_hash {
return Ok(());
}
// If we went behind final head, then block cannot be finalized on top of final head.
if header.height() < final_head.height {
return Err(Error::CannotBeFinalized);
}
// Otherwise go to parent block.
header = match self.get_previous_header(&header) {
Ok(header) => header,
Err(_) => {
// We couldn't find previous header. Return Ok because it can be an orphaned block which can be
// connected to canonical chain later.
return Ok(());
}
}
}

// If we traversed too many blocks, return Ok to avoid long delays.
Ok(())
}

/// Validate header. Returns error if the header is invalid.
/// `challenges`: the function will add new challenges generated from validating this header
/// to the vector. You can pass an empty vector here, or a vector with existing
Expand Down Expand Up @@ -1287,6 +1322,9 @@ impl Chain {
if header.challenges_root() != &MerkleHash::default() {
return Err(Error::InvalidChallengeRoot);
}

// Check if block can be finalized.
self.check_if_finalizable(&header)?;
}

Ok(())
Expand Down
54 changes: 54 additions & 0 deletions integration-tests/src/tests/client/process_blocks.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2728,6 +2728,60 @@ fn test_epoch_protocol_version_change() {
assert_eq!(protocol_version, PROTOCOL_VERSION);
}

#[test]
fn test_discard_non_finalizable_block() {
let genesis = Genesis::test(vec!["test0".parse().unwrap(), "test1".parse().unwrap()], 1);
let chain_genesis = ChainGenesis::new(&genesis);
let mut env = TestEnv::builder(chain_genesis)
.runtime_adapters(create_nightshade_runtimes(&genesis, 1))
.build();

let first_block = env.clients[0].produce_block(1).unwrap().unwrap();
env.process_block(0, first_block.clone(), Provenance::PRODUCED);
// Produce, but not process test block on top of block (1).
let non_finalizable_block = env.clients[0].produce_block(6).unwrap().unwrap();
env.clients[0]
.chain
.mut_store()
.save_latest_known(LatestKnown {
height: first_block.header().height(),
seen: first_block.header().raw_timestamp(),
})
.unwrap();

let second_block = env.clients[0].produce_block(2).unwrap().unwrap();
env.process_block(0, second_block.clone(), Provenance::PRODUCED);
// Produce, but not process test block on top of block (2).
let finalizable_block = env.clients[0].produce_block(7).unwrap().unwrap();
env.clients[0]
.chain
.mut_store()
.save_latest_known(LatestKnown {
height: second_block.header().height(),
seen: second_block.header().raw_timestamp(),
})
.unwrap();

// Produce and process two more blocks.
for i in 3..5 {
env.produce_block(0, i);
}

assert_eq!(env.clients[0].chain.final_head().unwrap().height, 2);
// Check that the first test block can't be finalized, because it is produced behind final head.
assert_matches!(
env.clients[0]
.process_block_test(non_finalizable_block.into(), Provenance::NONE)
.unwrap_err(),
Error::CannotBeFinalized
);
// Check that the second test block still can be finalized.
assert_matches!(
env.clients[0].process_block_test(finalizable_block.into(), Provenance::NONE),
Ok(_)
);
}

/// Final state should be consistent when a node switches between forks in the following scenario
/// /-----------h+2
/// h-2 ---- h-1 ------ h
Expand Down