Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: don't panic if new flat head is not reachable #7724

Merged
merged 12 commits into from
Oct 3, 2022

Conversation

Longarithm
Copy link
Member

In tests::client::process_blocks::test_query_final_state, there is a chain with two forks:

///                  /-------> 6
/// 1 -> 2 -> 3 -> 4 
///                  \---> 5

When (5) is postprocessed, flat head moves from (2) to (3).
When then (6) is postprocessed, the last final block it sees is (2). It attempts to move flat head from (3) to (2) and the node crashes.

Also we can't simply take final head which always moves forward, due to catchups. So we add comparison of old and new heights.

Testing

https://buildkite.com/nearprotocol/nearcore-flat-state/builds/62#01838983-7f4d-4bd1-9eea-a376da2e0a62

@Longarithm Longarithm self-assigned this Sep 29, 2022
@Longarithm Longarithm requested a review from a team as a code owner September 29, 2022 13:52
@Longarithm
Copy link
Member Author

Longarithm commented Sep 30, 2022

@mzhangmzz moved heights comparison inside update_flat_head.
As long as we are looking at FlatStorageState safety - I think we should include flat head in BlockNotSupported error, otherwise someone can update flat head before we print an error, which can make error text confusing. I proposed the fix here.

};
// If new head height is not greater than current head height, new head is not a child of flat head, so we
// return Ok.
if new_head_height <= current_head_height {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I wonder if it's cleaner to not change this function and check at the its callsite to ignore the "BlockNotFoundError". A lot of the checks you did here is included at get_deltas_between_blocks

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I agree. If get_deltas_between_blocks fails, then new code fails as well.

@near-bulldozer near-bulldozer bot merged commit 1f09ac7 into master Oct 3, 2022
@near-bulldozer near-bulldozer bot deleted the fs-final-head branch October 3, 2022 14:12
@Longarithm Longarithm changed the title fix: compare head heights before updating flat storage fix: don't panic if new flat head is not reachable Oct 3, 2022
nikurt pushed a commit that referenced this pull request Nov 9, 2022
In `tests::client::process_blocks::test_query_final_state`, there is a chain with two forks:

```
///                  /-------> 6
/// 1 -> 2 -> 3 -> 4 
///                  \---> 5
```
When (5) is postprocessed, flat head moves from (2) to (3).
When then (6) is postprocessed, the last final block it sees is (2). It attempts to move flat head from (3) to (2) and the node crashes.

Also we can't simply take final head which always moves forward, due to catchups. So we add comparison of old and new heights.

## Testing

https://buildkite.com/nearprotocol/nearcore-flat-state/builds/62#01838983-7f4d-4bd1-9eea-a376da2e0a62
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants