Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove DBCol::ChunkPerHeightShard #7671

Merged
merged 1 commit into from
Sep 26, 2022
Merged

Conversation

robin-near
Copy link
Contributor

This column is only used by ShardsManager to reject an incoming partial chunk. Previously, the column is written in two ways:

  1. When a partial encoded chunk's header is validated by ShardsManager, it will write this column. As long as the header is valid, this column will map the (height, shard ID) to the chunk's hash.
  2. When a block is processed by the Client, we will write each chunk in that block to this column.

This PR moves the mapping into memory. That means that upon a restart we will have an empty mapping. This should be OK: consider two nodes, one running the old version and one running the in-memory version; since the old version's mapping is a superset of the new version's in-memory mapping, there are two possible cases when the two nodes obtain different lookup results:

  1. the old node sees a chunk hash, but the new node does not see any. This can happen for two reasons: (a) if the hash was supposed to be in memory but the node had just restarted recently, then at most this can happen for a few heights because there are only a few heights within the height horizon, which means ShardsManager will potentially process a few more chunks than it would have before, which is fine (processing redundant chunks by itself does not affect correctness); (b) if the chunk hash was written by the Client upon processing a block, then because the Client should've waited for the chunks in this block to be completed before writing this, the ShardsManager in the new node should also have written this chunk to the in-memory mapping (and we're doing this check within the horizon so the mapping would not have expired) which is a contradiction.
  2. the old node and new node both see a hash, but the hashes differ. This can result in (a) an extra chunk (the hash the new node sees) being processed by ShardsManager, which is OK; (b) a chunk (that the old node sees) being ignored; however, we only make ShardsManager reject a partial encoded chunk if we didn't actively request for parts/receipts for the chunk (i.e. we would never reject partial encoded chunk responses); so if we needed to complete a chunk as part of a block, we would not have any issues since we actively send requests for these, and if instead we were the block producer collecting chunks, then we would've already had another valid chunk header to work anyway, and our choice of which valid chunk header would be just as arbitrary as the old node.

This PR is motivated by the ShardsManager refactoring to move it out to its own actor; to that end we need to split the ChainStore because the Client owns the mutable ChainStore. Since we decided to keep chunk-persisting writes on the Client side, the ShardsManager side's only mutation right now is this ChunkPerHeightShard column. Removing this column means the ShardsManager will not need any store mutations which makes the split much easier.

@robin-near robin-near requested a review from a team as a code owner September 22, 2022 23:10
@robin-near robin-near force-pushed the async-3 branch 5 times, most recently from a2dc8bb to b379c50 Compare September 23, 2022 15:56
@robin-near robin-near force-pushed the async-3 branch 2 times, most recently from bf9670f to 3a93d80 Compare September 26, 2022 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Move ShardsManager chunk management logic to a separate Actix thread
2 participants