Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

service/block: Implement ring buffer to cache latest X blocks for easy sampling access #120

Closed
renaynay opened this issue Oct 8, 2021 · 2 comments
Labels
architecture Architecture / design-related issues area:block Raw blocks and erasure coded blocks area:shares Shares and samples

Comments

@renaynay
Copy link
Member

renaynay commented Oct 8, 2021

Question: Do we store the block data in mem for a while and then pop it out after X number of new blocks?

@mattdf 's answer: This sounds reasonable to me, perhaps do what geth does and allow the user to specify a command line argument that sets the max RAM allowed to be used by the cache, so that very big blocks don't DoS nodes in the future. If a node has to restart for whatever reason, then you'd also have to rebuild this cache by reading the disk. I think some kind of ring buffer struct would be fine here since for sampling you'll want to access the entire block.

Ref

@renaynay renaynay added area:shares Shares and samples area:block Raw blocks and erasure coded blocks architecture Architecture / design-related issues labels Oct 8, 2021
@Wondertan
Copy link
Member

Wondertan commented Oct 15, 2021

We store chain blocks as IPFS blocks/shares rn and for caching to be implemented we can rely on IPFS caching on the share level by #108 and as implemented in #136.

By applying share-level caching we also power up caching for DASing and block syncing. Though we still may have a ring buffer for block pointers to keep them in-mem for some time.

We should elaborately consider caching for blocks here in both per block and per share levels.

@Wondertan
Copy link
Member

We now rely on shares and we have shares being cached in #108

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
architecture Architecture / design-related issues area:block Raw blocks and erasure coded blocks area:shares Shares and samples
Projects
None yet
Development

No branches or pull requests

2 participants