Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reference cached beacon states with WeakRefs #3094

Closed
5 tasks
dapplion opened this issue Sep 7, 2021 · 3 comments
Closed
5 tasks

Reference cached beacon states with WeakRefs #3094

dapplion opened this issue Sep 7, 2021 · 3 comments
Labels
prio-low This is nice to have. scope-memory Issues to reduce and improve memory usage.

Comments

@dapplion
Copy link
Contributor

dapplion commented Sep 7, 2021

Teku does it with good results. Since the cache memory size is technically unbounded this strategy is great for preventing OOMs.

  • The head state should be kept separately with a strong reference.
  • Test how JS chooses to garbage collect multiple WeakRef. By age? By least used?
  • Test performance of WeakRef.unref() if it's called very often (attestation processing)
  • Consider adding a shufflings cache to mitigate the risk of a key state getting dropped
  • Don't regen for attestation verification if the head state already has the correct shuffling

Note: this is a huge en-devour with uncertain payoff. Teku has had issues in the past with this approach as all states disapears and the node death spirals

@dapplion dapplion added the scope-memory Issues to reduce and improve memory usage. label Sep 7, 2021
@q9f q9f added mod1-beaconchain prio-medium Resolve this some time soon (tm). and removed mod1-beaconchain labels Sep 14, 2021
@philknows philknows added prio-high Resolve issues as soon as possible. and removed prio-medium Resolve this some time soon (tm). labels Nov 29, 2021
@dapplion dapplion added prio-medium Resolve this some time soon (tm). and removed prio-high Resolve issues as soon as possible. labels Jun 29, 2022
@philknows
Copy link
Member

Refer to #6008 . May not be required anymore.

@philknows philknows added prio-low This is nice to have. and removed prio-medium Resolve this some time soon (tm). labels Nov 5, 2023
@wemeetagain
Copy link
Member

@twoeths I don't think this is necessary anymore after n-historical-states?

@twoeths
Copy link
Contributor

twoeths commented Oct 11, 2024

yes, block state cache is bounded by max size and checkpoint state cache is bounded by max epoch count

@twoeths twoeths closed this as completed Oct 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
prio-low This is nice to have. scope-memory Issues to reduce and improve memory usage.
Projects
None yet
Development

No branches or pull requests

5 participants