Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sync latest changes from terence's eip rebase repo #12

Closed
wants to merge 127 commits into from
Closed

sync latest changes from terence's eip rebase repo #12

wants to merge 127 commits into from

Conversation

roberto-bayardo
Copy link
Collaborator

@roberto-bayardo roberto-bayardo commented Aug 17, 2022 •
edited
Loading

pull latest changes from terence's eip4844 rebasing repo into the branch used by Inphi/eip4844-interop repo.

terencechain and others added 30 commits August 1, 2022 14:43
@terencechain
Rename pow to execution (prysmaticlabs#11135)
699bfdf 
* Rename pow to execution

* Fix complain

* Fix complain
@prestonvanloon
Static Analyzer: require log.WithError instead of log.Errorf("bad: %v…
d17826f 
…", err) (prysmaticlabs#11143)

* Add analysis template with failing test

* Works for the most common use case

* Progress on tool and more test cases

* Improvements

* handle nil

* works for the most part

* Fix some TODOs in the tool

* Fix some TODOs in the tool
@nisdas @prylabs-bulldozer
Make Batch Gossip Aggregation The Default (prysmaticlabs#11144)
881b5c8 
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
@james-prysm @prylabs-bulldozer
fixing log parsing of public validator key (prysmaticlabs#11142)
c1f89cc 
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
@terencechain
Encapsulate sidecar within signed block interface
a390c74
@potuz @terencechain @nisdas
only viable head is invalid (prysmaticlabs#11117)
4b46dea 
* failing onBlock syncing

* passing merge check

* failing signature verification

* still failing block signature

* mock full bellatrix blocks

* working unit test

* return error from FCU if head fails to update

* move bellatrix block generator

* remove bellatrix signature function

* Add liveness unit tests

* revert removal of sync_aggregate.go

* gaz

* Terence's suggestion

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* go fmt

* Nishant's suggestion

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* Fix build

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
@rkapka
Native Blocks Ep. 2 - Switch usages to new package (prysmaticlabs#10885)
879e310 
* panic in SizeSSZ

* moving slowly

* adapt old code to new interfaces

* return interfaces from factory functions

* replace the rest of WrappedSignedBeaconBlock

* WrappedBeaconBlock

* WrappedBeaconBlockBody

* miscellaneous

* Test_BeaconBlockIsNil

* replace usages of BeaconBlockIsNil

* replace usages of mutator

* fix all build errors

* fix some more issues

* mutator changes

* relax assertions when initializing

* revert changes in object_mapping.go

* allow calling Proto on nil

* Revert "allow calling Proto on nil"

This reverts commit ecc84e4.

* modify Copy and Proto methods

* remove unused var

* fix block batch tests

* correct BUILD file

* Error when initializing nil objects

* one more error fix

* add missing comma

* rename alias to blocktest

* add logging

* error when SignedBeaconBlock is nil

* fix last test

* import fix

* broken

* working

* test fixes

* reduce complexity of processPendingBlocks

* simplified
@potuz
Add unit test for reboot behavior (prysmaticlabs#11151)
a5cab4e 
* Add unit test for reboot behavior

* fix conflicts
@terencechain
Update lighthouse prater bootnodes (prysmaticlabs#11152)
19bc691
@terencechain
Clean up core
37118e7
@rkapka
Native Blocks Ep. 3 - Remove old code (prysmaticlabs#11158)
7f4c694 
* Native Blocks Ep. 3 - Remove old code

* rename back package

* fix errors
@potuz @prylabs-bulldozer
Forkchoice: expose if no tip is viable (prysmaticlabs#11153)
19e4cd3 
* Forkchoice: expose if no tip is viable

This PR changes the behavior on when the node is considered optimistic.
A call to `blockchain.IsOptimistic()` relies solely on forkchoice, if
all tips are invalid, then it's optimistic. If the current headroot is
not in forkchoice then it's optimistic.

A call to `blockchain.IsOptimisticForRoot()` will return true if the
requested root is headroot and it's not found in forkchoice

* update comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
@potuz
Wrap missing state for checkpoint error (prysmaticlabs#11162)
2349899 
* Wrap missing state for checkpoint error

* terence's review
@prestonvanloon
Remove unused proto import (prysmaticlabs#11164)
0c3df40 
* Remove unused proto import

* Update generated go files
@terencechain @prylabs-bulldozer
Clean up: fix typos (prysmaticlabs#11165)
d4ab080 
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
@michaelneuder @james-prysm
Wallet edit CLI Manager migration (prysmaticlabs#11136)
8aa6057 
* Wallet edit CLI Manager migration

* TODO for code deduplication

* s/walletEdit/remoteWalletEdit/g

* s/walletEdit/remoteWalletEdit/g

* remove unused struct field

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
@terencechain @potuz
Can recover from missing state summary in DB (prysmaticlabs#11167)
a823256 
* Can recover from missing state summary in DB

* Tests

* fix tests

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
@potuz @terencechain @prylabs-bulldozer
Revert --experimental-disable-boundary-checks (prysmaticlabs#11160)
cef95dd 
* Revert --experimental-disable-boundary-checks

* add deprecation

* gaz

* spectests

* spectests fix

* fix previous justified checkpoint

* copy previous jcp

* Revert "copy previous jcp"

This reverts commit 6e2b865.

* update previous jcp on on_tick

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
@nisdas @prylabs-bulldozer
Fix Goodbye Codes In Prysm (prysmaticlabs#11168)
e169ce9 
* fix goodbye codes

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
@nisdas
Bring Back Mplex (prysmaticlabs#11169)
82ef9f1 
* bring back

* fix mod
@james-prysm
Validator Client: auth-token path log (prysmaticlabs#11170)
0ed739a
@terencechain @symbolpunk
Improve terminal difficulty has not been reached yet logging (prysm…
c05c187 
…aticlabs#11171)

* Improve terminal difficulty has not been reached yet logging

* Update beacon-chain/execution/check_transition_config.go

Co-authored-by: mick <103775631+symbolpunk@users.noreply.github.com>

Co-authored-by: mick <103775631+symbolpunk@users.noreply.github.com>
@terencechain
Merge branch 'develop' of https://github.com/prysmaticlabs/prysm into…
080810e 
… eip4844
@terencechain
Sync with native block changes
f04b2be
@terencechain
Clean up db and generated pbs
e71124e
@infosecual
Node by root mutex fix (prysmaticlabs#11172)
1323912
@prestonvanloon @prylabs-bulldozer @potuz
Enforce log.WithError(err) static analysis and fix all violations (pr…
9d37596 
…ysmaticlabs#11163)

* Use log.WithError static analysis from prysmaticlabs#11143 and fix all violations

* Fix another log violation after pulling from develop

* Update beacon-chain/sync/pending_blocks_queue.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* @potuz feedback

* Copy paste fail

* fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
@vieyang @prestonvanloon
improve log for public key not found (prysmaticlabs#11174)
ea93a68 
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
@nisdas @james-prysm
Update E2E (prysmaticlabs#11140)
9261da7 
* save progress

* tidy

* Update go.yml

* make CI happy

* gosec

* revert back

* Update go.yml

* change go version

* remove fixed test case

* fix ci

* fix updates

* fix up

* fix race tests

* fix bad mock

* lock it

* fix it

* fix e2e builds

* use gotags

* Revert "use gotags"

This reverts commit 808863f.

* Revert "fix e2e builds"

This reverts commit eb351e7.

* Revert "fix it"

This reverts commit 9e99dac.

* Revert "lock it"

This reverts commit 1a3c60a.

* different approach

* better

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
@terencechain
Use correct fork and genesis root to verify bid (prysmaticlabs#11176)
4e225fc
rauljordan and others added 13 commits August 19, 2022 09:26
@rauljordan @rkapka
Add a Generate Genesis State Command to Prysmctl (prysmaticlabs#11259)
20ed47a 
* genesis tool

* done with tool

* delete old tool

* no fatal

* fix up

* Update cmd/prysmctl/testnet/generate_genesis.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* radek feedback

* more feedback

* required

* fix up

* gaz

* radek feedback

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
@nisdas
Tag Gosec To Last Stable Version (prysmaticlabs#11267)
56c48b4
@nisdas @prylabs-bulldozer
Switch Down Libp2p Logging Level By Default (prysmaticlabs#11266)
029c81a 
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
@terencechain @prylabs-bulldozer
Update spec tests to v1.2.0-rc.3 (prysmaticlabs#11261)
f089405 
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
@potuz @terencechain @prylabs-bulldozer
DoublyLinkedTree double locks (prysmaticlabs#11269)
8fecfae 
* DoublyLinkedTree double locks

* fix updateCheckpoints

* add comment

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
@potuz @rauljordan @prylabs-bulldozer
Call on_tick on every skipped slot for spectests (prysmaticlabs#11262)
1775cf8 
New spectests ensure that we call on_tick for every slot between the
last tick and the current one. This PR fixes that

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
@potuz @terencechain @prylabs-bulldozer
Protoarray deadlocks (prysmaticlabs#11271)
76f9587 
Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
@james-prysm
Gas_Limit as string (prysmaticlabs#11264)
5e2498b 
* setting gas limit as string in json and yaml

* adding more tests

* fixing unit test

* fixing string conversion
@potuz @rauljordan
@prylabs-bulldozer @terencechain
Insert attestations on block (prysmaticlabs#11260)
e687fff 
* Insert attestations from blocks to forkchoice

* do not insert twice

* use OnAttestation instead

* Revert "use OnAttestation instead"

This reverts commit 577c37b.

* unit test

* fix test

* Update beacon-chain/blockchain/receive_block_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terencechain <terence@prysmaticlabs.com>
@nisdas
Bring back old execution flag as an alias (prysmaticlabs#11275)
2c89ce8
@nisdas
Handle Deprecated Flags Correctly (prysmaticlabs#11276)
b787fd8
@terencechain
Sync with upstream
a22d62e
@terencechain
Can build
5c2ebec
@roberto-bayardo roberto-bayardo marked this pull request as draft August 22, 2022 02:19
@roberto-bayardo
Copy link
Collaborator Author

The following commit is the first one where I notice the e2e test starts failing:

"Encapsulate sidecar within signed block interface"

The sidecar pointer is added at the top level and since the Block type has a non-pointer receiver, setting the sidecar is a no-op (it modifies only a temporary copy).

There's a good chance this is fixed in a later change as it seems later commits affect this particular code quite a bit, so not sure if this is going to end up relevant to the current issue.

@Inphi
Copy link
Owner

Inphi commented Aug 23, 2022

There's a good chance this is fixed in a later change as it seems later commits affect this particular code quite a bit, so not sure if this is going to end up relevant to the current issue.

Do you mean this bug could be fixed on the latest upstream? Prysm just put out a 3.0 merge release. Worth rebasing that since that should be stable enough.

@terencechain
Copy link

There's a good chance this is fixed in a later change as it seems later commits affect this particular code quite a bit, so not sure if this is going to end up relevant to the current issue.

Worth rebasing that since that should be stable enough.

I've rebased it here: terencechain#1

@roberto-bayardo roberto-bayardo force-pushed the roberto-bayardo/update-rebase branch from 75ae2ac to f0da81e Compare August 25, 2022 18:39
@roberto-bayardo
Copy link
Collaborator Author

I just rebased against the latest version of @terencechain 's eip repo to get the v3 changes in but chain is still not advancing. Will continue digging....

@roberto-bayardo
Copy link
Collaborator Author

Let me close this PR as it's become quite a mess. I'll re-open when I get things in a better state.

@terencechain
Copy link

terencechain commented Aug 28, 2022 •
edited
Loading

clone https://github.com/Inphi/eip4844-interop and then "make devnet-up" to make sure it's working (you should see the beacon-node advancing properly....)

then cd into the prysm submodule and checkout my branch (roberto-bayardo/update-rebase) ... "make devnet-clean" followed by "make devnet-up" and then you should see the beacon-node failing to make progress.

make devnet-up failed #0 0.329 go: go.mod file not found in current directory or any parent directory; see 'go help modules'... Will look into this...

Resolved. Sorry for the noise. Fixing the flags now

@roberto-bayardo
Copy link
Collaborator Author

roberto-bayardo commented Aug 28, 2022 via email

@terencechain
Copy link

I've taken a closer look...
The first issue is it fails to propose a block at the start. Times out when broadcasting. --min-sync-peers=0 in docker-compose didn't work for me for some reason. I manually added it to run_beacon_node.sh, and that's been resolved
The second issue is post-Bellatrix, the node is always in optimistic mode. This is usually due to the execution layer client up and how ttd is configured. I haven't had a chance to look into this yet

@roberto-bayardo
Copy link
Collaborator Author

yeah I was seeing that too so just for grins I coded "isOptimistic" to always return false, but at that point I just see it re-proposing the fork block over and over again.

@Inphi
Copy link
Owner

Inphi commented Aug 28, 2022

The old clean-slate branch made a couple changes to optimistic sync and fork choice. We no longer need those changes with coupled blocks. Wanna make sure that those changes werent merged here.

@roberto-bayardo roberto-bayardo deleted the roberto-bayardo/update-rebase branch November 17, 2022 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Inphi Inphi Inphi approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.