Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add broadcast_validation to block publishing #12432

Merged
merged 28 commits into from
Jun 1, 2023
Merged

Conversation

rkapka
Copy link
Contributor

@rkapka rkapka commented May 19, 2023

What type of PR is this?

Feature

What does this PR do? Why is it needed?

This PR implements ethereum/beacon-APIs#317, which defines two new endpoints: publishBlockV2 and publishBlindedBlockV2.

Things to keep in mind when reading the code:

  • I implemented the new endpoints as HTTP handlers, whereas existing publishBlock and publishBlindedBlock are implemented as gRPC handlers. To make the code more manageable, it would be a good idea to call the HTTP handler directly from the gRPC handler because the only difference is an additional query parameter. This will be done in a follow-up PR.
  • rpchelpers.ValidateSync used gRPC-specific constructs, so I renamed it to ValidateSyncGRPC and created a new HTTP version ValidateSyncHTTP.
  • Consensus and equivocation validation is based on @terencechain 's PR [DO NOT MERGE]: Apply consensus and equivocation check before broadcast #12335. Comparing to Terence's code, I reduced the scope of the cache because it is only used in the validator server.
  • Example JSON blocks in tests are based on the Beacon API spec.
  • I created yet another set of structs in structs.go, but maybe it would have been a better idea to reuse the ones from API Middleware. I didn't want to do this because we want to remove all middleware code in the future.
  • Both endpoints have a oneOf specification, which means that values representing several versions of blocks can be submitted. This makes JSON unmarshalling problematic because missing fields get ignored (as an example, a JSON representing a Phase 0 block will get unmarshalled to a Capella block without errors, but it shouldn't). To fix the issue I imported the https://github.com/go-playground/validator package to validate that fields are not empty. This package looks very useful and we can later extend validations to check for things like the length of a hex string.

@rkapka rkapka added Ready For Review API Api related tasks labels May 25, 2023
@rkapka rkapka marked this pull request as ready for review May 25, 2023 17:03
@rkapka rkapka requested a review from a team as a code owner May 25, 2023 17:03
}

func (vs *Server) validateEquivocation(blk interfaces.ReadOnlyBeaconBlock) error {
if vs.seenProposerIndex(blk.Slot(), blk.ProposerIndex()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no longer needs to use the cache here, just have to check if there exists a block of same slot in forkchoice store

Comment on lines 350 to 362
if validation == types.Consensus {
if err = vs.validateConsensus(ctx, blk); err != nil {
return nil, errors.Wrap(err, types.ErrConsensusValidationFailed.Error())
}
} else if validation == types.ConsensusAndEquivocation {
if err = vs.validateConsensus(ctx, blk); err != nil {
return nil, errors.Wrap(err, types.ErrConsensusValidationFailed.Error())
}
if err = vs.validateEquivocation(blk.Block()); err != nil {
return nil, errors.Wrap(err, types.ErrEquivocationValidationFailed.Error())
}
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer these checks to be outside of v1alpha1 pkg. It should be in the beacon-api pkg.
1.) It's less risky to current validator code
2.) v1alpha1 will be deprecated eventually then we shouldn't have to move these checks again

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the spec:

consensus_and_equivocation: the same as consensus, with an extra equivocation check immediately before the block is broadcast

Broadcast is done here, so it's hard to move this check anywhere else. Also I believe the function will stay as it is even when we deprecate v1alpha1. It will be just no longer a method on the v1alpha1 server, but rather a regular function either in Beacon APIs or somewhere else.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not break it up and call Broadcast directly like we did here: https://github.com/prysmaticlabs/prysm/pull/12335/files#diff-50db3f4907c1c24a971fc3a1ef6264d8386f7ec2b42cf60928238b6dfc94d058R1044

I think we should delete v1alpha1 eventually so avoid adding more dependency to it will be nice.

Copy link
Member

@terencechain terencechain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main concern is having these checks in ProposeGenericBeaconBlock, I think it should be scoped within SubmitBlock & SubmitBlockSSZ. Reasons being existence risk and future compatibility

@rkapka rkapka changed the title publishBlockV2 Add broadcast_validation to block publishing May 25, 2023
@rkapka rkapka changed the title Add broadcast_validation to block publishing Add broadcast_validation to block publishing May 25, 2023
Comment on lines 155 to 176
isSyncing, syncDetails, err := helpers.ValidateSyncHTTP(r.Context(), bs.SyncChecker, bs.HeadFetcher, bs.TimeFetcher, bs.OptimisticModeFetcher)
if err != nil {
errJson := &network.DefaultErrorJson{
Message: "Could not check if node is syncing: " + err.Error(),
Code: http.StatusInternalServerError,
}
network.WriteError(w, errJson)
return
}
if isSyncing {
msg := "Beacon node is currently syncing and not serving request on that endpoint"
details, err := json.Marshal(syncDetails)
if err == nil {
msg += " Details: " + string(details)
}
errJson := &network.DefaultErrorJson{
Message: msg,
Code: http.StatusServiceUnavailable,
}
network.WriteError(w, errJson)
return
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this whole part is duplicated and we'll use it in a few more places. Might as well abstract into its own function

}

var broadcastValidation types.BroadcastValidation
switch r.URL.Query().Get("broadcast_validation") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

magic strings here, same with the cases. Might be useful to make these variables instead

rkapka and others added 2 commits May 29, 2023 17:06
@prylabs-bulldozer prylabs-bulldozer bot merged commit c32b581 into develop Jun 1, 2023
@prylabs-bulldozer prylabs-bulldozer bot deleted the publishBlockV2 branch June 1, 2023 11:22
james-prysm pushed a commit that referenced this pull request Jun 7, 2023
* day 1

* day 2

* day 2+

* day 3

* day 4

* making bazel happy

* PublishBlindedBlockV2

* remove file

* use lock in insertSeenProposerIndex

* remove EquivocationChecker interface

* update deps.bzl

* remove middleware json tags

* go mod tidy

* remove redundant return statements

* validate in handler

* improvements

* extract common code

* remove import

* sync test fix

* Update beacon-chain/rpc/eth/beacon/handlers.go

Co-authored-by: terencechain <terence@prysmaticlabs.com>

---------

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Api related tasks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants