Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EIP-6110: Add deposits in EL #5055

Merged
merged 43 commits into from
Mar 28, 2023
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
d57d70b
Initial commit
ensi321 Feb 5, 2023
1c8de5a
Merge branch 'main' into eip-6110
ensi321 Feb 19, 2023
559b12a
Added object classes for deposit fields
ensi321 Feb 19, 2023
22d4aa4
Spotless apply
ensi321 Feb 19, 2023
97289b2
Fix up compilation issue
ensi321 Feb 19, 2023
1247429
Fix up compilation issue
ensi321 Feb 19, 2023
2e68cca
Fix up compilation issue
ensi321 Feb 20, 2023
91a851f
Update existed tests
ensi321 Feb 20, 2023
e0eb573
Merge branch 'main' into eip-6110
ensi321 Feb 20, 2023
e161250
Remove feature flag for this EIP for now
ensi321 Feb 23, 2023
ba3e19a
Renmae WithdrawalCredential
ensi321 Feb 25, 2023
0ca48a0
Remove DepositProcessor
ensi321 Feb 25, 2023
8707fc0
Remove all certain validation on deposits
ensi321 Feb 25, 2023
bb66d97
Remove all uncertain validation on deposits
ensi321 Feb 25, 2023
6837699
Merge remote-tracking branch 'remote/main' into eip-6110
ensi321 Feb 25, 2023
ff28502
Fix test
ensi321 Feb 25, 2023
ae25f12
Fix typo
ensi321 Mar 1, 2023
672a745
Fix test
ensi321 Mar 3, 2023
307582d
Update plugin-api hash
ensi321 Mar 3, 2023
d8da6b7
Partial commit
ensi321 Mar 4, 2023
2e07bb7
Partial commit
ensi321 Mar 5, 2023
48cda2a
Added unit test
ensi321 Mar 5, 2023
f645938
SpotlessApply
ensi321 Mar 6, 2023
cae7435
Merge branch 'main' into eip-6110
ensi321 Mar 6, 2023
8cfa587
Fix comment
ensi321 Mar 6, 2023
30d3083
Rename TODO and isExperimentalEipsTime in GenesisState
ensi321 Mar 9, 2023
225b56d
Update copyright info
ensi321 Mar 9, 2023
17ce42b
Remove deposits from createBlock() param
ensi321 Mar 10, 2023
15fd353
Add unstable annotation
ensi321 Mar 11, 2023
ba461ce
Put deposits root after excess gas in BlockHeader
ensi321 Mar 11, 2023
8ae831f
Add deposits root to GetBodiesFromPeerTask.BodyIdentifier
ensi321 Mar 11, 2023
3470caa
Update tests
ensi321 Mar 12, 2023
7065425
Remove DepositWithdrawalCredential
ensi321 Mar 19, 2023
6995414
Update test
ensi321 Mar 19, 2023
7d3fc83
Update test
ensi321 Mar 19, 2023
46fa21a
Merge remote-tracking branch 'remote/main' into eip-6110
ensi321 Mar 19, 2023
ed6f99f
Update test
ensi321 Mar 19, 2023
bce7b56
Apply spotless
ensi321 Mar 19, 2023
e8f5b96
Update hash
ensi321 Mar 19, 2023
33adc69
Fix up some test
ensi321 Mar 19, 2023
da838f6
Add DepositsRoot test and related block files
ensi321 Mar 26, 2023
4efac87
Merge remote-tracking branch 'remote/main' into eip-6110
ensi321 Mar 28, 2023
34453d4
Fix conflict
ensi321 Mar 28, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ public static BlockHeader createBlockHeader(
new BigInteger(block.getNonceRaw().substring(2), 16).longValue(),
null,
null,
null,
blockHeaderFunctions);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@ private void setSyncTarget() {
mock(EthPeer.class),
new org.hyperledger.besu.ethereum.core.BlockHeader(
null, null, null, null, null, null, null, null, 1, 1, 1, 1, null, null, null, 1, null,
null, null));
null, null, null));
}

private void clearSyncTarget() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
import org.hyperledger.besu.ethereum.blockcreation.AbstractBlockCreator;
import org.hyperledger.besu.ethereum.core.BlockHeader;
import org.hyperledger.besu.ethereum.core.BlockHeaderBuilder;
import org.hyperledger.besu.ethereum.core.Deposit;
import org.hyperledger.besu.ethereum.core.Difficulty;
import org.hyperledger.besu.ethereum.core.SealableBlockHeader;
import org.hyperledger.besu.ethereum.core.Transaction;
Expand Down Expand Up @@ -82,17 +83,20 @@ public MergeBlockCreator(
* @param random the random
* @param timestamp the timestamp
* @param withdrawals optional list of withdrawals
* @param deposits optional list of deposits
* @return the block creation result
*/
public BlockCreationResult createBlock(
final Optional<List<Transaction>> maybeTransactions,
final Bytes32 random,
final long timestamp,
final Optional<List<Withdrawal>> withdrawals) {
final Optional<List<Withdrawal>> withdrawals,
final Optional<List<Deposit>> deposits) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the deposits are derived from the transaction receipt logs after the transactions are processed and so should be generated within createBlock rather than passed in.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah doesn't make sense to pass in deposits as these are derived from the state after processing the transactions

Copy link
Contributor Author

@ensi321 ensi321 Mar 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I am not wrong I think in AbstractBlockCreator.createBlock() I will get the deposits from transactionResults and stuff them into the block header and body. But this involves ABI decoding which will be included in the next PR so for now I am gonna leave the deposits to be empty and add a TODO here

return createBlock(
maybeTransactions,
Optional.of(Collections.emptyList()),
withdrawals,
deposits,
Optional.of(random),
timestamp,
false);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
import org.hyperledger.besu.ethereum.core.Block;
import org.hyperledger.besu.ethereum.core.BlockHeader;
import org.hyperledger.besu.ethereum.core.BlockWithReceipts;
import org.hyperledger.besu.ethereum.core.Deposit;
import org.hyperledger.besu.ethereum.core.Difficulty;
import org.hyperledger.besu.ethereum.core.MiningParameters;
import org.hyperledger.besu.ethereum.core.MutableWorldState;
Expand Down Expand Up @@ -241,7 +242,8 @@ public PayloadIdentifier preparePayload(
final Long timestamp,
final Bytes32 prevRandao,
final Address feeRecipient,
final Optional<List<Withdrawal>> withdrawals) {
final Optional<List<Withdrawal>> withdrawals,
final Optional<List<Deposit>> deposits) {

// we assume that preparePayload is always called sequentially, since the RPC Engine calls
// are sequential, if this assumption changes then more synchronization should be added to
Expand All @@ -266,7 +268,8 @@ public PayloadIdentifier preparePayload(
// put the empty block in first
final Block emptyBlock =
mergeBlockCreator
.createBlock(Optional.of(Collections.emptyList()), prevRandao, timestamp, withdrawals)
.createBlock(
Optional.of(Collections.emptyList()), prevRandao, timestamp, withdrawals, deposits)
.getBlock();

BlockProcessingResult result = validateProposedBlock(emptyBlock);
Expand All @@ -288,7 +291,8 @@ public PayloadIdentifier preparePayload(
}
}

tryToBuildBetterBlock(timestamp, prevRandao, payloadIdentifier, mergeBlockCreator, withdrawals);
tryToBuildBetterBlock(
timestamp, prevRandao, payloadIdentifier, mergeBlockCreator, withdrawals, deposits);

return payloadIdentifier;
}
Expand All @@ -309,10 +313,13 @@ private void tryToBuildBetterBlock(
final Bytes32 random,
final PayloadIdentifier payloadIdentifier,
final MergeBlockCreator mergeBlockCreator,
final Optional<List<Withdrawal>> withdrawals) {
final Optional<List<Withdrawal>> withdrawals,
final Optional<List<Deposit>> deposits) {

final Supplier<BlockCreationResult> blockCreator =
() -> mergeBlockCreator.createBlock(Optional.empty(), random, timestamp, withdrawals);
() ->
mergeBlockCreator.createBlock(
Optional.empty(), random, timestamp, withdrawals, deposits);

LOG.debug(
"Block creation started for payload id {}, remaining time is {}ms",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
import org.hyperledger.besu.ethereum.blockcreation.MiningCoordinator;
import org.hyperledger.besu.ethereum.core.Block;
import org.hyperledger.besu.ethereum.core.BlockHeader;
import org.hyperledger.besu.ethereum.core.Deposit;
import org.hyperledger.besu.ethereum.core.Withdrawal;

import java.util.List;
Expand All @@ -41,14 +42,16 @@ public interface MergeMiningCoordinator extends MiningCoordinator {
* @param prevRandao the prev randao
* @param feeRecipient the fee recipient
* @param withdrawals the optional list of withdrawals
* @param deposits the optional list of deposits
* @return the payload identifier
*/
PayloadIdentifier preparePayload(
final BlockHeader parentHeader,
final Long timestamp,
final Bytes32 prevRandao,
final Address feeRecipient,
final Optional<List<Withdrawal>> withdrawals);
final Optional<List<Withdrawal>> withdrawals,
final Optional<List<Deposit>> deposits);

@Override
default boolean isCompatibleWithEngineApi() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
import org.hyperledger.besu.ethereum.chain.PoWObserver;
import org.hyperledger.besu.ethereum.core.Block;
import org.hyperledger.besu.ethereum.core.BlockHeader;
import org.hyperledger.besu.ethereum.core.Deposit;
import org.hyperledger.besu.ethereum.core.Transaction;
import org.hyperledger.besu.ethereum.core.Withdrawal;

Expand Down Expand Up @@ -147,9 +148,10 @@ public PayloadIdentifier preparePayload(
final Long timestamp,
final Bytes32 prevRandao,
final Address feeRecipient,
final Optional<List<Withdrawal>> withdrawals) {
final Optional<List<Withdrawal>> withdrawals,
final Optional<List<Deposit>> deposits) {
return mergeCoordinator.preparePayload(
parentHeader, timestamp, prevRandao, feeRecipient, withdrawals);
parentHeader, timestamp, prevRandao, feeRecipient, withdrawals, deposits);
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@
import org.hyperledger.besu.ethereum.core.BlockHeader;
import org.hyperledger.besu.ethereum.core.BlockHeaderTestFixture;
import org.hyperledger.besu.ethereum.core.BlockWithReceipts;
import org.hyperledger.besu.ethereum.core.Deposit;
import org.hyperledger.besu.ethereum.core.Difficulty;
import org.hyperledger.besu.ethereum.core.MiningParameters;
import org.hyperledger.besu.ethereum.core.Transaction;
Expand Down Expand Up @@ -114,6 +115,7 @@ public class MergeCoordinatorTest implements MergeGenesisConfigHelper {
private static final KeyPair KEYS1 =
new KeyPair(PRIVATE_KEY1, SIGNATURE_ALGORITHM.get().createPublicKey(PRIVATE_KEY1));
private static final Optional<List<Withdrawal>> EMPTY_WITHDRAWALS = Optional.empty();
private static final Optional<List<Deposit>> EMPTY_DEPOSITS = Optional.empty();

private static final long REPETITION_MIN_DURATION = 100;
@Mock MergeContext mergeContext;
Expand Down Expand Up @@ -224,7 +226,8 @@ public void coinbaseShouldMatchSuggestedFeeRecipient() {
System.currentTimeMillis() / 1000,
Bytes32.ZERO,
suggestedFeeRecipient,
EMPTY_WITHDRAWALS);
EMPTY_WITHDRAWALS,
EMPTY_DEPOSITS);

ArgumentCaptor<BlockWithReceipts> blockWithReceipts =
ArgumentCaptor.forClass(BlockWithReceipts.class);
Expand Down Expand Up @@ -262,7 +265,8 @@ public void exceptionDuringBuildingBlockShouldNotBeInvalid()
.doThrow(new MerkleTrieException("missing leaf"))
.doCallRealMethod()
.when(beingSpiedOn)
.createBlock(any(), any(Bytes32.class), anyLong(), eq(Optional.empty()));
.createBlock(
any(), any(Bytes32.class), anyLong(), eq(Optional.empty()), eq(Optional.empty()));
return beingSpiedOn;
};

Expand Down Expand Up @@ -298,6 +302,7 @@ public void exceptionDuringBuildingBlockShouldNotBeInvalid()
System.currentTimeMillis() / 1000,
Bytes32.random(),
suggestedFeeRecipient,
Optional.empty(),
Optional.empty());

verify(willThrow, never()).addBadBlock(any(), any());
Expand Down Expand Up @@ -330,6 +335,7 @@ public void shouldNotRecordProposedBadBlockToBadBlockManager()
System.currentTimeMillis() / 1000,
Bytes32.ZERO,
suggestedFeeRecipient,
Optional.empty(),
Optional.empty());

verify(badBlockManager, never()).addBadBlock(any(), any());
Expand Down Expand Up @@ -362,6 +368,7 @@ public void shouldContinueBuildingBlocksUntilFinalizeIsCalled()
System.currentTimeMillis() / 1000,
Bytes32.ZERO,
suggestedFeeRecipient,
Optional.empty(),
Optional.empty());

blockCreationTask.get();
Expand Down Expand Up @@ -413,6 +420,7 @@ public void blockCreationRepetitionShouldTakeNotLessThanRepetitionMinDuration()
System.currentTimeMillis() / 1000,
Bytes32.ZERO,
suggestedFeeRecipient,
Optional.empty(),
Optional.empty());

blockCreationTask.get();
Expand Down Expand Up @@ -459,6 +467,7 @@ public void shouldRetryBlockCreationOnRecoverableError()
System.currentTimeMillis() / 1000,
Bytes32.ZERO,
suggestedFeeRecipient,
Optional.empty(),
Optional.empty());

blockCreationTask.get();
Expand Down Expand Up @@ -493,6 +502,7 @@ public void shouldStopRetryBlockCreationIfTimeExpired() throws InterruptedExcept
System.currentTimeMillis() / 1000,
Bytes32.ZERO,
suggestedFeeRecipient,
Optional.empty(),
Optional.empty());

try {
Expand Down Expand Up @@ -535,6 +545,7 @@ public void shouldStopInProgressBlockCreationIfFinalizedIsCalled()
System.currentTimeMillis() / 1000,
Bytes32.ZERO,
suggestedFeeRecipient,
Optional.empty(),
Optional.empty());

waitForBlockCreationInProgress.await();
Expand Down Expand Up @@ -581,6 +592,7 @@ public void shouldNotStartAnotherBlockCreationJobIfCalledAgainWithTheSamePayload
timestamp,
Bytes32.ZERO,
suggestedFeeRecipient,
Optional.empty(),
Optional.empty());

final CompletableFuture<Void> task1 = blockCreationTask;
Expand All @@ -591,6 +603,7 @@ public void shouldNotStartAnotherBlockCreationJobIfCalledAgainWithTheSamePayload
timestamp,
Bytes32.ZERO,
suggestedFeeRecipient,
Optional.empty(),
Optional.empty());

assertThat(payloadId1).isEqualTo(payloadId2);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
/*
* Copyright contributors to Hyperledger Besu
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.datatypes;

import static com.google.common.base.Preconditions.checkArgument;

import org.hyperledger.besu.ethereum.rlp.RLPException;
import org.hyperledger.besu.ethereum.rlp.RLPInput;

import com.fasterxml.jackson.annotation.JsonCreator;
import org.apache.tuweni.bytes.Bytes;
import org.apache.tuweni.bytes.DelegatingBytes;

/** A BLS public key. */
public class BLSPublicKey extends DelegatingBytes
implements org.hyperledger.besu.plugin.data.PublicKey {
Comment on lines +27 to +28
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's your thinking with respect to the inheritance hierarchy here?

I would need to understand the class hierarchy a bit more myself, but just throwing out another possible place for this to live: https://github.com/hyperledger/besu/tree/main/crypto/src/main/java/org/hyperledger/besu/crypto

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea I was debating whether to put it under datatypes or crypto. My line of thought is crypto is more dedicated to classes that need crypto-related operations such as encryption and such. BLSPublicKey is mainly a CL thing, all we are doing in EL is just view this as a 48-byte field and we don't perform any crypto operation for BLSPublicKey hence putting it under datatypes. I am happy to move it to crypto if you think otherwise.

Copy link
Contributor

@siladu siladu Mar 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to have a chat with someone who knows this area of besu better, or wait until another maintainer reviews to get a second opinion about the appropriate location for this and the other new datatypes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The datatypes package does seem like a more appropriate place than crypto as it is more focused on operations.

However, the intent of the datatype module is for broad-based types. So I don't think the DepositWithdrawalCredential would qualify and should probably just be a Bytes32 since we only care about bytes anyway or move it closer to where it is needed rather than in the top-level module.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The datatypes package does seem like a more appropriate place than crypto as it is more focused on operations.

However, the intent of the datatype module is for broad-based types. So I don't think the DepositWithdrawalCredential would qualify and should probably just be a Bytes32 since we only care about bytes anyway or move it closer to where it is needed rather than in the top-level module.

I agree we should just remove DepositWithdrawalCredential and make the field as Bytes32. What do you think about BLSPublicKey and BLSSignature? AFAIK we are not doing anything special for these 3 fields as we only care about the bytes. Having dedicated data classes are kind of like syntactic sugar

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think BLSPublicKey and BLSSignature are OK as new types since they are quite a general concept and could imagine them being reused in the future.
Whereas DepositWithdrawalsCredential is quite specific Bytes32 is better IMO...it's not clear to me from the EIP what format this credential takes exactly, maybe there's an existing data type that is appropriate?

Possibly BLSPublicKey and BLSSignature shouldn't live in data types though since they are not broadly used. Maybe should live alongside Deposit.java for now?

Copy link
Contributor Author

@ensi321 ensi321 Mar 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think BLSPublicKey and BLSSignature are OK as new types since they are quite a general concept and could imagine them being reused in the future. Whereas DepositWithdrawalsCredential is quite specific Bytes32 is better IMO...it's not clear to me from the EIP what format this credential takes exactly, maybe there's an existing data type that is appropriate?

The format of withdrawal credential is one of its kind and is rather specific. To summarize what it contains: there are two types one with 0x00 and one with 0x01 prefix. The first type is a hash of public key with the first byte being replaced by 0. The second type is simply a 20-byte eth1 address. I think there are some discussions in CL on coming up with new prefixes and new encoding for the withdrawal credential. So in short, I don't think we can use existing data type for it.

Possibly BLSPublicKey and BLSSignature shouldn't live in data types though since they are not broadly used. Maybe should live alongside Deposit.java for now?

Yea I agree. We can move them to Deposit.


/** The constant SIZE. */
public static final int SIZE = 48;

/**
* Instantiates a new BLSPublicKey.
*
* @param bytes the bytes
*/
protected BLSPublicKey(final Bytes bytes) {
super(bytes);
}

/**
* Wrap public key.
*
* @param value the value
* @return the BLS public key
*/
public static BLSPublicKey wrap(final Bytes value) {
checkArgument(
value.size() == SIZE, "A BLS public key must be %s bytes long, got %s", SIZE, value.size());
return new BLSPublicKey(value);
}

/**
* Creates a bls public key from the given RLP-encoded input.
*
* @param input The input to read from
* @return the input's corresponding public key
*/
public static BLSPublicKey readFrom(final RLPInput input) {
final Bytes bytes = input.readBytes();
if (bytes.size() != SIZE) {
throw new RLPException(
String.format("BLSPublicKey unexpected size of %s (needs %s)", bytes.size(), SIZE));
}
return BLSPublicKey.wrap(bytes);
}

/**
* Parse a hexadecimal string representing an public key.
*
* @param str A hexadecimal string (with or without the leading '0x') representing a valid bls
* public key.
* @return The parsed bls public key: {@code null} if the provided string is {@code null}.
* @throws IllegalArgumentException if the string is either not hexadecimal, or not the valid
* representation of a BLSPublicKey.
*/
@JsonCreator
public static BLSPublicKey fromHexString(final String str) {
if (str == null) return null;
return wrap(Bytes.fromHexStringLenient(str, SIZE));
}
}
Loading