Skip to content

Commit 8405a1a

Browse files
committed
Eliminate setup steps and replace with pre-built launch artifacts
1 parent 9e31ff8 commit 8405a1a

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

44 files changed

+320
-805
lines changed

.gitignore

+12-4
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,14 @@
1-
examples/tmp
21
boot/node_modules
32
boot/package-lock.json
4-
test/node_modules
5-
test/package-lock.json
6-
test/coverage
3+
examples/qdata_1/args.txt
4+
examples/qdata_1/ethereum/geth/
5+
examples/qdata_1/logs/geth.log
6+
examples/qdata_2/args.txt
7+
examples/qdata_2/ethereum/geth/
8+
examples/qdata_2/logs/geth.log
9+
examples/qdata_3/args.txt
10+
examples/qdata_3/ethereum/geth/
11+
examples/qdata_3/logs/geth.log
12+
examples/qdata_4/args.txt
13+
examples/qdata_4/ethereum/geth/
14+
examples/qdata_4/logs/geth.log

Makefile

+3-14
Original file line numberDiff line numberDiff line change
@@ -19,24 +19,13 @@ clean: docker-clean
1919
# Docker builds
2020
docker-builder:
2121
@echo "Building docker image for builder"
22-
docker build -t $(DOCKER_NS)/quorum-builder builder
23-
24-
docker-bootnode: docker-builder
25-
@echo "Building docker image for bootnode"
26-
# build geth and bootnode commands
27-
docker run -v $(abspath ../quorum):/work $(DOCKER_NS)/quorum-builder make all
28-
# build the "bootnode" docker image
29-
docker build -t $(DOCKER_NS)/bootnode -f bootnode/Dockerfile ..
22+
docker build --platform linux/amd64 -t $(DOCKER_NS)/quorum-builder builder
3023

3124
docker-geth: docker-builder
3225
@echo "Building docker image for geth"
3326
# build geth and bootnode commands
3427
docker run -v $(abspath ../quorum):/work $(DOCKER_NS)/quorum-builder make all
3528
# build the "quorum" docker image
36-
docker build -t $(DOCKER_NS)/quorum -f geth/Dockerfile ..
37-
38-
istanbul-tools:
39-
@echo "Building docker image for istanbul-tools"
40-
docker build -t istanbul-tools -f istanbul/Dockerfile ..
29+
docker build --platform linux/amd64 -t $(DOCKER_NS)/quorum -f geth/Dockerfile ..
4130

42-
docker: docker-geth istanbul-tools
31+
docker: docker-geth

README.md

+10-70
Original file line numberDiff line numberDiff line change
@@ -7,68 +7,30 @@ This tests the setup to have Quorum (based on go-ethereum), without a private tr
77
### Repositories
88

99
This repository depends on the following repositories to be cloned and reside next to it:
10+
1011
```
11-
git clone git@github.com:kaleido-io/quorum-tools.git
1212
git clone -b v21.4.2 git@github.com:kaleido-io/quorum.git
13-
git clone git@github.com:getamis/istanbul-tools.git
1413
```
1514

16-
Build the docker images by launching from the project root:
15+
Build the docker images by launching from the project root of `quorum-tools`:
16+
1717
```
1818
make docker
1919
```
2020

2121
It should produce the following docker images:
2222

23-
| Image | Description | Needed for runtime? |
24-
| -------------------------------- |:---------------------:| -------------------:|
25-
| jpmorganchase/quorum | geth node | YES |
26-
| jpmorganchase/quorum-builder | build environment | NO |
27-
| istanbul-tools | IBFT CLI Tool | NO |
28-
29-
### Tools
30-
31-
`setup.sh` depends upon [jq](https://stedolan.github.io/jq/). Please ensure this `jq` is installed
32-
33-
## Generate configuration artifacts and docker-compose.yml
34-
35-
```
36-
cd examples
37-
./setup.sh
38-
```
39-
40-
The *setup.sh* script creates a basic Quorum network with Raft consensus. There's a whole bunch of things it needs to do in order to achieve this, some specific to Quorum, some common to private Ethereum chains in general.
41-
42-
The following arguments are supported:
43-
```
44-
-n, --nodes Number of Quorum nodes to generate. Default: 5. Can use "x+y" syntax to specify validators and non-validators. For instance, "./setup.sh -n 3+2 -c ibft" means generating 3 validators and 2 non-validators
45-
-c, --consensus Consensus to use. Valid values are raft and ibft. Default: raft
46-
-b, --blockperiod Applicable to IBFT only. The interval to produce blocks. The closely related parameter, istanbul.requesttimeout, will be automatically calculated by adding 10sec to the blockperiod value in order to make the configuration work.
47-
```
48-
49-
1. bootnode
50-
A bootnode is used in the network so that the geth nodes does not attempt to contact the well-known nodes in the public networks during p2p discovery. The script generates a node key for the bootnode and calculates its public address to be used in the geth node's `--bootnodes` argument.
51-
52-
2. for each Quorum node
53-
The script generates all configuration files for the geth node in the `ethereum` folder. Inside the folder:
23+
| Image | Description | Needed for runtime? |
24+
| ---------------------------- | :---------------: | ------------------: |
25+
| jpmorganchase/quorum | geth node | YES |
26+
| jpmorganchase/quorum-builder | build environment | NO |
5427

55-
* *nodekey* file to uniquely identify this node on the network.
56-
* *static-nodes.json* file that lists the Enode IDs of nodes that participate in the initial Raft cluster. Additional nodes can be added to the Raft cluster which is described [here](#adding-new-nodes-to-an-existing-network).
57-
* *permissioned-nodes.json* file that captures the list of Enode IDs allowed to connect to each other in this network instance.
58-
* Ether accounts are generated in the *keystore* directory
59-
* The accounts get written into the *genesis.json* file with an initial balance
60-
61-
3. docker-compose.yml
62-
This makes it trivial to launch the network
63-
64-
Refer to the *setup.sh* file itself for the full code.
65-
66-
## Launch the network
28+
## Launch the sample network
6729

6830
run the command below from the `examples` directory
6931

7032
```
71-
docker-compose -f tmp/docker-compose.yml up
33+
docker-compose up
7234
```
7335

7436
## Advanced Topics
@@ -89,26 +51,4 @@ The configuration files for the Quorum nodes in Quorum are saved under the `ethe
8951
│   └── permissioned-nodes.json
9052
└── logs/
9153

92-
On the Docker host, a *qdata_N/* directory for each node is created with the structure above. When the network is started, this will be mapped by the *docker-compose.yml* file to each container's internal */qdata/* directory.
93-
94-
### Adding new nodes to an existing network
95-
96-
The docker image jpmorganchase/quorum uses a startup script that is designed to start the node in one of the following modes:
97-
1. part of the initial Raft cluster. Notice in the docker-compose.yml file, the following command is used for this purpose:
98-
```
99-
command: start.sh --bootnode="enode://c3475a286a...6ebc6@172.13.0.100:30301" --raftInit
100-
```
101-
2. new node joining an existing network. You can modify the docker service start command as the following:
102-
```
103-
command: start.sh --bootnode="enode://c3475a286a...6ebc6@172.13.0.100:30301" --raftID=5
104-
```
105-
Note: the number `5` is the placeholder of the Raft node ID returned by calling `raft.addPeer()` in the geth console connected to an existing geth node of the network.
106-
107-
3. an IBFT node:
108-
```
109-
command: start.sh --bootnode="enode://c3475a286a...6ebc6@172.13.0.100:30301" --ibft
110-
```
111-
4. an IBFT node using 5 seconds block period:
112-
```
113-
command: start.sh --bootnode="enode://c3475a286a...6ebc6@172.13.0.100:30301" --ibft --blockperiod=5
114-
```
54+
On the Docker host, a _qdata_N/_ directory for each node is created with the structure above. When the network is started, this will be mapped by the _docker-compose.yml_ file to each container's internal _/qdata/_ directory.

boot/lib/boot.js

+6-6
Original file line numberDiff line numberDiff line change
@@ -33,9 +33,9 @@ const bootnode = argv.bootnode;
3333
const blockPeriod = argv.blockperiod;
3434
const roundChangeTimer = argv.roundchangetimer;
3535

36-
const rpcOrigins = argv.rpcOrigins || "*";
37-
const wsOrigins = argv.wsOrigins || "*";
38-
const consensus = (poa) ? 'POA' : ((ibft) ? 'IBFT' : 'RAFT');
36+
const rpcOrigins = argv.rpcOrigins || '*';
37+
const wsOrigins = argv.wsOrigins || '*';
38+
const consensus = poa ? 'POA' : ibft ? 'IBFT' : 'RAFT';
3939

4040
class Bootstrapper {
4141
constructor() {
@@ -47,7 +47,7 @@ class Bootstrapper {
4747
let data;
4848
try {
4949
data = await fs.readFile(this.configfile);
50-
} catch(err) {
50+
} catch (err) {
5151
logger.info(`No ${this.configfile} found, will look for required parameters in the command line args`);
5252
}
5353

@@ -66,7 +66,7 @@ class Bootstrapper {
6666

6767
let _config = {
6868
bootnode: _bootnode,
69-
network_id: _networkID
69+
network_id: _networkID,
7070
};
7171

7272
let _raftID = raftID || config.raft_id;
@@ -82,7 +82,7 @@ class Bootstrapper {
8282

8383
async writeCommandLineArgs(config) {
8484
const COMMON_ARGS = `--datadir ${DATADIR}/ethereum --nodekey /qdata/ethereum/nodekey --targetgaslimit 804247552 --miner.gasprice 0 --txpool.pricelimit 0 --port 30303 --rpc --rpcport 8545 --rpcaddr 0.0.0.0 --ws --wsport 8546 --wsaddr 0.0.0.0 --allow-insecure-unlock --unlock 0 --password /qdata/ethereum/passwords.txt --verbosity 4 --nousb`;
85-
const COMMON_APIS = "admin,db,eth,debug,miner,net,shh,txpool,personal,web3";
85+
const COMMON_APIS = 'admin,db,eth,debug,miner,net,shh,txpool,personal,web3';
8686
const RAFT_APIS = `${COMMON_APIS},raft`;
8787
const IBFT_APIS = `${COMMON_APIS},istanbul`;
8888
const POA_APIS = `${COMMON_APIS},clique`;

bootnode/Dockerfile

-10
This file was deleted.

examples/docker-compose.yml

+80
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
version: '2'
2+
services:
3+
bootnode:
4+
container_name: bootnode
5+
image: jpmorganchase/quorum
6+
command: bootnode -nodekey /qdata/nodekey
7+
volumes:
8+
- './qdata_0:/qdata'
9+
networks:
10+
quorum_net:
11+
ipv4_address: '172.13.0.100'
12+
13+
node_1:
14+
container_name: node_1
15+
image: jpmorganchase/quorum
16+
command: start.sh --bootnode="enode://973504942e8260307a1442046241e5dfc6a55a3065e3ce25d1a00b40315e127b62ba435237c7133b1d11940f1a8aadfea6c54766f337718b5d78ac26c20f4283@172.13.0.100:30301" --ibft --blockperiod=5 --roundchangetimer=15000 --networkid 2018
17+
volumes:
18+
- './qdata_1:/qdata'
19+
networks:
20+
quorum_net:
21+
ipv4_address: '172.13.3.1'
22+
ports:
23+
- 22001:8545
24+
- 23001:8546
25+
depends_on:
26+
- bootnode
27+
28+
node_2:
29+
container_name: node_2
30+
image: jpmorganchase/quorum
31+
command: start.sh --bootnode="enode://973504942e8260307a1442046241e5dfc6a55a3065e3ce25d1a00b40315e127b62ba435237c7133b1d11940f1a8aadfea6c54766f337718b5d78ac26c20f4283@172.13.0.100:30301" --ibft --blockperiod=5 --roundchangetimer=15000 --networkid 2018
32+
volumes:
33+
- './qdata_2:/qdata'
34+
networks:
35+
quorum_net:
36+
ipv4_address: '172.13.5.1'
37+
ports:
38+
- 22002:8545
39+
- 23002:8546
40+
depends_on:
41+
- bootnode
42+
43+
node_3:
44+
container_name: node_3
45+
image: jpmorganchase/quorum
46+
command: start.sh --bootnode="enode://973504942e8260307a1442046241e5dfc6a55a3065e3ce25d1a00b40315e127b62ba435237c7133b1d11940f1a8aadfea6c54766f337718b5d78ac26c20f4283@172.13.0.100:30301" --ibft --blockperiod=5 --roundchangetimer=15000 --networkid 2018
47+
volumes:
48+
- './qdata_3:/qdata'
49+
networks:
50+
quorum_net:
51+
ipv4_address: '172.13.7.1'
52+
ports:
53+
- 22003:8545
54+
- 23003:8546
55+
depends_on:
56+
- bootnode
57+
58+
node_4:
59+
container_name: node_4
60+
image: jpmorganchase/quorum
61+
command: start.sh --bootnode="enode://973504942e8260307a1442046241e5dfc6a55a3065e3ce25d1a00b40315e127b62ba435237c7133b1d11940f1a8aadfea6c54766f337718b5d78ac26c20f4283@172.13.0.100:30301" --ibft --blockperiod=5 --roundchangetimer=15000 --networkid 2018
62+
volumes:
63+
- './qdata_4:/qdata'
64+
networks:
65+
quorum_net:
66+
ipv4_address: '172.13.9.1'
67+
ports:
68+
- 22004:8545
69+
- 23004:8546
70+
depends_on:
71+
- bootnode
72+
73+
74+
networks:
75+
quorum_net:
76+
driver: bridge
77+
ipam:
78+
driver: default
79+
config:
80+
- subnet: 172.13.0.0/16

examples/qdata_0/nodekey

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
a8a535aeca0652612445543e79b6975c7470edbe8fc4ec9921dc6eeccdc7200d
+38
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
{
2+
"config": {
3+
"homesteadBlock": 1,
4+
"eip150Block": 2,
5+
"eip150Hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
6+
"eip155Block": 3,
7+
"eip158Block": 3,
8+
"istanbul": {
9+
"epoch": 30000,
10+
"policy": 0
11+
},
12+
"isQuorum": true
13+
},
14+
"nonce": "0x0",
15+
"timestamp": "0x63360676",
16+
"extraData": "0x0000000000000000000000000000000000000000000000000000000000000000f89af854947bead28484303a84322908a9072c448ec9ad374094d8cc52b186e9492fd1239d829b76d27a69cc296f94a8cbb3f6e49439003fbbf4a59f5aeabed110782e945491b9623b6413098211d70829d448ec83867111b8410000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000c0",
17+
"gasLimit": "0x47b760",
18+
"difficulty": "0x1",
19+
"mixHash": "0x63746963616c2062797a616e74696e65206661756c7420746f6c6572616e6365",
20+
"coinbase": "0x0000000000000000000000000000000000000000",
21+
"alloc": {
22+
"0xbE33f32C40fAF650c6B621035CC52E5C3831961c": {
23+
"balance": "1000000000000000000000000000"
24+
},
25+
"0xC2cd55b4195eCd8734E03F05914C4Ee1859b0c42": {
26+
"balance": "1000000000000000000000000000"
27+
},
28+
"0xc67DfeEf376da01eEe7ABd287F4D25fb05AE47eE": {
29+
"balance": "1000000000000000000000000000"
30+
},
31+
"0x90ffeD24A5D1bed608BFa27F0a05B12813888375": {
32+
"balance": "1000000000000000000000000000"
33+
}
34+
},
35+
"number": "0x0",
36+
"gasUsed": "0x0",
37+
"parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000"
38+
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{"address":"be33f32c40faf650c6b621035cc52e5c3831961c","crypto":{"cipher":"aes-128-ctr","ciphertext":"12188243ad7822143a462de48eec06c4875046a49653f5389073289a5c07f846","cipherparams":{"iv":"fe23c6c5e3a7d2f992547016263486d5"},"kdf":"scrypt","kdfparams":{"dklen":32,"n":262144,"p":1,"r":8,"salt":"bd893324d6014878aba90248dbe7d1832edf8f8a0c21ba0e03bb5a4fba542766"},"mac":"14fa4b6b9b28f1982f193485b0f6d28fede6ffb4163c566622ad3f8ecc16a60a"},"id":"d88caec8-3048-4b6d-9076-940bfeeb60a0","version":3}

examples/qdata_1/ethereum/nodekey

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
4fca7c920a382ac6d7e2654188a3b7cc356202732d3aa7838d859041435d8f71

examples/qdata_1/ethereum/passwords.txt

Whitespace-only changes.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
[
2+
"enode://670a0e1fbeb6a15c01588c5f264790a3624de668aa4c98157be4c6daf135431a87e0f0195b2311c25786e2e9ea0727a68932450fc40bdd9ec22527e2b6296e82@172.13.3.1:30303?discport=0&raftport=50400",
3+
"enode://ae3a290edfd48107c01736be30358d7686aa7bc02f1c2f9e35743f6b3de6a0d44967af8c91ca9657d9ff878b1e16abc81ba5d2504e2c67ed525d8855eb0b0436@172.13.5.1:30303?discport=0&raftport=50400",
4+
"enode://4c07cf65a51b31badf0b6b6d79a585a1ba3b802cdf53fc3d25c86d76f692dab8e8197019f7ea4fb8ee1d1ed89cb7ccc52ffa511998903b95b796c63477f761d4@172.13.7.1:30303?discport=0&raftport=50400",
5+
"enode://292312e5a2649f29504af80760abdcecbdfca90084e800225dd6d10cc0f60f6a12aa862d4f517fc4a485be0ce9c20f98099a8138ce04d7f257356dc9a70a0679@172.13.9.1:30303?discport=0&raftport=50400"
6+
]
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
[
2+
"enode://670a0e1fbeb6a15c01588c5f264790a3624de668aa4c98157be4c6daf135431a87e0f0195b2311c25786e2e9ea0727a68932450fc40bdd9ec22527e2b6296e82@172.13.3.1:30303?discport=0&raftport=50400",
3+
"enode://ae3a290edfd48107c01736be30358d7686aa7bc02f1c2f9e35743f6b3de6a0d44967af8c91ca9657d9ff878b1e16abc81ba5d2504e2c67ed525d8855eb0b0436@172.13.5.1:30303?discport=0&raftport=50400",
4+
"enode://4c07cf65a51b31badf0b6b6d79a585a1ba3b802cdf53fc3d25c86d76f692dab8e8197019f7ea4fb8ee1d1ed89cb7ccc52ffa511998903b95b796c63477f761d4@172.13.7.1:30303?discport=0&raftport=50400",
5+
"enode://292312e5a2649f29504af80760abdcecbdfca90084e800225dd6d10cc0f60f6a12aa862d4f517fc4a485be0ce9c20f98099a8138ce04d7f257356dc9a70a0679@172.13.9.1:30303?discport=0&raftport=50400"
6+
]

examples/qdata_1/logs/dummy.txt

Whitespace-only changes.
+38
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
{
2+
"config": {
3+
"homesteadBlock": 1,
4+
"eip150Block": 2,
5+
"eip150Hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
6+
"eip155Block": 3,
7+
"eip158Block": 3,
8+
"istanbul": {
9+
"epoch": 30000,
10+
"policy": 0
11+
},
12+
"isQuorum": true
13+
},
14+
"nonce": "0x0",
15+
"timestamp": "0x63360676",
16+
"extraData": "0x0000000000000000000000000000000000000000000000000000000000000000f89af854947bead28484303a84322908a9072c448ec9ad374094d8cc52b186e9492fd1239d829b76d27a69cc296f94a8cbb3f6e49439003fbbf4a59f5aeabed110782e945491b9623b6413098211d70829d448ec83867111b8410000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000c0",
17+
"gasLimit": "0x47b760",
18+
"difficulty": "0x1",
19+
"mixHash": "0x63746963616c2062797a616e74696e65206661756c7420746f6c6572616e6365",
20+
"coinbase": "0x0000000000000000000000000000000000000000",
21+
"alloc": {
22+
"0xbE33f32C40fAF650c6B621035CC52E5C3831961c": {
23+
"balance": "1000000000000000000000000000"
24+
},
25+
"0xC2cd55b4195eCd8734E03F05914C4Ee1859b0c42": {
26+
"balance": "1000000000000000000000000000"
27+
},
28+
"0xc67DfeEf376da01eEe7ABd287F4D25fb05AE47eE": {
29+
"balance": "1000000000000000000000000000"
30+
},
31+
"0x90ffeD24A5D1bed608BFa27F0a05B12813888375": {
32+
"balance": "1000000000000000000000000000"
33+
}
34+
},
35+
"number": "0x0",
36+
"gasUsed": "0x0",
37+
"parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000"
38+
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{"address":"c2cd55b4195ecd8734e03f05914c4ee1859b0c42","crypto":{"cipher":"aes-128-ctr","ciphertext":"ea8257fedc775efdc1c264bbed671e6da36675267f0eb50b284db29e788143cc","cipherparams":{"iv":"95d627e79226ff24b86061ddb058b8e6"},"kdf":"scrypt","kdfparams":{"dklen":32,"n":262144,"p":1,"r":8,"salt":"a8fe321d5278c8a198d9d88f886422742b7877f5aa810646a9f3ef0e7b87e920"},"mac":"a67ee08326972297aae30f0fc2fd27764991d6d580fcd154cf1648897a8362a6"},"id":"f41606c3-6f68-49cd-b5f1-1829b2491c73","version":3}

examples/qdata_2/ethereum/nodekey

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
5fa039db3218ce735a2574655287aa9c0dbb3bc70fea8dbe35fc30fad1da2dad

examples/qdata_2/ethereum/passwords.txt

Whitespace-only changes.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
[
2+
"enode://670a0e1fbeb6a15c01588c5f264790a3624de668aa4c98157be4c6daf135431a87e0f0195b2311c25786e2e9ea0727a68932450fc40bdd9ec22527e2b6296e82@172.13.3.1:30303?discport=0&raftport=50400",
3+
"enode://ae3a290edfd48107c01736be30358d7686aa7bc02f1c2f9e35743f6b3de6a0d44967af8c91ca9657d9ff878b1e16abc81ba5d2504e2c67ed525d8855eb0b0436@172.13.5.1:30303?discport=0&raftport=50400",
4+
"enode://4c07cf65a51b31badf0b6b6d79a585a1ba3b802cdf53fc3d25c86d76f692dab8e8197019f7ea4fb8ee1d1ed89cb7ccc52ffa511998903b95b796c63477f761d4@172.13.7.1:30303?discport=0&raftport=50400",
5+
"enode://292312e5a2649f29504af80760abdcecbdfca90084e800225dd6d10cc0f60f6a12aa862d4f517fc4a485be0ce9c20f98099a8138ce04d7f257356dc9a70a0679@172.13.9.1:30303?discport=0&raftport=50400"
6+
]

0 commit comments

Comments
 (0)