(v2.2.3) Spec json is invalid: unknown field gasLimitBoundDivisor

[drandreaskrueger]

@5chdn asked to try version v2.2.3:

does this happen with 2.2.3?

but besides #85 it looks as if v2 needs a different spec.json ?

reproduce this

git clone https://github.com/paritytech/parity-deploy paritytech_parity-deploy
cd paritytech_parity-deploy
git log | head -n 5

commit 0e0624814f21e9fe05a77b2168fa257f915bbb3d
Merge: 359c3c2 eaf31d4
...
Date:   Thu Nov 1 15:27:06 2018 +0100

sudo ./clean.sh
./parity-deploy.sh -r v2.2.3 --config dev --geth

Custom parity build set: v2.2.3
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3856  100  3856    0     0   6072      0 --:--:-- --:--:-- --:--:--  6072
Release selected is: v2.2.3
Parity 2.2.3 already installed

using instantseal

Invalid argument: generate

Spec json is invalid: unknown field `gasLimitBoundDivisor`, 
expected one of `stepDuration`, `validators`, `startStep`, `validateScoreTransition`, `validateStepTransition`, `immediateTransitions`, `blockReward`, `blockRewardContractTransition`, `blockRewardContractAddress`, `blockRewardContractCode`, `maximumUncleCountTransition`, `maximumUncleCount`, `emptyStepsTransition`, `maximumEmptySteps` at line 6 column 39

[5chdn]

yeah we don't accept invalid specs anymore

[drandreaskrueger]

we don't accept invalid specs anymore

That might make maintaining paritytech/parity-deploy a bit more difficult. Because then there might need to be a whole set of spec.json files, depending on the version that a user is requesting. And to allow backwards compatibility is a real advantage of this amazing tool here, we have seen that in several cases. Until all glitches are solved, I must still run chainhammer on parity v1.11.11, for example. Also it is a nice feature to be able to test different parity versions against each other, to e.g. benchmark recent improvements.

What about a switch parity --ignore-invalid-specs, or parity --ignore-deprecated-specs that bypasses the checking, or simply turns it from error to warning?

But perhaps I am wrong, and there is an easy solution for version-dependent spec.json deployment, @ddorgan ?

[5chdn]

backwards compatibility

backwards compatibility is given if you use a valid spec anyways. note v1.11.11 is not supported anymore. all we need to do here is to fix the spec.

[drandreaskrueger]

note v1.11.11 is not supported anymore.

I herewith submit my formal protest against this ;-)

v1.11.11 is the last version that was still working fine when querying singlethreaded AND multithreaded.
See e.g. openethereum/parity-ethereum#9582 (comment) for a recent run on v2.2.0.

[drandreaskrueger]

all we need to do here is to fix the spec.

yes, then I can run a test on v2.2.3

[drandreaskrueger]

Also, wouldn't you want to know how much faster v2.2.3 is versus v1.11.11 ??

If you put an end-of-live to parity-deploy-for-v1.11.11 now, then it becomes harder to finding out the difference (I would then have to work with an outdated September or October commit of parity-deploy, which was since then possibly improved or even de-bug-ed, etc)

[ddorgan]

@drandreaskrueger can you check the latest master...

[drandreaskrueger]

I will as soon as I can. Sorry for the delay.

[drandreaskrueger]

I have just tried it out:

PARITY_VERSION=v2.2.3

rm -rf paritytech_parity-deploy
git clone https://github.com/paritytech/parity-deploy.git paritytech_parity-deploy
cd paritytech_parity-deploy/
git checkout 11b64e11cc9403101aa1e0db0bfbc72997e01a74
sudo ./clean.sh
./parity-deploy.sh -r $PARITY_VERSION --config dev --geth

Custom parity build set: v2.2.3
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3856  100  3856    0     0   6333      0 --:--:-- --:--:-- --:--:--  6342
Release selected is: v2.2.3
Parity 2.2.3 already installed
using instantseal

sed -i 's/parity:stable/parity:'$PARITY_VERSION'/g' docker-compose.yml
docker-compose up

Please consider to do the second last step in parity-deploy.sh - otherwise with every version upgrade we run into danger of new problems.

Actually, now I get the other problem:

Pulling host1 (parity/parity:v2.2.3)...
v2.2.3: Pulling from parity/parity
3b37166ec614: Pull complete
504facff238f: Pull complete
ebbcacd28e10: Pull complete
c7fb3351ecad: Pull complete
2e3debadcbf7: Pull complete
9b43fb0c9ad5: Pull complete
692ae35adfca: Pull complete
36566c131ff9: Pull complete
7aba27ce4ae9: Pull complete
6720da0dddc4: Pull complete
83c58c1ef2ac: Pull complete
6f34088fcbf0: Pull complete
54bafce8432f: Pull complete
486b365d89e8: Pull complete
Creating host1 ... done
Attaching to host1
host1    | Loading config file from /home/parity/authority.toml
host1    | Error upgrading parity data: CannotCreateConfigPath
host1 exited with code 1

but then after opening up data completely:

sudo rm -rf data
mkdir data
sudo chmod -R 777 data
docker-compose up

it first fails:

Starting host1 ... done
Attaching to host1
host1    | Loading config file from /home/parity/authority.toml
host1    | Error upgrading parity data: CannotCreateConfigPath
host1 exited with code 1

but then after doing that once more, and restarting once more ...

sudo chmod -R 777 data
docker-compose up

it finally works.

Please consider to somehow automate this last step in your tool, thanks.

[ddorgan]

@drandreaskrueger is this all ok with the latest master? I know there was an issue with instant sealing but with aura this should be working fine.

[drandreaskrueger]

with the latest master

I have tried the above with git checkout 11b64e1 that is the newest, right?

And the only way I could get it working was accepting that it fails once:

sudo chmod -R 777 data
docker-compose up
sudo chmod -R 777 data
docker-compose up

see above.

[drandreaskrueger]

Ha, I have an idea for you. Why not have a look at the other two projects that I am using: quorum, geth because neither has such user&access-rights problems. Hope that helps somehow?

[ddorgan]

Hi Andreas,

Yes there's a permission issue here. On your docker containers everything runs as root so you do not bump into these issues. The base parity/parity images now set the user to parity with a uid of 1000.

To avoid the CannotCreateConfigPath error can you try https://github.com/paritytech/parity-deploy/tree/dd-uid-set, which sets the uid on the data and deployments directory.

This is just for now, we should be able to find a better way to handle this in the base parity docker image.

David

[drandreaskrueger]

Thx.

When I tried yesterday with your newest master 1a6afd1 I did not run into permission problems anymore.

So I thought you had merged that?

See
#76 (comment)
and
#76 (comment)

So I think that is solved with 1a6afd1 now?

[drandreaskrueger]

What about creating docker data containers instead of storing directly on host disk?
Then you can choose any uid, no?

Isn't the problem the confusion between inside-docker-container-uid and hostmachine-uid?

[ddorgan]

Will change how this works. I had wanted to avoid using docker volumes but it looks like it is required.

Note that just adding volumes to a path and driver_opts uid=1000,gid=1000 has problems in a lot of cases so I'll test some better ways. Hopefully something that also plays nice with kubernetes.

[drandreaskrueger]

Good, thanks. Yes perhaps that is the only way? Or perhaps check these interesting pages before?

1, 2,
or perhaps this trick: CURRENT_UID=$(id -u):$(id -g) docker-compose up in 3 ?

[ddorgan]

All those suggestions break in at least some circumstances, it's a real pain. And since this is really a development tool I've just switched it back to using root with: #100

Looking to rewrite this from scratch soon with the learning from this version.

[drandreaskrueger]

it's a real pain

sorry to hear that.

yes, complexity is a bitch ;-)

When I have my next release out, I can show you how I solved it for now. A few more days.

I still think, parity v1.11.11 is important, and should stay supported. But I am fine fine with closing this issue for now and possibly move this bottom discussion elsewhere, as this issue was originally about the specs version jump v1 --> v2 anyways, and not about those root/parity user problems.