Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ICS 23 Implementation #4515

Merged
merged 123 commits into from
Nov 5, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
123 commits
Select commit Hold shift + click to select a range
9375649
add mapping
mossid Jun 7, 2019
16eea68
rm unused mapping/*, rm interfaces
mossid Jun 11, 2019
f73215e
rm unused code
mossid Jun 11, 2019
cf0b8f2
mv mapping -> state, rm x/ibc
mossid Jun 11, 2019
e25f652
Merge branch 'master' of github.com:cosmos/cosmos-sdk into joon/store…
mossid Jun 17, 2019
2b17a39
rm GetIfExists
mossid Jun 25, 2019
dc84e1e
add key
mossid Jun 25, 2019
19d7155
rm custom encoding/decoding in enum/bool
mossid Jun 26, 2019
6764bc4
Merge branch 'master' of github.com:cosmos/cosmos-sdk into joon/store…
mossid Jun 29, 2019
8affaa7
fix lint
mossid Jun 29, 2019
a7898b4
rm tests
mossid Jul 7, 2019
afcca90
add commitment
mossid Jun 7, 2019
d29b601
newtyped remote values
mossid Jun 11, 2019
dd86fa9
newtyped context
mossid Jun 11, 2019
6aad15f
revert context newtype
mossid Jun 11, 2019
088836d
add README, keypath
mossid Jun 11, 2019
cd6f1f2
reflect downstream ics
mossid Jun 11, 2019
d0777cd
add merkle
mossid Jun 11, 2019
df8c05e
add test for proving
mossid Jun 12, 2019
47c33f0
soft coded root keypath
mossid Jun 25, 2019
12ef402
add update
mossid Jun 25, 2019
03d5d17
remove RootUpdate
mossid Jun 25, 2019
0b79b1d
separate keypath and keuprefix
mossid Jun 25, 2019
7046953
add codec
mossid Jun 26, 2019
3939bd4
Merge branch 'master' of github.com:cosmos/cosmos-sdk into joon/store…
mossid Jul 9, 2019
db93a8a
separate root/path
mossid Jul 11, 2019
37387bf
add path to codec
mossid Jul 13, 2019
f609249
add docs in progre
mossid Jul 24, 2019
7da69c1
add comments
mossid Jul 25, 2019
600445a
add comments
mossid Jul 25, 2019
98720bb
Apply suggestions from code review
mossid Jul 25, 2019
cdee396
add comments in progress
mossid Jul 25, 2019
ce588a4
Merge branch 'joon/store-mapping' into joon/ics-23-implementation
mossid Jul 25, 2019
39dca11
Merge branch 'joon/ics-23-implementation' of github.com:cosmos/cosmos…
mossid Jul 25, 2019
94e2cdc
Merge branch 'master' of github.com:cosmos/cosmos-sdk into joon/store…
mossid Jul 26, 2019
8d9e779
add comments
mossid Jul 26, 2019
92d0fb4
fix comment
mossid Jul 26, 2019
d896152
Merge branch 'joon/store-mapping' into joon/ics-23-implementation
mossid Jul 26, 2019
f70d3bd
add comments in progress
mossid Jul 27, 2019
5d913b7
recover IntEncoding scheme for integer
mossid Jul 29, 2019
699b0ba
add uint tests, don't use codec in custom types
mossid Jul 29, 2019
323818f
Merge branch 'joon/store-mapping' into joon/ics-23-implementation
mossid Jul 29, 2019
9a077eb
finalize merge
mossid Jul 29, 2019
a4a20d6
add godoc
mossid Jul 30, 2019
a9adb58
reformat test
mossid Jul 30, 2019
abb2f51
rm XXX
mossid Jul 30, 2019
361aa42
add godoc
mossid Jul 30, 2019
97edd89
Merge branch 'master' of github.com:cosmos/cosmos-sdk into joon/store…
mossid Jul 30, 2019
23debb9
Merge branch 'joon/store-mapping' into joon/ics-23-implementation
mossid Jul 30, 2019
24e621a
add query
mossid Jul 31, 2019
64c0381
update query.go
mossid Jul 31, 2019
971a642
update query.go
mossid Jul 31, 2019
2968dd3
Merge branch 'joon/store-mapping' into joon/ics-23-implementation
mossid Jul 31, 2019
fcad487
add Query to boolean.go
mossid Aug 1, 2019
3142d4b
Merge branch 'joon/store-mapping' into joon/ics-23-implementation
mossid Aug 1, 2019
6de0ed5
fix key
mossid Aug 1, 2019
05d39a1
Merge branch 'joon/store-mapping' into joon/ics-23-implementation
mossid Aug 1, 2019
6dc912f
godoc cleanup
alexanderbez Aug 7, 2019
63c6f3c
godoc cleanup
alexanderbez Aug 7, 2019
b6441c0
godoc cleanup
alexanderbez Aug 7, 2019
4d723ae
godoc cleanup
alexanderbez Aug 7, 2019
244d2c2
godoc cleanup
alexanderbez Aug 7, 2019
7bf4d57
merge from ics04 branch
mossid Aug 15, 2019
7471bdf
Merge branch 'joon/store-mapping' of github.com:cosmos/cosmos-sdk int…
mossid Aug 15, 2019
3c86363
Merge branch 'joon/store-mapping' into joon/ics-23-implementation
mossid Aug 15, 2019
32cb0ca
merge from ics04 branch
mossid Aug 15, 2019
71d2620
fix lint
mossid Aug 15, 2019
04543f8
Merge branch 'master' of github.com:cosmos/cosmos-sdk into joon/store…
mossid Sep 2, 2019
59d2b3e
Merge branch 'joon/store-mapping' into joon/ics-23-implementation
mossid Sep 2, 2019
10d1c46
Apply suggestions from code review
mossid Sep 10, 2019
09e6600
applying review in progress
mossid Sep 11, 2019
6f5f633
Merge branch 'joon/store-mapping' of github.com:cosmos/cosmos-sdk int…
mossid Sep 11, 2019
4116bb7
apply review - make querier interface
mossid Sep 11, 2019
8040c81
Merge branch 'master' of github.com:cosmos/cosmos-sdk into joon/store…
mossid Sep 17, 2019
48bb32d
Merge branch 'joon/store-mapping' into joon/ics-23-implementation
mossid Sep 17, 2019
9b63a6b
fix dependency
mossid Sep 17, 2019
828badd
revise querier interface to work both on cli & store
mossid Sep 18, 2019
a59d1e7
Merge branch 'joon/store-mapping' into joon/ics-23-implementation
mossid Sep 18, 2019
9f63629
rm commented lines
mossid Sep 28, 2019
6b966a3
rename Path -> Prefix
mossid Oct 1, 2019
21bd8d0
Store accessor upstream changes (#5119)
fedekunze Oct 1, 2019
af2564b
Merge branch 'joon/store-mapping' of https://github.com/cosmos/cosmos…
fedekunze Oct 1, 2019
303ebf3
Merge branch 'master' of https://github.com/cosmos/cosmos-sdk into jo…
fedekunze Oct 1, 2019
9b3b120
Merge branch 'master' into joon/store-mapping
fedekunze Oct 1, 2019
18f8d70
Merge branch 'joon/store-mapping' of https://github.com/cosmos/cosmos…
fedekunze Oct 1, 2019
df5b21a
add comments, reformat merkle querier
mossid Oct 1, 2019
7afe221
Merge branch 'joon/ics-23-implementation' of github.com:cosmos/cosmos…
mossid Oct 1, 2019
1ac982c
rm merkle/utils
mossid Oct 1, 2019
fa04419
ICS 23 upstream changes (#5120)
fedekunze Oct 1, 2019
db8d382
update Value
fedekunze Oct 1, 2019
a7d2d98
remove Mapping
fedekunze Oct 7, 2019
1b7e466
remove store accessors
fedekunze Oct 7, 2019
c09a84c
Merge branch 'master' of https://github.com/cosmos/cosmos-sdk into jo…
fedekunze Oct 7, 2019
a1c22e9
Merge branch 'master' into joon/ics-23-implementation
fedekunze Oct 15, 2019
d3a533a
Merge branch 'master' of https://github.com/cosmos/cosmos-sdk into jo…
fedekunze Oct 16, 2019
7ba777e
Merge branch 'master' of https://github.com/cosmos/cosmos-sdk into jo…
fedekunze Oct 18, 2019
499abb4
refactor ICS23
fedekunze Oct 18, 2019
a925d88
cleanup types
fedekunze Oct 18, 2019
53f5fad
Merge branch 'master' of https://github.com/cosmos/cosmos-sdk into jo…
fedekunze Oct 21, 2019
c12c298
implement batch verification
fedekunze Oct 21, 2019
a34b741
gosimple suggestion
fedekunze Oct 21, 2019
bf4f096
Merge branch 'master' into joon/ics-23-implementation
fedekunze Oct 21, 2019
386489b
Merge branch 'joon/ics-23-implementation' of https://github.com/cosmo…
fedekunze Oct 21, 2019
40fca1a
alias
fedekunze Oct 22, 2019
f8fd580
add tests
AdityaSripal Oct 22, 2019
cbc1106
ICS 24 Implementation (#5229)
AdityaSripal Oct 22, 2019
f8d99ca
Merge branch 'master' of https://github.com/cosmos/cosmos-sdk into jo…
fedekunze Oct 23, 2019
23ae994
flatten ICS23 structure
fedekunze Oct 23, 2019
e1fa5f8
fix ApplyPrefix
fedekunze Oct 23, 2019
2edb03f
complete types testing
AdityaSripal Oct 23, 2019
5783b03
merge
AdityaSripal Oct 23, 2019
696db2e
delete empty test file
AdityaSripal Oct 23, 2019
69db1f2
remove ibc errors from core error package
AdityaSripal Oct 23, 2019
33312c2
start batch-verify tests
AdityaSripal Oct 24, 2019
49d08e3
Merge branch 'master' of https://github.com/cosmos/cosmos-sdk into jo…
fedekunze Oct 25, 2019
5239720
minor changes on commitment types
fedekunze Oct 25, 2019
d1f71dd
Merge branch 'master' into joon/ics-23-implementation
fedekunze Oct 30, 2019
0a461b5
use testsuite
AdityaSripal Oct 30, 2019
dac68df
Merge branch 'joon/ics-23-implementation' of https://github.com/cosmo…
AdityaSripal Oct 30, 2019
a74ebf3
Merge branch 'master' into joon/ics-23-implementation
fedekunze Nov 4, 2019
17abbda
Merge branch 'master' of https://github.com/cosmos/cosmos-sdk into jo…
fedekunze Nov 5, 2019
820482f
upstream changes
fedekunze Nov 5, 2019
7c67236
context changes
fedekunze Nov 5, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions client/context/context.go
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,53 @@ func NewCLIContextWithFrom(from string) CLIContext {
return ctx.WithVerifier(verifier)
}

// NewCLIContextIBC takes additional arguements
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

arguements is a misspelling of arguments (from misspell)

Suggested change
// NewCLIContextIBC takes additional arguements

func NewCLIContextIBC(from string, chainID string, nodeURI string) CLIContext {
var rpc rpcclient.Client

genOnly := viper.GetBool(flags.FlagGenerateOnly)
fromAddress, fromName, err := GetFromFields(from, genOnly)
if err != nil {
fmt.Printf("failed to get from fields: %v", err)
os.Exit(1)
}

if !genOnly {
if nodeURI != "" {
rpc = rpcclient.NewHTTP(nodeURI, "/websocket")
}
}

ctx := CLIContext{
Client: rpc,
ChainID: chainID,
Output: os.Stdout,
NodeURI: nodeURI,
From: from,
OutputFormat: viper.GetString(cli.OutputFlag),
Height: viper.GetInt64(flags.FlagHeight),
HomeDir: viper.GetString(flags.FlagHome),
TrustNode: viper.GetBool(flags.FlagTrustNode),
UseLedger: viper.GetBool(flags.FlagUseLedger),
BroadcastMode: viper.GetString(flags.FlagBroadcastMode),
Simulate: viper.GetBool(flags.FlagDryRun),
GenerateOnly: genOnly,
FromAddress: fromAddress,
FromName: fromName,
Indent: viper.GetBool(flags.FlagIndentResponse),
SkipConfirm: viper.GetBool(flags.FlagSkipConfirmation),
}

// create a verifier for the specific chain ID and RPC client
verifier, err := CreateVerifier(ctx, DefaultVerifierCacheSize)
if err != nil && viper.IsSet(flags.FlagTrustNode) {
fmt.Printf("failed to create verifier: %s\n", err)
os.Exit(1)
}

return ctx.WithVerifier(verifier)
}

// NewCLIContext returns a new initialized CLIContext with parameters from the
// command line using Viper.
func NewCLIContext() CLIContext { return NewCLIContextWithFrom(viper.GetString(flags.FlagFrom)) }
Expand Down
113 changes: 97 additions & 16 deletions client/context/query.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package context
import (
"fmt"
"strings"
"time"

"github.com/pkg/errors"

Expand All @@ -12,6 +13,7 @@ import (
tmliteErr "github.com/tendermint/tendermint/lite/errors"
tmliteProxy "github.com/tendermint/tendermint/lite/proxy"
rpcclient "github.com/tendermint/tendermint/rpc/client"
ctypes "github.com/tendermint/tendermint/rpc/core/types"
tmtypes "github.com/tendermint/tendermint/types"

"github.com/cosmos/cosmos-sdk/store/rootmulti"
Expand All @@ -28,6 +30,55 @@ func (ctx CLIContext) GetNode() (rpcclient.Client, error) {
return ctx.Client, nil
}

// WaitForNBlocks blocks until the node defined on the context has advanced N blocks
func (ctx CLIContext) WaitForNBlocks(n int64) {
node, err := ctx.GetNode()
if err != nil {
panic(err)
}

resBlock, err := node.Block(nil)
var height int64
if err != nil || resBlock.Block == nil {
// wait for the first block to exist
ctx.waitForHeight(1)
height = 1 + n
} else {
height = resBlock.Block.Height + n
}
ctx.waitForHeight(height)
}

func (ctx CLIContext) waitForHeight(height int64) {
node, err := ctx.GetNode()
if err != nil {
panic(err)
}

for {
// get url, try a few times
var resBlock *ctypes.ResultBlock
var err error
INNER:
for i := 0; i < 5; i++ {
resBlock, err = node.Block(nil)
if err == nil {
break INNER
}
time.Sleep(time.Millisecond * 200)
}
if err != nil {
panic(err)
}

if resBlock.Block != nil && resBlock.Block.Height >= height {
return
}

time.Sleep(time.Millisecond * 100)
}
}

// Query performs a query to a Tendermint node with the provided path.
// It returns the result and height of the query upon success or an error if
// the query fails.
Expand All @@ -49,6 +100,12 @@ func (ctx CLIContext) QueryStore(key cmn.HexBytes, storeName string) ([]byte, in
return ctx.queryStore(key, storeName, "key")
}

// QueryABCI performs a query to a Tendermint node with the provide RequestQuery.
// It returns the ResultQuery obtained from the query.
func (ctx CLIContext) QueryABCI(req abci.RequestQuery) (abci.ResponseQuery, error) {
return ctx.queryABCI(req)
}

// QuerySubspace performs a query to a Tendermint node with the provided
// store name and subspace. It returns key value pair and height of the query
// upon success or an error if the query fails.
Expand All @@ -72,40 +129,64 @@ func (ctx CLIContext) GetFromName() string {
return ctx.FromName
}

// query performs a query to a Tendermint node with the provided store name
// and path. It returns the result and height of the query upon success
// or an error if the query fails. In addition, it will verify the returned
// proof if TrustNode is disabled. If proof verification fails or the query
// height is invalid, an error will be returned.
func (ctx CLIContext) query(path string, key cmn.HexBytes) (res []byte, height int64, err error) {
func (ctx CLIContext) queryABCI(req abci.RequestQuery) (resp abci.ResponseQuery, err error) {

node, err := ctx.GetNode()
if err != nil {
return res, height, err
return resp, err
}

// When a client did not provide a query height, manually query for it so it can
// be injected downstream into responses.
if ctx.Height == 0 {
status, err := node.Status()
if err != nil {
return resp, err
}
ctx = ctx.WithHeight(status.SyncInfo.LatestBlockHeight)
}

opts := rpcclient.ABCIQueryOptions{
Height: ctx.Height,
Prove: !ctx.TrustNode,
Prove: req.Prove || !ctx.TrustNode,
}

result, err := node.ABCIQueryWithOptions(path, key, opts)
result, err := node.ABCIQueryWithOptions(req.Path, req.Data, opts)
if err != nil {
return res, height, err
return
}

resp := result.Response
resp = result.Response
if !resp.IsOK() {
return res, resp.Height, errors.New(resp.Log)
err = errors.New(resp.Log)
return
}

// data from trusted node or subspace query doesn't need verification
if ctx.TrustNode || !isQueryStoreWithProof(path) {
return resp.Value, resp.Height, nil
if ctx.TrustNode || !isQueryStoreWithProof(req.Path) {
return resp, nil
}

err = ctx.verifyProof(path, resp)
err = ctx.verifyProof(req.Path, resp)
if err != nil {
return
}

return
}

// query performs a query to a Tendermint node with the provided store name
// and path. It returns the result and height of the query upon success
// or an error if the query fails. In addition, it will verify the returned
// proof if TrustNode is disabled. If proof verification fails or the query
// height is invalid, an error will be returned.
func (ctx CLIContext) query(path string, key cmn.HexBytes) (res []byte, height int64, err error) {
resp, err := ctx.queryABCI(abci.RequestQuery{
Path: path,
Data: key,
})
if err != nil {
return res, resp.Height, err
return
}

return resp.Value, resp.Height, nil
Expand Down
18 changes: 18 additions & 0 deletions x/ibc/23-commitment/codec.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
package commitment

import (
"github.com/cosmos/cosmos-sdk/codec"
)

// RegisterCodec registers types declared in this package
func RegisterCodec(cdc *codec.Codec) {
cdc.RegisterInterface((*RootI)(nil), nil)
cdc.RegisterInterface((*PrefixI)(nil), nil)
cdc.RegisterInterface((*PathI)(nil), nil)
cdc.RegisterInterface((*ProofI)(nil), nil)

cdc.RegisterConcrete(Root{}, "ibc/commitment/merkle/Root", nil)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ICS23 is supposed to be a generic format, not just for the sdk, but also for agoric (js), for rust chains (polkadot), etc.

As there is no truly cross-language support for go-amino, I wonder about the usage here. Also, the very unclear definition of the pieces (merkle.Proof used below is a (string, []byte) blob without clear spec.

I can see this approach for a quick prototype, but if you are attempting a 1.0 of something interoperable, there needs to be a clear spec and possibility to implement in other languages. I spent considerable time getting that to work in github.com/confio/ics23 and I wonder why that was not used (not comments left on that repo to explain what was lacking/wrong with that approach)?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ethanfrey correct, afaik for the IBC completeness your code should be the one that we’ll need to integrate here. With respect to what was lacking I think @cwgoes created an issue somewhere (not sure if it was here or on the ICS repo)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's right - the plan is to integrate confio/ics23 prior to 1.0.0. - #5199

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that the hackathon has passed, maybe this PR should include confio/ics23?

Up to you & the SDK team @fedekunze.

cdc.RegisterConcrete(Prefix{}, "ibc/commitment/merkle/Prefix", nil)
cdc.RegisterConcrete(Path{}, "ibc/commitment/merkle/Path", nil)
cdc.RegisterConcrete(Proof{}, "ibc/commitment/merkle/Proof", nil)
}
37 changes: 37 additions & 0 deletions x/ibc/23-commitment/commitment_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
package commitment_test

import (
"testing"

"github.com/stretchr/testify/suite"

"github.com/cosmos/cosmos-sdk/store/iavl"
"github.com/cosmos/cosmos-sdk/store/rootmulti"
storetypes "github.com/cosmos/cosmos-sdk/store/types"

dbm "github.com/tendermint/tm-db"
)

type MerkleTestSuite struct {
suite.Suite

store *rootmulti.Store
storeKey *storetypes.KVStoreKey
iavlStore *iavl.Store
}

func (suite *MerkleTestSuite) SetupTest() {
db := dbm.NewMemDB()
suite.store = rootmulti.NewStore(db)

suite.storeKey = storetypes.NewKVStoreKey("iavlStoreKey")

suite.store.MountStoreWithDB(suite.storeKey, storetypes.StoreTypeIAVL, nil)
suite.store.LoadVersion(0)

suite.iavlStore = suite.store.GetCommitStore(suite.storeKey).(*iavl.Store)
}

func TestMerkleTestSuite(t *testing.T) {
suite.Run(t, new(MerkleTestSuite))
}
Loading