diff --git a/CHANGELOG.md b/CHANGELOG.md index 5b2b0a944f..04661236c3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -77,6 +77,7 @@ - [#1149](https://github.com/crypto-org-chain/cronos/pull/1149) memiavl support `WorkingHash` api required by `FinalizeBlock`. - [#1151](https://github.com/crypto-org-chain/cronos/pull/1151) memiavl `CacheMultiStoreWithVersion` supports `io.Closer`. - [#1154](https://github.com/crypto-org-chain/cronos/pull/1154) Remove dependency on cosmos-sdk. +- [#1171](https://github.com/crypto-org-chain/cronos/pull/1171) Add memiavl background snapshot writing concurrency limit. *April 13, 2023* @@ -90,13 +91,13 @@ ### Bug Fixes -* [#953](https://github.com/crypto-org-chain/cronos/pull/953) Include third-party bug fixes: +- [#953](https://github.com/crypto-org-chain/cronos/pull/953) Include third-party bug fixes: - update ethermint to include two bug fixes - - https://github.com/crypto-org-chain/ethermint/pull/234 - - https://github.com/crypto-org-chain/ethermint/pull/233 + - + - - update cosmos-sdk to include one bug fix - - https://github.com/cosmos/cosmos-sdk/pull/15667 -* [#945](https://github.com/crypto-org-chain/cronos/pull/945) Fix no handler exists for proposal type error when update-client due to wrong ibc route. + - +- [#945](https://github.com/crypto-org-chain/cronos/pull/945) Fix no handler exists for proposal type error when update-client due to wrong ibc route. *Mar 16, 2023* @@ -104,7 +105,7 @@ ### Bug Fixes -* [#932](https://github.com/crypto-org-chain/cronos/pull/932) Backport multiple json-rpc bug fixes in ethermint ([commits](https://github.com/crypto-org-chain/ethermint/compare/v0.20.8-cronos...v0.20.9-cronos)). +- [#932](https://github.com/crypto-org-chain/cronos/pull/932) Backport multiple json-rpc bug fixes in ethermint ([commits](https://github.com/crypto-org-chain/ethermint/compare/v0.20.8-cronos...v0.20.9-cronos)). *Mar 6, 2023* @@ -243,6 +244,7 @@ ## v0.9.0 ### State Machine Breaking + - [cronos#429](https://github.com/crypto-org-chain/cronos/pull/429) Update ethermint to main, ibc-go to v3.0.0, cosmos sdk to v0.45.4 and gravity to latest, remove v0.7.0 related upgradeHandler. - [cronos#532](https://github.com/crypto-org-chain/cronos/pull/532) Add SendtoChain and CancelSendToChain support from evm call. - [cronos#600](https://github.com/crypto-org-chain/cronos/pull/600) Implement bidirectional token mapping. @@ -258,6 +260,7 @@ - [cronos#639](https://github.com/crypto-org-chain/cronos/pull/639) init and validate-genesis commands don't include experimental modules by default. ### Improvements + - [cronos#418](https://github.com/crypto-org-chain/cronos/pull/418) Support logs in evm-hooks and return id for SendToEthereum events - [cronos#489](https://github.com/crypto-org-chain/cronos/pull/489) Enable jemalloc memory allocator, and update rocksdb src to `v6.29.5`. - [cronos#511](https://github.com/crypto-org-chain/cronos/pull/511) Replace ibc-hook with ibc middleware, use ibc-go upstream version. @@ -313,6 +316,7 @@ - [cronos#315](https://github.com/crypto-org-chain/cronos/pull/315) Update cosmos-sdk to `v0.45.0` ### Improvements + - [cronos#210](https://github.com/crypto-org-chain/cronos/pull/210) re-enabling gravity bridge conditionally - [cronos#322](https://github.com/crypto-org-chain/cronos/pull/322) Merge min-gas-price change in ethermint: don't check min-gas-price for EVM tx when feemarket enabled. - [cronos#345](https://github.com/crypto-org-chain/cronos/pull/345) disable the url query parameter in swagger-ui. @@ -320,10 +324,11 @@ - [cronos#441](https://github.com/crypto-org-chain/cronos/pull/441) Update cosmos-sdk to `v0.45.4` ### Bug Fixes + - [cronos#287](https://github.com/crypto-org-chain/cronos/pull/287) call upgrade handler before sealing app - [cronos#323](https://github.com/crypto-org-chain/cronos/pull/323) Upgrade gravity bridge to v0.3.9 which contain a bugfix on `batchTxExecuted.` - [cronos#324](https://github.com/crypto-org-chain/cronos/pull/324) Update to cosmos-sdk `v0.45.1`, which fixes an OOM issue. -- [cronos#329](https://github.com/crypto-org-chain/cronos/pull/329) Fix panic of eth_call on blocks prior to upgrade. +- [cronos#329](https://github.com/crypto-org-chain/cronos/pull/329) Fix panic of eth_call on blocks prior to upgrade. - [cronos#340](https://github.com/crypto-org-chain/cronos/pull/340) Update dependencies to include several bug fixes: a) fix subscription deadlock issue in ethermint, b) fix data races `traceContext`. - [cronos#370](https://github.com/crypto-org-chain/cronos/pull/370) Update ethermint to fix a websocket bug, add websockets integration tests. - [cronos#378](https://github.com/crypto-org-chain/cronos/pull/378) Backport recent ethermint bug fixes: a) fix tx inclusion issue by report correct gasWanted of eth tx, b) Add buffer to eth_gasPrice response to fix client UX, c) Quick fix for eth_feeHistory when reward is nil, d) add returnValue message on tracing. @@ -343,6 +348,7 @@ ## v0.6.4 ### Bug Fixes + - [crypto-org-chain/ethermint#19](https://github.com/crypto-org-chain/ethermint/pull/19) revert tharsis#786 because it contains consensus breaking changes *November 29, 2021* @@ -360,12 +366,12 @@ - [tharsis#786](https://github.com/tharsis/ethermint/pull/786) Improve error message of `SendTransaction`/`SendRawTransaction` JSON-RPC APIs. - [cronos#222](https://github.com/crypto-org-chain/cronos/pull/222) change solc 0.6.11 to 0.6.8 (from dapp cachix) and update hermes to 0.8. - *November 19, 2021* ## v0.6.2 ### Bug Fixes + - [tharsis#720](https://github.com/tharsis/ethermint/pull/720) traceTransaction fails for succesful tx - [tharsis#743](https://github.com/tharsis/ethermint/pull/743) missing debug_tranceBlockByHash RPC method and fix debug_traceBlock* - [tharsis#746](https://github.com/tharsis/ethermint/pull/746) set debug based on tracer @@ -377,16 +383,16 @@ ## v0.6.1 ### State Machine Breaking + - [cronos#190](https://github.com/crypto-org-chain/cronos/pull/190) upgrade ethermint to v0.7.2 with (#661) and (#689) ### Bug Fixes + - [cronos#187](https://github.com/crypto-org-chain/cronos/pull/187) multiple denoms can be mapped to same contract - [cronos#157](https://github.com/crypto-org-chain/cronos/pull/185) cronos params name has an unnecessary Key prefix - [cronos#179](https://github.com/crypto-org-chain/cronos/pull/179) fix denom (symbol) in CRC20Module - [cronos#178](https://github.com/crypto-org-chain/cronos/pull/178) version CLI command doesn't output any text - - *October 13, 2021* ## v0.6.0 @@ -398,16 +404,19 @@ This version removes gravity-bridge from cronos, also includes multiple bug fixe - [cronos#171](https://github.com/crypto-org-chain/cronos/pull/171) remove gravity-bridge for mainnet launch ### Bug Fixes + - [cronos#144](https://github.com/crypto-org-chain/cronos/pull/144) fix events in autodeploy crc20 module contract - [gravity-bridge#17](https://github.com/crypto-org-chain/gravity-bridge/pull/17) processEthereumEvent does not persist hooks emitted event - [gravity-bridge#20](https://github.com/crypto-org-chain/gravity-bridge/pull/20) fix undeterministic in consensus - [cronos#167](https://github.com/crypto-org-chain/cronos/pull/167) upgrade cosmos-sdk to 0.44.2 ### Improvements + - [cronos#162](https://github.com/crypto-org-chain/cronos/pull/162) bump ibc-go to v1.2.1 with hooks support - [cronos#169](https://github.com/crypto-org-chain/cronos/pull/169) bump ethermint to v0.7.1 and go-ethereum to v10.1.3-patched which include (CVE-2021-39137) hotfix *October 4, 2021* + ## v0.5.5 This version fixes various bugs regarding ibc fund transfer and EVM-related in ethermint. @@ -428,6 +437,7 @@ We also enable swagger doc ui and add the token mapping state in genesis. - [cronos#128](https://github.com/crypto-org-chain/cronos/pull/128) add native message to update token mapping *September 22, 2021* + ## v0.5.4 This version is the same as v0.5.3 with a patched version of ethermint which include a bug fix on the transaction receipts events and on concurrent query. @@ -438,12 +448,14 @@ This version is the same as v0.5.3 with a patched version of ethermint which inc - [cronos#98](https://github.com/crypto-org-chain/cronos/pull/98) node crash under concurrent query *September 21, 2021* + ## v0.5.3 This version contains several new features, it enables gravity bridge in Cronos and automatic token conversion for bridging tokens to crc20 tokens. It also fixes the decimal conversion issues in the CRO tokens from Crypto.org Chain. In addition to that, it also upgrade ethermint to its latest version (v0.5.0.x) which bring several breaking changes (see [changelog](https://github.com/tharsis/ethermint/blob/1a01c6a992c0fb94d70bb1c7127715874cefd057/CHANGELOG.md)). ### Consensus breaking changes + - [cronos#87](https://github.com/crypto-org-chain/cronos/pull/87) upgrade ethermint to v0.4.2-0.20210920104419-1a01c6a992c0 ### Features @@ -474,8 +486,6 @@ In addition to that, it also upgrade ethermint to its latest version (v0.5.0.x) - (ethermint) [tharsis#446](https://github.com/tharsis/ethermint/pull/446) fix chain state export issue - - *August 16, 2021* ## v0.5.1 diff --git a/go.mod b/go.mod index 48d051684a..ab6b3d7301 100644 --- a/go.mod +++ b/go.mod @@ -53,7 +53,7 @@ require ( github.com/HdrHistogram/hdrhistogram-go v1.1.2 // indirect github.com/StackExchange/wmi v0.0.0-20180116203802-5d049714c4a6 // indirect github.com/VictoriaMetrics/fastcache v1.6.0 // indirect - github.com/alitto/pond v1.8.2 // indirect + github.com/alitto/pond v1.8.3 // indirect github.com/aws/aws-sdk-go v1.44.203 // indirect github.com/beorn7/perks v1.0.1 // indirect github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d // indirect diff --git a/go.sum b/go.sum index 42e4ce6f0c..b9bddae45b 100644 --- a/go.sum +++ b/go.sum @@ -281,8 +281,8 @@ github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRF github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho= github.com/alexkohler/prealloc v1.0.0/go.mod h1:VetnK3dIgFBBKmg0YnD9F9x6Icjd+9cvfHR56wJVlKE= github.com/alingse/asasalint v0.0.10/go.mod h1:nCaoMhw7a9kSJObvQyVzNTPBDbNpdocqrSP7t/cW5+I= -github.com/alitto/pond v1.8.2 h1:k0k3GIE7CFLW/kyMJj5DDKLFg1VH09l8skZqg/yJNng= -github.com/alitto/pond v1.8.2/go.mod h1:CmvIIGd5jKLasGI3D87qDkQxjzChdKMmnXMg3fG6M6Q= +github.com/alitto/pond v1.8.3 h1:ydIqygCLVPqIX/USe5EaV/aSRXTRXDEI9JwuDdu+/xs= +github.com/alitto/pond v1.8.3/go.mod h1:CmvIIGd5jKLasGI3D87qDkQxjzChdKMmnXMg3fG6M6Q= github.com/allegro/bigcache v1.2.1-0.20190218064605-e24eb225f156/go.mod h1:Cb/ax3seSYIx7SuZdm2G2xzfwmv3TPSk2ucNfQESPXM= github.com/allegro/bigcache v1.2.1 h1:hg1sY1raCwic3Vnsvje6TT7/pnZba83LeFck5NrFKSc= github.com/andybalholm/brotli v1.0.2/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y= diff --git a/gomod2nix.toml b/gomod2nix.toml index 0e1480f7a5..a72c048cb2 100644 --- a/gomod2nix.toml +++ b/gomod2nix.toml @@ -66,8 +66,8 @@ schema = 3 version = "v1.6.0" hash = "sha256-u1dkRJ2Y5+hnYlkyMPm14HxKkAv999bjN622nZDjaBo=" [mod."github.com/alitto/pond"] - version = "v1.8.2" - hash = "sha256-GbTih6gDKGuCP0V0iEo1DfEePAjm6lBxiUpqmL8hD1I=" + version = "v1.8.3" + hash = "sha256-Iaiw4a8Up0tB2iu8xdmXKHwXBGWBBGnqMGfZYWNc1ng=" [mod."github.com/armon/go-metrics"] version = "v0.4.1" hash = "sha256-usxTUHA0QQMdM6sHi2z51nmnEKMbA0qUilxJFpWHlYE=" diff --git a/memiavl/db.go b/memiavl/db.go index 68748fcb95..06e9dee111 100644 --- a/memiavl/db.go +++ b/memiavl/db.go @@ -11,13 +11,15 @@ import ( "sync" "time" + "github.com/alitto/pond" "github.com/cosmos/iavl" "github.com/tidwall/wal" ) const ( - DefaultSnapshotInterval = 1000 - LockFileName = "LOCK" + DefaultSnapshotInterval = 1000 + LockFileName = "LOCK" + DefaultSnapshotWriterLimit = 4 ) var errReadOnly = errors.New("db is read-only") @@ -72,6 +74,8 @@ type DB struct { // this method is the sole entry point for tree modifications, and there's no concurrency internally // (the background snapshot rewrite is handled separately), so we don't need locks in the Tree. mtx sync.Mutex + // worker goroutine IdleTimeout = 5s + snapshotWriterPool *pond.WorkerPool } type Options struct { @@ -97,6 +101,8 @@ type Options struct { // truncate the versions after the `TargetVersion`, the `TargetVersion` becomes the latest version. // it do nothing if the target version is `0`. LoadForOverwriting bool + + SnapshotWriterLimit int } func (opts Options) Validate() error { @@ -119,6 +125,10 @@ func (opts *Options) FillDefaults() { if opts.SnapshotInterval == 0 { opts.SnapshotInterval = DefaultSnapshotInterval } + + if opts.SnapshotWriterLimit <= 0 { + opts.SnapshotWriterLimit = DefaultSnapshotWriterLimit + } } const ( @@ -217,6 +227,8 @@ func Load(dir string, opts Options) (*DB, error) { return nil, fmt.Errorf("fail to prune snapshots: %w", err) } } + // create worker pool. recv tasks to write snapshot + workerPool := pond.New(opts.SnapshotWriterLimit, opts.SnapshotWriterLimit*10) db := &DB{ MultiTree: *mtree, @@ -229,6 +241,7 @@ func Load(dir string, opts Options) (*DB, error) { snapshotKeepRecent: opts.SnapshotKeepRecent, snapshotInterval: opts.SnapshotInterval, triggerStateSyncExport: opts.TriggerStateSyncExport, + snapshotWriterPool: workerPool, } if !db.readOnly && db.Version() == 0 && len(opts.InitialStores) > 0 { @@ -604,10 +617,12 @@ func (db *DB) Copy() *DB { func (db *DB) copy(cacheSize int) *DB { mtree := db.MultiTree.Copy(cacheSize) + return &DB{ - MultiTree: *mtree, - logger: db.logger, - dir: db.dir, + MultiTree: *mtree, + logger: db.logger, + dir: db.dir, + snapshotWriterPool: db.snapshotWriterPool, } } @@ -623,7 +638,7 @@ func (db *DB) RewriteSnapshot() error { snapshotDir := snapshotName(db.lastCommitInfo.Version) tmpDir := snapshotDir + "-tmp" path := filepath.Join(db.dir, tmpDir) - if err := db.MultiTree.WriteSnapshot(path); err != nil { + if err := db.MultiTree.WriteSnapshot(path, db.snapshotWriterPool); err != nil { return errors.Join(err, os.RemoveAll(path)) } if err := os.Rename(path, filepath.Join(db.dir, snapshotDir)); err != nil { @@ -801,7 +816,7 @@ func (db *DB) WriteSnapshot(dir string) error { db.mtx.Lock() defer db.mtx.Unlock() - return db.MultiTree.WriteSnapshot(dir) + return db.MultiTree.WriteSnapshot(dir, db.snapshotWriterPool) } func snapshotName(version int64) string { @@ -901,7 +916,11 @@ func walPath(root string) string { func initEmptyDB(dir string, initialVersion uint32) error { tmp := NewEmptyMultiTree(initialVersion, 0) snapshotDir := snapshotName(0) - if err := tmp.WriteSnapshot(filepath.Join(dir, snapshotDir)); err != nil { + // create tmp worker pool + pool := pond.New(DefaultSnapshotWriterLimit, DefaultSnapshotWriterLimit*10) + defer pool.Stop() + + if err := tmp.WriteSnapshot(filepath.Join(dir, snapshotDir), pool); err != nil { return err } return updateCurrentSymlink(dir, snapshotDir) diff --git a/memiavl/export.go b/memiavl/export.go index 7455ea50b3..ec54185f2b 100644 --- a/memiavl/export.go +++ b/memiavl/export.go @@ -31,9 +31,10 @@ func NewMultiTreeExporter(dir string, version uint32, supportExportNonSnapshotVe ) if supportExportNonSnapshotVersion { db, err = Load(dir, Options{ - TargetVersion: version, - ZeroCopy: true, - ReadOnly: true, + TargetVersion: version, + ZeroCopy: true, + ReadOnly: true, + SnapshotWriterLimit: DefaultSnapshotWriterLimit, }) if err != nil { return nil, fmt.Errorf("invalid height: %d, %w", version, err) diff --git a/memiavl/go.mod b/memiavl/go.mod index ebabc1beff..dd0a1fed17 100644 --- a/memiavl/go.mod +++ b/memiavl/go.mod @@ -3,6 +3,7 @@ module github.com/crypto-org-chain/cronos/memiavl go 1.20 require ( + github.com/alitto/pond v1.8.3 github.com/cometbft/cometbft-db v0.8.0 github.com/confio/ics23/go v0.9.0 github.com/cosmos/gogoproto v1.4.10 @@ -14,7 +15,6 @@ require ( github.com/tidwall/wal v1.1.7 github.com/zbiljic/go-filelock v0.0.0-20170914061330-1dbf7103ab7d golang.org/x/exp v0.0.0-20230711153332-06a737ee72cb - golang.org/x/sync v0.1.0 ) require ( diff --git a/memiavl/go.sum b/memiavl/go.sum index 6e976413a5..0f4bfaad56 100644 --- a/memiavl/go.sum +++ b/memiavl/go.sum @@ -49,6 +49,8 @@ github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuy github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho= +github.com/alitto/pond v1.8.3 h1:ydIqygCLVPqIX/USe5EaV/aSRXTRXDEI9JwuDdu+/xs= +github.com/alitto/pond v1.8.3/go.mod h1:CmvIIGd5jKLasGI3D87qDkQxjzChdKMmnXMg3fG6M6Q= github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8= github.com/aymerick/raymond v2.0.3-0.20180322193309-b565731e1464+incompatible/go.mod h1:osfaiScAUVup+UC9Nfq76eWqDhXlp+4UYaA8uhTBO6g= github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= @@ -532,7 +534,6 @@ golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.1.0 h1:wsuoTGHzEhffawBOhz5CYhcrV4IdKZbEyZjBMuTp12o= -golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= diff --git a/memiavl/mmap.go b/memiavl/mmap.go index c4dbb2621c..402156aeef 100644 --- a/memiavl/mmap.go +++ b/memiavl/mmap.go @@ -52,11 +52,9 @@ func (m *MmapFile) Data() []byte { func Mmap(f *os.File) ([]byte, *[mmap.MaxMapSize]byte, error) { fi, err := f.Stat() - if err != nil { + if err != nil || fi.Size() == 0 { return nil, nil, err } - if fi.Size() == 0 { - return nil, nil, nil - } + return mmap.Mmap(f, int(fi.Size())) } diff --git a/memiavl/multitree.go b/memiavl/multitree.go index 1b9857bf35..789f8d2639 100644 --- a/memiavl/multitree.go +++ b/memiavl/multitree.go @@ -9,10 +9,10 @@ import ( "path/filepath" "sort" + "github.com/alitto/pond" "github.com/cosmos/iavl" "github.com/tidwall/wal" "golang.org/x/exp/slices" - "golang.org/x/sync/errgroup" ) const MetadataFileName = "__metadata" @@ -88,7 +88,7 @@ func LoadMultiTree(dir string, zeroCopy bool, cacheSize int) (*MultiTree, error) treeMap[name] = NewFromSnapshot(snapshot, zeroCopy, cacheSize) } - sort.Strings(treeNames) + slices.Sort(treeNames) trees := make([]NamedTree, len(treeNames)) treesByName := make(map[string]int, len(trees)) @@ -357,20 +357,22 @@ func (t *MultiTree) CatchupWAL(wal *wal.Log, endVersion int64) error { return nil } -func (t *MultiTree) WriteSnapshot(dir string) error { +func (t *MultiTree) WriteSnapshot(dir string, wp *pond.WorkerPool) error { if err := os.MkdirAll(dir, os.ModePerm); err != nil { return err } - // write the snapshots in parallel - g, _ := errgroup.WithContext(context.Background()) + // write the snapshots in parallel and wait all jobs done + group, _ := wp.GroupContext(context.Background()) + for _, entry := range t.trees { - tree, name := entry.Tree, entry.Name // https://github.com/golang/go/wiki/CommonMistakes#using-goroutines-on-loop-iterator-variables - g.Go(func() error { + tree, name := entry.Tree, entry.Name + group.Submit(func() error { return tree.WriteSnapshot(filepath.Join(dir, name)) }) } - if err := g.Wait(); err != nil { + + if err := group.Wait(); err != nil { return err } diff --git a/store/go.mod b/store/go.mod index 3a100b8f4a..5768717a74 100644 --- a/store/go.mod +++ b/store/go.mod @@ -22,6 +22,7 @@ require ( github.com/99designs/keyring v1.2.1 // indirect github.com/ChainSafe/go-schnorrkel v1.0.0 // indirect github.com/DataDog/zstd v1.4.5 // indirect + github.com/alitto/pond v1.8.3 // indirect github.com/armon/go-metrics v0.4.1 // indirect github.com/beorn7/perks v1.0.1 // indirect github.com/bgentry/speakeasy v0.1.1-0.20220910012023-760eaf8b6816 // indirect @@ -119,7 +120,6 @@ require ( golang.org/x/crypto v0.11.0 // indirect golang.org/x/exp v0.0.0-20230711153332-06a737ee72cb // indirect golang.org/x/net v0.12.0 // indirect - golang.org/x/sync v0.1.0 // indirect golang.org/x/sys v0.11.0 // indirect golang.org/x/term v0.10.0 // indirect golang.org/x/text v0.12.0 // indirect diff --git a/store/go.sum b/store/go.sum index 926784f554..54e3bd02a5 100644 --- a/store/go.sum +++ b/store/go.sum @@ -63,6 +63,8 @@ github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuy github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= +github.com/alitto/pond v1.8.3 h1:ydIqygCLVPqIX/USe5EaV/aSRXTRXDEI9JwuDdu+/xs= +github.com/alitto/pond v1.8.3/go.mod h1:CmvIIGd5jKLasGI3D87qDkQxjzChdKMmnXMg3fG6M6Q= github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY= github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8= github.com/armon/go-metrics v0.4.1 h1:hR91U9KYmb6bLBYLQjyM+3j+rcd/UhE+G78SFnF8gJA= @@ -636,7 +638,6 @@ golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.1.0 h1:wsuoTGHzEhffawBOhz5CYhcrV4IdKZbEyZjBMuTp12o= -golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= diff --git a/store/setup.go b/store/setup.go index 9549784467..378f422dfb 100644 --- a/store/setup.go +++ b/store/setup.go @@ -15,12 +15,13 @@ import ( ) const ( - FlagMemIAVL = "memiavl.enable" - FlagAsyncCommitBuffer = "memiavl.async-commit-buffer" - FlagZeroCopy = "memiavl.zero-copy" - FlagSnapshotKeepRecent = "memiavl.snapshot-keep-recent" - FlagSnapshotInterval = "memiavl.snapshot-interval" - FlagCacheSize = "memiavl.cache-size" + FlagMemIAVL = "memiavl.enable" + FlagAsyncCommitBuffer = "memiavl.async-commit-buffer" + FlagZeroCopy = "memiavl.zero-copy" + FlagSnapshotKeepRecent = "memiavl.snapshot-keep-recent" + FlagSnapshotInterval = "memiavl.snapshot-interval" + FlagCacheSize = "memiavl.cache-size" + FlagSnapshotWriterLimit = "memiavl.snapshot-writer-limit" ) // SetupMemIAVL insert the memiavl setter in front of baseapp options, so that @@ -28,11 +29,12 @@ const ( func SetupMemIAVL(logger log.Logger, homePath string, appOpts servertypes.AppOptions, sdk46Compact bool, supportExportNonSnapshotVersion bool, baseAppOptions []func(*baseapp.BaseApp)) []func(*baseapp.BaseApp) { if cast.ToBool(appOpts.Get(FlagMemIAVL)) { opts := memiavl.Options{ - AsyncCommitBuffer: cast.ToInt(appOpts.Get(FlagAsyncCommitBuffer)), - ZeroCopy: cast.ToBool(appOpts.Get(FlagZeroCopy)), - SnapshotKeepRecent: cast.ToUint32(appOpts.Get(FlagSnapshotKeepRecent)), - SnapshotInterval: cast.ToUint32(appOpts.Get(FlagSnapshotInterval)), - CacheSize: cast.ToInt(appOpts.Get(FlagCacheSize)), + AsyncCommitBuffer: cast.ToInt(appOpts.Get(FlagAsyncCommitBuffer)), + ZeroCopy: cast.ToBool(appOpts.Get(FlagZeroCopy)), + SnapshotKeepRecent: cast.ToUint32(appOpts.Get(FlagSnapshotKeepRecent)), + SnapshotInterval: cast.ToUint32(appOpts.Get(FlagSnapshotInterval)), + CacheSize: cast.ToInt(appOpts.Get(FlagCacheSize)), + SnapshotWriterLimit: cast.ToInt(appOpts.Get(FlagSnapshotWriterLimit)), } if opts.ZeroCopy { diff --git a/versiondb/go.mod b/versiondb/go.mod index c952d14b0b..0d33214bcc 100644 --- a/versiondb/go.mod +++ b/versiondb/go.mod @@ -4,7 +4,7 @@ go 1.20 require ( cosmossdk.io/errors v1.0.0 - github.com/alitto/pond v1.8.2 + github.com/alitto/pond v1.8.3 github.com/cometbft/cometbft v0.37.2 github.com/cometbft/cometbft-db v0.8.0 github.com/cosmos/cosmos-sdk v0.47.3 @@ -138,7 +138,6 @@ require ( golang.org/x/crypto v0.11.0 // indirect golang.org/x/exp v0.0.0-20230711153332-06a737ee72cb // indirect golang.org/x/net v0.12.0 // indirect - golang.org/x/sync v0.1.0 // indirect golang.org/x/sys v0.11.0 // indirect golang.org/x/term v0.10.0 // indirect golang.org/x/text v0.12.0 // indirect diff --git a/versiondb/go.sum b/versiondb/go.sum index 53ecdd5a1a..e19a395eb1 100644 --- a/versiondb/go.sum +++ b/versiondb/go.sum @@ -80,8 +80,8 @@ github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuy github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho= -github.com/alitto/pond v1.8.2 h1:k0k3GIE7CFLW/kyMJj5DDKLFg1VH09l8skZqg/yJNng= -github.com/alitto/pond v1.8.2/go.mod h1:CmvIIGd5jKLasGI3D87qDkQxjzChdKMmnXMg3fG6M6Q= +github.com/alitto/pond v1.8.3 h1:ydIqygCLVPqIX/USe5EaV/aSRXTRXDEI9JwuDdu+/xs= +github.com/alitto/pond v1.8.3/go.mod h1:CmvIIGd5jKLasGI3D87qDkQxjzChdKMmnXMg3fG6M6Q= github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY= github.com/apache/thrift v0.12.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ= github.com/apache/thrift v0.13.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ= @@ -901,7 +901,6 @@ golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.1.0 h1:wsuoTGHzEhffawBOhz5CYhcrV4IdKZbEyZjBMuTp12o= -golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=