-
Notifications
You must be signed in to change notification settings - Fork 87
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: bump all major dependencies #2007
Draft
SuperFluffy
wants to merge
1
commit into
main
Choose a base branch
from
superfluffy/tendermint-0.40
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bfebff2
to
bbe36c1
Compare
major bumps are: 1. axum: 0.7.9 2. celestia-types: 0.10.0 3. cnidarium: 0.83.0 (instead of upstream rev ac7abacc) 4. http: 1.2.0 5. hyper: 1.6.0 6. ibc-types: 0.15.0 7. jsonrpsee: 0.24.8 8. metrics: 0.24.1 9. metrics-exporter-prometheus: 0.16.2 10. opentelemetry: 0.27.1 11. opentelemetry-otlp: 0.27.0 12. opentelemetry-semantic-conventions: 0.27.0 13. opentelemetry_sdk: 0.27.1 14. pbjson-build: 0.7.0 15. pbjson-types: 0.7.0 16. penumbra-ibc (renamed to penumbra-sdk-ibc): 1.0.1 16. penumbra-proto (renamed to penumbra-sdk-proto): 1.0.1 17. penumbra-tower-trace (renamed to penumbra-sdk-tower-trace): 1.0.1 18. prost: 0.13.4 19. prost-build: 0.13.4 20. reqwest: 0.12.12 21. tendermint: 0.40.1 22. tendermint-config: 0.40.1 23. tendermint-proto: 0.40.1 24. tendermint-rpc: 0.40.1 25. tonic: 0.12.3 26. tonic-build: 0.12.3 27. tower: 0.5.2 28. tower-http: 0.6.2 29. tracing-opentelemetry: 0.28.0 removed: 1. celestia-tendermint: celestia-types now relies on upstream tendermint and celestia-tendermint is no longer maintained. 2. opentelemetry-stdout: not to be used in production code and the APIs we used were removed. 3. isahc: replaced by reqwest because it has not seen updates in a long time and was not updated to the work with hyper and http 1.0 stabilization. notable overrides: this patch relies on changes to a number of dependencies that have not yet been upstreamed: 1. penumbra-sdk-ibc, penumbra-sdk-proto, penumbra-sdk-tower-trace need a bump to their dependencies on tonic, tower, tower-actor. Tracked in penumbra-zone/penumbra#5055 2. tower-actor: needs a bump to tower. Tracked in penumbra-zone/tower-actor#5 3. tonic: has its tower dependency bumped to 0.5.2 upstream but is still awaiting a release to 0.13.0. The tower patch was merged in hyperium/tonic@58345a6 Several crates that had a custom defined dependency now use the workspace definition. notable conductor changes: upgrade the following crates: celestia-types no longer has celestia-tendermint as a dependency and instead uses tendermint 0.40 (the same as Astria uses). This required changing a bunch imports and type paths. `Blob::new` now requires an `AppVersion` argument (only needed in tests. Mainnet is running v3.2.0, so the blackbox tests sets the same, but the conductor business logic is oblivious to that). The DataAvailability no longer has public fields and instead provides a constructor (only in tests, Conductor business logic is again oblivious). From celestia-node v0.15.0 blob.GetAll no longer returns an error if a single requested namespace (out of a range of namespaces) does not contain blobs. Instead, the namespace does not show up in the list of blobs. If there are no blobs at all then the RPC omits the field entirely, which the Rust celestia-rpc crate represents as a `Option<Vec<Blob>>`. All code related to mapping errors like `BlobNotFound` to an empty vec have been removed. The current version of celestia-node on Astria's mainnet is 0.20.0. tower introduced the BoxCloneSyncService, which makes constructing a rate-limited, buffered service a bit easier. notable sequencer-client changes: changes except for tests worked out of the box, but tokio for some reason was a direct dependency. Changed it to derive from the workspace. tendermint_rpc::endpoint::broadcast::Response has a new field codespace, which needed to be populated in test code. noteable composer changes: the api server needed refactoring because axum had breaking changes. To avoid dealing with the type parameters in the new axum::serve::Serve future, composer turns it into a trait object and wraps it in a api::Serve future. The geth related blackbox tests also started to be more flaky for unknown reasons - extending the timeout duration for the tests mostly fixes the flakiness. But this will have to be be investigated in a follow-up PR. notable bridge-withdrawer changes: axum::Server was replaced by axum::serve::Serve, which requires a tokio::net::TcpListener to be instantiated. We have introduced `api::Serve` as a type-erased wrapper to `axum::serve::Serve` to avoid having to specify its type parameteres while still providing access to its bound socket. However, because of these changes BridgeWithdrawer::new is now async. the http dependency is now specified through the workspace. The shutdown signal for the API server was also changed to be a CancellationToken. notable sequencer-relayer changes: axum underwent heavy changes and has replaced axum::Server by axum::serve::Serve future. This patch wraps the future to avoid having to name its type, erasing it, while still giving access to the bound tcp socket address. Because its constructor takes a tokio::net::TcpListener, the API server setup is now async, which in turn required SequencerRelayer::new to become async. Due to type erasure + wrapping, the shutdown signal closure now has to be passed in early on instead of right before entering sequencer-relayer's runloop. While doing so, the oneshot channel was replaced by a cancellation token. Because Celestia blobs now require an app version, the relayer write path now sets it to version 3 (because astria mainnet is targetting celestia-app 3.2.0, and the consensus node is targetting 3.3.0). In tests, isahc was replaced by reqwest. The entire workspace is migrating to hyper 1+ and http 1+ and isahc has not seen upgrades in several months and there is no work underway to make it compatible with http 1+. dependency tower was removed because it was not used. notable auctioneer changes: there was no need for the direct dependencies on http-body and http-body-util because tonic already provided reexports and/or type aliases. notable telemetry changes: opentelemetry had a couple of breaking changes that required reworking how the otel pipeline is set up. the most visible change is opentelemetry-stdout no longer allowing to write to stderr or another sink. because the otel stdout writer was never used and because opentelemetry-stdout itself notes that is should not be used in production contexts this patch removes it entirely (and with it the the need to explicitly set pretty printed logs; if services are connected to a TTY or the FORCE_STDOUT flag is set, they will always write to stdout).
bbe36c1
to
b8d771b
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
cd
composer
pertaining to composer
conductor
pertaining to the astria-conductor crate
override-freeze
proto
pertaining to the Astria Protobuf spec
sequencer
pertaining to the astria-sequencer crate
sequencer-relayer
pertaining to the astria-sequencer-relayer crate
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Bump all major dependencies
Background
An update to our Astria's dependencies was long overdue. Notable developments have been:
penumbra-*
prefix topenumbra-sdk-*
).Changes
major dependency upgrades:
removed dependencies
now relies on upstream tendermint and
celestia-tendermint is no longer maintained.
production code and the APIs we used were removed.
seen updates in a long time and was not updated
to the work with hyper and http 1.0 stabilization.
notable overrides:
this patch relies on changes to a number of
dependencies that have not yet been upstreamed:
penumbra-sdk-tower-trace need a bump to their
dependencies on tonic, tower, tower-actor. Tracked
in
bump tower@v0.5.2, tower-abci@v0.19.1, tower-actor@v0.2.0 penumbra-zone/penumbra#5055
prepare v0.2.0, bump tower@v0.5.2, quiet warnings and minimize tokio penumbra-zone/tower-actor#5
0.5.2 upstream but is still awaiting a release to
0.13.0. The tower patch was merged in
hyperium/tonic@58345a6
Other and crate/service specific changes
Several crates that had a custom defined
dependency now use the workspace definition.
All services lost their
<SERVICVE_NAME>_PRETTY_PRINT
env var, because it was no longer necessary. See the
telemetry changes below.
Conductor
celestia-types no longer has celestia-tendermint
as a dependency and instead uses tendermint 0.40
(the same as Astria uses). This required changing
a bunch imports and type paths.
Blob::new
now requires anAppVersion
argument(only needed in tests. Mainnet is running v3.2.0,
so the blackbox tests sets the same, but the
conductor business logic is oblivious to that).
Type
celestia_types::ataAvailabilityHeader
no longerhas public fields and instead provides a constructor
(only in tests, Conductor business logic is again oblivious).
From celestia-node v0.15.0 blob.GetAll no longer
returns an error if a single requested namespace
(out of a range of namespaces) does not contain
blobs. Instead, the namespace does not show up in
the list of blobs. If there are no blobs at all
then the RPC omits the field entirely, which
the Rust celestia-rpc crate represents as a
Option<Vec<Blob>>
. All code related tomapping errors like
BlobNotFound
to an emptyvec have been removed. The current version of
celestia-node on Astria's mainnet is 0.20.0.
tower introduced the BoxCloneSyncService, which
makes constructing a rate-limited, buffered
service a bit easier.
notable sequencer-client changes:
changes except for tests worked out of the box,
but tokio for some reason was a direct dependency.
Changed it to derive from the workspace.
tendermint_rpc::endpoint::broadcast::Response
has a new field codespace, which needed to be
populated in test code.
Composer
the api server needed refactoring because axum
had breaking changes. To avoid dealing with
the type parameters in the new axum::serve::Serve
future, composer turns it into a trait object and
wraps it in a api::Serve future.
The geth related blackbox tests also started
to be more flaky for unknown reasons - extending
the timeout duration for the tests mostly fixes
the flakiness. But this will have to be
be investigated in a follow-up PR.
Bridge-Withdrawer
axum::Server was replaced by axum::serve::Serve,
which requires a tokio::net::TcpListener to be
instantiated. We have introduced
api::Serve
as atype-erased wrapper to
axum::serve::Serve
toavoid having to specify its type parameteres while
still providing access to its bound socket.
However, because of these changes
BridgeWithdrawer::new is now async.
the http dependency is now specified through the
workspace.
The shutdown signal for the API server was also
changed to be a CancellationToken.
Sequencer-Relayer
axum underwent heavy changes and has replaced
axum::Server by axum::serve::Serve future. This
patch wraps the future to avoid having to name its
type, erasing it, while still giving access to the
bound tcp socket address. Because its constructor
takes a tokio::net::TcpListener, the API server
setup is now async, which in turn required
SequencerRelayer::new to become async.
Due to type erasure + wrapping, the shutdown
signal closure now has to be passed in early on
instead of right before entering
sequencer-relayer's runloop. While doing so,
the oneshot channel was replaced by a cancellation
token.
Because Celestia blobs now require an app version,
the relayer write path now sets it to version 3
(because astria mainnet is targetting celestia-app
3.2.0, and the consensus node is targetting
3.3.0).
In tests, isahc was replaced by reqwest. The
entire workspace is migrating to hyper 1+ and
http 1+ and isahc has not seen upgrades in several
months and there is no work underway to make it
compatible with http 1+.
dependency tower was removed because it was not
used.
Auctioneer
there was no need for the direct dependencies
on http-body and http-body-util because tonic
already provided reexports and/or type aliases.
Telemetry
opentelemetry had a couple of breaking changes
that required reworking how the otel pipeline
is set up.
the most visible change is opentelemetry-stdout
no longer allowing to write to stderr or another
sink. because the otel stdout writer was never
used and because opentelemetry-stdout itself
notes that is should not be used in production
contexts this patch removes it entirely (and
with it the the need to explicitly set pretty
printed logs; if services are connected to a
TTY or the FORCE_STDOUT flag is set, they will
always write to stdout).
Testing
Only dependencies were upgraded. If tests and smoke tests still work then nothing should be broken.
Changelogs
Ensure all relevant changelog files are updated as necessary. See
keepachangelog for change
categories. Replace this text with e.g. "Changelogs updated." or "No updates
required." to acknowledge changelogs have been considered.
Related Issues
Link any issues that are related, prefer full GitHub links.
closes