feat: adding column of table schema change #8063
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
BugenZhao
changed the title
github-actions
bot
added
type/feature
user-facing-changes
…-part-2 Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
6 tasks
mergify bot
pushed a commit
that referenced
this pull request
Feb 23, 2023
This PR moves the `ColIndexMapping` utility from the frontend crate to the common crate, since the meta service will also rely on this (#8063). The frontend-specific methods are provided by an extension trait. Approved-By: st1page Approved-By: xxchan
…-part-2 Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
github-actions
bot
removed
the
user-facing-changes
BugenZhao
changed the title
BugenZhao
commented
Feb 24, 2023
|
|
||
| table_fragments.insert(table_id, table_fragment.clone()); | ||
|
|
||
| // Update downstream `Merge`s. |
There was a problem hiding this comment.
This part is too dirty. 🥵
| @@ -842,18 +924,26 @@ where | |||
| pub async fn get_downstream_chain_fragments( | |||
| &self, | |||
| table_id: TableId, | |||
| ) -> MetaResult<Vec<Fragment>> { | |||
| ) -> MetaResult<Vec<(DispatchStrategy, Fragment)>> { | |||
There was a problem hiding this comment.
We also need the output indices in the dispatchers (then map it with column index mapping), so Fragment is not sufficient here.
| let added_upstream_actor_id = update.added_upstream_actor_id.clone(); | ||
| let removed_upstream_actor_id: HashSet<_> = | ||
| if update.new_upstream_fragment_id.is_some() { | ||
| select_all.upstream_actor_ids().iter().copied().collect() |
There was a problem hiding this comment.
Remove all actors if fragment id is changed.
risingwave/proto/stream_plan.proto
Lines 47 to 50 in 54a92dd
Codecov Report
@@ Coverage Diff @@
## main #8063 +/- ##
==========================================
- Coverage 71.70% 71.59% -0.11%
==========================================
Files 1131 1131
Lines 182325 182672 +347
==========================================
+ Hits 130731 130780 +49
- Misses 51594 51892 +298
Flags with carried forward coverage won't be shown. Click here to find out more.
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
yezizp2012
approved these changes
Feb 27, 2023
fuyufjh
reviewed
Feb 27, 2023
Comment on lines
+346
to
+349
| self.notify_fragment_mapping(&old_table_fragment, Operation::Delete) | ||
| .await; | ||
| self.notify_fragment_mapping(&table_fragment, Operation::Add) | ||
| .await; |
There was a problem hiding this comment.
Can this 2 lines be merged into 1 Update?
There was a problem hiding this comment.
I guess we can't. By specifying Update we will update the vnode mapping for all fragment in the list that is used for scaling, while in this case, the fragment list is replaced and we should delete the entry of vnode mapping for old fragments.
…-part-2 Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
BugenZhao
force-pushed
the
bz/schema-change-meta-part-2
branch
from
6722da1 to
7bb5515
Compare
7 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Bugen Zhao i@bugenzhao.com
I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
This PR implements adding columns of table schema change on the meta service and compute node.
ReplaceTablewhich will generateUpdatemutation, sharing some logic with scaling.Note: this PR should not be marked as user-facing, until #7906 is done.
Tracking issue: #6903.
Close #7908. Close #7910.
Checklist
./risedev check(or alias,./risedev c)Documentation
Click here for Documentation
Types of user-facing changes
Please keep the types that apply to your changes, and remove the others.
Release note