[SPVR-122] Node should be up instead of down after restarting eru-agent for node #497
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
DuodenumL
force-pushed
the
feature/set-node
branch
from
941cd6f to
4a9b0a2
Compare
|
如果只是想删除 node status, 那规定 SetNodeStatus(ctx, node, ttl) 的 ttl<0 的语义是删除就可以了? |
DuodenumL
force-pushed
the
feature/set-node
branch
from
4a9b0a2 to
b83ff93
Compare
cluster/calcium/node.go
Outdated
| err := c.store.SetNodeStatus(ctx, node, -1) | ||
| if err != nil { | ||
| // don't return here | ||
| log.Errorf(ctx, "[SetNode] failed to set node status, err: %v", err) |
There was a problem hiding this comment.
"%+v", errors.WithStack(err)
| // this is heartbeat of node | ||
| func (m *Mercury) SetNodeStatus(ctx context.Context, node *types.Node, ttl int64) error { | ||
| if ttl <= 0 { | ||
| if ttl == 0 { |
There was a problem hiding this comment.
还是记录一下, 虽然不知道为啥 ttl=0 就报错, 隔壁 workload status ttl=0 是不过期, 不过先这样吧, 反正没这需求...
DuodenumL
force-pushed
the
feature/set-node
branch
from
35fe68a to
266dc06
Compare
jschwinger233
approved these changes
Oct 27, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
在SetNode的时候，如果
node.available == false，就把node status给删掉。在这个PR之前，agent在crash的时候会调用
SetNode来把当前node设置为available: false，但是没有删除node status。如果在node status过期之前，agent重启成功，那么node status就会一直是alive: true，并且selfmon无法从NodeStatusStream里监听到任何变化，导致这个node一直是available: false。