release tensor by ReleaseTensor instruction #4737
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…leasing when exiting Signed-off-by: daquexian <daquexian566@gmail.com>
lixinqi
reviewed
Apr 26, 2021
| @@ -77,6 +77,12 @@ EagerMirroredTensorImpl::EagerMirroredTensorImpl( | |||
| eager_blob_object_(eager_blob_object) { | |||
| dtype_ = CHECK_JUST(DType::GetDTypeByDataType(eager_blob_object->blob_desc().data_type())); | |||
| tensor_storage_ = std::make_shared<TensorStorage>(eager_blob_object->tensor_buffer()); | |||
| tensor_storage_->set_releaser_hook( | |||
There was a problem hiding this comment.
const auto& eager_blob_object = this->eager_blob_object_;
const auto& parallel_desc = this->parallel_desc();| @@ -77,6 +77,12 @@ EagerMirroredTensorImpl::EagerMirroredTensorImpl( | |||
| eager_blob_object_(eager_blob_object) { | |||
| dtype_ = CHECK_JUST(DType::GetDTypeByDataType(eager_blob_object->blob_desc().data_type())); | |||
| tensor_storage_ = std::make_shared<TensorStorage>(eager_blob_object->tensor_buffer()); | |||
| tensor_storage_->set_releaser_hook( | |||
| [this](const std::shared_ptr<eager::TensorBuffer>& tensor_buffer) { | |||
There was a problem hiding this comment.
[eager_blob_object, parallel_desc]
There was a problem hiding this comment.
this只是裸指针，不能保证执行的时候该指针还有效。
| @@ -77,6 +77,12 @@ EagerMirroredTensorImpl::EagerMirroredTensorImpl( | |||
| eager_blob_object_(eager_blob_object) { | |||
| dtype_ = CHECK_JUST(DType::GetDTypeByDataType(eager_blob_object->blob_desc().data_type())); | |||
| tensor_storage_ = std::make_shared<TensorStorage>(eager_blob_object->tensor_buffer()); | |||
| tensor_storage_->set_releaser_hook( | |||
| [this](const std::shared_ptr<eager::TensorBuffer>& tensor_buffer) { | |||
| PhysicalRun([this](const std::shared_ptr<InstructionsBuilder>& builder) { | |||
lixinqi
approved these changes
Apr 26, 2021
liujuncheng
pushed a commit
that referenced
this pull request
Jun 3, 2021
* release tensor by instructions, update shut_down_util, skip tensor releasing when exiting Signed-off-by: daquexian <daquexian566@gmail.com> * Captures shared_ptr instread of raw pointer Co-authored-by: lixinqi <lixinqi0703106@163.com> Former-commit-id: 5170ffa
mergify bot
added a commit
that referenced
this pull request
Mar 22, 2023
1. <del>#4737 这个很早之前的 PR 在 python 退出后会跳过 tensor 的析构，导致 oom，当时这么做的原因已经不记得了，现在来看不跳过也想不到会有什么问题，所以本 PR 恢复为正常执行析构。</del> 更新：CI 遇到问题，待进一步定位，这个 PR 先不包含这部分改动了。 原 oom 问题可能和 #9681 这个 PR 把设置 is_shutting_down 的时机提前了有关 3. 修复线程不安全导致的 python stack getter 偶发 segfault 的 bug --------- Signed-off-by: daquexian <daquexian566@gmail.com> Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
添加对 ReleaseTensor 指令的调用、在 python 退出时跳过 tensor 析构