Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[xgb] Fix xgb intern to close replaced array #3558

Merged
merged 4 commits into from
Jan 22, 2025

Conversation

ewan0x79
Copy link
Contributor

Description

close replaced array for xgb engine

@ewan0x79 ewan0x79 requested review from zachgk and a team as code owners December 10, 2024 03:43
if (handle != null && handle.get() != 0L) {
long pointer = handle.getAndSet(0L);
JniUtils.deleteDMatrix(pointer);
if (replaced == null) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not necessary, the instanceof will check null:

if (!(replaced instanceof XgbNDArray))

handle = array.handle;
format = array.format;

synchronized (this) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NDArray operation is not thread-safe. Adding synchronized here won't improve thread-safty

data = array.data;
format = array.format;

if (array.handle != null && array.handle.get() != 0L) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about this null check, if we want to prevent a closed NDArray, we should just do:

if (this.closed()

array.data = null;
array.handle = null;
array.format = null;
array.close();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The memory leak issue may caused by alternativeArray leak, I think we should close current array's alternative array as will. The code can be re-implemented as the following:

        if (!(replaced instanceof XgbNDArray)) {
            throw new IllegalArgumentException(
                    "The replaced NDArray must be an instance of XgbNDArray.");
        }
        XgbNDArray array = (XgbNDArray) replaced;
        if (isReleased()) {
            throw new IllegalArgumentException("This array is already closed");
        }
        if (replaced.isReleased()) {
            throw new IllegalArgumentException("This target array is already closed");
        }

        long pointer = handle.getAndSet(0L);
        JniUtils.deleteDMatrix(pointer);
        if (alternativeArray != null) {
            alternativeArray.close();
        }

        data = array.data;
        handle = array.handle;
        format = array.format;
        alternativeArray = array.alternativeArray;
        array.handle = null;
        array.alternativeArray = null;
        array.close();

@codecov-commenter
Copy link

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 38.46154% with 8 lines in your changes missing coverage. Please review.

Project coverage is 72.40%. Comparing base (ca39062) to head (77fa14a).
Report is 14 commits behind head on master.

Files with missing lines Patch % Lines
...st/src/main/java/ai/djl/ml/xgboost/XgbNDArray.java 38.46% 4 Missing and 4 partials ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3558      +/-   ##
============================================
+ Coverage     72.36%   72.40%   +0.03%     
- Complexity     7410     7450      +40     
============================================
  Files           674      679       +5     
  Lines         33050    33290     +240     
  Branches       3521     3548      +27     
============================================
+ Hits          23918    24103     +185     
- Misses         7499     7540      +41     
- Partials       1633     1647      +14     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@xyang16 xyang16 merged commit 0ea9091 into deepjavalibrary:master Jan 22, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants