Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix sst_dump not able to open ingested file #6673

Closed
wants to merge 1 commit into from

Conversation

Connor1996
Copy link
Contributor

@Connor1996 Connor1996 commented Apr 9, 2020

When investigating #6666, we encounter an error for sst_dump to dump an ingested SST file with global seqno.

Corruption: An external sst file with version 2 have global seqno property with value ��/, while largest seqno in the file is 0)

Same as #5097, it is due to SstFileReader don't know the largest seqno of a file, it will fail this check when it open a file with global seqno.

if (global_seqno != 0 && global_seqno != largest_seqno) {

Test:
run it manually

Signed-off-by: Connor1996 <zbk602423539@gmail.com>
@yiwu-arbug
Copy link
Contributor

@ajkr mind also merge this one? thanks.

@yiwu-arbug yiwu-arbug requested a review from ajkr April 9, 2020 04:06
Copy link
Contributor

@ajkr ajkr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm.

Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@Connor1996
Copy link
Contributor Author

have no idea about why the test is failed, could you help me take a look? @ajkr

@ajkr
Copy link
Contributor

ajkr commented Apr 10, 2020

test failure appears unrelated:

db_test::DBTest.SparseMerge State: Completed
Note: Google Test filter = DBTest.SparseMerge
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from DBTest
[ RUN      ] DBTest.SparseMerge
c:\projects\rocksdb\db\db_test.cc(1341): error: Expected: (dbfull()->TEST_MaxNextLevelOverlappingBytes(handles_[1])) <= (20 * 1048576), actual: 32700451 vs 20971520
[  FAILED  ] DBTest.SparseMerge (5461 ms)
[----------] 1 test from DBTest (5461 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (5461 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] DBTest.SparseMerge

@facebook-github-bot
Copy link
Contributor

@ajkr merged this pull request in c8c739a.

Connor1996 added a commit to Connor1996/rocksdb that referenced this pull request Apr 14, 2020
Summary:
When investigating facebook#6666, we encounter an error for sst_dump to dump an ingested SST file with global seqno.
```
Corruption: An external sst file with version 2 have global seqno property with value ��/, while largest seqno in the file is 0)
```

Same as facebook#5097, it is due to SstFileReader don't know the largest seqno of a file, it will fail this check when it open a file with global seqno. https://github.com/facebook/rocksdb/blob/ca89ac2ba997dfa0e135bd75d4ccf6f5774a7eff/table/block_based_table_reader.cc#L730
Pull Request resolved: facebook#6673

Test Plan: run it manually

Reviewed By: cheng-chang

Differential Revision: D20937546

Pulled By: ajkr

fbshipit-source-id: c3fd04d60916a738533ee1885f3ea844669a9479
tabokie added a commit to tabokie/rocksdb that referenced this pull request Jul 5, 2021
Signed-off-by: tabokie <xy.tao@outlook.com>
Connor1996 added a commit to Connor1996/rocksdb that referenced this pull request Jul 5, 2021
Summary:
When investigating facebook#6666, we encounter an error for sst_dump to dump an ingested SST file with global seqno.
```
Corruption: An external sst file with version 2 have global seqno property with value ��/, while largest seqno in the file is 0)
```

Same as facebook#5097, it is due to SstFileReader don't know the largest seqno of a file, it will fail this check when it open a file with global seqno. https://github.com/facebook/rocksdb/blob/ca89ac2ba997dfa0e135bd75d4ccf6f5774a7eff/table/block_based_table_reader.cc#L730
Pull Request resolved: facebook#6673

Test Plan: run it manually

Reviewed By: cheng-chang

Differential Revision: D20937546

Pulled By: ajkr

fbshipit-source-id: c3fd04d60916a738533ee1885f3ea844669a9479
Connor1996 added a commit to Connor1996/rocksdb that referenced this pull request Jul 5, 2021
Summary:
When investigating facebook#6666, we encounter an error for sst_dump to dump an ingested SST file with global seqno.
```
Corruption: An external sst file with version 2 have global seqno property with value ��/, while largest seqno in the file is 0)
```

Same as facebook#5097, it is due to SstFileReader don't know the largest seqno of a file, it will fail this check when it open a file with global seqno. https://github.com/facebook/rocksdb/blob/ca89ac2ba997dfa0e135bd75d4ccf6f5774a7eff/table/block_based_table_reader.cc#L730
Pull Request resolved: facebook#6673

Test Plan: run it manually

Reviewed By: cheng-chang

Differential Revision: D20937546

Pulled By: ajkr

fbshipit-source-id: c3fd04d60916a738533ee1885f3ea844669a9479
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Connor1996 added a commit to tikv/rocksdb that referenced this pull request Jul 6, 2021
Summary:
When investigating facebook#6666, we encounter an error for sst_dump to dump an ingested SST file with global seqno.
```
Corruption: An external sst file with version 2 have global seqno property with value ��/, while largest seqno in the file is 0)
```

Same as facebook#5097, it is due to SstFileReader don't know the largest seqno of a file, it will fail this check when it open a file with global seqno. https://github.com/facebook/rocksdb/blob/ca89ac2ba997dfa0e135bd75d4ccf6f5774a7eff/table/block_based_table_reader.cc#L730
Pull Request resolved: facebook#6673

Test Plan: run it manually

Reviewed By: cheng-chang

Differential Revision: D20937546

Pulled By: ajkr

fbshipit-source-id: c3fd04d60916a738533ee1885f3ea844669a9479
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants