Skip to content

Commit

Permalink
chore: ensure generation is included when a grpc ReadObject is retried (
Browse files Browse the repository at this point in the history
#2741)

If a read is opened without a generation, we want to ensure we restart for the same generation. We have this behavior for json

This is mainly to avoid a race condition of reading an object (gen=1), that object being replaced (gen=2) then retry attempts to read it again we need to read gen=1 otherwise we will return corrupted data.
  • Loading branch information
BenWhitehead authored Sep 26, 2024
1 parent 3bce87a commit f195f55
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 5 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -264,12 +264,15 @@ private void ensureStreamOpen() {
alg,
() -> {
ReadObjectObserver tmp = new ReadObjectObserver();
ReadObjectRequest request = req;
ReadObjectRequest.Builder builder = req.toBuilder();
long currentFetchOffset = fetchOffset.get();
if (request.getReadOffset() != currentFetchOffset) {
request = req.toBuilder().setReadOffset(currentFetchOffset).build();
if (req.getReadOffset() != currentFetchOffset) {
builder.setReadOffset(currentFetchOffset);
}
read.call(request, tmp);
if (metadata != null && req.getGeneration() == 0) {
builder.setGeneration(metadata.getGeneration());
}
read.call(builder.build(), tmp);
ApiExceptions.callAndTranslateApiException(tmp.open);
return tmp;
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,8 @@ public final class ITGapicUnbufferedReadableByteChannelTest {

private final ReadObjectRequest req1 =
ReadObjectRequest.newBuilder().setObject(objectName).setReadOffset(0).build();
private final ReadObjectRequest req2 = req1.toBuilder().setReadOffset(20).build();
private final ReadObjectRequest req2 =
req1.toBuilder().setGeneration(3L).setReadOffset(20).build();
private final ReadObjectResponse resp1 =
ReadObjectResponse.newBuilder()
.setMetadata(expectedResult)
Expand Down

0 comments on commit f195f55

Please sign in to comment.