Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Fix for getSingleObjectSummary, replacing keyCount with objectSummaries().size (#7000) 

* Instead of using keyCount, changing it to check the size of objectSummaries.

For issue:
https://github.com/apache/incubator-druid/issues/6980
https://github.com/apache/incubator-druid/issues/6980#issuecomment-460006580

* Changing another usage of keyCount with size of objectSummaries.

* Adding some comments to explain why using keyCount is not working as expected.
This commit is contained in:
anantmf 2019-02-05 15:45:45 -08:00 committed by Clint Wylie
parent ef451d3603
commit 315ccb76b8
2 changed files with 8 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3DataSegmentMover.java
View File

@ -174,7 +174,10 @@ public class S3DataSegmentMover implements DataSegmentMover
.withPrefix(s3Path)
.withMaxKeys(1)
);
if (listResult.getKeyCount() == 0) {
// Using getObjectSummaries().size() instead of getKeyCount as, in some cases
// it is observed that even though the getObjectSummaries returns some data
// keyCount is still zero.
if (listResult.getObjectSummaries().size() == 0) {
// should never happen
throw new ISE("Unable to list object [s3://%s/%s]", s3Bucket, s3Path);
}

5
extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3Utils.java
View File

@ -251,7 +251,10 @@ public class S3Utils
.withMaxKeys(1);
final ListObjectsV2Result result = s3Client.listObjectsV2(request);
if (result.getKeyCount() == 0) {
// Using getObjectSummaries().size() instead of getKeyCount as, in some cases
// it is observed that even though the getObjectSummaries returns some data
// keyCount is still zero.
if (result.getObjectSummaries().size() == 0) {
throw new ISE("Cannot find object for bucket[%s] and key[%s]", bucket, key);
}
final S3ObjectSummary objectSummary = result.getObjectSummaries().get(0);