Apache
/
nifi
mirror of https://github.com/apache/nifi.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

NIFI-9055 Added handling for 0- read range to FetchS3Object (#5317)

This commit is contained in:
Joey 2021-09-13 10:19:19 -07:00 committed by GitHub
parent 3fbd9c9e14
commit 38a1f476e3
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 15 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/s3/FetchS3Object.java
View File

@ -171,9 +171,23 @@ public class FetchS3Object extends AbstractS3Processor {
request = new GetObjectRequest(bucket, key, versionId);
}
request.setRequesterPays(requesterPays);
// tl;dr don't setRange(0) on GetObjectRequest because it results in
// InvalidRange errors on zero byte objects.
//
// Amazon S3 sets byte ranges using HTTP Range headers as described in
// https://datatracker.ietf.org/doc/html/rfc2616#section-14.35 and
// https://datatracker.ietf.org/doc/html/rfc7233#section-2.1. There
// isn't a satisfiable byte range specification for zero length objects
// so 416 (Request range not satisfiable) is returned.
//
// Since the effect of the byte range 0- is equivalent to not sending a
// byte range and works for both zero and non-zero length objects,
// the single argument setRange() only needs to be called when the
// first byte position is greater than zero.
if (rangeLength != null) {
request.setRange(rangeStart, rangeStart + rangeLength - 1);
} else {
} else if (rangeStart > 0) {
request.setRange(rangeStart);
}