Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HADOOP-17869. `fs.s3a.connection.maximum` should be bigger than `fs.s3a.threads.max` (#3337). 

The value of `fs.s3a.connection.maximum` has been increased to 96

Contributed by Dongjoon Hyun

Change-Id: I9020a2bfd2a67fa7a2ec0598ed9d63e78ee99c73
This commit is contained in:
Dongjoon Hyun 2021-08-30 10:30:43 -07:00 committed by Steve Loughran
parent 4a93ca78f9
commit 8606b2cddd
No known key found for this signature in database
GPG Key ID: D22CF846DBB162A0
2 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
View File

@ -1372,7 +1372,7 @@
<property>
<name>fs.s3a.connection.maximum</name>
<value>48</value>
<value>96</value>
<description>Controls the maximum number of simultaneous connections to S3.
This must be bigger than the value of fs.s3a.threads.max so as to stop
threads being blocked waiting for new HTTPS connections.

2
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java
View File

@ -147,7 +147,7 @@ public final class Constants {
// number of simultaneous connections to s3
public static final String MAXIMUM_CONNECTIONS = "fs.s3a.connection.maximum";
public static final int DEFAULT_MAXIMUM_CONNECTIONS = 48;
public static final int DEFAULT_MAXIMUM_CONNECTIONS = 96;
// connect to s3 over ssl?
public static final String SECURE_CONNECTIONS =