hadoop/hadoop-tools
Mukund Thakur 56350664a7
HADOOP-13873. log DNS addresses on s3a initialization.
Contributed by Mukund Thakur.

If you set the log org.apache.hadoop.fs.s3a.impl.NetworkBinding
to DEBUG, then when the S3A bucket probe is made -the DNS address
of the S3 endpoint is calculated and printed.

This is useful to see if a large set of processes are all using
the same IP address from the pool of load balancers to which AWS
directs clients when an AWS S3 endpoint is resolved.

This can have implications for performance: if all clients
access the same load balancer performance may be suboptimal.

Note: if bucket probes are disabled, fs.s3a.bucket.probe = 0,
the DNS logging does not take place.

Change-Id: I21b3ac429dc0b543f03e357fdeb94c2d2a328dd8
2020-04-17 14:15:38 +01:00
..
hadoop-aliyun Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-archive-logs Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-archives Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-aws HADOOP-13873. log DNS addresses on s3a initialization. 2020-04-17 14:15:38 +01:00
hadoop-azure HADOOP-16855. Changing wildfly dependency scope in hadoop-azure to compile 2020-04-14 19:17:12 +01:00
hadoop-azure-datalake Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-datajoin Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-distcp HADOOP-16932. distcp copy calls getFileStatus() needlessly and can fail against S3 (#1936) 2020-04-07 17:55:55 +01:00
hadoop-dynamometer Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-extras Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-fs2img Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-gridmix Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-kafka Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-openstack Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-pipes Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-resourceestimator Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-rumen Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-sls Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-streaming Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-tools-dist Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
pom.xml Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30