belugabehr
eca05917d6
HADOOP-16951: Tidy Up Text and ByteWritables Classes.
...
1. Remove superfluous code
2. Remove superfluous comments
3. Checkstyle fixes
4. Remove methods that simply call super.method()
5. Use Java 8 facilities to streamline code where applicable
6. Simplify and unify some of the constructs between the two classes
7. Expanding of the arrays be 1.5x instead of 2x per expansion.
2020-04-17 08:16:12 -07:00
Sunil G
2fe122e322
YARN-10154. CS Dynamic Queues cannot be configured with absolute resources. Contributed by Manikandan R.
2020-04-17 18:51:19 +05:30
Mukund Thakur
56350664a7
HADOOP-13873. log DNS addresses on s3a initialization.
...
Contributed by Mukund Thakur.
If you set the log org.apache.hadoop.fs.s3a.impl.NetworkBinding
to DEBUG, then when the S3A bucket probe is made -the DNS address
of the S3 endpoint is calculated and printed.
This is useful to see if a large set of processes are all using
the same IP address from the pool of load balancers to which AWS
directs clients when an AWS S3 endpoint is resolved.
This can have implications for performance: if all clients
access the same load balancer performance may be suboptimal.
Note: if bucket probes are disabled, fs.s3a.bucket.probe = 0,
the DNS logging does not take place.
Change-Id: I21b3ac429dc0b543f03e357fdeb94c2d2a328dd8
2020-04-17 14:15:38 +01:00
Ayush Saxena
37d6582223
HDFS-15266. Add missing DFSOps Statistics in WebHDFS. Contributed by Ayush Saxena.
2020-04-17 13:32:02 +05:30
Stephen O'Donnell
3481895f8a
HDFS-15283. Cache pool MAXTTL is not persisted and restored on cluster restart. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-04-16 20:18:25 -07:00
Eric Yang
053e39e1e4
MAPREDUCE-7273. Fixed Job History server token renewal.
...
Contributed by Peter Bacsko via eyang
2020-04-16 20:17:21 -04:00
Mukund Thakur
8505840c26
HADOOP-16979. S3Guard auth mode should be set to false by default in integration tests. ( #1958 ) Contributed by Mukund Thakur.
2020-04-16 13:02:30 +02:00
Ayush Saxena
cc5c1da7c1
HDFS-15277. Parent directory in the explorer does not support all path formats. Contributed by Jianfei Jiang.
2020-04-15 23:01:58 +05:30
Vinayakumar B
4d24d99e85
HADOOP-16985. Handle release package related issues ( #1957 )
2020-04-15 20:38:19 +05:30
Szilard Nemeth
4db598e0e6
YARN-9999. TestFSSchedulerConfigurationStore: Extend from ConfigurationStoreBaseTest, general code cleanup. Contributed by Benjamin Teke
2020-04-15 13:18:44 +02:00
Prabhu Joseph
94f7470c71
YARN-10233. Fix YARN UI2 Daemon Logs
...
Contributed by Akhil PB.
2020-04-15 13:39:37 +05:30
Szilard Nemeth
55fcbcb5c2
YARN-10234. FS-CS converter: don't enable auto-create queue property for root. Contributed by Peter Bacsko
2020-04-15 07:24:04 +02:00
sunlisheng
ae474e1ae5
HADOOP-16982. Update Netty to 4.1.48.Final. Contributed by Lisheng Sun.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2020-04-15 06:41:32 +09:00
bilaharith
0ad0102678
HADOOP-16855. Changing wildfly dependency scope in hadoop-azure to compile
...
Contributed by Biliharith
2020-04-14 19:17:12 +01:00
Mukund Thakur
7b2d84d19c
HADOOP-16465 listLocatedStatus() optimisation ( #1943 )
...
Contributed by Mukund Thakur
Optimize S3AFileSystem.listLocatedStatus() to perform list
operations directly and then fallback to head checks for files
2020-04-14 17:19:51 +01:00
Steve Loughran
aeeebc5e79
HADOOP-16941. ITestS3GuardOutOfBandOperations.testListingDelete failing on versioned bucket ( #1919 )
...
Contributed by Steve Loughran.
Removed the failing probe and replacing with two probes which will fail
on both versioned and unversioned buckets.
2020-04-14 10:56:00 +01:00
Prabhu Joseph
c791b0e90e
YARN-10219. Fix YARN Native Service Placement Constraints with Node Attributes.
...
Contributed by Eric Yang.
2020-04-14 12:34:53 +05:30
Ayush Saxena
3edbe8708a
HADOOP-16958. NPE when hadoop.security.authorization is enabled but the input PolicyProvider for ZKFCRpcServer is NULL. Contributed by Ctest.
2020-04-14 02:13:42 +05:30
Eric E Payne
11d17417ce
MAPREDUCE-7272. TaskAttemptListenerImpl excessive log messages. Contributed by Ahmed Hussein (ahussein)
2020-04-13 18:20:07 +00:00
Ayush Saxena
8d49229c37
HDFS-15247. RBF: Provide Non DFS Used per DataNode in DataNode UI. Contributed by Lisheng Sun.
2020-04-12 12:05:50 +05:30
Ayush Saxena
2ea5adf2de
HADOOP-16967. TestSequenceFile#testRecursiveSeqFileCreate fails in subsequent run. Contributed by Ctest.
2020-04-12 11:31:07 +05:30
Ayush Saxena
275c478330
HDFS-15261. RBF: Add Block Related Metrics. Contributed by Ayush Saxena.
2020-04-09 23:44:36 +05:30
Jonathan Hung
23481ad378
YARN-10212. Create separate configuration for max global AM attempts. Contributed by Bilwa S T
2020-04-09 11:06:11 -07:00
Ayush Saxena
5746533cde
HADOOP-16962. Making `getBoolean` log warning message for unrecognized value. Contributed by Ctest.
2020-04-09 23:30:28 +05:30
Wei-Chiu Chuang
061afcdf30
HDFS-15269. NameNode should check the authorization API version only … ( #1945 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
2020-04-09 09:19:35 -07:00
Szilard Nemeth
c04a484aed
YARN-9995. Code cleanup in TestSchedConfCLI. Contributed by Bilwa S T.
2020-04-09 15:46:25 +02:00
Sunil G
6fb29d0f17
YARN-10226. NPE in Capacity Scheduler while using %primary_group queue mapping. Contributed by Peter Bacsko.
2020-04-09 18:51:42 +05:30
Szilard Nemeth
e2c87b7917
MAPREDUCE-7266. historyContext doesn't need to be a class attribute inside JobHistoryServer. Contributed by Siddharth Ahuja
2020-04-09 12:38:54 +02:00
Szilard Nemeth
8e4517f251
YARN-5277. When localizers fail due to resource timestamps being out, provide more diagnostics. Contributed by Siddharth Ahuja
2020-04-09 12:31:01 +02:00
Akira Ajisaka
e53d472bb0
HADOOP-14836. Upgrade maven-clean-plugin to 3.1.0 ( #1933 )
2020-04-09 01:47:31 +09:00
Ayush Saxena
1189af4746
HDFS-15263. Fix the logic of scope and excluded scope in Network Topology. Contributed by Ayush Saxena.
2020-04-08 15:55:14 +05:30
Wilfred Spiegelenburg
2214005c0f
YARN-10063. Add container-executor arguments --http/--https to usage. Contributed by Siddharth Ahuja
2020-04-08 11:30:03 +10:00
Steve Loughran
20eec95867
HADOOP-16932. distcp copy calls getFileStatus() needlessly and can fail against S3 ( #1936 )
...
Contributed by Steve Loughran.
This strips out all the -p preservation options which have already been
processed when uploading a file before deciding whether or not to query
the far end for the status of the (existing/uploaded) file to see if any
other attributes need changing.
This will avoid 404 caching-related issues in S3, wherein a newly created
file can have a 404 entry in the S3 load balancer's cache from the
probes for the file's existence prior to the upload.
It partially addresses a regression caused by HADOOP-8143,
"Change distcp to have -pb on by default" that causes a resurfacing
of HADOOP-13145, "In DistCp, prevent unnecessary getFileStatus call when
not preserving metadata"
2020-04-07 17:55:55 +01:00
Szilard Nemeth
bffb43b00e
YARN-10207. CLOSE_WAIT socket connection leaks during rendering of (corrupted) aggregated logs on the JobHistoryServer Web UI. Contributed by Siddharth Ahuja
2020-04-07 17:03:17 +02:00
Szilard Nemeth
45362a9f4c
YARN-10001. Add explanation of unimplemented methods in InMemoryConfigurationStore. Contributed by Siddharth Ahuja
2020-04-07 15:51:55 +02:00
Toshihiro Suzuki
c12ddbd1de
HDFS-15249 ThrottledAsyncChecker is not thread-safe. ( #1922 )
2020-04-07 14:38:09 +09:00
Inigo Goiri
0b855b9f35
HDFS-15256. Fix typo in DataXceiverServer#run(). Contributed by Lisheng Sun.
2020-04-06 13:05:33 -07:00
Akira Ajisaka
ab7495dc7d
MAPREDUCE-7269. TestNetworkedJob fails ( #1940 )
2020-04-06 17:57:37 +09:00
Takanobu Asanuma
b89d875f7b
HADOOP-16954. Add -S option in "Count" command to show only Snapshot Counts. Contributed by hemanthboyina.
2020-04-06 11:03:10 +09:00
Akira Ajisaka
e6455cc864
HADOOP-16949. pylint fails in the build environment ( #1926 )
2020-04-04 16:52:04 +09:00
Masatake Iwasaki
8f8be6b92a
HADOOP-16647. Support OpenSSL 1.1.1 LTS. Contributed by Rakesh Radhakrishnan.
2020-04-04 07:20:51 +09:00
Akira Ajisaka
1695d8d59c
HDFS-15258. RBF: Mark Router FSCK unstable. ( #1934 )
2020-04-04 00:13:41 +09:00
Ayush Saxena
ff50ecdc10
HADOOP-16952. Add .diff to gitignore. Contributed by Ayush Saxena.
2020-04-03 13:57:02 +05:30
Prabhu Joseph
7a39da3537
YARN-10120. Amendment fix for Java Doc.
...
Contributed by Bilwa S T.
2020-04-03 13:07:41 +05:30
Wanqiang Ji
f8e598f278
MAPREDUCE-7268. Fix TestMapreduceConfigFields ( #1935 )
2020-04-03 10:59:07 +09:00
Prabhu Joseph
25361b077b
YARN-10120. Https Support in Router WebServiceClient.
...
Contributed by Bilwa S T.
2020-04-02 18:11:43 +05:30
Takanobu Asanuma
736659e0e1
HDFS-15252. HttpFS: setWorkingDirectory should not accept invalid paths. Contributed by hemanthboyina.
2020-04-02 19:21:02 +09:00
Ayush Saxena
4a3eb10972
HDFS-15051. RBF: Impose directory level permissions for Mount entries. Contributed by Xiaoqiao He.
2020-04-02 11:06:01 +05:30
He Xiaoqiao
d3b5951572
HDFS-15242. Add metrics for operations hold lock times of FsDatasetImpl. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
2020-04-01 16:36:46 -07:00
Jason Lowe
c613296dc8
MAPREDUCE-7241. FileInputFormat listStatus with less memory footprint. Contributed by Zhihua Deng
2020-04-01 07:46:33 -05:00