Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,080 Commits 326 Branches 371 Tags
Commit Graph

12080 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Xiao Chen
d81706cd99 HDFS-11120. TestEncryptionZones should waitActive. Contributed by John Zhuge. 
(cherry picked from commit 907b1431c0ead934e26cf7dbc51bfab5d7cc5ddc)
 2016-11-09 17:16:36 -08:00
Naganarasimha
108f09a763 YARN-4498. Application level node labels stats to be available in REST (addendum patch). Contributed by Bibin A Chundatt. 
(cherry picked from commit 8a2998c08c614f9c64f012f40d31404fe9217e63)
 2016-11-10 05:24:12 +05:30
Wei-Chiu Chuang
1d75da8e26 HDFS-11056. Concurrent append and read operations lead to checksum error. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 16430c64b788bce6997f3a9e4401a2914f37b959)
 2016-11-09 13:32:35 -08:00
Xiao Chen
b1b609c003 HADOOP-13590. Retry until TGT expires even if the UGI renewal thread encountered exception. 
(cherry picked from commit 367c3d41217728c2e61252c5a5235e5bc1f9822f)
(cherry picked from commit ede909144da6fe8bde2f6561fa44a458674f96e2)
 2016-11-09 09:10:06 -08:00
Mingliang Liu
ad992e05a6 HDFS-11083. Add unit test for DFSAdmin -report command. Contributed by Xiaobing Zhou 
(cherry picked from commit 62d8c17dfda75a6a6de06aedad2f22699a1cbad6)
 2016-11-08 17:10:59 -08:00
Zhe Zhang
d93dc01e3d HADOOP-13782. Make MutableRates metrics thread-local write, aggregate-on-read. Contributed by Erik Krogen. 
(cherry picked from commit 77c13c385774c51766fe505397fa916754ac08d4)
(cherry picked from commit 758638d08d3337159dd3e9fa651093d0430b399a)
 2016-11-08 16:10:53 -08:00
Naganarasimha
5f386967a1 YARN-5720. Update document for "rmadmin -replaceLabelOnNode". Contributred by Tao Jie 
(cherry picked from commit 0e75496049001b00c00901fcbfe3d4229b2fd64a)
 2016-11-08 10:37:34 +05:30
Varun Vasudev
3603e525b1 YARN-4884. Fix missing documentation about rmadmin command regarding node labels. Contributed by Kai Sasaki. 
(cherry picked from commit f1b8f6b2c16403869f78a54268ae1165982a7050)
 2016-11-08 10:22:59 +05:30
Zhe Zhang
384b7b71a3 HADOOP-13804. MutableStat mean loses accuracy if add(long, long) is used. Contributed by Erik Krogen. 
(cherry picked from commit 3dbad5d823b8bf61b643dd1057165044138b99e0)
(cherry picked from commit b245e9ce2f20bb84690bffe902a60d5e96130cdb)
 2016-11-07 16:15:20 -08:00
Steve Loughran
5231c527aa HADOOP-13797 Remove hardcoded absolute path for ls. Contributed by Christine Koppelt 2016-11-07 12:36:38 +00:00
Akira Ajisaka
21f210bada HADOOP-13798. TestHadoopArchives times out. 
(cherry picked from commit b970446b2c59f8897bb2c3a562fa192ed3452db5)
(cherry picked from commit 3b279fe4260d99f95787ea5cce428f9ef337a3cb)
 2016-11-07 19:55:10 +09:00
Sunil
ee5dd2c388 YARN-5802. updateApplicationPriority api in scheduler should ensure to re-insert app to correct ordering policy. Contributed by Bibin A Chundatt 2016-11-05 13:06:31 +05:30
Xiaoyu Yao
5073289c4a Revert "HADOOP-13565. KerberosAuthenticationHandler#authenticate should not rebuild SPN based on client request. Contributed by Xiaoyu Yao." 
This reverts commit 5abc78882590bc3c0dfb71cba2f08fd671bc1e33.
 2016-11-04 16:22:45 -07:00
Jason Lowe
4cb4e4c0a1 YARN-5837. NPE when getting node status of a decommissioned node after an RM restart. Contributed by Robert Kanter 
(cherry picked from commit 6bb741ff0ef208a8628bc64d6537999d4cd67955)
 2016-11-04 22:25:35 +00:00
Mingliang Liu
00625004b7 HDFS-11085. Add unit test for NameNode failing to start when name dir is unwritable. Contributed by Xiaobing Zhou 
(cherry picked from commit 0c0ab102ab392ba07ed2aa8d8a67eef4c20cad9b)
 2016-11-04 14:45:43 -07:00
Robert Kanter
f9a7b83a53 HADOOP-7930. Kerberos relogin interval in UserGroupInformation should be configurable (xiaochen via rkanter) 
(cherry picked from commit 0b36dcda7cbac51c725224a8e0abef4eaa004fd8)
 2016-11-04 09:24:34 -07:00
Rohith Sharma K S
c89d462761 YARN-5608. TestAMRMClient.setup() fails with ArrayOutOfBoundsException. Contributed by Daniel Templeton. 
(cherry picked from commit f9557127b3660e138cab8d9535a589677809384a)
 2016-11-04 13:52:11 +00:00
Mingliang Liu
b082d4b746 HDFS-11076. Add unit test for extended Acls. Contributed by Chen Liang 
(cherry picked from commit 7534aee09af47c6961100588312da8d133be1b27)
 2016-11-03 13:55:43 -07:00
Xiao Chen
a22ca0aa58 HADOOP-12453. Support decoding KMS Delegation Token with its own Identifier. Contributed by Xiaoyu Yao. 
(cherry picked from commit 7154a20bcb1559c23aeb3b78b920bed03d834cb5)
 2016-11-03 13:10:41 -07:00
Arpit Agarwal
3f2bfdba3e HDFS-11097. Fix warnings for deprecated StorageReceivedDeletedBlocks constructor. Contributed by Yiqun Lin. 2016-11-03 10:33:28 -07:00
Naganarasimha
605e9bf3bf YARN-5697. Use CliParser to parse options in RMAdminCLI. Contributed by Tao Jie. 2016-11-03 10:31:30 +05:30
Andrew Wang
e393dd788e HDFS-11080. Update HttpFS to use ConfigRedactor. Contributed by Sean Mackrory. 2016-11-02 19:11:25 -07:00
Varun Saxena
8602e2cb0f YARN-5815. Random failure of TestApplicationPriority.testOrderOfActivatingThePriorityApplicationOnRMRestart (Bibin A Chundatt via Varun Saxena) 2016-11-03 00:43:28 +05:30
Brahma Reddy Battula
77142827e8 HDFS-11031. Add additional unit test for DataNode startup behavior when volumes fail. Contributed by Mingliang Liu. 
(cherry picked from commit 35e6081020d34be966724f44df678f1f5ee6f6d8)
 2016-11-02 10:40:22 +05:30
Andrew Wang
9b64e30ccf HADOOP-13583. Incorporate checkcompatibility script which runs Java API Compliance Checker. 
(cherry picked from commit 0cd22d66691fdbc3fb3e0a35ad1625236c3ae3f7)
(cherry picked from commit 92fff7d065d11df4115063e4bbfb98fa0a763718)
 2016-11-01 16:36:40 -07:00
Mingliang Liu
10eb21e52e HDFS-11065. Add space quota tests for heterogenous storages. Contributed by Xiaobing Zhou 
(cherry picked from commit fa1512a740b2ed2661743d6b5483ef3eb49e5634)
 2016-11-01 14:50:16 -07:00
Jason Lowe
4a70bd86da YARN-5001. Aggregated Logs root directory is created with wrong group if nonexistent. Contributed by Haibo Chen 
(cherry picked from commit 76893a41003d57d94eb1a5f486010815266046af)
 2016-11-01 20:24:44 +00:00
Ravi Prakash
8504e20b2e HADOOP-13773. Wrong HADOOP_CLIENT_OPTS in hadoop-env on branch-2. Contributed by Fei Hui 
(cherry picked from commit 045501a538800a48b77e688e46c55dce54473041)
 2016-11-01 10:49:45 -07:00
Kihwal Lee
db6dfeca1a HDFS-10997. Reduce number of path resolving methods. Contributed by Daryn Sharp. 
(cherry picked from commit 7c1a1834e49cf88c64837d92b78bbc07ea9e9efc)
 2016-11-01 08:26:25 -05:00
Akira Ajisaka
689bbade9a HADOOP-13603. Ignore package line length checkstyle rule. Contributed by Shane Kumpf. 
(cherry picked from commit 5b577f43a269caeee776a59695427985d0cd1697)
(cherry picked from commit f0d4d7a1782ed743feb6625e207fb89a19fd6957)
 2016-11-01 21:00:04 +09:00
Akira Ajisaka
61128c50de MAPREDUCE-6795. Update the document for JobConf#setNumReduceTasks. Contributed by Yiqun Lin. 
(cherry picked from commit 235f39b7c44c5907bd4de063892905577c35cd80)
(cherry picked from commit a3897f77aefaf0a9823b31a1d3e8f4d1eded8642)
 2016-11-01 20:52:52 +09:00
Brahma Reddy Battula
c895f08936 HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Rakesh R. 
(cherry picked from commit 1c5708f6ded6ba9fc007eed93c1f65ba44451241)
 2016-11-01 10:53:37 +05:30
Kihwal Lee
91640a0170 Addendum of HDFS-10722 to include HDFS-7964 logic. 
(cherry picked from commit 4a8e6dc02fb1d0dc89c3981b444575494511444a)
(cherry picked from commit 3b2554f3b0f686a66e3df20b7c381babb9bc74d3)
(cherry picked from commit c2bb9bf00c4a32644f96c673078a812087ef3c46)
 2016-10-31 16:37:00 -07:00
Zhe Zhang
7900b1e803 Revert "HDFS-10722. Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs. Contributed by Daryn Sharp." 
This reverts commit c2bb9bf00c4a32644f96c673078a812087ef3c46.
 2016-10-31 16:35:05 -07:00
Kihwal Lee
c2bb9bf00c HDFS-10722. Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs. Contributed by Daryn Sharp. 
(cherry picked from commit 4a8e6dc02fb1d0dc89c3981b444575494511444a)
(cherry picked from commit 3b2554f3b0f686a66e3df20b7c381babb9bc74d3)
 2016-10-31 16:30:39 -07:00
Akira Ajisaka
a17fc1dca3 HDFS-10485. Fix findbugs warning in FSEditLog.java. (aajisaka) 
(cherry picked from commit e620530301fd3e62537d4b7bc3d8ed296bda1ffc)
(cherry picked from commit b8c1db6f63f9d01f44f8f43921aadfe999eefd76)
 2016-10-31 16:24:35 -07:00
Jing Zhao
e327325a37 HDFS-7964. Add support for async edit logging. Contributed by Daryn Sharp. 
(cherry picked from commit 2151716832ad14932dd65b1a4e47e64d8d6cd767)
(cherry picked from commit 3cb7ae11a839c01b8be629774874c1873f51b747)
 2016-10-31 16:24:35 -07:00
Mingliang Liu
fece3c881b HDFS-11030. TestDataNodeVolumeFailure#testVolumeFailure is flaky (though passing). Contributed by Mingliang Liu 
(cherry picked from commit 6bb12ecc3f63afb2070296c2db009c6f9a2d884b)
 2016-10-31 16:06:41 -07:00
Steve Loughran
a83f10c90d HADOOP-13680. fs.s3a.readahead.range to use getLongBytes. Contributed by Abhishek Modi. 2016-10-31 20:53:22 +00:00
Arpit Agarwal
f0a6d17e3b HDFS-11074. Remove unused method FsDatasetSpi#getFinalizedBlocksOnPersistentStorage. Contributed by Hanisha Koneru. 2016-10-31 12:40:03 -07:00
Naganarasimha
b8ae7a8ee4 YARN-4498. Application level node labels stats to be available in REST. Contributed by Bibin A Chundatt. 
(cherry picked from commit e4023f0d8177ac37a75685b55c11991ca4ea2c23)
 2016-10-31 19:09:53 +05:30
Akira Ajisaka
757b38ea43 MAPREDUCE-6799. Document mapreduce.jobhistory.webapp.https.address in mapred-default.xml. Contributed by Yiqun Lin. 
(cherry picked from commit f646fe34be704a22cbea322cce2826f8f201bd34)
(cherry picked from commit ef13a5ef9d7859cbb720ee46ebed8172358ad0d5)
 2016-10-31 15:19:37 +09:00
Varun Saxena
367cb029d4 YARN-5773. RM recovery too slow due to LeafQueue#activateApplications (Bibin A Chundatt via Varun Saxena) 2016-10-30 11:50:54 +05:30
Wei-Chiu Chuang
9b6d27793d HDFS-11061. Update dfs -count -t command line help and documentation. Contributed by Yiqun Lin. 
(cherry picked from commit 112f04eb1b771e410ffd232a46e0ee2bc7b99d06)
(cherry picked from commit 9ba283c27d73962a894b33a89f17d3ec697bb388)
 2016-10-28 15:37:53 -07:00
Mingliang Liu
616c1194fe HDFS-11047. Remove deep copies of FinalizedReplica to alleviate heap consumption on DataNode. Contributed by Xiaobing Zhou 
(cherry picked from commit 2848a27700e76fc2c2d3ab87adba54ca8e93ba39)
 2016-10-28 11:18:26 -07:00
Mingliang Liu
a8a99f3423 HDFS-11048. Audit Log should escape control characters. Contributed by Eric Badger 
(cherry picked from commit 8a9388e5f6d622152798aaaa256064919e4fec37)
 2016-10-28 11:01:09 -07:00
Jason Lowe
93165c1ade YARN-4963. capacity scheduler: Make number of OFF_SWITCH assignments per heartbeat configurable. Contributed by Nathan Roberts 2016-10-28 17:44:56 +00:00
Jason Lowe
9cc7ab4e96 YARN-5767. Fix the order that resources are cleaned up from the local Public/Private caches. Contributed by Chris Trezzo 
(cherry picked from commit 1b79c417dca17bcd2e031864bc6ca07254c61b47)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
 2016-10-28 16:11:59 +00:00
Jason Lowe
83bb428e6c YARN-5027. NM should clean up app log dirs after NM restart. Contributed by sandflee 
(cherry picked from commit 7146359bfd436a76585fb1f3ea93716795308cec)
 2016-10-28 15:50:22 +00:00
Jason Lowe
558e53b10b HADOOP-13770. Shell.checkIsBashSupported swallowed an interrupted exception. Contributed by Wei-Chiu Chuang 
(cherry picked from commit c017171da00a6cd71a2901c84a0298ce14a49e23)
 2016-10-28 15:06:57 +00:00