Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,926 Commits 326 Branches 371 Tags
Commit Graph

11926 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Xuan
e0401e0781 YARN-4559. Make leader elector and zk store share the same curator 
client. Contributed by Jian He

(cherry picked from commit 890a2ebd1af51d24ccbbc5d1d65d17b24ad8ab9b)
(cherry picked from commit be3322792f8961eb3e1bec7a0119a4bf092543ee)
 2016-12-20 16:15:59 -08:00
Mingliang Liu
059d33bb8b HDFS-11094. Send back HAState along with NamespaceInfo during a versionRequest as an optional parameter. Contributed by Eric Badger 
(cherry picked from commit 012d28a1fa4a0dd7e5cdfefd85cdc8c27f6cfff6)
 2016-12-20 14:29:33 -08:00
Xiao Chen
aec0d06a34 HDFS-11195. Return error when appending files by webhdfs rest api fails. Contributed by Yuanbo Liu. 
(cherry picked from commit 5b7acdd206f5a7d1b7af29b68adaa7587d7d8c43)
 2016-12-20 12:27:44 -08:00
Brahma Reddy Battula
a43bbea3c8 HDFS-11263. ClassCastException when we use Bzipcodec for Fsimage compression. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 1b401f6a734df4e23a79b3bd89c816a1fc0de574)
 2016-12-20 21:16:54 +05:30
Akira Ajisaka
1e3304ad23 HDFS-11262. Remove unused variables in FSImage.java. Contributed by Jagadesh Kiran N. 
(cherry picked from commit d2026ae9e6c86f4fd9e48b80e71233181c1853d9)
(cherry picked from commit a95acbcb6e955cf1673f053cb9dab9e5e5dc6803)
 2016-12-20 14:37:57 +09:00
Akira Ajisaka
07a221e6aa HDFS-11249. Redundant toString() in DFSConfigKeys.java. Contributed by Jagadesh Kiran N. 
(cherry picked from commit 36947f79ba66bf8ce150813e5dc1d06c789f2b2a)
(cherry picked from commit eaa50a1087ce1c29349bc266bc485fb2222ac5c5)
 2016-12-16 00:41:03 +09:00
Xiaoyu Yao
691a32c6f6 HADOOP-13890. Maintain HTTP/host as SPNEGO SPN support and fix KerberosName parsing. Contributed by Xiaoyu Yao. 
(cherry picked from commit f5e0bd30fde654ed48fe73e5c0523030365385a4)
(cherry picked from commit 85083567be22eba76dbf1f8c18149a0493f07526)
 2016-12-14 14:01:32 -08:00
Akira Ajisaka
901bc5eef5 HDFS-11204. Document the missing options of hdfs zkfc command. Contributed by Yiqun Lin. 
(cherry picked from commit 72bff192cd37ff97442e0f8dd477fbc2e58fc12d)
(cherry picked from commit 190b2dca6204e659ed8561c682c4a03f30258298)
 2016-12-14 18:54:39 +09:00
Andrew Wang
158292fd7e HDFS-10684. WebHDFS DataNode calls fail without parameter createparent. Contributed by John Zhuge. 
(cherry picked from commit fbdbbd57cdc3d8c778fca9266a7cadf298c8ff6c)
(cherry picked from commit 6b795c34d0c534a8d5214c64036ba73708d58fd0)
 2016-12-13 18:01:40 -08:00
Xuan
120f3a0ff9 YARN-4438. Implement RM leader election with curator. Contributed by Jian He 
(cherry picked from commit 89022f8d4bac0e9d0b848fd91e9c4d700fe1cdbe)
(cherry picked from commit 2cbbf76c3dd9570e08914077de8e610d93ee0c9e)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-12-13 13:30:15 -08:00
Wangda Tan
6a18ae849f YARN-2009. CapacityScheduler: Add intra-queue preemption for app priority support. (Sunil G via wangda) 
(cherry-picked from commit 90dd3a8148468ac37a3f2173ad8d45e38bfcb0c9)
 2016-12-12 23:13:08 +00:00
Mingliang Liu
01b50b36b6 HADOOP-13871. ITestS3AInputStreamPerformance.testTimeToOpenAndReadWholeFileBlocks performance awful. Contributed by Steve Loughran 
(cherry picked from commit c6a39232456fa0c98b2b9b6dbeaec762294ca01e)
 2016-12-12 15:11:11 -08:00
Vinayakumar B
69db9eecfa HDFS-10570. Remove classpath conflicts of netty-all jar in hadoop-hdfs-client (Contributed by Vinayakumar B) 
(cherry picked from commit c6fb9f0d0b63fdd735a6f455b6aee9d248852b41)
 2016-12-12 12:12:30 +05:30
Karthik Kambatla
a0fa1f8386 YARN-5677. RM should transition to standby when connection is lost for an extended period. (Daniel Templeton via kasha) 
(cherry picked from commit aedd5c4c1b6695d46d7660597685a9a71aaffc64)
 2016-12-10 09:05:33 -08:00
Xiaoyu Yao
bd4e5bc501 HADOOP-13565. KerberosAuthenticationHandler#authenticate should not rebuild SPN based on client request. Contributed by Xiaoyu Yao. 
(cherry picked from commit 4c38f11cec0664b70e52f9563052dca8fb17c33f)
 2016-12-09 21:38:42 -08:00
Akira Ajisaka
0c58241b22 MAPREDUCE-6820. Fix dead links in Job relevant classes. Contributed by Yiqun Lin. 
(cherry picked from commit 92a8917ca5a25bf6bee75acbb554aae87766a7e7)
(cherry picked from commit 1e1ab9bc1bde48593eceafb7380d7501d4bbc200)
 2016-12-10 10:57:33 +09:00
Wei-Chiu Chuang
fa3b2bee61 HDFS-11229. HDFS-11056 failed to close meta file. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 292bd78b4494ba1a1073b32b3f89d95e72f3204d)
 2016-12-09 16:12:09 -08:00
Wei-Chiu Chuang
983d487e98 HADOOP-13824. FsShell can suppress the real error if no error message is present. Contributed by John Zhuge. 
(cherry picked from commit b606e025f10daed18b90b45ac00cd0c82e818581)

Conflicts:
   hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShell.java

(cherry picked from commit 6c93c1870c08be16876ee379cba6d59775ab6a14)
 2016-12-09 15:40:14 -08:00
Andrew Wang
e8b0ef0618 HDFS-8630. WebHDFS : Support get/set/unset StoragePolicy. Contributed by Surendra Singh Lilhore. 2016-12-09 14:56:49 -08:00
Jian He
40367c8da3 YARN-4390. Do surgical preemption based on reserved container in CapacityScheduler. Contributed by Wangda Tan 
(cherry picked from commit bb62e0592566b2fcae7136b30972aad2d3ac55b0)
 2016-12-09 20:33:54 +00:00
Andrew Wang
bced9e2e5b HDFS-11217. Annotate NameNode and DataNode MXBean interfaces as Private/Stable. Contributed by Jagadesh Kiran N. 
(cherry picked from commit 5bd7dece920b20da95755cf7fbba9b3f39c7da69)
(cherry picked from commit 85adc6677ba28bf106361be595e709941cb36496)
 2016-12-09 12:05:50 -08:00
Brahma Reddy Battula
3957161a97 HADOOP-13883. Add description of -fs option in generic command usage. Contributed By Yiqun Lin. 
(cherry picked from commit 80b8023276126353229ab8d6427c1761b2e4bb38)
 2016-12-09 18:44:27 +05:30
Vinayakumar B
b1c08737d8 HDFS-11224. Lifeline message should be ignored for dead nodes (Contributed by Vinayakumar B) 
(cherry picked from commit d1d4aba71b21871140b162583a4b94ce118e1fb3)
(cherry picked from commit 199d8b559434c296c22ce7cfe5fb8de0dead0d12)
 2016-12-09 14:58:28 +05:30
Xiao Chen
892ee3f191 HDFS-11197. Listing encryption zones fails when deleting a EZ that is on a snapshotted directory. Contributed by Wellington Chevreuil. 
(cherry picked from commit 401c7318723d8d62c7fc29728f7f4e8d336b4d2f)
 2016-12-08 12:41:31 -08:00
Akira Ajisaka
39bf840398 HDFS-11223. Fix typos in HttpFs documentations. Contributed by Yiqun Lin. 
(cherry picked from commit 4c2cf5560f6d952cfa36ef656f0b04dc3150f8b3)
(cherry picked from commit 0478597ea9ac1a0ccfa28f5faf017b78c93d16ab)
 2016-12-08 20:53:36 +09:00
Akira Ajisaka
59b0857c6b HDFS-11140. Directory Scanner should log startup message time correctly. Contributed by Yiqun Lin. 
(cherry picked from commit 9ef89ede2f18c76c601fd585cb9d47511f5fc3ed)
(cherry picked from commit e00a124cb5c43e34674faa8b450254074b7fd3a6)
 2016-12-08 13:42:02 +09:00
Naganarasimha
5b44b0c9ed YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena 
(cherry picked from commit f3b8ff54ab08545d7093bf8861b44ec9912e8dc3)
 2016-12-08 07:37:12 +05:30
Vinayakumar B
7dc22dbe0c HDFS-9057. allow/disallow snapshots via webhdfs (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 6d2332ae375e26d024358c6e75fdb3c68a781a66)
(cherry picked from commit 4b21b47f5607027672403fce211165abe03b9a77)
 2016-12-07 15:58:29 -08:00
Junping Du
879caf688b YARN-5184. Fix up incompatible changes introduced on ContainerStatus and NodeReport. Contributed by Sangjin Lee. 2016-12-06 14:48:36 -08:00
Akira Ajisaka
460fe6594a MAPREDUCE-6816. Progress bars in Web UI always at 100%. Contributed by Shen Yinjie. 
(cherry picked from commit da4ecc9f8223ac82bff8483594fb29be2f8d4d34)
(cherry picked from commit 03854f0bc51c3357161dd7cefb5acd3487868546)
 2016-12-07 06:44:31 +09:00
Kihwal Lee
ac9801fec8 HDFS-11198. NN UI should link DN web address using hostnames. Contributed by Weiwei Yang. 2016-12-06 08:08:28 -06:00
Andrew Wang
092d0602c2 HADOOP-13861. Spelling errors in logging and exceptions for code. Contributed by Grant Sohn. 
(cherry picked from commit 7b988e88992528a0cac2ca8893652c5d4a90c6b9)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/util/GF256.java
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/KDiag.java

(cherry picked from commit 6bd600eef2f4a2ab1515947bd4e909c9954d6acf)
 2016-12-05 23:21:17 -08:00
Andrew Wang
1a7b2ab9bb Revert "HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang" 
This reverts commit 3b80424d4f2ff073310c9628f4645aab6cfb6d4d.
 2016-12-05 23:09:30 -08:00
Andrew Wang
79f830ed22 HDFS-10581. Hide redundant table on NameNode WebUI when no nodes are decomissioning. Contributed by Weiwei Yang. 
(cherry picked from commit 8e63fa98eabac55bdb2254306584ad1e759c79eb)
(cherry picked from commit 2fdae7324b1c98a2262d0de8e73c0c9099ca6ff0)
 2016-12-05 18:14:09 -08:00
Jian He
802a1fb2fc YARN-5559. Analyse 2.8.0/3.0.0 jdiff reports and fix any issues. Contributed by Akira Ajisaka & Wangda Tan 
(cherry picked from commit 43ebff2e354142bddcb42755766a965ae8a503a6)
 2016-12-05 11:40:46 -08:00
Xiao Chen
49f9e7cf71 HADOOP-13847. KMSWebApp should close KeyProviderCryptoExtension. Contributed by John Zhuge. 
(cherry picked from commit 291df5c7fb713d5442ee29eb3f272127afb05a3c)
 2016-12-05 09:35:54 -08:00
Wei-Chiu Chuang
590c7f0d78 HDFS-11181. Fuse wrapper has a typo. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit b36af9b76c37bbad0b33a16e39e69fd86dc0faee)
 2016-12-05 08:56:24 -08:00
Wangda Tan
a87abf9b69 YARN-4934. Reserved Resource for QueueMetrics needs to be handled correctly in few cases. (Sunil G via wangda) 
(cherry picked from commit fdc46bfb37776d8c41b68f6c33a2379d0f329994)
 2016-12-02 22:53:19 +00:00
Mingliang Liu
3232f5ffe6 HADOOP-13855. Fix a couple of the s3a statistic names to be consistent with the rest. Contributed by Steve Loughran 
(cherry picked from commit 6b6bc4705d2b608e810ae7d1912eac49bfd4c6ca)
 2016-12-02 13:55:17 -08:00
Mingliang Liu
4a1b7d6dee HADOOP-13857. S3AUtils.translateException to map (wrapped) InterruptedExceptions to InterruptedIOEs. Contributed by Steve Loughran 
(cherry picked from commit 2ff84a00405e977b1fd791cfb974244580dd5ae8)
 2016-12-02 13:42:58 -08:00
Wangda Tan
f9ef3e3719 YARN-4865. Track Reserved resources in ResourceUsage and QueueCapacities. (Sunil G via wangda) 2016-12-02 21:17:18 +00:00
Mingliang Liu
3b80424d4f HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang 
(cherry picked from commit c7ff34f8dcca3a2024230c5383abd9299daa1b20)
(cherry picked from commit be969e591883aa6cdd69bb62cea4e8904ece65f1)
 2016-12-02 11:38:36 -08:00
Jason Lowe
69f66424b9 MAPREDUCE-6815. Fix flaky TestKill.testKillTask(). Contributed by Haibo Chen 
(cherry picked from commit 0cfd7ad21f4457513ed3416e5d77f3123bfe9da0)
 2016-12-02 17:23:33 +00:00
Jason Lowe
e6394e3a35 YARN-5915. ATS 1.5 FileSystemTimelineWriter causes flush() to be called after every event write. Contributed by Atul Sikaria 
(cherry picked from commit 6b17d449bf93e55c4f373ba96f12af6319a8411c)
 2016-12-02 17:00:05 +00:00
Akira Ajisaka
5025a898e1 HDFS-11180. Intermittent deadlock in NameNode when failover happens. 
(cherry picked from commit 5a7941a4fc193259ab4a306f7fe4f68bf101d0e0)
 2016-12-02 11:35:22 +09:00
Kihwal Lee
dd4acebb41 HDFS-8674. Improve performance of postponed block scans. Contributed by Daryn Sharp. 
(cherry picked from commit 0d8a35bd6de5d2a5a9b816ca98f31975e94bd7c6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2016-12-01 12:20:30 -06:00
Mingliang Liu
afad13f8d9 HADOOP-13830. Intermittent failure of ITestS3NContractRootDir#testRecursiveRootListing: "Can not create a Path from an empty string". Contributed by Steve Loughran 
(cherry picked from commit 3fd844b99fdfae6be6e5e261f371d175aad14229)
 2016-11-30 13:06:22 -08:00
Junping Du
7f4bd1af52 Revert "MAPREDUCE-6565. Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution. Contributed by Li Lu." 
This reverts commit f2284f44ddaa086d60b1a917b0bfff461a2f9287.
 2016-11-30 11:23:00 -08:00
Akira Ajisaka
9a99d8c334 HADOOP-13790. Make qbt script executable. Contributed by Andrew Wang. 
(cherry picked from commit be5a757096246d5c4ef73da9d233adda67bd3d69)
(cherry picked from commit 20c544d117cefb804bd8d11764302b9d64a07196)
 2016-12-01 03:53:44 +09:00
Junping Du
f2284f44dd MAPREDUCE-6565. Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution. Contributed by Li Lu. 
(cherry picked from commit 8f6e14399a3e77e1bdcc5034f7601e9f62163dea)
(cherry picked from commit 376a2439db9cd55ed1f2965cd876b811109990ba)
(cherry picked from commit 05a570a1981597ab8d2615ed4752e63f0a21b32b)
 2016-11-29 16:52:11 -08:00