Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,301 Commits 326 Branches 371 Tags
Commit Graph

12301 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Akira Ajisaka
59b0857c6b HDFS-11140. Directory Scanner should log startup message time correctly. Contributed by Yiqun Lin. 
(cherry picked from commit 9ef89ede2f18c76c601fd585cb9d47511f5fc3ed)
(cherry picked from commit e00a124cb5c43e34674faa8b450254074b7fd3a6)
 2016-12-08 13:42:02 +09:00
Naganarasimha
5b44b0c9ed YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena 
(cherry picked from commit f3b8ff54ab08545d7093bf8861b44ec9912e8dc3)
 2016-12-08 07:37:12 +05:30
Vinayakumar B
7dc22dbe0c HDFS-9057. allow/disallow snapshots via webhdfs (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 6d2332ae375e26d024358c6e75fdb3c68a781a66)
(cherry picked from commit 4b21b47f5607027672403fce211165abe03b9a77)
 2016-12-07 15:58:29 -08:00
Junping Du
879caf688b YARN-5184. Fix up incompatible changes introduced on ContainerStatus and NodeReport. Contributed by Sangjin Lee. 2016-12-06 14:48:36 -08:00
Akira Ajisaka
460fe6594a MAPREDUCE-6816. Progress bars in Web UI always at 100%. Contributed by Shen Yinjie. 
(cherry picked from commit da4ecc9f8223ac82bff8483594fb29be2f8d4d34)
(cherry picked from commit 03854f0bc51c3357161dd7cefb5acd3487868546)
 2016-12-07 06:44:31 +09:00
Kihwal Lee
ac9801fec8 HDFS-11198. NN UI should link DN web address using hostnames. Contributed by Weiwei Yang. 2016-12-06 08:08:28 -06:00
Andrew Wang
092d0602c2 HADOOP-13861. Spelling errors in logging and exceptions for code. Contributed by Grant Sohn. 
(cherry picked from commit 7b988e88992528a0cac2ca8893652c5d4a90c6b9)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/util/GF256.java
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/KDiag.java

(cherry picked from commit 6bd600eef2f4a2ab1515947bd4e909c9954d6acf)
 2016-12-05 23:21:17 -08:00
Andrew Wang
1a7b2ab9bb Revert "HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang" 
This reverts commit 3b80424d4f2ff073310c9628f4645aab6cfb6d4d.
 2016-12-05 23:09:30 -08:00
Andrew Wang
79f830ed22 HDFS-10581. Hide redundant table on NameNode WebUI when no nodes are decomissioning. Contributed by Weiwei Yang. 
(cherry picked from commit 8e63fa98eabac55bdb2254306584ad1e759c79eb)
(cherry picked from commit 2fdae7324b1c98a2262d0de8e73c0c9099ca6ff0)
 2016-12-05 18:14:09 -08:00
Jian He
802a1fb2fc YARN-5559. Analyse 2.8.0/3.0.0 jdiff reports and fix any issues. Contributed by Akira Ajisaka & Wangda Tan 
(cherry picked from commit 43ebff2e354142bddcb42755766a965ae8a503a6)
 2016-12-05 11:40:46 -08:00
Xiao Chen
49f9e7cf71 HADOOP-13847. KMSWebApp should close KeyProviderCryptoExtension. Contributed by John Zhuge. 
(cherry picked from commit 291df5c7fb713d5442ee29eb3f272127afb05a3c)
 2016-12-05 09:35:54 -08:00
Wei-Chiu Chuang
590c7f0d78 HDFS-11181. Fuse wrapper has a typo. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit b36af9b76c37bbad0b33a16e39e69fd86dc0faee)
 2016-12-05 08:56:24 -08:00
Wangda Tan
a87abf9b69 YARN-4934. Reserved Resource for QueueMetrics needs to be handled correctly in few cases. (Sunil G via wangda) 
(cherry picked from commit fdc46bfb37776d8c41b68f6c33a2379d0f329994)
 2016-12-02 22:53:19 +00:00
Mingliang Liu
3232f5ffe6 HADOOP-13855. Fix a couple of the s3a statistic names to be consistent with the rest. Contributed by Steve Loughran 
(cherry picked from commit 6b6bc4705d2b608e810ae7d1912eac49bfd4c6ca)
 2016-12-02 13:55:17 -08:00
Mingliang Liu
4a1b7d6dee HADOOP-13857. S3AUtils.translateException to map (wrapped) InterruptedExceptions to InterruptedIOEs. Contributed by Steve Loughran 
(cherry picked from commit 2ff84a00405e977b1fd791cfb974244580dd5ae8)
 2016-12-02 13:42:58 -08:00
Wangda Tan
f9ef3e3719 YARN-4865. Track Reserved resources in ResourceUsage and QueueCapacities. (Sunil G via wangda) 2016-12-02 21:17:18 +00:00
Mingliang Liu
3b80424d4f HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang 
(cherry picked from commit c7ff34f8dcca3a2024230c5383abd9299daa1b20)
(cherry picked from commit be969e591883aa6cdd69bb62cea4e8904ece65f1)
 2016-12-02 11:38:36 -08:00
Jason Lowe
69f66424b9 MAPREDUCE-6815. Fix flaky TestKill.testKillTask(). Contributed by Haibo Chen 
(cherry picked from commit 0cfd7ad21f4457513ed3416e5d77f3123bfe9da0)
 2016-12-02 17:23:33 +00:00
Jason Lowe
e6394e3a35 YARN-5915. ATS 1.5 FileSystemTimelineWriter causes flush() to be called after every event write. Contributed by Atul Sikaria 
(cherry picked from commit 6b17d449bf93e55c4f373ba96f12af6319a8411c)
 2016-12-02 17:00:05 +00:00
Akira Ajisaka
5025a898e1 HDFS-11180. Intermittent deadlock in NameNode when failover happens. 
(cherry picked from commit 5a7941a4fc193259ab4a306f7fe4f68bf101d0e0)
 2016-12-02 11:35:22 +09:00
Kihwal Lee
dd4acebb41 HDFS-8674. Improve performance of postponed block scans. Contributed by Daryn Sharp. 
(cherry picked from commit 0d8a35bd6de5d2a5a9b816ca98f31975e94bd7c6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2016-12-01 12:20:30 -06:00
Mingliang Liu
afad13f8d9 HADOOP-13830. Intermittent failure of ITestS3NContractRootDir#testRecursiveRootListing: "Can not create a Path from an empty string". Contributed by Steve Loughran 
(cherry picked from commit 3fd844b99fdfae6be6e5e261f371d175aad14229)
 2016-11-30 13:06:22 -08:00
Junping Du
7f4bd1af52 Revert "MAPREDUCE-6565. Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution. Contributed by Li Lu." 
This reverts commit f2284f44ddaa086d60b1a917b0bfff461a2f9287.
 2016-11-30 11:23:00 -08:00
Akira Ajisaka
9a99d8c334 HADOOP-13790. Make qbt script executable. Contributed by Andrew Wang. 
(cherry picked from commit be5a757096246d5c4ef73da9d233adda67bd3d69)
(cherry picked from commit 20c544d117cefb804bd8d11764302b9d64a07196)
 2016-12-01 03:53:44 +09:00
Junping Du
f2284f44dd MAPREDUCE-6565. Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution. Contributed by Li Lu. 
(cherry picked from commit 8f6e14399a3e77e1bdcc5034f7601e9f62163dea)
(cherry picked from commit 376a2439db9cd55ed1f2965cd876b811109990ba)
(cherry picked from commit 05a570a1981597ab8d2615ed4752e63f0a21b32b)
 2016-11-29 16:52:11 -08:00
Akira Ajisaka
129beeeddd HADOOP-13812. Upgrade Tomcat to 6.0.48. Contributed by John Zhuge. 
(cherry picked from commit 00096dcc0c23af38e697cd7c6c29e3f00d744e54)
(cherry picked from commit 0c895e8a62b4b69a020a5f004e97323520a1b1d0)
 2016-11-29 19:51:38 +09:00
Robert Kanter
92ee5e2077 HADOOP-13838. KMSTokenRenewer should close providers (xiaochen via rkanter) 
(cherry picked from commit 47ca9e26fba4a639e43bee5bfc001ffc4b42330d)
(cherry picked from commit 285edf691db42a3e17138d960f8560c62d8d0dde)
 2016-11-28 18:23:23 -08:00
Mingliang Liu
59f6067568 HADOOP-13823. s3a rename: fail if dest file exists. Contributed by Steve Loughran 
(cherry picked from commit d60a60be8aa450c44d3be69d26c88025e253ac0c)
 2016-11-28 16:42:41 -08:00
Brahma Reddy Battula
713d800e7b HADOOP-13820. Replace ugi.getUsername() with ugi.getShortUserName() in viewFS. Contributed By Brahma Reddy Battula. 
(cherry picked from commit ee3d437a33c2106ea416078e722ff6184106a0bc)
(cherry picked from commit 1c0fa25dc1c4bc787fe4d2809f162bea475a8b76)
 2016-11-25 20:20:37 +05:30
Steve Loughran
db37ad0ebf HADOOP-12804. Read Proxy Password from Credential Providers in S3 FileSystem. Contributed by Larry McCay. 2016-11-25 13:33:32 +00:00
Akira Ajisaka
c9297b608c HDFS-11174. Wrong HttpFS test command in doc. Contributed by John Zhuge. 
(cherry picked from commit bb1b5e083b6f32646412b292d061ce5721f5dbe1)
(cherry picked from commit 6fe5ffd05d41727552834e54695d69a26eddd8d8)
 2016-11-25 21:29:48 +09:00
Akira Ajisaka
011d478aab HDFS-11175. Document uppercase key names are not supported in TransparentEncryption.md. Contributed by Yiqun Lin. 
(cherry picked from commit 636f7547b7067c315cf5448d77e49a8fe55651b3)
(cherry picked from commit 0df7f5cdf673d9b0f97f85099c52a0a77342836e)
 2016-11-25 17:50:43 +09:00
Steve Loughran
24715cefe8 HADOOP-10776 Open up already widely-used APIs for delegation-token fetching & renewal to ecosystem projects. Contributed by Vinod Kumar Vavilapalli 2016-11-24 17:51:49 +00:00
Brahma Reddy Battula
1e25fd6b3b HADOOP-11603. Metric Snapshot log can be changed #MetricsSystemImpl.java since all the services will be initialized. Contributed By Brahma Reddy Battula. 
(cherry picked from commit 13501053dd95f41ac14091f5bbd79bcb2a3896a3)
 2016-11-24 22:25:36 +05:30
Steve Loughran
01e3b0dae8 HADOOP-13801 regression: ITestS3AMiniYarnCluster failing. Contributed by Steve Loughran 2016-11-23 21:37:23 +00:00
Mingliang Liu
834e2ddbab HADOOP-13166. Add getFileStatus("/") test to AbstractContractGetFileStatusTest. Contributed by Steve Loughran 
(cherry picked from commit 2bf9a15e8a1ac92923b45130c3369720d4a76fe1)
 2016-11-22 14:35:32 -08:00
Mingliang Liu
ed9dcdae57 HADOOP-13655. document object store use with fs shell and distcp. Contributed by Steve Loughran 2016-11-22 13:24:42 -08:00
Brahma Reddy Battula
9be7b353dd HADOOP-13646. Remove outdated overview.html. Contributed By Brahma Reddy Battula. 
(cherry picked from commit afcf8d38e750f935c06629e641a1321b79c4cace)
 2016-11-22 20:36:32 +05:30
Kihwal Lee
f5410f0814 HDFS-10966. Enhance Dispatcher logic on deciding when to give up a source DataNode. Contributed by Mark Wagner and Zhe Zhang. 
(cherry picked from commit 774b0cd845a67bc0e2ef99b415ce1e5432769551)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
 2016-11-21 10:52:42 -06:00
Jason Lowe
74503ed6b4 YARN-5859. TestResourceLocalizationService#testParallelDownloadAttemptsForPublicResource sometimes fails. Contributed by Eric Badger 
(cherry picked from commit 009452bb6dbe5dffb0b304d67a2f360fe0eee1e2)
 2016-11-21 16:40:00 +00:00
Brahma Reddy Battula
ce4cd98784 HDFS-11101. TestDFSShell#testMoveWithTargetPortEmpty fails intermittently. Contributed By Brahma Reddy Battula. 
(cherry picked from commit f92206774899b84e8abaecdf97399b65e0995ed8)
 2016-11-21 19:36:57 +05:30
Brahma Reddy Battula
23ddb98e1b HDFS-11144. TestFileCreationDelete#testFileCreationDeleteParent fails wind bind exception. Contributed By Brahma Reddy Battula. 
(cherry picked from commit c68dad18ab5cdf01f3dea1bb5988f896609956b4)
 2016-11-21 11:00:15 +05:30
Akira Ajisaka
402606323a HADOOP-13814. Sample configuration of KMS HTTP Authentication signature is misleading. Contributed by Masahiro Tanaka. 
(cherry picked from commit c65d6b65415742288b53f8e38314b71794e47ecc)
(cherry picked from commit d29e3d0f52e869e0c87839056637d815ea308353)
 2016-11-21 11:29:00 +09:00
Jian He
f1e2e7354c YARN-4822. Refactor existing Preemption Policy of CS for easier adding new approach to select preemption candidates. Contributed by Wangda Tan 2016-11-19 01:57:11 +00:00
Wangda Tan
c34072a890 YARN-4108. CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request. (Wangda Tan) 
(cherry picked from commit 7e8c9beb4156dcaeb3a11e60aaa06d2370626913)
 2016-11-18 16:13:54 +00:00
Wei-Chiu Chuang
20c90e13b8 YARN-5271. ATS client doesn't work with Jersey 2 on the classpath. Contributed by Weiwei Yang. 
(cherry picked from commit 09520cb439f8b002e3f2f3d8f5080ffc34f4bd5c)
(cherry picked from commit c5a3ec493b8ad2fdec8a23e078c5914317d7b7c5)
 2016-11-17 22:28:37 -06:00
Mingliang Liu
416f6f3605 HADOOP-11601. Enhance FS spec & tests to mandate FileStatus.getBlocksize() >0 for non-empty files. Contributed by Steve Loughran 
(cherry picked from commit ae8849fe378e11b9db642ef7784c8e6a08199b96)
 2016-11-17 14:27:34 -08:00
Kihwal Lee
0fb1306f93 HADOOP-13742. Expose NumOpenConnectionsPerUser as a metric. Brahma Reddy Battula. 
(cherry picked from commit bd3735554fa5c3bc064c57ec78f4308430b14b48)
 2016-11-17 12:20:01 -06:00
Brahma Reddy Battula
f0e2271149 HDFS-11134. Fix bind exception threw in TestRenameWhileOpen. Contributed By Yiqun Lin. 
(cherry picked from commit b2d4b7b1b902aa830d44889d7747a4e1b816ab9d)
 2016-11-17 10:26:50 +05:30
Mingliang Liu
c60ed3a3e4 HADOOP-13822. Use GlobalStorageStatistics.INSTANCE.reset() at FileSystem#clearStatistics(). Contribute by Brahma Reddy Battula 
(cherry picked from commit aab9737a058c0bdeeb17c173e530b7e571315a56)
 2016-11-16 20:11:10 -08:00