Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,114 Commits 326 Branches 371 Tags
Commit Graph

12114 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Brahma Reddy Battula
9be7b353dd HADOOP-13646. Remove outdated overview.html. Contributed By Brahma Reddy Battula. 
(cherry picked from commit afcf8d38e750f935c06629e641a1321b79c4cace)
 2016-11-22 20:36:32 +05:30
Kihwal Lee
f5410f0814 HDFS-10966. Enhance Dispatcher logic on deciding when to give up a source DataNode. Contributed by Mark Wagner and Zhe Zhang. 
(cherry picked from commit 774b0cd845a67bc0e2ef99b415ce1e5432769551)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
 2016-11-21 10:52:42 -06:00
Jason Lowe
74503ed6b4 YARN-5859. TestResourceLocalizationService#testParallelDownloadAttemptsForPublicResource sometimes fails. Contributed by Eric Badger 
(cherry picked from commit 009452bb6dbe5dffb0b304d67a2f360fe0eee1e2)
 2016-11-21 16:40:00 +00:00
Brahma Reddy Battula
ce4cd98784 HDFS-11101. TestDFSShell#testMoveWithTargetPortEmpty fails intermittently. Contributed By Brahma Reddy Battula. 
(cherry picked from commit f92206774899b84e8abaecdf97399b65e0995ed8)
 2016-11-21 19:36:57 +05:30
Brahma Reddy Battula
23ddb98e1b HDFS-11144. TestFileCreationDelete#testFileCreationDeleteParent fails wind bind exception. Contributed By Brahma Reddy Battula. 
(cherry picked from commit c68dad18ab5cdf01f3dea1bb5988f896609956b4)
 2016-11-21 11:00:15 +05:30
Akira Ajisaka
402606323a HADOOP-13814. Sample configuration of KMS HTTP Authentication signature is misleading. Contributed by Masahiro Tanaka. 
(cherry picked from commit c65d6b65415742288b53f8e38314b71794e47ecc)
(cherry picked from commit d29e3d0f52e869e0c87839056637d815ea308353)
 2016-11-21 11:29:00 +09:00
Jian He
f1e2e7354c YARN-4822. Refactor existing Preemption Policy of CS for easier adding new approach to select preemption candidates. Contributed by Wangda Tan 2016-11-19 01:57:11 +00:00
Wangda Tan
c34072a890 YARN-4108. CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request. (Wangda Tan) 
(cherry picked from commit 7e8c9beb4156dcaeb3a11e60aaa06d2370626913)
 2016-11-18 16:13:54 +00:00
Wei-Chiu Chuang
20c90e13b8 YARN-5271. ATS client doesn't work with Jersey 2 on the classpath. Contributed by Weiwei Yang. 
(cherry picked from commit 09520cb439f8b002e3f2f3d8f5080ffc34f4bd5c)
(cherry picked from commit c5a3ec493b8ad2fdec8a23e078c5914317d7b7c5)
 2016-11-17 22:28:37 -06:00
Mingliang Liu
416f6f3605 HADOOP-11601. Enhance FS spec & tests to mandate FileStatus.getBlocksize() >0 for non-empty files. Contributed by Steve Loughran 
(cherry picked from commit ae8849fe378e11b9db642ef7784c8e6a08199b96)
 2016-11-17 14:27:34 -08:00
Kihwal Lee
0fb1306f93 HADOOP-13742. Expose NumOpenConnectionsPerUser as a metric. Brahma Reddy Battula. 
(cherry picked from commit bd3735554fa5c3bc064c57ec78f4308430b14b48)
 2016-11-17 12:20:01 -06:00
Brahma Reddy Battula
f0e2271149 HDFS-11134. Fix bind exception threw in TestRenameWhileOpen. Contributed By Yiqun Lin. 
(cherry picked from commit b2d4b7b1b902aa830d44889d7747a4e1b816ab9d)
 2016-11-17 10:26:50 +05:30
Mingliang Liu
c60ed3a3e4 HADOOP-13822. Use GlobalStorageStatistics.INSTANCE.reset() at FileSystem#clearStatistics(). Contribute by Brahma Reddy Battula 
(cherry picked from commit aab9737a058c0bdeeb17c173e530b7e571315a56)
 2016-11-16 20:11:10 -08:00
Jason Lowe
cb0fccad19 YARN-5836. Malicious AM can kill containers of other apps running in any node its containers are running. Contributed by Botong Huang 
(cherry picked from commit 59bfcbf3579e45ddf96db3aafccf669c8e03648f)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java
 2016-11-16 22:43:56 +00:00
Mingliang Liu
a8dadd8c53 HDFS-11105. TestRBWBlockInvalidation#testRWRInvalidation fails intermittently. Contributed by Yiqun Lin 
(cherry picked from commit c90891e7b3afaa66dbca967b83a9997e80e893d8)
 2016-11-16 13:13:19 -08:00
Jason Lowe
e27fbcb50c MAPREDUCE-6804. Add timeout when starting JobHistoryServer in MiniMRYarnCluster. Contributed by Andras Bokor 
(cherry picked from commit 41f8275c50b34c919cca26510d505775d676ebb4)
 2016-11-16 15:27:32 +00:00
Xiao Chen
cf366a0fd6 HADOOP-13815. TestKMS#testDelegationTokensOpsSimple and TestKMS#testDelegationTokensOpsKerberized Fails in Trunk. 
(cherry picked from commit 61c0bedf775e6e794d4704485ec2c41a95aecae9)
(cherry picked from commit efc9ffc3b62a3de7660ab6fadeb59c8c2bca368b)
 2016-11-15 16:26:59 -08:00
Xiao Chen
fc4c0595be YARN-5875. TestTokenClientRMService#testTokenRenewalWrongUser fails. Contributed by Gergely Novák. 
(cherry picked from commit d5c396ccb361e1a0cf4307ce1d1b37fae0eec250)
 2016-11-15 13:57:43 -08:00
Kihwal Lee
15f08e9298 HDFS-8870. Lease is leaked on write failure. Contributed by Kuhu Shukla. 
(cherry picked from commit 4fcea8a0c8019d6d9a5e6f315c83659938b93a40)
 2016-11-15 14:57:04 -06:00
Naganarasimha
5c13bb5008 YARN-4355. NPE while processing localizer heartbeat. Contributed by Varun Saxena & Jonathan Hung. 
(cherry picked from commit 7ffb9943b8838a3bb56684e0722db40d800743a2)
 2016-11-15 15:52:05 +05:30
Mingliang Liu
0feb9bb4a9 HADOOP-13810. Add a test to verify that Configuration handles &-encoded characters. Contributed by Steve Loughran 
This closes #158

(cherry picked from commit 04014c4c739bb4e3bc3fdf9299abc0f47521e8fd)
 2016-11-14 19:55:45 -08:00
Zhe Zhang
efc0e640ed Addendum of HDFS-10872. Removed chan ge in MiniHadoopClusterManager.java 2016-11-14 16:43:43 -08:00
Zhe Zhang
c6e7b631bd HDFS-10872. Add MutableRate metrics for FSNamesystemLock operations. Contributed by Erik Krogen. 
(cherry picked from commit 818915aabb22653e212f4646b1b3cd0673330328)
 2016-11-14 16:39:26 -08:00
Jason Lowe
5b1b825ac8 MAPREDUCE-6797. Job history server scans can become blocked on a single, slow entry. Contributed by Prabhu Joseph 
(cherry picked from commit 99c2bbd337942e4bc7b246a88dff53f98e530651)
 2016-11-14 20:31:38 +00:00
Brahma Reddy Battula
4dc40fe28e HDFS-11135. The tests in TestBalancer run fails due to NPE. Contributed By Yiqun Lin 
(cherry picked from commit 6efb8c9c6018ec688eeba8c61c220009ec350a44)
 2016-11-14 20:42:50 +05:30
Erik Krogen
3d52459712 HDFS-11087. NamenodeFsck should check if the output writer is still writable. Contributed by Erik Krogen. 
(cherry picked from commit 98d9e9c4c823d865ddfc017e71c82d12c8328e53)
 2016-11-13 14:54:49 -08:00
Naganarasimha
59a3d9a576 Reverting it because of YARN-5287, blocker issue has been reported. 
Revert "YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang."

This reverts commit 16c8fd9dca75e8d4f36e63ef0940aed83f8ebecf.
 2016-11-13 17:43:01 +05:30
Mingliang Liu
4bcef1c616 HADOOP-13813. TestDelegationTokenFetcher#testDelegationTokenWithoutRenewer is failing. Contributed by Mingliang Liu 
(cherry picked from commit fb3a5085e8057c79e0785e7e7bf7da031c3f3c12)
 2016-11-12 10:37:46 -08:00
Mingliang Liu
fcedb6812b HDFS-11129. TestAppendSnapshotTruncate fails with bind exception. Contributed by Brahma Reddy Battula 
(cherry picked from commit 2ee18fc15e60dff9a42a80eb0c30e8bd8cedc26a)
 2016-11-11 22:19:39 -08:00
Xiaoyu Yao
c443ecb267 HDFS-10941. Improve BlockManager#processMisReplicatesAsync log. Contributed by Chen Liang. 
(cherry picked from commit 4484b48498b2ab2a40a404c487c7a4e875df10dc)
 2016-11-11 14:32:11 -08:00
Mingliang Liu
ae7a20f524 HDFS-11122. TestDFSAdmin#testReportCommand fails due to timed out. Contributed by Yiqun Lin 
(cherry picked from commit aa6010ccca3045ce9f0bb819fb2cb7ff65e1822b)
 2016-11-11 12:35:21 -08:00
Brahma Reddy Battula
121b56de14 HDFS-11128. CreateEditsLog throws NullPointerException. Contributed by Hanisha Koneru. 
(cherry picked from commit 1ae57f0f75695178dc135b0c259d066f7da31c9d)
 2016-11-11 20:11:13 +05:30
Yongjun Zhang
51f5bab695 HADOOP-13720. Add more info to the msgs printed in AbstractDelegationTokenSecretManager. Contributed by Yongjun Zhang. 
(cherry picked from commit fd2f22adec5c2f21f792045dbfde9385c21403ec)
 2016-11-10 23:41:48 -08:00
Eric Payne
1a6a5af44a YARN-4218. Metric for resource*time that was preempted. Contributed by Chang Li. 
(cherry picked from commit dd5b9dabf90dbd68afdc87421101a5f4bad357d9)
 2016-11-10 23:32:17 +00:00
Xiao Chen
d81706cd99 HDFS-11120. TestEncryptionZones should waitActive. Contributed by John Zhuge. 
(cherry picked from commit 907b1431c0ead934e26cf7dbc51bfab5d7cc5ddc)
 2016-11-09 17:16:36 -08:00
Naganarasimha
108f09a763 YARN-4498. Application level node labels stats to be available in REST (addendum patch). Contributed by Bibin A Chundatt. 
(cherry picked from commit 8a2998c08c614f9c64f012f40d31404fe9217e63)
 2016-11-10 05:24:12 +05:30
Wei-Chiu Chuang
1d75da8e26 HDFS-11056. Concurrent append and read operations lead to checksum error. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 16430c64b788bce6997f3a9e4401a2914f37b959)
 2016-11-09 13:32:35 -08:00
Xiao Chen
b1b609c003 HADOOP-13590. Retry until TGT expires even if the UGI renewal thread encountered exception. 
(cherry picked from commit 367c3d41217728c2e61252c5a5235e5bc1f9822f)
(cherry picked from commit ede909144da6fe8bde2f6561fa44a458674f96e2)
 2016-11-09 09:10:06 -08:00
Mingliang Liu
ad992e05a6 HDFS-11083. Add unit test for DFSAdmin -report command. Contributed by Xiaobing Zhou 
(cherry picked from commit 62d8c17dfda75a6a6de06aedad2f22699a1cbad6)
 2016-11-08 17:10:59 -08:00
Zhe Zhang
d93dc01e3d HADOOP-13782. Make MutableRates metrics thread-local write, aggregate-on-read. Contributed by Erik Krogen. 
(cherry picked from commit 77c13c385774c51766fe505397fa916754ac08d4)
(cherry picked from commit 758638d08d3337159dd3e9fa651093d0430b399a)
 2016-11-08 16:10:53 -08:00
Naganarasimha
5f386967a1 YARN-5720. Update document for "rmadmin -replaceLabelOnNode". Contributred by Tao Jie 
(cherry picked from commit 0e75496049001b00c00901fcbfe3d4229b2fd64a)
 2016-11-08 10:37:34 +05:30
Varun Vasudev
3603e525b1 YARN-4884. Fix missing documentation about rmadmin command regarding node labels. Contributed by Kai Sasaki. 
(cherry picked from commit f1b8f6b2c16403869f78a54268ae1165982a7050)
 2016-11-08 10:22:59 +05:30
Zhe Zhang
384b7b71a3 HADOOP-13804. MutableStat mean loses accuracy if add(long, long) is used. Contributed by Erik Krogen. 
(cherry picked from commit 3dbad5d823b8bf61b643dd1057165044138b99e0)
(cherry picked from commit b245e9ce2f20bb84690bffe902a60d5e96130cdb)
 2016-11-07 16:15:20 -08:00
Steve Loughran
5231c527aa HADOOP-13797 Remove hardcoded absolute path for ls. Contributed by Christine Koppelt 2016-11-07 12:36:38 +00:00
Akira Ajisaka
21f210bada HADOOP-13798. TestHadoopArchives times out. 
(cherry picked from commit b970446b2c59f8897bb2c3a562fa192ed3452db5)
(cherry picked from commit 3b279fe4260d99f95787ea5cce428f9ef337a3cb)
 2016-11-07 19:55:10 +09:00
Sunil
ee5dd2c388 YARN-5802. updateApplicationPriority api in scheduler should ensure to re-insert app to correct ordering policy. Contributed by Bibin A Chundatt 2016-11-05 13:06:31 +05:30
Xiaoyu Yao
5073289c4a Revert "HADOOP-13565. KerberosAuthenticationHandler#authenticate should not rebuild SPN based on client request. Contributed by Xiaoyu Yao." 
This reverts commit 5abc78882590bc3c0dfb71cba2f08fd671bc1e33.
 2016-11-04 16:22:45 -07:00
Jason Lowe
4cb4e4c0a1 YARN-5837. NPE when getting node status of a decommissioned node after an RM restart. Contributed by Robert Kanter 
(cherry picked from commit 6bb741ff0ef208a8628bc64d6537999d4cd67955)
 2016-11-04 22:25:35 +00:00
Mingliang Liu
00625004b7 HDFS-11085. Add unit test for NameNode failing to start when name dir is unwritable. Contributed by Xiaobing Zhou 
(cherry picked from commit 0c0ab102ab392ba07ed2aa8d8a67eef4c20cad9b)
 2016-11-04 14:45:43 -07:00
Robert Kanter
f9a7b83a53 HADOOP-7930. Kerberos relogin interval in UserGroupInformation should be configurable (xiaochen via rkanter) 
(cherry picked from commit 0b36dcda7cbac51c725224a8e0abef4eaa004fd8)
 2016-11-04 09:24:34 -07:00