Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,120 Commits 326 Branches 371 Tags
Commit Graph

12120 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Subru Krishnan
2f4df2b851 YARN-5203.Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API. Contributed by Ellen Hui. 
(cherry picked from commit 4e756d72719ec3c6d64a1e3daccbc0b8e8de998c)
 2016-07-28 16:04:30 -07:00
Jason Lowe
2f854b8386 YARN-5438. TimelineClientImpl leaking FileSystem Instances causing Long running services like HiverServer2 daemon going OOM. Contributed by Rohith Sharma K S 
(cherry picked from commit a1890c32c52fed69ec09efad0fccf49ed8c2e21e)
 2016-07-28 21:48:40 +00:00
Li Lu
310245063b YARN-5440. Use AHSClient in YarnClient when TimelineServer is running. (Xuan Gong via gtcarrera9) 
(cherry picked from commit 26de4f0de789f58736d1dc383125cffb54debdd0)
 2016-07-28 10:25:55 -07:00
Junping Du
932bf79244 YARN-5432. Lock already held by another process while LevelDB cache store creation for dag. Contributed by Li Lu. 
(cherry picked from commit 7f3c306e2ed9e865171324898decb9b587d90e73)
 2016-07-28 06:36:31 -07:00
Akira Ajisaka
ab49c2b399 HDFS-10696. TestHDFSCLI fails. Contributed by Kai Sasaki. 
(cherry picked from commit 414fbfab41470923eb82a21628709e51fd3a3f6e)
 2016-07-28 16:59:25 +09:00
Subru Krishnan
4ecfd05095 YARN-5441. Fixing minor Scheduler test case failures 
(cherry picked from commit d2cbfd7de33fde526089a395550deafb4628fc6f)
 2016-07-27 18:26:29 -07:00
Chris Nauroth
59eec8248a HADOOP-13354. Update WASB driver to use the latest version (4.2.0) of SDK for Microsoft Azure Storage Clients. Contributed by Sivaguru Sankaridurg. 
(cherry picked from commit b43de80031d1272e8a08ea5bd31027efe45e9d70)

Conflicts:
	hadoop-project/pom.xml
 2016-07-27 15:52:07 -07:00
Yongjun Zhang
bd84cf3036 HDFS-10667. Report more accurate info about data corruption location. (Yuanbo Liu via Yongjun Zhang) 
(cherry picked from commit eb7ff0c9927131f4a797148b970a95a1abf7d847)
 2016-07-27 15:23:38 -07:00
Brahma Reddy Battula
096d3fa25c HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu. 2016-07-27 10:28:31 +05:30
Wangda Tan
77e0b6d1bf YARN-5195. RM intermittently crashed with NPE while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler. (sandflee via wangda) 
(cherry picked from commit d62e121ffc0239e7feccc1e23ece92c5fac685f6)
 2016-07-26 21:36:37 -07:00
Wangda Tan
55fa19d576 YARN-5342. Improve non-exclusive node partition resource allocation in Capacity Scheduler. (Sunil G via wangda) 
(cherry picked from commit 49969b16cdba0f251b9f8bf3d8df9906e38b5c61)
 2016-07-26 21:16:12 -07:00
Arun Suresh
0f6ff20652 YARN-5351. ResourceRequest should take ExecutionType into account during comparison. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 2d8d183b1992b82c4d8dd3d6b41a1964685d909e)
 2016-07-26 19:09:22 -07:00
Chris Nauroth
df2ed6b2c4 HADOOP-13422. ZKDelegationTokenSecretManager JaasConfig does not work well with other ZK users in process. Contributed by Sergey Shelukhin. 
(cherry picked from commit 255ea45e50e102505ee61eb0ba45ea93035abe3c)
 2016-07-26 15:33:30 -07:00
Arun Suresh
df509a8066 YARN-5392. Replace use of Priority in the Scheduling infrastructure with an opaque ShedulerRequestKey. (asuresh and subru) 
(cherry picked from commit 5aace38b748ba71aaadd2c4d64eba8dc1f816828)
 2016-07-26 14:58:54 -07:00
Arpit Agarwal
c08c7b7a94 HDFS-10642. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. (Contributed by Mingliang Liu) 2016-07-26 12:44:36 -07:00
Wei-Chiu Chuang
0ee1644e39 HDFS-9937. Update dfsadmin command line help and HdfsQuotaAdminGuide. Contributed by Kai Sasaki. 
(cherry picked from commit 25f0a9b3ab136d44f70eb2805598e2df8325612a)
 2016-07-26 08:05:14 -07:00
Steve Loughran
e5a4b6cb94 Revert "HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu." 
This reverts commit b3ce778268a441b742b625594d1a55b83960f355.
 2016-07-26 14:30:31 +01:00
Masatake Iwasaki
f4207d0336 HDFS-10671. Fix typo in HdfsRollingUpgrade.md. Contributed by Yiqun Lin. 
(cherry picked from commit 59466b8c180716dda7aa670728580a88e54eb4d2)
 2016-07-26 19:50:39 +09:00
Brahma Reddy Battula
b3ce778268 HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu. 
(cherry picked from commit 7cac7655fd84ac394250705b31e3927fe548e34a)
 2016-07-26 11:02:23 +05:30
Akira Ajisaka
a04b193374 Revert "HDFS-10620. StringBuilder created and appended even if logging is disabled. Contributed by Staffan Friberg." 
This reverts commit 2685e16f381532d53aea413ce1c0a14181ee7c89.
 2016-07-26 11:04:05 +09:00
Vinitha Reddy Gankidi
0dde4035c9 HDFS-10301. Interleaving processing of storages from repeated block reports causes false zombie storage detection, removes valid blocks. Contributed by Vinitha Gankidi. 2016-07-25 18:55:00 -07:00
Jing Zhao
30c772fc59 HDFS-10688. BPServiceActor may run into a tight loop for sending block report when hitting IOException. Contributed by Chen Liang. 
(cherry picked from commit 0cde9e12a7175e4d8bc4ccd5c36055b280d1fbd6)
 2016-07-25 18:43:25 -07:00
Jason Lowe
26df6bd31a MAPREDUCE-6744. Increase timeout on TestDFSIO tests. Contributed by Eric Badger 
(cherry picked from commit 703fdf86c6e42d6d555090f670ca5cbd1ef48bf9)
 2016-07-25 21:55:58 +00:00
Steve Loughran
b757eff30c HADOOP-13188 S3A file-create should throw error rather than overwrite directories. Contributed by Steve Loughran 2016-07-25 15:49:54 +01:00
Steve Loughran
271e7bb00b HADOOP-13406 S3AFileSystem: Consider reusing filestatus in delete() and mkdirs(). Contributed by Rajesh Balamohan 2016-07-25 14:44:03 +01:00
Steve Loughran
bdde7edba5 HADOOP-13389 TestS3ATemporaryCredentials.testSTS error when using IAM credentials. Contributed by Steven K. Wong 2016-07-23 17:59:50 +01:00
Chris Nauroth
e6ba69b4db HADOOP-13207. Specify FileSystem listStatus, listFiles and RemoteIterator. Contributed by Steve Loughran. 
(cherry picked from commit 5fda66e4a38c7597bc0bc9c94748c8491b7f3750)
 2016-07-22 15:38:34 -07:00
Xiao Chen
1a115ea317 HDFS-10225. DataNode hot swap drives should disallow storage type changes. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 132deb4cacc413a85a6af2f390ec79a76c91961c)
 2016-07-22 15:04:17 -07:00
Arpit Agarwal
2f1e352912 HDFS-10660. Expose storage policy apis via HDFSAdmin interface. (Contributed by Rakesh R) 2016-07-22 12:12:29 -07:00
mattf-horton
ea10e1384f HADOOP-13382. Remove unneeded commons-httpclient dependencies from POM files in Hadoop and sub-projects, while adding it in to the only place it is still needed, hadoop-openstack. Contributed by Matt Foley. 2016-07-21 15:58:05 -07:00
Chris Nauroth
382bcf21e7 HADOOP-13240. TestAclCommands.testSetfaclValidations fail. Contributed by John Zhuge. 
(cherry picked from commit 43cf6b101dacd96bacfd199826b717f6946109af)
 2016-07-21 14:12:39 -07:00
Jakob Homan
43a40fa009 HADOOP-12009. Clarify FileSystem.listStatus() sorting order & fix FileSystemContractBaseTest:testListStatus. backport to 2.8 2016-07-21 21:50:38 +01:00
Akira Ajisaka
f4d94f2246 MAPREDUCE-6738. TestJobListCache.testAddExisting failed intermittently in slow VM testbed. Contributed by Junping Du. 
(cherry picked from commit ecff7d03595b45d7a5d6aed73b9931fef85a56ee)
 2016-07-21 11:38:02 -07:00
Akira Ajisaka
1222889f1a HDFS-10287. MiniDFSCluster should implement AutoCloseable. Contributed by Andras Bokor. 
(cherry picked from commit fcde6940e0cbdedb1105007e4857137ecdfa1284)
 2016-07-21 11:28:19 -07:00
Jing Zhao
419d6ce19c HDFS-10653. Optimize conversion from path string to components. Contributed by Daryn Sharp. 
(cherry picked from commit bd3dcf46e263b6e6aa3fca6a5d9936cc49e3280f)
 2016-07-21 11:15:30 -07:00
Rohith Sharma K S
eff65463de YARN-5092. TestRMDelegationTokens fails intermittently. Contributed by Jason Lowe. 
(cherry picked from commit 557a245d83c358211f51b3ab1950d707a70679d9)
 2016-07-21 12:49:06 +05:30
Rohith Sharma K S
d9ed29e8c6 YARN-1126. Add validation of users input nodes-states options to nodes CLI. Contributed by Wei Yan. 
(cherry picked from commit be34b2a8fd30a1a0e803ac8b2198c695600a9e46)
 2016-07-21 12:22:34 +05:30
Akira Ajisaka
2bf5c4eb1a YARN-4883. Make consistent operation name in AdminService. Contributed by Kai Sasaki. 
(cherry picked from commit c63afdbe14df7359841aa4b213252a8a275e8c4b)
 2016-07-20 16:51:44 -07:00
Sangjin Lee
a7e533444c MAPREDUCE-6365. Refactor JobResourceUploader#uploadFilesInternal (Chris Trezzo via sjlee) 
(cherry picked from commit 8f0d3d69d65a252439610e6f13d679808d768569)
 2016-07-20 11:25:25 -07:00
Vinod Kumar Vavilapalli
5641c28c1c YARN-5340. Fixed a race condition in RollingLevelDBTimelineStore that caused loss of Timeline events. Contributed by Li Lu. 
(cherry picked from commit 1c9d2ab503ea5a3f16757351af9603041059b390)
 2016-07-20 08:38:56 -07:00
Steve Loughran
d3df810dce HADOOP-13387 users always get told off for using S3 —even when not using it. Contributed by stevel. 2016-07-20 14:01:30 +01:00
Steve Loughran
5490ed5f87 HADOOP-13212 Provide an option to set the socket buffers in S3AFileSystem (Rajesh Balamohan) 2016-07-20 13:42:33 +01:00
Varun Vasudev
9e326f4eeb YARN-5309. Fix SSLFactory truststore reloader thread leak in TimelineClientImpl. Contributed by Weiwei Yang. 
(cherry picked from commit 9ccf935b2cba5ac8a3e8124360d2203931397391)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/impl/TimelineClientImpl.java
 2016-07-20 12:55:38 +05:30
Arun Suresh
586cd0daa1 YARN-5350. Distributed Scheduling: Ensure sort order of allocatable nodes returned by the RM is not lost. (asuresh) 
(cherry picked from commit 8fbe6ece24e38ee24fee0abdbed5f7dc5d3c16da)
 2016-07-19 23:22:47 -07:00
Akira Ajisaka
8fe2c450b3 HADOOP-12991. Conflicting default ports in DelegateToFileSystem. Contributed by Kai Sasaki. 
(cherry picked from commit dc065dd64ca3e101b0c0a7bcc7d7a067b77d6c82)
 2016-07-19 16:06:16 -07:00
Akira Ajisaka
2685e16f38 HDFS-10620. StringBuilder created and appended even if logging is disabled. Contributed by Staffan Friberg. 
(cherry picked from commit f7dabe3addf3f6eb32ea9b8ec1354fb442ce4222)
 2016-07-19 15:17:31 -07:00
Allen Wittenauer
84c785036f HDFS-9852. hdfs dfs -setfacl error message is misleading (Wei-Chiu Chuang via aw) 
(cherry picked from commit b3649adf6a1b2dc47566b4b0d652bd4e0a6a8056)
 2016-07-19 14:40:51 -07:00
Akira Ajisaka
c92601e981 HDFS-10603. Fix flaky tests in org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot. Contributed by Yiqun Lin. 
(cherry picked from commit fc570b55b9d1d1d57329eca223f443e2c86f34b7)
 2016-07-19 13:52:06 -07:00
Junping Du
ad8180fc6c YARN-5213. Fix a bug in LogCLIHelpers which cause TestLogsCLI#testFetchApplictionLogs fails intermittently. Contributed by Xuan Gong. 
(cherry picked from commit dc2f4b6ac8a6f8848457046cf9e1362d8f48495d)
 2016-07-19 07:19:17 -07:00
Varun Saxena
2153c9f04c YARN-4996. Make TestNMReconnect.testCompareRMNodeAfterReconnect() scheduler agnostic (Kai Sasaki via Varun Saxena) 2016-07-19 16:07:56 +05:30