Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,578 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

11578 Commits

Author SHA1 Message Date
Jian He db3cf5b354 MAPREDUCE-6696. Add a configuration to limit the number of map tasks allowed per job. Contributed by Zhihai Xu 
(cherry picked from commit 21d2b90213)
 2016-05-19 19:25:48 -07:00
Tsz-Wo Nicholas Sze 658072d62e HADOOP-13168. Support Future.get with timeout in ipc async calls. 2016-05-19 15:58:15 -07:00
Junping Du 2703ec6871 YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron. 2016-05-19 14:10:15 -07:00
Kihwal Lee bae151801d HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu. 
(cherry picked from commit 204de430bf)
 2016-05-19 15:53:41 -05:00
Jian He 2d846fdcbd YARN-4002. Make ResourceTrackerService#nodeHeartbeat more concurrent. Contributed by Rohith Sharma K S & Zhiguo Hong 
(cherry picked from commit feb90ffcca)
 2016-05-19 13:02:05 -07:00
Allen Wittenauer f7ffa1a534 HADOOP-13177. Native tests fail on OS X, because DYLD_LIBRARY_PATH is not defined to include libhadoop.dylib (Chris Nauroth via aw) 
(cherry picked from commit c7159f0240)
 2016-05-19 09:58:48 -07:00
Junping Du 416274b53e YARN-5100. The YarnApplicationState is always running in ATS even application is finished. Contributed by Xuan Gong. 
(cherry picked from commit 141873ca7d)
 2016-05-19 09:14:42 -07:00
Steve Loughran 151ecdf26d HADOOP-12767. Update apache httpclient version to 4.5.2; httpcore to 4.4.4. Artem Aliev via stevel. 2016-05-19 14:44:44 +01:00
Arun Suresh 15808717f3 YARN-5110. Fix OpportunisticContainerAllocator to insert complete HostAddress in issued ContainerTokenIds. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 1597630681)
 2016-05-18 23:20:54 -07:00
Arun Suresh a555a320e8 YARN-5090. Add End-to-End test-cases for DistributedScheduling using MiniYarnCluster. (asuresh) 
(cherry picked from commit 8a9ecb7584)
 2016-05-18 23:20:36 -07:00
Arun Suresh f31602893e YARN-5075. Fix findbugs warnings in hadoop-yarn-common module. (asuresh) 
(cherry picked from commit ccc93e7812)
 2016-05-18 23:20:08 -07:00
Arun Suresh bbcea3abf3 YARN-4738. Notify the RM about the status of OPPORTUNISTIC containers (Konstantinos Karanasos via asuresh) 
(cherry picked from commit f45bc5a83e)
 2016-05-18 23:12:19 -07:00
Arun Suresh 69a9ce3494 YARN-2888. Corrective mechanisms for rebalancing NM container queues. (asuresh) 
(cherry picked from commit f0ac18d001)
 2016-05-18 23:09:11 -07:00
Arun Suresh 005dae521c YARN-5073. Refactor startContainerInternal() in ContainerManager to remove unused parameter. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit d971bf252f)
 2016-05-18 22:36:28 -07:00
Arun Suresh 1b4600abcf YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh) 
(cherry picked from commit 341888a0aa)
 2016-05-18 22:35:55 -07:00
Arun Suresh 307cda70db YARN-5049. Extend NMStateStore to save queued container information. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit d464f4d1c4)
 2016-05-18 22:09:30 -07:00
Arun Suresh d80d24aabf YARN-2885. Create AMRMProxy request interceptor and ContainerAllocator to distribute OPPORTUNISTIC containers to appropriate Nodes (asuresh) 
(cherry picked from commit 2bf025278a318b0452fdc9ece4427b4c42124e39)
(cherry picked from commit c282a08f38)
 2016-05-18 22:08:55 -07:00
Karthik Kambatla b56fc51b70 YARN-2883. Queuing of container requests in the NM. (Konstantinos Karanasos and Arun Suresh via kasha) 
(cherry picked from commit c8172f5f14)
 2016-05-18 22:08:20 -07:00
Arun Suresh 68964bbaa2 YARN-4335. Allow ResourceRequests to specify ExecutionType of a request ask (kkaranasos via asuresh) 
(cherry picked from commit 8ffabfdf4f)
(cherry picked from commit b2a654c5ee)
 2016-05-18 22:02:43 -07:00
Karthik Kambatla 2c6c701aff YARN-2882. Add an OPPORTUNISTIC ExecutionType. (Konstantinos Karanasos and Inigo Goiri via kasha) 
(cherry picked from commit fb00794368)
 2016-05-18 22:02:28 -07:00
Andrew Wang 686691df60 HADOOP-13157. Follow-on improvements to hadoop credential commands. Contributed by Mike Yoder. 2016-05-18 14:55:31 -07:00
Andrew Wang a44c87c2d0 HDFS-2173. saveNamespace should not throw IOE when only one storage directory fails to write VERSION file. Contributed by Andras Bokor. 
(cherry picked from commit 010e6ac328)
 2016-05-18 14:13:47 -07:00
Robert Kanter 27aabcab45 MAPREDUCE-6686. Add a way to download the job config from the mapred CLI (rkanter) 
(cherry picked from commit 992a49353f)
 2016-05-18 12:16:45 -07:00
Xiaoyu Yao 1470af8c95 HDFS-10383. Safely close resources in DFSTestUtil. Contributed by Mingliang Liu. 
(cherry picked from commit ccb1cade5b)
 2016-05-18 11:02:31 -07:00
Naganarasimha a166286b9e YARN-4925. ContainerRequest in AMRMClient, application should be able to specify nodes/racks together with nodeLabelExpression. Contributed by Bibin A Chundatt 
(cherry picked from commit f04c81c9ce)
 2016-05-19 01:35:47 +08:00
Varun Vasudev 3c9392a31d YARN-4913. Yarn logs should take a -out option to write to a directory. Contributed by Xuan Gong. 
(cherry picked from commit ef1757790d)
 2016-05-18 22:45:06 +05:30
Chris Nauroth 42f64b3bcc HADOOP-13138. Unable to append to a SequenceFile with Compression.NONE. Contributed by Vinayakumar B. 
(cherry picked from commit 0a527196dcb35a699e12121165bc3738766b569e)
 2016-05-18 09:32:15 -07:00
Arpit Agarwal 24bed7768d HDFS-9926. MiniDFSCluster leaks dependency Mockito via DataNodeTestUtils. (Contributed by Josh Elser) 2016-05-18 09:21:41 -07:00
Colin Patrick Mccabe feabfe9cda HADOOP-13140. FileSystem#initialize must not attempt to create StorageStatistics objects with null or empty schemes (Mingliang Liu via cmccabe) 
(cherry picked from commit f2c1d9181f)
 2016-05-18 09:19:45 -07:00
Akira Ajisaka e9dd0d84e5 Revert "HADOOP-13157. Follow-on improvements to hadoop credential commands. Contributed by Mike Yoder." 
This reverts commit e205421555.
 2016-05-18 17:57:46 +09:00
Andrew Wang e205421555 HADOOP-13157. Follow-on improvements to hadoop credential commands. Contributed by Mike Yoder. 
(cherry picked from commit 7154ace712)
 2016-05-17 17:44:34 -07:00
Lei Xu 2d1d2dde17 HDFS-10360. DataNode may format directory and lose blocks if current/VERSION is missing. (Wei-Chiu Chuang via lei) 
(cherry picked from commit bae11b95d492e0be7c5576097424879c2539b474)
 2016-05-17 16:37:36 -07:00
Yongjun Zhang 456fe08a6d HDFS-10381, DataStreamer DataNode exclusion log message should be warning. (John Zhuge via Yongjun Zhang) 
(cherry picked from commit 16c07cc68a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/StripedDataStreamer.java
 2016-05-17 15:59:00 -07:00
Jing Zhao dd7f5a3b88 HDFS-10397. Distcp should ignore -delete option if -diff option is provided instead of exiting. Contributed by Mingliang Liu. 
(cherry picked from commit 03788d3015)
 2016-05-17 15:53:39 -07:00
Xiaoyu Yao 937bc924cb HADOOP-13159. Fix potential NPE in Metrics2 source for DecayRpcScheduler. Contributed by Xiaoyu Yao. 
(cherry picked from commit 9478484845)
(cherry picked from commit 744d5edc87)
 2016-05-17 15:51:47 -07:00
Jian He 264c06a438 YARN-4832. NM side resource value should get updated if change applied in RM side. Contributed by Junping Du 2016-05-17 15:02:14 -07:00
Junping Du 7e36b9159e MAPREDUCE-6657. Job history server can fail on startup when NameNode is in start phase. Contributed by Haibo Chen. 
(cherry picked from commit f6ef876fe1)
 2016-05-17 14:42:11 -07:00
Kihwal Lee 1e7eb2d2ac Revert "HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu." 
This reverts commit 70ee477e5f.
 2016-05-17 15:20:59 -05:00
Kihwal Lee 70ee477e5f HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu. 
(cherry picked from commit 0942954e8a)
 2016-05-17 15:10:47 -05:00
Colin Patrick Mccabe 6ebb92c9c1 HDFS-10404. Fix formatting of CacheAdmin command usage help text (Yiqun Lin via cmccabe) 
(cherry picked from commit 7cd5ae62f6)
 2016-05-17 11:59:10 -07:00
Jason Lowe 9c47862d87 MAPREDUCE-6698. Increase timeout on TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser. Contributed by Haibo Chen 
(cherry picked from commit 34fddd1e91)
 2016-05-17 15:04:02 +00:00
Jason Lowe 51e06c085f MAPREDUCE-6701. application master log can not be available when clicking jobhistory's am logs link. Contributed by Haibo Chen 
(cherry picked from commit 12fa4ec141)
 2016-05-17 14:56:23 +00:00
Kihwal Lee 48be0ff67a HDFS-10303. DataStreamer#ResponseProcessor calculates packet ack latency incorrectly. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 4a5819dae2)
 2016-05-17 08:58:53 -05:00
Steve Loughran dd0a8201bc HADOOP-13158 S3AFileSystem#toString might throw NullPointerException due to null cannedACL. (Chris Nauroth via stevel) 2016-05-17 13:19:08 +01:00
Steve Loughran dfc062a092 HADOOP-13163 Reuse pre-computed filestatus in Distcp-CopyMapper (Rajesh Balamohan via stevel) 2016-05-17 13:01:50 +01:00
Akira Ajisaka 19db965c27 MAPREDUCE-6693. ArrayIndexOutOfBoundsException occurs when the length of the job name is equal to mapreduce.jobhistory.jobname.limit. Contributed by Ajith S. 
(cherry picked from commit b357930526)
 2016-05-17 20:22:46 +09:00
Akira Ajisaka 7f03c1c90e HDFS-10242. Cannot create space quota of zero. Contributed by Takashi Ohnishi. 
(cherry picked from commit 9fe5828f05)
 2016-05-17 15:56:21 +09:00
Sangjin Lee 9330a7b4de HDFS-10208. Addendum for HDFS-9579: to handle the case when client machine can't resolve network path (Ming Ma via sjlee) 
(cherry picked from commit 61f46be071)
 2016-05-16 18:59:19 -07:00
Lei Xu 09a613b023 HDFS-10410. RedundantEditLogInputStream.LOG is set to wrong class. (John Zhuge via lei) 
(cherry picked from commit 6a6e74acf5)
 2016-05-16 17:07:03 -07:00
Jing Zhao af70fc42e7 HADOOP-13146. Refactor RetryInvocationHandler. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit a9a8297cad)
 2016-05-16 15:26:48 -07:00