Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,329 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2235 Commits

Author SHA1 Message Date
Robert Kanter 568b5eecd8 MAPREDUCE-6765. MR should not schedule container requests in cases where reducer or mapper containers demand resource larger than the maximum supported (haibochen via rkanter) 
(cherry picked from commit fc2b69eba1)
 2016-11-01 20:47:58 -07:00
Akira Ajisaka a3897f77ae MAPREDUCE-6795. Update the document for JobConf#setNumReduceTasks. Contributed by Yiqun Lin. 
(cherry picked from commit 235f39b7c4)
 2016-11-01 20:52:35 +09:00
Akira Ajisaka ef13a5ef9d MAPREDUCE-6799. Document mapreduce.jobhistory.webapp.https.address in mapred-default.xml. Contributed by Yiqun Lin. 
(cherry picked from commit f646fe34be)
 2016-10-31 15:18:34 +09:00
Naganarasimha 1455f56b63 MAPREDUCE-2631. Potential resource leaks in BinaryProtocol$TeeOutputStream.java. Contributed by Sunil G. 
(cherry picked from commit 28660f51af)
 2016-10-28 06:00:29 +05:30
Varun Saxena f572448809 YARN-4388. Cleanup mapreduce.job.hdfs-servers from yarn-default.xml (Junping Du via Varun Saxena) 2016-10-28 02:24:24 +05:30
Naganarasimha fe5c1f249a MAPREDUCE-6541. Exclude scheduled reducer memory when calculating available mapper slots from headroom to avoid deadlock. Contributed by Varun Saxena 
(cherry picked from commit 060558c6f2)
 2016-10-27 18:05:00 +05:30
Varun Saxena 65fa9827fc YARN-5757. RM REST API documentation is not up to date (Miklos Szegedi via Varun Saxena) 2016-10-27 17:24:24 +05:30
Varun Saxena 14ce42a8ed MAPREDUCE-6798. Fix intermittent failure of TestJobHistoryParsing.testJobHistoryMethods (Haibo Chen via Varun Saxena) 2016-10-27 15:41:27 +05:30
Akira Ajisaka e29d0a2fd8 YARN-5575. Many classes use bare yarn. properties instead of the defined constants. Contributed by Daniel Templeton. 
(cherry picked from commit d3bb69a667)
 2016-10-26 15:34:40 +09:00
Junping Du 432657780d MAPREDUCE-6792. Allow user's full principal name as owner of MapReduce staging directory in JobSubmissionFiles#JobStagingDir(). Contributed by Santhosh G Nayak. 
(cherry picked from commit 9b7ce948a5)
 2016-10-25 11:21:33 -07:00
Robert Kanter c76eb90f0e MAPREDUCE-6728. Give fetchers hint when ShuffleHandler rejects a shuffling connection (haibochen via rkanter) 2016-10-24 17:47:09 -07:00
Robert Kanter 48ca6be76e MAPREDUCE-6776. yarn.app.mapreduce.client.job.max-retries should have a more useful default (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit f3f37e6fb8)
 2016-10-07 14:47:23 -07:00
Akira Ajisaka 5f1432d98e MAPREDUCE-6789. Fix TestAMWebApp failure. Contributed by Daniel Templeton. 
(cherry picked from commit 272a21747e)
 2016-10-06 15:59:19 +09:00
Karthik Kambatla 876fc2e61a MAPREDUCE-6638. Do not attempt to recover progress from previous job attempts if spill encryption is enabled. (Haibo Chen via kasha) 
(cherry picked from commit de7a0a92ca)
 2016-10-03 10:30:44 -07:00
Jason Lowe c52ad9ee86 MAPREDUCE-6771. RMContainerAllocator sends container diagnostics event after corresponding completion event. Contributed by Haibo Chen 2016-09-29 15:50:50 +00:00
Robert Kanter 85724268c2 MAPREDUCE-6718. add progress log to JHS during startup (haibochen via rkanter) 
(cherry picked from commit 0d6778d800)
 2016-09-28 16:04:48 -07:00
Wangda Tan 269401dc83 HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs. (vinodkv via wangda) 
(cherry picked from commit 875062b5bc)
 2016-09-27 11:27:25 -07:00
Arun Suresh 7484d0b1b9 YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh 2016-09-26 23:55:13 +08:00
Naganarasimha 0e4a5611ff YARN-3692. Allow REST API to set a user generated message when killing an application. Contributed by Rohith Sharma K S 2016-09-24 21:13:01 +05:30
Robert Kanter 254041b71a MAPREDUCE-6632. Master.getMasterAddress() should be updated to use YARN-4629 (templedf via rkanter) 
(cherry picked from commit 4fc632ae19)
 2016-09-22 16:13:24 -07:00
Karthik Kambatla a0b0767856 MAPREDUCE-6740. Enforce mapreduce.task.timeout to be at least mapreduce.task.progress-report.interval. (Haibo Chen via kasha) 
(cherry picked from commit 537095d13c)
 2016-09-21 18:30:42 -07:00
Naganarasimha fa80972efb MAPREDUCE-6777. Typos in 4 log messages. Contributed by Mehran Hassani 2016-09-17 10:21:13 +05:30
Chris Douglas 806a8dea38 MAPREDUCE-6628. Potential memory leak in CryptoOutputStream. Contributed by Mariappan Asokan 
(cherry picked from commit 9f192cc5ac)
 2016-09-09 11:14:30 -07:00
Arun Suresh b279f42d79 YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 
(cherry picked from commit d6d9cff21b)
 2016-08-30 16:33:31 -07:00
Jason Lowe f70f522e80 MAPREDUCE-4784. TestRecovery occasionally fails. Contributed by Haibo Chen 
(cherry picked from commit af508605a9)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
 2016-08-30 14:09:14 +00:00
Jason Lowe 44d0af2611 MAPREDUCE-6768. TestRecovery.testSpeculative failed with NPE. Contributed by Haibo Chen 2016-08-29 19:57:38 +00:00
Junping Du af1502fcab YARN-5557. Add localize API to the ContainerManagementProtocol. Contributed by Jian He. 
(cherry picked from commit 9ef632f3b0)
 2016-08-26 09:05:33 -07:00
Wei-Chiu Chuang 13cbf1677e MAPREDUCE-6764. Teragen LOG initialization bug. Contributed by Yufei Gu. 
(cherry picked from commit 1360bd2d54)
 2016-08-25 09:51:17 -07:00
Jason Lowe 5cdc4ba9f4 MAPREDUCE-6761. Regression when handling providers - invalid configuration ServiceConfiguration causes Cluster initialization failure. Contributed by Peter Vary 
(cherry picked from commit 6fa9bf4407)
 2016-08-24 14:42:06 +00:00
Naganarasimha 5fa241daa0 YARN-4491. yarn list command to support filtering by tags. Contributed by Varun Saxena 
(cherry picked from commit 143c59e4c5)
 2016-08-24 01:55:24 +05:30
Akira Ajisaka 41fe85f868 MAPREDUCE-6587. Remove unused params in connection-related methods of Fetcher. Contributed by Yiqun Lin. 
(cherry picked from commit 8cc4a67059)
 2016-08-23 17:05:45 +09:00
Varun Saxena 4ba8dbf00e MAPREDUCE-6762. ControlledJob#toString failed with NPE when job status is not successfully updated (Weiwei Yang via Varun Saxena) 2016-08-21 21:50:44 +05:30
Wangda Tan 65822e55fd MAPREDUCE-6310. Add jdiff support to MapReduce. (Li Lu/vinodkv via wangda) 
(cherry picked from commit 3d937457ee)
 2016-08-19 16:41:47 -07:00
Varun Saxena 5128e3320e MAPREDUCE-6763. Shuffle server listen queue is too small (Jason Lowe via Varun Saxena) 2016-08-20 04:05:01 +05:30
Jason Lowe 848ae35a93 MAPREDUCE-6690. Limit the number of resources a single map reduce job can submit for localization. Contributed by Chris Trezzo 
(cherry picked from commit f80a729832)
 2016-08-17 16:24:16 +00:00
Ray Chiang d1c4ba2d7c MAPREDUCE-6751. Add debug log message when splitting is not possible due to unsplittable compression. (Peter Vary via rchiang) 
(cherry picked from commit 6c154abd33)
 2016-08-16 12:14:30 -07:00
Varun Saxena 705307eda3 MAPREDUCE-6750. Fix TestHSAdminServer#testRefreshSuperUserGroups (Kihwal Lee via Varun Saxena) 2016-08-09 23:43:27 +05:30
Jason Lowe f18febd933 MAPREDUCE-6741. Add MR support to redact job conf properties. Contributed by Haibo Chen 
(cherry picked from commit 4aba858750)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEventHandler.java
 2016-08-09 16:32:38 +00:00
Arun Suresh 5f7edb79d1 YARN-5457. Refactor DistributedScheduling framework to pull out common functionality. (asuresh) 
(cherry picked from commit 82c9e06101)
 2016-08-09 00:46:08 -07:00
Varun Saxena 8b09292f4f MAPREDUCE-6748. Enhance logging for Cluster.java around InetSocketAddress (Vrushali C via Varun Saxena) 2016-08-08 21:53:29 +05:30
Akira Ajisaka e54de94e5a MAPREDUCE-6730. Use StandardCharsets instead of String overload in TextOutputFormat. Contributed by Sahil Kang. 
This closes #114

(cherry picked from commit 70c2781152)
 2016-08-04 20:18:13 +09:00
Gera Shegalov 7eeba35495 MAPREDUCE-6724. Single shuffle to memory must not exceed Integer#MAX_VALUE. (Haibo Chen via gera) 
(cherry picked from commit 6890d5b472)
 2016-08-01 23:41:35 -07:00
Akira Ajisaka 96d5607e7a MAPREDUCE-6729. Accurately compute the test execute time in DFSIO. Contributed by mingleizhang. 
This closes #112

(cherry picked from commit ce93595d7a)
 2016-07-30 22:16:13 +09:00
Akira Ajisaka 0fc838540b MAPREDUCE-6746. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 
(cherry picked from commit 3f0bffea2d)
 2016-07-30 11:47:15 +09:00
Jason Lowe 26df6bd31a MAPREDUCE-6744. Increase timeout on TestDFSIO tests. Contributed by Eric Badger 
(cherry picked from commit 703fdf86c6)
 2016-07-25 21:55:58 +00:00
Akira Ajisaka f4d94f2246 MAPREDUCE-6738. TestJobListCache.testAddExisting failed intermittently in slow VM testbed. Contributed by Junping Du. 
(cherry picked from commit ecff7d0359)
 2016-07-21 11:38:02 -07:00
Sangjin Lee a7e533444c MAPREDUCE-6365. Refactor JobResourceUploader#uploadFilesInternal (Chris Trezzo via sjlee) 
(cherry picked from commit 8f0d3d69d6)
 2016-07-20 11:25:25 -07:00
Robert Kanter 233ffaf3f7 MAPREDUCE-6652. Add configuration property to prevent JHS from loading jobs with a task count greater than X (haibochen via rkanter) 2016-07-15 13:37:04 -07:00
Junping Du 51a3131a87 MAPREDUCE-6733. MapReduce JerseyTest tests failing with java.net.BindException: Address already in use. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit 1d053836b1)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServices.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesAttempt.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesAttempts.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobConf.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesTasks.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServices.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesAttempts.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobConf.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobs.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobsQuery.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesTasks.java
 2016-07-15 11:36:04 -07:00
Robert Kanter a2ef30cb2f MAPREDUCE-6625. TestCLI#testGetJob fails occasionally (haibochen via rkanter) 
(cherry picked from commit ad256beb50)
 2016-07-12 15:51:30 -07:00