Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,239 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2357 Commits

Author SHA1 Message Date
Daniel Templeton d1aa844dc6 MAPREDUCE-6715. Fix Several Unsafe Practices (Contributed by Yufei Gu via Daniel Templeton) 
(cherry picked from commit 0b8a7c18dd)
 2017-01-05 17:57:14 -08:00
Akira Ajisaka 3e2d26a97f HADOOP-12733. Remove references to obsolete io.seqfile configuration variables. Contributed by Ray Chiang. 
(cherry picked from commit 01d31fe938)
 2017-01-04 14:15:00 +09:00
Brahma Reddy Battula d6e5e4288c HADOOP-13883. Addendum patch to correct the message and alphabetized with the earlier patch. Contributed by Yiqun Lin. 
(cherry picked from commit 6938b67711)
 2016-12-31 16:17:15 +05:30
Daniel Templeton 28b605dd1a HADOOP-13940. Document the missing envvars commands (Contributed by Yiqun Lin via Daniel Templeton) 
(cherry picked from commit d432173737a532b7106682116013b751ba162ff0)
 2016-12-27 07:19:37 -08:00
Akira Ajisaka 522bc98026 YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor. 
(cherry picked from commit ae401539ea)
 2016-12-23 04:06:53 +09:00
Ravi Prakash d7d9006afc HADOOP-13898. Should set HADOOP_JOB_HISTORYSERVER_HEAPSIZE only if it's empty. Contributed by Fei Hui. 2016-12-13 13:08:54 -08:00
Akira Ajisaka 1e1ab9bc1b MAPREDUCE-6820. Fix dead links in Job relevant classes. Contributed by Yiqun Lin. 
(cherry picked from commit 92a8917ca5)
 2016-12-10 10:57:06 +09:00
Karthik Kambatla 0aaaeea268 MAPREDUCE-6817. The format of job start time in JHS is different from those of submit and finish time. (Haibo Chen via kasha) 
(cherry picked from commit a793cec2c9)
 2016-12-07 13:37:12 -08:00
Akira Ajisaka 03854f0bc5 MAPREDUCE-6816. Progress bars in Web UI always at 100%. Contributed by Shen Yinjie. 
(cherry picked from commit da4ecc9f82)
 2016-12-07 06:44:03 +09:00
Robert Kanter 1585b4f7ce MAPREDUCE-6571. JobEndNotification info logs are missing in AM container syslog (haibochen via rkanter) 
(cherry picked from commit 6b4a9970f1)
 2016-12-06 13:11:20 -08:00
Jason Lowe e8cd9dc200 MAPREDUCE-6815. Fix flaky TestKill.testKillTask(). Contributed by Haibo Chen 
(cherry picked from commit 0cfd7ad21f)
 2016-12-02 17:23:10 +00:00
Robert Kanter 99b046f8a9 MAPREDUCE-6787. Allow job_conf.xml to be downloadable on the job overview page in JHS (haibochen via rkanter) 
(cherry picked from commit c87b3a448a)
 2016-12-01 17:30:02 -08:00
Junping Du 376a2439db MAPREDUCE-6565. Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution. Contributed by Li Lu. 
(cherry picked from commit 8f6e14399a)
 2016-11-29 16:50:17 -08:00
Rohith Sharma K S 4b289d567e MAPREDUCE-6793. io.sort.factor code default and mapred-default.xml values inconsistent. Contributed by Gera Shegalov. 
(cherry picked from commit 683e0c71fe)
 2016-11-21 22:11:59 +05:30
Varun Saxena ca3b40306b MAPREDUCE-6801. Fix flaky TestKill.testKillJob (Haibo Chen via Varun Saxena) 
(cherry picked from commit 7584fbf4cb)
 2016-11-19 01:39:47 +05:30
Varun Saxena bcfa6d5384 MAPREDUCE-6811. TestPipeApplication#testSubmitter fails after HADOOP-13802 (Brahma Reddy Battula via Varun Saxena) 
(cherry picked from commit b8690a9d25)
 2016-11-17 00:55:12 +05:30
Jason Lowe 41f8275c50 MAPREDUCE-6804. Add timeout when starting JobHistoryServer in MiniMRYarnCluster. Contributed by Andras Bokor 2016-11-16 15:26:19 +00:00
Mingliang Liu 4e0fcff8ab HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu 
(cherry picked from commit 5af572b644)
 2016-11-15 11:36:08 -08:00
Jason Lowe 5ffb26c2a9 MAPREDUCE-6797. Job history server scans can become blocked on a single, slow entry. Contributed by Prabhu Joseph 
(cherry picked from commit 99c2bbd337)
 2016-11-14 20:30:36 +00:00
Jian He a422740bd7 YARN-5611. Provide an API to update lifetime of an application. Contributed by Rohith Sharma K S 2016-11-09 16:39:46 -08:00
Andrew Wang 4f25fd975e HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey. 
(cherry picked from commit e1c6ef2efa)
 2016-11-08 16:33:58 -08:00
Varun Saxena a942d968c9 MAPREDUCE-6782. JHS task page search based on each individual column not working (Ajith S via Varun Saxena) 
(cherry picked from commit 026b39ad9d)
 2016-11-08 15:02:46 +05:30
Robert Kanter 568b5eecd8 MAPREDUCE-6765. MR should not schedule container requests in cases where reducer or mapper containers demand resource larger than the maximum supported (haibochen via rkanter) 
(cherry picked from commit fc2b69eba1)
 2016-11-01 20:47:58 -07:00
Akira Ajisaka a3897f77ae MAPREDUCE-6795. Update the document for JobConf#setNumReduceTasks. Contributed by Yiqun Lin. 
(cherry picked from commit 235f39b7c4)
 2016-11-01 20:52:35 +09:00
Akira Ajisaka ef13a5ef9d MAPREDUCE-6799. Document mapreduce.jobhistory.webapp.https.address in mapred-default.xml. Contributed by Yiqun Lin. 
(cherry picked from commit f646fe34be)
 2016-10-31 15:18:34 +09:00
Naganarasimha 1455f56b63 MAPREDUCE-2631. Potential resource leaks in BinaryProtocol$TeeOutputStream.java. Contributed by Sunil G. 
(cherry picked from commit 28660f51af)
 2016-10-28 06:00:29 +05:30
Varun Saxena f572448809 YARN-4388. Cleanup mapreduce.job.hdfs-servers from yarn-default.xml (Junping Du via Varun Saxena) 2016-10-28 02:24:24 +05:30
Naganarasimha fe5c1f249a MAPREDUCE-6541. Exclude scheduled reducer memory when calculating available mapper slots from headroom to avoid deadlock. Contributed by Varun Saxena 
(cherry picked from commit 060558c6f2)
 2016-10-27 18:05:00 +05:30
Varun Saxena 65fa9827fc YARN-5757. RM REST API documentation is not up to date (Miklos Szegedi via Varun Saxena) 2016-10-27 17:24:24 +05:30
Varun Saxena 14ce42a8ed MAPREDUCE-6798. Fix intermittent failure of TestJobHistoryParsing.testJobHistoryMethods (Haibo Chen via Varun Saxena) 2016-10-27 15:41:27 +05:30
Akira Ajisaka e29d0a2fd8 YARN-5575. Many classes use bare yarn. properties instead of the defined constants. Contributed by Daniel Templeton. 
(cherry picked from commit d3bb69a667)
 2016-10-26 15:34:40 +09:00
Junping Du 432657780d MAPREDUCE-6792. Allow user's full principal name as owner of MapReduce staging directory in JobSubmissionFiles#JobStagingDir(). Contributed by Santhosh G Nayak. 
(cherry picked from commit 9b7ce948a5)
 2016-10-25 11:21:33 -07:00
Robert Kanter c76eb90f0e MAPREDUCE-6728. Give fetchers hint when ShuffleHandler rejects a shuffling connection (haibochen via rkanter) 2016-10-24 17:47:09 -07:00
Robert Kanter 48ca6be76e MAPREDUCE-6776. yarn.app.mapreduce.client.job.max-retries should have a more useful default (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit f3f37e6fb8)
 2016-10-07 14:47:23 -07:00
Akira Ajisaka 5f1432d98e MAPREDUCE-6789. Fix TestAMWebApp failure. Contributed by Daniel Templeton. 
(cherry picked from commit 272a21747e)
 2016-10-06 15:59:19 +09:00
Karthik Kambatla 876fc2e61a MAPREDUCE-6638. Do not attempt to recover progress from previous job attempts if spill encryption is enabled. (Haibo Chen via kasha) 
(cherry picked from commit de7a0a92ca)
 2016-10-03 10:30:44 -07:00
Jason Lowe c52ad9ee86 MAPREDUCE-6771. RMContainerAllocator sends container diagnostics event after corresponding completion event. Contributed by Haibo Chen 2016-09-29 15:50:50 +00:00
Robert Kanter 85724268c2 MAPREDUCE-6718. add progress log to JHS during startup (haibochen via rkanter) 
(cherry picked from commit 0d6778d800)
 2016-09-28 16:04:48 -07:00
Wangda Tan 269401dc83 HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs. (vinodkv via wangda) 
(cherry picked from commit 875062b5bc)
 2016-09-27 11:27:25 -07:00
Arun Suresh 7484d0b1b9 YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh 2016-09-26 23:55:13 +08:00
Naganarasimha 0e4a5611ff YARN-3692. Allow REST API to set a user generated message when killing an application. Contributed by Rohith Sharma K S 2016-09-24 21:13:01 +05:30
Robert Kanter 254041b71a MAPREDUCE-6632. Master.getMasterAddress() should be updated to use YARN-4629 (templedf via rkanter) 
(cherry picked from commit 4fc632ae19)
 2016-09-22 16:13:24 -07:00
Karthik Kambatla a0b0767856 MAPREDUCE-6740. Enforce mapreduce.task.timeout to be at least mapreduce.task.progress-report.interval. (Haibo Chen via kasha) 
(cherry picked from commit 537095d13c)
 2016-09-21 18:30:42 -07:00
Naganarasimha fa80972efb MAPREDUCE-6777. Typos in 4 log messages. Contributed by Mehran Hassani 2016-09-17 10:21:13 +05:30
Chris Douglas 806a8dea38 MAPREDUCE-6628. Potential memory leak in CryptoOutputStream. Contributed by Mariappan Asokan 
(cherry picked from commit 9f192cc5ac)
 2016-09-09 11:14:30 -07:00
Arun Suresh b279f42d79 YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 
(cherry picked from commit d6d9cff21b)
 2016-08-30 16:33:31 -07:00
Jason Lowe f70f522e80 MAPREDUCE-4784. TestRecovery occasionally fails. Contributed by Haibo Chen 
(cherry picked from commit af508605a9)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
 2016-08-30 14:09:14 +00:00
Jason Lowe 44d0af2611 MAPREDUCE-6768. TestRecovery.testSpeculative failed with NPE. Contributed by Haibo Chen 2016-08-29 19:57:38 +00:00
Junping Du af1502fcab YARN-5557. Add localize API to the ContainerManagementProtocol. Contributed by Jian He. 
(cherry picked from commit 9ef632f3b0)
 2016-08-26 09:05:33 -07:00
Wei-Chiu Chuang 13cbf1677e MAPREDUCE-6764. Teragen LOG initialization bug. Contributed by Yufei Gu. 
(cherry picked from commit 1360bd2d54)
 2016-08-25 09:51:17 -07:00
Jason Lowe 5cdc4ba9f4 MAPREDUCE-6761. Regression when handling providers - invalid configuration ServiceConfiguration causes Cluster initialization failure. Contributed by Peter Vary 
(cherry picked from commit 6fa9bf4407)
 2016-08-24 14:42:06 +00:00
Naganarasimha 5fa241daa0 YARN-4491. yarn list command to support filtering by tags. Contributed by Varun Saxena 
(cherry picked from commit 143c59e4c5)
 2016-08-24 01:55:24 +05:30
Akira Ajisaka 41fe85f868 MAPREDUCE-6587. Remove unused params in connection-related methods of Fetcher. Contributed by Yiqun Lin. 
(cherry picked from commit 8cc4a67059)
 2016-08-23 17:05:45 +09:00
Varun Saxena 4ba8dbf00e MAPREDUCE-6762. ControlledJob#toString failed with NPE when job status is not successfully updated (Weiwei Yang via Varun Saxena) 2016-08-21 21:50:44 +05:30
Wangda Tan 65822e55fd MAPREDUCE-6310. Add jdiff support to MapReduce. (Li Lu/vinodkv via wangda) 
(cherry picked from commit 3d937457ee)
 2016-08-19 16:41:47 -07:00
Varun Saxena 5128e3320e MAPREDUCE-6763. Shuffle server listen queue is too small (Jason Lowe via Varun Saxena) 2016-08-20 04:05:01 +05:30
Jason Lowe 848ae35a93 MAPREDUCE-6690. Limit the number of resources a single map reduce job can submit for localization. Contributed by Chris Trezzo 
(cherry picked from commit f80a729832)
 2016-08-17 16:24:16 +00:00
Ray Chiang d1c4ba2d7c MAPREDUCE-6751. Add debug log message when splitting is not possible due to unsplittable compression. (Peter Vary via rchiang) 
(cherry picked from commit 6c154abd33)
 2016-08-16 12:14:30 -07:00
Varun Saxena 705307eda3 MAPREDUCE-6750. Fix TestHSAdminServer#testRefreshSuperUserGroups (Kihwal Lee via Varun Saxena) 2016-08-09 23:43:27 +05:30
Jason Lowe f18febd933 MAPREDUCE-6741. Add MR support to redact job conf properties. Contributed by Haibo Chen 
(cherry picked from commit 4aba858750)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEventHandler.java
 2016-08-09 16:32:38 +00:00
Arun Suresh 5f7edb79d1 YARN-5457. Refactor DistributedScheduling framework to pull out common functionality. (asuresh) 
(cherry picked from commit 82c9e06101)
 2016-08-09 00:46:08 -07:00
Varun Saxena 8b09292f4f MAPREDUCE-6748. Enhance logging for Cluster.java around InetSocketAddress (Vrushali C via Varun Saxena) 2016-08-08 21:53:29 +05:30
Akira Ajisaka e54de94e5a MAPREDUCE-6730. Use StandardCharsets instead of String overload in TextOutputFormat. Contributed by Sahil Kang. 
This closes #114

(cherry picked from commit 70c2781152)
 2016-08-04 20:18:13 +09:00
Gera Shegalov 7eeba35495 MAPREDUCE-6724. Single shuffle to memory must not exceed Integer#MAX_VALUE. (Haibo Chen via gera) 
(cherry picked from commit 6890d5b472)
 2016-08-01 23:41:35 -07:00
Akira Ajisaka 96d5607e7a MAPREDUCE-6729. Accurately compute the test execute time in DFSIO. Contributed by mingleizhang. 
This closes #112

(cherry picked from commit ce93595d7a)
 2016-07-30 22:16:13 +09:00
Akira Ajisaka 0fc838540b MAPREDUCE-6746. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 
(cherry picked from commit 3f0bffea2d)
 2016-07-30 11:47:15 +09:00
Jason Lowe 26df6bd31a MAPREDUCE-6744. Increase timeout on TestDFSIO tests. Contributed by Eric Badger 
(cherry picked from commit 703fdf86c6)
 2016-07-25 21:55:58 +00:00
Akira Ajisaka f4d94f2246 MAPREDUCE-6738. TestJobListCache.testAddExisting failed intermittently in slow VM testbed. Contributed by Junping Du. 
(cherry picked from commit ecff7d0359)
 2016-07-21 11:38:02 -07:00
Sangjin Lee a7e533444c MAPREDUCE-6365. Refactor JobResourceUploader#uploadFilesInternal (Chris Trezzo via sjlee) 
(cherry picked from commit 8f0d3d69d6)
 2016-07-20 11:25:25 -07:00
Robert Kanter 233ffaf3f7 MAPREDUCE-6652. Add configuration property to prevent JHS from loading jobs with a task count greater than X (haibochen via rkanter) 2016-07-15 13:37:04 -07:00
Junping Du 51a3131a87 MAPREDUCE-6733. MapReduce JerseyTest tests failing with java.net.BindException: Address already in use. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit 1d053836b1)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServices.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesAttempt.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesAttempts.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobConf.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesTasks.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServices.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesAttempts.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobConf.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobs.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobsQuery.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesTasks.java
 2016-07-15 11:36:04 -07:00
Robert Kanter a2ef30cb2f MAPREDUCE-6625. TestCLI#testGetJob fails occasionally (haibochen via rkanter) 
(cherry picked from commit ad256beb50)
 2016-07-12 15:51:30 -07:00
Jian He e94e6be772 YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan 2016-07-11 22:37:52 -07:00
Akira Ajisaka b3b81b4f0e HADOOP-13320. Fix arguments check in documentation for WordCount v2.0. Contributed by niccolo becchi. 
(cherry picked from commit 9d46a49c74)
 2016-07-07 15:32:05 -07:00
Rohith Sharma K S 0a1a2ce070 MAPREDUCE-5803. Counters page display all task nevertheless of task type( Map or Reduce). Contributed by Kai Sasaki. 
(cherry picked from commit 4fd37eed90)
 2016-06-28 09:45:15 +05:30
Akira Ajisaka 2830ee5f0f MAPREDUCE-6725. Javadoc for CLI#listEvents() contains no-existent param. Contributed by shenyinjie. 
(cherry picked from commit 0f23cd497b)
 2016-06-25 00:27:18 +09:00
Tsuyoshi Ozawa 3ecbfdfdc2 MAPREDUCE-6721. mapreduce.reduce.shuffle.memory.limit.percent=0.0 should be legal to enforce shuffle to disk. (Gera Shegalov via ozawa) 
This closes #102

(cherry picked from commit 79a7289165)
 2016-06-22 17:20:49 -07:00
Jian He e0f4397232 MAPREDUCE-6197. Cache MapOutputLocations in ShuffleHandler. Contributed by Junping Du 
(cherry picked from commit d8107fcd1c)
 2016-06-21 14:26:23 -07:00
Sangjin Lee 910742ad12 MAPREDUCE-6719. The list of -libjars archives should be replaced with a wildcard in the distributed cache to reduce the application footprint in the state store (Daniel Templeton via sjlee) 
(cherry picked from commit 605b4b6136)
 2016-06-21 11:26:55 -07:00
Akira Ajisaka 1ef89a767d MAPREDUCE-6542. HistoryViewer uses SimpleDateFormat, but SimpleDateFormat is not threadsafe. Contributed by zhangyubiao. 
(cherry picked from commit 51d497fa93)
 2016-06-17 11:05:13 +09:00
Wangda Tan 7eb524ddee YARN-4844. Addendum patch, revert few changes to public and evolving to avoid breaking downstream projects as much as possible. 2016-06-16 11:09:56 -07:00
Wangda Tan 863bfa4d6a YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:12:00 -07:00
Akira Ajisaka d219550e8b HADOOP-13270. BZip2CompressionInputStream finds the same compression marker twice in corner case, causing duplicate data blocks. Contributed by Kai Sasaki. 
(cherry picked from commit e3ba9ad3f1)
 2016-06-14 10:20:43 +09:00
Akira Ajisaka db2d96b378 MAPREDUCE-6741. Refactor UncompressedSplitLineReader.fillBuffer(). Contributed by Daniel Templeton. 
(cherry picked from commit 0b7b8a3776)
 2016-06-10 19:17:05 +09:00
Gera Shegalov 5a9929e19b MAPREDUCE-6240. Hadoop client displays confusing error message. (gera) 
(cherry picked from commit 0af96a1c08)
 2016-06-08 13:49:31 -07:00
Ming Ma 74e2b5efa2 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 
(cherry picked from commit 4a1cedc010)
 2016-06-06 14:34:47 -07:00
Arun Suresh 384a84828f YARN-5180. Allow ResourceRequest to specify an enforceExecutionType flag. (asuresh) 
(cherry picked from commit dc26601d8f)
 2016-06-02 09:14:30 -07:00
Varun Vasudev 4f36c3d214 YARN-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 20:55:39 +05:30
Carlo Curino c656977961 YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 
(cherry picked from commit 013532a95e)
 2016-05-25 16:58:19 -07:00
Jian He 396d222d7a MAPREDUCE-6703. Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers. Contributed by Arun Suresh 
(cherry picked from commit ae353ea969)
 2016-05-24 19:48:11 -07:00
Akira Ajisaka a59f30272d MAPREDUCE-6607. Enable regex pattern matching when mapreduce.task.files.preserve.filepattern is set. Contributed by Kai Sasaki. 2016-05-23 07:25:52 +09:00
Jian He db3cf5b354 MAPREDUCE-6696. Add a configuration to limit the number of map tasks allowed per job. Contributed by Zhihai Xu 
(cherry picked from commit 21d2b90213)
 2016-05-19 19:25:48 -07:00
Junping Du 2703ec6871 YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron. 2016-05-19 14:10:15 -07:00
Robert Kanter 27aabcab45 MAPREDUCE-6686. Add a way to download the job config from the mapred CLI (rkanter) 
(cherry picked from commit 992a49353f)
 2016-05-18 12:16:45 -07:00
Junping Du 7e36b9159e MAPREDUCE-6657. Job history server can fail on startup when NameNode is in start phase. Contributed by Haibo Chen. 
(cherry picked from commit f6ef876fe1)
 2016-05-17 14:42:11 -07:00
Jason Lowe 9c47862d87 MAPREDUCE-6698. Increase timeout on TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser. Contributed by Haibo Chen 
(cherry picked from commit 34fddd1e91)
 2016-05-17 15:04:02 +00:00
Jason Lowe 51e06c085f MAPREDUCE-6701. application master log can not be available when clicking jobhistory's am logs link. Contributed by Haibo Chen 
(cherry picked from commit 12fa4ec141)
 2016-05-17 14:56:23 +00:00
Akira Ajisaka 19db965c27 MAPREDUCE-6693. ArrayIndexOutOfBoundsException occurs when the length of the job name is equal to mapreduce.jobhistory.jobname.limit. Contributed by Ajith S. 
(cherry picked from commit b357930526)
 2016-05-17 20:22:46 +09:00
Jason Lowe 08743edd70 MAPREDUCE-6558. multibyte delimiters with compressed input files generate duplicate records. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 9227dfc25f)
 2016-05-13 14:35:34 +00:00
Steve Loughran eab6e633ee MAPREDUCE-6639 Process hangs in LocatedFileStatusFetcher if FileSystem.get throws. Ryan Blue via stevel 2016-05-12 19:00:08 +01:00
Jason Lowe d3dfed3fd0 MAPREDUCE-6684. High contention on scanning of user directory under immediate_done in Job History Server. Contributed by Haibo Chen 
(cherry picked from commit 5ffb54694b)
 2016-05-10 16:06:26 +00:00
Eric Payne 77d147e0ec MAPREDUCE-6678. Allow ShuffleHandler readahead without drop-behind. Contributed by Nathan Roberts. 
(cherry picked from commit cd35b692de)
 2016-05-10 16:04:09 +00:00
Jason Lowe d1aa3e0f89 MAPREDUCE-6689. MapReduce job can infinitely increase number of reducer resource requests. Contributed by Wangda Tan 
(cherry picked from commit c9bb96fa81)
 2016-05-06 22:27:24 +00:00
Vinod Kumar Vavilapalli e9c7a17bc8 MAPREDUCE-6514. Fixed MapReduce ApplicationMaster to properly updated resources ask after ramping down of all reducers avoiding job hangs. Contributed by Varun Saxena and Wangda Tan. 
(cherry picked from commit 8d48266720)
 2016-05-05 19:05:44 -07:00
Robert Kanter f9d6092108 MAPREDUCE-6675. TestJobImpl.testUnusableNode failed (haibochen via rkanter) 
(cherry picked from commit 9d3fcdfbb3)
 2016-05-04 22:42:34 -07:00
Robert Kanter cca8e9b90c MAPREDUCE-6677. LocalContainerAllocator doesn't specify resource of the containers allocated (haibochen via rkanter) 
(cherry picked from commit ecce3b7d53)
 2016-05-04 22:19:42 -07:00
Chris Nauroth 752a813283 MAPREDUCE-6672. TestTeraSort fails on Windows. Contributed by Tibor Kiss. 
(cherry picked from commit 26ac0f8f0b)
 2016-04-29 11:33:41 -07:00
Harsh J 2d7e4a8228 MAPREDUCE-2398. MRBench: setting the baseDir parameter has no effect. Contributed by Michael Noll and Wilfred Spiegelenburg. 
(cherry picked from commit 7da5847cf1)
 2016-04-21 17:54:24 +05:30
Jian He 70a31458e0 MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du 
(cherry picked from commit 1e48eefe58)
 2016-04-20 19:02:58 -07:00
Xuan 94e4d349b0 YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du 
(cherry picked from commit e6c0742012)
 2016-04-16 19:40:11 -07:00
Eric Payne edb096eab7 MAPREDUCE-6649. getFailureInfo not returning any failure info. Contributed by Eric Badger 2016-04-16 22:22:42 +00:00
Wangda Tan ab23acdfda MAPREDUCE-6513. MR job got hanged forever when one NM unstable for some time. (Varun Saxena via wangda) 2016-04-14 11:16:37 -07:00
Eric Payne 17c4f34b3d MAPREDUCE-6633. AM should retry map attempts if the reduce task encounters commpression related errors. Contributed by Rushabh Shah 
(cherry picked from commit 1fec06e037)
 2016-04-09 18:27:03 +00:00
Robert Kanter 3584cbc355 MAPREDUCE-6647. MR usage counters use the resources requested instead of the resources allocated (haibochen via rkanter) 
(cherry picked from commit 3be1ab485f)
 2016-04-06 17:16:10 -07:00
Junping Du 5574f82799 MAPREDUCE-6670. TestJobListCache#testEviction sometimes fails on Windows with timeout. Contributed by Gergely Novák. 
(cherry picked from commit de96d7c88a)
 2016-04-06 08:35:51 -07:00
Akira Ajisaka 94b2ba89a4 MAPREDUCE-6663. [NNBench] Refactor nnbench as a Tool implementation. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 690d8a368d)
 2016-03-30 11:44:33 +09:00
Akira Ajisaka 7b2cf29f4f MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote. 
(cherry picked from commit 0050fa5f1c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/partition/TestTotalOrderPartitioner.java
 2016-03-29 18:20:01 +09:00
Rohith Sharma K S f22ce96b50 MAPREDUCE-6662. Clear ASF Warnings on test data files. Contributed by Vinayakumar B 
(cherry picked from commit 0ef8bbfd87)
 2016-03-29 10:35:04 +08:00
Junping Du ae29b1a17b MAPREDUCE-6655. Fix a typo (STRICT_IE6) in Encrypted Shuffle. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 90fcb16c0f)
 2016-03-28 05:31:46 -07:00
Akira Ajisaka 2b1f066d2d Revert "MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote." 
This reverts commit 0dc1453626.
 2016-03-28 16:05:55 +09:00
Jason Lowe 8360d98d6a MAPREDUCE-6535. TaskID default constructor results in NPE on toString(). Contributed by Daniel Templeton 
(cherry picked from commit 3f622a143c)
 2016-03-25 20:07:56 +00:00
Akira Ajisaka 0dc1453626 MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote. 
(cherry picked from commit 2c268cc936)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/partition/TestTotalOrderPartitioner.java
 2016-03-25 15:23:23 +09:00
Akira Ajisaka 7a3fd1bc1a MAPREDUCE-6656. [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363. Contributed by J.Andreina. 
(cherry picked from commit 368c77376a)
 2016-03-23 23:24:30 +09:00
Akira Ajisaka 12c1cc889a MAPREDUCE-6580. Test failure: TestMRJobsWithProfiler. Contributed by Eric Badger. 
(cherry picked from commit 0bfe5a0167)
 2016-03-23 13:19:08 +09:00
Li Lu 8bbb5efbfb MAPREDUCE-6110. JobHistoryServer CLI throws NullPointerException with job ids that do not exist. (Kai Sasaki via gtcarrera9) 
(cherry picked from commit 57ead18a85)
 2016-03-22 14:29:12 -07:00
Karthik Kambatla 95b59c17a7 YARN-4732. *ProcessTree classes have too many whitespace issues 
(cherry picked from commit 7fae4c68e6)
 2016-03-20 10:47:56 -07:00
Akira Ajisaka 365d236175 MAPREDUCE-6645. TestWordStats outputs logs under directories other than target/test-dir. Contributed by Gabor Liptak. 
(cherry picked from commit 80fa70c4e1)
 2016-03-17 18:09:20 +09:00
Wangda Tan abfb2957d1 MAPREDUCE-6579. JobStatus#getFailureInfo should not output diagnostic information when the job is running. (Akira AJISAKA via wangda) 
(cherry picked from commit 6529c87551)
 2016-03-15 17:30:37 -07:00
Akira Ajisaka c2e14d1bbb MAPREDUCE-6520. Migrate MR Client test cases part 1. Contributed by Dustin Cote. 
(cherry picked from commit 247a790609)
 2016-03-11 22:54:40 +09:00
Akira Ajisaka 99f61afcdd Revert "MAPREDUCE-6520. Migrate MR Client test cases part 1." 
This reverts commit 7d665f0a98.
 2016-03-11 22:52:10 +09:00
Akira Ajisaka 7d665f0a98 MAPREDUCE-6520. Migrate MR Client test cases part 1. 
(cherry picked from commit ef68b44195)
 2016-03-11 22:25:29 +09:00
Allen Wittenauer 9f4b3eaac0 Revert "HADOOP-12366. expose calculated paths (aw)" 
This reverts commit 45bb467918.
 2016-03-09 09:59:19 -08:00
Chris Nauroth 38a031ce68 HADOOP-12905. Clean up CHANGES.txt RAT exclusions from pom.xml files. Contributed by Chris Nauroth. 
(cherry picked from commit 89b16d27e2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/pom.xml
	hadoop-mapreduce-project/pom.xml
 2016-03-08 21:22:57 -08:00
Varun Vasudev e9a0ffc7f1 YARN-4737. Add CSRF filter support in YARN. Contributed by Jonathan Maron. 
(cherry picked from commit 43416187c07afb35e3267f94d0a41d8d3cfb5735)
 2016-03-07 15:23:36 +05:30
Harsh J ed22444a60 MAPREDUCE-6648. Add yarn.app.mapreduce.am.log.level to mapred-default.xml (harsh) 2016-03-07 13:13:05 +05:30
Masatake Iwasaki 722182fdfb HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims) 
(cherry picked from commit cbd31328a6)

 Conflicts:
	hadoop-common-project/hadoop-common/src/site/markdown/CommandsManual.md
	hadoop-common-project/hadoop-common/src/site/markdown/UnixShellGuide.md
 2016-03-04 14:12:44 +09:00
Andrew Wang 670944810c HADOOP-11792. Remove all of the CHANGES.txt files. 2016-03-03 21:02:58 -08:00
Robert Kanter f9f57265cf MAPREDUCE-4785. TestMRApp occasionally fails (haibochen via rkanter) 
(cherry picked from commit ff0ee84d77)
 2016-03-03 16:38:25 -08:00
Robert Kanter 48cb4b9ba7 MAPREDUCE-6622. Add capability to set JHS job cache to a task-based limit (rchiang via rkanter) 
(cherry picked from commit 0f72da7e28)
 2016-02-26 17:57:45 -08:00
Robert Kanter ec48848711 YARN-4701. When task logs are not available, port 8041 is referenced instead of port 8042 (haibochen via rkanter) 
(cherry picked from commit c4d4df8de0)
 2016-02-25 12:48:29 -08:00
Robert Kanter d9f01f28a0 MAPREDUCE-6640. mapred job -history command should be able to take Job ID (rkanter) 
(cherry picked from commit 4343a4cf77)
 2016-02-23 17:38:25 -08:00
Varun Vasudev f1999fe275 MAPREDUCE-6635. Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException. Contributed by Junping Du. 
(cherry picked from commit c6f2d761d5)
 2016-02-23 13:05:52 +05:30
Tsuyoshi Ozawa 929e869ae0 YARN-4708. Missing default mapper type in TimelineServer performance test tool usage. Contributed by Kai Sasaki. 
(cherry picked from commit b68901d7dd)
 2016-02-22 07:55:35 +09:00
Andrew Wang 7f6737951a MAPREDUCE-6637. Testcase Failure : TestFileInputFormat.testSplitLocationInfo. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 37577852ba)
 2016-02-19 16:33:40 -08:00
Robert Kanter a33109a0b8 MAPREDUCE-6627. Add machine-readable output to mapred job -history command (rkanter) 
(cherry picked from commit 8eee59ce6b)
 2016-02-18 17:58:40 -08:00
Robert Kanter f864faaafa HADOOP-12817. Enable TLS v1.1 and 1.2 (rkanter) 
(cherry picked from commit a365a3941c)
 2016-02-18 11:11:32 -08:00
Varun Vasudev 212c519ad3 MAPREDUCE-6634. Log uncaught exceptions/errors in various thread pools in mapreduce. Contributed by Sidharta Seethana. 
(cherry picked from commit c50d1e54000c76880a041ce5959a2eb23c86bd35)
 2016-02-18 14:19:01 +05:30
Tsuyoshi Ozawa a33cdfb875 MAPREDUCE-6341. addendum patch to fix typos. Contributed by John Michael Luy. 
(cherry picked from commit 453e7e0d10)
 2016-02-17 14:18:04 +09:00
Junping Du a28be9edc7 Set the release date for 2.6.4. 
(cherry picked from commit ec12ce8f48)
 2016-02-13 01:44:12 -08:00
Jason Lowe 303c650d0d MAPREDUCE-6191. Improve clearing stale state of Java serialization 
testcase.  (Sam Liu via Eric Yang)
(cherry picked from commit c379e102dd)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-02-10 16:44:12 +00:00