Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,970 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

717 Commits

Author SHA1 Message Date
Daniel Templeton 2f32accaaf MAPREDUCE-6846. Fragments specified for libjar paths are not handled correctly 
(Contributed by Chris Trezzo via Daniel Templeton)

(cherry picked from commit a92bc8a6c1c50d34d8229a67dc9d197823001551)
 2017-04-05 17:26:06 -07:00
Zhe Zhang 3f5dc51ec9 MAPREDUCE-6873. MR Job Submission Fails if MR framework application path not on defaultFS. Contributed by Erik Krogen. 
(cherry picked from commit de7efd2687)
 2017-03-29 22:11:22 -07:00
Ming Ma 2ae9ae1864 MAPREDUCE-6862. Fragments are not handled correctly by resource limit checking. (Chris Trezzo via mingma) 
(cherry picked from commit ceab00ac62)
 2017-03-29 17:42:40 -07:00
Joe Mészáros 34a11b9108
MAPREDUCE-6866. Fix getNumMapTasks() documentation in JobConf. 
This closes #205

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e0a2eb71e3)
 2017-03-27 13:52:26 +09:00
Naganarasimha 0cfa6ec6c8 MAPREDUCE-6865. Fix typo in javadoc for DistributedCache. Contributed by Attila Sasvari. 
(cherry picked from commit 34a931c0fa)
 2017-03-20 08:26:05 +05:30
Akira Ajisaka 120f680318
Treat encrypted files as private. Contributed by Daniel Templeton. 
(cherry picked from commit f01a69f84f)
 2017-03-07 13:38:20 +09:00
Akira Ajisaka e563cd8692 MAPREDUCE-6841. Fix dead link in MapReduce tutorial document. Contributed by Victor Nee. 
(cherry picked from commit 4d33683882)
 2017-03-03 16:12:56 +09:00
Jian He b8b605bfc5 MAPREDUCE-6852. Job#updateStatus() failed with NPE due to race condition. Contributed by Junping Du 2017-03-02 10:59:18 -08:00
Junping Du 81e2d54ac6 MAPREDUCE-6404. Allow AM to specify a port range for starting its webapp. Contributed by Varun Saxena. 
(cherry picked from commit 7afe5b1a31)
 2017-02-07 01:46:17 -08:00
Masatake Iwasaki 2f4cf07fb1 MAPREDUCE-6644. Use doxia macro to generate in-page TOC of MapReduce site documentation. (iwasakims) 
(cherry picked from commit 3619ae32be)
 2017-02-02 01:23:02 +09:00
Karthik Kambatla a91cc63d37 MAPREDUCE-6829. Add peak memory usage counter for each task. (Miklos Szegedi via kasha) 
(cherry picked from commit c65f884fc7)
 2017-01-26 11:08:53 -08:00
Jason Lowe 94b326f2c8 YARN-5910. Support for multi-cluster delegation tokens. Contributed by Jian He 
(cherry picked from commit 69fa81679f)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java
 2017-01-23 17:23:32 +00:00
Arun Suresh 4985217de4 YARN-6066. Opportunistic containers Minor fixes : API annotations, parameter name changes, checkstyles. (asuresh) 2017-01-08 08:27:10 -08:00
Arun Suresh 2b4d3e8506 YARN-5646. Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 2273a74c1f)
(cherry picked from commit 9e17ffe599e12c59c48f4355de645c31a4735d04)
 2017-01-06 11:15:10 -08:00
Daniel Templeton d1aa844dc6 MAPREDUCE-6715. Fix Several Unsafe Practices (Contributed by Yufei Gu via Daniel Templeton) 
(cherry picked from commit 0b8a7c18dd)
 2017-01-05 17:57:14 -08:00
Daniel Templeton 28b605dd1a HADOOP-13940. Document the missing envvars commands (Contributed by Yiqun Lin via Daniel Templeton) 
(cherry picked from commit d432173737a532b7106682116013b751ba162ff0)
 2016-12-27 07:19:37 -08:00
Akira Ajisaka 1e1ab9bc1b MAPREDUCE-6820. Fix dead links in Job relevant classes. Contributed by Yiqun Lin. 
(cherry picked from commit 92a8917ca5)
 2016-12-10 10:57:06 +09:00
Rohith Sharma K S 4b289d567e MAPREDUCE-6793. io.sort.factor code default and mapred-default.xml values inconsistent. Contributed by Gera Shegalov. 
(cherry picked from commit 683e0c71fe)
 2016-11-21 22:11:59 +05:30
Mingliang Liu 4e0fcff8ab HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu 
(cherry picked from commit 5af572b644)
 2016-11-15 11:36:08 -08:00
Akira Ajisaka a3897f77ae MAPREDUCE-6795. Update the document for JobConf#setNumReduceTasks. Contributed by Yiqun Lin. 
(cherry picked from commit 235f39b7c4)
 2016-11-01 20:52:35 +09:00
Akira Ajisaka ef13a5ef9d MAPREDUCE-6799. Document mapreduce.jobhistory.webapp.https.address in mapred-default.xml. Contributed by Yiqun Lin. 
(cherry picked from commit f646fe34be)
 2016-10-31 15:18:34 +09:00
Naganarasimha 1455f56b63 MAPREDUCE-2631. Potential resource leaks in BinaryProtocol$TeeOutputStream.java. Contributed by Sunil G. 
(cherry picked from commit 28660f51af)
 2016-10-28 06:00:29 +05:30
Varun Saxena f572448809 YARN-4388. Cleanup mapreduce.job.hdfs-servers from yarn-default.xml (Junping Du via Varun Saxena) 2016-10-28 02:24:24 +05:30
Junping Du 432657780d MAPREDUCE-6792. Allow user's full principal name as owner of MapReduce staging directory in JobSubmissionFiles#JobStagingDir(). Contributed by Santhosh G Nayak. 
(cherry picked from commit 9b7ce948a5)
 2016-10-25 11:21:33 -07:00
Robert Kanter c76eb90f0e MAPREDUCE-6728. Give fetchers hint when ShuffleHandler rejects a shuffling connection (haibochen via rkanter) 2016-10-24 17:47:09 -07:00
Robert Kanter 48ca6be76e MAPREDUCE-6776. yarn.app.mapreduce.client.job.max-retries should have a more useful default (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit f3f37e6fb8)
 2016-10-07 14:47:23 -07:00
Robert Kanter 254041b71a MAPREDUCE-6632. Master.getMasterAddress() should be updated to use YARN-4629 (templedf via rkanter) 
(cherry picked from commit 4fc632ae19)
 2016-09-22 16:13:24 -07:00
Karthik Kambatla a0b0767856 MAPREDUCE-6740. Enforce mapreduce.task.timeout to be at least mapreduce.task.progress-report.interval. (Haibo Chen via kasha) 
(cherry picked from commit 537095d13c)
 2016-09-21 18:30:42 -07:00
Naganarasimha fa80972efb MAPREDUCE-6777. Typos in 4 log messages. Contributed by Mehran Hassani 2016-09-17 10:21:13 +05:30
Chris Douglas 806a8dea38 MAPREDUCE-6628. Potential memory leak in CryptoOutputStream. Contributed by Mariappan Asokan 
(cherry picked from commit 9f192cc5ac)
 2016-09-09 11:14:30 -07:00
Jason Lowe 5cdc4ba9f4 MAPREDUCE-6761. Regression when handling providers - invalid configuration ServiceConfiguration causes Cluster initialization failure. Contributed by Peter Vary 
(cherry picked from commit 6fa9bf4407)
 2016-08-24 14:42:06 +00:00
Akira Ajisaka 41fe85f868 MAPREDUCE-6587. Remove unused params in connection-related methods of Fetcher. Contributed by Yiqun Lin. 
(cherry picked from commit 8cc4a67059)
 2016-08-23 17:05:45 +09:00
Varun Saxena 4ba8dbf00e MAPREDUCE-6762. ControlledJob#toString failed with NPE when job status is not successfully updated (Weiwei Yang via Varun Saxena) 2016-08-21 21:50:44 +05:30
Wangda Tan 65822e55fd MAPREDUCE-6310. Add jdiff support to MapReduce. (Li Lu/vinodkv via wangda) 
(cherry picked from commit 3d937457ee)
 2016-08-19 16:41:47 -07:00
Varun Saxena 5128e3320e MAPREDUCE-6763. Shuffle server listen queue is too small (Jason Lowe via Varun Saxena) 2016-08-20 04:05:01 +05:30
Jason Lowe 848ae35a93 MAPREDUCE-6690. Limit the number of resources a single map reduce job can submit for localization. Contributed by Chris Trezzo 
(cherry picked from commit f80a729832)
 2016-08-17 16:24:16 +00:00
Ray Chiang d1c4ba2d7c MAPREDUCE-6751. Add debug log message when splitting is not possible due to unsplittable compression. (Peter Vary via rchiang) 
(cherry picked from commit 6c154abd33)
 2016-08-16 12:14:30 -07:00
Jason Lowe f18febd933 MAPREDUCE-6741. Add MR support to redact job conf properties. Contributed by Haibo Chen 
(cherry picked from commit 4aba858750)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/jobhistory/TestJobHistoryEventHandler.java
 2016-08-09 16:32:38 +00:00
Varun Saxena 8b09292f4f MAPREDUCE-6748. Enhance logging for Cluster.java around InetSocketAddress (Vrushali C via Varun Saxena) 2016-08-08 21:53:29 +05:30
Akira Ajisaka e54de94e5a MAPREDUCE-6730. Use StandardCharsets instead of String overload in TextOutputFormat. Contributed by Sahil Kang. 
This closes #114

(cherry picked from commit 70c2781152)
 2016-08-04 20:18:13 +09:00
Gera Shegalov 7eeba35495 MAPREDUCE-6724. Single shuffle to memory must not exceed Integer#MAX_VALUE. (Haibo Chen via gera) 
(cherry picked from commit 6890d5b472)
 2016-08-01 23:41:35 -07:00
Akira Ajisaka 0fc838540b MAPREDUCE-6746. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 
(cherry picked from commit 3f0bffea2d)
 2016-07-30 11:47:15 +09:00
Sangjin Lee a7e533444c MAPREDUCE-6365. Refactor JobResourceUploader#uploadFilesInternal (Chris Trezzo via sjlee) 
(cherry picked from commit 8f0d3d69d6)
 2016-07-20 11:25:25 -07:00
Robert Kanter 233ffaf3f7 MAPREDUCE-6652. Add configuration property to prevent JHS from loading jobs with a task count greater than X (haibochen via rkanter) 2016-07-15 13:37:04 -07:00
Robert Kanter a2ef30cb2f MAPREDUCE-6625. TestCLI#testGetJob fails occasionally (haibochen via rkanter) 
(cherry picked from commit ad256beb50)
 2016-07-12 15:51:30 -07:00
Akira Ajisaka b3b81b4f0e HADOOP-13320. Fix arguments check in documentation for WordCount v2.0. Contributed by niccolo becchi. 
(cherry picked from commit 9d46a49c74)
 2016-07-07 15:32:05 -07:00
Akira Ajisaka 2830ee5f0f MAPREDUCE-6725. Javadoc for CLI#listEvents() contains no-existent param. Contributed by shenyinjie. 
(cherry picked from commit 0f23cd497b)
 2016-06-25 00:27:18 +09:00
Tsuyoshi Ozawa 3ecbfdfdc2 MAPREDUCE-6721. mapreduce.reduce.shuffle.memory.limit.percent=0.0 should be legal to enforce shuffle to disk. (Gera Shegalov via ozawa) 
This closes #102

(cherry picked from commit 79a7289165)
 2016-06-22 17:20:49 -07:00
Sangjin Lee 910742ad12 MAPREDUCE-6719. The list of -libjars archives should be replaced with a wildcard in the distributed cache to reduce the application footprint in the state store (Daniel Templeton via sjlee) 
(cherry picked from commit 605b4b6136)
 2016-06-21 11:26:55 -07:00
Akira Ajisaka 1ef89a767d MAPREDUCE-6542. HistoryViewer uses SimpleDateFormat, but SimpleDateFormat is not threadsafe. Contributed by zhangyubiao. 
(cherry picked from commit 51d497fa93)
 2016-06-17 11:05:13 +09:00