Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,069 Commits 326 Branches 371 Tags
Commit Graph

1912 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli
cab5d86738 Revert "MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev" 
This reverts commit ba02e8f6d15cdfad4faac0ff3bad0ba34d305aea.
 2015-12-16 15:17:01 -08:00
Zhihai Xu
e1a2f9e6e0 Update CHANGES.txt to move MAPREDUCE-6436 from YARN to MAPREDUCE 
(cherry picked from commit 7092d47fc0b3b792dd31f967c01d460dc089f60b)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-yarn-project/CHANGES.txt
 2015-12-15 22:27:32 -08:00
Zhihai Xu
35a5b8929e MAPREDUCE-6436. JobHistory cache issue. Contributed by Kai Sasaki 
(cherry picked from commit 5b7078d06921893200163a3d29c8901c3c0107cb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-15 01:24:03 -08:00
Xuan
ba02e8f6d1 MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev 
(cherry picked from commit fc470840a0b1f5dc8b3b13f7ed99fd68ba728216)
(cherry picked from commit 3b10ff6a6f7becea181506c306a52973602b0481)
 2015-12-07 14:18:26 -08:00
Jian He
6a30d93aa4 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 
(cherry picked from commit 9f77ccad735f4843ce2c38355de9f434838d4507)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestAHSWebApp.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
 2015-12-03 14:11:48 -08:00
Jason Lowe
e1b6a5413f MAPREDUCE-6549. multibyte delimiters with LineRecordReader cause duplicate records (wilfreds via rkanter) 
(cherry picked from commit 7fd00b3db4b7d73afd41276ba9a06ec06a0e1762)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-11-30 17:45:27 +00:00
Jason Lowe
4ba5667aed MAPREDUCE-5883. "Total megabyte-seconds" in job counters is slightly misleading. Contributed by Nathan Roberts 
(cherry picked from commit cab3c7c8892ad33a7eb0955b01e99872ab95e192)
 2015-11-24 22:02:21 +00:00
Jason Lowe
9bd9544bda Update CHANGES.txt for recent commits to branch-2.6. 
(cherry picked from commit aa2009c37bc3c47035a5aea4a7449d3fe48569d4)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-11-23 22:00:27 +00:00
Jason Lowe
b96377334c MAPREDUCE-6377. JHS sorting on state column not working in webUi. 
Contributed by zhihai xu.
(cherry picked from commit 790a861766dcd60212d83f444f2f96d3acf20a94)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-11-18 17:12:50 +00:00
Jian He
27b786660e Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan" 
This reverts commit cb4d4d5918c4d488056e5521e343aaa9ffbd24d1.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-11-16 20:20:46 -08:00
Jian He
f95f416fad MAPREDUCE-5485. Allow repeating job commit by extending OutputCommitter API. Contributed by Junping Du 2015-11-16 17:13:49 -08:00
Sangjin Lee
adede3e53d MAPREDUCE-6540. TestMRTimelineEventHandling fails (sjlee) 
(cherry picked from commit 9ad708a0dfa5e82d1a17471b1cf3e76204b32cbc)
 2015-11-11 16:35:46 -08:00
Jian He
cb4d4d5918 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan 
(cherry picked from commit 8fbea531d7f7b665f6f55af54c8ebf330118ff37)
(cherry picked from commit a4ff03b200cf02a49ab81fa3c71cb707f6d2fee1)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestAHSWebApp.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
 2015-11-09 14:30:47 -08:00
Jason Lowe
7499f71e77 Add 2.7.3 release to CHANGES.txt 
(cherry picked from commit 0383a3973b3b734fb23c331a2256dc92cff05365)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-yarn-project/CHANGES.txt
 2015-11-04 16:29:56 +00:00
Vinod Kumar Vavilapalli
09348c4bce Preparing for 2.7.3 development 2015-11-03 15:18:10 -08:00
Sangjin Lee
c8767fb68c Updated the 2.6.2 final release date. 
(cherry picked from commit a4a6b5b4b470b1e7a3c5e2d38433429c455bc709)
 2015-10-30 18:53:10 -07:00
Kihwal Lee
7b783911c4 MAPREDUCE-6451. DistCp has incorrect chunkFilePath for multiple jobs when strategy is dynamic. Contributed by Kuhu Shukla. 
(cherry picked from commit 2868ca0328d908056745223fb38d9a90fd2811ba)
 2015-10-30 14:58:54 -05:00
Jason Lowe
431f48f653 MAPREDUCE-6528. Memory leak for HistoryFileManager.getJobSummary(). Contributed by Junping Du 
(cherry picked from commit 6344b6a7694c70f296392b6462dba452ff762109)
 2015-10-30 15:33:08 +00:00
Jason Lowe
51783bb88a Update CHANGES.txt to reflect commit of MR-6273 to branch-2.6 
(cherry picked from commit 6ae9efaf5949bd5a5f4fd99b5777ce8f6d7f3a2c)
 2015-10-30 15:20:23 +00:00
Jason Lowe
e894be93f6 Creating 2.6.3 entries in CHANGES.txt files. 
(cherry picked from commit eadf7b3096cb010eb7f0afd9afd4ae0d67b2645f)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-yarn-project/CHANGES.txt
 2015-10-30 14:55:33 +00:00
Tsuyoshi Ozawa
c84ec0657f YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188. Contributed by Varun Saxena. 
(cherry picked from commit ce31b22739512804da38cf87e0ce1059e3128da3)
 2015-10-30 18:00:45 +09:00
Xuan
ceefabc44a YARN-4313. Race condition in MiniMRYarnCluster when getting history 
server address. Contributed by Jian He

(cherry picked from commit 7412ff48eeb967c972c19c1370c77a41c5b3b81f)
(cherry picked from commit b505bf9e582f5bca59be28296a4b4a640b3389eb)
 2015-10-29 17:38:09 -07:00
Jason Lowe
7499981d3c MAPREDUCE-6273. HistoryFileManager should check whether summaryFile exists 
to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED
state. Contributed by zhihai xu.

(cherry picked from commit 1ea90144d58443a7431ff33ba49ee19278ebe42b)
(cherry picked from commit b9865cbf3d435903835afa23c74c5e2ecfbba1bc)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-10-29 18:29:45 +00:00
Jason Lowe
0dc69afdc7 MAPREDUCE-6518. Set SO_KEEPALIVE on shuffle connections. Contributed by Chang Li 
(cherry picked from commit ee9b80acf84718b952cbb82380cd0a4dbe22dc79)
 2015-10-21 21:46:59 +00:00
Akira Ajisaka
3096d3e1a1 MAPREDUCE-6497. Fix wrong value of JOB_FINISHED event in JobHistoryEventHandler. Contributed by Shinichi Yamashita. 
(cherry picked from commit 195793c6f3e53a5c0527020477fe9c9158576f77)
(cherry picked from commit e42f57623251f0a5d5607e6a8b3d732de8c8a5a8)
 2015-10-01 10:26:24 +02:00
Jason Lowe
fc10c99839 MAPREDUCE-6492. AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask. Contributed by Bibin A Chundatt 
(cherry picked from commit ab11085b81353e1617875deb10f3c8e2a8b91a1e)
 2015-09-28 22:13:40 +00:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
2b4c7fcc67 Release process for 2.6.1: Set the release date for 2.6.1 
(cherry picked from commit 1f707ecffd80142c25ef0dee8708da49920646bc)
(cherry picked from commit f48af332a76caa89c9d8fa1d2654d290bffdc183)
 2015-09-23 15:13:09 -07:00
Jason Lowe
7429438c85 MAPREDUCE-5982. Task attempts that fail from the ASSIGNED state can disappear. Contributed by Chang Li 2015-09-18 19:21:35 +00:00
Jason Lowe
521196874c MAPREDUCE-6481. LineRecordReader may give incomplete record and wrong position/key information for uncompressed input sometimes. Contributed by Zhihai Xu 
(cherry picked from commit 58d1a02b8d66b1d2a6ac2158be32bd35ad2e69bd)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-09-18 15:10:38 +00:00
Jason Lowe
850d6a3fd3 MAPREDUCE-5948. org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well. Contributed by Vinayakumar B, Rushabh Shah, and Akira AJISAKA 
(cherry picked from commit 077250d8d7b4b757543a39a6ce8bb6e3be356c6f)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-09-18 15:04:02 +00:00
Jason Lowe
ca6d5b4b92 MAPREDUCE-6472. MapReduce AM should have java.io.tmpdir=./tmp to be consistent with tasks. Contributed by Naganarasimha G R 
(cherry picked from commit 62943b8e3aff3b274c439f72a8bb86094c1ab0e8)
 2015-09-15 19:45:06 +00:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
c05ba079e3 Updating all CHANGES.txt files to move entires from future releases into 2.6.1 
section given the large number of backports to 2.6.1.

(cherry picked from commit fbbb7ff1ed111884d0375c177239073153139848)
(cherry picked from commit cbf90f93f9ab3463307292ddfe6fe98ab8b37842)
 2015-09-10 16:10:56 -07:00
Jason Lowe
4ea42b84b7 MAPREDUCE-5649. Reduce cannot use more than 2G memory for the final merge. Contributed by Gera Shegalov 
(cherry picked from commit 7dc3c1203d1ab14c09d0aaf0869a5bcdfafb0a5a)

(cherry picked from commit 87c2d915f1cc799cb4020c945c04d3ecb82ee963)
 2015-09-10 11:00:00 -07:00
Hitesh Shah
f6b7dc24f0 YARN-2890. MiniYarnCluster should turn on timeline service if configured to do so. Contributed by Mit Desai. 
(cherry picked from commit 265ed1fe804743601a8b62cabc1e4dc2ec8e502f)
(cherry picked from commit 55b794e7fa205df655c19bbfe1de99091fa9dc64)
 2015-09-10 10:55:23 -07:00
Jason Lowe
d1186dbafa MAPREDUCE-6474. ShuffleHandler can possibly exhaust nodemanager file descriptors. Contributed by Kuhu Shukla 
(cherry picked from commit 8e615588d5216394d0251a9c97bd706537856c6d)
 2015-09-10 16:05:48 +00:00
Tsuyoshi Ozawa
e8a15495bc MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. 
(cherry picked from commit 9b685773ec1031e90cc2b8aedb42670721757e22)
 2015-09-05 11:24:30 +09:00
Vinod Kumar Vavilapalli
f333e367e1 MAPREDUCE-6454. Fixed MapReduce to modify HADOOP_CLASSPATH to have distributed cache files so that child processes running hadoop scripts can access these files. Contributed by Junping Du. 
(cherry picked from commit 9da81a395584cb1dbfdee012222fd4de53522681)
 2015-08-20 17:39:43 -07:00
Vinod Kumar Vavilapalli
9faee040f9 Creating 2.6.2 entries in CHANGES.txt files. 
(cherry picked from commit 33474939a048ac7bbe07df772f0000c84979722f)
 2015-08-20 17:39:29 -07:00
Karthik Kambatla
d610144d45 MAPREDUCE-6439. AM may fail instead of retrying if RM shuts down during the allocate call. (Anubhav Dhoot via kasha) 
(cherry picked from commit 8dfec7a1979e8f70f8355c096874921d368342ef)
(cherry picked from commit 20734320a18f12a28e6e4cd7b73dfcb5df869696)
 2015-08-15 00:56:22 -07:00
Devaraj K
7e5f73a952 MAPREDUCE-6426. TestShuffleHandler#testGetMapOutputInfo is failing. 
Contributed by zhihai xu.

(cherry picked from commit fffb15bb431fcdd7ca4ce0e249f9d45f4968497b)
 2015-07-09 15:05:11 +05:30
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
dcafd3ccd2 Release process for 2.7.1: Set the release date for 2.7.1. 
(cherry picked from commit bf89ddb9b8ca27a34074b415f85599dd48b8fc50)
 2015-07-06 16:41:38 -07:00
Devaraj K
c9c7a00a82 MAPREDUCE-6425. ShuffleHandler passes wrong "base" parameter to 
getMapOutputInfo if mapId is not in the cache. Contributed by zhihai xu.

(cherry picked from commit bff67dfe2f811654ffb1bbcbd87509c185f452b6)
 2015-07-06 13:48:44 +05:30
Vinod Kumar Vavilapalli
a36f090c55 Preparing for 2.7.2 development: mvn versions:set -DnewVersion=2.7.2 2015-06-28 16:41:12 -07:00
Vinod Kumar Vavilapalli
2e99675c72 Adding release 2.7.2 to CHANGES.txt. 
(cherry picked from commit aad6a7d5dba5858d6e9845f18c4baec16c91911d)
 2015-06-28 16:33:49 -07:00
Vinod Kumar Vavilapalli
a90e9b2f6f MAPREDUCE-6410. Fixed MapReduce JobHistory server to use the right (login) UGI to refresh log and cleaner settings. Contributed by Varun Saxena. 
(cherry picked from commit d481684c7c9293a94f54ef622a92753531c6acc7)
 2015-06-23 11:41:31 -07:00
Karthik Kambatla
d404caaa43 MAPREDUCE-6387. Serialize the recently added Task#encryptedSpillKey field at the end. (Arun Suresh via kasha) 
(cherry picked from commit 6786daab33792c9c426adf0fc6b361f193d356a8)
(cherry picked from commit 1a2e6e8091d8578950a8623a023422ba5ee87d53)
 2015-06-05 09:19:52 -07:00
Akira Ajisaka
718dca4859 HADOOP-12058. Fix dead links to DistCp and Hadoop Archives pages. Contributed by Kazuho Fujii. 
(cherry picked from commit dbed757cbaf3ddff0c9ae2d905d42828aadedf2a)
(cherry picked from commit 1cccd1eeb3c1c22f570f657ba1eb80a9a37b1496)
 2015-06-05 01:47:01 +09:00
Xuan
1ad0d43fb4 Revert "YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong." 
This reverts commit 2032e8d1a0c6abe154a64df69b0124cdae9f78e0.
 2015-06-01 11:29:36 -07:00
Zhijie Shen
2032e8d1a0 YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong. 2015-05-30 09:39:49 -07:00
cnauroth
02ab202a2b YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch. 
(cherry picked from commit 7bc33b63c78c6ee9a04c0c1511ef16e63f7f3074)
(cherry picked from commit 6ed8989a6f897437ab53c85c42fa8b58fe556359)
 2015-05-27 14:21:26 -07:00