Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,002 Commits 326 Branches 371 Tags
Commit Graph

1625 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
6f38ccc674 (Re)Set the release date for 2.7.2-RC0 2015-11-11 10:54:53 -08:00
Vinod Kumar Vavilapalli
3843243b59 Set the release date for 2.7.2-RC0 2015-11-10 12:44:37 -08:00
Sangjin Lee
c8767fb68c Updated the 2.6.2 final release date. 
(cherry picked from commit a4a6b5b4b470b1e7a3c5e2d38433429c455bc709)
 2015-10-30 18:53:10 -07:00
Kihwal Lee
7b783911c4 MAPREDUCE-6451. DistCp has incorrect chunkFilePath for multiple jobs when strategy is dynamic. Contributed by Kuhu Shukla. 
(cherry picked from commit 2868ca0328d908056745223fb38d9a90fd2811ba)
 2015-10-30 14:58:54 -05:00
Jason Lowe
431f48f653 MAPREDUCE-6528. Memory leak for HistoryFileManager.getJobSummary(). Contributed by Junping Du 
(cherry picked from commit 6344b6a7694c70f296392b6462dba452ff762109)
 2015-10-30 15:33:08 +00:00
Jason Lowe
51783bb88a Update CHANGES.txt to reflect commit of MR-6273 to branch-2.6 
(cherry picked from commit 6ae9efaf5949bd5a5f4fd99b5777ce8f6d7f3a2c)
 2015-10-30 15:20:23 +00:00
Jason Lowe
e894be93f6 Creating 2.6.3 entries in CHANGES.txt files. 
(cherry picked from commit eadf7b3096cb010eb7f0afd9afd4ae0d67b2645f)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-yarn-project/CHANGES.txt
 2015-10-30 14:55:33 +00:00
Jason Lowe
7499981d3c MAPREDUCE-6273. HistoryFileManager should check whether summaryFile exists 
to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED
state. Contributed by zhihai xu.

(cherry picked from commit 1ea90144d58443a7431ff33ba49ee19278ebe42b)
(cherry picked from commit b9865cbf3d435903835afa23c74c5e2ecfbba1bc)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-10-29 18:29:45 +00:00
Jason Lowe
0dc69afdc7 MAPREDUCE-6518. Set SO_KEEPALIVE on shuffle connections. Contributed by Chang Li 
(cherry picked from commit ee9b80acf84718b952cbb82380cd0a4dbe22dc79)
 2015-10-21 21:46:59 +00:00
Akira Ajisaka
3096d3e1a1 MAPREDUCE-6497. Fix wrong value of JOB_FINISHED event in JobHistoryEventHandler. Contributed by Shinichi Yamashita. 
(cherry picked from commit 195793c6f3e53a5c0527020477fe9c9158576f77)
(cherry picked from commit e42f57623251f0a5d5607e6a8b3d732de8c8a5a8)
 2015-10-01 10:26:24 +02:00
Jason Lowe
fc10c99839 MAPREDUCE-6492. AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask. Contributed by Bibin A Chundatt 
(cherry picked from commit ab11085b81353e1617875deb10f3c8e2a8b91a1e)
 2015-09-28 22:13:40 +00:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
2b4c7fcc67 Release process for 2.6.1: Set the release date for 2.6.1 
(cherry picked from commit 1f707ecffd80142c25ef0dee8708da49920646bc)
(cherry picked from commit f48af332a76caa89c9d8fa1d2654d290bffdc183)
 2015-09-23 15:13:09 -07:00
Jason Lowe
7429438c85 MAPREDUCE-5982. Task attempts that fail from the ASSIGNED state can disappear. Contributed by Chang Li 2015-09-18 19:21:35 +00:00
Jason Lowe
521196874c MAPREDUCE-6481. LineRecordReader may give incomplete record and wrong position/key information for uncompressed input sometimes. Contributed by Zhihai Xu 
(cherry picked from commit 58d1a02b8d66b1d2a6ac2158be32bd35ad2e69bd)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-09-18 15:10:38 +00:00
Jason Lowe
850d6a3fd3 MAPREDUCE-5948. org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well. Contributed by Vinayakumar B, Rushabh Shah, and Akira AJISAKA 
(cherry picked from commit 077250d8d7b4b757543a39a6ce8bb6e3be356c6f)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-09-18 15:04:02 +00:00
Jason Lowe
ca6d5b4b92 MAPREDUCE-6472. MapReduce AM should have java.io.tmpdir=./tmp to be consistent with tasks. Contributed by Naganarasimha G R 
(cherry picked from commit 62943b8e3aff3b274c439f72a8bb86094c1ab0e8)
 2015-09-15 19:45:06 +00:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
c05ba079e3 Updating all CHANGES.txt files to move entires from future releases into 2.6.1 
section given the large number of backports to 2.6.1.

(cherry picked from commit fbbb7ff1ed111884d0375c177239073153139848)
(cherry picked from commit cbf90f93f9ab3463307292ddfe6fe98ab8b37842)
 2015-09-10 16:10:56 -07:00
Jason Lowe
4ea42b84b7 MAPREDUCE-5649. Reduce cannot use more than 2G memory for the final merge. Contributed by Gera Shegalov 
(cherry picked from commit 7dc3c1203d1ab14c09d0aaf0869a5bcdfafb0a5a)

(cherry picked from commit 87c2d915f1cc799cb4020c945c04d3ecb82ee963)
 2015-09-10 11:00:00 -07:00
Jason Lowe
d1186dbafa MAPREDUCE-6474. ShuffleHandler can possibly exhaust nodemanager file descriptors. Contributed by Kuhu Shukla 
(cherry picked from commit 8e615588d5216394d0251a9c97bd706537856c6d)
 2015-09-10 16:05:48 +00:00
Tsuyoshi Ozawa
e8a15495bc MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. 
(cherry picked from commit 9b685773ec1031e90cc2b8aedb42670721757e22)
 2015-09-05 11:24:30 +09:00
Vinod Kumar Vavilapalli
f333e367e1 MAPREDUCE-6454. Fixed MapReduce to modify HADOOP_CLASSPATH to have distributed cache files so that child processes running hadoop scripts can access these files. Contributed by Junping Du. 
(cherry picked from commit 9da81a395584cb1dbfdee012222fd4de53522681)
 2015-08-20 17:39:43 -07:00
Vinod Kumar Vavilapalli
9faee040f9 Creating 2.6.2 entries in CHANGES.txt files. 
(cherry picked from commit 33474939a048ac7bbe07df772f0000c84979722f)
 2015-08-20 17:39:29 -07:00
Karthik Kambatla
d610144d45 MAPREDUCE-6439. AM may fail instead of retrying if RM shuts down during the allocate call. (Anubhav Dhoot via kasha) 
(cherry picked from commit 8dfec7a1979e8f70f8355c096874921d368342ef)
(cherry picked from commit 20734320a18f12a28e6e4cd7b73dfcb5df869696)
 2015-08-15 00:56:22 -07:00
Devaraj K
7e5f73a952 MAPREDUCE-6426. TestShuffleHandler#testGetMapOutputInfo is failing. 
Contributed by zhihai xu.

(cherry picked from commit fffb15bb431fcdd7ca4ce0e249f9d45f4968497b)
 2015-07-09 15:05:11 +05:30
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
dcafd3ccd2 Release process for 2.7.1: Set the release date for 2.7.1. 
(cherry picked from commit bf89ddb9b8ca27a34074b415f85599dd48b8fc50)
 2015-07-06 16:41:38 -07:00
Devaraj K
c9c7a00a82 MAPREDUCE-6425. ShuffleHandler passes wrong "base" parameter to 
getMapOutputInfo if mapId is not in the cache. Contributed by zhihai xu.

(cherry picked from commit bff67dfe2f811654ffb1bbcbd87509c185f452b6)
 2015-07-06 13:48:44 +05:30
Vinod Kumar Vavilapalli
2e99675c72 Adding release 2.7.2 to CHANGES.txt. 
(cherry picked from commit aad6a7d5dba5858d6e9845f18c4baec16c91911d)
 2015-06-28 16:33:49 -07:00
Vinod Kumar Vavilapalli
a90e9b2f6f MAPREDUCE-6410. Fixed MapReduce JobHistory server to use the right (login) UGI to refresh log and cleaner settings. Contributed by Varun Saxena. 
(cherry picked from commit d481684c7c9293a94f54ef622a92753531c6acc7)
 2015-06-23 11:41:31 -07:00
Karthik Kambatla
d404caaa43 MAPREDUCE-6387. Serialize the recently added Task#encryptedSpillKey field at the end. (Arun Suresh via kasha) 
(cherry picked from commit 6786daab33792c9c426adf0fc6b361f193d356a8)
(cherry picked from commit 1a2e6e8091d8578950a8623a023422ba5ee87d53)
 2015-06-05 09:19:52 -07:00
Tsuyoshi Ozawa
6b03ec5137 MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. 2015-05-19 12:01:14 -07:00
Vinod Kumar Vavilapalli
0e68e304f7 MAPREDUCE-6251. Added a new config for JobClient to retry JobStatus calls so that they don't fail on history-server backed by DFSes with not so strong guarantees. Contributed by Craig Welch. 
(cherry picked from commit f24452d14e9ba48cdb82e5e6e5c10ce5b1407308)
 2015-05-12 12:16:16 -07:00
Jason Lowe
e7b6a076db MAPREDUCE-6259. IllegalArgumentException due to missing job submit time. Contributed by zhihai xu 
(cherry picked from commit bf70c5ae2824a9139c1aa9d7c14020018881cec2)
 2015-05-04 20:40:43 +00:00
Devaraj K
9828638fac MAPREDUCE-6339. Job history file is not flushed correctly because 
isTimerActive flag is not set true when flushTimerTask is scheduled.
Contributed by zhihai xu.

(cherry picked from commit f5b38477f9d0827b238fadd260c1dd2889531fd4)
 2015-04-30 12:37:36 +05:30
Jason Lowe
30d0f10458 MAPREDUCE-6334. Fetcher#copyMapOutput is leaking usedMemory upon IOException during InMemoryMapOutput shuffle handler. Contributed by Eric Payne 
(cherry picked from commit bc1bd7e5c4047b374420683d36a8c30eda6d75b6)
 2015-04-28 20:20:32 +00:00
Devaraj K
a940f19226 MAPREDUCE-6252. JobHistoryServer should not fail when encountering a 
missing directory. Contributed by Craig Welch.

(cherry picked from commit 5e67c4d384193b38a85655c8f93193596821faa5)
 2015-04-28 12:31:46 +05:30
Vinod Kumar Vavilapalli
3a7bfdf318 MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM tokens when they roll-over. Contributed by Jason Lowe. 
(cherry picked from commit 9fc32c5c4d1d5f50c605bdb0e3b13f44c86660c8)
 2015-04-27 15:01:15 -07:00
Robert Kanter
0eae3bc0f6 MAPREDUCE-6238. MR2 can't run local jobs with -libjars command options which is a regression from MR1 (zxu via rkanter) 
(cherry picked from commit d50e8f09287deeb51012d08e326a2ed71a6da869)
 2015-04-23 17:47:00 -07:00
Vinod Kumar Vavilapalli
7dd5f42a76 Set the release date for 2.7.0 
(cherry picked from commit d52de61544060a04a273114f55979e7e929a576c)
 2015-04-20 20:18:38 -07:00
Akira Ajisaka
48d6e994bf MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 1fa8075794894b3e0053d251788277fb88e85533)
(cherry picked from commit 51c458a5c4562c54ff147824816f7b44c4b9b8f6)
 2015-04-16 10:16:48 -05:00
Akira Ajisaka
a41c1c80b0 Revert "MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li." 
This reverts commit 743c74a5730aa0bbd2acd6880f69a0133d4f6c7c.
 2015-04-11 12:38:05 +09:00
Akira Ajisaka
743c74a573 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 4cec9975c0e2c2fcca090888dc2c5e149c5be1e9)
(cherry picked from commit f267fe567cab1db6508412509743e80f59ea69da)
 2015-04-11 01:30:41 +09:00
Vinod Kumar Vavilapalli
2b44fcaa53 Adding release 2.7.1 to CHANGES.txt 
(cherry picked from commit 623fd46c1eac057ea9bc4b503a47ad1a831cdacf)
 2015-04-09 14:01:23 -07:00
Junping Du
86e8c9958a MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal to a reducer. Contributed by Jason Lowe. 
(cherry picked from commit eccb7d46efbf07abcc6a01bd5e7d682f6815b824)
 2015-04-02 12:16:22 -07:00
Vinod Kumar Vavilapalli
ec123abdc9 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit 7bfd9e068d205f752c05d51778a4a4702329d31e.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-03-31 11:30:04 -07:00
Vinod Kumar Vavilapalli
1c050bd135 Revert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)" 
This reverts commit f9341c1e2cffc4e3ac37cb7da4e535ca5c6d3913.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-03-31 11:28:51 -07:00
Vinod Kumar Vavilapalli
02646ddba2 Revert "MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu." 
This reverts commit 929b04ce3a4fe419dece49ed68d4f6228be214c1.
 2015-03-31 11:26:13 -07:00
Vinod Kumar Vavilapalli
96d04307ff Revert "MAPREDUCE-6288. Changed permissions on JobHistory server's done directory so that user's client can load the conf files directly. Contributed by Robert Kanter." 
This reverts commit 5358b83167777a7108b32c9900fb0d01ca0fe961.

(cherry picked from commit cc0a01c50308fe599b4bdd5a80068d6244062e4f)
 2015-03-30 14:55:17 -07:00
Vinod Kumar Vavilapalli
fa30500d72 MAPREDUCE-6288. Changed permissions on JobHistory server's done directory so that user's client can load the conf files directly. Contributed by Robert Kanter. 
(cherry picked from commit 5358b83167777a7108b32c9900fb0d01ca0fe961)
 2015-03-30 10:29:05 -07:00
Akira Ajisaka
da6228e320 MAPREDUCE-6292. Use org.junit package instead of junit.framework in TestCombineFileInputFormat. (aajisaka) 
(cherry picked from commit c770df49b4c9d5ac5ddaa4ff2ec541e4bd1a8e56)
(cherry picked from commit 2e509908eed4cf2e365ca1fd94aa2ff3d59193c1)
 2015-03-25 19:02:44 +09:00
Tsuyoshi Ozawa
01460c0bc9 MAPREDUCE-6285. ClientServiceDelegate should not retry upon AuthenticationException. Contributed by Jonathan Eagles. 
(cherry picked from commit 4170c99147b0cb6d561ff626cea140e0a061b314)
 2015-03-25 00:57:26 +09:00