Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,063 Commits 326 Branches 371 Tags
Commit Graph

2125 Commits

Author SHA1 Message Date
Devaraj K
b9865cbf3d MAPREDUCE-6273. HistoryFileManager should check whether summaryFile exists 
to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED
state. Contributed by zhihai xu.

(cherry picked from commit 1ea90144d58443a7431ff33ba49ee19278ebe42b)
 2015-05-15 12:34:08 +05:30
Vinod Kumar Vavilapalli
87862970f1 Fixing MR intermediate spills. Contributed by Arun Suresh. 
(cherry picked from commit 6b710a42e00acca405e085724c89cda016cf7442)
 2015-05-14 16:56:56 -07:00
Devaraj K
454236ec19 MAPREDUCE-5708. Duplicate String.format in 
YarnOutputFiles.getSpillFileForWrite. Contributed by Konstantin Weitz.

(cherry picked from commit 05ff54c66c49301c4ec2549704d9d459e784572c)
 2015-05-14 22:11:15 +05:30
Xuan
487d9b0f3f YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch 
(cherry picked from commit 0f95921447ea547bdf9caf18f7fde46bc66031f8)
 2015-05-13 13:12:04 -07:00
cnauroth
ddc4afeff2 HADOOP-11966. Variable cygwin is undefined in hadoop-config.sh when executed through hadoop-daemon.sh. Contributed by Chris Nauroth. 2015-05-13 12:25:49 -07:00
Tsuyoshi Ozawa
780d4f416e MAPREDUCE-6366. mapreduce.terasort.final.sync configuration in TeraSort doesn't work. Contributed by Takuya Fukudome. 
(cherry picked from commit e82067bfe680ce04acc0153693cce3cd385e5567)
 2015-05-13 16:45:46 +09:00
Akira Ajisaka
345e62c6d5 Revert "MAPREDUCE-5762. Port MAPREDUCE-3223 (Remove MRv1 config from mapred-default.xml) to branch-2. (aajisaka)" 
This reverts commit 853cc0466811c310486fff6042d7b76ab3b78947.
 2015-05-13 14:02:41 +09:00
Vinod Kumar Vavilapalli
d5755ba241 MAPREDUCE-6251. Added a new config for JobClient to retry JobStatus calls so that they don't fail on history-server backed by DFSes with not so strong guarantees. Contributed by Craig Welch. 
(cherry picked from commit f24452d14e9ba48cdb82e5e6e5c10ce5b1407308)
 2015-05-12 12:13:23 -07:00
Tsuyoshi Ozawa
fb5b0ebb45 MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. 
(cherry picked from commit f4e2b3cc0b1f4e49c306bc09a9dddd0495225bb2)
 2015-05-13 00:28:36 +09:00
Vinayakumar B
cda1962996 MAPREDUCE-6360. TestMapreduceConfigFields is placed in wrong dir, introducing compile error (Contributed by Arshad Mohammad) 
(cherry picked from commit 360dff5903085c3c7f02ccf9f17d71a842275e48)
 2015-05-12 12:50:44 +05:30
Jason Lowe
53e9973afc MAPREDUCE-5465. Tasks are often killed before they exit on their own. Contributed by Ming Ma 2015-05-11 22:40:05 +00:00
Junping Du
a1a7deebf8 YARN-3587. Fix the javadoc of DelegationTokenSecretManager in yarn, etc. projects. Contributed by Gabor Liptak. 
(cherry picked from commit 7e543c27fa2881aa65967be384a6203bd5b2304f)
 2015-05-11 05:28:25 -07:00
Karthik Kambatla
52469b10dd MAPREDUCE-6353. Divide by zero error in MR AM when calculating available containers. (Anubhav Dhoot via kasha) 
(cherry picked from commit 1773aac780585871072960a5863af461e112a030)
 2015-05-09 14:43:57 -07:00
Karthik Kambatla
e8dd0bc4fd YARN-1287. Consolidate MockClocks. (Sebastian Wong and Anubhav Dhoot via kasha) 
(cherry picked from commit 70fb37cd79a7eba6818313960624380bacfe0bb2)
 2015-05-09 14:35:13 -07:00
Junping Du
39c188449e MAPREDUCE-6359. In RM HA setup, Cluster tab links populated with AM hostname instead of RM. Contributed by zhaoyunjiong. 
(cherry picked from commit df36ad0a08261b03c250b6f745b27e5f83e4286e)
 2015-05-09 06:12:01 -07:00
Ravi Prakash
23065e6051 MAPREDUCE-4750. Enable NNBenchWithoutMR in MapredTestDriver (Liang Xie and Jason Lowe via raviprak) 
(cherry picked from commit 5aab014340b53ebc9363ee244b2cbea7a4c1f573)
 2015-05-08 15:58:29 -07:00
Jason Lowe
0f6c971282 MAPREDUCE-5248. Let NNBenchWithoutMR specify the replication factor for its test. Contributed by Erik Paulson 
(cherry picked from commit 30099a36c6b0f658d25fb505a9f3ce15d19f7ba6)
 2015-05-08 22:30:21 +00:00
Chris Douglas
4f301f92c1 MAPREDUCE-2094. LineRecordReader should not seek into non-splittable, compressed streams. 2015-05-08 14:30:56 -07:00
Jason Lowe
dfc4a62f7d MAPREDUCE-3383. Duplicate job.getOutputValueGroupingComparator() in ReduceTask. Contributed by Binglin Chang 
(cherry picked from commit c39012f4a0444f9e4b7d67957d5192127d143d90)
 2015-05-08 21:17:05 +00:00
Devaraj K
a957baeb71 MAPREDUCE-5981. Log levels of certain MR logs can be changed to DEBUG. 
Contributed by Varun Saxena.

(cherry picked from commit dc2b2ae31f2eb6dae324c2e14ed7660ce605a89b)
 2015-05-08 21:28:52 +05:30
Devaraj K
ade5165ecf MAPREDUCE-6342. Make POM project names consistent. Contributed by Rohith. 2015-05-08 15:20:35 +05:30
Akira Ajisaka
5cc036d473 MAPREDUCE-6079. Rename JobImpl#username to reporterUserName. Contributed by Tsuyoshi Ozawa. 
(cherry picked from commit ecfa05227489724f12775b94a9f9184b66e865af)
 2015-05-08 18:17:35 +09:00
Tsuyoshi Ozawa
1ae8bf5b21 MAPREDUCE-6284. Add Task Attempt State API to MapReduce Application Master REST API. Contributed by Ryu Kobayashi. 
(cherry picked from commit d18f10ad1b3e497fa1aaaeb85ba055f87d9849f7)
 2015-05-08 16:00:41 +09:00
Jason Lowe
976a3c1f9d MAPREDUCE-6279. AM should explicity exit JVM after all services have stopped. Contributed by Eric Payne 
(cherry picked from commit f30065c8b6099372f57015b505434120fe83c2b0)
 2015-05-07 22:06:54 +00:00
Akira Ajisaka
f7c011cbaa MAPREDUCE-6356. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d335071398fdb14153c854b740814b4fd385b658)
 2015-05-07 19:41:44 +09:00
Robert Kanter
bd207b6cc0 MAPREDUCE-6192. Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) 2015-05-05 14:47:52 -07:00
Xuan
02cdcaec58 YARN-2123. Progress bars in Web UI always at 100% due to non-US locale. 
Contributed by Akira AJISAKA

(cherry picked from commit b7dd3a4f04f712b7594c4e6e7ce50fd314f7c342)
 2015-05-05 10:45:18 -07:00
Tsuyoshi Ozawa
4e96175b33 MAPREDUCE-6165. [JDK8] TestCombineFileInputFormat failed on JDK8. Contributed by Akira AJISAKA. 
(cherry picked from commit 551615fa13f65ae996bae9c1bacff189539b6557)
 2015-05-05 10:23:25 +09:00
Jason Lowe
81f128f293 MAPREDUCE-6259. IllegalArgumentException due to missing job submit time. Contributed by zhihai xu 
(cherry picked from commit bf70c5ae2824a9139c1aa9d7c14020018881cec2)
 2015-05-04 20:40:16 +00:00
Jason Lowe
87c2d915f1 MAPREDUCE-5649. Reduce cannot use more than 2G memory for the final merge. Contributed by Gera Shegalov 
(cherry picked from commit 7dc3c1203d1ab14c09d0aaf0869a5bcdfafb0a5a)
 2015-05-04 19:03:43 +00:00
Tsuyoshi Ozawa
7e03bda3e7 MAPREDUCE-6349. Fix typo in property org.apache.hadoop.mapreduce.lib.chain.Chain.REDUCER_INPUT_VALUE_CLASS. Contributed by Ray Chiang. 
(cherry picked from commit bb6ef2984d8f117711b806c4ebdc757bd182c06e)
 2015-05-04 15:36:53 +09:00
Tsuyoshi Ozawa
1035f550b9 MAPREDUCE-5905. CountersStrings.toEscapedCompactStrings outputs unnecessary null strings. Contributed by Akira AJISAKA. 
(cherry picked from commit 3ba18362f2a4b83635b89aa0adc5ebaf27d9ca83)
 2015-05-04 15:02:40 +09:00
Gera Shegalov
ff6492d603 MAPREDUCE-6345. Documentation fix for when CRLA is enabled for MRAppMaster logs. (Rohit Agarwal via gera) 
(cherry picked from commit f1a152cc0adc071277c80637ea6f5faa0bf06a1a)
 2015-05-01 19:07:14 -07:00
Devaraj K
c3d083a9ff MAPREDUCE-6339. Job history file is not flushed correctly because 
isTimerActive flag is not set true when flushTimerTask is scheduled.
Contributed by zhihai xu.

(cherry picked from commit f5b38477f9d0827b238fadd260c1dd2889531fd4)
 2015-04-30 12:36:00 +05:30
Jason Lowe
ea828578a3 MAPREDUCE-6334. Fetcher#copyMapOutput is leaking usedMemory upon IOException during InMemoryMapOutput shuffle handler. Contributed by Eric Payne 
(cherry picked from commit bc1bd7e5c4047b374420683d36a8c30eda6d75b6)
 2015-04-28 20:19:52 +00:00
Devaraj K
1d03ac3533 Moving MAPREDUCE-6252 to the 2.7.1 CHANGES.txt 
(cherry picked from commit 99fe03e439b0f9afd01754d998c6eb64f0f70300)
 2015-04-28 12:44:07 +05:30
Vinod Kumar Vavilapalli
32dc13d907 MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM tokens when they roll-over. Contributed by Jason Lowe. 
(cherry picked from commit 9fc32c5c4d1d5f50c605bdb0e3b13f44c86660c8)
 2015-04-27 14:59:57 -07:00
Tsuyoshi Ozawa
fa9def5a2a MAPREDUCE-6341. Fix typo in mapreduce tutorial. Contributed by John Michael Luy. 
(cherry picked from commit 9a3dda3d345b94cf6f9062c00395500596829cf1)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/MapReduceTutorial.md
 2015-04-28 01:58:12 +09:00
Devaraj K
84ae26573c MAPREDUCE-6252. JobHistoryServer should not fail when encountering a 
missing directory. Contributed by Craig Welch.

(cherry picked from commit 5e67c4d384193b38a85655c8f93193596821faa5)
 2015-04-27 15:02:44 +05:30
Gera Shegalov
fa915f73e2 MAPREDUCE-6333. TestEvents,TestAMWebServicesTasks,TestAppController are broken due to MAPREDUCE-6297. (Siqi Li via gera) 
(cherry picked from commit 78c6b462412bbadad4a1a13ed4c597927b0cf188)
 2015-04-24 17:38:16 -07:00
Vinod Kumar Vavilapalli
db1a33e956 Moving YARN-3351, YARN-3382, YARN-3472, MAPREDUCE-6238 to the 2.7.1 CHANGES.txt 
sections given the recent merge into branch-2.7.

(cherry picked from commit 2f82ae042a6f3110742aaa57c076bb9ebd7888d1)
 2015-04-24 17:20:02 -07:00
Vinod Kumar Vavilapalli
0ec6e7e7f5 YARN-3413. Changed Nodelabel attributes (like exclusivity) to be settable only via addToClusterNodeLabels but not changeable at runtime. (Wangda Tan via vinodkv) 
(cherry picked from commit f5fe35e297ed4a00a1ba93d090207ef67cebcc9d)
 2015-04-23 11:25:30 -07:00
Tsuyoshi Ozawa
a479669e96 Fix a misleading summary of MAPREDUCE-6330 in CHANGES.txt. 
(cherry picked from commit 252e69f562383383cf539b29f1da68676b7ce0c0)
 2015-04-23 03:28:46 +09:00
Tsuyoshi Ozawa
542d53a268 MAPREDUCE-6330. Fix typo in some Task Attempt API's URL. Contributed by Ryu Kobayashi. 
(cherry picked from commit e33594dd87f83953b59b7416c82992de760eb4cd)
 2015-04-23 03:22:02 +09:00
Gera Shegalov
6f06e3b7ea MAPREDUCE-6297. Task Id of the failed task in diagnostics should link to the task page. (Siqi Li via gera) 
(cherry picked from commit 89ded89e86e5d9a634d92a5d8a7c889744d97f94)
 2015-04-21 14:21:57 -07:00
Gera Shegalov
6f5dc9f5af MAPREDUCE-6293. Set job classloader on uber-job's LocalContainerLauncher event thread. (Sangjin Lee via gera) 
(cherry picked from commit 725eb52ddc647074f0bf1cc73c3029f1352f51d5)
 2015-04-21 14:21:56 -07:00
Vinod Kumar Vavilapalli
27fc4fb998 Set the release date for 2.7.0 
(cherry picked from commit d52de61544060a04a273114f55979e7e929a576c)
 2015-04-20 20:18:01 -07:00
Robert Kanter
756c254293 MAPREDUCE-6238. MR2 can't run local jobs with -libjars command options which is a regression from MR1 (zxu via rkanter) 
(cherry picked from commit d50e8f09287deeb51012d08e326a2ed71a6da869)
 2015-04-20 14:14:49 -07:00
Jian He
42cf8e0554 YARN-3021. YARN's delegation-token handling disallows certain trust setups to operate properly over DistCp. Contributed by Yongjun Zhang 
(cherry picked from commit bb6dde68f19be1885a9e7f7949316a03825b6f3e)
 2015-04-16 19:44:32 -07:00
Akira Ajisaka
51c458a5c4 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 1fa8075794894b3e0053d251788277fb88e85533)
 2015-04-16 10:06:03 -05:00