Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,874 Commits 326 Branches 371 Tags
Commit Graph

1431 Commits

Author SHA1 Message Date
Chris Douglas
4f301f92c1 MAPREDUCE-2094. LineRecordReader should not seek into non-splittable, compressed streams. 2015-05-08 14:30:56 -07:00
Jason Lowe
dfc4a62f7d MAPREDUCE-3383. Duplicate job.getOutputValueGroupingComparator() in ReduceTask. Contributed by Binglin Chang 
(cherry picked from commit c39012f4a0444f9e4b7d67957d5192127d143d90)
 2015-05-08 21:17:05 +00:00
Devaraj K
a957baeb71 MAPREDUCE-5981. Log levels of certain MR logs can be changed to DEBUG. 
Contributed by Varun Saxena.

(cherry picked from commit dc2b2ae31f2eb6dae324c2e14ed7660ce605a89b)
 2015-05-08 21:28:52 +05:30
Devaraj K
ade5165ecf MAPREDUCE-6342. Make POM project names consistent. Contributed by Rohith. 2015-05-08 15:20:35 +05:30
Akira Ajisaka
5cc036d473 MAPREDUCE-6079. Rename JobImpl#username to reporterUserName. Contributed by Tsuyoshi Ozawa. 
(cherry picked from commit ecfa05227489724f12775b94a9f9184b66e865af)
 2015-05-08 18:17:35 +09:00
Tsuyoshi Ozawa
1ae8bf5b21 MAPREDUCE-6284. Add Task Attempt State API to MapReduce Application Master REST API. Contributed by Ryu Kobayashi. 
(cherry picked from commit d18f10ad1b3e497fa1aaaeb85ba055f87d9849f7)
 2015-05-08 16:00:41 +09:00
Jason Lowe
976a3c1f9d MAPREDUCE-6279. AM should explicity exit JVM after all services have stopped. Contributed by Eric Payne 
(cherry picked from commit f30065c8b6099372f57015b505434120fe83c2b0)
 2015-05-07 22:06:54 +00:00
Akira Ajisaka
f7c011cbaa MAPREDUCE-6356. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d335071398fdb14153c854b740814b4fd385b658)
 2015-05-07 19:41:44 +09:00
Robert Kanter
bd207b6cc0 MAPREDUCE-6192. Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) 2015-05-05 14:47:52 -07:00
Xuan
02cdcaec58 YARN-2123. Progress bars in Web UI always at 100% due to non-US locale. 
Contributed by Akira AJISAKA

(cherry picked from commit b7dd3a4f04f712b7594c4e6e7ce50fd314f7c342)
 2015-05-05 10:45:18 -07:00
Tsuyoshi Ozawa
4e96175b33 MAPREDUCE-6165. [JDK8] TestCombineFileInputFormat failed on JDK8. Contributed by Akira AJISAKA. 
(cherry picked from commit 551615fa13f65ae996bae9c1bacff189539b6557)
 2015-05-05 10:23:25 +09:00
Jason Lowe
81f128f293 MAPREDUCE-6259. IllegalArgumentException due to missing job submit time. Contributed by zhihai xu 
(cherry picked from commit bf70c5ae2824a9139c1aa9d7c14020018881cec2)
 2015-05-04 20:40:16 +00:00
Jason Lowe
87c2d915f1 MAPREDUCE-5649. Reduce cannot use more than 2G memory for the final merge. Contributed by Gera Shegalov 
(cherry picked from commit 7dc3c1203d1ab14c09d0aaf0869a5bcdfafb0a5a)
 2015-05-04 19:03:43 +00:00
Tsuyoshi Ozawa
7e03bda3e7 MAPREDUCE-6349. Fix typo in property org.apache.hadoop.mapreduce.lib.chain.Chain.REDUCER_INPUT_VALUE_CLASS. Contributed by Ray Chiang. 
(cherry picked from commit bb6ef2984d8f117711b806c4ebdc757bd182c06e)
 2015-05-04 15:36:53 +09:00
Tsuyoshi Ozawa
1035f550b9 MAPREDUCE-5905. CountersStrings.toEscapedCompactStrings outputs unnecessary null strings. Contributed by Akira AJISAKA. 
(cherry picked from commit 3ba18362f2a4b83635b89aa0adc5ebaf27d9ca83)
 2015-05-04 15:02:40 +09:00
Gera Shegalov
ff6492d603 MAPREDUCE-6345. Documentation fix for when CRLA is enabled for MRAppMaster logs. (Rohit Agarwal via gera) 
(cherry picked from commit f1a152cc0adc071277c80637ea6f5faa0bf06a1a)
 2015-05-01 19:07:14 -07:00
Devaraj K
c3d083a9ff MAPREDUCE-6339. Job history file is not flushed correctly because 
isTimerActive flag is not set true when flushTimerTask is scheduled.
Contributed by zhihai xu.

(cherry picked from commit f5b38477f9d0827b238fadd260c1dd2889531fd4)
 2015-04-30 12:36:00 +05:30
Jason Lowe
ea828578a3 MAPREDUCE-6334. Fetcher#copyMapOutput is leaking usedMemory upon IOException during InMemoryMapOutput shuffle handler. Contributed by Eric Payne 
(cherry picked from commit bc1bd7e5c4047b374420683d36a8c30eda6d75b6)
 2015-04-28 20:19:52 +00:00
Vinod Kumar Vavilapalli
32dc13d907 MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM tokens when they roll-over. Contributed by Jason Lowe. 
(cherry picked from commit 9fc32c5c4d1d5f50c605bdb0e3b13f44c86660c8)
 2015-04-27 14:59:57 -07:00
Tsuyoshi Ozawa
fa9def5a2a MAPREDUCE-6341. Fix typo in mapreduce tutorial. Contributed by John Michael Luy. 
(cherry picked from commit 9a3dda3d345b94cf6f9062c00395500596829cf1)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/MapReduceTutorial.md
 2015-04-28 01:58:12 +09:00
Devaraj K
84ae26573c MAPREDUCE-6252. JobHistoryServer should not fail when encountering a 
missing directory. Contributed by Craig Welch.

(cherry picked from commit 5e67c4d384193b38a85655c8f93193596821faa5)
 2015-04-27 15:02:44 +05:30
Gera Shegalov
fa915f73e2 MAPREDUCE-6333. TestEvents,TestAMWebServicesTasks,TestAppController are broken due to MAPREDUCE-6297. (Siqi Li via gera) 
(cherry picked from commit 78c6b462412bbadad4a1a13ed4c597927b0cf188)
 2015-04-24 17:38:16 -07:00
Vinod Kumar Vavilapalli
0ec6e7e7f5 YARN-3413. Changed Nodelabel attributes (like exclusivity) to be settable only via addToClusterNodeLabels but not changeable at runtime. (Wangda Tan via vinodkv) 
(cherry picked from commit f5fe35e297ed4a00a1ba93d090207ef67cebcc9d)
 2015-04-23 11:25:30 -07:00
Tsuyoshi Ozawa
542d53a268 MAPREDUCE-6330. Fix typo in some Task Attempt API's URL. Contributed by Ryu Kobayashi. 
(cherry picked from commit e33594dd87f83953b59b7416c82992de760eb4cd)
 2015-04-23 03:22:02 +09:00
Gera Shegalov
6f06e3b7ea MAPREDUCE-6297. Task Id of the failed task in diagnostics should link to the task page. (Siqi Li via gera) 
(cherry picked from commit 89ded89e86e5d9a634d92a5d8a7c889744d97f94)
 2015-04-21 14:21:57 -07:00
Gera Shegalov
6f5dc9f5af MAPREDUCE-6293. Set job classloader on uber-job's LocalContainerLauncher event thread. (Sangjin Lee via gera) 
(cherry picked from commit 725eb52ddc647074f0bf1cc73c3029f1352f51d5)
 2015-04-21 14:21:56 -07:00
Robert Kanter
756c254293 MAPREDUCE-6238. MR2 can't run local jobs with -libjars command options which is a regression from MR1 (zxu via rkanter) 
(cherry picked from commit d50e8f09287deeb51012d08e326a2ed71a6da869)
 2015-04-20 14:14:49 -07:00
Jian He
42cf8e0554 YARN-3021. YARN's delegation-token handling disallows certain trust setups to operate properly over DistCp. Contributed by Yongjun Zhang 
(cherry picked from commit bb6dde68f19be1885a9e7f7949316a03825b6f3e)
 2015-04-16 19:44:32 -07:00
Akira Ajisaka
51c458a5c4 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 1fa8075794894b3e0053d251788277fb88e85533)
 2015-04-16 10:06:03 -05:00
Akira Ajisaka
4801b0b478 Revert "MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li." 
This reverts commit 3de0bf9a35094f3c2ac216992d861729251b6a3d.
 2015-04-11 12:37:44 +09:00
Jian He
d3daf9665c YARN-3348. Add a 'yarn top' tool to help understand cluster usage. Contributed by Varun Vasudev 2015-04-10 10:40:31 -07:00
Akira Ajisaka
3de0bf9a35 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 4cec9975c0e2c2fcca090888dc2c5e149c5be1e9)
 2015-04-11 01:31:44 +09:00
Akira Ajisaka
f603b9ccca MAPREDUCE-6307. Remove property mapreduce.tasktracker.taskmemorymanager.monitoringinterval. Contributed by J.Andreina. 
(cherry picked from commit 83979e61a49daa67a05eed1ba64c24d31a41610a)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-04-10 20:23:46 +09:00
Harsh J
b817f0924d MAPREDUCE-6314. TestPipeApplication fails on trunk. Contributed by Varun Vasudev. 
(cherry picked from commit c91b2de68ad8137a909641fdec498f2024af4476)
 2015-04-10 14:03:37 +05:30
Robert Kanter
0fdd5c23c3 MAPREDUCE-6266. Job#getTrackingURL should consistently return a proper URL (rchiang via rkanter) 
(cherry picked from commit c4986b2d00d327f18d0c7e2f8805b69a4e07a19b)
 2015-04-09 13:49:03 -07:00
Hitesh Shah
55b794e7fa YARN-2890. MiniYarnCluster should turn on timeline service if configured to do so. Contributed by Mit Desai. 
(cherry picked from commit 265ed1fe804743601a8b62cabc1e4dc2ec8e502f)
 2015-04-08 14:14:06 -07:00
Gera Shegalov
3757e731e6 MAPREDUCE-4844. Counters / AbstractCounters have constant references not declared final. (Brahma Reddy Battula via gera) 
(cherry picked from commit a0795c1e81685b812288cf2f4df6f8d23da30ab8)
 2015-04-03 10:35:18 -07:00
Junping Du
cacadea632 MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal to a reducer. Contributed by Jason Lowe. 
(cherry picked from commit eccb7d46efbf07abcc6a01bd5e7d682f6815b824)
 2015-04-02 12:15:12 -07:00
Vinod Kumar Vavilapalli
defae1d399 Revert "MAPREDUCE-6288. Changed permissions on JobHistory server's done directory so that user's client can load the conf files directly. Contributed by Robert Kanter." 
This reverts commit 5358b83167777a7108b32c9900fb0d01ca0fe961.

(cherry picked from commit cc0a01c50308fe599b4bdd5a80068d6244062e4f)
 2015-03-30 14:52:36 -07:00
Vinod Kumar Vavilapalli
cc130a033a MAPREDUCE-6288. Changed permissions on JobHistory server's done directory so that user's client can load the conf files directly. Contributed by Robert Kanter. 
(cherry picked from commit 5358b83167777a7108b32c9900fb0d01ca0fe961)
 2015-03-30 10:28:28 -07:00
Vinod Kumar Vavilapalli
c5bc48946d YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla. 
(cherry picked from commit c358368f511963ad8e35f030b9babee541e1bd01)
 2015-03-30 10:11:12 -07:00
Harsh J
274db918c3 MAPREDUCE-6291. Correct mapred queue usage command. Contributed by Brahma Reddy Battula. 
(cherry picked from commit f5432cb47dfacaa57faed32a29d00bbeff8a5385)
 2015-03-28 11:58:44 +05:30
Akira Ajisaka
853cc04668 MAPREDUCE-5762. Port MAPREDUCE-3223 (Remove MRv1 config from mapred-default.xml) to branch-2. (aajisaka) 2015-03-28 03:35:03 +09:00
Tsuyoshi Ozawa
f0aa2a7466 MAPREDUCE-6294. Remove an extra parameter described in Javadoc of TockenCache. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 05499b1093ea6ba6a39a1354d67b0a46a2982824)
 2015-03-28 00:08:56 +09:00
Akira Ajisaka
2e509908ee MAPREDUCE-6292. Use org.junit package instead of junit.framework in TestCombineFileInputFormat. (aajisaka) 
(cherry picked from commit c770df49b4c9d5ac5ddaa4ff2ec541e4bd1a8e56)
 2015-03-25 19:01:38 +09:00
Tsuyoshi Ozawa
94eb8e5b3d MAPREDUCE-6285. ClientServiceDelegate should not retry upon AuthenticationException. Contributed by Jonathan Eagles. 
(cherry picked from commit 4170c99147b0cb6d561ff626cea140e0a061b314)
 2015-03-25 00:57:08 +09:00
Devaraj K
943d9ee603 MAPREDUCE-6242. Progress report log is incredibly excessive in application 
master. Contributed by Varun Saxena.
 2015-03-23 22:48:00 +05:30
Harsh J
6e6e0e4a37 MAPREDUCE-5448. Addendum fix to remove deprecation warning by junit.Assert import in TestFileOutputCommitter. 
(cherry picked from commit 4cd54d9a297435150ab61803284eb05603f114e2)
 2015-03-22 10:33:29 +05:30
Harsh J
d43c0da7d7 MAPREDUCE-5190. Unnecessary condition test in RandomSampler. Contributed by Jingguo Yao. 
(cherry picked from commit 1d5c796d654c8959972d15cc6742731a99380bfc)
 2015-03-22 10:04:19 +05:30
Harsh J
a7f1c1b993 MAPREDUCE-5448. MapFileOutputFormat#getReaders bug with invisible files/folders. Contributed by Maysam Yabandeh. 
(cherry picked from commit b46c2bb51ae524e6640756620f70e5925cda7592)
 2015-03-22 09:46:54 +05:30