Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,690 Commits 326 Branches 371 Tags
Commit Graph

2782 Commits

Author SHA1 Message Date
Kihwal Lee
0c35e44184 MAPREDUCE-7069. Add ability to specify user environment variables individually. Contributed by Jim Brennan 
(cherry picked from commit 4571351cccf6d4977469d3d623cf045b06a5f5f0)
 2020-08-13 19:02:03 -05:00
Wanqiang Ji
f3ac1dde5e
MAPREDUCE-7288. Fix TestLongLong#testRightShift (#2183) 
(cherry picked from commit dc5470ae86b96d6f84abd7354c3c056d3627871b)
 2020-08-06 09:37:28 +09:00
ywheel
2dfc6bf471
MAPREDUCE-7051. Fix typo in MultipleOutputFormat (#338) 
(cherry picked from commit cf4eb75608527a8a56ad9d13a3009ccc38b29c8e)
 2020-07-30 13:31:18 +09:00
Ayush Saxena
4592af898b HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. Contributed by Ahmed Hussein. 2020-07-22 19:05:13 +05:30
Ahmed Hussein
34a6cbb5bb HADOOP-17101. Replace Guava Function with Java8+ Function 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 98fcffe93f9ef910654574f69591fcdc621523af)
(cherry picked from commit 43a865dc07437e0f767a902c4cfe6ca0472bf57f)
 2020-07-15 11:01:34 -05:00
Eric Badger
0799901d48 Revert "MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles)." 
This reverts commit 7e05577c4755802e24d14d80282edfdbe14304ab.
 2020-06-08 20:12:00 +00:00
Wilfred Spiegelenburg
0e0d88ca97
MAPREDUCE-7278. Speculative execution behavior is observed even when mapreduce.map.speculative and mapreduce.reduce.speculative are false 
Contributed by Tarun Parimi.

(cherry picked from commit 10db97df1c8562a9e29b00e60d5bde1773c09188)
 2020-05-28 16:38:34 +10:00
Gabor Bota
ec6d2a8402 Preparing for 3.1.5 development 
Change-Id: Iabc64aba7392e3b6f9e4e18109fcaa2cfc01d1f9
 2020-04-29 11:18:18 +02:00
Eric E Payne
7e05577c47 MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles). 
(cherry picked from commit e2322e1117a2a3435aabecd49de0ad3c8d2f64b9)
 2020-04-27 20:12:27 +00:00
Eric E Payne
8bcd7cfc0b MAPREDUCE-7272. TaskAttemptListenerImpl excessive log messages. Contributed by Ahmed Hussein (ahussein) 
(cherry picked from commit 11d17417ceba0f1a2944e0c8b4286515c4883889)
 2020-04-13 19:22:56 +00:00
Eric E Payne
d4ecb65629 MAPREDUCE-7079: JobHistory#ServiceStop implementation is incorrect. Contributed by Ahmed Hussein (ahussein) 
(cherry picked from commit b897f6834ba69d443c3acd1fab52261c00d675a1)
 2020-01-29 17:21:40 +00:00
Ahmed Hussein
f3ff61865a MAPREDUCE-7259. testSpeculateSuccessfulWithUpdateEvents fails Intermittently 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 08251538fe2550d9dd86f9daf79994f5b8bdf7fa)
 2020-01-28 11:04:25 -06:00
Ahmed Hussein
1f315c28d9 MAPREDUCE-7262. MRApp helpers block for long intervals (500ms) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 3f01c481060585ccd37be9db8aa4d1e33d2e2d6b)
 2020-01-27 16:23:06 -06:00
kevinzhao1661
ad34d7032e
MAPREDUCE-7247. Modify HistoryServerRest.html content,change The job attempt id‘s datatype from string to int (#1684) 
(cherry picked from commit b8518e6dabc1cf73fa7c43b3f8904d2e2beb2bf3)
 2020-01-16 12:58:30 +09:00
Masatake Iwasaki
ce52a66fcb
MAPREDUCE-7256. Fix javadoc error in SimpleExponentialSmoothing. (#1804) 
(cherry picked from commit a2fdd7c2b59463c39a4d4250f908a9e2d3653ead)
 2020-01-15 12:12:27 +09:00
Eric E Payne
ec40c1f400 YARN-9018. Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path. Contributed by Kuhu Shukla (kshukla) 
(cherry picked from commit 93233a7d6e4d6b8098622a1aa830355cc18d9589)
 2020-01-09 17:41:10 +00:00
Ahmed Hussein
6ad2f662eb MAPREDUCE-7252. Handling 0 progress in SimpleExponential task runtime estimator 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit cdd6efd3ab6917e30b4c5c7b261f61838901bb37)
 2020-01-08 11:14:40 -06:00
Sergey Pogorelov
d55d02d25a
MAPREDUCE-7255. Fix typo in MapReduce documentaion example (#1793) 
(cherry picked from commit b343e1533b1f4ebbbf10b063c35d79d53a741321)
 2020-01-06 12:41:03 +09:00
prabhujoseph
a97f7776bd MAPREDUCE-7249. Fix Invalid event TA_TOO_MANY_FETCH_FAILURE at SUCCESS_CONTAINER_CLEANUP causes job failure. 
Contributed by Wilfred Spiegelenburg.
 2019-11-28 14:34:52 +05:30
prabhujoseph
6a4966f336 MAPREDUCE-7240. Fix Invalid event: TA_TOO_MANY_FETCH_FAILURE at SUCCESS_FINISHING_CONTAINER. 
Contributed by Huachao and Peter Bacsko. Reviewed by Wilfred Spiegelenburg.
 2019-11-27 16:51:47 +05:30
Ahmed Hussein
cbd501636c MAPREDUCE-7208. Tuning TaskRuntimeEstimator. (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit ed302f1fed6d124d682486d24dae958946dba9be)
 2019-11-05 15:29:19 -06:00
Jonathan Hung
4d274f60bc Make upstream aware of 2.10.0 release 
(cherry picked from commit 7663db59c097c82eeed2df7a91168a4d7123c96b)
(cherry picked from commit 5d2ffcc7aac608746946dc2ed5711549a01a2055)
 2019-10-30 21:00:55 -07:00
Szilard Nemeth
19755b9b36 MAPREDUCE-6441. Improve temporary directory name generation in LocalDistributedCacheManager for concurrent processes (wattsinabox, rchiang, haibochen via snemeth) 2019-10-18 15:25:50 +02:00
Jonathan Hung
0dc85bc182 YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena 2019-10-08 11:18:52 -07:00
Tao Yang
3cc21350b5 YARN-9817. Fix failing testcases due to not initialized AsyncDispatcher - ArithmeticException: / by zero. Contributed by Prabhu Joseph. 2019-09-07 07:40:50 +08:00
Zhankun Tang
ef79d98788 Preparing for 3.1.4 development 2019-09-04 16:11:36 +08:00
Szilard Nemeth
a3fb032010 MAPREDUCE-7230. TestHSWebApp.testLogsViewSingle fails. Contributed by Prabhu Joseph 
(cherry picked from commit 3f4f097f1f29249c8ad5d59ccefcddf0a66b2e6b)
 2019-08-15 17:08:53 +02:00
Szilard Nemeth
639763c561 MAPREDUCE-7225: Fix broken current folder expansion during MR job start. Contributed by Peter Bacsko. 2019-08-01 16:08:14 +02:00
pingsutw
1a2aba85d8
MAPREDUCE-7076. TestNNBench#testNNBenchCreateReadAndDelete failing in our internal build 
This closes #1089

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ee87e9a42e4ff1f27a6f1e5b7c7de97f8989d9b2)
 2019-07-23 10:36:01 +09:00
Sean Mackrory
fee1e67453 HADOOP-16213. Update guava to 27.0-jre. Contributed by Gabor Bota. 2019-06-13 07:38:43 -06:00
Takanobu Asanuma
8098ddaf40 HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:39:49 +09:00
Akira Ajisaka
f8bd5deec1
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:28:21 +09:00
Wanqiang Ji
6690955358
MAPREDUCE-7205. Treat container scheduler kill exit code as a task attempt killing event. 
This closes #821

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 67f9a7b165edecbec7c8063758202be4d8cff0f5)
 2019-05-22 14:41:20 +09:00
Adam Antal
52536ff887 MAPREDUCE-7190. Add SleepJob additional parameter to make parallel runs distinguishable. Contributed by Adam Antal. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 856cbf62d3fd1c885ae8374f08d7c18c83941791)
(cherry picked from commit ddf88596550be13889ff7002aadd0ebf797996a1)
 2019-04-01 10:39:18 -07:00
Weiwei Yang
fdd56b5b05 MAPREDUCE-7191. JobHistoryServer should log exception when loading/parsing history file failed. Contributed by Jiandan Yang. 
(cherry picked from commit f0605146b34ddbd19fc9e52d4761f9a347d5173e)
 2019-03-11 16:40:01 +08:00
Wangda Tan
a1e09b4c0c Make upstream aware of 3.1.2 release 
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
(cherry picked from commit 308f3168fa221a336ef0dbb1a17358b1ad6f9356)
(cherry picked from commit 649da5af04d57ee910dbe7fb0f4a7d9cb4c40eb0)
 2019-02-05 14:09:33 -08:00
Akira Ajisaka
72b3baf207
MAPREDUCE-7177. Disable speculative execution in TestDFSIO. Contributed by Zhaohui Xin. 
(cherry picked from commit c991e2ca5a65eae62bad7e5baf81f1eb633b7ec0)
 2019-02-04 14:04:26 +09:00
Haibo Chen
61a6cc8d23 YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 
(cherry picked from commit bb92bfb4ef96baa234966b60e464d1773fbf3f22)
 2019-01-24 15:58:37 -08:00
Wangda Tan
31ea2f7806 Preparing for 3.1.3 development 
Change-Id: I3c3d3ee47dc4fef239127b4452ff14676fa26e3d
 2019-01-07 10:04:58 -08:00
Weiwei Yang
2a5b2eda7c MAPREDUCE-7174. Mapreduce example wordmedian should handle generic options. Contributed by Fei Hui. 
(cherry picked from commit 9e7196749ee025db7668c648743d1deb72d3ca28)
 2019-01-02 21:56:52 +08:00
Akira Ajisaka
51b7376a8d
MAPREDUCE-7170. Doc typo in PluggableShuffleAndPluggableSort.md. Contributed by Zhaohui Xin. 
(cherry picked from commit e0d75088f24283985a4e4032fa547dad2ebb54db)
 2018-12-13 16:28:18 +09:00
Jason Lowe
6e6f43afd8 MAPREDUCE-7159. FrameworkUploader: ensure proper permissions of generated framework tar.gz if restrictive umask is used. Contributed by Peter Bacsko 
(cherry picked from commit 9886c27c71147a2bcef87b42ff21c2bbe98c8ab4)
 2018-12-06 15:02:00 -06:00
Jason Lowe
e7fa638fe8 MAPREDUCE-7164. FileOutputCommitter does not report progress while merging paths. Contributed by Kuhu Shukla 
(cherry picked from commit 4d8de7ab690ef919b392b12d856482a6a1f2bb3d)
 2018-11-28 16:01:05 -06:00
Akira Ajisaka
5e2f123748
MAPREDUCE-7162. TestEvents#testEvents fails. Contributed by Zhaohui Xin. 
(cherry picked from commit 1aad99a71813660b83628cacfed393d0b3a123cc)
 2018-11-27 20:02:34 +09:00
Wangda Tan
abd9d93a55 MAPREDUCE-7158. Inefficient Flush Logic in JobHistory EventWriter. (Zichen Sun via wangda) 
Change-Id: I99ace87980da03bb35a8012cea7218d602a8817a
(cherry picked from commit 762a56cc64bc07d57f94e253920534b8e049f238)
 2018-11-13 11:36:04 -08:00
Jason Lowe
afbcb51ac1 MAPREDUCE-7156. NullPointerException when reaching max shuffle connections. Contributed by Peter Bacsko 
(cherry picked from commit ba1f9d66d94ed0b85084d7c40c09a87478b3a05a)
 2018-11-06 18:07:57 -06:00
Jason Lowe
7335d940de YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit ab6aa4c7265db5bcbb446c2f779289023d454b81)
 2018-11-06 08:52:29 -06:00
bibinchundatt
4167275e89 MAPREDUCE-7151. RMContainerAllocator#handleJobPriorityChange expects application_priority always. Contributed by Bilwa S T. 
(cherry picked from commit 74a5e683fe65ee624a094de77940a42777ed7a26)
 2018-10-25 00:27:40 +05:30
Xiao Chen
30526f2c5c MAPREDUCE-7132. JobSplitWriter prints unnecessary warnings if EC(RS10,4) is used. Contributed by Peter Bacsko. 
(cherry picked from commit 25f8fcb06476938826cdc92858a61124b18cd98d)
 2018-10-16 10:24:24 -07:00
Todd Lipcon
89d4481020 HADOOP-15550. Avoid static initialization of ObjectMappers 
(cherry picked from commit 7a3c6e9c3cd9ffdc71946fd12f5c3d59718c4939)
 2018-10-09 13:50:00 -05:00