Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,353 Commits 326 Branches 371 Tags
Commit Graph

519 Commits

Author SHA1 Message Date
Jason Lowe
b1d5e73c1f HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka 
(cherry picked from commit 0898ff42e9e5c53f2fce7ccdeb4e1cd7d0f123b3)
 2018-02-16 15:00:10 -06:00
Jason Lowe
a87bccdc3f MAPREDUCE-7052. TestFixedLengthInputFormat#testFormatCompressedIn is flaky. Contributed by Peter Bacsko 
(cherry picked from commit a53d62ab26e170a0338f93e228718da52e9196e4)
 2018-02-15 15:27:46 -06:00
Jason Lowe
d0fc1cd0c8 MAPREDUCE-5124. AM lacks flow control for task events. Contributed by Peter Bacsko 2017-12-01 14:15:25 -06:00
Arun Suresh
71a55bebfb Preparing for 2.10.0 development 2017-10-31 12:51:14 -07:00
Jason Lowe
347997c959 HADOOP-14919. BZip2 drops records when reading data in splits. Contributed by Jason Lowe 
(cherry picked from commit 2fae63aa60c43b62bd908a9499562fe528603185)
 2017-10-31 09:32:45 -05:00
Varun Saxena
56e0873e22 Fix javac, javadoc and checkstyle issues before branch-2 merge. 2017-10-17 23:36:41 +05:30
Sangjin Lee
c8a8e680bd MAPREDUCE-6818. Remove direct reference to TimelineClientImpl. Contributed by Li Lu. 
(cherry picked from commit 6217b87f4a056cf704cef2e073b386b7803415de)
 2017-10-17 23:36:35 +05:30
Sangjin Lee
b8cfb4fcb3 YARN-5792. Adopt the id prefix for YARN, MR, and DS entities. Contributed by Varun Saxena. 
(cherry picked from commit f734977b27a514ce0561638c0a6a17b1ef093026)
 2017-10-17 23:36:35 +05:30
Varun Saxena
0820df611d MAPREDUCE-6731. TestMRTimelineEventHandling#testMRNewTimelineServiceEventHandling fails randomly for concurrent tests. (Sangjin Lee via Varun Saxena). 
(cherry picked from commit d14e729a55c0ca4053c537290392b74f732ce939)
 2017-10-17 23:36:33 +05:30
Sangjin Lee
e212e562ec YARN-5355. Backported YARN-2928 into our branch-2 feature branch. 2017-10-17 23:36:33 +05:30
Chris Trezzo
3e7c06af41 MAPREDUCE-5951. Add support for the YARN Shared Cache. 
(cherry picked from commit e46d5bb962b0c942f993afc505b165b1cd96e51b)
 2017-10-12 11:30:38 -07:00
Robert Kanter
72823b2738 MAPREDUCE-6953. Skip the testcase testJobWithChangePriority if FairScheduler is used (pbacsko via rkanter) 
(cherry picked from commit a323f73bae65483652de290ef819268850803a17)
(cherry picked from commit f5f343a6654647721120380556fdc2e401a39275)
 2017-09-08 13:18:59 -07:00
Konstantin V Shvachko
9a0aac4170 MAPREDUCE-6931. Remove TestDFSIO "Total Throughput" calculation. Contributed by Dennis Huo. 
(cherry picked from commit 3e0e2033cc28d48a7909822416478aff7487bfe6)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
 2017-08-30 14:42:36 -07:00
Yufei Gu
3ef7adc748 MAPREDUCE-6914. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu) 
(cherry picked from commit b8e8241854904e3a92b636ffaa6f4c651e47cea2)
 2017-08-03 11:47:09 -07:00
Akira Ajisaka
338d84ab71
MAPREDUCE-6921. TestUmbilicalProtocolWithJobToken#testJobTokenRpc fails. Contributed by Sonia Garudi. 
(cherry picked from commit ceacadc51e58bb94ad3f3669488515a61e886d88)
 2017-08-01 15:02:20 +09:00
Jason Lowe
67d31a8e7e Revert "MAPREDUCE-6829. Add peak memory usage counter for each task. (Miklos Szegedi via kasha)" 
This reverts commit a91cc63d374a5770b17cd8465080caf7f91da2a8.
 2017-07-31 16:44:13 -05:00
Junping Du
cae33098e6 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit e8a31f2e1c34514fba2f480e8db652f6e2ed65d8.

(cherry picked from commit fbb7d6bcbb887ce52ab1e9d5a1fed67a7f8a4be8)
 2017-07-31 14:15:06 -07:00
Junping Du
bf360da63f Revert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)" 
This reverts commit f9341c1e2cffc4e3ac37cb7da4e535ca5c6d3913.
 2017-07-31 14:13:26 -07:00
Akira Ajisaka
22f237c5d9
HADOOP-14681. Remove MockitoMaker class. Contributed by Andras Bokor. 
(cherry picked from commit cca51e916b7387ea358688e8f8188ead948fbdcc)
 2017-07-25 15:25:36 +09:00
Sunil G
958835a3dc MAPREDUCE-6889. Add Job#close API to shutdown MR client services. Contributed by Rohith Sharma K S. 
(cherry picked from commit fb3b5d33ffb29ee8e1ffbd2eee7a603a5777ebaf)
 2017-07-17 13:36:00 +05:30
Junping Du
f823f9fd78 MAPREDUCE-6246. DBOutputFormat.java appending extra semicolon to query which is incompatible with DB2. Contributed by ramtin and Gergely Novák. 
(cherry picked from commit f484a6ff602d48413556a1d046670e2003c71c2e)
 2017-07-07 13:27:56 -07:00
Naganarasimha
4fd6aad010 MAPREDUCE-6905. HADOOP_JOB_HISTORY_OPTS should be HADOOP_JOB_HISTORYSERVER_OPTS in mapred-config.sh. Contributed by LiXin Ge. 
(cherry picked from commit bf1f59937dd5d860c9ed7fefce203d6a9f645182)
 2017-07-02 16:01:02 +05:30
Brahma Reddy Battula
a5a5bcdb60 MAPREDUCE-6676. NNBench should Throw IOException when rename and delete fails. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 24181f559f008fc7945702c0fad7d1028f75bfc6)
 2017-06-07 16:37:11 +08:00
Sunil G
ec346d6377 YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang. 2017-05-23 07:57:36 +05:30
Robert Kanter
d686560b35 MAPREDUCE-6871. Allow users to specify racks and nodes for strict locality for AMs (rkanter) 
(cherry picked from commit 3721cfe1fbd98c5b6aa46aefdfcf62276c28c4a4)
 2017-04-21 16:12:45 -07:00
Haibo Chen
d4f553d42f MAPREDUCE-6673. Add a test example job that grows in memory usage over time (Karthik Kambatla via Haibo Chen) 
Change-Id: Iccfc8c67c38c526cc61726d87bfcbcf69ac36fea
(cherry picked from commit 25ac44709b4bbed78b607ea48021237b64e01b9f)
 2017-04-14 17:37:37 -07:00
Robert Kanter
a65011a128 YARN-6050. AMs can't be scheduled on racks or nodes (rkanter) 2017-03-28 14:33:08 -07:00
Akira Ajisaka
aa1f944bba
MAPREDUCE-6855. Specify charset when create String in CredentialsTestJob. Contributed by Kai Sasaki. 
(cherry picked from commit 14413989cac9acc1fa6f8d330fac32f772613325)
 2017-03-07 13:12:58 +09:00
Akira Ajisaka
ed55bcee1f
HADOOP-14108. CLI MiniCluster: add an option to specify NameNode HTTP port. Contributed by Takanobu Asanuma. 
(cherry picked from commit 209ecd1a5c056e096a5d804759f88302608d503b)
 2017-03-07 01:36:10 +09:00
Akira Ajisaka
e2fb9538f0 MAPREDUCE-6753. Variable in byte printed directly in mapreduce client. Contributed by Kai Sasaki. 
(cherry picked from commit 19a1fc6373f0dea56e5845f19990d469428489bc)
 2017-03-03 16:13:26 +09:00
Wangda Tan
a82480c7d4 YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda) 2017-02-23 13:59:20 -08:00
Daniel Templeton
4c883f331c MAPREDUCE-6825. YARNRunner#createApplicationSubmissionContext method is longer than 150 lines (Contributed by Gergely Novák via Daniel Templeton) 
(cherry picked from commit 732ee6f0b58a12500198c0d934cc570c7490b520)
 2017-02-22 15:40:12 -08:00
Karthik Kambatla
a91cc63d37 MAPREDUCE-6829. Add peak memory usage counter for each task. (Miklos Szegedi via kasha) 
(cherry picked from commit c65f884fc7e08118524f8c88737119d8196b4c1b)
 2017-01-26 11:08:53 -08:00
Jason Lowe
94b326f2c8 YARN-5910. Support for multi-cluster delegation tokens. Contributed by Jian He 
(cherry picked from commit 69fa81679f59378fd19a2c65db8019393d7c05a2)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java
 2017-01-23 17:23:32 +00:00
Arun Suresh
4985217de4 YARN-6066. Opportunistic containers Minor fixes : API annotations, parameter name changes, checkstyles. (asuresh) 2017-01-08 08:27:10 -08:00
Arun Suresh
2b4d3e8506 YARN-5646. Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 2273a74c1f3895163046cca09ff5e983df301d22)
(cherry picked from commit 9e17ffe599e12c59c48f4355de645c31a4735d04)
 2017-01-06 11:15:10 -08:00
Arun Suresh
368565f0ce YARN-4597. Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle. (asuresh) 
(cherry picked from commit 3219b7b4ac7d12aee343f6ab2980b3357fc618b6)
(cherry picked from commit 49921678858a17ca58ca2ae704ced1ff584e0859)
 2017-01-06 11:15:09 -08:00
Jason Lowe
937f739804 MAPREDUCE-6682. TestMRCJCFileOutputCommitter fails intermittently Contributed by Akira Ajisaka. 
(cherry picked from commit 8f1c374bec2451568f463ab68f7eb6db734ab14e)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java
 2017-01-06 18:46:28 +00:00
Brahma Reddy Battula
d6e5e4288c HADOOP-13883. Addendum patch to correct the message and alphabetized with the earlier patch. Contributed by Yiqun Lin. 
(cherry picked from commit 6938b677111a5fa6bf654717b310220f98b7cf20)
 2016-12-31 16:17:15 +05:30
Varun Saxena
bcfa6d5384 MAPREDUCE-6811. TestPipeApplication#testSubmitter fails after HADOOP-13802 (Brahma Reddy Battula via Varun Saxena) 
(cherry picked from commit b8690a9d25cb212d53a92bfa1f76a33b8c74413c)
 2016-11-17 00:55:12 +05:30
Jason Lowe
41f8275c50 MAPREDUCE-6804. Add timeout when starting JobHistoryServer in MiniMRYarnCluster. Contributed by Andras Bokor 2016-11-16 15:26:19 +00:00
Jian He
a422740bd7 YARN-5611. Provide an API to update lifetime of an application. Contributed by Rohith Sharma K S 2016-11-09 16:39:46 -08:00
Naganarasimha
1455f56b63 MAPREDUCE-2631. Potential resource leaks in BinaryProtocol$TeeOutputStream.java. Contributed by Sunil G. 
(cherry picked from commit 28660f51af161a9fa301523d96a6f8ae4ebd6edd)
 2016-10-28 06:00:29 +05:30
Akira Ajisaka
e29d0a2fd8 YARN-5575. Many classes use bare yarn. properties instead of the defined constants. Contributed by Daniel Templeton. 
(cherry picked from commit d3bb69a66776e9f410150c4030ddb15981f58fb9)
 2016-10-26 15:34:40 +09:00
Robert Kanter
48ca6be76e MAPREDUCE-6776. yarn.app.mapreduce.client.job.max-retries should have a more useful default (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit f3f37e6fb8172f6434e06eb9a137c0c155b3952e)
 2016-10-07 14:47:23 -07:00
Naganarasimha
0e4a5611ff YARN-3692. Allow REST API to set a user generated message when killing an application. Contributed by Rohith Sharma K S 2016-09-24 21:13:01 +05:30
Naganarasimha
5fa241daa0 YARN-4491. yarn list command to support filtering by tags. Contributed by Varun Saxena 
(cherry picked from commit 143c59e4c5a811eb2c12cf6626d558f9b8796e03)
 2016-08-24 01:55:24 +05:30
Wangda Tan
65822e55fd MAPREDUCE-6310. Add jdiff support to MapReduce. (Li Lu/vinodkv via wangda) 
(cherry picked from commit 3d937457ee60d617d71b36bb6a29344e771670b4)
 2016-08-19 16:41:47 -07:00
Jason Lowe
848ae35a93 MAPREDUCE-6690. Limit the number of resources a single map reduce job can submit for localization. Contributed by Chris Trezzo 
(cherry picked from commit f80a7298325a4626638ee24467e2012442e480d4)
 2016-08-17 16:24:16 +00:00
Arun Suresh
5f7edb79d1 YARN-5457. Refactor DistributedScheduling framework to pull out common functionality. (asuresh) 
(cherry picked from commit 82c9e061017c32e633e0b0cbb7978749a6df4fb2)
 2016-08-09 00:46:08 -07:00