Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,421 Commits 326 Branches 371 Tags
Commit Graph

479 Commits

Author SHA1 Message Date
Konstantin V Shvachko
7ab03ffbf0 MAPREDUCE-6931. Remove TestDFSIO "Total Throughput" calculation. Contributed by Dennis Huo. 
(cherry picked from commit 3e0e2033cc28d48a7909822416478aff7487bfe6)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
 2017-08-30 14:46:39 -07:00
Junping Du
5ad710bc7a MAPREDUCE-6246. DBOutputFormat.java appending extra semicolon to query which is incompatible with DB2. Contributed by ramtin and Gergely Novák. 
(cherry picked from commit f484a6ff602d48413556a1d046670e2003c71c2e)
(cherry picked from commit f823f9fd784fa4944178247a82df24fba03c2051)
 2017-07-07 13:32:21 -07:00
Naganarasimha
acc504dc58 MAPREDUCE-6905. HADOOP_JOB_HISTORY_OPTS should be HADOOP_JOB_HISTORYSERVER_OPTS in mapred-config.sh. Contributed by LiXin Ge. 
(cherry picked from commit bf1f59937dd5d860c9ed7fefce203d6a9f645182)
 2017-07-02 16:18:30 +05:30
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
396de0fd8f Preparing for 2.8.3 development 2017-06-07 15:38:20 -07:00
Brahma Reddy Battula
c60516aee5 MAPREDUCE-6676. NNBench should Throw IOException when rename and delete fails. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 24181f559f008fc7945702c0fad7d1028f75bfc6)
 2017-06-07 16:38:05 +08:00
Junping Du
dfac95a3a2 Preparing for 2.8.1 development 2017-04-11 11:21:49 -07:00
Akira Ajisaka
8b5307c0e7
MAPREDUCE-6855. Specify charset when create String in CredentialsTestJob. Contributed by Kai Sasaki. 
(cherry picked from commit 14413989cac9acc1fa6f8d330fac32f772613325)
(cherry picked from commit aa1f944bba02dcb9a14d9d37cfc113845d437ffa)
 2017-03-07 13:13:52 +09:00
Akira Ajisaka
c6bf43e0be MAPREDUCE-6753. Variable in byte printed directly in mapreduce client. Contributed by Kai Sasaki. 
(cherry picked from commit 19a1fc6373f0dea56e5845f19990d469428489bc)
(cherry picked from commit e2fb9538f0e2855d757bcbd586c37278824700ea)
 2017-03-03 16:14:03 +09:00
Wangda Tan
920847edce YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda) 
(cherry picked from commit 0566b30875dfc61dbbba4cde0d42e15b116e85b8)
 2017-02-22 11:21:57 -08:00
Junping Du
7706a63fb4 Preparing for 2.8.1 development after cutting off 2.8.0 branch. 2017-01-10 18:03:51 -08:00
Jason Lowe
8ffe5eb7bd MAPREDUCE-6682. TestMRCJCFileOutputCommitter fails intermittently Contributed by Akira Ajisaka. 
(cherry picked from commit 8f1c374bec2451568f463ab68f7eb6db734ab14e)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java
 2017-01-06 18:55:48 +00:00
Rohith Sharma K S
0c1615c368 MAPREDUCE-6662. Clear ASF Warnings on test data files. Contributed by Vinayakumar B 
(cherry picked from commit 0ef8bbfd8791899c7cfed3dd9c1670182fd87575)
(cherry picked from commit f22ce96b50f8148e95a453ff41f84b99a9197bf6)
 2017-01-05 18:03:48 +09:00
Jian He
42d1b38989 YARN-4164. Changed updateApplicationPriority API to return the updated application priority. Contributed by Rohith Sharma K S 2017-01-04 15:43:06 -08:00
Brahma Reddy Battula
b8be8f0d4f HADOOP-13883. Addendum patch to correct the message and alphabetized with the earlier patch. Contributed by Yiqun Lin. 2016-12-31 16:27:17 +05:30
Junping Du
5f36b62265 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit 7bfd9e068d205f752c05d51778a4a4702329d31e.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryViewer.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
 2016-12-28 17:37:20 -08:00
Junping Du
8755a6f5ac Revert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)" 
This reverts commit f9341c1e2cffc4e3ac37cb7da4e535ca5c6d3913.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-12-28 17:13:04 -08:00
Jason Lowe
e27fbcb50c MAPREDUCE-6804. Add timeout when starting JobHistoryServer in MiniMRYarnCluster. Contributed by Andras Bokor 
(cherry picked from commit 41f8275c50b34c919cca26510d505775d676ebb4)
 2016-11-16 15:27:32 +00:00
Zhe Zhang
efc0e640ed Addendum of HDFS-10872. Removed chan ge in MiniHadoopClusterManager.java 2016-11-14 16:43:43 -08:00
Zhe Zhang
c6e7b631bd HDFS-10872. Add MutableRate metrics for FSNamesystemLock operations. Contributed by Erik Krogen. 
(cherry picked from commit 818915aabb22653e212f4646b1b3cd0673330328)
 2016-11-14 16:39:26 -08:00
Naganarasimha
8a664eba7d MAPREDUCE-2631. Potential resource leaks in BinaryProtocol$TeeOutputStream.java. Contributed by Sunil G. 
(cherry picked from commit 28660f51af161a9fa301523d96a6f8ae4ebd6edd)
 2016-10-28 06:04:03 +05:30
Naganarasimha
fa2025316d YARN-3692. Allow REST API to set a user generated message when killing an application. Contributed by Rohith Sharma K S 2016-09-24 22:42:28 +05:30
Wangda Tan
e2bd490236 MAPREDUCE-6310. Add jdiff support to MapReduce. (Li Lu/vinodkv via wangda) 
(cherry picked from commit 3d937457ee60d617d71b36bb6a29344e771670b4)
(cherry picked from commit 65822e55fd99511ab0c39f802d3ec58290445f9d)
 2016-08-19 16:49:54 -07:00
Akira Ajisaka
fa0b6ff97b MAPREDUCE-6729. Accurately compute the test execute time in DFSIO. Contributed by mingleizhang. 
This closes #112

(cherry picked from commit ce93595d7a18bd0561ae2c1983b51f6aba887179)
(cherry picked from commit 96d5607e7a2e6deebdb76858955127d09c125729)
 2016-07-30 22:16:43 +09:00
Jason Lowe
e098a732ef MAPREDUCE-6744. Increase timeout on TestDFSIO tests. Contributed by Eric Badger 
(cherry picked from commit 703fdf86c6e42d6d555090f670ca5cbd1ef48bf9)
 2016-07-25 21:56:33 +00:00
Jian He
23eb3c7ceb YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan 2016-07-11 22:38:35 -07:00
Wangda Tan
93b406b66d YARN-4844. Addendum patch, revert few changes to public and evolving APIs to avoid breaking downstream projects as much as possible. 
(cherry picked from commit 7eb524ddee4af80c87b134070945977bf73e950f)
 2016-06-16 11:10:46 -07:00
Wangda Tan
d838c6443d YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:21:41 -07:00
Akira Ajisaka
8295351e54 HADOOP-13270. BZip2CompressionInputStream finds the same compression marker twice in corner case, causing duplicate data blocks. Contributed by Kai Sasaki. 
(cherry picked from commit e3ba9ad3f116306910f74645ded91506345b9f6e)
(cherry picked from commit d219550e8bf4c24be68a74f7b8032b6a8b8af0fc)
 2016-06-14 10:24:53 +09:00
Wangda Tan
19e578870d YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. (wangda) 2016-06-07 12:41:50 -07:00
Ming Ma
ec4f9a14f9 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 
(cherry picked from commit 4a1cedc010d3fa1d8ef3f2773ca12acadfee5ba5)
(cherry picked from commit 74e2b5efa26f27027fed212b4b2108f0e95587fb)
 2016-06-06 14:49:43 -07:00
Wangda Tan
2f3e1d965d Revert "YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan." 
This reverts commit 457884737f75c796413ce860b1859a31cc5292ca.
 2016-05-31 22:16:53 -07:00
Varun Vasudev
457884737f YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 20:57:56 +05:30
Carlo Curino
7982933c09 YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 
(cherry picked from commit 013532a95e63d7c53e601be530021d6d5a15ab7f)
(cherry picked from commit c656977961e2ba0f9dfd349ed59bf1d0d41c57f5)
 2016-05-25 17:02:22 -07:00
Akira Ajisaka
73d51f5c47 MAPREDUCE-6663. [NNBench] Refactor nnbench as a Tool implementation. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 690d8a368d3e967495eafea27659b6124989c89e)
(cherry picked from commit 94b2ba89a467c131d1cd27249ecd81363b0da809)
 2016-03-30 11:44:49 +09:00
Akira Ajisaka
68cd8112e7 MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote. 
(cherry picked from commit 0050fa5f1c20087009bd76a0bb2183a479f787f0)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/partition/TestTotalOrderPartitioner.java

(cherry picked from commit 7b2cf29f4f673a955910b335d46687989cde478b)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
 2016-03-29 18:24:22 +09:00
Akira Ajisaka
0286ac2576 Revert "MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote." 
This reverts commit 416ea158c2aa197b4a997ed420842154a1294b1d.
 2016-03-28 16:06:10 +09:00
Akira Ajisaka
416ea158c2 MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote. 
(cherry picked from commit 2c268cc9365851f5b02d967d13c8c0cbca850a86)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/partition/TestTotalOrderPartitioner.java

(cherry picked from commit 0dc145362642f6e5f9def8f50775b5c16599f6e6)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
 2016-03-25 15:33:01 +09:00
Akira Ajisaka
f362b8a403 MAPREDUCE-6656. [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363. Contributed by J.Andreina. 
(cherry picked from commit 368c77376a3704e3fa395d72b1816d388b5cd782)
(cherry picked from commit 7a3fd1bc1a2ce93100b6887a97733e262b5e1d56)
 2016-03-23 23:24:40 +09:00
Akira Ajisaka
9d7c88f275 MAPREDUCE-6580. Test failure: TestMRJobsWithProfiler. Contributed by Eric Badger. 
(cherry picked from commit 0bfe5a0167f0af65f1d7a8bc251a1c404ed41316)
(cherry picked from commit 12c1cc889ab955c30a8dc5ed68f1df1a5c557933)
 2016-03-23 13:19:26 +09:00
Wangda Tan
2d9f42d685 MAPREDUCE-6579. JobStatus#getFailureInfo should not output diagnostic information when the job is running. (Akira AJISAKA via wangda) 
(cherry picked from commit 6529c87551e07e9da1e1b273eff4c6bad6b3e838)
 2016-03-15 17:32:32 -07:00
Akira Ajisaka
cb97d8fa63 MAPREDUCE-6520. Migrate MR Client test cases part 1. Contributed by Dustin Cote. 
(cherry picked from commit 247a7906092065289ea81139e71badcac6abef1e)
(cherry picked from commit c2e14d1bbb0cf87f76ceba5c958a46e7a4de7215)
 2016-03-11 22:55:16 +09:00
Akira Ajisaka
097a03fb6d Revert "MAPREDUCE-6520. Migrate MR Client test cases part 1." 
This reverts commit 70e232f36e8400b99551ce6f4f4fc48b44e28518.
 2016-03-11 22:55:01 +09:00
Akira Ajisaka
70e232f36e MAPREDUCE-6520. Migrate MR Client test cases part 1. 
(cherry picked from commit ef68b441957c6deb65adae2272d60ac929d00733)
(cherry picked from commit 7d665f0a987c17dcf8a26113c61a70dee196ee3d)
 2016-03-11 22:26:03 +09:00
Jason Lowe
83985eed2a MAPREDUCE-6191. Improve clearing stale state of Java serialization 
testcase.  (Sam Liu via Eric Yang)
(cherry picked from commit c379e102ddab1b83fd69e4492a40c9901fb50675)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
(cherry picked from commit 303c650d0dfcb10760cea589b8cef6ace8e36417)
 2016-02-10 16:46:58 +00:00
Wangda Tan
c487453b91 YARN-4340. Add list API to reservation system. (Sean Po via wangda) 
(cherry picked from commit 9875325d5c63f343809907d06bf48a298035a611)
 2016-02-02 10:23:25 +08:00
Jason Lowe
7bb48ed162 MAPREDUCE-6618. YarnClientProtocolProvider leaking the YarnClient thread. Contributed by Xuan Gong 
(cherry picked from commit 59a212b6e1265adfa9b55c71b65a22157dfccf77)
 2016-02-01 16:11:20 +00:00
Akira Ajisaka
d4d368d4e8 MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Vlad Sharanhovich and Bibin A Chundatt. 
(cherry picked from commit 7b0964f354e90968c2dac2f7acc17214732aed64)
(cherry picked from commit 6efee8907af3f21d8526927cf5e676d9922b686b)
 2016-01-14 10:41:45 +09:00
Sangjin Lee
ae535ec93e MAPREDUCE-6577. MR AM unable to load native library without MR_AM_ADMIN_USER_ENV set (sjlee) 
(cherry picked from commit f6f16118d38fcfe3b724f05fad752cb223f441ec)
(cherry picked from commit 28fd4c70cab27c2872dc6a63ddee223ccf53f21d)
 2016-01-06 08:44:37 -08:00
Jason Lowe
2f18218508 MAPREDUCE-5870. Support for passing Job priority through Application Submission Context in Mapreduce Side. Contributed by Sunil G 
(cherry picked from commit f634505d48d97e4d461980d68a0cbdf87223646d)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java
 2015-11-24 22:15:37 +00:00
Haohui Mai
469fef2a4f HDFS-9153. Pretty-format the output for DFSIO. Contributed by Kai Zheng. 2015-11-22 16:18:16 -08:00