9155 Commits

Author SHA1 Message Date
Devaraj K
5fceebd696 YARN-3629. NodeID is always printed as "null" in node manager
initialization log. Contributed by nijel.

(cherry picked from commit 5c2f05cd9bad9bf9beb0f4ca18f4ae1bc3e84499)
2015-05-12 22:22:40 +05:30
Tsuyoshi Ozawa
fb5b0ebb45 MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du.
(cherry picked from commit f4e2b3cc0b1f4e49c306bc09a9dddd0495225bb2)
2015-05-13 00:28:36 +09:00
yliu
c31c6fbda7 HDFS-8255. Rename getBlockReplication to getPreferredBlockReplication. (Contributed by Zhe Zhang) 2015-05-12 21:26:57 +08:00
Devaraj K
f887243f88 YARN-3513. Remove unused variables in ContainersMonitorImpl and add debug
log for overall resource usage by all containers.  Contributed by
Naganarasimha G R.

(cherry picked from commit 8badd82ce256e4dc8c234961120d62a88358ab39)
2015-05-12 16:56:26 +05:30
Vinayakumar B
cda1962996 MAPREDUCE-6360. TestMapreduceConfigFields is placed in wrong dir, introducing compile error (Contributed by Arshad Mohammad)
(cherry picked from commit 360dff5903085c3c7f02ccf9f17d71a842275e48)
2015-05-12 12:50:44 +05:30
Vinayakumar B
e1b8bb9d84 HDFS-8362. Java Compilation Error in TestHdfsConfigFields.java (Contributed by Arshad Mohammad)
(cherry picked from commit 987abc99b0309a07f0a342746b2a5048d5c36ce0)
2015-05-12 12:10:14 +05:30
Wangda Tan
12584ac10e Move YARN-3493 from 2.8 to 2.7.1 in CHANGES.txt 2015-05-11 18:08:25 -07:00
Wangda Tan
d06d2a96f5 YARN-3489. RMServerUtils.validateResourceRequests should only obtain queue info once. (Varun Saxena via wangda)
(cherry picked from commit d6f6741296639a73f5306e3ebefec84a40ca03e5)
2015-05-11 17:35:01 -07:00
Jason Lowe
53e9973afc MAPREDUCE-5465. Tasks are often killed before they exit on their own. Contributed by Ming Ma 2015-05-11 22:40:05 +00:00
Wangda Tan
2c80a3fe19 Moved YARN-3434. (Interaction between reservations and userlimit can result in significant ULF violation.) From 2.8.0 to 2.7.1
(cherry picked from commit 1952f88889395870e7b631d43418e075e774b9d2)
2015-05-11 15:22:25 -07:00
Kihwal Lee
50778f9d45 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop. Contributed by Rushabh Shah.
(cherry picked from commit ea11590aad952b5b560a5101d064adf27d8656db)
2015-05-11 14:33:45 -05:00
Allen Wittenauer
733579ddc8 re-commit of HADOOP-11881 2015-05-11 12:13:55 -07:00
Allen Wittenauer
93bbbddda2 HADOOP-11947. test-patch should return early from determine-issue when run in jenkins mode. (Sean Busbey via aw) 2015-05-11 12:08:23 -07:00
Allen Wittenauer
5c7c7b7b74 HADOOP-11948. test-patch's issue matching regex should be configurable. (Sean Busbey via aw) 2015-05-11 11:54:21 -07:00
Allen Wittenauer
cea8213c3e HADOOP-11950. Add cli option to test-patch to set the project-under-test (Sean Busbey via aw) 2015-05-11 11:51:04 -07:00
Allen Wittenauer
6307687f7d HADOOP-11951. test-patch should give better info about failures to handle dev-support updates without resetrepo option (Sean Busbey via aw) 2015-05-11 11:46:14 -07:00
Allen Wittenauer
fa1cf5a51a HADOOP-11928. Test-patch check for @author tags incorrectly flags removal of @author tags (Kengo Seki via aw) 2015-05-11 11:35:42 -07:00
Akira Ajisaka
77587fe198 HADOOP-11663. Remove description about Java 6 from docs. Contributed by Masatake Iwasaki.
(cherry picked from commit b9cebfc0ba871beb99f634a1a8c797e143ecc6d4)
2015-05-12 00:31:28 +09:00
Junping Du
a1a7deebf8 YARN-3587. Fix the javadoc of DelegationTokenSecretManager in yarn, etc. projects. Contributed by Gabor Liptak.
(cherry picked from commit 7e543c27fa2881aa65967be384a6203bd5b2304f)
2015-05-11 05:28:25 -07:00
Akira Ajisaka
703f1e9d16 HDFS-8351. Remove namenode -finalize option from document. (aajisaka)
(cherry picked from commit 3fa2efc09f051b6fc6244f0edca46d3d06f4ae3b)
2015-05-11 15:37:12 +09:00
Haohui Mai
1f499a61dc HDFS-8357. Consolidate parameters of INode.CleanSubtree() into a parameter objects. Contributed by Li Lu. 2015-05-09 22:52:07 -07:00
Karthik Kambatla
3cf4f24a9f YARN-3395. FairScheduler: Trim whitespaces when using username for queuename. (Zhihai Xu via kasha)
(cherry picked from commit a60f78e98ed73ab320576c652c577f119ce70901)
2015-05-09 15:41:49 -07:00
Karthik Kambatla
52469b10dd MAPREDUCE-6353. Divide by zero error in MR AM when calculating available containers. (Anubhav Dhoot via kasha)
(cherry picked from commit 1773aac780585871072960a5863af461e112a030)
2015-05-09 14:43:57 -07:00
Karthik Kambatla
e8dd0bc4fd YARN-1287. Consolidate MockClocks. (Sebastian Wong and Anubhav Dhoot via kasha)
(cherry picked from commit 70fb37cd79a7eba6818313960624380bacfe0bb2)
2015-05-09 14:35:13 -07:00
Junping Du
39c188449e MAPREDUCE-6359. In RM HA setup, Cluster tab links populated with AM hostname instead of RM. Contributed by zhaoyunjiong.
(cherry picked from commit df36ad0a08261b03c250b6f745b27e5f83e4286e)
2015-05-09 06:12:01 -07:00
Haohui Mai
300d36cee7 HDFS-8327. Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncate(). Contributed by Haohui Mai. 2015-05-08 23:09:57 -07:00
Haohui Mai
51bfb66ec8 Add missing entry in CHANGES.txt for HDFS-6757. 2015-05-08 23:09:55 -07:00
Haohui Mai
73d61f7ae7 HDFS-6757. Simplify lease manager with INodeID. Contributed by Haohui Mai. 2015-05-08 23:04:57 -07:00
Xuan
acb342b77c YARN-1912. ResourceLocalizer started without any jvm memory control.
Contributed by Masatake Iwasaki

(cherry picked from commit 6471d18bc72bc6c83ce31a03b5c5f5737847bb6d)
2015-05-08 20:02:23 -07:00
Xuan
fc980247a6 YARN-3602. TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IOException from cleanup. Contributed by zhihai xu
(cherry picked from commit 333f9a896d8a4407ce69cfd0dc8314587a339233)
2015-05-08 17:54:46 -07:00
Zhijie Shen
ef096599c3 YARN-2206. Updated document for applications REST API response examples. Contributed by Kenji Kikushima and Brahma Reddy Battula.
(cherry picked from commit 08f0ae403c649b28925d3b339b7b6de1d7ec2c0c)
2015-05-08 16:41:29 -07:00
Karthik Kambatla
a7d8f21630 YARN-3271. FairScheduler: Move tests related to max-runnable-apps from TestFairScheduler to TestAppRunnability. (nijel via kasha)
(cherry picked from commit 2fb44c8aaf7f35f425d80b133a28b1c1f305c3e6)
2015-05-08 16:40:36 -07:00
Karthik Kambatla
21a07769d4 YARN-1050. Document the Fair Scheduler REST API. (Kenji Kikushima and Roman Shaposhnik via kasha)
(cherry picked from commit 96473bdc2b090c13708fd467fd626621ef1d47eb)
2015-05-08 16:31:30 -07:00
Arpit Agarwal
969a8f6c96 HDFS-8097. TestFileTruncate is failing intermittently. (Contributed by Rakesh R) 2015-05-08 16:29:01 -07:00
Robert Kanter
453488cff2 YARN-3473. Fix RM Web UI configuration for some properties (rchiang via rkanter)
(cherry picked from commit 5658998845bbeb3f09037a891f6b254585848de7)
2015-05-08 16:23:20 -07:00
Robert Kanter
015b30c3ab HADOOP-9737. JarFinder#getJar should delete the jar file upon destruction of the JVM (jbonofre via rkanter)
(cherry picked from commit 315074bb08e0f2c9508834ada62f1533ffd82a26)
2015-05-08 16:05:59 -07:00
Ravi Prakash
23065e6051 MAPREDUCE-4750. Enable NNBenchWithoutMR in MapredTestDriver (Liang Xie and Jason Lowe via raviprak)
(cherry picked from commit 5aab014340b53ebc9363ee244b2cbea7a4c1f573)
2015-05-08 15:58:29 -07:00
Jason Lowe
743ca1a056 YARN-3476. Nodemanager can fail to delete local logs if log aggregation fails. Contributed by Rohith
(cherry picked from commit 25e2b02122c4ed760227ab33c49d3445c23b9276)
2015-05-08 22:46:34 +00:00
Jason Lowe
0f6c971282 MAPREDUCE-5248. Let NNBenchWithoutMR specify the replication factor for its test. Contributed by Erik Paulson
(cherry picked from commit 30099a36c6b0f658d25fb505a9f3ce15d19f7ba6)
2015-05-08 22:30:21 +00:00
Allen Wittenauer
a1484cf771 HADOOP-11906. test-patch.sh should use file command for patch determinism (Sean Busbey via aw) 2015-05-08 15:14:13 -07:00
Jian He
5a1c2fed34 YARN-3604. Fixed ZKRMStateStore#removeApplication to also disable watch. Contributed Zhihai Xu
(cherry picked from commit d0b73287c26983f32b0efe01862150e9fd28086f)
2015-05-08 15:13:43 -07:00
Xuan
8d07116266 YARN-2331. Distinguish shutdown during supervision vs. shutdown for
rolling upgrade. Contributed by Jason Lowe

(cherry picked from commit 088156de43abb07bec590a3fcd1a5af2feb02cd2)
2015-05-08 15:11:55 -07:00
Vinod Kumar Vavilapalli
24c8d3e54b YARN-3018. Unified the default value for the configuration property yarn.scheduler.capacity.node-locality-delay in code and default xml file. Contributed by Nijel SF.
(cherry picked from commit f7a74d249173292035839810a1595d7a028bfdda)
2015-05-08 14:45:55 -07:00
Kihwal Lee
f264a5aeed HDFS-8245. Standby namenode doesn't process DELETED_BLOCK if the addblock request is in edit log. Contributed by Rushabh S Shah.
(cherry picked from commit 2d4ae3d18bc530fa9f81ee616db8af3395705fb9)
2015-05-08 16:37:26 -05:00
Chris Douglas
4f301f92c1 MAPREDUCE-2094. LineRecordReader should not seek into non-splittable, compressed streams. 2015-05-08 14:30:56 -07:00
Jason Lowe
1523ed5a76 HADOOP-9729. The example code of org.apache.hadoop.util.Tool is incorrect. Contributed by hellojinjie
(cherry picked from commit ec2748dedbc500bd554aa62ca76ccab593129966)
2015-05-08 21:29:18 +00:00
Jason Lowe
dfc4a62f7d MAPREDUCE-3383. Duplicate job.getOutputValueGroupingComparator() in ReduceTask. Contributed by Binglin Chang
(cherry picked from commit c39012f4a0444f9e4b7d67957d5192127d143d90)
2015-05-08 21:17:05 +00:00
Colin Patrick Mccabe
ab80a572be HDFS-8113. Add check for null BlockCollection pointers in BlockInfoContiguous structures (Chengbing Liu via Colin P. McCabe)
(cherry picked from commit f523e963e4d88e4e459352387c6efeab59e7a809)
2015-05-08 14:00:13 -07:00
Yongjun Zhang
fbb2cf54bc HDFS-8311. DataStreamer.transfer() should timeout the socket InputStream. (Esteban Gutierrez via Yongjun Zhang)
(cherry picked from commit 730f9930a48259f34e48404aee51e8d641cc3d36)
2015-05-08 01:38:01 -07:00
Jason Lowe
4b96834e0d HADOOP-7165. listLocatedStatus(path, filter) is not redefined in FilterFs. Contributed by Hairong Kuang
(cherry picked from commit c4bd2cddede779acfd3c5cdff16bf552068dfb64)
2015-05-08 20:36:55 +00:00