Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,754 Commits 326 Branches 371 Tags
Commit Graph

1510 Commits

Author SHA1 Message Date
Rohith Sharma K S
3207762192 YARN-5988. RM unable to start in secure setup. Contributed by Ajith S. 
(cherry picked from commit e49e0a6e37f4a32535d7d4a07015fbf9eb33c74a)
 2017-01-04 12:10:05 +05:30
Wangda Tan
23f6a33156 YARN-6025. Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda) 
(cherry picked from commit e0f2379312c48e26b0cb2c1e1e803ef71d1839cf)
(cherry picked from commit 554e6656bc9cf898d14d1fe2da5be5bff8fc4284)
 2017-01-03 14:52:31 -08:00
Wangda Tan
cb64e8eb19 Revert "Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda)" because forgot to add JIRA Number 
This reverts commit 554e6656bc9cf898d14d1fe2da5be5bff8fc4284.
 2017-01-03 14:51:59 -08:00
Wangda Tan
554e6656bc Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda) 
(cherry picked from commit e0f2379312c48e26b0cb2c1e1e803ef71d1839cf)
 2017-01-03 14:50:19 -08:00
Daniel Templeton
8cb7aa2b52 YARN-5931. Document timeout interfaces CLI and REST APIs (Contributed by Rohith Sharma K S via Daniel Templeton) 
(cherry picked from commit 165d01a73e5f328108a9e876c7a751a3c2cec4a8)
 2016-12-30 08:56:03 -08:00
Robert Kanter
f9b48206dd YARN-4882. Change the log level to DEBUG for recovering completed applications (templedf via rkanter) 
(cherry picked from commit f216276d2164c6564632c571fd3adbb03bc8b3e4)
 2016-12-28 15:29:06 -08:00
Daniel Templeton
47f0a0e15b HADOOP-13932. Fix indefinite article in comments (Contributed by LiXin Ge via Daniel Templeton) 2016-12-28 07:03:51 -08:00
Sunil G
b3231b480d YARN-6024. Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to max. Contributed by Wangda Tan. 
(cherry picked from commit f6715b26b65fc127b35368d0460b786ece88b5b8)
 2016-12-28 16:00:52 +05:30
Wangda Tan
42f571728c YARN-5756. Add state-machine implementation for scheduler queues. (Xuan Gong via wangda) 
(cherry picked from commit 0840b4329b2428b20b862f70d72cbdcd6d1618ed)
 2016-12-27 21:19:10 -08:00
Robert Kanter
8fbd36c60f YARN-5962. Spelling errors in logging and exceptions for resource manager code (gsohn via rkanter) 
(cherry picked from commit 1bbd023275db535ab80fcb60e022151e9679d468)
 2016-12-27 13:35:55 -08:00
Karthik Kambatla
10a154cedd YARN-5969. FairShareComparator: Cache value of getResourceUsage for better performance. (zhangshilong via kasha) 
(cherry picked from commit c3973e7080bf71b57ace4a6adf4bb43f3c5d35b5)
 2016-12-27 12:30:50 -08:00
Sunil G
238863c11b YARN-5650. Render Application Timeout value in web UI. Contributed by Akhil PB. 2016-12-20 11:03:57 +05:30
Sunil G
23bd68a4a4 YARN-5970. Validate application update timeout request parameters. Contributed by Rohith Sharma K S. 2016-12-12 11:59:14 +05:30
Jian He
71fdca4600 YARN-5709. Cleanup leader election configs and pluggability. Contribtued by Karthik Kambatla 
(cherry picked from commit b817c565c8be1d4a682d119bfac6f43ee09e87f0)
 2016-12-09 16:59:52 -08:00
Sunil
a636a87270 YARN-5965. Retrospect ApplicationReport#getApplicationTimeouts. Contributed by Rohith Sharma K S 
(cherry picked from commit ab923a53fcf55d4d75aa027d46e3c4a659015325)
 2016-12-08 00:34:45 +05:30
Daniel Templeton
d8b4961515 YARN-5136. Error in handling event type APP_ATTEMPT_REMOVED to the scheduler 
(Contributed by Wilfred Spiegelenburg via Daniel Templeton)
 2016-12-07 11:01:09 -08:00
Rohith Sharma K S
602c998443 YARN-5932. Retrospect moveApplicationToQueue in align with YARN-5611. Contributed by Sunil G. 2016-12-07 22:45:25 +05:30
Naganarasimha
54c5880cf7 YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena 
(cherry picked from commit f3b8ff54ab08545d7093bf8861b44ec9912e8dc3)
 2016-12-06 11:01:14 +05:30
Jian He
0f6fbfc0db YARN-5559. Analyse 2.8.0/3.0.0 jdiff reports and fix any issues. Contributed by Akira Ajisaka & Wangda Tan 
(cherry picked from commit 43ebff2e354142bddcb42755766a965ae8a503a6)
 2016-12-05 11:40:26 -08:00
Jian He
d58fca0102 YARN-5746. The state of the parentQueue and its childQueues should be synchronized. Contributed by Xuan Gong 2016-12-02 16:18:39 -08:00
Daniel Templeton
1216bad0f7 YARN-5929. Missing scheduling policy in the FS queue metric. (Contributed by Yufei Gu via Daniel Templeton) 
(cherry picked from commit 9b076327eac9da244f94e226816465e7d17fda54)
 2016-12-02 14:02:35 -08:00
Li Lu
8ffe86f780 YARN-5761. Separate QueueManager from Scheduler. (Xuan Gong via gtcarrera9) 
(cherry picked from commit 69fb70c31aa277f7fb14b05c0185ddc5cd90793d)
 2016-11-30 14:02:31 -08:00
Daniel Templeton
803aba03f2 YARN-5890. FairScheduler should log information about AM-resource-usage and max-AM-share for queues 
(Contributed by Yufei Gu via Daniel Templeton)
 2016-11-29 14:14:43 -08:00
Daniel Templeton
b8bebb8607 YARN-5774. MR Job stuck in ACCEPTED status without any progress in Fair Scheduler 
if set yarn.scheduler.minimum-allocation-mb to 0. (Contributed by Yufei Gu via Daniel Templeton)

(cherry picked from commit 25f9872be63423ada6a18481eaad2888e731fdac)
 2016-11-29 10:12:10 -08:00
Sunil
c28dc5e5a4 YARN-4206. Add Application timeouts in Application report and CLI. Contributed by Rohith Sharma K S. 
(cherry picked from commit eb0a483ed07399c7a06ed9db4bfbec382de470da)
 2016-11-24 18:27:50 +05:30
Sunil
7b676c5632 YARN-5649. Add REST endpoints for updating application timeouts. Contributed by Rohith Sharma K S 2016-11-24 17:37:36 +05:30
Rohith Sharma K S
6b47a7fb8e YARN-5920. Fix deadlock in TestRMHA.testTransitionedToStandbyShouldNotHang. Contributed by Varun Saxena. 
(cherry picked from commit e15c20edba1e9a23475ee6a4dfbadbdb8c1f668a)
 2016-11-24 12:19:19 +05:30
Rohith Sharma K S
87bb3c51d6 Reverting for fixing compilation errors in branch-2. 
Revert "YARN-5649. Add REST endpoints for updating application timeouts. Contributed by Rohith Sharma K S"

This reverts commit 43796580421346f254d066c0c58ae13937cdbd45.
 2016-11-24 09:18:07 +05:30
Jian He
4379658042 YARN-5649. Add REST endpoints for updating application timeouts. Contributed by Rohith Sharma K S 2016-11-23 16:27:12 -08:00
Rohith Sharma K S
2d068f1793 YARN-5865. Retrospect updateApplicationPriority api to handle state store exception in align with YARN-5611. Contributed by Sunil G. 2016-11-23 23:29:39 +05:30
Daniel Templeton
06f5d27bd6 YARN-5722. FairScheduler hides group resolution exceptions when assigning queue (Contributed by Wilfred Spiegelenburg via Daniel Templeton) 
(cherry picked from commit 83cc7263af632939dc3b2ee58d8f03f98ed4d96a)
 2016-11-22 11:17:39 -08:00
Rohith Sharma K S
74ac78b3c0 YARN-5375. invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures. Contributed by sandflee. 
(cherry picked from commit d65603517e52843f11cd9d3b6f6e28fca9336ee3)
 2016-11-16 15:17:46 +05:30
Xiao Chen
3497221fcc YARN-5875. TestTokenClientRMService#testTokenRenewalWrongUser fails. Contributed by Gergely Novák. 
(cherry picked from commit d5c396ccb361e1a0cf4307ce1d1b37fae0eec250)
 2016-11-15 13:58:02 -08:00
Rohith Sharma K S
528ef2407c YARN-5874. RM -format-state-store and -remove-application-from-state-store commands fail with NPE. Contributed by Varun Saxena. 
(cherry picked from commit b7070f3308fc4c6a8a9a25021562169cae87d223)
 2016-11-15 11:03:38 +05:30
Rohith Sharma K S
fad6de3080 YARN-5873. RM crashes with NPE if generic application history is enabled. Contributed by Varun Saxena. 
(cherry picked from commit 296c5de0cfee88389cf9f90263280b2034e54cd5)
 2016-11-15 11:03:27 +05:30
Karthik Kambatla
ea74c7b7ae YARN-4033. In FairScheduler, parent queues should also display queue status. (Siqi Li via kasha) 
(cherry picked from commit 79448d4ab1f298e50565b161e6605d994ee5e058)
 2016-11-13 21:39:44 -08:00
Jian He
9b0a2cb321 YARN-5825. ProportionalPreemptionalPolicy should use readLock over LeafQueue instead of synchronized block. Contributed by Sunil G 
(cherry picked from commit fad9609d13e76e9e3a4e01c96f698bb60b03807e)
 2016-11-11 15:16:43 -08:00
Wangda Tan
a76e214556 YARN-5552. Add Builder methods for common yarn API records. (Tao Jie via wangda) 
(cherry picked from commit ede1a473f5061cf40f6affc1c8c30a645c1fef6c)
 2016-11-11 13:42:45 -08:00
Naganarasimha
bde95f90e7 YARN-5545, Fix issues related to Max App in capacity scheduler. Contributed by Bibin A Chundatt 2016-11-12 00:52:35 +05:30
Eric Payne
dd5b9dabf9 YARN-4218. Metric for resource*time that was preempted. Contributed by Chang Li. 2016-11-10 23:02:42 +00:00
Karthik Kambatla
f7b25420a1 YARN-5453. FairScheduler#update may skip update demand resource of child queue/app if current demand reached maxResource. (sandflee via kasha) 
(cherry picked from commit 86ac1ad9fd65c7dd12278372b369de38dc4616db)
 2016-11-09 23:44:26 -08:00
Jian He
a422740bd7 YARN-5611. Provide an API to update lifetime of an application. Contributed by Rohith Sharma K S 2016-11-09 16:39:46 -08:00
Naganarasimha
8a2998c08c YARN-4498. Application level node labels stats to be available in REST (addendum patch). Contributed by Bibin A Chundatt. 2016-11-10 05:21:07 +05:30
Daniel Templeton
42147bbbf7 YARN-4329. [YARN-5437] Allow fetching exact reason as to why a submitted app 
is in ACCEPTED state in Fair Scheduler (Contributed by Yufei Gu)

(cherry picked from commit 59ee8b7a88603e94b5661a8d5d088f7aa99fe049)
 2016-11-09 13:18:28 -08:00
Andrew Wang
4f25fd975e HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey. 
(cherry picked from commit e1c6ef2efa9d87fdfc7474ca63998a13a3929874)
 2016-11-08 16:33:58 -08:00
Jason Lowe
8bb264c70d YARN-5356. NodeManager should communicate physical resource capability to ResourceManager. Contributed by Inigo Goiri 
(cherry picked from commit 3f93ac0733058238a2c8f23960c986c71dca0e02)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
 2016-11-08 22:16:53 +00:00
Jian He
6cdcab9078 YARN-5716. Add global scheduler interface definition and update CapacityScheduler to use it. Contributed by Wangda Tan 2016-11-07 10:31:43 -08:00
Jason Lowe
27ed2c526a YARN-5837. NPE when getting node status of a decommissioned node after an RM restart. Contributed by Robert Kanter 
(cherry picked from commit 6bb741ff0ef208a8628bc64d6537999d4cd67955)
 2016-11-04 22:21:45 +00:00
Sunil
3eed75a9ab YARN-5802. updateApplicationPriority api in scheduler should ensure to re-insert app to correct ordering policy. Contributred by Bibin A Chundatt 
(cherry picked from commit 19b3779ae7455230ed89971141b2667eae624aab)
 2016-11-04 17:09:19 +05:30
Jason Lowe
e2917180ee YARN-4862. Handle duplicate completed containers in RMNodeImpl. Contributed by Rohith Sharma K S 
(cherry picked from commit 352cbaa7a54a94bad2bed131d6a250c5b21a7701)
 2016-11-03 14:03:56 +00:00