Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,483 Commits 326 Branches 371 Tags
Commit Graph

1446 Commits

Author SHA1 Message Date
Sunil
a636a87270 YARN-5965. Retrospect ApplicationReport#getApplicationTimeouts. Contributed by Rohith Sharma K S 
(cherry picked from commit ab923a53fcf55d4d75aa027d46e3c4a659015325)
 2016-12-08 00:34:45 +05:30
Daniel Templeton
d8b4961515 YARN-5136. Error in handling event type APP_ATTEMPT_REMOVED to the scheduler 
(Contributed by Wilfred Spiegelenburg via Daniel Templeton)
 2016-12-07 11:01:09 -08:00
Rohith Sharma K S
602c998443 YARN-5932. Retrospect moveApplicationToQueue in align with YARN-5611. Contributed by Sunil G. 2016-12-07 22:45:25 +05:30
Naganarasimha
54c5880cf7 YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena 
(cherry picked from commit f3b8ff54ab08545d7093bf8861b44ec9912e8dc3)
 2016-12-06 11:01:14 +05:30
Jian He
0f6fbfc0db YARN-5559. Analyse 2.8.0/3.0.0 jdiff reports and fix any issues. Contributed by Akira Ajisaka & Wangda Tan 
(cherry picked from commit 43ebff2e354142bddcb42755766a965ae8a503a6)
 2016-12-05 11:40:26 -08:00
Jian He
d58fca0102 YARN-5746. The state of the parentQueue and its childQueues should be synchronized. Contributed by Xuan Gong 2016-12-02 16:18:39 -08:00
Daniel Templeton
1216bad0f7 YARN-5929. Missing scheduling policy in the FS queue metric. (Contributed by Yufei Gu via Daniel Templeton) 
(cherry picked from commit 9b076327eac9da244f94e226816465e7d17fda54)
 2016-12-02 14:02:35 -08:00
Li Lu
8ffe86f780 YARN-5761. Separate QueueManager from Scheduler. (Xuan Gong via gtcarrera9) 
(cherry picked from commit 69fb70c31aa277f7fb14b05c0185ddc5cd90793d)
 2016-11-30 14:02:31 -08:00
Daniel Templeton
803aba03f2 YARN-5890. FairScheduler should log information about AM-resource-usage and max-AM-share for queues 
(Contributed by Yufei Gu via Daniel Templeton)
 2016-11-29 14:14:43 -08:00
Daniel Templeton
b8bebb8607 YARN-5774. MR Job stuck in ACCEPTED status without any progress in Fair Scheduler 
if set yarn.scheduler.minimum-allocation-mb to 0. (Contributed by Yufei Gu via Daniel Templeton)

(cherry picked from commit 25f9872be63423ada6a18481eaad2888e731fdac)
 2016-11-29 10:12:10 -08:00
Sunil
c28dc5e5a4 YARN-4206. Add Application timeouts in Application report and CLI. Contributed by Rohith Sharma K S. 
(cherry picked from commit eb0a483ed07399c7a06ed9db4bfbec382de470da)
 2016-11-24 18:27:50 +05:30
Sunil
7b676c5632 YARN-5649. Add REST endpoints for updating application timeouts. Contributed by Rohith Sharma K S 2016-11-24 17:37:36 +05:30
Rohith Sharma K S
6b47a7fb8e YARN-5920. Fix deadlock in TestRMHA.testTransitionedToStandbyShouldNotHang. Contributed by Varun Saxena. 
(cherry picked from commit e15c20edba1e9a23475ee6a4dfbadbdb8c1f668a)
 2016-11-24 12:19:19 +05:30
Rohith Sharma K S
87bb3c51d6 Reverting for fixing compilation errors in branch-2. 
Revert "YARN-5649. Add REST endpoints for updating application timeouts. Contributed by Rohith Sharma K S"

This reverts commit 43796580421346f254d066c0c58ae13937cdbd45.
 2016-11-24 09:18:07 +05:30
Jian He
4379658042 YARN-5649. Add REST endpoints for updating application timeouts. Contributed by Rohith Sharma K S 2016-11-23 16:27:12 -08:00
Rohith Sharma K S
2d068f1793 YARN-5865. Retrospect updateApplicationPriority api to handle state store exception in align with YARN-5611. Contributed by Sunil G. 2016-11-23 23:29:39 +05:30
Daniel Templeton
06f5d27bd6 YARN-5722. FairScheduler hides group resolution exceptions when assigning queue (Contributed by Wilfred Spiegelenburg via Daniel Templeton) 
(cherry picked from commit 83cc7263af632939dc3b2ee58d8f03f98ed4d96a)
 2016-11-22 11:17:39 -08:00
Rohith Sharma K S
74ac78b3c0 YARN-5375. invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures. Contributed by sandflee. 
(cherry picked from commit d65603517e52843f11cd9d3b6f6e28fca9336ee3)
 2016-11-16 15:17:46 +05:30
Xiao Chen
3497221fcc YARN-5875. TestTokenClientRMService#testTokenRenewalWrongUser fails. Contributed by Gergely Novák. 
(cherry picked from commit d5c396ccb361e1a0cf4307ce1d1b37fae0eec250)
 2016-11-15 13:58:02 -08:00
Rohith Sharma K S
528ef2407c YARN-5874. RM -format-state-store and -remove-application-from-state-store commands fail with NPE. Contributed by Varun Saxena. 
(cherry picked from commit b7070f3308fc4c6a8a9a25021562169cae87d223)
 2016-11-15 11:03:38 +05:30
Rohith Sharma K S
fad6de3080 YARN-5873. RM crashes with NPE if generic application history is enabled. Contributed by Varun Saxena. 
(cherry picked from commit 296c5de0cfee88389cf9f90263280b2034e54cd5)
 2016-11-15 11:03:27 +05:30
Karthik Kambatla
ea74c7b7ae YARN-4033. In FairScheduler, parent queues should also display queue status. (Siqi Li via kasha) 
(cherry picked from commit 79448d4ab1f298e50565b161e6605d994ee5e058)
 2016-11-13 21:39:44 -08:00
Jian He
9b0a2cb321 YARN-5825. ProportionalPreemptionalPolicy should use readLock over LeafQueue instead of synchronized block. Contributed by Sunil G 
(cherry picked from commit fad9609d13e76e9e3a4e01c96f698bb60b03807e)
 2016-11-11 15:16:43 -08:00
Wangda Tan
a76e214556 YARN-5552. Add Builder methods for common yarn API records. (Tao Jie via wangda) 
(cherry picked from commit ede1a473f5061cf40f6affc1c8c30a645c1fef6c)
 2016-11-11 13:42:45 -08:00
Naganarasimha
bde95f90e7 YARN-5545, Fix issues related to Max App in capacity scheduler. Contributed by Bibin A Chundatt 2016-11-12 00:52:35 +05:30
Eric Payne
dd5b9dabf9 YARN-4218. Metric for resource*time that was preempted. Contributed by Chang Li. 2016-11-10 23:02:42 +00:00
Karthik Kambatla
f7b25420a1 YARN-5453. FairScheduler#update may skip update demand resource of child queue/app if current demand reached maxResource. (sandflee via kasha) 
(cherry picked from commit 86ac1ad9fd65c7dd12278372b369de38dc4616db)
 2016-11-09 23:44:26 -08:00
Jian He
a422740bd7 YARN-5611. Provide an API to update lifetime of an application. Contributed by Rohith Sharma K S 2016-11-09 16:39:46 -08:00
Naganarasimha
8a2998c08c YARN-4498. Application level node labels stats to be available in REST (addendum patch). Contributed by Bibin A Chundatt. 2016-11-10 05:21:07 +05:30
Daniel Templeton
42147bbbf7 YARN-4329. [YARN-5437] Allow fetching exact reason as to why a submitted app 
is in ACCEPTED state in Fair Scheduler (Contributed by Yufei Gu)

(cherry picked from commit 59ee8b7a88603e94b5661a8d5d088f7aa99fe049)
 2016-11-09 13:18:28 -08:00
Andrew Wang
4f25fd975e HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey. 
(cherry picked from commit e1c6ef2efa9d87fdfc7474ca63998a13a3929874)
 2016-11-08 16:33:58 -08:00
Jason Lowe
8bb264c70d YARN-5356. NodeManager should communicate physical resource capability to ResourceManager. Contributed by Inigo Goiri 
(cherry picked from commit 3f93ac0733058238a2c8f23960c986c71dca0e02)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
 2016-11-08 22:16:53 +00:00
Jian He
6cdcab9078 YARN-5716. Add global scheduler interface definition and update CapacityScheduler to use it. Contributed by Wangda Tan 2016-11-07 10:31:43 -08:00
Jason Lowe
27ed2c526a YARN-5837. NPE when getting node status of a decommissioned node after an RM restart. Contributed by Robert Kanter 
(cherry picked from commit 6bb741ff0ef208a8628bc64d6537999d4cd67955)
 2016-11-04 22:21:45 +00:00
Sunil
3eed75a9ab YARN-5802. updateApplicationPriority api in scheduler should ensure to re-insert app to correct ordering policy. Contributred by Bibin A Chundatt 
(cherry picked from commit 19b3779ae7455230ed89971141b2667eae624aab)
 2016-11-04 17:09:19 +05:30
Jason Lowe
e2917180ee YARN-4862. Handle duplicate completed containers in RMNodeImpl. Contributed by Rohith Sharma K S 
(cherry picked from commit 352cbaa7a54a94bad2bed131d6a250c5b21a7701)
 2016-11-03 14:03:56 +00:00
Varun Saxena
b818773919 YARN-5815. Random failure of TestApplicationPriority.testOrderOfActivatingThePriorityApplicationOnRMRestart (Bibin A Chundatt via Varun Saxena) 2016-11-03 00:41:53 +05:30
Varun Saxena
91ddea5bfb YARN-5788. Apps not activiated and AM limit resource in UI and REST not updated after -replaceLabelsOnNode (Bibin A Chundatt via Varun Saxena) 2016-11-01 16:26:38 +05:30
Karthik Kambatla
2226ae4924 YARN-4998. Minor cleanup to UGI use in AdminService. (Daniel Templeton via kasha) 
(cherry picked from commit 733aa993134ba324c712590fa92b8ef230b0839a)
 2016-10-31 16:26:15 -07:00
Wangda Tan
cef281abe6 YARN-2009. CapacityScheduler: Add intra-queue preemption for app priority support. (Sunil G via wangda) 
(cherry picked from commit 90dd3a8148468ac37a3f2173ad8d45e38bfcb0c9)
 2016-10-31 15:23:16 -07:00
Daniel Templeton
ce13463e7a YARN-4396. Log the trace information on FSAppAttempt#assignContainer (Contributed by Yiqun Li via Daniel Templeton) 
(cherry picked from commit 2528bea67ff80fae597f10e26c5f70d601af9fb1)
 2016-10-31 13:36:24 -07:00
Daniel Templeton
adceebc400 YARN-4907. Make all MockRM#waitForState consistent. (Contributed by Yufei Gu via Daniel Templeton) 
(cherry picked from commit 155f06e0c1a5a15365fb620f3802dca8aa46c287)
 2016-10-31 13:21:45 -07:00
Naganarasimha
e4023f0d81 YARN-4498. Application level node labels stats to be available in REST. Contributed by Bibin A Chundatt. 2016-10-31 19:07:56 +05:30
Varun Saxena
0e6fcd3bd2 YARN-5773. RM recovery too slow due to LeafQueue#activateApplications (Bibin A Chundatt via Varun Saxena) 2016-10-29 13:51:02 +05:30
Jason Lowe
b85b5c6b94 YARN-4963. capacity scheduler: Make number of OFF_SWITCH assignments per heartbeat configurable. Contributed by Nathan Roberts 
(cherry picked from commit 1eae719bcead45915977aa220324650eab3c1b9e)
 2016-10-28 17:31:29 +00:00
Karthik Kambatla
950bfed1d3 YARN-4743. FairSharePolicy breaks TimSort assumption. (Zephyr Guo and Yufei Gu via kasha) 
(cherry picked from commit 4df8ed63ed93f2542e4b48f521b0cc6624ab59c1)
 2016-10-27 17:48:24 -07:00
Subru Krishnan
334fd9e83f YARN-2306. Add test for leakage of reservation metrics in fair scheduler. (Hong Zhiguo and Yufei Gu via subru). 
(cherry picked from commit b2c4f24c31e73faa8f71d44db5de3aa91e3b7d5e)
 2016-10-27 17:44:05 -07:00
Carlo Curino
58ac40b552 YARN-3432. Cluster metrics have wrong Total Memory when there is reserved memory on CS. (Brahma Reddy Battula via curino) 
(cherry picked from commit 892a8348fceb42069ea9877251c413fe33415e16)
 2016-10-27 15:22:04 -07:00
Subru Krishnan
4274600b95 YARN-3568. TestAMRMTokens should use some random port. (Takashi Ohnishi via Subru). 
(cherry picked from commit 79ae78dcbec183ab53b26de408b4517e5a151878)
 2016-10-27 15:13:01 -07:00
Varun Saxena
c34bc3d661 YARN-5308. FairScheduler: Move continuous scheduling related tests to TestContinuousScheduling (Kai Sasaki via Varun Saxena) 2016-10-28 02:23:25 +05:30