Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,525 Commits 326 Branches 371 Tags
Commit Graph

3023 Commits

Author SHA1 Message Date
Jian He
9b0a2cb321 YARN-5825. ProportionalPreemptionalPolicy should use readLock over LeafQueue instead of synchronized block. Contributed by Sunil G 
(cherry picked from commit fad9609d13e76e9e3a4e01c96f698bb60b03807e)
 2016-11-11 15:16:43 -08:00
Wangda Tan
a76e214556 YARN-5552. Add Builder methods for common yarn API records. (Tao Jie via wangda) 
(cherry picked from commit ede1a473f5061cf40f6affc1c8c30a645c1fef6c)
 2016-11-11 13:42:45 -08:00
Naganarasimha
bde95f90e7 YARN-5545, Fix issues related to Max App in capacity scheduler. Contributed by Bibin A Chundatt 2016-11-12 00:52:35 +05:30
Eric Payne
dd5b9dabf9 YARN-4218. Metric for resource*time that was preempted. Contributed by Chang Li. 2016-11-10 23:02:42 +00:00
Karthik Kambatla
f4c8dc4783 YARN-5834. TestNodeStatusUpdater.testNMRMConnectionConf compares nodemanager wait time to the incorrect value. (Chang Li via kasha) 
(cherry picked from commit 3a98419532687e4362ffc26abbc1264232820db7)
 2016-11-10 14:09:33 -08:00
Karthik Kambatla
f7b25420a1 YARN-5453. FairScheduler#update may skip update demand resource of child queue/app if current demand reached maxResource. (sandflee via kasha) 
(cherry picked from commit 86ac1ad9fd65c7dd12278372b369de38dc4616db)
 2016-11-09 23:44:26 -08:00
Varun Saxena
06c8ebdb83 YARN-5862. TestDiskFailures.testLocalDirsFailures failed (Yufei Gu via Varun Saxena) 
(cherry picked from commit c202a10923a46a6e7f7f518e6e3dbb6545dbb971)
 2016-11-10 11:43:57 +05:30
Jian He
a422740bd7 YARN-5611. Provide an API to update lifetime of an application. Contributed by Rohith Sharma K S 2016-11-09 16:39:46 -08:00
Naganarasimha
8a2998c08c YARN-4498. Application level node labels stats to be available in REST (addendum patch). Contributed by Bibin A Chundatt. 2016-11-10 05:21:07 +05:30
Daniel Templeton
42147bbbf7 YARN-4329. [YARN-5437] Allow fetching exact reason as to why a submitted app 
is in ACCEPTED state in Fair Scheduler (Contributed by Yufei Gu)

(cherry picked from commit 59ee8b7a88603e94b5661a8d5d088f7aa99fe049)
 2016-11-09 13:18:28 -08:00
Arun Suresh
4a023b4414 YARN-5833. Addendum patch to include missing changes to yarn-default.xml 
(cherry picked from commit 280357c29f867e3ef6386ea5bd0f7b7ca6fe04eb)
 2016-11-09 07:16:39 -08:00
Andrew Wang
4f25fd975e HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey. 
(cherry picked from commit e1c6ef2efa9d87fdfc7474ca63998a13a3929874)
 2016-11-08 16:33:58 -08:00
Subru Krishnan
797c7bfab9 YARN-5833. Add validation to ensure default ports are unique in Configuration. (Konstantinos Karanasos via Subru). 
(cherry picked from commit 29e3b3417c16c83dc8e753f94d7ca9957dddbedd)
 2016-11-08 14:39:09 -08:00
Jason Lowe
8bb264c70d YARN-5356. NodeManager should communicate physical resource capability to ResourceManager. Contributed by Inigo Goiri 
(cherry picked from commit 3f93ac0733058238a2c8f23960c986c71dca0e02)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
 2016-11-08 22:16:53 +00:00
Jian He
6cdcab9078 YARN-5716. Add global scheduler interface definition and update CapacityScheduler to use it. Contributed by Wangda Tan 2016-11-07 10:31:43 -08:00
Jason Lowe
27ed2c526a YARN-5837. NPE when getting node status of a decommissioned node after an RM restart. Contributed by Robert Kanter 
(cherry picked from commit 6bb741ff0ef208a8628bc64d6537999d4cd67955)
 2016-11-04 22:21:45 +00:00
Sunil
3eed75a9ab YARN-5802. updateApplicationPriority api in scheduler should ensure to re-insert app to correct ordering policy. Contributred by Bibin A Chundatt 
(cherry picked from commit 19b3779ae7455230ed89971141b2667eae624aab)
 2016-11-04 17:09:19 +05:30
Varun Vasudev
7676b89f43 YARN-5822. Log ContainerRuntime initialization error in LinuxContainerExecutor. Contributed by Sidharta Seethana. 
(cherry picked from commit 9ee0e3172ed20b951b458cbbdc4799675f2a2a51)
 2016-11-03 22:34:38 +05:30
Jason Lowe
e2917180ee YARN-4862. Handle duplicate completed containers in RMNodeImpl. Contributed by Rohith Sharma K S 
(cherry picked from commit 352cbaa7a54a94bad2bed131d6a250c5b21a7701)
 2016-11-03 14:03:56 +00:00
Naganarasimha
0c7caba087 YARN-5720. Update document for "rmadmin -replaceLabelOnNode". Contributred by Tao Jie 
(cherry picked from commit 0e75496049001b00c00901fcbfe3d4229b2fd64a)
 2016-11-03 12:50:38 +05:30
Varun Saxena
b818773919 YARN-5815. Random failure of TestApplicationPriority.testOrderOfActivatingThePriorityApplicationOnRMRestart (Bibin A Chundatt via Varun Saxena) 2016-11-03 00:41:53 +05:30
Naganarasimha
2612b509ef YARN-5697. Use CliParser to parse options in RMAdminCLI. Contributed by Tao jie 
(cherry picked from commit aacf214a274fb5149437f287c542339966c03ea5)
 2016-11-02 04:46:24 +05:30
Jason Lowe
f8d653ef34 YARN-5001. Aggregated Logs root directory is created with wrong group if nonexistent. Contributed by Haibo Chen 
(cherry picked from commit 76893a41003d57d94eb1a5f486010815266046af)
 2016-11-01 20:24:22 +00:00
Varun Saxena
91ddea5bfb YARN-5788. Apps not activiated and AM limit resource in UI and REST not updated after -replaceLabelsOnNode (Bibin A Chundatt via Varun Saxena) 2016-11-01 16:26:38 +05:30
Varun Saxena
2f0a101ab9 YARN-5805. Add isDebugEnabled check for debug logs in nodemanager (Bibin A Chundatt via Varun Saxena) 2016-11-01 16:23:12 +05:30
Karthik Kambatla
2226ae4924 YARN-4998. Minor cleanup to UGI use in AdminService. (Daniel Templeton via kasha) 
(cherry picked from commit 733aa993134ba324c712590fa92b8ef230b0839a)
 2016-10-31 16:26:15 -07:00
Varun Saxena
92b7491f23 YARN-5809. AsyncDispatcher possibly invokes multiple shutdown threads when handling exception (Jian He via Varun Saxena) 2016-11-01 04:37:56 +05:30
Wangda Tan
cef281abe6 YARN-2009. CapacityScheduler: Add intra-queue preemption for app priority support. (Sunil G via wangda) 
(cherry picked from commit 90dd3a8148468ac37a3f2173ad8d45e38bfcb0c9)
 2016-10-31 15:23:16 -07:00
Daniel Templeton
9d13a13160 YARN-5793. Trim configuration values in DockerLinuxContainerRuntime (Contributed by Tianyin Xu via Daniel Templeton) 
(cherry picked from commit f3eb4c3c738204e099cbaa03471497c46530efbf)
 2016-10-31 14:01:15 -07:00
Daniel Templeton
ce13463e7a YARN-4396. Log the trace information on FSAppAttempt#assignContainer (Contributed by Yiqun Li via Daniel Templeton) 
(cherry picked from commit 2528bea67ff80fae597f10e26c5f70d601af9fb1)
 2016-10-31 13:36:24 -07:00
Daniel Templeton
adceebc400 YARN-4907. Make all MockRM#waitForState consistent. (Contributed by Yufei Gu via Daniel Templeton) 
(cherry picked from commit 155f06e0c1a5a15365fb620f3802dca8aa46c287)
 2016-10-31 13:21:45 -07:00
Naganarasimha
e4023f0d81 YARN-4498. Application level node labels stats to be available in REST. Contributed by Bibin A Chundatt. 2016-10-31 19:07:56 +05:30
Varun Saxena
0e6fcd3bd2 YARN-5773. RM recovery too slow due to LeafQueue#activateApplications (Bibin A Chundatt via Varun Saxena) 2016-10-29 13:51:02 +05:30
Jason Lowe
b85b5c6b94 YARN-4963. capacity scheduler: Make number of OFF_SWITCH assignments per heartbeat configurable. Contributed by Nathan Roberts 
(cherry picked from commit 1eae719bcead45915977aa220324650eab3c1b9e)
 2016-10-28 17:31:29 +00:00
Jason Lowe
ea9a1be109 YARN-5767. Fix the order that resources are cleaned up from the local Public/Private caches. Contributed by Chris Trezzo 
(cherry picked from commit 1b79c417dca17bcd2e031864bc6ca07254c61b47)
 2016-10-28 15:59:57 +00:00
Jason Lowe
2ab80a148a YARN-5027. NM should clean up app log dirs after NM restart. Contributed by sandflee 
(cherry picked from commit 7146359bfd436a76585fb1f3ea93716795308cec)
 2016-10-28 15:49:54 +00:00
Jason Lowe
0f224d405f HADOOP-13770. Shell.checkIsBashSupported swallowed an interrupted exception. Contributed by Wei-Chiu Chuang 
(cherry picked from commit c017171da00a6cd71a2901c84a0298ce14a49e23)
 2016-10-28 15:06:23 +00:00
Naganarasimha
62d8536df6 Revert "YARN-5794. Fix the asflicense warnings. Contributed by Yiqun Lin" 
This reverts commit b92ad5d8577ff16557c354292044c4a20be93145.
 2016-10-28 12:58:42 +05:30
Naganarasimha
b92ad5d857 YARN-5794. Fix the asflicense warnings. Contributed by Yiqun Lin 
(cherry picked from commit 3de130252dd81b45b051f8aa01d281983ca7e541)
 2016-10-28 12:44:14 +05:30
Karthik Kambatla
950bfed1d3 YARN-4743. FairSharePolicy breaks TimSort assumption. (Zephyr Guo and Yufei Gu via kasha) 
(cherry picked from commit 4df8ed63ed93f2542e4b48f521b0cc6624ab59c1)
 2016-10-27 17:48:24 -07:00
Subru Krishnan
334fd9e83f YARN-2306. Add test for leakage of reservation metrics in fair scheduler. (Hong Zhiguo and Yufei Gu via subru). 
(cherry picked from commit b2c4f24c31e73faa8f71d44db5de3aa91e3b7d5e)
 2016-10-27 17:44:05 -07:00
Li Lu
2b0fd1f4e2 YARN-4668. Reuse objectMapper instance in Yarn. (Yiqun Lin via gtcarrera9) 
(cherry picked from commit 221582c4ab0ff1d5936f754f23da140aac656654)
 2016-10-27 15:26:05 -07:00
Carlo Curino
58ac40b552 YARN-3432. Cluster metrics have wrong Total Memory when there is reserved memory on CS. (Brahma Reddy Battula via curino) 
(cherry picked from commit 892a8348fceb42069ea9877251c413fe33415e16)
 2016-10-27 15:22:04 -07:00
Subru Krishnan
4274600b95 YARN-3568. TestAMRMTokens should use some random port. (Takashi Ohnishi via Subru). 
(cherry picked from commit 79ae78dcbec183ab53b26de408b4517e5a151878)
 2016-10-27 15:13:01 -07:00
Varun Saxena
f572448809 YARN-4388. Cleanup mapreduce.job.hdfs-servers from yarn-default.xml (Junping Du via Varun Saxena) 2016-10-28 02:24:24 +05:30
Varun Saxena
c34bc3d661 YARN-5308. FairScheduler: Move continuous scheduling related tests to TestContinuousScheduling (Kai Sasaki via Varun Saxena) 2016-10-28 02:23:25 +05:30
Jason Lowe
e50215306d YARN-4831. Recovered containers will be killed after NM stateful restart. Contributed by Siqi Li 
(cherry picked from commit 7e3c327d316b33d6a09bfd4e65e7e5384943bb1d)
 2016-10-27 20:44:17 +00:00
Jason Lowe
8935b23706 YARN-5172. Update yarn daemonlog documentation due to HADOOP-12847. Contributed by Wei-Chiu Chuang 
(cherry picked from commit b4a8fbcbbc5ea4ab3087ecf913839a53f32be113)
 2016-10-27 19:45:00 +00:00
Robert Kanter
b7f7d42c27 YARN-4456. Clean up Lint warnings in nodemanager (templedf via rkanter) 
(cherry picked from commit 6fbfb501f2e27045da5ce8f48dde881b29328b4a)
 2016-10-27 12:37:20 -07:00
Naganarasimha
15f9c18ae4 YARN-5420. Delete org.apache.hadoop.yarn.server.resourcemanager.resource.Priority as its not necessary. Contributed by Sunil G. 
(cherry picked from commit b3c15e4ef763ebc4b033c686114fe627350824ac)
 2016-10-27 18:23:27 +05:30