Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,708 Commits 326 Branches 371 Tags
Commit Graph

818 Commits

Author SHA1 Message Date
Sunil G
7b2b6caf12 YARN-6118. Add javadoc for Resources.isNone. Contributed by Andres Perez. 
(cherry picked from commit 3a0a0a485a4145e3bd2e5946731ee1d901b51ead)
 2017-02-10 15:54:59 +05:30
Junping Du
33e7c3044f YARN-6145. Improve log message on fail over. Contributed by Jian He. 
(cherry picked from commit eec52e158b7bc14b2d3d53512323ba05e15e09e3)
 2017-02-08 02:15:11 -08:00
Junping Du
ab41da7e64 YARN-6108. Improve AHS webservice to accept NM address as a parameter to get container logs. Contributed by Xuan Gong. 
(cherry picked from commit 9dbfab1284015af6983b4a55b79c9e65bc0194e5)
 2017-02-07 02:14:09 -08:00
Junping Du
b7d59e790b HADOOP-12097. Allow port range to be specified while starting webapp. Contributed by Varun Saxena. 
(cherry picked from commit cce35c38159b23eb55204b3c9afcaa3215f4f4ef)

Conflicts:

	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java
 2017-02-05 20:13:49 -08:00
Junping Du
c699ce70ea YARN-6100. Improve YARN webservice to output aggregated container logs. Contributed by Xuan Gong. 2017-02-02 00:45:27 -08:00
Daniel Templeton
443f2803ac YARN-4997. Update fair scheduler to use pluggable auth provider (Contributed by Tao Jie via Daniel Templeton) 
(cherry picked from commit b3befc021b0e2d63d1a3710ea450797d1129f1f5)
 2017-01-26 14:07:46 -08:00
Sangjin Lee
c4d3b28c2d YARN-6117. SharedCacheManager does not start up. Contributed by Chris Trezzo. 
(cherry picked from commit dc6ec9704829f180ce0e182c436fe1a435744c88)
 2017-01-25 15:36:07 -08:00
Karthik Kambatla
0ceb7149d9 YARN-4752. Improved preemption in FairScheduler. (kasha) 2017-01-25 10:45:02 -08:00
Junping Du
8027c3e8b9 YARN-6099. Improve webservice to list aggregated log files. Contributed by Xuan Gong. 2017-01-24 15:28:52 -08:00
Jason Lowe
94b326f2c8 YARN-5910. Support for multi-cluster delegation tokens. Contributed by Jian He 
(cherry picked from commit 69fa81679f59378fd19a2c65db8019393d7c05a2)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java
 2017-01-23 17:23:32 +00:00
bibinchundatt
580275628b YARN-6057. yarn.scheduler.minimum-allocation-* descriptions are incorrect about behavior when a request is out of bounds. Contributed by Julia Sommer. 
(cherry picked from commit a9d946d4df91c29ea86851bb1c5046e95dc80406)
 2017-01-17 14:18:41 +05:30
Daniel Templeton
718ae2d477 YARN-5849. Automatically create YARN control group for pre-mounted cgroups (Contributed by Miklos Szegedi via Daniel Templeton) 
(cherry picked from commit 0fffebe51a91023c4949ab560f5c828f1b568072)
 2017-01-11 15:36:18 -08:00
Junping Du
ba6a013341 YARN-6079. Fix simple spelling errors in yarn test code. Contributed by vijay. 
(cherry picked from commit 4db119b7b55213929e5b86f2abb0ed84a21719b5)
 2017-01-10 18:22:45 -08:00
Naganarasimha
ea38256fab YARN-6015. AsyncDispatcher thread name can be set to improved debugging. Contributed by Ajith S. 2017-01-10 15:10:54 +05:30
Wangda Tan
447f71326b YARN-3955. Support for application priority ACLs in queues of CapacityScheduler. (Sunil G via wangda) 
(cherry picked from commit 287d3d6804a869723ae36605a3c2d2b3eae3941e)
 2017-01-09 08:42:38 -08:00
Junping Du
f615c1fbbc Addendum patch to YARN-3866: fix incompatible API change. Contributed by Wangda Tan. 2017-01-07 19:58:42 -08:00
Arun Suresh
81da7d1d30 YARN-5938. Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes (asuresh) 
(cherry picked from commit ac1e5d4f77e3b9df8dcacb0b1f72eecc27931eb8)
 2017-01-06 11:25:49 -08:00
Arun Suresh
2b4d3e8506 YARN-5646. Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 2273a74c1f3895163046cca09ff5e983df301d22)
(cherry picked from commit 9e17ffe599e12c59c48f4355de645c31a4735d04)
 2017-01-06 11:15:10 -08:00
Arun Suresh
3d3bb30a25 YARN-5982. Simplify opportunistic container parameters and metrics. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit b0aace21b1ef3436ba9d516186208fee9a9ceef2)
(cherry picked from commit 3faa0b0d682687099b253feee9b41340edef7111)
 2017-01-06 11:15:09 -08:00
Arun Suresh
368565f0ce YARN-4597. Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle. (asuresh) 
(cherry picked from commit 3219b7b4ac7d12aee343f6ab2980b3357fc618b6)
(cherry picked from commit 49921678858a17ca58ca2ae704ced1ff584e0859)
 2017-01-06 11:15:09 -08:00
Daniel Templeton
8cb7aa2b52 YARN-5931. Document timeout interfaces CLI and REST APIs (Contributed by Rohith Sharma K S via Daniel Templeton) 
(cherry picked from commit 165d01a73e5f328108a9e876c7a751a3c2cec4a8)
 2016-12-30 08:56:03 -08:00
Daniel Templeton
47f0a0e15b HADOOP-13932. Fix indefinite article in comments (Contributed by LiXin Ge via Daniel Templeton) 2016-12-28 07:03:51 -08:00
Robert Kanter
88597991b0 YARN-5257. Fix unreleased resources and null dereferences (yufeigu via rkanter) 
(cherry picked from commit 9262797e86453fc04b7ca3710b73b21fcdf9e6b4)
 2016-12-27 14:14:41 -08:00
Sunil G
23bd68a4a4 YARN-5970. Validate application update timeout request parameters. Contributed by Rohith Sharma K S. 2016-12-12 11:59:14 +05:30
Robert Kanter
9cba7c6d71 YARN-5963. Spelling errors in logging and exceptions for node manager, client, web-proxy, common, and app history code (gsohn via rkanter) 
(cherry picked from commit 72fe54684198b7df5c5fb2114616dff6d17a4402)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/impl/TimelineClientImpl.java
 2016-12-07 15:09:37 -08:00
Sunil
a636a87270 YARN-5965. Retrospect ApplicationReport#getApplicationTimeouts. Contributed by Rohith Sharma K S 
(cherry picked from commit ab923a53fcf55d4d75aa027d46e3c4a659015325)
 2016-12-08 00:34:45 +05:30
Jian He
0f6fbfc0db YARN-5559. Analyse 2.8.0/3.0.0 jdiff reports and fix any issues. Contributed by Akira Ajisaka & Wangda Tan 
(cherry picked from commit 43ebff2e354142bddcb42755766a965ae8a503a6)
 2016-12-05 11:40:26 -08:00
Jason Lowe
6b17d449bf YARN-5915. ATS 1.5 FileSystemTimelineWriter causes flush() to be called after every event write. Contributed by Atul Sikaria 2016-12-02 16:58:43 +00:00
Robert Kanter
99b046f8a9 MAPREDUCE-6787. Allow job_conf.xml to be downloadable on the job overview page in JHS (haibochen via rkanter) 
(cherry picked from commit c87b3a448a00df97149a4e93a8c39d9ad0268bdb)
 2016-12-01 17:30:02 -08:00
Daniel Templeton
b8bebb8607 YARN-5774. MR Job stuck in ACCEPTED status without any progress in Fair Scheduler 
if set yarn.scheduler.minimum-allocation-mb to 0. (Contributed by Yufei Gu via Daniel Templeton)

(cherry picked from commit 25f9872be63423ada6a18481eaad2888e731fdac)
 2016-11-29 10:12:10 -08:00
Sunil
c28dc5e5a4 YARN-4206. Add Application timeouts in Application report and CLI. Contributed by Rohith Sharma K S. 
(cherry picked from commit eb0a483ed07399c7a06ed9db4bfbec382de470da)
 2016-11-24 18:27:50 +05:30
Naganarasimha
2d94f938c5 YARN-4330. MiniYARNCluster is showing multiple Failed to instantiate default resource calculator warning messages. Contributed by Varun Saxena 
(cherry picked from commit 3541ed80685f25486f33ef0f553854ccbdeb51d4)
 2016-11-23 14:14:56 +05:30
Naganarasimha
b96ed43897 YARN-5911. DrainDispatcher does not drain all events on stop even if setDrainEventsOnStop is true. Contributed by Varun Saxena. 
(cherry picked from commit 466756416214a4bbc77af8a29da1a33e01106864)
 2016-11-23 09:36:43 +05:30
Rohith Sharma K S
74ac78b3c0 YARN-5375. invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures. Contributed by sandflee. 
(cherry picked from commit d65603517e52843f11cd9d3b6f6e28fca9336ee3)
 2016-11-16 15:17:46 +05:30
Mingliang Liu
4e0fcff8ab HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu 
(cherry picked from commit 5af572b6443715b7a741296c1bd520a1840f9a7c)
 2016-11-15 11:36:08 -08:00
Wangda Tan
a76e214556 YARN-5552. Add Builder methods for common yarn API records. (Tao Jie via wangda) 
(cherry picked from commit ede1a473f5061cf40f6affc1c8c30a645c1fef6c)
 2016-11-11 13:42:45 -08:00
Eric Payne
dd5b9dabf9 YARN-4218. Metric for resource*time that was preempted. Contributed by Chang Li. 2016-11-10 23:02:42 +00:00
Jian He
a422740bd7 YARN-5611. Provide an API to update lifetime of an application. Contributed by Rohith Sharma K S 2016-11-09 16:39:46 -08:00
Arun Suresh
4a023b4414 YARN-5833. Addendum patch to include missing changes to yarn-default.xml 
(cherry picked from commit 280357c29f867e3ef6386ea5bd0f7b7ca6fe04eb)
 2016-11-09 07:16:39 -08:00
Andrew Wang
4f25fd975e HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey. 
(cherry picked from commit e1c6ef2efa9d87fdfc7474ca63998a13a3929874)
 2016-11-08 16:33:58 -08:00
Jason Lowe
8bb264c70d YARN-5356. NodeManager should communicate physical resource capability to ResourceManager. Contributed by Inigo Goiri 
(cherry picked from commit 3f93ac0733058238a2c8f23960c986c71dca0e02)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
 2016-11-08 22:16:53 +00:00
Varun Saxena
92b7491f23 YARN-5809. AsyncDispatcher possibly invokes multiple shutdown threads when handling exception (Jian He via Varun Saxena) 2016-11-01 04:37:56 +05:30
Naganarasimha
62d8536df6 Revert "YARN-5794. Fix the asflicense warnings. Contributed by Yiqun Lin" 
This reverts commit b92ad5d8577ff16557c354292044c4a20be93145.
 2016-10-28 12:58:42 +05:30
Naganarasimha
b92ad5d857 YARN-5794. Fix the asflicense warnings. Contributed by Yiqun Lin 
(cherry picked from commit 3de130252dd81b45b051f8aa01d281983ca7e541)
 2016-10-28 12:44:14 +05:30
Li Lu
2b0fd1f4e2 YARN-4668. Reuse objectMapper instance in Yarn. (Yiqun Lin via gtcarrera9) 
(cherry picked from commit 221582c4ab0ff1d5936f754f23da140aac656654)
 2016-10-27 15:26:05 -07:00
Varun Saxena
f572448809 YARN-4388. Cleanup mapreduce.job.hdfs-servers from yarn-default.xml (Junping Du via Varun Saxena) 2016-10-28 02:24:24 +05:30
Subru Krishnan
81595a127f YARN-5711. Propogate exceptions back to client when using hedging RM failover provider. 
(cherry picked from commit 0a166b13472213db0a0cd2dfdaddb2b1746b3957)
 2016-10-24 19:02:03 -07:00
Varun Saxena
e9a58691ab YARN-5599. Publish AM launch command to ATS (Rohith Sharma K S via Varun Saxena) 2016-10-14 23:32:29 +05:30
Jason Lowe
28bd08798d YARN-5551. Ignore file backed pages from memory computation when smaps is enabled. Contributed by Rajesh Balamohan 
(cherry picked from commit ecb51b857ac7faceff981b2b6f22ea1af0d42ab1)
 2016-10-11 15:14:08 +00:00
Chris Douglas
539c3e4dc0 YARN-5702. Refactor TestPBImplRecords for reuse in other YARN modules. Contributed by Subru Krishnan 
(cherry picked from commit d65b957776c4f055f82549a610fd1e8494580fe6)
 2016-10-05 12:19:21 -07:00