Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,269 Commits 326 Branches 371 Tags
Commit Graph

936 Commits

Author SHA1 Message Date
Jian He
615c912b61 YARN-3673. Create a FailoverProxy for Federation services. Contributed by Subru Krishnan 
(cherry picked from commit 3307564a5f8c8abc5fe84efcd05ee0f7dfdd921c)
(cherry picked from commit bdfad4523f1a5a776e20773c371ce99d0c538ac1)
 2017-09-21 16:13:06 -07:00
Jian He
fb3f9b48fb YARN-3674. Create Facade for Federation State and Policy Store. Contributed by Subru Krishnan 
(cherry picked from commit 5e976cd2b90ccf1bccb6039edf14140677804c4e)
(cherry picked from commit 5c843823972d030555e25729ae7b4b8fbafb59fd)
 2017-09-21 16:12:34 -07:00
Subru Krishnan
31d92378ba YARN-5390. Federation Subcluster Resolver. Contributed by Ellen Hui. 
(cherry picked from commit d3dc461a935c2af4ec3f0312ff0c26918c408467)
(cherry picked from commit d19b6773012d78cb20b92e98e13546013b6622c2)
 2017-09-21 16:00:36 -07:00
Jason Lowe
fc6d9612c7 YARN-4266. Allow users to enter containers as UID:GID pair instead of by username. Contributed by luhuichun, Zhankun Tang, and Eric Badger. 
(cherry picked from commit bfd1a72ba8fbb06da73fede2a85e0b544d6ab43f)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2017-09-21 17:52:11 -05:00
Sangjin Lee
a79422f717 YARN-6771. Use classloader inside configuration class to make new 
classes. Contributed by Jongyoul Lee.

(cherry picked from commit 8b336632acad10e45d029596c5e3196e1857d891)
 2017-09-20 22:46:10 -07:00
Arun Suresh
8a12a914fd YARN-7203. Add container ExecutionType into ContainerReport. (Botong Huang via asuresh) 
(cherry picked from commit 56ef5279c1db93d03b2f1e04badbfe804f548918)
 2017-09-18 15:51:19 -07:00
Jason Lowe
cd6cf0caf6 YARN-7192. Add a pluggable StateMachine Listener that is notified of NM Container State changes. Contributed by Arun Suresh 
(cherry picked from commit a4f9c7c9247801dd37beec6fc195622af1b884ad)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
 2017-09-18 10:34:53 -05:00
Arun Suresh
52bf458616 YARN-5216. Expose configurable preemption policy for OPPORTUNISTIC containers running on the NM. (Hitesh Sharma via asuresh) 
(cherry picked from commit 4f8194430fc6a69d9cc99b78828fd7045d5683e8)
 2017-09-14 10:29:54 -07:00
Junping Du
7e6cc17133 YARN-7194. Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat. Contributed by Xuan Gong. 
(cherry picked from commit c92c1d521eadfd8a4cd8205cc6aee74816f353f4)
 2017-09-13 22:18:39 -07:00
Xuan
02ae3f8202 YARN-7072. Add a new log aggregation file format controller (xgong) 2017-09-10 23:18:31 -07:00
Sunil G
a39829c456 YARN-6600. Introduce default and max lifetime of application at LeafQueue level. Contributed by Rohith Sharma K S. 2017-09-08 19:20:52 +05:30
Jason Lowe
c004d57d47 YARN-6930. Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager. Contributed by Shane Kumpf 2017-09-07 16:24:55 -05:00
Junping Du
1ef48f1d4b YARN-7144. Log Aggregation controller should not swallow the exceptions when it calls closeWriter and closeReader. Contributed by Xuan Gong. 
(cherry picked from commit 22de9449f8aa72c5b0bb586b8253390773502189)
 2017-09-06 14:54:29 -07:00
Junping Du
eaf134c46c YARN-7148. TestLogsCLI fails in trunk and branch-2 and javadoc error. Contributed by Xuan Gong. 
(cherry picked from commit 63231a2a3008794c5ebcbc9d3855638fc5f28216)
 2017-09-06 13:20:58 -07:00
Junping Du
119220b88f YARN-6877. Create an abstract log reader for extendability. Contributed by Xuan Gong. 2017-09-01 03:04:55 -07:00
Junping Du
b906bddd95 YARN-7037. Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices. Contributed by Tao Yang. 
(cherry picked from commit ad45d19998c1b0da25754d0016854046731fa623)
 2017-08-29 15:17:21 -07:00
Junping Du
e12358c5ce YARN-6876. Create an abstract log writer for extendability. Contributed by Xuan Gong. 
(cherry picked from commit c2cb7ea1ef6532020b69031dbd18b0f9b8369f0f)
 2017-08-24 15:27:40 -07:00
Arun Suresh
f8f193c8e8 YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 8410d862d3a72740f461ef91dddb5325955e1ca5)
 2017-08-20 08:31:17 -07:00
Subru Krishnan
a70efb6138 HADOOP-14741. Refactor curator based ZooKeeper communication into common library. (Íñigo Goiri via Subru). 2017-08-14 11:03:50 -07:00
Junping Du
e9c21d14b4 YARN-6811. [ATS1.5] All history logs should be kept under its own User Directory. Contributed by Rohith Sharma K S. 2017-08-08 12:45:55 -07:00
Daniel Templeton
675d368950 YARN-6757. Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path 
(Contributed by Miklos Szegedi via Daniel Templeton)

(cherry picked from commit 47b145b9b4e81d781891abce8a6638f0b436acc4)
 2017-08-08 11:12:34 -07:00
Arun Suresh
4cfd248a72 YARN-6777. Support for ApplicationMasterService processing chain of interceptors. (asuresh) 
(cherry picked from commit 077fcf6a96e420e7f36350931722b8603d010cf1)
 2017-08-04 16:50:21 -07:00
Arun Suresh
2d62af6545 YARN-5966. AMRMClient changes to support ExecutionType update. (asuresh) 
(cherry picked from commit aaf106fde35ec97e2e2ea4d7a67434038c4273ac)
 2017-08-04 16:19:44 -07:00
Arun Suresh
88d6adf419 YARN-5977. ContainerManagementProtocol changes to support change of container ExecutionType. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 35dc7829236f92054d5ce6ea78d3a44ca6c8f3d3)
 2017-08-03 21:18:43 -07:00
Yufei Gu
ce609d06c0 YARN-6832. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu) 2017-08-03 13:52:58 -07:00
Jason Lowe
e76e7b9f0b YARN-6837. Null LocalResource visibility or resource type can crash the nodemanager. Contributed by Jinjiang Ling 
(cherry picked from commit c8df3668ecc37c2d58cad35520a762eaec3c8539)
 2017-07-20 11:06:39 -05:00
Sunil G
d83e8712f6 YARN-5892. Support user-specific minimum user limit percentage in Capacity Scheduler. Contributed by Eric Payne. 
(cherry picked from commit ca13b224b2feb9c44de861da9cbba8dd2a12cb35)
 2017-07-13 14:50:28 -05:00
Wangda Tan
cbb5f601f1 YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Sunil G via wangda) 
Change-Id: I9b19f69788068be05b3295247cdd7b972f8a573c
(cherry picked from commit c583ab02c730be0a63d974039a78f2dc67dc2db6)
 2017-07-13 10:44:07 -05:00
Xuan
edc0aa95f1 YARN-6689. PlacementRule should be configurable. (Jonathan Hung via xgong) 2017-07-12 15:45:51 -07:00
Jason Lowe
401b191a3c YARN-6797. TimelineWriter does not fully consume the POST response. Contributed by Jason Lowe 
(cherry picked from commit 655110393bf90a7dac5d85f831ea225071f5f3c2)
 2017-07-12 15:54:17 -05:00
Eric Payne
e6cdf770ca Revert "YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Contributed by Sunil G)" 
This reverts commit eda4ac07c1835031aca7e27cc673f1c5913813bb.
Commit eda4ac07c1835031aca7e27cc673f1c5913813bb was a separate patch from trunk rather than a cherry-pick. I will cherryp-ick dependencies and then cherry-pick the trunk commit for YARN-2113.
 2017-07-12 08:56:40 -05:00
Naganarasimha
b08ef44d9e YARN-6428. Queue AM limit is not honored in CS always. Contributed by Bibin A Chundatt. 
(cherry picked from commit 626d730bfc4c1f6ea54e8a246a0a676cdb74937f)
 2017-07-09 17:27:50 +05:30
Eric Payne
eda4ac07c1 YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Contributed by Sunil G) 2017-07-06 10:43:00 -05:00
Daniel Templeton
15242786b6 YARN-6743. yarn.resourcemanager.zk-max-znode-size.bytes description needs spaces in yarn-default.xml 
(Contributed by Lori Loberg via Daniel Templeton)

(cherry picked from commit 25d891a784304fcf02f57bc7984c31af45003553)
 2017-06-28 17:17:56 -07:00
Naganarasimha
92e266c2b6 YARN-5006. ResourceManager quit due to ApplicationStateData exceed the limit size of znode in zk. Contributed by Bibin A Chundatt. 2017-06-24 23:14:51 +05:30
Naganarasimha
578318473c YARN-6680. Avoid locking overhead for NO_LABEL lookups. Contributed by Daryn Sharp 
(cherry picked from commit ee89ac84e68d3e181b75c63f74a0444f9d28146f)
 2017-06-19 22:45:21 +05:30
Naganarasimha
3106f256e2 YARN-6517. Fix warnings from Spotbugs in hadoop-yarn-common(addendum). Contributed by Weiwei Yang. 
(cherry picked from commit 7582dedad1c73eabdc3eeece0a3a860e7bb33c1a)
 2017-06-18 19:56:11 +05:30
Eric Payne
33605e0896 YARN-6585. RM fails to start when upgrading from 2.7 for clusters with node labels. Contributed by Sunil G. 
(cherry picked from commit 5578af860335ae44c9780082508c3dcf726f60fc)
 2017-06-09 16:18:16 -05:00
Daniel Templeton
c5b28c03a1 YARN-6679. Reduce Resource instance overhead via non-PBImpl 
(Contributed by Daryn Sharp via Daniel Templeton)
 2017-06-08 10:41:32 -07:00
Robert Kanter
2719cc0388 YARN-6602. Impersonation does not work if standby RM is contacted first (rkanter) 
(cherry picked from commit 9855225a790e66835c79816712f432ffa8c599b4)
 2017-05-19 15:43:43 -07:00
Haibo Chen
86fcb8a66c YARN-6457. Allow custom SSL configuration to be supplied in WebApps. (Sanjay M Pujare via Haibo Chen) 
(cherry picked from commit 1769b12a773dc6c83f13663a77da33fa78878730)
 2017-05-08 15:47:01 -07:00
Jason Lowe
6ca0c134e4 YARN-3839. Quit throwing NMNotYetReadyException. Contributed by Manikandan R 
(cherry picked from commit 424887ecb7d11a72837f2757ed3ff9e0fe8c5b5d)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManagerRecovery.java
 2017-05-08 17:25:11 -05:00
Steve Loughran
5ca4f0f82e YARN-524 TestYarnVersionInfo failing if generated properties doesn't include an SVN URL 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463300 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 7c65dbb9385567fe1d8e280b81ecb97b109d18a8)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2017-05-01 08:58:12 -05:00
Naganarasimha
29370e7784 YARN-6517. Fix warnings from Spotbugs in hadoop-yarn-common. Contributed by Weiwei Yang 
(cherry picked from commit 4b5bd73ac53f7de4899b5b70078249ad20216048)
 2017-05-01 16:35:45 +05:30
Daniel Templeton
a8512d5aeb YARN-2962. ZKRMStateStore: Limit the number of znodes under a znode (Contributed by Varun Sexena via Daniel Templeton) 
(cherry picked from commit 2e52789edf68016e7a3f450164f8bd3d8e6cb210)
 2017-04-28 13:39:11 -07:00
Haibo Chen
894521673b YARN-6510. Fix profs stat file warning caused by process names that includes parenthesis. (Wilfred Spiegelenburg via Haibo Chen) 
(cherry picked from commit 4f3ca0396a810f54f7fd0489a224c1bb13143aa4)
 2017-04-26 11:47:23 -07:00
Sunil G
a9495b206b YARN-6164. Expose Queue Configurations per Node Label through YARN client api. Contributed by Benson Qiu. 2017-04-24 12:16:18 +05:30
Junping Du
d77cada182 YARN-3760. FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close(). Contributed by Haibo Chen. 
(cherry picked from commit c26ccf1adb3a72df3f68e1150b86b813c691203a)
(cherry picked from commit 9d9087a67ad599e678948d5c1036e51cb8cf5e3a)
 2017-04-12 13:54:58 -07:00
Jason Lowe
84b7f2e956 YARN-6288. Exceptions during aggregated log writes are mishandled. Contributed by Akira Ajisaka 
(cherry picked from commit 1b081ca27e05e97d8b7d284ca24200d43763e481)
 2017-04-06 16:25:41 -05:00
Jason Lowe
903278559a YARN-6403. Invalid local resource request can raise NPE and make NM exit. Contributed by Tao Yang 
(cherry picked from commit e8071aa249c7b21b1de084ee5a9ca2a44efd3bf0)
 2017-04-05 12:31:48 -05:00