Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,642 Commits 326 Branches 371 Tags
Commit Graph

3467 Commits

Author SHA1 Message Date
Subru Krishnan
1889dfe409 YARN-5519. Add SubClusterId in AddApplicationHomeSubClusterResponse for Router Failover. (Ellen Hui via Subru) 
(cherry picked from commit e4f928cf9cb2a0d57131e4b7960f47a8dac7940c)
(cherry picked from commit 52558df6204954456fc27cf762e5495513928cbe)
 2017-09-21 16:00:40 -07:00
Subru Krishnan
cf15768f45 YARN-5407. In-memory based implementation of the FederationApplicationStateStore/FederationPolicyStateStore. (Ellen Hui via Subru) 
(cherry picked from commit b747d59f41f08dabe4f3a486a2dbd4bed5723867)
(cherry picked from commit 0733088c296eb29ef10399cae4ec6069a233c72e)
 2017-09-21 16:00:39 -07:00
Subru Krishnan
4c3d45a1cf YARN-5408. Compose Federation membership/application/policy APIs into an uber FederationStateStore API. (Ellen Hui via Subru). 
(cherry picked from commit 268389d0211a757439289bea84d502bed6f6ad52)
(cherry picked from commit e0c3a44396f075c6f402b000a6b286e1d9178eb8)
 2017-09-21 16:00:38 -07:00
Subru Krishnan
d0dddb4ebd YARN-3664. Federation PolicyStore internal APIs 
(cherry picked from commit 2f377263a17ec36547625dff23cc65c9aa89c58e)
(cherry picked from commit f1a508bdef5db7455588d586e898781fb47a0bf1)
 2017-09-21 16:00:38 -07:00
Subru Krishnan
07834ad678 YARN-5307. Federation Application State Store internal APIs 
(cherry picked from commit 1af5292042a336a5789b72f0fb6d5f4636d64d49)
(cherry picked from commit 047a16bf1540acc009b441640f2f5841dfed3997)
 2017-09-21 16:00:37 -07:00
Subru Krishnan
31d92378ba YARN-5390. Federation Subcluster Resolver. Contributed by Ellen Hui. 
(cherry picked from commit d3dc461a935c2af4ec3f0312ff0c26918c408467)
(cherry picked from commit d19b6773012d78cb20b92e98e13546013b6622c2)
 2017-09-21 16:00:36 -07:00
Subru Krishnan
8b6a2afa6e YARN-5406. In-memory based implementation of the FederationMembershipStateStore. Contributed by Ellen Hui. 
(cherry picked from commit 20d1d2be91bf841e0fee32522ee7b560a6f2598f)
(cherry picked from commit dcf8c52d80118edce70e310da4fc05045b4cf175)
 2017-09-21 16:00:35 -07:00
Subru Krishnan
a939d2fdb8 YARN-3662. Federation Membership State Store internal APIs. 
(cherry picked from commit 5424d0899cd68cb728f60adcefd2a95877cddbca)
(cherry picked from commit 3e22896f865ca05fd2944b24e159253fe0da034d)
 2017-09-21 16:00:35 -07:00
Subru Krishnan
194524bdf8 YARN-5300. Exclude generated federation protobuf sources from YARN Javadoc/findbugs build 
(cherry picked from commit 9d23570172ae7984f894406dfc6ad066648c61f4)
(cherry picked from commit dd75ffb26b36eaa1a18d33dacbd9ee645dcd35e4)
 2017-09-21 16:00:34 -07:00
Jason Lowe
fc6d9612c7 YARN-4266. Allow users to enter containers as UID:GID pair instead of by username. Contributed by luhuichun, Zhankun Tang, and Eric Badger. 
(cherry picked from commit bfd1a72ba8fbb06da73fede2a85e0b544d6ab43f)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2017-09-21 17:52:11 -05:00
Junping Du
e67c8347c4 YARN-7034. DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor. Contributed by Miklos Szegedi. 2017-09-21 14:12:57 -07:00
Yufei Gu
c3c8b0ffab YARN-7045. Remove FSLeafQueue#addAppSchedulable. (Contributed by Sen Zhao via Yufei Gu) 
(cherry picked from commit a92ef030a2707182e90acee644e47c8ef7e1fd8d)
 2017-09-21 12:40:37 -07:00
Jason Lowe
ae42bf73c0 YARN-6968. Hardcoded absolute pathname in DockerLinuxContainerRuntime. Contributed by Eric Badger 
(cherry picked from commit 10d7493587643b52cee5fde87eca9ef99c422a70)
 2017-09-21 08:39:31 -05:00
Sangjin Lee
a79422f717 YARN-6771. Use classloader inside configuration class to make new 
classes. Contributed by Jongyoul Lee.

(cherry picked from commit 8b336632acad10e45d029596c5e3196e1857d891)
 2017-09-20 22:46:10 -07:00
Junping Du
bdd8433d4c YARN-7186. Fix finicky TestContainerManager tests. Contributed by Arun Suresh. 
(cherry picked from commit 647b7527a9cdf4717e7dcbbb660e5812b67a17f1)
 2017-09-19 18:36:25 -07:00
Yufei Gu
64878193d5 YARN-6499. Remove the doc about Schedulable#redistributeShare(). (Contributed by Chetna Chaudhari via Yufei Gu) 
(cherry picked from commit a9019e1fb753f15c1927e3f9355996fd6544c14f)
 2017-09-19 18:29:24 -07:00
Arun Suresh
8a12a914fd YARN-7203. Add container ExecutionType into ContainerReport. (Botong Huang via asuresh) 
(cherry picked from commit 56ef5279c1db93d03b2f1e04badbfe804f548918)
 2017-09-18 15:51:19 -07:00
Xuan
6aaca3d08e YARN-6570. No logs were found for running application, running 
container. Contributed by Junping Du

(cherry picked from commit 7c732924a889cd280e972882619a1827877fbafa)
(cherry picked from commit e91447ccc9ff438dd300e2564c251b2c4b0f369a)
 2017-09-18 14:05:36 -07:00
Wangda Tan
74708d3817 YARN-7149. Cross-queue preemption sometimes starves an underserved queue. (Eric Payne via wangda) 
Change-Id: Ib269991dbebce160378e8372ee6d24849c4a5ed6
(cherry picked from commit 3dfa937a1fadfc62947755872515f549b3b15e6a)
(cherry picked from commit 38c14ef8d8a094a7101917eb77d90f5e62324f61)
 2017-09-18 13:58:51 -05:00
Jason Lowe
cd6cf0caf6 YARN-7192. Add a pluggable StateMachine Listener that is notified of NM Container State changes. Contributed by Arun Suresh 
(cherry picked from commit a4f9c7c9247801dd37beec6fc195622af1b884ad)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
 2017-09-18 10:34:53 -05:00
Junping Du
c34c823d26 YARN-7174. Add retry logic in LogsCLI when fetch running application logs. Contributed by Xuan Gong. 
(cherry picked from commit 1a84c24b0cf6674fa755403971fa57d8e412b320)
 2017-09-15 15:34:00 -07:00
Robert Kanter
4e36452d02 YARN-7162. Remove XML excludes file format (rkanter) 2017-09-15 09:53:55 -07:00
Sunil G
317af87add Addendum patch for YARN-7163 in branch-2 2017-09-15 11:34:14 +05:30
Arun Suresh
b93a23f411 YARN-6059. Update paused container state in the NM state store. (Hitesh Sharma via asuresh) 
(cherry picked from commit 66ca0a65408521d5f9b080dd16b353b49fb8eaea)
 2017-09-14 10:30:30 -07:00
Arun Suresh
52bf458616 YARN-5216. Expose configurable preemption policy for OPPORTUNISTIC containers running on the NM. (Hitesh Sharma via asuresh) 
(cherry picked from commit 4f8194430fc6a69d9cc99b78828fd7045d5683e8)
 2017-09-14 10:29:54 -07:00
Arun Suresh
1af9f962c9 YARN-5292. NM Container lifecycle and state transitions to support for PAUSED container state. (Hitesh Sharma via asuresh) 
(cherry picked from commit 864fbacd4548004b1de8b0812627976acd22aff5)
 2017-09-14 10:26:34 -07:00
Junping Du
7e6cc17133 YARN-7194. Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat. Contributed by Xuan Gong. 
(cherry picked from commit c92c1d521eadfd8a4cd8205cc6aee74816f353f4)
 2017-09-13 22:18:39 -07:00
Eric Payne
fb3d53e001 YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable for container. Contributed by Jason Lowe. 
(cherry picked from commit 3860be7961580ac20dd505d665b580f0a04ac4f8)
 2017-09-13 15:07:27 -05:00
Nathan Roberts
1d33290a0a YARN-7014. Fix off-by-one error causing heap corruption (Jason Lowe via nroberts) 
(cherry picked from commit d265459024b8e5f5eccf421627f684ca8f162112)
 2017-09-13 09:47:55 -07:00
Varun Vasudev
4647761341 YARN-5719. Enforce a C standard for native container-executor. Contributed by Chris Douglas. 
(cherry picked from commit 972da46cb48725ad49d3e0a033742bd1a8228f51)
 2017-09-13 18:15:02 +05:30
Robert Kanter
055bde9806
YARN-5753. fix NPE in AMRMClientImpl.getMatchingRequests() (haibochen via rkanter) 
(cherry picked from commit 44fdf009642ae4e99b15f89ec0ca43834f991ef3)
 2017-09-13 16:48:43 +09:00
Arun Suresh
0d32a38ac3 YARN-7185. [Addendum patch] Minor javadoc and checkstyle fix. 
(cherry picked from commit a1c398e0dc85e424b45c50a3c3a0a784c04ed737)
 2017-09-12 17:05:01 -07:00
Arun Suresh
d735c1f26b YARN-7185. ContainerScheduler should only look at availableResource for GUARANTEED containers when OPPORTUNISTIC container queuing is enabled. (Wangda Tan via asuresh) 
(cherry picked from commit 2ae72692fc370267141a1ee55ef372ff62302b54)
 2017-09-12 16:33:05 -07:00
Arun Suresh
7803a93fff YARN-6706. Refactor ContainerScheduler to make oversubscription change easier. (Haibo Chen via asuresh) 
(cherry picked from commit 5b007921cdf01ecc8ed97c164b7d327b8304c529)
 2017-09-12 16:33:05 -07:00
Yufei Gu
1d40bc225a YARN-7057. FSAppAttempt#getResourceUsage doesn't need to consider resources queued for preemption. (Contributed by Karthik via Yufei Gu) 
(cherry picked from commit 82c5dd1d508292ed88eda0f5356776437ba67d4c)
 2017-09-12 16:14:31 -07:00
Wangda Tan
10803bf08d YARN-7173. Container update RM-NM communication fix for backward compatibility. (Arun Suresh via wangda) 
Change-Id: Ia7d61e0d9df1e703bc983a31e6856e84a5a0521c
 2017-09-11 20:56:17 -07:00
Daniel Templeton
fac4172e02 YARN-6622. Document Docker work as experimental (Contributed by Varun Vasudev) 
(cherry picked from commit 6651cbcc72d92caf86b744fa76fba513b36b12c7)
 2017-09-11 16:16:15 -07:00
Daniel Templeton
2e5a48afe3 Revert "YARN-6022. Document Docker work as experimental (Contributed by Varun Vasudev)" 
This reverts commit 1acd0047275ec8a15ee18f76dbb392d31e8f607a.
 2017-09-11 16:15:06 -07:00
Daniel Templeton
1acd004727 YARN-6022. Document Docker work as experimental (Contributed by Varun Vasudev) 
(cherry picked from commit 66be6c848b47ae7248b0658e3bb2149d2387cebd)
 2017-09-11 15:48:21 -07:00
Wangda Tan
c093e4622c YARN-6726. Fix issues with docker commands executed by container-executor. (Shane Kumpf via wangda) 
Change-Id: I8fa7d70526505d4082ac9b51cd6066376a599108
 2017-09-11 11:34:51 -07:00
Yufei Gu
416a44004f YARN-6799. Remove the duplicated code in CGroupsHandlerImp.java. (Contributed by Weiyuan via Yufei Gu) 
(cherry picked from commit 5ab21dfe9c8e14585db71e02bcb021a526507f98)
 2017-09-10 23:24:14 -07:00
Xuan
02ae3f8202 YARN-7072. Add a new log aggregation file format controller (xgong) 2017-09-10 23:18:31 -07:00
Sunil G
23dc6ef8b0 YARN-7163. RMContext need not to be injected to webapp and other Always Running services. Contributed by Rohith Sharma K S. 2017-09-11 10:34:59 +05:30
Subru Krishnan
2c051dbe9e YARN-5330. SharingPolicy enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru). 
(cherry picked from commit fa6137501c1499ae33f6e0e2adc31671a7e782dc)
 2017-09-08 14:44:46 -07:00
Chris Douglas
2f8825588e YARN-5164. Use plan RLE to improve CapacityOverTimePolicy efficiency 
(cherry picked from commit d383bfdcd40c2315197fa5f85c3e5cb3bb83167f)
 2017-09-08 14:44:34 -07:00
Nathan Roberts
b3a478832a YARN-6219. Fix conf usage in NMWebServer unit tests. Contributed by Jason Lowe. 2017-09-08 16:01:06 -05:00
Arun Suresh
2b09cec218 YARN-6849. NMContainerStatus should have the Container ExecutionType. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 1f53ae79728065417c6a99eb6fcc8d3a080ab4cc)
 2017-09-08 09:26:50 -07:00
Sunil G
a39829c456 YARN-6600. Introduce default and max lifetime of application at LeafQueue level. Contributed by Rohith Sharma K S. 2017-09-08 19:20:52 +05:30
Jason Lowe
c004d57d47 YARN-6930. Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager. Contributed by Shane Kumpf 2017-09-07 16:24:55 -05:00
Sunil G
ef342455cb YARN-6248. user is not removed from UsersManager’s when app is killed with pending container requests. Contributed by Eric Payne. 
(cherry picked from commit c1386a90d7ff65b57be00a7245de82c31abb4665)
 2017-09-07 14:47:07 -05:00