Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,633 Commits 326 Branches 371 Tags
Commit Graph

3458 Commits

Author SHA1 Message Date
Jason Lowe
fc6d9612c7 YARN-4266. Allow users to enter containers as UID:GID pair instead of by username. Contributed by luhuichun, Zhankun Tang, and Eric Badger. 
(cherry picked from commit bfd1a72ba8fbb06da73fede2a85e0b544d6ab43f)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2017-09-21 17:52:11 -05:00
Junping Du
e67c8347c4 YARN-7034. DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor. Contributed by Miklos Szegedi. 2017-09-21 14:12:57 -07:00
Yufei Gu
c3c8b0ffab YARN-7045. Remove FSLeafQueue#addAppSchedulable. (Contributed by Sen Zhao via Yufei Gu) 
(cherry picked from commit a92ef030a2707182e90acee644e47c8ef7e1fd8d)
 2017-09-21 12:40:37 -07:00
Jason Lowe
ae42bf73c0 YARN-6968. Hardcoded absolute pathname in DockerLinuxContainerRuntime. Contributed by Eric Badger 
(cherry picked from commit 10d7493587643b52cee5fde87eca9ef99c422a70)
 2017-09-21 08:39:31 -05:00
Sangjin Lee
a79422f717 YARN-6771. Use classloader inside configuration class to make new 
classes. Contributed by Jongyoul Lee.

(cherry picked from commit 8b336632acad10e45d029596c5e3196e1857d891)
 2017-09-20 22:46:10 -07:00
Junping Du
bdd8433d4c YARN-7186. Fix finicky TestContainerManager tests. Contributed by Arun Suresh. 
(cherry picked from commit 647b7527a9cdf4717e7dcbbb660e5812b67a17f1)
 2017-09-19 18:36:25 -07:00
Yufei Gu
64878193d5 YARN-6499. Remove the doc about Schedulable#redistributeShare(). (Contributed by Chetna Chaudhari via Yufei Gu) 
(cherry picked from commit a9019e1fb753f15c1927e3f9355996fd6544c14f)
 2017-09-19 18:29:24 -07:00
Arun Suresh
8a12a914fd YARN-7203. Add container ExecutionType into ContainerReport. (Botong Huang via asuresh) 
(cherry picked from commit 56ef5279c1db93d03b2f1e04badbfe804f548918)
 2017-09-18 15:51:19 -07:00
Xuan
6aaca3d08e YARN-6570. No logs were found for running application, running 
container. Contributed by Junping Du

(cherry picked from commit 7c732924a889cd280e972882619a1827877fbafa)
(cherry picked from commit e91447ccc9ff438dd300e2564c251b2c4b0f369a)
 2017-09-18 14:05:36 -07:00
Wangda Tan
74708d3817 YARN-7149. Cross-queue preemption sometimes starves an underserved queue. (Eric Payne via wangda) 
Change-Id: Ib269991dbebce160378e8372ee6d24849c4a5ed6
(cherry picked from commit 3dfa937a1fadfc62947755872515f549b3b15e6a)
(cherry picked from commit 38c14ef8d8a094a7101917eb77d90f5e62324f61)
 2017-09-18 13:58:51 -05:00
Jason Lowe
cd6cf0caf6 YARN-7192. Add a pluggable StateMachine Listener that is notified of NM Container State changes. Contributed by Arun Suresh 
(cherry picked from commit a4f9c7c9247801dd37beec6fc195622af1b884ad)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
 2017-09-18 10:34:53 -05:00
Junping Du
c34c823d26 YARN-7174. Add retry logic in LogsCLI when fetch running application logs. Contributed by Xuan Gong. 
(cherry picked from commit 1a84c24b0cf6674fa755403971fa57d8e412b320)
 2017-09-15 15:34:00 -07:00
Robert Kanter
4e36452d02 YARN-7162. Remove XML excludes file format (rkanter) 2017-09-15 09:53:55 -07:00
Sunil G
317af87add Addendum patch for YARN-7163 in branch-2 2017-09-15 11:34:14 +05:30
Arun Suresh
b93a23f411 YARN-6059. Update paused container state in the NM state store. (Hitesh Sharma via asuresh) 
(cherry picked from commit 66ca0a65408521d5f9b080dd16b353b49fb8eaea)
 2017-09-14 10:30:30 -07:00
Arun Suresh
52bf458616 YARN-5216. Expose configurable preemption policy for OPPORTUNISTIC containers running on the NM. (Hitesh Sharma via asuresh) 
(cherry picked from commit 4f8194430fc6a69d9cc99b78828fd7045d5683e8)
 2017-09-14 10:29:54 -07:00
Arun Suresh
1af9f962c9 YARN-5292. NM Container lifecycle and state transitions to support for PAUSED container state. (Hitesh Sharma via asuresh) 
(cherry picked from commit 864fbacd4548004b1de8b0812627976acd22aff5)
 2017-09-14 10:26:34 -07:00
Junping Du
7e6cc17133 YARN-7194. Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat. Contributed by Xuan Gong. 
(cherry picked from commit c92c1d521eadfd8a4cd8205cc6aee74816f353f4)
 2017-09-13 22:18:39 -07:00
Eric Payne
fb3d53e001 YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable for container. Contributed by Jason Lowe. 
(cherry picked from commit 3860be7961580ac20dd505d665b580f0a04ac4f8)
 2017-09-13 15:07:27 -05:00
Nathan Roberts
1d33290a0a YARN-7014. Fix off-by-one error causing heap corruption (Jason Lowe via nroberts) 
(cherry picked from commit d265459024b8e5f5eccf421627f684ca8f162112)
 2017-09-13 09:47:55 -07:00
Varun Vasudev
4647761341 YARN-5719. Enforce a C standard for native container-executor. Contributed by Chris Douglas. 
(cherry picked from commit 972da46cb48725ad49d3e0a033742bd1a8228f51)
 2017-09-13 18:15:02 +05:30
Robert Kanter
055bde9806
YARN-5753. fix NPE in AMRMClientImpl.getMatchingRequests() (haibochen via rkanter) 
(cherry picked from commit 44fdf009642ae4e99b15f89ec0ca43834f991ef3)
 2017-09-13 16:48:43 +09:00
Arun Suresh
0d32a38ac3 YARN-7185. [Addendum patch] Minor javadoc and checkstyle fix. 
(cherry picked from commit a1c398e0dc85e424b45c50a3c3a0a784c04ed737)
 2017-09-12 17:05:01 -07:00
Arun Suresh
d735c1f26b YARN-7185. ContainerScheduler should only look at availableResource for GUARANTEED containers when OPPORTUNISTIC container queuing is enabled. (Wangda Tan via asuresh) 
(cherry picked from commit 2ae72692fc370267141a1ee55ef372ff62302b54)
 2017-09-12 16:33:05 -07:00
Arun Suresh
7803a93fff YARN-6706. Refactor ContainerScheduler to make oversubscription change easier. (Haibo Chen via asuresh) 
(cherry picked from commit 5b007921cdf01ecc8ed97c164b7d327b8304c529)
 2017-09-12 16:33:05 -07:00
Yufei Gu
1d40bc225a YARN-7057. FSAppAttempt#getResourceUsage doesn't need to consider resources queued for preemption. (Contributed by Karthik via Yufei Gu) 
(cherry picked from commit 82c5dd1d508292ed88eda0f5356776437ba67d4c)
 2017-09-12 16:14:31 -07:00
Wangda Tan
10803bf08d YARN-7173. Container update RM-NM communication fix for backward compatibility. (Arun Suresh via wangda) 
Change-Id: Ia7d61e0d9df1e703bc983a31e6856e84a5a0521c
 2017-09-11 20:56:17 -07:00
Daniel Templeton
fac4172e02 YARN-6622. Document Docker work as experimental (Contributed by Varun Vasudev) 
(cherry picked from commit 6651cbcc72d92caf86b744fa76fba513b36b12c7)
 2017-09-11 16:16:15 -07:00
Daniel Templeton
2e5a48afe3 Revert "YARN-6022. Document Docker work as experimental (Contributed by Varun Vasudev)" 
This reverts commit 1acd0047275ec8a15ee18f76dbb392d31e8f607a.
 2017-09-11 16:15:06 -07:00
Daniel Templeton
1acd004727 YARN-6022. Document Docker work as experimental (Contributed by Varun Vasudev) 
(cherry picked from commit 66be6c848b47ae7248b0658e3bb2149d2387cebd)
 2017-09-11 15:48:21 -07:00
Wangda Tan
c093e4622c YARN-6726. Fix issues with docker commands executed by container-executor. (Shane Kumpf via wangda) 
Change-Id: I8fa7d70526505d4082ac9b51cd6066376a599108
 2017-09-11 11:34:51 -07:00
Yufei Gu
416a44004f YARN-6799. Remove the duplicated code in CGroupsHandlerImp.java. (Contributed by Weiyuan via Yufei Gu) 
(cherry picked from commit 5ab21dfe9c8e14585db71e02bcb021a526507f98)
 2017-09-10 23:24:14 -07:00
Xuan
02ae3f8202 YARN-7072. Add a new log aggregation file format controller (xgong) 2017-09-10 23:18:31 -07:00
Sunil G
23dc6ef8b0 YARN-7163. RMContext need not to be injected to webapp and other Always Running services. Contributed by Rohith Sharma K S. 2017-09-11 10:34:59 +05:30
Subru Krishnan
2c051dbe9e YARN-5330. SharingPolicy enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru). 
(cherry picked from commit fa6137501c1499ae33f6e0e2adc31671a7e782dc)
 2017-09-08 14:44:46 -07:00
Chris Douglas
2f8825588e YARN-5164. Use plan RLE to improve CapacityOverTimePolicy efficiency 
(cherry picked from commit d383bfdcd40c2315197fa5f85c3e5cb3bb83167f)
 2017-09-08 14:44:34 -07:00
Nathan Roberts
b3a478832a YARN-6219. Fix conf usage in NMWebServer unit tests. Contributed by Jason Lowe. 2017-09-08 16:01:06 -05:00
Arun Suresh
2b09cec218 YARN-6849. NMContainerStatus should have the Container ExecutionType. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 1f53ae79728065417c6a99eb6fcc8d3a080ab4cc)
 2017-09-08 09:26:50 -07:00
Sunil G
a39829c456 YARN-6600. Introduce default and max lifetime of application at LeafQueue level. Contributed by Rohith Sharma K S. 2017-09-08 19:20:52 +05:30
Jason Lowe
c004d57d47 YARN-6930. Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager. Contributed by Shane Kumpf 2017-09-07 16:24:55 -05:00
Sunil G
ef342455cb YARN-6248. user is not removed from UsersManager’s when app is killed with pending container requests. Contributed by Eric Payne. 
(cherry picked from commit c1386a90d7ff65b57be00a7245de82c31abb4665)
 2017-09-07 14:47:07 -05:00
Arun Suresh
48279f7876 YARN-6978. Add updateContainer API to NMClient. (Kartheek Muthyala via asuresh) 
(cherry picked from commit c41118a7f826dcbb269b4bd3d5877b35fbbee2b1)
 2017-09-07 10:30:41 -07:00
Sunil G
8cce19241e YARN-6992. Kill application button is visible even if the application is FINISHED in RM UI. Contributed by Suma Shivaprasad. 
(cherry picked from commit 6f101e7df10d477cf7469f6979d0cb8269315a3e)
 2017-09-07 13:18:30 +05:30
Subru Krishnan
5ccdd83923 YARN-5328. Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem. 
(cherry picked from commit b6e7d1369690eaf50ce9ea7968f91a72ecb74de0)
 2017-09-06 16:46:49 -07:00
Junping Du
1ef48f1d4b YARN-7144. Log Aggregation controller should not swallow the exceptions when it calls closeWriter and closeReader. Contributed by Xuan Gong. 
(cherry picked from commit 22de9449f8aa72c5b0bb586b8253390773502189)
 2017-09-06 14:54:29 -07:00
Jason Lowe
fc444da891 YARN-7164. TestAMRMClientOnRMRestart fails sporadically with bind address in use. Contributed by Jason Lowe 
(cherry picked from commit 704267cb4909bf9968dd30067617f74543e07ffb)
 2017-09-06 15:25:37 -05:00
Junping Du
eaf134c46c YARN-7148. TestLogsCLI fails in trunk and branch-2 and javadoc error. Contributed by Xuan Gong. 
(cherry picked from commit 63231a2a3008794c5ebcbc9d3855638fc5f28216)
 2017-09-06 13:20:58 -07:00
Wei-Chiu Chuang
92d9ad7761 Revert "Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem." 
This reverts commit 80516b3de79e9aac4dba1374638257b4611f199e.
 2017-09-05 14:52:43 -07:00
Subru Krishnan
80516b3de7 Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem. 
(cherry picked from commit 7996eca7dcfaa1bdf970e32022274f2699bef8a1)
 2017-09-01 15:17:27 -07:00
Oleg Danilov
b616462eef
YARN-7023. Incorrect ReservationId.compareTo() implementation. 
This closes #262

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c5281a85e108c7cab6e84dfe351843483d720b9b)
 2017-09-01 21:36:12 +02:00