Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,805 Commits 326 Branches 371 Tags
Commit Graph

3584 Commits

Author SHA1 Message Date
Arun Suresh
7803a93fff YARN-6706. Refactor ContainerScheduler to make oversubscription change easier. (Haibo Chen via asuresh) 
(cherry picked from commit 5b007921cdf01ecc8ed97c164b7d327b8304c529)
 2017-09-12 16:33:05 -07:00
Yufei Gu
1d40bc225a YARN-7057. FSAppAttempt#getResourceUsage doesn't need to consider resources queued for preemption. (Contributed by Karthik via Yufei Gu) 
(cherry picked from commit 82c5dd1d508292ed88eda0f5356776437ba67d4c)
 2017-09-12 16:14:31 -07:00
Wangda Tan
10803bf08d YARN-7173. Container update RM-NM communication fix for backward compatibility. (Arun Suresh via wangda) 
Change-Id: Ia7d61e0d9df1e703bc983a31e6856e84a5a0521c
 2017-09-11 20:56:17 -07:00
Daniel Templeton
fac4172e02 YARN-6622. Document Docker work as experimental (Contributed by Varun Vasudev) 
(cherry picked from commit 6651cbcc72d92caf86b744fa76fba513b36b12c7)
 2017-09-11 16:16:15 -07:00
Daniel Templeton
2e5a48afe3 Revert "YARN-6022. Document Docker work as experimental (Contributed by Varun Vasudev)" 
This reverts commit 1acd0047275ec8a15ee18f76dbb392d31e8f607a.
 2017-09-11 16:15:06 -07:00
Daniel Templeton
1acd004727 YARN-6022. Document Docker work as experimental (Contributed by Varun Vasudev) 
(cherry picked from commit 66be6c848b47ae7248b0658e3bb2149d2387cebd)
 2017-09-11 15:48:21 -07:00
Wangda Tan
c093e4622c YARN-6726. Fix issues with docker commands executed by container-executor. (Shane Kumpf via wangda) 
Change-Id: I8fa7d70526505d4082ac9b51cd6066376a599108
 2017-09-11 11:34:51 -07:00
Yufei Gu
416a44004f YARN-6799. Remove the duplicated code in CGroupsHandlerImp.java. (Contributed by Weiyuan via Yufei Gu) 
(cherry picked from commit 5ab21dfe9c8e14585db71e02bcb021a526507f98)
 2017-09-10 23:24:14 -07:00
Xuan
02ae3f8202 YARN-7072. Add a new log aggregation file format controller (xgong) 2017-09-10 23:18:31 -07:00
Sunil G
23dc6ef8b0 YARN-7163. RMContext need not to be injected to webapp and other Always Running services. Contributed by Rohith Sharma K S. 2017-09-11 10:34:59 +05:30
Subru Krishnan
2c051dbe9e YARN-5330. SharingPolicy enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru). 
(cherry picked from commit fa6137501c1499ae33f6e0e2adc31671a7e782dc)
 2017-09-08 14:44:46 -07:00
Chris Douglas
2f8825588e YARN-5164. Use plan RLE to improve CapacityOverTimePolicy efficiency 
(cherry picked from commit d383bfdcd40c2315197fa5f85c3e5cb3bb83167f)
 2017-09-08 14:44:34 -07:00
Nathan Roberts
b3a478832a YARN-6219. Fix conf usage in NMWebServer unit tests. Contributed by Jason Lowe. 2017-09-08 16:01:06 -05:00
Arun Suresh
2b09cec218 YARN-6849. NMContainerStatus should have the Container ExecutionType. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 1f53ae79728065417c6a99eb6fcc8d3a080ab4cc)
 2017-09-08 09:26:50 -07:00
Sunil G
a39829c456 YARN-6600. Introduce default and max lifetime of application at LeafQueue level. Contributed by Rohith Sharma K S. 2017-09-08 19:20:52 +05:30
Jason Lowe
c004d57d47 YARN-6930. Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager. Contributed by Shane Kumpf 2017-09-07 16:24:55 -05:00
Sunil G
ef342455cb YARN-6248. user is not removed from UsersManager’s when app is killed with pending container requests. Contributed by Eric Payne. 
(cherry picked from commit c1386a90d7ff65b57be00a7245de82c31abb4665)
 2017-09-07 14:47:07 -05:00
Arun Suresh
48279f7876 YARN-6978. Add updateContainer API to NMClient. (Kartheek Muthyala via asuresh) 
(cherry picked from commit c41118a7f826dcbb269b4bd3d5877b35fbbee2b1)
 2017-09-07 10:30:41 -07:00
Sunil G
8cce19241e YARN-6992. Kill application button is visible even if the application is FINISHED in RM UI. Contributed by Suma Shivaprasad. 
(cherry picked from commit 6f101e7df10d477cf7469f6979d0cb8269315a3e)
 2017-09-07 13:18:30 +05:30
Subru Krishnan
5ccdd83923 YARN-5328. Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem. 
(cherry picked from commit b6e7d1369690eaf50ce9ea7968f91a72ecb74de0)
 2017-09-06 16:46:49 -07:00
Junping Du
1ef48f1d4b YARN-7144. Log Aggregation controller should not swallow the exceptions when it calls closeWriter and closeReader. Contributed by Xuan Gong. 
(cherry picked from commit 22de9449f8aa72c5b0bb586b8253390773502189)
 2017-09-06 14:54:29 -07:00
Jason Lowe
fc444da891 YARN-7164. TestAMRMClientOnRMRestart fails sporadically with bind address in use. Contributed by Jason Lowe 
(cherry picked from commit 704267cb4909bf9968dd30067617f74543e07ffb)
 2017-09-06 15:25:37 -05:00
Junping Du
eaf134c46c YARN-7148. TestLogsCLI fails in trunk and branch-2 and javadoc error. Contributed by Xuan Gong. 
(cherry picked from commit 63231a2a3008794c5ebcbc9d3855638fc5f28216)
 2017-09-06 13:20:58 -07:00
Wei-Chiu Chuang
92d9ad7761 Revert "Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem." 
This reverts commit 80516b3de79e9aac4dba1374638257b4611f199e.
 2017-09-05 14:52:43 -07:00
Subru Krishnan
80516b3de7 Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem. 
(cherry picked from commit 7996eca7dcfaa1bdf970e32022274f2699bef8a1)
 2017-09-01 15:17:27 -07:00
Oleg Danilov
b616462eef
YARN-7023. Incorrect ReservationId.compareTo() implementation. 
This closes #262

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c5281a85e108c7cab6e84dfe351843483d720b9b)
 2017-09-01 21:36:12 +02:00
Jason Lowe
57b5b186ce YARN-7120. CapacitySchedulerPage NPE in "Aggregate scheduler counts" section. Contributed by Eric Payne 
(cherry picked from commit 99a7f5d451fb91ef956c926f891283f70f0ada1c)
 2017-09-01 11:10:40 -05:00
Junping Du
119220b88f YARN-6877. Create an abstract log reader for extendability. Contributed by Xuan Gong. 2017-09-01 03:04:55 -07:00
Daniel Templeton
004231dc08 YARN-6780. ResourceWeights.toString() cleanup (Contributed by weiyuan via Daniel Templeton) 
(cherry picked from commit d4417dae4f38b58d41517a0263484da73ed41b75)
 2017-08-31 09:38:31 -07:00
Sunil G
e8b4427259 YARN-7116. CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage. Contributed by Wangda Tan. 
(cherry picked from commit f9e0cc8cdca387bf2811c07beccae4af07362952)
 2017-08-31 18:43:17 +05:30
Robert Kanter
8523319e15 YARN-7094. Document the current known issue with server-side NM graceful decom (rkanter) 
(cherry picked from commit ce79f7be29dfbad444614cb340c59e4ff8137e89)
 2017-08-30 18:06:03 -07:00
Rohith Sharma K S
877d96b83d YARN-6982. Potential issue on setting AMContainerSpec#tokenConf to null before app is completed. Contributed by Manikandan R. 
(cherry picked from commit 4cae120c619811006b26b9a95680a98732572af6)
 2017-08-30 10:50:57 +05:30
Jian He
036cbd7453 YARN-6756. ContainerRequest#executionTypeRequest causes NPE. Contributed by Jian He 2017-08-29 20:36:23 -07:00
Wangda Tan
9bcd64d233 YARN-5219. When an export var command fails in launch_container.sh, the full container launch should fail. (Sunil G via wangda) 
Change-Id: Iaa6b978bb89482e9d1d77ba57f4adfdc48e39a3c
(cherry picked from commit f59332b97b9a57e3cf1dcdeb47d7838d287100eb)
 2017-08-29 15:29:53 -07:00
Junping Du
b906bddd95 YARN-7037. Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices. Contributed by Tao Yang. 
(cherry picked from commit ad45d19998c1b0da25754d0016854046731fa623)
 2017-08-29 15:17:21 -07:00
Jason Lowe
c930aa3ff9 YARN-5816. TestDelegationTokenRenewer#testCancelWithMultipleAppSubmissions is still flakey. Contributed by Robert Kanter 2017-08-29 10:02:44 -05:00
Junping Du
f252e1fe28 YARN-7076. yarn application -list -appTypes is not working. Contributed by Jian He. 
(cherry picked from commit 312b1fd9da2781da97f8c76fe1262c4d99b9c37f)
 2017-08-28 18:27:38 -07:00
Wangda Tan
82e1cd08eb YARN-7112. TestAMRMProxy is failing with invalid request. (Jason Lowe via wangda) 
Change-Id: I432f4523921666ac218fcf9936f826a0d8d635e0
(cherry picked from commit 51881a863cb596ff01106d8bd386066e56c38a2a)
 2017-08-28 11:38:05 -07:00
Yufei Gu
221c1bdf7e YARN-7099. ResourceHandlerModule.parseConfiguredCGroupPath only works for privileged yarn users. (Contributed by Miklos Szegedi via Yufei Gu) 
(cherry picked from commit d5bc9582793de6ab2d2e489cde293bf076229a1c)
 2017-08-28 10:11:01 -07:00
Sunil G
80570737a8 YARN-7051. Avoid concurrent modification exception in FifoIntraQueuePreemptionPlugin. Contributed by Eric Payne. 
(cherry picked from commit 02599bda04e0ef46f4628b006f2430ad63cac97e)
 2017-08-28 12:27:29 +05:30
Li Lu
b7f319c53c YARN-6999. Add log about how to solve Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster. (Linlin Zhou via gtcarrera9) 
(cherry picked from commit 3bee7046c52bff681a5aecc956ba9be6c3543397)
 2017-08-25 22:45:20 -07:00
Eric Payne
1617744801 YARN-7087. NM failed to perform log aggregation due to absent container. Contributed by Jason Lowe. 
(cherry picked from commit e864f81471407a384395fefe1ceb3b66fc7f87f2)
 2017-08-25 15:52:29 -05:00
Jason Lowe
ea5bb5898f YARN-7052. RM SchedulingMonitor gives no indication why the spawned thread crashed. Contributed by Eric Payne 
(cherry picked from commit 39a9dc8e4a6e1d13658867ad756878d3dd6352b0)
 2017-08-25 15:32:29 -05:00
Jason Lowe
82957ce0d0 YARN-6640. AM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 
(cherry picked from commit 3a4e861169dc3da9df0158ba6f44a9bc8576e217)
 2017-08-25 09:20:50 -05:00
Yufei Gu
389b659c94 YARN-7049. FSAppAttempt preemption related fields have confusing names. (Contributed by Karthik Kambatla via Yufei Gu) 
(cherry picked from commit 9e2699ac2c99d8df85191dd7fbf9468b00f5b5aa)
 2017-08-24 17:54:00 -07:00
Junping Du
e12358c5ce YARN-6876. Create an abstract log writer for extendability. Contributed by Xuan Gong. 
(cherry picked from commit c2cb7ea1ef6532020b69031dbd18b0f9b8369f0f)
 2017-08-24 15:27:40 -07:00
Akira Ajisaka
bf903396a0
YARN-7047. Moving logging APIs over to slf4j in hadoop-yarn-server-nodemanager. Contributed by Yeliang Cang. 2017-08-25 03:28:02 +09:00
bibinchundatt
cc50ca0722 YARN-7074. Fix NM state store update comment. Contributed by Botong Huang 
(cherry picked from commit de0cba700bcf4276726c0aa9df8d738787debc17)
 2017-08-24 16:40:18 +05:30
Junping Du
c98917f798 YARN-7090. testRMRestartAfterNodeLabelDisabled get failed when CapacityScheduler is configured. Contributed by Wangda Tan. 
(cherry picked from commit 652dd434d96cd1a1fe25cd8c636b1859c29f462b)
 2017-08-23 18:07:07 -07:00
Wangda Tan
245b49332d YARN-6251. Do async container release to prevent deadlock during container updates. (Arun Suresh via wangda) 
Change-Id: I6c67d20c5dd4d22752830ebf0ed2340824976ecb
(cherry picked from commit f49843a9888ad8fe5c1bb4c16bfb5217d693009d)
 2017-08-23 10:35:37 -07:00