Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,702 Commits 326 Branches 371 Tags
Commit Graph

4880 Commits

Author SHA1 Message Date
Jason Lowe
f6d6a2ab04 YARN-8244. TestContainerSchedulerQueuing.testStartMultipleContainers failed. Contributed by Jim Brennan 
(cherry picked from commit dc912994a1bcb511dfda32a0649cef0c9bdc47d3)
 2018-05-11 14:13:55 -05:00
Eric Yang
645429ca28 YARN-8274. Fixed a bug on docker start command. 
Contributed by Jason Lowe
 2018-05-11 14:26:20 -04:00
Weiwei Yang
b4213bfd90 YARN-7003. DRAINING state of queues is not recovered after RM restart. Contributed by Tao Yang. 
(Cherry picked from commit 9db9cd95bd0348070a286e69e7965c03c9bd39d6)
 2018-05-11 10:55:05 +08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
4c0a6b67f3 YARN-8249. Fixed few REST APIs in RMWebServices to have static-user check. Contributed by Sunil Govindan. 
(cherry picked from commit d76fbbc9b82e720d7d5188f9ae2f56a8d78f3a98)
 2018-05-10 19:07:23 -07:00
Robert Kanter
0506c762b2 YARN-8202. DefaultAMSProcessor should properly check units of requested custom resource types against minimum/maximum allocation (snemeth via rkanter) 
(cherry picked from commit c8b53c43644b4ad22d5385c22cad8ed573c0b1ba)
 2018-05-10 09:52:11 -07:00
Vrushali C
ad4d4153da YARN-8247 Incorrect HTTP status code returned by ATSv2 for non-whitelisted users. Contributed by Rohith Sharma K S 2018-05-09 22:30:59 -07:00
bibinchundatt
b29479c93a YARN-8201. Skip stacktrace of few exception from ClientRMService. Contributed by Bilwa S T. 
(cherry picked from commit cc0310a5266c8b8351f338f5fc8087a203c68cac)
 2018-05-10 09:29:43 +05:30
Eric Yang
6ce89ea2a8 YARN-8261. Fixed a bug in creation of localized container directory. 
Contributed by Jason Lowe

(cherry picked from commit af4fc2e62805a5bd392214521a93650085b8d020)
 2018-05-09 13:34:54 -04:00
Rohith Sharma K S
ee19c47d08 YARN-8260. [UI2] Per-application tracking URL is no longer available in YARN UI2. Contributed by Sunil G. 
(cherry picked from commit cd68c7cc6942970285dce04775e80f75a5d55afa)
 2018-05-09 12:34:11 +05:30
Rohith Sharma K S
438c5b12aa YARN-8264. [UI2] GPU Info tab disappears if we click any sub link under List of Applications or List of Containers. Contributed by Sunil G. 
(cherry picked from commit 9acfaee1f080245744f191a7fd080f91413249a6)
 2018-05-09 12:34:01 +05:30
Rohith Sharma K S
e3a7dbc4ec YARN-8239. [UI2] Clicking on Node Manager UI under AM container info / App Attempt page goes to old RM UI. Contributed by Sunil G. 
(cherry picked from commit eb7b256cc2c74e1b461e45444179584668d34ac6)
 2018-05-09 12:33:49 +05:30
Billie Rinaldi
f7cbfeb726 YARN-7894. Improve ATS response for DShell DS_CONTAINER when container launch fails. Contributed by Chandni Singh 
(cherry picked from commit 1ef0a1db1d6a412b2a26782329a8325635866d0a)
 2018-05-08 13:51:02 -07:00
Jason Lowe
39236da2ff YARN-8207. Docker container launch use popen have risk of shell expansion. Contributed by Eric Yang. 
(cherry picked from commit a2ea7564209dce896a5badee3949ee41e4bc00bf)
 2018-05-08 15:37:24 -05:00
Wangda Tan
0873cfcb56 YARN-8232. RMContainer lost queue name when RM HA happens. (Hu Ziqian via wangda) 
Change-Id: Ia21e1da6871570c993bbedde76ce32929e95970f
(cherry picked from commit 6b96a73bb0f0ad1c877a062b19091e3e15a33ec4)
 2018-05-08 11:35:11 -07:00
Rohith Sharma K S
b69c65fc6b YARN-8253. HTTPS Ats v2 api call fails with 'bad HTTP parsed'. Contributed by Charan Hebri. 
(cherry picked from commit 7450583721757b8af2945ebd9be1a9efed11444c)
 2018-05-08 12:31:35 +05:30
Sunil G
9355f56b36 YARN-8251. [UI2] Clicking on Application link at the header goes to Diagnostics Tab instead of AppAttempt Tab. Contributed by Yesha Vora. 
(cherry picked from commit 2916c90500643f82aabcedd74b349938d8c86518)
 2018-05-08 11:37:43 +05:30
Sunil G
9eb51a3004 YARN-5151. [UI2] Support kill application from new YARN UI. Contributed by Gergely Novák. 
(cherry picked from commit 9832265e1deeefaa6d58f9f62052c8ef6a8e82b7)
 2018-05-08 10:38:04 +05:30
Weiwei Yang
68dd01efc1 YARN-8025. UsersManangers#getComputedResourceLimitForActiveUsers throws NPE due to preComputedActiveUserLimit is empty. Contributed by Tao Yang. 
(Cherry picked from commit 67f239c42f676237290d18ddbbc9aec369267692)
 2018-05-07 11:41:01 +08:00
Billie Rinaldi
e933ed0ee3 YARN-8079. Support static and archive unmodified local resources in service AM. Contributed by Suma Shivaprasad 
(cherry picked from commit 6795f8072ffbe6138857e77d51af173f33e4e5c1)
 2018-05-04 09:47:06 -07:00
Eric Yang
a984abc472 YARN-8223. Improved yarn auxiliary service to load jar file from HDFS. 
Contributed by Zian Chen

(cherry picked from commit 8cdb032aff4237d8d3970057d82290e4e32c4040)
 2018-05-04 12:38:28 -04:00
Billie Rinaldi
04c3f11de4 YARN-7818. Remove privileged operation warnings during container launch for the ContainerRuntimes. Contributed by Shane Kumpf 
(cherry picked from commit 502914ca32ac02b19116fd681eb8301b92fccbb3)
 2018-05-04 08:55:30 -07:00
Eric Yang
2148fc1c3a YARN-8226. Improved anti-affinity description in YARN Service doc. 
Contributed by Gour Saha

(cherry picked from commit 7698737207b01e80b1be2b4df60363f952a1c2b4)
 2018-05-03 13:38:21 -04:00
Eric Yang
b09af11117 YARN-7961. Improve status message for YARN service. 
Contributed by Gour Saha
(cherry picked from commit 7fe3214d4bb810c0da18dd936875b4e2588ba518)
 2018-05-03 13:29:07 -04:00
Eric Yang
cf46533bca YARN-7973. Added ContainerRelaunch feature for Docker containers. 
Addendum patch 001
            Contributed by Shane Kumpf
 2018-05-03 12:46:04 -04:00
Eric Yang
f729fb0baa YARN-8194. Fixed reinitialization error for LinuxContainerExecutor. 
Contributed by Chandni Singh

(cherry picked from commit f4d280f02b557885cd5e5cf36abc36eb579ccfb4)
 2018-05-03 11:59:35 -04:00
Eric Yang
e4b9981612 YARN-7973. Added ContainerRelaunch feature for Docker containers. 
Contributed by Shane Kumpf
 2018-05-03 11:57:01 -04:00
Rohith Sharma K S
4c93cd228c YARN-8217. RmAuthenticationFilterInitializer and TimelineAuthenticationFilterInitializer should use Configuration.getPropsWithPrefix instead of iterator. Contributed by Suma Shivaprasad. 
(cherry picked from commit ee2ce923a922bfc3e89ad6f0f6a25e776fe91ffb)
 2018-05-03 18:20:21 +05:30
Eric Yang
61fac20c9e YARN-8209. Fixed NPE in Yarn Service deletion. 
Contributed by Eric Badger
 2018-05-02 15:05:30 -04:00
Konstantinos Karanasos
62ad9d512d YARN-8113. Update placement constraints doc with application namespaces and inter-app constraints. Contributed by Weiwei Yang. 
(cherry picked from commit 3b34fca4b5d67a2685852f30bb61e7c408a0e886)
 2018-05-02 11:51:45 -07:00
Weiwei Yang
6fce88765a YARN-8222. Fix potential NPE when gets RMApp from RM context. Contributed by Tao Yang. 2018-05-02 18:02:24 +08:00
Yiqun Lin
2741af505f
YARN-6385. Fix checkstyle warnings in TestFileSystemApplicationHistoryStore 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3265b55119d39ecbda6d75be04a9a1bf59c631f1)
 2018-05-02 18:15:53 +09:00
Eric Yang
a6f3d6ffcb YARN-7799. Improved YARN service jar file handling. 
Contributed by Billie Rinaldi
(cherry picked from commit 24eeea8b18749e02ea05b03eb18e3cf0455077c5)
 2018-05-01 16:49:29 -04:00
Billie Rinaldi
4174a0ed25 YARN-2674. Fix distributed shell AM container relaunch during RM work preserving restart. Contributed by Shane Kumpf 
(cherry picked from commit 4e1382aca4cf23ca229bdd24e0f143c22449b329)
 2018-05-01 08:04:39 -07:00
Sunil G
12fe48b5e5 YARN-8187. [UI2] Individual Node page does not contain breadcrumb trail. Contributed by Zian Chen. 
(cherry picked from commit d6139c5106a469df72c1551100d550371f6cb7c7)
 2018-05-01 14:02:20 +05:30
Weiwei Yang
f34f4e9c11 YARN-8212. Pending backlog for async allocation threads should be configurable. Contributed by Tao Yang. 
(cherry picked from commit 2d319e37937c1e20c6a7dc4477ef88defd1f8464)
 2018-05-01 09:57:15 +08:00
Eric Yang
0fc55a8f57 YARN-8228. Added hostname length check for docker container. 
Contributed by Shane Kumpf

(cherry picked from commit a966ec6e23b3ac8e233b2cf9b9ddaa6628a8c996)
 2018-04-30 19:14:37 -04:00
Konstantinos Karanasos
9d2967098d YARN-8195. Fix constraint cardinality check in the presence of multiple target allocation tags. Contributed by Weiwei Yang. 
(cherry picked from commit 9b0955545174abe16fd81240db30f175145ee89b)
 2018-04-30 11:55:26 -07:00
Weiwei Yang
36fc9a6939 YARN-8225. YARN precommit build failing in TestPlacementConstraintTransformations. Contributed by Shane Kumpf. 
(cherry picked from commit 2c95eb8111a7b03fd4683f740123cd4720b62c3e)
 2018-04-28 18:50:51 +08:00
Wangda Tan
090a8a274e YARN-8005. Add unit tests for queue priority with dominant resource calculator. (Zian Chen via wangda) 
Change-Id: I17a645f20869a1e5d86fa7a325c93fec908b91dc
(cherry picked from commit 92c5331423afee172e843198f4824469eb92e89b)
 2018-04-27 13:13:52 -07:00
Wangda Tan
f9add5fdd9 YARN-8210. AMRMClient logging on every heartbeat to track updation of AM RM token causes too many log lines to be generated in AM logs. (Suma Shivaprasad via wangda) 
Change-Id: I70edd6e301fd5e78d479e1882aedc9332a0827aa
(cherry picked from commit b1833d9ba2c078582161da45ac392dd5c361dcdf)
 2018-04-27 13:13:47 -07:00
Rohith Sharma K S
b3fd464e55 YARN-8221. RMWebServices also need to honor yarn.resourcemanager.display.per-user-apps. Contributed by Sunil G. 
(cherry picked from commit ef3ecc308dbea41c6a88bd4d16739c7bbc10cdda)
 2018-04-27 22:59:16 +05:30
Eric Yang
e75aeacf29 YARN-7781. Update YARN service documentation. 
Contributed by Gour Saha
(cherry picked from commit 24a5ccbf4bda413a98480d52c204d56f82ef9ac5)
 2018-04-27 12:40:24 -04:00
Eric Yang
0e19c0dd72 YARN-8204. Added a flag to disable YARN service upgrade. 
Contributed by Chandni Singh
(cherry picked from commit 14b47990af39de71b0a09d995208f45ea3b79c23)
 2018-04-27 12:25:57 -04:00
Billie Rinaldi
90a31dc61c YARN-8211. Yarn registry dns log finds BufferUnderflowException on port ping. Contributed by Eric Yang 
(cherry picked from commit 9ab3f9708923b3429209006a12695f4826a045b9)
 2018-04-27 08:00:04 -07:00
Billie Rinaldi
b20f68ddaa YARN-8208. Add log statement for Docker client configuration file at INFO level. Contributed by Yesha Vora 
(cherry picked from commit 914d6b5cb143f4f9cadd9d689ea4e278383a6779)
 2018-04-27 06:11:34 -07:00
Sunil G
28e63f3d88 YARN-8004. Add unit tests for inter queue preemption for dominant resource calculator. Contributed by Zian Chen. 
(cherry picked from commit 71220d218db59cab0b03bbba427e5e9ef5b3003c)
 2018-04-27 10:42:59 +05:30
Sunil G
a1a2f1ddc1 YARN-8215. ATS v2 returns invalid YARN_CONTAINER_ALLOCATED_HOST_HTTP_ADDRESS from NM. Contributed by Rohith Sharma K S. 
(cherry picked from commit 7fc09c4f7255645025ddf902dff70a17092ea039)
 2018-04-27 10:34:29 +05:30
Sunil G
e2901d97ba YARN-8205. Application State is not updated to ATS if AM launching is delayed. Contributed by Rohith Sharma K S. 
(cherry picked from commit 1634de0fc1430d86b7688d16259a81462fba482f)
 2018-04-27 10:28:18 +05:30
Eric Yang
2253616064 YARN-8122. Added component health monitoring for YARN service. 
Contributed by Gour Saha
(cherry picked from commit 93979db8400186d6e557c46a2a1e2622821013c1)
 2018-04-26 19:19:29 -04:00
Billie Rinaldi
91b24278da YARN-8140. Improve log message when launch cmd is ran for stopped yarn service. Contributed by Eric Yang 
(cherry picked from commit ed4e0676c357d5d18c15998b1f6b5d305848cb04)
 2018-04-26 13:18:56 -07:00