Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,446 Commits 326 Branches 371 Tags
Commit Graph

3281 Commits

Author SHA1 Message Date
Weiwei Yang
4488ad5297 YARN-8394. Improve data locality documentation for Capacity Scheduler. Contributed by Weiwei Yang. 2018-06-13 13:54:09 +08:00
Inigo Goiri
d5cd12c082 HADOOP-15529. ContainerLaunch#testInvalidEnvVariableSubstitutionType is not supported in Windows. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 6e756e8a620e4d6dc3192986679060c52063489b)
 2018-06-12 10:25:05 -07:00
Rohith Sharma K S
af0309d793 YARN-8405. RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773. Contributed by Íñigo Goiri. 
(cherry picked from commit 2df73dace06cfd2b3193a14cd455297f8f989617)
 2018-06-12 17:23:22 +05:30
Inigo Goiri
65d2554ad8 YARN-8370. Some Node Manager tests fail on Windows due to improper path/file separator. Contributed by Anbang Hu. 
(cherry picked from commit 2b2f672022547e8c19658213ac5a4090bf5b6c72)
 2018-06-11 19:26:27 -07:00
Inigo Goiri
63c20f9021 YARN-8359. Exclude containermanager.linux test classes on Windows. Contributed by Jason Lowe. 
(cherry picked from commit 3492a1db2c0654ce5375360caa74a34f928f23be)
 2018-06-07 17:10:09 -07:00
Rohith Sharma K S
a95f216945 YARN-8399. NodeManager is giving 403 GSS exception post upgrade to 3.1 in secure mode. Contributed by Sunil Govindan. 
(cherry picked from commit 58bc34f1e347034af566d6968eb3b3439a91cc74)
 2018-06-07 10:27:08 +05:30
Robert Kanter
93d6ed859e YARN-4677. RMNodeResourceUpdateEvent update from scheduler can lead to race condition (wilfreds and gphillips via rkanter) 
(cherry picked from commit 0cd145a44390bc1a01113dce4be4e629637c3e8a)
 2018-06-04 15:32:47 -07:00
Miklos Szegedi
21948b048e YARN-8382. cgroup file leak in NM. Contributed by Hu Ziqian. 
(cherry picked from commit 925fdf761a513130e23c10575c7328c8681cff1d)
 2018-06-04 10:56:13 -07:00
Billie Rinaldi
e59e9feefa YARN-8342. Enable untrusted docker image to run with launch command. Contributed by Eric Yang 
(cherry picked from commit 31998643a51f1e08f723f18dc5476ac1512d5b81)
 2018-06-02 14:49:21 -07:00
Rohith Sharma K S
f7d0ca71ac More YARN pages need to honor yarn.resourcemanager.display.per-user-apps(addendum). Contributed by Sunil G. 
(cherry picked from commit 8261f9e5710038ccbc475dbfcea3b9ae79b6f482)
 2018-06-02 09:43:08 +05:30
Wangda Tan
0be93a7973 YARN-8349. Remove YARN registry entries when a service is killed by the RM. (Billie Rinaldi via wangda) 
Change-Id: Ia58db3637789a8921482f564aa9bdf99c45cc36c
(cherry picked from commit ff583d3fa3325029bc691ec22d817aee37e5e85d)
 2018-06-01 15:53:26 -07:00
Wangda Tan
1cee85d8c4 YARN-7962. Race Condition When Stopping DelegationTokenRenewer causes RM crash during failover. (BELUGA BEHR via wangda) 
Change-Id: I617e2645f60a57080058ad5f06af860fb3f682c8
(cherry picked from commit 931f78718f3a09775bfa1f9a952c069c416d0914)
 2018-06-01 15:47:51 -07:00
Wangda Tan
cc460fea2d YARN-8384. stdout.txt, stderr.txt logs of a launched docker container is coming with primary group of submit user instead of hadoop. (Eric Yang via wangda) 
Change-Id: Idbb2cd250f4f62dc32993e0d3ca3ec5684616baa
(cherry picked from commit 3a6bd775500343632bad5f9c1a2bfacd408f7760)
 2018-06-01 15:47:45 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
a468400bad YARN-8197. Fixed AM IP Filter and Webapp proxy to redirect app tracking-URLs correctly when UI is secure. Contributed by Sunil Govindan. 
(cherry picked from commit 6b74f5d7fc509c55c331249256eec78b7e53b6ce)
 2018-05-31 17:04:45 -07:00
Weiwei Yang
ec4240a7fa YARN-8367. Fix NPE in SingleConstraintAppPlacementAllocator when placement constraint in SchedulingRequest is null. Contributed by Weiwei Yang. 
(Cherry picked from commit 6468071f137e6d918a7b4799ad54558fa26b25ce)
 2018-05-31 21:04:41 +08:00
Eric E Payne
3d06d75423 YARN-8377: Javadoc build failed in hadoop-yarn-server-nodemanager. Contributed by Takanobu Asanuma 
(cherry picked from commit e44c0849d7982c8f1ed43af25d2092090881d19f)
 2018-05-30 17:28:14 +00:00
Eric Yang
03209e8966 YARN-8362. Bugfix logic in container retries in node manager. 
Contributed by Chandni Singh

(cherry picked from commit 135941e00d762a417c3b4cc524cdc59b0d1810b1)
 2018-05-29 17:04:01 -04:00
Jason Lowe
a1fd04c4f4 YARN-8329. Docker client configuration can still be set incorrectly. Contributed by Shane Kumpf 
(cherry picked from commit 4827e9a9085b306bc379cb6e0b1fe4b92326edcd)
 2018-05-29 14:48:01 -05:00
Wangda Tan
3eb1cb18c7 YARN-8369. Javadoc build failed due to 'bad use of >'. (Takanobu Asanuma via wangda) 
Change-Id: I79a42154e8f86ab1c3cc939b3745024b8eebe5f4
(cherry picked from commit 17aa40f669f197d43387d67dc00040d14cd00948)
 2018-05-29 09:28:34 -07:00
Jason Lowe
500b0ee2ce YARN-8338. TimelineService V1.5 doesn't come up after HADOOP-15406. Contributed by Vinod Kumar Vavilapalli 
(cherry picked from commit 31ab960f4f931df273481927b897388895d803ba)
 2018-05-29 11:03:19 -05:00
Sunil G
7f9caf8986 YARN-4781. Support intra-queue preemption for fairness ordering policy. Contributed by Eric Payne. 
(cherry picked from commit 7c343669baf660df3b70d58987d6e68aec54d6fa)
 2018-05-28 16:33:48 +05:30
Wangda Tan
4d41cb1696 YARN-8213. Add Capacity Scheduler performance metrics. (Weiwei Yang via wangda) 
Change-Id: Ieea6f3eeb83c90cd74233fea896f0fcd0f325d5f
(cherry picked from commit f24c842d52e166e8566337ef93c96438f1c870d8)
 2018-05-25 22:04:54 -07:00
Eric E Payne
053937f436 YARN-8292: Fix the dominant resource preemption cannot happen when some of the resource vector becomes negative. Contributed by Wangda Tan. 
(cherry picked from commit 8d5509c68156faaa6641f4e747fc9ff80adccf88)
 2018-05-25 16:27:09 +00:00
Shane Kumpf
de43b41d0f YARN-6919. Add default volume mount list. Contributed by Eric Badger 2018-05-24 09:35:35 -06:00
Rohith Sharma K S
923c833da4 YARN-8319. More YARN pages need to honor yarn.resourcemanager.display.per-user-apps. Contributed by Sunil G. 
(cherry picked from commit c05b5d424b000bab766f57e88a07f2b4e9a56647)
 2018-05-24 14:22:43 +05:30
Inigo Goiri
61b5b2f4f7 YARN-8344. Missing nm.stop() in TestNodeManagerResync to fix testKillContainersOnResync. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit e99e5bf104e9664bc1b43a2639d87355d47a77e2)
 2018-05-23 14:16:03 -07:00
Wangda Tan
6aaf33e50c YARN-8290. SystemMetricsPublisher.appACLsUpdated should be invoked after application information is published to ATS to avoid "User is not set in the application report" Exception. (Eric Yang via wangda) 
Change-Id: I0ac6ddd19740d1aa7dd07111cd11af71ddc2fcaf
(cherry picked from commit bd15d2396ef0c24fb6b60c6393d16b37651b828e)
 2018-05-22 13:39:15 -07:00
Eric Yang
7c3f236783 YARN-7960. Added security flag no-new-privileges for YARN Docker integration. 
Contributed by Eric Badger

(cherry picked from commit 6176d2b35c85715aae93526236c29540f71ecac8)
 2018-05-22 13:47:08 -04:00
Jason Lowe
9788c8c01e YARN-8206. Sending a kill does not immediately kill docker containers. Contributed by Eric Badger 
(cherry picked from commit 5f11288e41fca2e414dcbea130c7702e29d4d610)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/TestDockerContainerRuntime.java
 2018-05-22 09:49:31 -05:00
Eric E Payne
5b2bbff141 YARN-8179: Preemption does not happen due to natural_termination_factor when DRF is used. Contributed by Kyungwan Nam. 
(cherry picked from commit 0b4c44bdeef62945b592d5761666ad026b629c0b)
 2018-05-21 20:19:44 +00:00
Eric Yang
aafaa5f99b YARN-8141. Removed YARN_CONTAINER_RUNTIME_DOCKER_LOCAL_RESOURCE_MOUNTS flag. 
Contributed by Chandni Singh

(cherry-picked from commit d45a0b7d73519acb78cd94ac3186bd8481f6c13e)
 2018-05-17 17:32:29 -04:00
Jason Lowe
79b2a508e0 YARN-8284. get_docker_command refactoring. Contributed by Eric Badger 
(cherry picked from commit d47c09dcb146d0a1cb50486f453a67560812fe5c)
 2018-05-16 09:34:24 -05:00
Weiwei Yang
2bb39330a4 YARN-8278. DistributedScheduling is not working in HA. Contributed by Bibin A Chundatt. 
(Cherry picked from commit 2bb647bb91439e82cf7298e963bb5f7f80bbc3cb)
 2018-05-15 18:47:15 +08:00
Haibo Chen
8d3598525d YARN-8130 Race condition when container events are published for KILLED applications. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit 2d00a0c71b5dde31e2cf8fcb96d9d541d41fb879)
 2018-05-15 11:40:47 +05:30
Eric Yang
8edcbe9460 YARN-8265. Improve DNS handling on docker IP changes. 
Contributed by Billie Rinaldi

(cherry picked from commit 0ff94563b9b62d0426d475dc0f84152b68f1ff0d)
 2018-05-11 22:41:30 -07:00
Jason Lowe
dfe73334c2 YARN-7654. Support ENTRY_POINT for docker container. Contributed by Eric Yang 
(cherry picked from commit 6c8e51ca7eaaeef0626658b3c45d446a537e4dc0)
 2018-05-11 18:57:20 -05:00
Jason Lowe
f6d6a2ab04 YARN-8244. TestContainerSchedulerQueuing.testStartMultipleContainers failed. Contributed by Jim Brennan 
(cherry picked from commit dc912994a1bcb511dfda32a0649cef0c9bdc47d3)
 2018-05-11 14:13:55 -05:00
Eric Yang
645429ca28 YARN-8274. Fixed a bug on docker start command. 
Contributed by Jason Lowe
 2018-05-11 14:26:20 -04:00
Weiwei Yang
b4213bfd90 YARN-7003. DRAINING state of queues is not recovered after RM restart. Contributed by Tao Yang. 
(Cherry picked from commit 9db9cd95bd0348070a286e69e7965c03c9bd39d6)
 2018-05-11 10:55:05 +08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
4c0a6b67f3 YARN-8249. Fixed few REST APIs in RMWebServices to have static-user check. Contributed by Sunil Govindan. 
(cherry picked from commit d76fbbc9b82e720d7d5188f9ae2f56a8d78f3a98)
 2018-05-10 19:07:23 -07:00
Robert Kanter
0506c762b2 YARN-8202. DefaultAMSProcessor should properly check units of requested custom resource types against minimum/maximum allocation (snemeth via rkanter) 
(cherry picked from commit c8b53c43644b4ad22d5385c22cad8ed573c0b1ba)
 2018-05-10 09:52:11 -07:00
Vrushali C
ad4d4153da YARN-8247 Incorrect HTTP status code returned by ATSv2 for non-whitelisted users. Contributed by Rohith Sharma K S 2018-05-09 22:30:59 -07:00
bibinchundatt
b29479c93a YARN-8201. Skip stacktrace of few exception from ClientRMService. Contributed by Bilwa S T. 
(cherry picked from commit cc0310a5266c8b8351f338f5fc8087a203c68cac)
 2018-05-10 09:29:43 +05:30
Eric Yang
6ce89ea2a8 YARN-8261. Fixed a bug in creation of localized container directory. 
Contributed by Jason Lowe

(cherry picked from commit af4fc2e62805a5bd392214521a93650085b8d020)
 2018-05-09 13:34:54 -04:00
Rohith Sharma K S
e3a7dbc4ec YARN-8239. [UI2] Clicking on Node Manager UI under AM container info / App Attempt page goes to old RM UI. Contributed by Sunil G. 
(cherry picked from commit eb7b256cc2c74e1b461e45444179584668d34ac6)
 2018-05-09 12:33:49 +05:30
Jason Lowe
39236da2ff YARN-8207. Docker container launch use popen have risk of shell expansion. Contributed by Eric Yang. 
(cherry picked from commit a2ea7564209dce896a5badee3949ee41e4bc00bf)
 2018-05-08 15:37:24 -05:00
Wangda Tan
0873cfcb56 YARN-8232. RMContainer lost queue name when RM HA happens. (Hu Ziqian via wangda) 
Change-Id: Ia21e1da6871570c993bbedde76ce32929e95970f
(cherry picked from commit 6b96a73bb0f0ad1c877a062b19091e3e15a33ec4)
 2018-05-08 11:35:11 -07:00
Rohith Sharma K S
b69c65fc6b YARN-8253. HTTPS Ats v2 api call fails with 'bad HTTP parsed'. Contributed by Charan Hebri. 
(cherry picked from commit 7450583721757b8af2945ebd9be1a9efed11444c)
 2018-05-08 12:31:35 +05:30
Weiwei Yang
68dd01efc1 YARN-8025. UsersManangers#getComputedResourceLimitForActiveUsers throws NPE due to preComputedActiveUserLimit is empty. Contributed by Tao Yang. 
(Cherry picked from commit 67f239c42f676237290d18ddbbc9aec369267692)
 2018-05-07 11:41:01 +08:00
Eric Yang
a984abc472 YARN-8223. Improved yarn auxiliary service to load jar file from HDFS. 
Contributed by Zian Chen

(cherry picked from commit 8cdb032aff4237d8d3970057d82290e4e32c4040)
 2018-05-04 12:38:28 -04:00