Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,282 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3506 Commits

Author SHA1 Message Date
Robert Kanter 27e2b4b364 Disable mounting cgroups by default (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit 351cf87c92)
 2018-07-18 16:07:48 -07:00
Eric Yang d82edec3c0 YARN-8538. Fixed memory leaks in container-executor and test cases. 
Contributed by Billie Rinaldi
 2018-07-18 13:44:49 -04:00
Wangda Tan 44beab0b63 YARN-8511. When AM releases a container, RM removes allocation tags before it is released by NM. (Weiwei Yang via wangda) 
Change-Id: I6f9f409f2ef685b405cbff547dea9623bf3322d9
(cherry picked from commit 752dcce5f4)
 2018-07-16 11:04:08 -07:00
Eric E Payne 9a79e893f7 YARN-8421: when moving app, activeUsers is increased, even though app does not have outstanding request. Contributed by Kyungwan Nam 
(cherry picked from commit 937ef39b3f)
 2018-07-16 16:32:05 +00:00
Jason Lowe 7cbb9597c4 YARN-8515. container-executor can crash with SIGPIPE after nodemanager restart. Contributed by Jim Brennan 
(cherry picked from commit 17118f446c)
 2018-07-13 10:06:38 -05:00
Sunil G 9b4ead92c8 YARN-8512. ATSv2 entities are not published to HBase from second attempt onwards. Contributed by Rohith Sharma K S. 
(cherry picked from commit 7f1d3d0e9d)
 2018-07-11 12:27:12 +05:30
Sunil G 6f10491e64 YARN-8473. Containers being launched as app tears down can leave containers in NEW state. Contributed by Jason Lowe. 
(cherry picked from commit 705e2c1f7c)
 2018-07-10 20:12:47 +05:30
Rohith Sharma K S 785ed3f8ae YARN-8302. ATS v2 should handle HBase connection issue properly. Contributed by Billie Rinaldi. 
(cherry picked from commit ba68320449)
 2018-07-06 15:20:39 -07:00
Sunil G 844538d669 YARN-8459. Improve Capacity Scheduler logs to debug invalid states. Contributed by Wangda Tan. 
(cherry picked from commit 51654a3962)
 2018-07-03 10:00:05 -07:00
Junping Du 3d6ba2dd4e yarn.resourcemanager.fail-fast is used inconsistently. Contributed by Yuanbo Liu. 
(cherry picked from commit d9ba6f3656)
 2018-07-03 22:36:24 +08:00
Sunil G 8ffc191f61 YARN-8415. TimelineWebServices.getEntity should throw ForbiddenException instead of 404 when ACL checks fail. Contributed by Suma Shivaprasad. 
(cherry picked from commit fa9ef15ecd)
 2018-07-02 15:35:45 -07:00
Shane Kumpf 2f264764a4 YARN-8485. Priviledged container app launch is failing intermittently. Contributed by Eric Yang 
(cherry picked from commit 53e267fa72)
 2018-07-02 16:23:23 -06:00
Eric Yang 0aa7272199 YARN-8465. Fixed docker container status for node manager restart. 
Contributed by Shane Kumpf
 2018-07-02 13:40:02 -04:00
Jason Lowe dbb0a5f51f YARN-8451. Multiple NM heartbeat thread created when a slow NM resync with RM. Contributed by Botong Huang 
(cherry picked from commit 100470140d)
 2018-06-29 13:11:41 -05:00
Sunil G e7f2c9886a YARN-8455. Add basic ACL check for all ATS v2 REST APIs. Contributed by Rohith Sharma K S. 
(cherry picked from commit 469b29c081)
 2018-06-29 10:03:53 -07:00
Sunil G 122cb81b65 YARN-8379. Improve balancing resources in already satisfied queues by using Capacity Scheduler preemption. Contributed by Zian Chen. 
(cherry picked from commit 291194302c)
 2018-06-28 10:24:03 -07:00
Eric Yang 3ce3571a34 YARN-8409. Fixed NPE in ActiveStandbyElectorBasedElectorService. 
Contributed by Chandni Singh
 2018-06-28 13:11:56 -04:00
Wangda Tan ec37e02bc9 YARN-8464. Async scheduling thread could be interrupted when there are no NodeManagers in cluster. (Sunil G via wangda) 
Change-Id: I4f5f856373378685713e77752ba6cf0988a66065
(cherry picked from commit bedc4fe079)
 2018-06-26 19:34:15 -07:00
Wangda Tan 8f226f4f1d YARN-8423. GPU does not get released even though the application gets killed. (Sunil G via wangda) 
Change-Id: I570db7d60f8c6c21762dd618a9207d1107c486a0
(cherry picked from commit ada8f63d0b)
 2018-06-26 19:34:10 -07:00
Eric Yang f6aa6a6981 YARN-8108. Added option to disable loading existing filters to prevent 
security filter from initialize twice.
            Contributed by Sunil Govindan

(cherry picked from commit b69ba0f330)
 2018-06-26 17:45:13 -04:00
Weiwei Yang bfdbc9dea2 YARN-8443. Total #VCores in cluster metrics is wrong when CapacityScheduler reserved some containers. Contributed by Tao Yang. 
(Cherry picked from commit 440140cea6)
 2018-06-25 09:28:57 +08:00
Eric Yang 3b3a85002d YARN-8326. Removed exit code file check for launched container. 
Contributed by Shane Kumpf

(cherry picked from commit 8a32bc39eb)
 2018-06-22 19:23:57 -04:00
Rohith Sharma K S 1aa6c9407c YARN-8155. Improve ATSv2 client logging in RM and NM publisher. Contributed by Abhishek Modi. 
(cherry picked from commit 9119b3cf8f)
 2018-06-14 13:51:13 +05:30
Eric Yang 934a005f72 YARN-8259. Improve privileged docker container liveliness checks. 
Contributed by Shane Kumpf

(cherry picked from commit 22994889dc)
 2018-06-13 19:26:23 -04:00
Sunil G ed20c59d36 YARN-8404. Timeline event publish need to be async to avoid Dispatcher thread leak in case ATS is down. Contributed by Rohith Sharma K S 
(cherry picked from commit 6307962b93)
 2018-06-13 16:09:52 +05:30
Weiwei Yang 4488ad5297 YARN-8394. Improve data locality documentation for Capacity Scheduler. Contributed by Weiwei Yang. 2018-06-13 13:54:09 +08:00
Inigo Goiri d5cd12c082 HADOOP-15529. ContainerLaunch#testInvalidEnvVariableSubstitutionType is not supported in Windows. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 6e756e8a62)
 2018-06-12 10:25:05 -07:00
Rohith Sharma K S af0309d793 YARN-8405. RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773. Contributed by Íñigo Goiri. 
(cherry picked from commit 2df73dace0)
 2018-06-12 17:23:22 +05:30
Inigo Goiri 65d2554ad8 YARN-8370. Some Node Manager tests fail on Windows due to improper path/file separator. Contributed by Anbang Hu. 
(cherry picked from commit 2b2f672022)
 2018-06-11 19:26:27 -07:00
Inigo Goiri 63c20f9021 YARN-8359. Exclude containermanager.linux test classes on Windows. Contributed by Jason Lowe. 
(cherry picked from commit 3492a1db2c0654ce5375360caa74a34f928f23be)
 2018-06-07 17:10:09 -07:00
Rohith Sharma K S a95f216945 YARN-8399. NodeManager is giving 403 GSS exception post upgrade to 3.1 in secure mode. Contributed by Sunil Govindan. 
(cherry picked from commit 58bc34f1e3)
 2018-06-07 10:27:08 +05:30
Robert Kanter 93d6ed859e YARN-4677. RMNodeResourceUpdateEvent update from scheduler can lead to race condition (wilfreds and gphillips via rkanter) 
(cherry picked from commit 0cd145a443)
 2018-06-04 15:32:47 -07:00
Miklos Szegedi 21948b048e YARN-8382. cgroup file leak in NM. Contributed by Hu Ziqian. 
(cherry picked from commit 925fdf761a513130e23c10575c7328c8681cff1d)
 2018-06-04 10:56:13 -07:00
Billie Rinaldi e59e9feefa YARN-8342. Enable untrusted docker image to run with launch command. Contributed by Eric Yang 
(cherry picked from commit 31998643a5)
 2018-06-02 14:49:21 -07:00
Rohith Sharma K S f7d0ca71ac More YARN pages need to honor yarn.resourcemanager.display.per-user-apps(addendum). Contributed by Sunil G. 
(cherry picked from commit 8261f9e571)
 2018-06-02 09:43:08 +05:30
Wangda Tan 0be93a7973 YARN-8349. Remove YARN registry entries when a service is killed by the RM. (Billie Rinaldi via wangda) 
Change-Id: Ia58db3637789a8921482f564aa9bdf99c45cc36c
(cherry picked from commit ff583d3fa3)
 2018-06-01 15:53:26 -07:00
Wangda Tan 1cee85d8c4 YARN-7962. Race Condition When Stopping DelegationTokenRenewer causes RM crash during failover. (BELUGA BEHR via wangda) 
Change-Id: I617e2645f60a57080058ad5f06af860fb3f682c8
(cherry picked from commit 931f78718f)
 2018-06-01 15:47:51 -07:00
Wangda Tan cc460fea2d YARN-8384. stdout.txt, stderr.txt logs of a launched docker container is coming with primary group of submit user instead of hadoop. (Eric Yang via wangda) 
Change-Id: Idbb2cd250f4f62dc32993e0d3ca3ec5684616baa
(cherry picked from commit 3a6bd77550)
 2018-06-01 15:47:45 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) a468400bad YARN-8197. Fixed AM IP Filter and Webapp proxy to redirect app tracking-URLs correctly when UI is secure. Contributed by Sunil Govindan. 
(cherry picked from commit 6b74f5d7fc)
 2018-05-31 17:04:45 -07:00
Weiwei Yang ec4240a7fa YARN-8367. Fix NPE in SingleConstraintAppPlacementAllocator when placement constraint in SchedulingRequest is null. Contributed by Weiwei Yang. 
(Cherry picked from commit 6468071f13)
 2018-05-31 21:04:41 +08:00
Eric E Payne 3d06d75423 YARN-8377: Javadoc build failed in hadoop-yarn-server-nodemanager. Contributed by Takanobu Asanuma 
(cherry picked from commit e44c0849d7)
 2018-05-30 17:28:14 +00:00
Eric Yang 03209e8966 YARN-8362. Bugfix logic in container retries in node manager. 
Contributed by Chandni Singh

(cherry picked from commit 135941e00d)
 2018-05-29 17:04:01 -04:00
Jason Lowe a1fd04c4f4 YARN-8329. Docker client configuration can still be set incorrectly. Contributed by Shane Kumpf 
(cherry picked from commit 4827e9a908)
 2018-05-29 14:48:01 -05:00
Wangda Tan 3eb1cb18c7 YARN-8369. Javadoc build failed due to 'bad use of >'. (Takanobu Asanuma via wangda) 
Change-Id: I79a42154e8f86ab1c3cc939b3745024b8eebe5f4
(cherry picked from commit 17aa40f669)
 2018-05-29 09:28:34 -07:00
Jason Lowe 500b0ee2ce YARN-8338. TimelineService V1.5 doesn't come up after HADOOP-15406. Contributed by Vinod Kumar Vavilapalli 
(cherry picked from commit 31ab960f4f)
 2018-05-29 11:03:19 -05:00
Sunil G 7f9caf8986 YARN-4781. Support intra-queue preemption for fairness ordering policy. Contributed by Eric Payne. 
(cherry picked from commit 7c343669ba)
 2018-05-28 16:33:48 +05:30
Wangda Tan 4d41cb1696 YARN-8213. Add Capacity Scheduler performance metrics. (Weiwei Yang via wangda) 
Change-Id: Ieea6f3eeb83c90cd74233fea896f0fcd0f325d5f
(cherry picked from commit f24c842d52)
 2018-05-25 22:04:54 -07:00
Eric E Payne 053937f436 YARN-8292: Fix the dominant resource preemption cannot happen when some of the resource vector becomes negative. Contributed by Wangda Tan. 
(cherry picked from commit 8d5509c681)
 2018-05-25 16:27:09 +00:00
Shane Kumpf de43b41d0f YARN-6919. Add default volume mount list. Contributed by Eric Badger 2018-05-24 09:35:35 -06:00
Rohith Sharma K S 923c833da4 YARN-8319. More YARN pages need to honor yarn.resourcemanager.display.per-user-apps. Contributed by Sunil G. 
(cherry picked from commit c05b5d424b)
 2018-05-24 14:22:43 +05:30
Inigo Goiri 61b5b2f4f7 YARN-8344. Missing nm.stop() in TestNodeManagerResync to fix testKillContainersOnResync. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit e99e5bf104)
 2018-05-23 14:16:03 -07:00
Wangda Tan 6aaf33e50c YARN-8290. SystemMetricsPublisher.appACLsUpdated should be invoked after application information is published to ATS to avoid "User is not set in the application report" Exception. (Eric Yang via wangda) 
Change-Id: I0ac6ddd19740d1aa7dd07111cd11af71ddc2fcaf
(cherry picked from commit bd15d2396e)
 2018-05-22 13:39:15 -07:00
Eric Yang 7c3f236783 YARN-7960. Added security flag no-new-privileges for YARN Docker integration. 
Contributed by Eric Badger

(cherry picked from commit 6176d2b35c)
 2018-05-22 13:47:08 -04:00
Jason Lowe 9788c8c01e YARN-8206. Sending a kill does not immediately kill docker containers. Contributed by Eric Badger 
(cherry picked from commit 5f11288e41)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/TestDockerContainerRuntime.java
 2018-05-22 09:49:31 -05:00
Eric E Payne 5b2bbff141 YARN-8179: Preemption does not happen due to natural_termination_factor when DRF is used. Contributed by Kyungwan Nam. 
(cherry picked from commit 0b4c44bdee)
 2018-05-21 20:19:44 +00:00
Eric Yang aafaa5f99b YARN-8141. Removed YARN_CONTAINER_RUNTIME_DOCKER_LOCAL_RESOURCE_MOUNTS flag. 
Contributed by Chandni Singh

(cherry-picked from commit d45a0b7d73)
 2018-05-17 17:32:29 -04:00
Jason Lowe 79b2a508e0 YARN-8284. get_docker_command refactoring. Contributed by Eric Badger 
(cherry picked from commit d47c09dcb1)
 2018-05-16 09:34:24 -05:00
Weiwei Yang 2bb39330a4 YARN-8278. DistributedScheduling is not working in HA. Contributed by Bibin A Chundatt. 
(Cherry picked from commit 2bb647bb91)
 2018-05-15 18:47:15 +08:00
Haibo Chen 8d3598525d YARN-8130 Race condition when container events are published for KILLED applications. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit 2d00a0c71b)
 2018-05-15 11:40:47 +05:30
Eric Yang 8edcbe9460 YARN-8265. Improve DNS handling on docker IP changes. 
Contributed by Billie Rinaldi

(cherry picked from commit 0ff94563b9)
 2018-05-11 22:41:30 -07:00
Jason Lowe dfe73334c2 YARN-7654. Support ENTRY_POINT for docker container. Contributed by Eric Yang 
(cherry picked from commit 6c8e51ca7e)
 2018-05-11 18:57:20 -05:00
Jason Lowe f6d6a2ab04 YARN-8244. TestContainerSchedulerQueuing.testStartMultipleContainers failed. Contributed by Jim Brennan 
(cherry picked from commit dc912994a1)
 2018-05-11 14:13:55 -05:00
Eric Yang 645429ca28 YARN-8274. Fixed a bug on docker start command. 
Contributed by Jason Lowe
 2018-05-11 14:26:20 -04:00
Weiwei Yang b4213bfd90 YARN-7003. DRAINING state of queues is not recovered after RM restart. Contributed by Tao Yang. 
(Cherry picked from commit 9db9cd95bd)
 2018-05-11 10:55:05 +08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 4c0a6b67f3 YARN-8249. Fixed few REST APIs in RMWebServices to have static-user check. Contributed by Sunil Govindan. 
(cherry picked from commit d76fbbc9b8)
 2018-05-10 19:07:23 -07:00
Robert Kanter 0506c762b2 YARN-8202. DefaultAMSProcessor should properly check units of requested custom resource types against minimum/maximum allocation (snemeth via rkanter) 
(cherry picked from commit c8b53c4364)
 2018-05-10 09:52:11 -07:00
Vrushali C ad4d4153da YARN-8247 Incorrect HTTP status code returned by ATSv2 for non-whitelisted users. Contributed by Rohith Sharma K S 2018-05-09 22:30:59 -07:00
bibinchundatt b29479c93a YARN-8201. Skip stacktrace of few exception from ClientRMService. Contributed by Bilwa S T. 
(cherry picked from commit cc0310a526)
 2018-05-10 09:29:43 +05:30
Eric Yang 6ce89ea2a8 YARN-8261. Fixed a bug in creation of localized container directory. 
Contributed by Jason Lowe

(cherry picked from commit af4fc2e628)
 2018-05-09 13:34:54 -04:00
Rohith Sharma K S e3a7dbc4ec YARN-8239. [UI2] Clicking on Node Manager UI under AM container info / App Attempt page goes to old RM UI. Contributed by Sunil G. 
(cherry picked from commit eb7b256cc2)
 2018-05-09 12:33:49 +05:30
Jason Lowe 39236da2ff YARN-8207. Docker container launch use popen have risk of shell expansion. Contributed by Eric Yang. 
(cherry picked from commit a2ea756420)
 2018-05-08 15:37:24 -05:00
Wangda Tan 0873cfcb56 YARN-8232. RMContainer lost queue name when RM HA happens. (Hu Ziqian via wangda) 
Change-Id: Ia21e1da6871570c993bbedde76ce32929e95970f
(cherry picked from commit 6b96a73bb0)
 2018-05-08 11:35:11 -07:00
Rohith Sharma K S b69c65fc6b YARN-8253. HTTPS Ats v2 api call fails with 'bad HTTP parsed'. Contributed by Charan Hebri. 
(cherry picked from commit 7450583721)
 2018-05-08 12:31:35 +05:30
Weiwei Yang 68dd01efc1 YARN-8025. UsersManangers#getComputedResourceLimitForActiveUsers throws NPE due to preComputedActiveUserLimit is empty. Contributed by Tao Yang. 
(Cherry picked from commit 67f239c42f)
 2018-05-07 11:41:01 +08:00
Eric Yang a984abc472 YARN-8223. Improved yarn auxiliary service to load jar file from HDFS. 
Contributed by Zian Chen

(cherry picked from commit 8cdb032aff)
 2018-05-04 12:38:28 -04:00
Billie Rinaldi 04c3f11de4 YARN-7818. Remove privileged operation warnings during container launch for the ContainerRuntimes. Contributed by Shane Kumpf 
(cherry picked from commit 502914ca32)
 2018-05-04 08:55:30 -07:00
Eric Yang cf46533bca YARN-7973. Added ContainerRelaunch feature for Docker containers. 
Addendum patch 001
            Contributed by Shane Kumpf
 2018-05-03 12:46:04 -04:00
Eric Yang f729fb0baa YARN-8194. Fixed reinitialization error for LinuxContainerExecutor. 
Contributed by Chandni Singh

(cherry picked from commit f4d280f02b)
 2018-05-03 11:59:35 -04:00
Eric Yang e4b9981612 YARN-7973. Added ContainerRelaunch feature for Docker containers. 
Contributed by Shane Kumpf
 2018-05-03 11:57:01 -04:00
Rohith Sharma K S 4c93cd228c YARN-8217. RmAuthenticationFilterInitializer and TimelineAuthenticationFilterInitializer should use Configuration.getPropsWithPrefix instead of iterator. Contributed by Suma Shivaprasad. 
(cherry picked from commit ee2ce923a9)
 2018-05-03 18:20:21 +05:30
Eric Yang 61fac20c9e YARN-8209. Fixed NPE in Yarn Service deletion. 
Contributed by Eric Badger
 2018-05-02 15:05:30 -04:00
Weiwei Yang 6fce88765a YARN-8222. Fix potential NPE when gets RMApp from RM context. Contributed by Tao Yang. 2018-05-02 18:02:24 +08:00
Yiqun Lin 2741af505f
YARN-6385. Fix checkstyle warnings in TestFileSystemApplicationHistoryStore 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3265b55119)
 2018-05-02 18:15:53 +09:00
Weiwei Yang f34f4e9c11 YARN-8212. Pending backlog for async allocation threads should be configurable. Contributed by Tao Yang. 
(cherry picked from commit 2d319e3793)
 2018-05-01 09:57:15 +08:00
Eric Yang 0fc55a8f57 YARN-8228. Added hostname length check for docker container. 
Contributed by Shane Kumpf

(cherry picked from commit a966ec6e23)
 2018-04-30 19:14:37 -04:00
Konstantinos Karanasos 9d2967098d YARN-8195. Fix constraint cardinality check in the presence of multiple target allocation tags. Contributed by Weiwei Yang. 
(cherry picked from commit 9b09555451)
 2018-04-30 11:55:26 -07:00
Wangda Tan 090a8a274e YARN-8005. Add unit tests for queue priority with dominant resource calculator. (Zian Chen via wangda) 
Change-Id: I17a645f20869a1e5d86fa7a325c93fec908b91dc
(cherry picked from commit 92c5331423)
 2018-04-27 13:13:52 -07:00
Rohith Sharma K S b3fd464e55 YARN-8221. RMWebServices also need to honor yarn.resourcemanager.display.per-user-apps. Contributed by Sunil G. 
(cherry picked from commit ef3ecc308d)
 2018-04-27 22:59:16 +05:30
Sunil G 28e63f3d88 YARN-8004. Add unit tests for inter queue preemption for dominant resource calculator. Contributed by Zian Chen. 
(cherry picked from commit 71220d218d)
 2018-04-27 10:42:59 +05:30
Sunil G a1a2f1ddc1 YARN-8215. ATS v2 returns invalid YARN_CONTAINER_ALLOCATED_HOST_HTTP_ADDRESS from NM. Contributed by Rohith Sharma K S. 
(cherry picked from commit 7fc09c4f72)
 2018-04-27 10:34:29 +05:30
Sunil G e2901d97ba YARN-8205. Application State is not updated to ATS if AM launching is delayed. Contributed by Rohith Sharma K S. 
(cherry picked from commit 1634de0fc1)
 2018-04-27 10:28:18 +05:30
Eric Yang 050bd4cd2a YARN-8043. Added the exception message for failed launches running under LCE. 
Contributed by Shane Kumpf
 2018-04-26 08:26:10 -07:00
Wangda Tan af286319fd YARN-8193. YARN RM hangs abruptly (stops allocating resources) when running successive applications. (Zian Chen via wangda) 
Change-Id: Ia83dd2499ee9000b9e09ae5a932f21a13c0ddee6
(cherry picked from commit af986b442b7be0f2fad1241ca9e267a15c20bf43)
 2018-04-25 22:16:07 -07:00
Sunil G 44d28692a8 YARN-8188. RM Nodes UI data table index for sorting column need to be corrected post Application tags display. Contributed by Weiwei Yang. 
(cherry picked from commit 626690612cd0957316628376744a8be62f891665)
 2018-04-25 14:58:07 +05:30
Wangda Tan 3043a93d46 YARN-8183. Fix ConcurrentModificationException inside RMAppAttemptMetrics#convertAtomicLongMaptoLongMap. (Suma Shivaprasad via wangda) 
Change-Id: I347871d672001653a3afe2e99adefd74e0d798cd
(cherry picked from commit bb3c504764f807fccba7f28298a12e2296f284cb)
 2018-04-24 17:44:58 -07:00
Robert Kanter ea7ad50499 HADOOP-15390. Yarn RM logs flooded by DelegationTokenRenewer trying to renew KMS tokens (xiaochen via rkanter) 
(cherry picked from commit 7ab08a9c37)
 2018-04-23 16:25:41 -07:00
Eric Yang 5556cf397c YARN-8064. Bugfix for clean up container-executor cmd helper file. 
Contributed by Eric Badger
 2018-04-20 19:12:06 -04:00
Konstantinos Karanasos 18c86a3fb6 YARN-8111. Simplify PlacementConstraints API by removing allocationTagToIntraApp. Contributed by Weiwei Yang. 
(cherry picked from commit 28e2244390)
 2018-04-20 12:25:08 -07:00
Jason Lowe 71b0d52987 YARN-7786. NullPointerException while launching ApplicationMaster. Contributed by lujie 
(cherry picked from commit 766544c0b0)
 2018-04-20 13:23:46 -05:00
Sunil G 162db6e826 YARN-6827. [ATS1/1.5] NPE exception while publishing recovering applications into ATS during RM restart. Contributed by Rohith Sharma K S. 
(cherry picked from commit 7d06806dfd)
 2018-04-20 00:06:44 +05:30
Weiwei Yang c3ca10c22a YARN-7734. Fix UT failure TestContainerLogsPage#testContainerLogPageAccess. Contributed by Tao Yang. 
(cherry picked from commit 411993f6e5)
 2018-04-18 19:40:34 -07:00
Shashikant Banerjee a13b3edfd8
HADOOP-15396. Some java source files are executable 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e4c39f3247)
 2018-04-19 08:36:32 +09:00
Rohith Sharma K S fa007d986b YARN-8145. yarn rmadmin -getGroups doesn't return updated groups for user. Contributed by Sunil G. 
(cherry picked from commit 241d985467)
 2018-04-18 13:40:32 +05:30
Inigo Goiri 8f300c9ce1 YARN-8165. Incorrect queue name logging in AbstractContainerAllocator. Contributed by Weiwei Yan. 
(cherry picked from commit dd5e18c4ae)
 2018-04-17 15:28:13 +05:30
Wangda Tan 6ee62e6b1c YARN-8138. Add unit test to validate queue priority preemption works under node partition. (Zian Chen via wangda) 
Change-Id: Ibebfab98a714c12c2dc643b6d7b9754a7f813632
 2018-04-14 11:04:49 -07:00
Rohith Sharma K S eb1026dffa YARN-8156. Increase the default value of yarn.timeline-service.app-collector.linger-period.ms. Contributed by Charan Hebri. 
(cherry picked from commit 669eb7bdea)
 2018-04-14 10:57:02 +05:30
Wangda Tan 994c7d66e0 YARN-8091. Revisit checkUserAccessToQueue RM REST API. (wangda) 
Change-Id: I5fab3fe229c34e967487b7327c7b3c8ddf7cb795
 2018-04-13 10:19:10 -07:00
Wangda Tan 7031a853f4 YARN-8048. Support auto-spawning of admin configured services during bootstrap of RM (Rohith Sharma K S via wangda) 
Change-Id: I2d8d61ccad55e1118009294d7e17822df3cd0fd5
 2018-04-13 10:18:48 -07:00
Sunil G 21717db6a0 YARN-8028. Support authorizeUserAccessToQueue in RMWebServices. Contributed by Wangda Tan. 2018-04-13 10:18:12 -07:00
Wangda Tan c962371430 YARN-1151. Ability to configure auxiliary services from HDFS-based JAR files. (Xuan Gong via wangda) 
Change-Id: Ied37ff11e507fc86847753ba79486652c8fadfe9
 2018-04-13 10:17:47 -07:00
Weiwei Yang 84531ad9b6 YARN-8153. Guaranteed containers always stay in SCHEDULED on NM after restart. Contributed by Yang Wang. 2018-04-13 13:29:43 +08:00
Eric E Payne 9693d8b99d YARN-8147. TestClientRMService#testGetApplications sporadically fails. Contributed by Jason Lowe 
(cherry picked from commit 18844599ae)
 2018-04-12 17:58:41 +00:00
Eric E Payne f955d52f39 YARN-8120. JVM can crash with SIGSEGV when exiting due to custom leveldb logger. Contributed by Jason Lowe. 
(cherry picked from commit 6bb128dfb8)
 2018-04-12 16:11:57 +00:00
Billie Rinaldi 11f1d49823 YARN-7221. Add security check for privileged docker container. Contributed by Eric Yang 
(cherry picked from commit 933477e9e0)
 2018-04-11 12:15:00 -07:00
Weiwei Yang c2036af196 YARN-8127. Resource leak when async scheduling is enabled. Contributed by Tao Yang. 
(cherry picked from commit 7eb783e263)
 2018-04-11 17:21:47 +08:00
Vrushali C 67cd7acf99 YARN-8073 TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration. Contributed by Rohith Sharma K S 
(cherry picked from commit 345e7624d5)
 2018-04-11 09:41:04 +05:30
Wangda Tan f5dc7d1edf YARN-8116. Nodemanager fails with NumberFormatException: For input string: . (Chandni Singh via wangda) 
Change-Id: Idd30cfca59982d3fc6e47aa1b88f844a78fae94d
(cherry picked from commit 2bf9cc2c73)
 2018-04-10 17:38:31 -07:00
Wangda Tan 25bf1d3ac6 Updated branch-3.1 to track 3.1.1 
Change-Id: Idd55c79921b8acacc8a8a3258450e476f3abb706
 2018-04-09 10:58:50 -07:00
Sunil G 8311fcc75b YARN-7574. Add support for Node Labels on Auto Created Leaf Queue Template. Contributed by Suma Shivaprasad. 
(cherry picked from commit 821b0de4c5)
 2018-04-09 21:17:56 +05:30
bibinchundatt 091db4d0eb YARN-7905. Parent directory permission incorrect during public localization. Contributed by Bilwa S T. 
(cherry picked from commit eb47c3de74)
 2018-04-08 14:49:36 +05:30
Subru Krishnan b9bad94596 YARN-8110. AMRMProxy recover should catch for all throwable to avoid premature exit. (Botong Huang via Subru). 
(cherry picked from commit 00905efab2)
 2018-04-06 16:35:29 -07:00
Haibo Chen b232dcab33 YARN-8107. Give an informative message when incorrect format is used in ATSv2 filter attributes. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit 024d7c0870)
 2018-04-06 09:39:45 -07:00
Haibo Chen 6658018410 YARN-6936. [Atsv2] Retrospect storing entities into sub application table from client perspective. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit f8b8bd53c4)
 2018-04-05 11:31:17 -07:00
Konstantinos Karanasos fa464c12ad YARN-8013. Support application tags when defining application namespaces for placement constraints. Contributed by Weiwei Yang. 
(cherry picked from commit 7853ec8d2f)
 2018-04-04 10:53:33 -07:00
Wangda Tan 78832eca8a YARN-8002. Support NOT_SELF and ALL namespace types for allocation tag. (Weiwei Yang via wangda) 
Change-Id: I63b4e4192a95bf7ded98c54e46a2871c72869700
 2018-04-04 09:42:49 +08:00
Sunil G 73cd67d7e3 YARN-8085. ResourceProfilesManager should be set in RMActiveServiceContext. Contributed by Tao Yang. 
(cherry picked from commit 7a59d60e0c)
 2018-03-29 21:41:52 +05:30
Subru Krishnan a0091ec4b3 YARN-8010. Add config in FederationRMFailoverProxy to not bypass facade cache when failing over. (Botong Huang via Subru). 
(cherry picked from commit 09999d7e01)
 2018-03-28 11:35:45 -07:00
Subru Krishnan ed49f59e18 Revert "YARN-8010. Add config in FederationRMFailoverProxy to not bypass facade cache when failing over. (Botong Huang via Subru)." 
This reverts commit 85eebf1beb.
 2018-03-28 11:35:30 -07:00
Wangda Tan a2b4daab55 YARN-6629. NPE occurred when container allocation proposal is applied but its resource requests are removed before. (Tao Yang via wangda) 
Change-Id: I805880f90b3f6798ec96ed8e8e75755f390a9ad5
(cherry picked from commit 47f711eebc)
 2018-03-28 11:22:26 -07:00
Subru Krishnan 85eebf1beb YARN-8010. Add config in FederationRMFailoverProxy to not bypass facade cache when failing over. (Botong Huang via Subru). 
(cherry picked from commit 2a2ef15caf)
 2018-03-27 17:44:00 -07:00
Wangda Tan dc99611878 YARN-8072. RM log is getting flooded with MemoryPlacementConstraintManager info logs. (Zian Chen via wangda) 
Change-Id: I2567f4669ad868e13c4301fe44c4bde85bb92c37
(cherry picked from commit 24bc5e0e4d)
 2018-03-26 11:27:40 -07:00
Wangda Tan 62b9537332 YARN-8068. Application Priority field causes NPE in app timeline publish when Hadoop 2.7 based clients to 2.8+ (Sunil G via wangda) 
Change-Id: I7910bd1064a1b4dbbe2084080c060822ea6f3b48
(cherry picked from commit 9eef19b2ad)
 2018-03-26 11:27:32 -07:00
Wangda Tan 54da6a0d78 YARN-8062. yarn rmadmin -getGroups returns group from which the user has been removed. (Sunil G via wangda) 
Change-Id: I80ed63846502bf7751b890b6c6c6a7c0679e2b4a
(cherry picked from commit 5d381570f8)
 2018-03-26 11:27:21 -07:00
Jason Lowe 5d4b2c31a1 YARN-8054. Improve robustness of the LocalDirsHandlerService MonitoringTimerTask thread. Contributed by Jonathan Eagles 
(cherry picked from commit 5aa7052e31)
 2018-03-21 17:20:36 -05:00
Rohith Sharma K S bbfe36d686 YARN-7581. HBase filters are not constructed correctly in ATSv2. Contributed by Habio Chen. 
(cherry picked from commit 29acea5000)
 2018-03-21 08:26:56 +05:30
Haibo Chen d8764f1c90 YARN-8053. Add hadoop-distcp in exclusion in hbase-server dependencies for timelineservice-hbase packages. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit 3ff6977d3e)
 2018-03-21 08:26:56 +05:30
Jason Lowe a4f0dbcb44 YARN-8027. Setting hostname of docker container breaks for --net=host in docker 1.13. Contributed by Jim Brennan 
(cherry picked from commit f480367af6)
 2018-03-19 17:44:06 -05:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 0bd99621c5 Preparing for 3.1.0 development 2018-03-19 15:16:54 -07:00
Yufei Gu 49c747ab18 YARN-8039. Clean up log dir configuration in TestLinuxContainerExecutorWithMocks.testStartLocalizer. Contributed by Miklos Szegedi. 2018-03-16 16:25:33 -07:00
Weiwei Yang 154cfb2b62 YARN-7636. Re-reservation count may overflow when cluster resource exhausted for a long time. contributed by Tao Yang. 2018-03-16 18:57:31 +08:00
Wangda Tan 4bf622043f YARN-7952. RM should be able to recover log aggregation status after restart/fail-over. (Xuan Gong via wangda) 
Change-Id: I725c9afe64831eda0aa6b0bebdbc79d2dd165707
 2018-03-15 13:26:45 -07:00
fang zhenyi 5e013d50d1
HADOOP-15305. Replace FileUtils.writeStringToFile(File, String) with (File, String, Charset) to fix deprecation warnings. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-15 16:12:07 +09:00
Eric Yang a82be7754d YARN-7999. Added file cache initializer for Linux container-executor. 
Contributed by Jason Lowe
 2018-03-14 12:21:57 -04:00
Rohith Sharma K S e6de10d0a6 YARN-8022. ResourceManager UI cluster/app/<app-id> page fails to render. Contributed by Tarun Parimi. 2018-03-14 09:42:20 +05:30
Wangda Tan b167d60763 YARN-7657. Queue Mapping could provide options to provide 'user' specific auto-created queues under a specified group parent queue. (Suma Shivaprasad via wangda) 
Change-Id: I32d566e8727840e43c0d66e39a77edef017e3a83
 2018-03-13 17:55:51 -07:00
Wangda Tan a5b27b3c67 YARN-5015. Support sliding window retry capability for container restart. (Chandni Singh via wangda) 
Change-Id: I07addd3e4ba8d98456ee2ff1d5c540a38fe61dea
 2018-03-13 17:55:17 -07:00
Miklos Szegedi a82d4a2e3a YARN-5764. NUMA awareness support for launching containers. Contributed by Devaraj K. 2018-03-13 12:36:57 -07:00
Yufei Gu ff31d8aefa YARN-8024. LOG in class MaxRunningAppsEnforcer is initialized with a faulty class. Contributed by Sen Zhao. 2018-03-12 16:35:26 -07:00
Wangda Tan ea18e70a74 Updated timeline reader to use AuthenticationFilter 
Change-Id: I961771589180c1eb377d36c37a79aa23754effbf
(cherry picked from commit 837338788eb903d0e8bbb1230694782a707891be)
 2018-03-09 22:51:08 -08:00
Owen O'Malley 3a8dade9b1 Revert "HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu." 
This reverts commit 172b23af33.

(cherry picked from commit d0d2d4c51e9534e08893ae14cf3fff7b2ee70b1d)
 2018-03-09 22:46:30 -08:00
Weiwei Yang b451889e8e YARN-8011. TestOpportunisticContainerAllocatorAMService#testContainerPromoteAndDemoteBeforeContainerStart fails intermittently. Contributed by Tao Yang. 2018-03-08 18:13:36 +08:00
Wangda Tan e718ac597f Revert "YARN-7891. LogAggregationIndexedFileController should support read from HAR file. (Xuan Gong via wangda)" 
This reverts commit 4d53ef7eef.
 2018-03-07 15:46:47 -08:00
Wangda Tan 037d783483 YARN-7626. Allow regular expression matching in container-executor.cfg for devices and named docker volumes mount. (Zian Chen via wangda) 
Change-Id: If461277d4557922ab7e4dce9dd8dc5d0d5f22710
(cherry picked from commit 88f9138e12d2d5a1bd13f0915acef93037c1d086)
 2018-03-07 11:30:15 -08:00
Wangda Tan 4d53ef7eef YARN-7891. LogAggregationIndexedFileController should support read from HAR file. (Xuan Gong via wangda) 
Change-Id: Ie16e34039d57df50128c73b37516ad0bc7c9590e
 2018-03-07 11:30:06 -08:00
Jason Lowe d69b31f7f7 YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan 2018-03-07 09:20:34 -06:00
Rohith Sharma K S 55ba49dd07 YARN-7346. Add a profile to allow optional compilation for ATSv2 with HBase-2.0. Contributed by Haibo Chen and Rohith. 2018-03-06 11:55:21 +05:30
Arun Suresh 1054b48c27 YARN-7972. Support inter-app placement constraints for allocation tags by application ID. (Weiwei Yang via asuresh) 2018-03-05 11:24:17 -08:00
Billie Rinaldi 628be58a4c YARN-7915. Trusted image log message repeated multiple times. Contributed by Shane Kumpf 2018-03-05 09:08:44 -08:00
Jason Lowe 83798f15f8 YARN-7511. NPE in ContainerLocalizer when localization failed for running container. Contributed by Tao Yang 2018-03-02 09:59:46 -06:00
Ray Chiang 923e1773e1 HADOOP-14652. Update metrics-core version to 3.2.4. (rchiang) 2018-03-01 15:36:10 -08:00
Sunil G 22928c0c60 YARN-7637. GPU volume creation command fails when work preserving is disabled at NM. Contributed by Zian Chen. 2018-03-01 22:39:32 +05:30
Yufei Gu 6ca4e3017b YARN-5028. RMStateStore should trim down app state for completed applications(addendum). Contributed by Rohith Sharma K S. 2018-03-01 08:38:53 -08:00
Haibo Chen 10663b78c8 Revert "yarn-7346.07.patch" 
This reverts commit 5e37ca5bb4.
 2018-02-28 21:11:36 -08:00
Haibo Chen d1274c3b71 YARN-7835. Race condition in NM while publishing events if second attempt is launched on the same node. (Rohith Sharma K S via Haibo Chen) 2018-02-28 21:10:42 -08:00
Haibo Chen 5e37ca5bb4 yarn-7346.07.patch 2018-02-28 21:04:54 -08:00
Billie Rinaldi 727c033997 YARN-7446. Remove --user flag when running privileged mode docker container. Contributed by Eric Yang 2018-02-27 14:33:57 -08:00
Arpit Agarwal 2fa7963c3d HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi. 2018-02-24 14:41:55 -08:00
Jason Lowe 8e728f39c9 YARN-5714. ContainerExecutor does not order environment map. Contributed by Remi Catherinot and Jim Brennan 2018-02-23 15:46:35 -06:00
Carlo Curino 514794e1a5 YARN-7934. [GQ] Refactor preemption calculators to allow overriding for Federation Global Algos. (Contributed by curino) 2018-02-22 18:12:12 -08:00
Jason Lowe 3132709b46 YARN-7916. Remove call to docker logs on failure in container-executor. Contributed by Shane Kumpf 2018-02-21 16:54:02 -06:00
Yufei Gu 92cbbfe79e YARN-5028. RMStateStore should trim down app state for completed applications. Contributed by Gergo Repas. 2018-02-21 11:42:51 -08:00
Sunil G bdd2a184d7 YARN-7947. Capacity Scheduler intra-queue preemption can NPE for non-schedulable apps. Contributed by Eric Payne. 2018-02-21 14:35:57 +05:30
Jason Lowe 9497215084 YARN-7813. Capacity Scheduler Intra-queue Preemption should be configurable for each queue. Contributed by Eric Payne 2018-02-19 14:06:28 -06:00
Jason Lowe b9a429bb28 Revert "YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan" 
This reverts commit 8013475d44.
 2018-02-19 08:16:25 -06:00
fang zhenyi 4d4dde5112
HADOOP-15223. Replace Collections.EMPTY* with empty* when available 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-02-18 22:19:39 +09:00
Rohith Sharma K S 9af30d46c6 YARN-7919. Refactor timelineservice-hbase module into submodules. Contributed by Haibo Chen. 2018-02-17 20:30:28 +05:30
Arun Suresh a1e56a6286 YARN-7918. Fix TestAMRMClientPlacementConstraints. (Gergely Novák via asuresh) 2018-02-17 03:24:55 -08:00
Sunil G aae629913c YARN-7292. Retrospect Resource Profile Behavior for overriding capability. Contributed by Wangda Tan. 2018-02-16 12:37:40 +05:30
Jason Lowe 8013475d44 YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan 2018-02-15 17:09:00 -06:00
Konstantinos Karanasos 0b489e564c YARN-7920. Simplify configuration for PlacementConstraints. Contributed by Wangda Tan. 2018-02-15 14:23:38 -08:00
Eric Payne bddfe42e2c Revert "YARN-7813: Capacity Scheduler Intra-queue Preemption should be configurable for each queue" 
This reverts commit c5e6e3de1c.
 2018-02-14 14:29:20 -06:00
Wangda Tan 042ef2fa7b YARN-7789. Should fail RM if 3rd resource type is configured but RM uses DefaultResourceCalculator. (Zian Chen via wangda) 
Change-Id: I1f6a451fe16758def6f47c046a9b8a67ad7255af
 2018-02-14 23:11:10 +08:00
Eric Payne c5e6e3de1c YARN-7813: Capacity Scheduler Intra-queue Preemption should be configurable for each queue 2018-02-13 10:11:02 -06:00
Jason Lowe 5a1db60ab1 YARN-7914. Fix exit code handling for short lived Docker containers. Contributed by Shane Kumpf 2018-02-12 15:50:10 -06:00
Jason Lowe 3414fd1243 YARN-7917. Fix failing test TestDockerContainerRuntime#testLaunchContainerWithDockerTokens. Contributed by Shane Kumpf 2018-02-12 09:27:43 -06:00
Wangda Tan d02e42cee4 YARN-7739. DefaultAMSProcessor should properly check customized resource types against minimum/maximum allocation. (wangda) 
Change-Id: I10cc9341237d9a2fc0f8c855efb98a36b91389e2
 2018-02-12 10:29:37 +08:00
Wangda Tan 60f9e60b3b Preparing for 3.2.0 development 
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
 2018-02-11 11:17:38 +08:00
Yufei Gu 1bc03ddf97 YARN-7655. Avoid AM preemption caused by RRs for specific nodes or racks. Contributed by Steven Rand. 2018-02-08 12:32:43 -08:00
Jian He eb2449d539 YARN-5428. Allow for specifying the docker client configuration directory. Contributed by Shane Kumpf 2018-02-08 11:35:30 -08:00
Jason Lowe 456705a07c YARN-7815. Make the YARN mounts added to Docker containers more restrictive. Contributed by Shane Kumpf 2018-02-07 13:09:08 -06:00
Billie Rinaldi 266da25c04 Revert "YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi." 
This reverts commit e14f03dfbf.
 2018-02-07 07:17:00 -08:00
Billie Rinaldi aa461f9091 YARN-7516. Add security check for trusted docker images. Contributed by Eric Yang 2018-02-07 07:09:45 -08:00
Robert Kanter 814d701d46 YARN-7841. Cleanup AllocationFileLoaderService's reloadAllocations method (snemeth via rkanter) 2018-02-06 14:36:49 -08:00
Anu Engineer 4304fcd5bd HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen. 2018-02-06 13:43:45 -08:00
Weiwei Yang bff858e910 YARN-7860. Fix UT failure TestRMWebServiceAppsNodelabel#testAppsRunning. Contributed by Sunil G. 2018-02-06 19:33:04 +08:00
Billie Rinaldi a196ee9362 YARN-7890. NPE during container relaunch. Contributed by Jason Lowe 2018-02-05 18:04:09 -08:00
Jason Lowe 60656bceca YARN-7849. TestMiniYarnClusterNodeUtilization#testUpdateNodeUtilization fails due to heartbeat sync error. Contributed by Botong Huang 2018-02-05 17:16:04 -06:00
Konstantinos Karanasos 50723889cc YARN-7778. Merging of placement constraints defined at different levels. Contributed by Weiwei Yang. 2018-02-02 14:46:20 -08:00
Arun Suresh 6e5ba9366f YARN-7839. Modify PlacementAlgorithm to Check node capacity before placing request on node. (Panagiotis Garefalakis via asuresh) 2018-02-02 10:28:22 -08:00
Jason Lowe 682ea21f2b Revert "YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan" 
This reverts commit 12eaae383a.
 2018-02-01 12:14:09 -06:00
Jason Lowe 12eaae383a YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan 2018-01-31 11:37:54 -06:00
Arun Suresh d481344783 YARN-7822. Constraint satisfaction checker support for composite OR and AND constraints. (Weiwei Yang via asuresh) 2018-01-31 01:51:08 -08:00
Arun Suresh add993e26a YARN-6597. Add RMContainer recovery test to verify tag population in the AllocationTagsManager. (Panagiotis Garefalakis via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh f8c5f5b237 YARN-7784. Fix Cluster metrics when placement processor is enabled. (asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh c23980c4f2 YARN-7795. Fix jenkins issues of YARN-6592 branch. (Sunil G via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh 644afe5fd8 YARN-7807. Assume intra-app anti-affinity as default for scheduling request inside AppPlacementAllocator. (Wangda Tan via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh a4c539fcdb YARN-7783. Add validation step to ensure constraints are not violated due to order in which a request is processed. (asuresh) 2018-01-31 01:30:17 -08:00
Sunil G 9b81cb0537 YARN-7779. Display allocation tags in RM web UI and expose same through REST API. Contributed by Weiwei Yang. 2018-01-31 01:30:17 -08:00
Konstantinos Karanasos adbe87abf8 YARN-7788. Factor out management of temp tags from AllocationTagsManager. (Arun Suresh via kkaranasos) 2018-01-31 01:30:17 -08:00
Arun Suresh 8bf7c44436 YARN-7763. Allow Constraints specified in the SchedulingRequest to override application level constraints. (Weiwei Yang via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh 28fe7f3318 YARN-7774. Miscellaneous fixes to the PlacementProcessor. (asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh 38af237969 YARN-6599. Support anti-affinity constraint via AppPlacementAllocator. (Wangda Tan via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh 8779a35742 YARN-7709. Remove SELF from TargetExpression type. (Konstantinos Karanasos via asuresh) 2018-01-31 01:30:17 -08:00
Wangda Tan 29d9e4d581 YARN-6619. AMRMClient Changes to use the PlacementConstraint and SchcedulingRequest objects. (Arun Suresh via wangda) 2018-01-31 01:30:17 -08:00
Arun Suresh a5c1fc881e YARN-7696. Add container tags to ContainerTokenIdentifier, api.Container and NMContainerStatus to handle all recovery cases. (asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh 4eda58c136 YARN-7681. Double-check placement constraints in scheduling phase before actual allocation is made. (Weiwei Yang via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh bdba01f73b YARN-7682. Expose canSatisfyConstraints utility function to validate a placement against a constraint. (Panagiotis Garefalakis via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh a52d11fb8c YARN-7613. Implement Basic algorithm for constraint based placement. (Panagiotis Garefalakis via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh f9af15d659 YARN-7612. Add Processor Framework for Rich Placement Constraints. (asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh 1efb2b6f25 YARN-6596. Introduce Placement Constraint Manager module. (Konstantinos Karanasos via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh 37f1a7b64f YARN-7653. Node group support for AllocationTagsManager. (Panagiotis Garefalakis via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh 06eb63e64b YARN-7669. API and interface modifications for placement constraint processor. (asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh 88d8d3f40b YARN-7670. Modifications to the ResourceScheduler API to support SchedulingRequests. (asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh 801c0988b5 YARN-7522. Introduce AllocationTagsManager to associate allocation tags to nodes. (Wangda Tan via asuresh) 2018-01-31 01:30:17 -08:00
Sunil G 6463e10c72 YARN-7723. Avoid using docker volume --format option to run against to older docker releases. Contributed by Wangda Tan 2018-01-30 15:58:11 +05:30
Sunil G e9c72d04be YARN-7790. Improve Capacity Scheduler Async Scheduling to better handle node failures. Contributed by Wangda Tan. 2018-01-29 20:44:38 +05:30
Jinjiang Ling 3400d0c535
YARN-7698. A misleading variable's name in ApplicationAttemptEventDispatcher 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-01-30 00:00:57 +09:00
Jian He 443523f9c0 YARN-7765. Fixed an issue that kerberos tgt not found when NM posting timeline events. Contributed by Rohith Sharma K S 2018-01-27 22:18:51 -08:00
Haibo Chen 649ef7ac33 YARN-7064. Use cgroup to get container resource utilization. (Miklos Szegedi via Haibo Chen) 2018-01-26 16:27:31 -08:00
Billie Rinaldi f2fa736f0a YARN-7797. Docker host network can not obtain IP address for RegistryDNS. Contributed by Eric Yang 2018-01-26 09:32:23 -08:00
Wangda Tan 75618c0dec YARN-7760. [UI2] Clicking 'Master Node' or link next to 'AM Node Web UI' under application's appAttempt page goes to OLD RM UI. (Vasudevan Skm / Sunil G via wangda) 
Change-Id: Ia7e2b955d3551ca43c6852fe253fd01983b56565
 2018-01-26 15:44:55 +08:00
Wangda Tan e0cfb0a31a YARN-7817. Add Resource reference to RM's NodeInfo object so REST API can get non memory/vcore resource usages. (Sunil G via wangda) 
Change-Id: Ia7ceeabd82046645ddeaf487c763288f36cfbdee
 2018-01-26 15:43:27 +08:00
Miklos Szegedi 2e5865606b YARN-7814. Remove automatic mounting of the cgroups root directory into Docker containers. Contributed by Shane Kumpf. 2018-01-25 17:37:23 -08:00
Jason Lowe ff8378eb1b YARN-7102. NM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 2018-01-25 17:47:19 -06:00
Sunil G 82cc6f6968 YARN-7728. Expose container preemptions related information in Capacity Scheduler queue metrics. Contributed by Eric Payne. 2018-01-25 14:32:54 +05:30
Eric Yang 59828be197 YARN-7810. Update TestDockerContainerRuntime to test with current user credential. 
(contributed by Shane Kumpf)
 2018-01-24 21:24:19 -05:00
Zhe Zhang fa8cf4d1b4 YARN-7737. prelaunch.err file not found exception on container failure. Contributed by Keqiu Hu. 2018-01-24 12:54:54 -08:00
Jian He 55c32776b1 YARN-7801. AmFilterInitializer should addFilter after fill all parameters. Contributed by Wangda Tan 2018-01-24 11:17:12 -08:00
Rohith Sharma K S f3e33aeb80 Revert "YARN-7537 [Atsv2] load hbase configuration from filesystem rather than URL. Contributed by Rohith Sharma" 
This reverts commit ec8f47e7fa.
 2018-01-24 15:04:15 +05:30
Miklos Szegedi e7642a3e6f YARN-7796. Container-executor fails with segfault on certain OS configurations. Contributed by Gergo Repas. 2018-01-23 21:07:05 -08:00
Billie Rinaldi e307edcb47 YARN-7540 and YARN-7605. Convert yarn app cli to call yarn api services and implement doAs for Api Service REST API. Contributed by Eric Yang and Jian He 2018-01-23 17:54:39 -08:00
Eric Yang 97fe3cc187 YARN-7729. Add support for setting Docker PID namespace mode. (Contributed by Billie Rinaldi) 2018-01-22 16:33:38 -05:00
Yufei Gu 22ee6f77e5 YARN-7755. Clean up deprecation messages for allocation increments in FS config. Contributed by Wilfred Spiegelenburg. 2018-01-22 10:30:01 -08:00
Vrushali C ec8f47e7fa YARN-7537 [Atsv2] load hbase configuration from filesystem rather than URL. Contributed by Rohith Sharma 2018-01-19 15:34:40 -08:00
Haibo Chen 4aca4ff759 YARN-5094. some YARN container events have timestamp of -1. 2018-01-19 10:27:46 -08:00
Miklos Szegedi bc93ac229e YARN-7139. FairScheduler: finished applications are always restored to default queue. Contributed by Wilfred Spiegelenburg. 2018-01-18 17:43:47 -08:00
Billie Rinaldi 37f4696a9c YARN-7740. Fix logging for destroy yarn service cli when app does not exist and some minor bugs. Contributed by Jian He 2018-01-18 12:11:19 -08:00
Sunil G 06cceba1cb YARN-7738. CapacityScheduler: Support refresh maximum allocation for multiple resource types. Contributed by Wangda Tan. 2018-01-18 19:05:26 +05:30
Miklos Szegedi a68e445dc6 YARN-7717. Add configuration consistency for module.enabled and docker.privileged-containers.enabled. Contributed by Eric Badger. 2018-01-17 14:11:14 -08:00
Miklos Szegedi 41049ba5d1 YARN-7758. Add an additional check to the validity of container and application ids passed to container-executor. Contributed by Yufei Gu. 2018-01-16 15:40:43 -08:00