Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,491 Commits 326 Branches 371 Tags
Commit Graph

4562 Commits

Author SHA1 Message Date
Eric Yang
03209e8966 YARN-8362. Bugfix logic in container retries in node manager. 
Contributed by Chandni Singh

(cherry picked from commit 135941e00d762a417c3b4cc524cdc59b0d1810b1)
 2018-05-29 17:04:01 -04:00
Jason Lowe
a1fd04c4f4 YARN-8329. Docker client configuration can still be set incorrectly. Contributed by Shane Kumpf 
(cherry picked from commit 4827e9a9085b306bc379cb6e0b1fe4b92326edcd)
 2018-05-29 14:48:01 -05:00
Wangda Tan
3eb1cb18c7 YARN-8369. Javadoc build failed due to 'bad use of >'. (Takanobu Asanuma via wangda) 
Change-Id: I79a42154e8f86ab1c3cc939b3745024b8eebe5f4
(cherry picked from commit 17aa40f669f197d43387d67dc00040d14cd00948)
 2018-05-29 09:28:34 -07:00
Wangda Tan
b262ea1381 YARN-8339. Service AM should localize static/archive resource types to container working directory instead of 'resources'. (Suma Shivaprasad via wangda) 
Change-Id: I9f8e8f621650347f6c2f9e3420edee9eb2f356a4
(cherry picked from commit 3061bfcde53210d2032df3814243498b27a997b7)
 2018-05-29 09:28:27 -07:00
Jason Lowe
500b0ee2ce YARN-8338. TimelineService V1.5 doesn't come up after HADOOP-15406. Contributed by Vinod Kumar Vavilapalli 
(cherry picked from commit 31ab960f4f931df273481927b897388895d803ba)
 2018-05-29 11:03:19 -05:00
Sunil G
7f9caf8986 YARN-4781. Support intra-queue preemption for fairness ordering policy. Contributed by Eric Payne. 
(cherry picked from commit 7c343669baf660df3b70d58987d6e68aec54d6fa)
 2018-05-28 16:33:48 +05:30
Rohith Sharma K S
f19e8e2246 MAPREDUCE-7097. MapReduce JHS should honor yarn.webapp.filter-entity-list-by-user. Contributed by Sunil Govindan. 
(cherry picked from commit 88cbe57c069a1d2dd3bfb32e3ad742566470a10b)
 2018-05-28 14:06:50 +05:30
Wangda Tan
4d41cb1696 YARN-8213. Add Capacity Scheduler performance metrics. (Weiwei Yang via wangda) 
Change-Id: Ieea6f3eeb83c90cd74233fea896f0fcd0f325d5f
(cherry picked from commit f24c842d52e166e8566337ef93c96438f1c870d8)
 2018-05-25 22:04:54 -07:00
Eric E Payne
053937f436 YARN-8292: Fix the dominant resource preemption cannot happen when some of the resource vector becomes negative. Contributed by Wangda Tan. 
(cherry picked from commit 8d5509c68156faaa6641f4e747fc9ff80adccf88)
 2018-05-25 16:27:09 +00:00
Miklos Szegedi
1483c90379 YARN-8310. Handle old NMTokenIdentifier, AMRMTokenIdentifier, and ContainerTokenIdentifier formats. Contributed by Robert Kanter. 
(cherry picked from commit 3e5f7ea986600e084fcac723b0423e7de1b3bb8a)
 2018-05-24 15:14:15 -07:00
Eric Yang
f49e697cc8 YARN-8357. Fixed NPE when YARN service is saved and not deployed. 
Contributed by Chandni Singh

(cherry picked from commit d9852eb5897a25323ab0302c2c0decb61d310e5e)
 2018-05-24 16:34:32 -04:00
Eric Yang
514c05abd0 YARN-8316. Improved diagnostic message for ATS unavailability for YARN Service. 
Contributed by Billie Rinaldi
 2018-05-24 16:26:53 -04:00
Shane Kumpf
de43b41d0f YARN-6919. Add default volume mount list. Contributed by Eric Badger 2018-05-24 09:35:35 -06:00
Rohith Sharma K S
923c833da4 YARN-8319. More YARN pages need to honor yarn.resourcemanager.display.per-user-apps. Contributed by Sunil G. 
(cherry picked from commit c05b5d424b000bab766f57e88a07f2b4e9a56647)
 2018-05-24 14:22:43 +05:30
Rohith Sharma K S
67842c56fe YARN-8346. Upgrading to 3.1 kills running containers with error 'Opportunistic container queue is full'. Contributed by Jason Lowe. 
(cherry picked from commit 4cc0c9b0baa93f5a1c0623eee353874e858a7caa)
 2018-05-24 12:25:33 +05:30
Eric Yang
05d905f586 YARN-7530. Refactored YARN service API project location. 
Contributed by Chandni Singh
 2018-05-23 22:43:54 -04:00
Inigo Goiri
fa8af4aab5 YARN-8327. Fix TestAggregatedLogFormat#testReadAcontainerLogs1 on Windows. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit f09dc73001fd5f3319765fa997f4b0ca9e8f2aff)
 2018-05-23 15:59:59 -07:00
Inigo Goiri
61b5b2f4f7 YARN-8344. Missing nm.stop() in TestNodeManagerResync to fix testKillContainersOnResync. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit e99e5bf104e9664bc1b43a2639d87355d47a77e2)
 2018-05-23 14:16:03 -07:00
Rohith Sharma K S
8dcc7b853b YARN-8297. Incorrect ATS Url used for Wire encrypted cluster.(addendum). Contributed by Sunil G. 
(cherry picked from commit f61e3e752eb1cf4a08030da04bc3d6c5a2b3926d)
 2018-05-23 18:32:57 +05:30
Wangda Tan
2bbf1b4435 YARN-8332. Incorrect min/max allocation property name in resource types doc. (Weiwei Yang via wangda) 
Change-Id: If74f1ceed9c045a2cb2d6593741278b65ac44a9f
(cherry picked from commit 83f53e5c6236de30c213dc41878cebfb02597e26)
 2018-05-22 13:39:20 -07:00
Wangda Tan
6aaf33e50c YARN-8290. SystemMetricsPublisher.appACLsUpdated should be invoked after application information is published to ATS to avoid "User is not set in the application report" Exception. (Eric Yang via wangda) 
Change-Id: I0ac6ddd19740d1aa7dd07111cd11af71ddc2fcaf
(cherry picked from commit bd15d2396ef0c24fb6b60c6393d16b37651b828e)
 2018-05-22 13:39:15 -07:00
Eric Yang
7c3f236783 YARN-7960. Added security flag no-new-privileges for YARN Docker integration. 
Contributed by Eric Badger

(cherry picked from commit 6176d2b35c85715aae93526236c29540f71ecac8)
 2018-05-22 13:47:08 -04:00
Jason Lowe
9788c8c01e YARN-8206. Sending a kill does not immediately kill docker containers. Contributed by Eric Badger 
(cherry picked from commit 5f11288e41fca2e414dcbea130c7702e29d4d610)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/TestDockerContainerRuntime.java
 2018-05-22 09:49:31 -05:00
Eric E Payne
5b2bbff141 YARN-8179: Preemption does not happen due to natural_termination_factor when DRF is used. Contributed by Kyungwan Nam. 
(cherry picked from commit 0b4c44bdeef62945b592d5761666ad026b629c0b)
 2018-05-21 20:19:44 +00:00
Eric Yang
61e8974049 YARN-7530. Refactored YARN service API project location. 
Contributed by Chandni Singh
 2018-05-18 17:33:18 -04:00
Eric Yang
0791785574 YARN-8296. Removed unique_component_support from YARN services. 
Contributed by Suma Shivaprasad

(cherry picked from commit 2f2dd22aad4a003228b9efe3b4c506a6922d09d8)
 2018-05-17 20:59:50 -04:00
Eric Yang
79351d4fda YARN-8141. Removed YARN_CONTAINER_RUNTIME_DOCKER_LOCAL_RESOURCE_MOUNTS flag. 
Contributed by Chandni Singh

(cherry picked from 989cfdc1e0f1c27b35679c668a9bde6a0be845b5)
 2018-05-17 20:57:14 -04:00
Eric Yang
aafaa5f99b YARN-8141. Removed YARN_CONTAINER_RUNTIME_DOCKER_LOCAL_RESOURCE_MOUNTS flag. 
Contributed by Chandni Singh

(cherry-picked from commit d45a0b7d73519acb78cd94ac3186bd8481f6c13e)
 2018-05-17 17:32:29 -04:00
Eric Yang
044573b628 YARN-8080. Add restart policy for YARN services. 
Contributed by Suma Shivaprasad

(cherry picked from commit 7f083ed8699a720d3fb82e4ec310356902a6ac30)
 2018-05-17 17:20:45 -04:00
Eric Yang
c6c3b0afa0 YARN-8293. Removed "User Name for service" for deploying secure YARN service. 
Contributed by Sunil G

(cherry picked from commit 7802af6e9a85f033e9515cc7b23c125a0e06c325)
 2018-05-17 17:07:55 -04:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
0460201c3a YARN-8128. Documenting the per-node per-app file limit in YARN log aggregation. Contributed by Xuan Gong. 
(cherry picked from commit 328f0847e3d98cad8c368d57499f31081c153237)
 2018-05-17 13:09:09 -07:00
Rohith Sharma K S
4222e8f21b YARN-8297. Incorrect ATS Url used for Wire encrypted cluster. Contributed by Sunil G. 
(cherry picked from commit c95795cd8f649c45f05df1d5faa1dcabd30d7a6e)
 2018-05-17 16:25:38 +05:30
Eric Yang
2c3f300464 YARN-8300. Fixed NPE in DefaultUpgradeComponentsFinder. 
Contributed by Suma Shivaprasad

(cherry picked from 91357c22ef0fb89652dd2898cf488b47234452ce)
 2018-05-16 12:42:22 -04:00
Jason Lowe
79b2a508e0 YARN-8284. get_docker_command refactoring. Contributed by Eric Badger 
(cherry picked from commit d47c09dcb146d0a1cb50486f453a67560812fe5c)
 2018-05-16 09:34:24 -05:00
Eric Yang
051d9cff9c YARN-8081. Add support to upgrade a component. 
Contributed by Chandni Singh

(cherry picked from commit 8d3b39de89809f5eed06f85f00ab223e2f75e49c)
 2018-05-15 20:42:24 -04:00
Weiwei Yang
2bb39330a4 YARN-8278. DistributedScheduling is not working in HA. Contributed by Bibin A Chundatt. 
(Cherry picked from commit 2bb647bb91439e82cf7298e963bb5f7f80bbc3cb)
 2018-05-15 18:47:15 +08:00
Sunil G
0a30a548f1 YARN-8236. Invalid kerberos principal file name cause NPE in native service. Contributed by Gour Saha. 
(cherry picked from commit 58b97c79e34901938d59acc84ed48c1f9344996a)
 2018-05-15 12:18:09 +05:30
Sunil G
d997959560 YARN-8166. [UI2] Service page header links are broken. Contributed by Yesha Vora. 
(cherry picked from commit ffb9210dedb79a56075448dc296251896bed49e6)
 2018-05-15 12:13:44 +05:30
Sunil G
d0452370cc YARN-8266. [UI2] Clicking on application from cluster view should redirect to application attempt page. Contributed by Yesha Vora. 
(cherry picked from commit 796b2b0ee36e8e9225fb76ae35edc58ad907b737)
 2018-05-15 12:02:40 +05:30
Haibo Chen
8d3598525d YARN-8130 Race condition when container events are published for KILLED applications. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit 2d00a0c71b5dde31e2cf8fcb96d9d541d41fb879)
 2018-05-15 11:40:47 +05:30
Naganarasimha
26e8fdffd6 YARN-8288. Fix wrong number of table columns in Resource Model doc. Contributed by Weiwei Yang. 
(cherry picked from commit 8a2b5914f3a68148f40f99105acf5dafcc326e89)
 2018-05-15 00:10:58 +08:00
Sunil G
e6c607cc2c YARN-8271. [UI2] Improve labeling of certain tables. Contributed by Yesha Vora. 
(cherry picked from commit 89d0b87ad324db09f14e00031d20635083d576ed)
 2018-05-14 21:11:59 +05:30
Eric Yang
8edcbe9460 YARN-8265. Improve DNS handling on docker IP changes. 
Contributed by Billie Rinaldi

(cherry picked from commit 0ff94563b9b62d0426d475dc0f84152b68f1ff0d)
 2018-05-11 22:41:30 -07:00
Jason Lowe
dfe73334c2 YARN-7654. Support ENTRY_POINT for docker container. Contributed by Eric Yang 
(cherry picked from commit 6c8e51ca7eaaeef0626658b3c45d446a537e4dc0)
 2018-05-11 18:57:20 -05:00
Billie Rinaldi
355ff085e6 YARN-8243. Flex down should remove instance with largest component instance ID first. Contributed by Gour Saha 
(cherry picked from commit ca612e353fc3e3766868ec0816de035e48b1f5b4)
 2018-05-11 12:50:40 -07:00
Jason Lowe
f6d6a2ab04 YARN-8244. TestContainerSchedulerQueuing.testStartMultipleContainers failed. Contributed by Jim Brennan 
(cherry picked from commit dc912994a1bcb511dfda32a0649cef0c9bdc47d3)
 2018-05-11 14:13:55 -05:00
Eric Yang
645429ca28 YARN-8274. Fixed a bug on docker start command. 
Contributed by Jason Lowe
 2018-05-11 14:26:20 -04:00
Weiwei Yang
b4213bfd90 YARN-7003. DRAINING state of queues is not recovered after RM restart. Contributed by Tao Yang. 
(Cherry picked from commit 9db9cd95bd0348070a286e69e7965c03c9bd39d6)
 2018-05-11 10:55:05 +08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
4c0a6b67f3 YARN-8249. Fixed few REST APIs in RMWebServices to have static-user check. Contributed by Sunil Govindan. 
(cherry picked from commit d76fbbc9b82e720d7d5188f9ae2f56a8d78f3a98)
 2018-05-10 19:07:23 -07:00
Robert Kanter
0506c762b2 YARN-8202. DefaultAMSProcessor should properly check units of requested custom resource types against minimum/maximum allocation (snemeth via rkanter) 
(cherry picked from commit c8b53c43644b4ad22d5385c22cad8ed573c0b1ba)
 2018-05-10 09:52:11 -07:00