Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,429 Commits 326 Branches 371 Tags
Commit Graph

4535 Commits

Author SHA1 Message Date
Inigo Goiri
63c20f9021 YARN-8359. Exclude containermanager.linux test classes on Windows. Contributed by Jason Lowe. 
(cherry picked from commit 3492a1db2c0654ce5375360caa74a34f928f23be)
 2018-06-07 17:10:09 -07:00
Inigo Goiri
1fa51d4542 YARN-8400. Fix typos in YARN Federation documentation page. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 67fc70e09f941e9b43b022d9f42a9486ad759e6e)
 2018-06-07 16:10:51 -07:00
Rohith Sharma K S
e9367ccb41 YARN-8386. App log can not be viewed from Logs tab in secure cluster. Contributed by Sunil Govindan. 
(cherry picked from commit 377ea1bcdfd523f9353647388881b82699f2df12)
 2018-06-07 21:43:17 +05:30
Rohith Sharma K S
a95f216945 YARN-8399. NodeManager is giving 403 GSS exception post upgrade to 3.1 in secure mode. Contributed by Sunil Govindan. 
(cherry picked from commit 58bc34f1e347034af566d6968eb3b3439a91cc74)
 2018-06-07 10:27:08 +05:30
Rohith Sharma K S
e4eac4a9f3 YARN-8396. Click on an individual container continuously spins and doesn't load the page. Contributed by Sunil Govindan. 
(cherry picked from commit 745f3a244166c1e6a1aee81eb1d6c2c692053d04)
 2018-06-05 19:29:04 +05:30
Eric Yang
9dd79d1dcd YARN-8365. Set DNS query type according to client request. 
Contributed by Shane Kumpf

(cherry picked from commit 5889a3c0f931ebe1a9e6b8023c0557922b7d9e49)
 2018-06-04 19:03:07 -04:00
Robert Kanter
93d6ed859e YARN-4677. RMNodeResourceUpdateEvent update from scheduler can lead to race condition (wilfreds and gphillips via rkanter) 
(cherry picked from commit 0cd145a44390bc1a01113dce4be4e629637c3e8a)
 2018-06-04 15:32:47 -07:00
Miklos Szegedi
21948b048e YARN-8382. cgroup file leak in NM. Contributed by Hu Ziqian. 
(cherry picked from commit 925fdf761a513130e23c10575c7328c8681cff1d)
 2018-06-04 10:56:13 -07:00
Inigo Goiri
83d0964d82 YARN-8389. Improve the description of machine-list property in Federation docs. Contributed by Takanobu Asanuma. 
(cherry picked from commit 61fc7f73f21b0949e27ef3893efda757d91a03f9)
 2018-06-04 09:28:40 -07:00
Sunil G
ea2fa52d92 YARN-8276. [UI2] After version field became mandatory, form-based submission of new YARN service doesn't work. Contributed by Gergely Novák. 
(cherry picked from commit 9c4cbed8d19ec0f486af454de6b117d77a0a0b84)
 2018-06-04 00:09:44 +05:30
Billie Rinaldi
e59e9feefa YARN-8342. Enable untrusted docker image to run with launch command. Contributed by Eric Yang 
(cherry picked from commit 31998643a51f1e08f723f18dc5476ac1512d5b81)
 2018-06-02 14:49:21 -07:00
Rohith Sharma K S
f7d0ca71ac More YARN pages need to honor yarn.resourcemanager.display.per-user-apps(addendum). Contributed by Sunil G. 
(cherry picked from commit 8261f9e5710038ccbc475dbfcea3b9ae79b6f482)
 2018-06-02 09:43:08 +05:30
Wangda Tan
0be93a7973 YARN-8349. Remove YARN registry entries when a service is killed by the RM. (Billie Rinaldi via wangda) 
Change-Id: Ia58db3637789a8921482f564aa9bdf99c45cc36c
(cherry picked from commit ff583d3fa3325029bc691ec22d817aee37e5e85d)
 2018-06-01 15:53:26 -07:00
Wangda Tan
ed20576413 YARN-8372. Distributed shell app master should not release containers when shutdown if keep-container is true. (Suma Shivaprasad via wangda) 
Change-Id: Ief04d1ca865621f348fba4ac85fa78bc47465904
(cherry picked from commit 8956e5b8db3059e0872e49f59adc6affc76e2274)
 2018-06-01 15:47:59 -07:00
Wangda Tan
1cee85d8c4 YARN-7962. Race Condition When Stopping DelegationTokenRenewer causes RM crash during failover. (BELUGA BEHR via wangda) 
Change-Id: I617e2645f60a57080058ad5f06af860fb3f682c8
(cherry picked from commit 931f78718f3a09775bfa1f9a952c069c416d0914)
 2018-06-01 15:47:51 -07:00
Wangda Tan
cc460fea2d YARN-8384. stdout.txt, stderr.txt logs of a launched docker container is coming with primary group of submit user instead of hadoop. (Eric Yang via wangda) 
Change-Id: Idbb2cd250f4f62dc32993e0d3ca3ec5684616baa
(cherry picked from commit 3a6bd775500343632bad5f9c1a2bfacd408f7760)
 2018-06-01 15:47:45 -07:00
Eric Yang
4193892de6 YARN-8308. Fixed YARN Service AM failure with HDFS token renewal. 
Contributed by Gour Saha

(cherry picked from commit 7c7ac07f013f46db89c73681806300699ee534d6)
 2018-05-31 21:10:04 -04:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
a468400bad YARN-8197. Fixed AM IP Filter and Webapp proxy to redirect app tracking-URLs correctly when UI is secure. Contributed by Sunil Govindan. 
(cherry picked from commit 6b74f5d7fc509c55c331249256eec78b7e53b6ce)
 2018-05-31 17:04:45 -07:00
Billie Rinaldi
1bb112cd96 YARN-8333. Load balance YARN services using RegistryDNS multiple A records. Contributed by Eric Yang 
(cherry picked from commit 6bc92e304fe05e80f13830104d1fd2c59da8344b)
 2018-05-31 06:49:04 -07:00
Weiwei Yang
ec4240a7fa YARN-8367. Fix NPE in SingleConstraintAppPlacementAllocator when placement constraint in SchedulingRequest is null. Contributed by Weiwei Yang. 
(Cherry picked from commit 6468071f137e6d918a7b4799ad54558fa26b25ce)
 2018-05-31 21:04:41 +08:00
Billie Rinaldi
58e7d6c3c0 YARN-8350. NPE in service AM related to placement policy. Contributed by Gour Saha 
(cherry picked from commit 778a4a24be176382a5704f709c00bdfcfe6ddc8c)
 2018-05-30 13:21:17 -07:00
Billie Rinaldi
7bd5d79177 YARN-8368. yarn app start cli should print applicationId. Contributed by Rohith Sharma K S 
(cherry picked from commit 96eefcc84aacc4cc82ad7e3e72c5bdad56f4a7b7)
 2018-05-30 12:38:53 -07:00
Eric E Payne
3d06d75423 YARN-8377: Javadoc build failed in hadoop-yarn-server-nodemanager. Contributed by Takanobu Asanuma 
(cherry picked from commit e44c0849d7982c8f1ed43af25d2092090881d19f)
 2018-05-30 17:28:14 +00:00
Eric Yang
03209e8966 YARN-8362. Bugfix logic in container retries in node manager. 
Contributed by Chandni Singh

(cherry picked from commit 135941e00d762a417c3b4cc524cdc59b0d1810b1)
 2018-05-29 17:04:01 -04:00
Jason Lowe
a1fd04c4f4 YARN-8329. Docker client configuration can still be set incorrectly. Contributed by Shane Kumpf 
(cherry picked from commit 4827e9a9085b306bc379cb6e0b1fe4b92326edcd)
 2018-05-29 14:48:01 -05:00
Wangda Tan
3eb1cb18c7 YARN-8369. Javadoc build failed due to 'bad use of >'. (Takanobu Asanuma via wangda) 
Change-Id: I79a42154e8f86ab1c3cc939b3745024b8eebe5f4
(cherry picked from commit 17aa40f669f197d43387d67dc00040d14cd00948)
 2018-05-29 09:28:34 -07:00
Wangda Tan
b262ea1381 YARN-8339. Service AM should localize static/archive resource types to container working directory instead of 'resources'. (Suma Shivaprasad via wangda) 
Change-Id: I9f8e8f621650347f6c2f9e3420edee9eb2f356a4
(cherry picked from commit 3061bfcde53210d2032df3814243498b27a997b7)
 2018-05-29 09:28:27 -07:00
Jason Lowe
500b0ee2ce YARN-8338. TimelineService V1.5 doesn't come up after HADOOP-15406. Contributed by Vinod Kumar Vavilapalli 
(cherry picked from commit 31ab960f4f931df273481927b897388895d803ba)
 2018-05-29 11:03:19 -05:00
Sunil G
7f9caf8986 YARN-4781. Support intra-queue preemption for fairness ordering policy. Contributed by Eric Payne. 
(cherry picked from commit 7c343669baf660df3b70d58987d6e68aec54d6fa)
 2018-05-28 16:33:48 +05:30
Rohith Sharma K S
f19e8e2246 MAPREDUCE-7097. MapReduce JHS should honor yarn.webapp.filter-entity-list-by-user. Contributed by Sunil Govindan. 
(cherry picked from commit 88cbe57c069a1d2dd3bfb32e3ad742566470a10b)
 2018-05-28 14:06:50 +05:30
Wangda Tan
4d41cb1696 YARN-8213. Add Capacity Scheduler performance metrics. (Weiwei Yang via wangda) 
Change-Id: Ieea6f3eeb83c90cd74233fea896f0fcd0f325d5f
(cherry picked from commit f24c842d52e166e8566337ef93c96438f1c870d8)
 2018-05-25 22:04:54 -07:00
Eric E Payne
053937f436 YARN-8292: Fix the dominant resource preemption cannot happen when some of the resource vector becomes negative. Contributed by Wangda Tan. 
(cherry picked from commit 8d5509c68156faaa6641f4e747fc9ff80adccf88)
 2018-05-25 16:27:09 +00:00
Miklos Szegedi
1483c90379 YARN-8310. Handle old NMTokenIdentifier, AMRMTokenIdentifier, and ContainerTokenIdentifier formats. Contributed by Robert Kanter. 
(cherry picked from commit 3e5f7ea986600e084fcac723b0423e7de1b3bb8a)
 2018-05-24 15:14:15 -07:00
Eric Yang
f49e697cc8 YARN-8357. Fixed NPE when YARN service is saved and not deployed. 
Contributed by Chandni Singh

(cherry picked from commit d9852eb5897a25323ab0302c2c0decb61d310e5e)
 2018-05-24 16:34:32 -04:00
Eric Yang
514c05abd0 YARN-8316. Improved diagnostic message for ATS unavailability for YARN Service. 
Contributed by Billie Rinaldi
 2018-05-24 16:26:53 -04:00
Shane Kumpf
de43b41d0f YARN-6919. Add default volume mount list. Contributed by Eric Badger 2018-05-24 09:35:35 -06:00
Rohith Sharma K S
923c833da4 YARN-8319. More YARN pages need to honor yarn.resourcemanager.display.per-user-apps. Contributed by Sunil G. 
(cherry picked from commit c05b5d424b000bab766f57e88a07f2b4e9a56647)
 2018-05-24 14:22:43 +05:30
Rohith Sharma K S
67842c56fe YARN-8346. Upgrading to 3.1 kills running containers with error 'Opportunistic container queue is full'. Contributed by Jason Lowe. 
(cherry picked from commit 4cc0c9b0baa93f5a1c0623eee353874e858a7caa)
 2018-05-24 12:25:33 +05:30
Eric Yang
05d905f586 YARN-7530. Refactored YARN service API project location. 
Contributed by Chandni Singh
 2018-05-23 22:43:54 -04:00
Inigo Goiri
fa8af4aab5 YARN-8327. Fix TestAggregatedLogFormat#testReadAcontainerLogs1 on Windows. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit f09dc73001fd5f3319765fa997f4b0ca9e8f2aff)
 2018-05-23 15:59:59 -07:00
Inigo Goiri
61b5b2f4f7 YARN-8344. Missing nm.stop() in TestNodeManagerResync to fix testKillContainersOnResync. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit e99e5bf104e9664bc1b43a2639d87355d47a77e2)
 2018-05-23 14:16:03 -07:00
Rohith Sharma K S
8dcc7b853b YARN-8297. Incorrect ATS Url used for Wire encrypted cluster.(addendum). Contributed by Sunil G. 
(cherry picked from commit f61e3e752eb1cf4a08030da04bc3d6c5a2b3926d)
 2018-05-23 18:32:57 +05:30
Wangda Tan
2bbf1b4435 YARN-8332. Incorrect min/max allocation property name in resource types doc. (Weiwei Yang via wangda) 
Change-Id: If74f1ceed9c045a2cb2d6593741278b65ac44a9f
(cherry picked from commit 83f53e5c6236de30c213dc41878cebfb02597e26)
 2018-05-22 13:39:20 -07:00
Wangda Tan
6aaf33e50c YARN-8290. SystemMetricsPublisher.appACLsUpdated should be invoked after application information is published to ATS to avoid "User is not set in the application report" Exception. (Eric Yang via wangda) 
Change-Id: I0ac6ddd19740d1aa7dd07111cd11af71ddc2fcaf
(cherry picked from commit bd15d2396ef0c24fb6b60c6393d16b37651b828e)
 2018-05-22 13:39:15 -07:00
Eric Yang
7c3f236783 YARN-7960. Added security flag no-new-privileges for YARN Docker integration. 
Contributed by Eric Badger

(cherry picked from commit 6176d2b35c85715aae93526236c29540f71ecac8)
 2018-05-22 13:47:08 -04:00
Jason Lowe
9788c8c01e YARN-8206. Sending a kill does not immediately kill docker containers. Contributed by Eric Badger 
(cherry picked from commit 5f11288e41fca2e414dcbea130c7702e29d4d610)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/TestDockerContainerRuntime.java
 2018-05-22 09:49:31 -05:00
Eric E Payne
5b2bbff141 YARN-8179: Preemption does not happen due to natural_termination_factor when DRF is used. Contributed by Kyungwan Nam. 
(cherry picked from commit 0b4c44bdeef62945b592d5761666ad026b629c0b)
 2018-05-21 20:19:44 +00:00
Eric Yang
61e8974049 YARN-7530. Refactored YARN service API project location. 
Contributed by Chandni Singh
 2018-05-18 17:33:18 -04:00
Eric Yang
0791785574 YARN-8296. Removed unique_component_support from YARN services. 
Contributed by Suma Shivaprasad

(cherry picked from commit 2f2dd22aad4a003228b9efe3b4c506a6922d09d8)
 2018-05-17 20:59:50 -04:00
Eric Yang
79351d4fda YARN-8141. Removed YARN_CONTAINER_RUNTIME_DOCKER_LOCAL_RESOURCE_MOUNTS flag. 
Contributed by Chandni Singh

(cherry picked from 989cfdc1e0f1c27b35679c668a9bde6a0be845b5)
 2018-05-17 20:57:14 -04:00