Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,235 Commits 326 Branches 371 Tags
Commit Graph

3845 Commits

Author SHA1 Message Date
Billie Rinaldi
43ea9a42d7 YARN-7890. NPE during container relaunch. Contributed by Jason Lowe 
(cherry picked from commit a196ee9362a1b35e5de20ee519f7c544ab1588e1)
 2018-02-06 12:06:25 -08:00
Jason Lowe
e73e820d2a YARN-7849. TestMiniYarnClusterNodeUtilization#testUpdateNodeUtilization fails due to heartbeat sync error. Contributed by Botong Huang 
(cherry picked from commit 60656bcecadd80e28c81bc943b44abf13d20abae)
 2018-02-05 17:29:49 -06:00
Haibo Chen
e78a701687 YARN-6868. Add test scope to certain entries in hadoop-yarn-server-resourcemanager pom.xml. (Ray Chiang via Haibo Chen) 
(cherry picked from commit a20e7105ea9d4e38d7f8f9fd48035e342bb22f1c)
(cherry picked from commit c25c082961a837b94016dcf0250bdc6d5f54e915)
 2018-02-05 10:51:39 -08:00
Jinjiang Ling
d70ca99594
YARN-7698. A misleading variable's name in ApplicationAttemptEventDispatcher 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3400d0c535aeb151c3f283cc41111b15d66990e5)
(cherry picked from commit 987a8972acf4201642881dc3dcda21f6cfcf3bc7)
 2018-01-30 00:13:26 +09:00
Jason Lowe
98499bb098 YARN-7102. NM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 
(cherry picked from commit 4d7bbee0a1dfc0b636cfbff0d0fd11f294985a31)
 2018-01-25 18:00:41 -06:00
Sunil G
b354357733 YARN-7728. Expose container preemptions related information in Capacity Scheduler queue metrics. Contributed by Eric Payne. 
(cherry picked from commit 82cc6f6968d2897ae006c5ef29eba74023f3483e)
 2018-01-25 15:55:45 +05:30
Jian He
cbaf6d242d YARN-7801. AmFilterInitializer should addFilter after fill all parameters. Contributed by Wangda Tan 2018-01-24 16:18:36 -08:00
Zhe Zhang
258fb117d2 YARN-7737. prelaunch.err file not found exception on container failure. Contributed by Keqiu Hu. 
(cherry picked from commit fa8cf4d1b4896a602dc383d5e266768392a9790c)
(cherry picked from commit 83a4f92a3f0c0134709bcb0a534e31b7c7056643)
(cherry picked from commit e9a338ff644e4ac06f09762b30e90af8027377d7)
 2018-01-24 13:10:58 -08:00
Jian He
6da08837da YARN-7782. Enable user re-mapping for Docker containers in yarn-default.xml. Contributed by Eric Yang 2018-01-23 14:07:41 -08:00
Haibo Chen
c4ba8eb003 YARN-5094. some YARN container events have timestamp of -1. 
(cherry picked from commit 4aca4ff759f773135f8a27dbaa9731196fac5233)
(cherry picked from commit 8793e45f53607315bf6c99902452ba6cf2e3eb72)
(cherry picked from commit 895a0391fe4d7dc2b19bea6cc0528b7fd607f643)
 2018-01-19 10:38:18 -08:00
Miklos Szegedi
6e11ef7c44 YARN-7590. Improve container-executor validation check. Contributed by Eric Yang. 2018-01-17 22:18:08 -08:00
Sunil G
4e28aa4dcf YARN-7727. Incorrect log levels in few logs with QueuePriorityContainerCandidateSelector. Contributed by Prabhu Joseph. 
(cherry picked from commit 128d773a2315fa6baaa3a52b13c53c77e741b69c)
 2018-01-12 15:57:04 +05:30
Akira Ajisaka
fff7f77c96
YARN-7735. Fix typo in YARN documentation. Contributed by Takanobu Asanuma. 
(cherry picked from commit fbbbf59c82658e18dad7e0e256613187b5b75d0f)
(cherry picked from commit 68b7908aa828a070534e7853df56e74ed281af7e)
 2018-01-11 16:00:50 +09:00
Haibo Chen
6602182a2b YARN-7716. metricsTimeStart and metricsTimeEnd should be all lower case in the doc. 
(cherry picked from commit f725b9e267c604fbcec09956e3a39caf04798809)
(cherry picked from commit cf816bc751e3b82be1f1931d6d0d5cbd77cb3e17)
(cherry picked from commit a58034cdfd592fa61a85c873b5249f9b6a3a825e)
 2018-01-09 09:05:25 -08:00
Jason Lowe
3701c399b3 YARN-6948. Invalid event: ATTEMPT_ADDED at FINAL_SAVING. Contributed by lujie 
(cherry picked from commit 90c86d6e090d668414681fcb5eb02add5d419b33)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttemptImpl.java
 2018-01-09 11:00:10 -06:00
Jason Lowe
52097cf2f4 YARN-7663. RMAppImpl:Invalid event: START at KILLED. Contributed by lujie 
(cherry picked from commit d2c2d8c913dfc12585394a1168d4a92934a4cfb4)
 2018-01-09 10:04:16 -06:00
bibinchundatt
7ee994a536 YARN-7508. NPE in FiCaSchedulerApp when debug log enabled in async-scheduling mode. Contributed by Tao Yang. 
(cherry picked from commit b26e30ab1655a8cdf369862a5512db5c2a88ec46)
 2018-01-09 20:04:40 +05:30
Rohith Sharma K S
8e0a5b1514 YARN-7699. queueUsagePercentage is coming as INF for getApp REST api call. Contributed by Sunil G. 
(cherry picked from commit 281350674c70d65233363849a8b60e9b7dc965cf)
 2018-01-08 15:57:56 +05:30
Sunil G
c4e0821d72 YARN-7619. Max AM Resource value in Capacity Scheduler UI has to be refreshed for every user. Contributed by Eric Payne. 
(cherry picked from commit a79abbc03e75a5d2982de12587bd3b45169cc884)
 2018-01-05 14:55:17 +05:30
Jason Lowe
ea78313d55 YARN-7678. Ability to enable logging of container memory stats. Contributed by Jim Brennan 
(cherry picked from commit 42964c2a66cfe6d7885c6231a2ec7dbde324eb67)
 2018-01-04 10:22:54 -06:00
Rohith Sharma K S
50f635ec73 YARN-7692. Skip validating priority acls while recovering applications. Contributed by Sunil G. 
(cherry picked from commit c9bf813c9a6c018d14f2bef49ba086ec0e60c761)
 2018-01-03 18:34:12 +05:30
Arun Suresh
7854745c5f YARN-7691. Add Unit Tests for ContainersLauncher. (Sampada Dehankar via asuresh) 
(cherry picked from commit c0c7cce81d5609f6347bff67929d5026d5893d75)
 2018-01-02 22:04:57 -08:00
Arun Suresh
41601363ca YARN-7542. Fix issue that causes some Running Opportunistic Containers to be recovered as PAUSED. (Sampada Dehankar via asuresh) 
(cherry picked from commit a55884c68eb175f1c9f61771386c086bf1ee65a9)
 2017-12-28 22:21:19 -08:00
Junping Du
c6f5d36ff2 Add 2.8.3 release jdiff files. 
(cherry picked from commit c89f99aade575ab1f6a9836df719cda272293d90)
 2017-12-18 22:38:26 -08:00
Jason Lowe
6c20eeb76d YARN-7661. NodeManager metrics return wrong value after update node resource. Contributed by Yang Wang 
(cherry picked from commit 811fabdebe881248756c0165bf7667bfc22be9bb)
 2017-12-18 15:29:31 -06:00
Jason Lowe
3929982ee6 YARN-7595. Container launching code suppresses close exceptions after writes. Contributed by Jim Brennan 
(cherry picked from commit 0f8de8cd91f6f74774d399378ecf8cbafcbe8f09)
 2017-12-15 16:01:40 -06:00
Weiwei Yang
108f8b8fa1 YARN-7642. Add test case to verify context update after container promotion or demotion with or without auto update. Contributed by Weiwei Yang. 
(cherry picked from commit 89b6c482c1720a7f7ac86ce022c403825a086fa0)
 2017-12-15 12:33:41 +08:00
Subru Krishnan
2b77729fdf YARN-7630. Fix AMRMToken rollover handling in AMRMProxy. Contributed by Botong Huang. 
(cherry picked from commit 17ba74be29193ac15474f73baaaf4e647a95078b)
(cherry picked from commit 8d3d7fa1c5fda4ec394b2f64baf1008e7b0d9524)
 2017-12-14 14:11:07 -08:00
Weiwei Yang
dff710632e YARN-7647. NM print inappropriate error log when node-labels is enabled. Contributed by Yang Wang. 2017-12-13 13:13:12 +08:00
Weiwei Yang
98eb4b2c6b YARN-7608. Incorrect sTarget column causing DataTable warning on RM application and scheduler web page. Contributed by Gergely Novák. 2017-12-11 10:37:22 +08:00
Wangda Tan
524ecb19ef YARN-7591. NPE in async-scheduling mode of CapacityScheduler. (Tao Yang via wangda) 
Change-Id: I67e086d6d35c3166ec0f6305490ea6527bd6d799
(cherry picked from commit 41796ea60ab9dbd6a41dd8fd3c6ff50990bba2f5)
(cherry picked from commit c27a5ba65c115f72a83c3bef9599579e38926aac)
 2017-12-08 21:57:26 -08:00
Subru Krishnan
39d0fdf1b2 YARN-6704. Add support for work preserving NM restart when FederationInterceptor is enabled in AMRMProxyService. (Botong Huang via Subru). 
(cherry picked from commit 670e8d4ec7e71fc3b054cd3b2826f869b649a788)
(cherry picked from commit 850bd0ed7cebdddf982c8a586ee4d3ad833cfc57)
 2017-12-08 15:46:13 -08:00
Weiwei Yang
1a8f6bad34 YARN-7607. Remove the trailing duplicated timestamp in container diagnostics message. Contributed by Weiwei Yang. 2017-12-07 17:30:35 +08:00
Weiwei Yang
42deb54049 YARN-7611. Node manager web UI should display container type in containers page. Contributed by Weiwei Yang. 
Previous commit 218d622529a45ea215cbf262221a06bc5b693f85 introduces a compile error,
  so it was reverted by commit db74ada80f4237726e15bd6def873081428a7210.
  This commit fixed the issue.

(cherry picked from commit 05c347fe51c01494ed8110f8f116a01c90205f13)
 2017-12-06 20:22:52 +08:00
Weiwei Yang
db74ada80f Revert "YARN-7611. Node manager web UI should display container type in containers page. Contributed by Weiwei Yang." 
This reverts commit 218d622529a45ea215cbf262221a06bc5b693f85.
 2017-12-06 19:58:51 +08:00
Weiwei Yang
218d622529 YARN-7611. Node manager web UI should display container type in containers page. Contributed by Weiwei Yang. 
(cherry picked from commit 05c347fe51c01494ed8110f8f116a01c90205f13)
 2017-12-06 12:57:22 +08:00
Arun Suresh
cc0c966a4e YARN-7587. Skip dispatching opportunistic containers to nodes whose queue is already full. (Weiwei Yang via asuresh) 
(cherry picked from commit 37ca4169508c3003dbe9044fefd37eb8cd8c0503)
 2017-12-03 22:25:06 -08:00
Jason Lowe
487c920bb0 YARN-7455. quote_and_append_arg can overflow buffer. Contributed by Jim Brennan 
(cherry picked from commit 60f95fb719f00a718b484c36a823ec5aa8b099f4)
 2017-12-01 15:54:03 -06:00
Jason Lowe
5a8a1e66c6 YARN-7495. Improve robustness of the AggregatedLogDeletionService. Contributed by Jonathan Eagles 
(cherry picked from commit 5cfaee2e6db8b2ac55708de0968ff5539ee3bd76)
 2017-11-30 12:47:43 -06:00
Wangda Tan
a6d7fa91d9 YARN-7509. AsyncScheduleThread and ResourceCommitterService are still running after RM is transitioned to standby. (Tao Yang via wangda) 
Change-Id: I7477fe355419fd4a0a6e2bdda7319abad4c4c748
(cherry picked from commit 834e91ee91d22d74866afbf6252107e969bf8370)
(cherry picked from commit 94f7b0f99fb24efbacaa36ee8f9c68646ddef440)
 2017-11-29 15:48:51 -08:00
Wei Yan
17289ca677 YARN-6851. Capacity Scheduler: document configs for controlling # containers allowed to be allocated per node heartbeat. 
(cherry picked from commit 333ef303ff0caf9adfd378652a8f966377901768)
 2017-11-29 14:48:19 -08:00
Haibo Chen
c08a0bcad8 YARN-7390. All reservation related test cases failed when TestYarnClient runs against Fair Scheduler. 
(cherry picked from commit db2651cc9dd90320907e5c510272520c582581a4)
 2017-11-21 13:29:00 -08:00
Subru Krishnan
a6db6c9855 YARN-6128. Add support for AMRMProxy HA. (Botong Huang via Subru). 
(cherry picked from commit ed3109136100a21d971484f242d80f2a7e7d337d)
 2017-11-20 14:22:59 -08:00
bibinchundatt
49ba091f71 YARN-7525. Incorrect query parameters in cluster nodes REST API document. Contributed by Tao Yang. 
(cherry picked from commit ea8a121423a3d6b098129af5bbbec88f98550653)
 2017-11-21 00:32:49 +05:30
bibinchundatt
dc4132395e YARN-7489. ConcurrentModificationException in RMAppImpl#getRMAppMetrics. Contributed by Tao Yang. 
(cherry picked from commit 5c2986e58dfc86fada8554a3f48cd417a4458abc)
 2017-11-20 23:29:14 +05:30
Sunil G
9eab9a25c6 YARN-7469. Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit. Contributed by Eric Payne. 
(cherry picked from commit 61ace174cdcbca9d22abce7aa0aa71148f37ad55)
 2017-11-17 19:42:00 +05:30
Varun Vasudev
f23f49d4ee YARN-7430. Enable user re-mapping for Docker containers by default. Contributed by Eric Yang. 
(cherry picked from commit 5f0b238a118f3992bd149d8c02e6a1376dee96d7)
(cherry picked from commit 9cf9627e20258147ee0a13c938c6165406c160dd)
 2017-11-17 12:05:55 +05:30
Jason Lowe
e4344d45cb YARN-7361. Improve the docker container runtime documentation. Contributed by Shane Kumpf 
(cherry picked from commit fac72eef23bb0a74a34f289dd6ef50ffa4303aa4)
 2017-11-15 12:34:22 -06:00
Junping Du
9229fb3523 YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi. 
(cherry picked from commit a72dcb9cad7df7f7236092f5b86446c2ef4ea874)
 2017-11-14 12:50:21 -08:00
Arun Suresh
238a70c6c3 YARN-5049. Addendum patch for branch-2. (asuresh) 
(cherry picked from commit c153bed6e3b38d60815c3f62d96c604f23116eb3)
 2017-11-12 10:17:01 -08:00