Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,725 Commits 326 Branches 371 Tags
Commit Graph

4061 Commits

Author SHA1 Message Date
bibinchundatt
5514f02a73 YARN-8535. Fix DistributedShell unit tests. Contributed by Abhishek Modi. 
(cherry picked from commit eed8415dc18fa7415ebd105350bd0532b3b1b6bb)
 2018-09-02 13:39:26 +05:30
Weiwei Yang
fdecbd94c1 YARN-8719. Typo correction for yarn configuration in OpportunisticContainers(federation) docs. Contributed by Y. SREENIVASULU REDDY. 
(cherry picked from commit e8b063f63049d781f4bd67e2ac928c03fd7b7941)
 2018-08-28 01:10:47 +08:00
Haibo Chen
49590e7c6b YARN-8051. TestRMEmbeddedElector#testCallbackSynchronization is flaky. (Robert Kanter via Haibo Chen) 
(cherry picked from commit 93d47a0ed504ee81d4b74d340c1815bdbb3c9b14)
 2018-08-24 13:27:54 -05:00
Inigo Goiri
c3ba06deeb YARN-7773. YARN Federation used Mysql as state store throw exception, Unknown column 'homeSubCluster' in 'field list'. Contributed by Yiran Wu. 
(cherry picked from commit 60e5c1b516647bf99ef3e9501c0ce04deea13ce9)
 2018-08-24 14:26:52 +05:30
Jason Lowe
79ebbec4c8 YARN-8649. NPE in localizer hearbeat processing if a container is killed while localizing. Contributed by lujie 
(cherry picked from commit 585ebd873a55bedd2a364d256837f08ada8ba032)
 2018-08-23 09:40:42 -05:00
Rohith Sharma K S
7278566f27 YARN-8129. Improve error message for invalid value in fields attribute. Contributed by Abhishek Modi. 
(cherry picked from commit d3fef7a5c5b83d27e87b5e49928254a7d1b935e5)
 2018-08-21 12:11:32 +05:30
Rohith Sharma K S
675aa2bbc0 YARN-8679. [ATSv2] If HBase cluster is down for long time, high chances that NM ContainerManager dispatcher get blocked. Contributed by Wangda Tan. 
(cherry picked from commit 4aacbfff605262aaf3dbd926258afcadc86c72c0)
 2018-08-18 11:06:14 +05:30
Haibo Chen
8118b14db8 YARN-7835. Race condition in NM while publishing events if second attempt is launched on the same node. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit d1274c3b71549cb000868500c293cafd880b3713)
 2018-08-18 11:06:02 +05:30
Rohith Sharma K S
90bf2d3b52 YARN-8612. Fix NM Collector Service Port issue in YarnConfiguration. Contributed by Prabha Manepalli. 
(cherry picked from commit 1697a0230696e1ed6d9c19471463b44a6d791dfa)
 2018-08-17 11:14:10 +05:30
Weiwei Yang
8c0cecb034 YARN-8614. Fix few annotation typos in YarnConfiguration. Contributed by Sen Zhao. 
(cherry picked from commit 8dc07b47235358653c5cccda354201564f7545ed)
 2018-08-15 13:37:21 +08:00
Robert Kanter
fd63be750a YARN-8568. Replace the deprecated zk-address property in the HA config example in ResourceManagerHA.md (bsteinbach via rkanter) 
(cherry picked from commit 8478732bb28e9e71061d6b4a043a3a1b5c688902)
 2018-08-14 10:16:44 -07:00
Jason Lowe
c72674ee3c YARN-8640. Restore previous state in container-executor after failure. Contributed by Jim Brennan 
(cherry picked from commit d1d129aa9deecebf42261947fcb0b2ca46dacad5)
 2018-08-14 10:29:58 -05:00
Jonathan Hung
0420ca5a6f YARN-8559. Expose mutable-conf scheduler's configuration in RM /scheduler-conf endpoint. Contributed by Weiwei Yang. 2018-08-10 15:22:11 -07:00
Jason Lowe
b0a364171d YARN-8331. Race condition in NM container launched after done. Contributed by Pradeep Ambati 
(cherry picked from commit cd04e954d2db27f0a15b7d1c492b7cdb656a51db)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/TestContainer.java
 2018-08-09 10:31:11 -05:00
Wangda Tan
dc41462af4 Make 3.1.1 awared by other branches - adding missing files 
(cherry picked from commit 3214cd75acd0474373951870e1ba2ec11833a3da)
(cherry picked from commit f16c311486b8e9236b78e31d1bfe4abb108d9d49)
 2018-08-08 13:07:15 -07:00
Wangda Tan
8b4392769c Make 3.1.1 awared by other branches 
(cherry picked from commit 49c687608b65b772faeed614700ece8e526432e8)
(cherry picked from commit 781620a44db1eea511406f6c300f5b12b3a24daf)
 2018-08-08 13:07:09 -07:00
Sunil G
a3675f382a YARN-8397. Potential thread leak in ActivitiesManager. Contributed by Rohith Sharma K S. 
(cherry picked from commit 6310c0d17d6422a595f856a55b4f1fb82be43739)
 2018-08-01 08:36:12 +05:30
Jonathan Hung
0d6e1a2aab YARN-7974. Allow updating application tracking url after registration. Contributed by Jonathan Hung 2018-07-30 18:06:25 -07:00
Haibo Chen
a4fc0279fc YARN-6966. NodeManager metrics may return wrong negative values when NM restart. (Szilard Nemeth via Haibo Chen) 2018-07-30 09:26:00 -07:00
bibinchundatt
9a3b006685 YARN-8558. NM recovery level db not cleaned up properly on container finish. Contributed by Bibin A Chundatt. 2018-07-30 21:31:47 +05:30
Eric E Payne
299dffc72d YARN-4606. CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps. Contributed by Manikandan R 
(cherry picked from commit 9485c9aee6e9bb935c3e6ae4da81d70b621781de)
 2018-07-25 16:49:03 +00:00
Sunil G
1d8fce0d2f YARN-7748. TestContainerResizing.testIncreaseContainerUnreservedWhenApplicationCompleted fails due to multiple container fail events. Contributed by Weiwei Yang. 
(cherry picked from commit 35ce6eb1f526ce3db7e015fb1761eee15604100c)
 2018-07-24 22:21:44 +05:30
bibinchundatt
1f713d6c66 YARN-8548. AllocationRespose proto setNMToken initBuilder not done. Contributed by Bilwa S T. 
(cherry picked from commit ff7c2eda34c2c40ad71b50df6462a661bd213fbd)
 2018-07-24 16:32:21 +05:30
bibinchundatt
1a270e3b07 YARN-8434. Update federation documentation of Nodemanager configurations. Contributed by Bibin A Chundatt. 2018-07-24 13:04:26 +05:30
Robert Kanter
6e0db6fe1a YARN-8518. test-container-executor test_is_empty() is broken (Jim_Brennan via rkanter) 
(cherry picked from commit 1bc106a738a6ce4f7ed025d556bb44c1ede022e3)
 2018-07-18 16:10:57 -07:00
Robert Kanter
c1dc4ca2c6 Only mount non-empty directories for cgroups (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit 0838fe833738e04f5e6f6408e97866d77bebbf30)
 2018-07-18 16:10:57 -07:00
Robert Kanter
d61d84279f Disable mounting cgroups by default (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit 351cf87c92872d90f62c476f85ae4d02e485769c)
 2018-07-18 16:10:57 -07:00
Eric E Payne
5738bd8a10 YARN-8421: when moving app, activeUsers is increased, even though app does not have outstanding request. Contributed by Kyungwan Nam 
(cherry picked from commit 937ef39b3ff90f72392b7a319e4346344db34e03)
 2018-07-16 16:53:07 +00:00
Jason Lowe
1ae35834a2 YARN-8515. container-executor can crash with SIGPIPE after nodemanager restart. Contributed by Jim Brennan 
(cherry picked from commit 17118f446c2387aa796849da8b69a845d9d307d3)
 2018-07-13 10:08:21 -05:00
Sunil G
ce7c6762f4 YARN-8473. Containers being launched as app tears down can leave containers in NEW state. Contributed by Jason Lowe. 
(cherry picked from commit 705e2c1f7cba51496b0d019ecedffbe5fb55c28b)
 2018-07-10 20:13:15 +05:30
Junping Du
0a6942d58c yarn.resourcemanager.fail-fast is used inconsistently. Contributed by Yuanbo Liu. 
(cherry picked from commit d9ba6f3656e8dc97d2813181e27d12e52dca4328)
(cherry picked from commit 3d6ba2dd4e4f1c42522c837ca96ee5d13f1491a4)
 2018-07-09 16:22:58 +08:00
Xiaoyu Yao
3042aa2685 HADOOP-15571. Multiple FileContexts created with the same configuration object should be allowed to have different umask. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit 498e3bfb6b93bf542e5581d83e64e920983fe87e)
 2018-07-06 21:20:57 -07:00
Jason Lowe
90631e6dbf YARN-8451. Multiple NM heartbeat thread created when a slow NM resync with RM. Contributed by Botong Huang 
(cherry picked from commit 100470140d86eede0fa240a9aa93226f274ee4f5)
 2018-06-29 13:14:20 -05:00
Sunil G
7dbe81895b YARN-8401. [UI2] new ui is not accessible with out internet connection. Contributed by Bibin A Chundatt. 
(cherry picked from commit fbaff369e9b9022723a7b2c6f25e71122a8f8a15)
 2018-06-27 10:36:08 -07:00
Rohith Sharma K S
1fa79f0063 YARN-8457. Compilation is broken with -Pyarn-ui. 
(cherry picked from commit 4ffe68a6f70ce01a5654da8991b4cdb35ae0bf1f)
 2018-06-25 10:40:02 -07:00
Weiwei Yang
3151e95d27 YARN-8443. Total #VCores in cluster metrics is wrong when CapacityScheduler reserved some containers. Contributed by Tao Yang. 2018-06-25 09:46:32 +08:00
Rohith Sharma K S
6542e31b78 YARN-8155. Improve ATSv2 client logging in RM and NM publisher. Contributed by Abhishek Modi. 
(cherry picked from commit 9119b3cf8f883aa2d5df534afc0c50249fed03c6)
 2018-06-14 13:51:05 +05:30
Sunil G
d9b00cdd6b YARN-8404. Timeline event publish need to be async to avoid Dispatcher thread leak in case ATS is down. Contributed by Rohith Sharma K S 
(cherry picked from commit 6307962b932e0ee69ba61f5796388c175d79195a)
 2018-06-13 16:10:24 +05:30
Weiwei Yang
ef105abb70 YARN-8394. Improve data locality documentation for Capacity Scheduler. Contributed by Weiwei Yang. 
(Cherry picked from commit 29024a62038c297f11e8992601f2522ffffc7da7)
 2018-06-13 13:57:35 +08:00
Inigo Goiri
6a31dc9927 HADOOP-15529. ContainerLaunch#testInvalidEnvVariableSubstitutionType is not supported in Windows. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 6e756e8a620e4d6dc3192986679060c52063489b)
 2018-06-12 10:25:36 -07:00
Rohith Sharma K S
96e7b7e8ae YARN-8405. RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773. Contributed by Íñigo Goiri. 
(cherry picked from commit 2df73dace06cfd2b3193a14cd455297f8f989617)
 2018-06-12 17:25:22 +05:30
Inigo Goiri
26ed145763 YARN-8370. Some Node Manager tests fail on Windows due to improper path/file separator. Contributed by Anbang Hu. 
(cherry picked from commit 2b2f672022547e8c19658213ac5a4090bf5b6c72)
 2018-06-11 19:26:58 -07:00
Inigo Goiri
500057c40c YARN-8359. Exclude containermanager.linux test classes on Windows. Contributed by Jason Lowe. 
(cherry picked from commit 3492a1db2c0654ce5375360caa74a34f928f23be)
 2018-06-07 17:10:38 -07:00
Robert Kanter
d5e6d0d5f4 YARN-4677. RMNodeResourceUpdateEvent update from scheduler can lead to race condition (wilfreds and gphillips via rkanter) 
(cherry picked from commit 0cd145a44390bc1a01113dce4be4e629637c3e8a)
 2018-06-04 15:42:46 -07:00
Miklos Szegedi
a3afd69051 YARN-8382. cgroup file leak in NM. Contributed by Hu Ziqian. 
(cherry picked from commit 925fdf761a513130e23c10575c7328c8681cff1d)
 2018-06-04 11:07:08 -07:00
Yongjun Zhang
f0de11ba98 Preparing for 3.0.4 development 2018-05-29 23:40:26 -07:00
Jason Lowe
d5708bbcdc YARN-8338. TimelineService V1.5 doesn't come up after HADOOP-15406. Contributed by Vinod Kumar Vavilapalli 
(cherry picked from commit 31ab960f4f931df273481927b897388895d803ba)
 2018-05-29 11:15:07 -05:00
Sunil G
521ada1a11 YARN-4781. Support intra-queue preemption for fairness ordering policy. Contributed by Eric Payne. 
(cherry picked from commit 7c343669baf660df3b70d58987d6e68aec54d6fa)
 2018-05-28 16:34:29 +05:30
Miklos Szegedi
b0f7b0dfd5 YARN-8310. Handle old NMTokenIdentifier, AMRMTokenIdentifier, and ContainerTokenIdentifier formats. Contributed by Robert Kanter. 2018-05-24 15:36:35 -07:00
Rohith Sharma K S
4711980d77 YARN-8346. Upgrading to 3.1 kills running containers with error 'Opportunistic container queue is full'. Contributed by Jason Lowe. 
(cherry picked from commit 4cc0c9b0baa93f5a1c0623eee353874e858a7caa)
 2018-05-24 12:26:25 +05:30