Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,951 Commits 326 Branches 371 Tags
Commit Graph

4118 Commits

Author SHA1 Message Date
Erik Krogen
849e1f018e HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2019-07-25 10:07:04 -07:00
Eric E Payne
6aa76eaee3 YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonanthan Hung) 
Addendum to fix 3.0 build
 2019-07-23 21:53:27 +00:00
Jonathan Hung
3b346f50de YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonanthan Hung) 2019-07-22 10:54:38 -07:00
Weiwei Yang
78a6e40994 YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. Contributed by hunshenshi. 2019-07-02 23:05:18 +08:00
Zhankun Tang
6da0151081 YARN-9584. Should put initializeProcessTrees method call before get pid. Contributed by Wanqiang Ji. 
(cherry picked from commit 67414a1a80039e70e0afc1de171831a6e981f37a)
 2019-06-18 13:19:28 +08:00
Eric E Payne
7da78f06a3 YARN-8625. Aggregate Resource Allocation for each job is not present in ATS. Contributed by Prabhu Joseph. 
(cherry picked from commit 3c63551101dc17038efb9f5345833d2883fce86e)
 2019-05-29 19:23:23 +00:00
Ahmed Hussein
9d63fc7ed6 YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2019-05-29 14:19:20 -05:00
Haibo Chen
aa7ff8552e YARN-9529. Log correct cpu controller path on error while initializing CGroups. (Contributed by Jonathan Hung) 
(cherry picked from commit 597fa47ad125c0871f5c4deb3a883e5b3341c67b)
(cherry picked from commit c6573562cbc43832e9332989996e5d07b8bce9b0)
(cherry picked from commit ea1f0f282bee7ca64ce9fbe1a59bb5e231420d29)
 2019-05-06 11:59:49 -07:00
Eric E Payne
01fa72a825 YARN-9285: RM UI progress column is of wrong type. Contributed by Ahmed Hussein. 2019-05-03 15:56:20 +00:00
Weiwei Yang
5d85a34b1f YARN-9463. Add queueName info when failing with queue capacity sanity check. Contributed by Aihua Xu. 
(cherry picked from commit 8c1bba375b144fd515b389174ddb349f2d9246fa)
 2019-04-10 23:05:12 +08:00
Weiwei Yang
0e167fde55 YARN-9413. Queue resource leak after app fail for CapacityScheduler. Contributed by Tao Yang. 2019-04-08 13:44:33 +08:00
Jason Lowe
32f96b6be4 YARN-5714. ContainerExecutor does not order environment map. Contributed by Remi Catherinot and Jim Brennan 
(cherry picked from commit 8e728f39c961f034369b43e087d68d01aa4a0e7d)
 2019-03-07 15:10:52 +00:00
Sunil G
c139fe7a37 YARN-9168. DistributedShell client timeout should be -1 by default. Contributed by Zhankun Tang. 
(cherry picked from commit 6cec90653dc2ae60a10262745997ea91dd872a8b)
 2019-02-25 15:30:17 +05:30
Adam Antal
bebbdd290e
YARN-9283. Javadoc of LinuxContainerExecutor#addSchedPriorityCommand has a wrong property name as reference 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9385ec45d75109a2e6565faa10527cc56637bf5f)
 2019-02-15 18:59:02 +09:00
Masatake Iwasaki
ba035dda76 YARN-9282. Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'. Contributed by Charan Hebri. 
(cherry picked from commit e0ab1bdecec3d7ba7ddc0849781d7f71714f8687)
 2019-02-09 00:34:22 +09:00
Eric E Payne
f3ff6d3aa7 YARN-7171: RM UI should sort memory / cores numerically. Contributed by Ahmed Hussein 
(cherry picked from commit d1ca9432dd0b3e9b46b4903e8c9d33f5c28fcc1b)
 2019-02-07 21:03:43 +00:00
Sunil G
ec0bed1008 YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla. 
(cherry picked from commit 6ffe6ea8999c1216b6e510e350535754431bac2d)
 2019-02-07 19:09:19 +05:30
Eric E Payne
50ef25464f YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph 
(cherry picked from commit 04105bbfdb041a41062c856632641140de84fba8)
 2019-01-29 18:12:36 +00:00
Weiwei Yang
d182c81938 YARN-9237. NM should ignore sending finished apps to RM during RM fail-over. Contributed by Jiandan Yang. 
(cherry picked from commit 4f63ffe444286dac91ed36fd647d8ce69e75b0f0)
 2019-01-29 11:07:51 +08:00
Jonathan Hung
bce8f28bec YARN-9222. Print launchTime in ApplicationSummary 
(cherry picked from commit 6cace58e212d3ee0aec988926a5a17c9cc58e645)
(cherry picked from commit bf760e7e81f8e02ad413c470fccf78aaa9cb9f86)
(cherry picked from commit 80a08c29adc9d07974bbe8087abcec750d5f7916)
 2019-01-25 13:51:21 -08:00
Haibo Chen
3368f141b2 YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 
(cherry picked from commit bb92bfb4ef96baa234966b60e464d1773fbf3f22)
(cherry picked from commit 3a764250262e745a6762dd6551b0d0a6af39e891)
 2019-01-24 15:58:27 -08:00
Sunil G
d11fdbb7ac YARN-8961. [UI2] Flow Run End Time shows 'Invalid date'. Contributed by Akhil PB 
(cherry picked from commit c7264459901e59021e180228cea3a6ae334a339f)
 2019-01-24 15:03:16 +05:30
Weiwei Yang
0e3d5f3064 YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang. 
(cherry picked from commit d43df31751bcadab77d42b31e3e1dd5748b471b5)
 2019-01-22 11:15:03 +08:00
Akira Ajisaka
1a38a61716
YARN-8747. [UI2] YARN UI2 page loading failed due to js error under some time zone configuration. Contributed by collinma. 
(cherry picked from commit 104ef5df36fd45f1c2d07a2f3d441263aa85e22e)
 2019-01-16 14:38:55 +09:00
Weiwei Yang
009f5b55ad YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed Wilfred Spiegelenburg. 
(cherry picked from commit 91e9c9f96eca01bbef7d22b4cff1ced3be1ac5e8)
 2019-01-08 14:09:49 +08:00
Jonathan Hung
a0291a015c YARN-9175. Null resources check in ResourceInfo for branch-3.0 2019-01-03 15:58:10 -05:00
Weiwei Yang
c4e39e3a59 YARN-9164. Shutdown NM may cause NPE when opportunistic container scheduling is enabled. Contributed by lujie. 
(cherry picked from commit cfe89e6f963ba25b5fff1ce48cad36d74b3c789c)
 2019-01-04 02:00:59 +08:00
Eric Yang
2ffd8487b4 YARN-9125. Fixed Carriage Return detection in Docker container launch. 
Contributed by Billie Rinaldi
 2018-12-17 12:29:49 -05:00
Eric Yang
55da3078b1 YARN-9040. Fixed memory leak in LevelDBCacheTimelineStore and DBIterator. 
Contributed by Tarun Parimi

(cherry picked from commit 71e0b0d8005ea1952dc7e582db15c2ac09df7c91)
 2018-12-17 12:08:56 -05:00
Weiwei Yang
d78234222d YARN-9009. Fix flaky test TestEntityGroupFSTimelineStore.testCleanLogs. Contributed by OrDTesters. 
(cherry picked from commit 1c09a10e9601bd628f1e887f3bf92c5a4ac286ab)
 2018-12-10 12:36:53 +08:00
Jonathan Hung
80ef3e7581 YARN-9085. Add Guaranteed and MaxCapacity to CSQueueMetrics 
(cherry picked from commit 978ab3e958227220cb6f1a08ae6e7cdb8a46628b)
(cherry picked from commit dca69d178dba21c41fd1293187f29143f7e81e19)
 2018-12-07 10:46:07 -08:00
Jonathan Hung
15057a7e67 YARN-9036. Escape newlines in health report in YARN UI. Contributed by Keqiu Hu 
(cherry picked from commit c3d10aa6eddadd0e00edfb87cc77c9483ec8fb13)
 2018-11-30 10:17:26 -08:00
bibinchundatt
b42c2679ad YARN-9069. Fix SchedulerInfo#getSchedulerType for custom schedulers. Contributed by Bilwa S T. 
(cherry picked from commit 07142f54a8c7f70857e99c041f3a2a5189c809b5)
 2018-11-29 22:08:48 +05:30
Jason Lowe
e4dcc3e606 YARN-8812. Containers fail during creating a symlink which started with hyphen for a resource file. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 3ce99e32f7d7887412cae8337cd4ebeb3b2ee308)
 2018-11-28 08:56:04 -06:00
Weiwei Yang
067e39f781 YARN-8833. Avoid potential integer overflow when computing fair shares. Contributed by liyakun. 
(cherry picked from commit d027a24f0349b60efa5125c330058f123771748f)
 2018-11-18 23:25:31 +08:00
Akira Ajisaka
50ce9fd4e2
YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang. 2018-11-10 18:02:57 +09:00
Weiwei Yang
be2a0a5e23 YARN-8977. Remove unnecessary type casting when calling AbstractYarnScheduler#getSchedulerNode. Contributed by Wanqiang Ji. 
(cherry picked from commit c96cbe8659587cfc114a96aab1be5cc85029fe44)
 2018-11-07 22:54:34 +08:00
Takanobu Asanuma
eb365039da YARN-8866. Fix a parsing error for crossdomain.xml. 
(cherry picked from commit 8dc1f6dbf712a65390a9a6859f62fec0481af31b)
 2018-11-07 18:28:15 +09:00
Jason Lowe
f3e6682ba7 YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit ab6aa4c7265db5bcbb446c2f779289023d454b81)
 2018-11-06 08:59:47 -06:00
Weiwei Yang
ddb349ceb3 YARN-8970. Improve the debug message in CS#allocateContainerOnSingleNode. Contributed by Zhankun Tang. 
(cherry picked from commit 5d6554c722f08f79bce904e021243605ee75bae3)
 2018-11-06 14:54:33 +08:00
Weiwei Yang
c3a0f07db0 YARN-8969. AbstractYarnScheduler#getNodeTracker should return generic type to avoid type casting. Contributed by Wanqiang Ji. 
(cherry picked from commit c7fcca0d7ec9e31d43ef3040ecd576ec808f1f8b)
 2018-11-06 13:43:24 +08:00
Jonathan Hung
68ec0fa0a8 YARN-7225. Add queue and partition info to RM audit log. Contributed by Eric Payne 
(cherry picked from commit 2ab611d48b7669b31bd2c9b918f47251da77d0f6)
 2018-11-01 14:31:57 -07:00
Jason Lowe
a9bc3e3708 YARN-8904. TestRMDelegationTokens can fail in testRMDTMasterKeyStateOnRollingMasterKey. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 93fb3b4b9cbc63e65d590676a5a318d555a25904)
 2018-10-23 12:57:49 -05:00
Rohith Sharma K S
6ff7056b58 YARN-8826. Fix lingering timeline collector after serviceStop in TimelineCollectorManager. Contributed by Prabha Manepalli. 
(cherry picked from commit 0b62983c5a9361eb832784f134f140f9926c9ec6)
 2018-10-23 14:09:18 +05:30
Sunil G
78cf8c8099 YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. 
(cherry picked from commit e50334513c781a10368e7353bec6674f89eb0a7b)
 2018-10-22 14:17:29 +05:30
Weiwei Yang
37af9580c8 YARN-8907. Fix incorrect logging message in TestCapacityScheduler. Contributed by Zhankun Tang. 
(cherry picked from commit 13cc0f50ea1c5f8978455e34f49716ddb0e3a143)
 2018-10-19 10:03:39 +08:00
Wangda Tan
b67dc5e1d1 YARN-7274. Ability to disable elasticity at leaf queue level. (Zian Chen via wangda) 
Change-Id: Ic8d43e297f0f5de788b562f7eff8106c5c35e8d2
(cherry picked from commit 74665e3a7d7f05644d9a5abad5a3f2d47597d6c8)
 2018-10-11 16:00:10 +00:00
Weiwei Yang
af85ce6ae4 YARN-8858. CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used. Contributed by Wangda Tan. 
(cherry picked from commit edce866489d83744f3f47a3b884b0c6136885e4a)
 2018-10-10 09:56:16 +08:00
Eric E Payne
b412bb2240 YARN-8774. Memory leak when CapacityScheduler allocates from reserved container with non-default label. Contributed by Tao Yang. 
(cherry picked from commit 8598b498bcaf4deffa822f871a26635bdf3d9d5c)
 2018-09-28 15:48:51 +00:00
Jason Lowe
17583e690a YARN-8804. resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues. Contributed by Tao Yang 
(cherry picked from commit 6b988d821e62d29c118e10a7213583b92c302baf)
 2018-09-26 16:54:29 -07:00