Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,936 Commits 326 Branches 371 Tags
Commit Graph

3806 Commits

Author SHA1 Message Date
Eric Badger
afa9a4084d YARN-9442. container working directory has group read permissions. Contributed by Jim Brennan. 
(cherry picked from commit 2ac029b949f041da2ee04da441c5f9f85e1f2c64)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c

(cherry picked from commit cec71691be76577718b22f936aea9e2b2cd100ea)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c

(cherry picked from commit db88224e8f9d164ac811fcca9efe4a350cebecd1)
 2019-08-13 17:27:43 +00:00
Erik Krogen
d5da529acf HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2019-08-09 15:49:55 -07:00
Haibo Chen
4a9fc45f6f YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung) 
(cherry picked from commit f51702d5398531835b24d812f6f95094a0e0493e)
(cherry picked from commit 8d357343c4bc9f18e25543583f8f217b8a2f621b)
(cherry picked from commit f943bff2546330bc9eb0914d1a0fa1ef4c709b72)
(cherry picked from commit 6109ecf16d6f0ebf1e314e163b5c41ad75758be1)
 2019-08-06 15:14:45 -07:00
Weiwei Yang
19af228957 YARN-8944. TestContainerAllocation.testUserLimitAllocationMultipleContainers failure after YARN-8896. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 1d90a0dd23c5d1bd52d04c303506806d0dc61cd2)
(cherry picked from commit 70efe253f369f579f05d833d45e6a1bae3e41dfe)
(cherry picked from commit cd8e9fa80b2903935ca9c337631e60161a6053b3)
 2019-08-05 10:04:09 -07:00
Weiwei Yang
4e5ba315c6 YARN-8915. Update the doc about the default value of maximum-container-assignments for capacity scheduler. Contributed by Zhankun Tang. 
(cherry picked from commit 78ea897b6701c82799a6231103ca7ee1f8b4efde)
(cherry picked from commit 7dd8eafe337a33689ff6f2f947ad152e06dbc55f)
(cherry picked from commit d095325234fe8fa8d2513c2f03c03347ef5f3606)
 2019-08-05 10:04:00 -07:00
Wangda Tan
75fc058290 YARN-8896. Limit the maximum number of container assignments per heartbeat. (Zhankun Tang via wangda) 
Change-Id: I6e72f8362bd7f5c2a844cb9e3c4732492314e9f1
(cherry picked from commit 780be14f07df2a3ed6273b96ae857c278fd72718)
(cherry picked from commit 46baafedf13446f5c1b973d667f437bf7158d3c2)
(cherry picked from commit bf4da6aacee62be906e71eca6e33821f5ee79a2d)
 2019-08-05 10:03:56 -07:00
Eric Badger
9aae72af84 YARN-8045. Reduce log output from container status calls. Contributed by Craig Condit 
(cherry picked from commit 144a55f0e3ba302327baf2e98d1e07b953dcbbfd)
 2019-08-02 21:06:54 +00:00
Eric E Payne
77d1aa9376 YARN-9596: QueueMetrics has incorrect metrics when labelled partitions are involved. Contributed by Muhammad Samir Khan. 
(cherry picked from commit c04ccf296dab6eaa53f008342e50f8729935a50d)
 2019-07-30 20:31:02 +00:00
Ahmed Hussein
dc4d7f69ce YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 9d63fc7ed61ca6c3d901fe9984a7cca98ce07ea9)
 2019-07-24 11:21:29 -05:00
Haibo Chen
68c24b322d YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonanthan Hung) 2019-07-19 11:00:38 -07:00
Eric E Payne
441339a716 YARN-8625. Aggregate Resource Allocation for each job is not present in ATS. Contributed by Prabhu Joseph. 2019-06-04 18:57:44 +00:00
Takanobu Asanuma
e3d4f6bfed HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:41:21 +09:00
Akira Ajisaka
cea1b0000c
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:28:54 +09:00
Haibo Chen
dc28c86270 YARN-9529. Log correct cpu controller path on error while initializing CGroups. (Contributed by Jonathan Hung) 
(cherry picked from commit 597fa47ad125c0871f5c4deb3a883e5b3341c67b)
(cherry picked from commit c6573562cbc43832e9332989996e5d07b8bce9b0)
(cherry picked from commit ea1f0f282bee7ca64ce9fbe1a59bb5e231420d29)
(cherry picked from commit aa7ff8552e0f1b9439926ae3476a3227dcf339f3)
 2019-05-06 12:01:26 -07:00
Eric E Payne
5bf40f592e YARN-9285: RM UI progress column is of wrong type. Contributed by Ahmed Hussein. 2019-05-03 16:17:43 +00:00
Vrushali C
396fcee0b0 YARN-9150 Making TimelineSchemaCreator support different backends for Timeline Schema Creation in ATSv2. Contributed by Sushil Ks 2019-03-07 23:04:36 -08:00
Vrushali C
b83ec96767 YARN-8549 Adding a NoOp timeline writer and reader plugin classes for ATSv2. Contributed by Prabha Manepalli 2019-03-07 22:54:32 -08:00
Eric E Payne
6e6ebc368c YARN-5714. ContainerExecutor does not order environment map. Contributed by Remi Catherinot and Jim Brennan. 2019-03-07 16:22:36 +00:00
Adam Antal
3c7c3366ff
YARN-9283. Javadoc of LinuxContainerExecutor#addSchedPriorityCommand has a wrong property name as reference 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9385ec45d75109a2e6565faa10527cc56637bf5f)
 2019-02-15 19:00:48 +09:00
Masatake Iwasaki
83a9369f7d YARN-9282. Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'. Contributed by Charan Hebri. 
(cherry picked from commit e0ab1bdecec3d7ba7ddc0849781d7f71714f8687)
 2019-02-09 00:34:57 +09:00
Eric E Payne
2a7dcc509b YARN-7171: RM UI should sort memory / cores numerically. Contributed by Ahmed Hussein 
(cherry picked from commit d1ca9432dd0b3e9b46b4903e8c9d33f5c28fcc1b)
 2019-02-07 21:29:23 +00:00
Sunil G
a91933620d YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla. 
(cherry picked from commit 6ffe6ea8999c1216b6e510e350535754431bac2d)
 2019-02-07 19:11:12 +05:30
Eric E Payne
c6c5e94de4 YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph 
(cherry picked from commit 04105bbfdb041a41062c856632641140de84fba8)
 2019-01-29 18:22:12 +00:00
Jonathan Hung
338dbbedf7 YARN-9222. Print launchTime in ApplicationSummary 
(cherry picked from commit 6cace58e212d3ee0aec988926a5a17c9cc58e645)
(cherry picked from commit bf760e7e81f8e02ad413c470fccf78aaa9cb9f86)
(cherry picked from commit 80a08c29adc9d07974bbe8087abcec750d5f7916)
(cherry picked from commit c71054c428b3abe00507792099e13a2dad337697)
 2019-01-25 13:40:51 -08:00
Haibo Chen
c02c19121e YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 
(cherry picked from commit bb92bfb4ef96baa234966b60e464d1773fbf3f22)
(cherry picked from commit 3a764250262e745a6762dd6551b0d0a6af39e891)
(cherry picked from commit 75df8bed6eed86ad7c34460a1167c35d48b559d7)
 2019-01-23 12:18:42 -08:00
Weiwei Yang
c77f7e25c9 YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang. 2019-01-23 11:02:15 +08:00
Akira Ajisaka
300d65c4d7
YARN-8747. [UI2] YARN UI2 page loading failed due to js error under some time zone configuration. Contributed by collinma. 
(cherry picked from commit 104ef5df36fd45f1c2d07a2f3d441263aa85e22e)
 2019-01-16 14:39:17 +09:00
Weiwei Yang
c54a689d96 YARN-8833. Avoid potential integer overflow when computing fair shares. Contributed by liyakun. 2019-01-09 14:33:06 +08:00
Botong Huang
15286d9166 YARN 9108. Fix FederationIntercepter merge home and secondary allocate response typo. Contributed by Abhishek Modi. 2018-12-22 12:46:13 -08:00
Jonathan Hung
eb8b1eaa9a YARN-9085. Add Guaranteed and MaxCapacity to CSQueueMetrics 
(cherry picked from commit cc51607ccdf60f4ebecf7afa3e2056297ca0ac5f)
 2018-12-07 10:47:45 -08:00
Jonathan Hung
7456da99c1 YARN-9036. Escape newlines in health report in YARN UI. Contributed by Keqiu Hu 2018-11-30 10:18:17 -08:00
Botong Huang
b7ca20e16e YARN-8933. [AMRMProxy] Fix potential empty fields in allocation response, move SubClusterTimeout to FederationInterceptor. Contributed by Botong Huang. 2018-11-11 19:41:08 -08:00
Akira Ajisaka
ee1cb3b6e8
YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang. 2018-11-10 21:44:46 +09:00
bibinchundatt
731b90d44f YARN-8905. [Router] Add JvmMetricsInfo and pause monitor. Contributed by Bilwa S T. 
(cherry picked from commit f84a278baad6cb871ea4196257f23a938826ca23)
 2018-11-03 20:50:28 +05:30
Giovanni Matteo Fumarola
81da8b262b YARN-8893. [AMRMProxy] Fix thread leak in AMRMClientRelayer and UAM client. Contributed by Botong Huang. 2018-11-02 16:02:56 -07:00
Jonathan Hung
62ec800bd4 YARN-7225. Add queue and partition info to RM audit log. Contributed by Eric Payne 2018-11-01 14:13:15 -07:00
Rohith Sharma K S
dbf36dbb8d YARN-8826. Fix lingering timeline collector after serviceStop in TimelineCollectorManager. Contributed by Prabha Manepalli. 
(cherry picked from commit 0b62983c5a9361eb832784f134f140f9926c9ec6)
 2018-10-23 14:10:24 +05:30
Sunil G
56a27284a7 YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. 
(cherry picked from commit e50334513c781a10368e7353bec6674f89eb0a7b)
 2018-10-22 14:18:37 +05:30
Vrushali C
7ed627af6b YARN-3879 [Storage implementation] Create HDFS backing storage implementation for ATS reads. Contributed by Abhishek Modi. 2018-10-11 21:23:34 -07:00
Wangda Tan
91f18a5148 YARN-7274. Ability to disable elasticity at leaf queue level. (Zian Chen via wangda) 
Change-Id: Ic8d43e297f0f5de788b562f7eff8106c5c35e8d2
(cherry picked from commit 74665e3a7d7f05644d9a5abad5a3f2d47597d6c8)
 2018-10-11 16:58:07 +00:00
Vrushali C
7a5d27dde4 YARN-6989 Ensure timeline service v2 codebase gets UGI from HttpServletRequest in a consistent way. Contributed by Abhishek Modi 2018-10-10 15:25:58 -07:00
Weiwei Yang
cc1bf7f894 YARN-8858. CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used. Contributed by Wangda Tan. 
(cherry picked from commit edce866489d83744f3f47a3b884b0c6136885e4a)
 2018-10-10 10:17:18 +08:00
Weiwei Yang
d412785175 YARN-8720. CapacityScheduler does not enforce max resource allocation check at queue level. Contributed by Tarun Parimi. 
(cherry picked from commit f1a893fdbc2dbe949cae786f08bdb2651b88d673)
 2018-10-10 10:16:16 +08:00
Inigo Goiri
b0900ad310 YARN-7652. Handle AM register requests asynchronously in FederationInterceptor. Contributed by Botong Huang. 
(cherry picked from commit c3d22d3b4569b7f87af4ee4abfcc284deebe90de)
 2018-10-09 10:30:42 -07:00
Akira Ajisaka
65aaa10173
YARN-8840. Add missing cleanupSSLConfig() call for TestTimelineClient test. Contributed by Aki Tanaka. 
(cherry picked from commit 39b35036ba47064149003046a7b59feb01575d1e)
 2018-10-04 10:15:37 +09:00
Daniel Templeton
1505058440
MAPREDUCE-6896. Document wrong spelling in usage of MapredTestDriver tools. 
(Contributed by LiXin Ge via Daniel Templeton)

(cherry picked from commit 036a24bab3f3ff89e3d52c5487fce0525c0d1d78)
 2018-10-04 10:12:57 +09:00
Giovanni Matteo Fumarola
57caab6f9b YARN-8760. [AMRMProxy] Fix concurrent re-register due to YarnRM failover in AMRMClientRelayer. Contributed by Botong Huang. 2018-10-01 14:27:01 -07:00
Eric E Payne
d7d0e55e0a YARN-8774. Memory leak when CapacityScheduler allocates from reserved container with non-default label. Contributed by Tao Yang. 2018-09-28 16:10:31 +00:00
Jason Lowe
1b0a110501 YARN-8804. resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues. Contributed by Tao Yang 
(cherry picked from commit 6b988d821e62d29c118e10a7213583b92c302baf)
 2018-09-26 17:06:07 -07:00
Arun Suresh
4d69741a61 YARN-7974. Allow updating application tracking url after registration. (Jonathan Hung via asuresh) 2018-09-26 00:08:10 -07:00