Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,025 Commits 326 Branches 371 Tags
Commit Graph

5021 Commits

Author SHA1 Message Date
Sunil G
f09befd2ea YARN-9519. TFile log aggregation file format is not working for yarn.log-aggregation.TFile.remote-app-log-dir config. Contributed by Adam Antal. 
(cherry picked from commit 7d831eca645f93d064975ebae35a7cbea3bbad31)
 2019-05-14 10:50:02 -07:00
Sunil G
8b306e34e0 YARN-9504. [UI2] Fair scheduler queue view page does not show actual capacity. Contributed by Zoltan Siegl. 
(cherry picked from commit 64c7f36ab12faba26a2ecf1e6a0cab7eb2bcffdd)
 2019-05-10 14:28:08 +05:30
Eric Yang
bf013aa06e YARN-8622. Fixed container-executor compilation on MacOSX. 
Contributed by Siyao Meng

(cherry picked from commit ef97a20831677c055aa6bff6ad0649cbb3a56a86)
 2019-05-09 14:55:38 -04:00
Haibo Chen
ea1f0f282b YARN-9529. Log correct cpu controller path on error while initializing CGroups. (Contributed by Jonathan Hung) 
(cherry picked from commit 597fa47ad125c0871f5c4deb3a883e5b3341c67b)
(cherry picked from commit c6573562cbc43832e9332989996e5d07b8bce9b0)
 2019-05-06 11:59:20 -07:00
Eric E Payne
41ffaea342 YARN-9285: RM UI progress column is of wrong type. Contributed by Ahmed Hussein. 
(cherry picked from commit b094b94d43a46af9ddb910da24f792b95f614b08)
 2019-05-02 19:57:44 +00:00
Weiwei Yang
94a895b94f YARN-9307. node_partitions constraint does not work. Contributed by kyungwan nam. 2019-04-26 13:16:43 +08:00
Weiwei Yang
d242b166ed YARN-9325. TestQueueManagementDynamicEditPolicy fails intermittent. Contributed by Prabhu Joseph. 
(cherry picked from commit 1c8046d67ec10710e7749ed1929b09fac4b1ba94)
 2019-04-23 14:25:33 +08:00
Eric Yang
8b228a42e9 YARN-8587. Added retries for fetching docker exit code. 
Contributed by Charo Zhang

(cherry picked from commit c16c49b8c3b8e2e42c00e79a50e7ae029ebe98e2)
 2019-04-19 15:40:56 -04:00
Eric Yang
68a98be8a2 YARN-6695. Fixed NPE in publishing appFinished events to ATSv2. 
Contributed by Prabhu Joseph

(cherry picked from commit df76cdc8959c51b71704ab5c38335f745a6f35d8)
 2019-04-18 12:31:34 -04:00
Weiwei Yang
c37065eae9 YARN-9463. Add queueName info when failing with queue capacity sanity check. Contributed by Aihua Xu. 
(cherry picked from commit 8c1bba375b144fd515b389174ddb349f2d9246fa)
 2019-04-10 23:04:27 +08:00
Weiwei Yang
bd0c9bc160 YARN-9413. Queue resource leak after app fail for CapacityScheduler. Contributed by Tao Yang. 
(cherry picked from commit ec143cbf678bd65f87fdd464c23022a2d2c54c07)
 2019-04-06 20:38:06 +08:00
Eric Yang
dbc02bcda7 YARN-9391. Fixed node manager environment leaks into Docker containers. 
Contributed by Jim Brennan

(cherry picked from commit 3c45762a0bfb403e069a03e30d35dd11432ee8b0)
 2019-03-25 15:55:46 -04:00
Sunil G
6941033396 YARN-8803. [UI2] Show flow runs in the order of recently created time in graph widgets. Contributed by Akhil PB. 
(cherry picked from commit c79f139519e9b2486de31b307d7811b4c2a6b5b0)
 2019-03-06 16:50:19 +05:30
Sunil G
379a9bfd9a YARN-9138. Improve test coverage for nvidia-smi binary execution of GpuDiscoverer. Contributed by Szilard Nemeth. 
(cherry picked from commit 46045c5cb3ab06a35df27879afbd1bc3c2a384dd)
 2019-03-06 16:02:39 +05:30
bibinchundatt
e663a6af89 Revert "YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph" 
This reverts commit 7db50ffcebf5413bbd1a80ee2cd8e2ffc11befe1.
 2019-03-04 17:03:45 +05:30
Sunil G
80d507d1a4 YARN-9139. Simplify initializer code of GpuDiscoverer. Contributed by Szilard Nemeth. 2019-03-01 19:28:33 +05:30
Eric Yang
72eacb3e10 YARN-9334. Allow YARN Service client to send SPNEGO challenge header when authentication type is not simple. 
Contributed by Billie Rinaldi

(cherry picked from commit 04b228e43b728d574d7ad97330aa4218cb7f8bf8)
 2019-02-28 10:12:09 -08:00
Sunil G
817028364a YARN-9121. Replace GpuDiscoverer.getInstance() to a readable object for easy access control. Contributed by Szilard Nemeth. 2019-02-27 17:46:43 +05:30
Weiwei Yang
10d4a9a7fb YARN-9248. RMContainerImpl:Invalid event: ACQUIRED at KILLED. Contributed by lujie. 
(cherry picked from commit 8c30114b0055bdb44608d4a6d1fa838a04821ff6)
 2019-02-27 17:39:37 +08:00
Sunil G
51b010b19f YARN-9087. Improve logging for initialization of Resource plugins. Contributed by Szilard Nemeth. 2019-02-27 11:57:32 +05:30
Sunil G
84928ba3d1 YARN-9168. DistributedShell client timeout should be -1 by default. Contributed by Zhankun Tang. 
(cherry picked from commit 6cec90653dc2ae60a10262745997ea91dd872a8b)
 2019-02-25 15:29:53 +05:30
Sunil G
cb0f45b75b YARN-9213. RM Web UI v1 does not show custom resource allocations for containers page. Contributed by Szilard Nemeth. 
(cherry picked from commit f282f9c362dc5a2b7a86b7e2f78d53835c2a6188)
 2019-02-25 11:39:05 +05:30
Weiwei Yang
dab22e74a4 YARN-9316. TestPlacementConstraintsUtil#testInterAppConstraintsByAppID fails intermittently. Contributed by Prabhu Joseph. 
(cherry picked from commit 9cd5c5447f008b38be43b21ca596d8a36ad0cd55)
 2019-02-24 22:57:40 +08:00
bibinchundatt
6d1cf7b395 YARN-9317. Avoid repeated YarnConfiguration#timelineServiceV2Enabled check. Contributed by Prabhu Joseph 2019-02-23 08:03:28 +05:30
bibinchundatt
7db50ffceb YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph 2019-02-22 20:52:40 +05:30
Sunil G
d6377c8b68 YARN-9118. Handle exceptions with parsing user defined GPU devices in GpuDiscoverer. Contributed by Szilard Nemeth. 
(cherry picked from commit 95fbbfed75dd309b5d56032ece64996165572287)
 2019-02-22 20:23:51 +05:30
Weiwei Yang
040d475030 YARN-9238. Avoid allocating opportunistic containers to previous/removed/non-exist application attempt. Contributed by lujie. 
(cherry picked from commit 9c88695bcda0ffe4c7f49d643c649dfa1dce9bde)
 2019-02-22 21:43:53 +08:00
Weiwei Yang
1ffa7f8349 YARN-9315. TestCapacitySchedulerMetrics fails intermittently. Contributed by Prabhu Joseph. 2019-02-21 18:16:22 +08:00
bibinchundatt
77c7e8492e YARN-9286. [Timeline Server] Sorting based on FinalStatus shows pop-up message. Contributed by Bilwa S T. 
(cherry picked from commit b8de78c570babe4f802d951957c495ea0a4b07da)
 2019-02-20 01:21:12 +05:30
Sunil G
2576aea729 YARN-7824. [UI2] Yarn Component Instance page should include link to container logs. Contributed by Akhil PB. 
(cherry picked from commit a060e8cb518445994d1ace4c0dd625f18ae11cd9)
 2019-02-17 20:20:40 +05:30
Adam Antal
511ffb5f70
YARN-9283. Javadoc of LinuxContainerExecutor#addSchedPriorityCommand has a wrong property name as reference 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9385ec45d75109a2e6565faa10527cc56637bf5f)
 2019-02-15 18:50:42 +09:00
Sunil G
8551831ac2 YARN-8295. [UI2] Improve Resource Usage tab error message when there are no data available. Contributed by Charan Hebri. 
(cherry picked from commit 5b55f3538cb27baf8ac08568a5f752423e7b29a4)
 2019-02-15 12:43:26 +05:30
Akira Ajisaka
3b10c07946
YARN-9284. Fix the unit of yarn.service.am-resource.memory in the document. Contributed by Masahiro Tanaka. 
(cherry picked from commit 3a39d9a2d28943316dd32196bba10ef326218d6e)
 2019-02-15 15:44:02 +09:00
bibinchundatt
8b074e6be8 YARN-9295. [UI2] Fix label typo in Cluster Overview page. Contributed by Charan Hebri. 
(cherry picked from commit b66d5ae9e26447113b146be0ffd81ed7d663a778)
 2019-02-14 23:11:59 +05:30
Sunil G
ec08eed542 YARN-7761. [UI2] Clicking 'master container log' or 'Link' next to 'log' under application's appAttempt goes to Old UI's Log link. Contributed by Akhil PB. 
(cherry picked from commit d321d0e747ec91f6b66fe8eb73ea3b9ef99825ae)
 2019-02-14 20:56:52 +05:30
Weiwei Yang
f3c1e456b0 YARN-9253. Add UT to verify Placement Constraint in Distributed Shell. Contributed by Prabhu Joseph. 
(cherry picked from commit 711d22f166b4bb76005ae876a9d7ff7238991f90)
 2019-02-12 16:26:38 +08:00
Giovanni Matteo Fumarola
c6582cc04c YARN-9191. Add cli option in DS to support enforceExecutionType in resource requests. Contributed by Abhishek Modi. 
(cherry picked from commit f738b397ae021c9be900e4ec51ab55cd69b075e0)
 2019-02-12 14:21:57 +08:00
Eric Yang
102db40870 YARN-8761. Service AM support for decommissioning component instances. 
Contributed by Billie Rinaldi

(cherry picked from commit 4c465f5535054dad2ef0b18128fb115129f6939e)
 2019-02-08 08:38:43 -08:00
Masatake Iwasaki
11ebdaab48 YARN-9282. Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'. Contributed by Charan Hebri. 
(cherry picked from commit e0ab1bdecec3d7ba7ddc0849781d7f71714f8687)
 2019-02-09 00:29:58 +09:00
Sunil G
73956d5de9 YARN-9257. Distributed Shell client throws a NPE for a non-existent queue. Contributed by Charan Hebri. 
(cherry picked from commit fbc08145cfb6a81395448c4b3463bf6d28a6272b)
 2019-02-08 11:23:21 +05:30
Eric E Payne
834a862bd0 YARN-7171: RM UI should sort memory / cores numerically. Contributed by Ahmed Hussein 
(cherry picked from commit d1ca9432dd0b3e9b46b4903e8c9d33f5c28fcc1b)
 2019-02-07 20:36:54 +00:00
Sunil G
6ffe6ea899 YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla. 2019-02-07 19:08:41 +05:30
Wangda Tan
a1e09b4c0c Make upstream aware of 3.1.2 release 
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
(cherry picked from commit 308f3168fa221a336ef0dbb1a17358b1ad6f9356)
(cherry picked from commit 649da5af04d57ee910dbe7fb0f4a7d9cb4c40eb0)
 2019-02-05 14:09:33 -08:00
Weiwei Yang
41bdcf4110 YARN-9262. TestRMAppAttemptTransitions is failing with an NPE. Contributed by lujie. 
(cherry picked from commit 28ad20a711735243bc10b10e33866dc525f415eb)
 2019-02-04 14:00:54 +05:30
Sunil G
3b03ff6fdd YARN-9099. GpuResourceAllocator#getReleasingGpus calculates number of GPUs in a wrong way. Contributed by Szilard Nemeth. 
(cherry picked from commit 71c49fa60faad2504b0411979a6e46e595b97a85)
 2019-01-31 09:26:38 +05:30
Eric E Payne
0cb05a9fe3 YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph 
(cherry picked from commit 04105bbfdb041a41062c856632641140de84fba8)
 2019-01-29 18:04:01 +00:00
Weiwei Yang
4257043232 YARN-9237. NM should ignore sending finished apps to RM during RM fail-over. Contributed by Jiandan Yang. 
(cherry picked from commit 4f63ffe444286dac91ed36fd647d8ce69e75b0f0)
 2019-01-29 11:03:26 +08:00
Eric Yang
29ccb8689f YARN-8901. Fixed restart policy NEVER/ON_FAILURE with component dependency. 
Contributed by Suma Shivaprasad

(cherry picked from commit f5a95f7998e110cab81e52acd99b07e13ea9653d)
 2019-01-28 18:12:39 -05:00
Rohith Sharma K S
6e059c7930 Revert "YARN-8270 Adding JMX Metrics for Timeline Collector and Reader. Contributed by Sushil Ks." 
This reverts commit 5b72aa04e104242d1761abf56822fb38e9915def.
 2019-01-28 10:55:12 +05:30
Jonathan Hung
6092d913b1 YARN-9222. Print launchTime in ApplicationSummary 
(cherry picked from commit 6cace58e212d3ee0aec988926a5a17c9cc58e645)
(cherry picked from commit bf760e7e81f8e02ad413c470fccf78aaa9cb9f86)
 2019-01-25 13:50:44 -08:00