Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,992 Commits 326 Branches 371 Tags
Commit Graph

5010 Commits

Author SHA1 Message Date
Eric Yang
dbc02bcda7 YARN-9391. Fixed node manager environment leaks into Docker containers. 
Contributed by Jim Brennan

(cherry picked from commit 3c45762a0bfb403e069a03e30d35dd11432ee8b0)
 2019-03-25 15:55:46 -04:00
Sunil G
6941033396 YARN-8803. [UI2] Show flow runs in the order of recently created time in graph widgets. Contributed by Akhil PB. 
(cherry picked from commit c79f139519e9b2486de31b307d7811b4c2a6b5b0)
 2019-03-06 16:50:19 +05:30
Sunil G
379a9bfd9a YARN-9138. Improve test coverage for nvidia-smi binary execution of GpuDiscoverer. Contributed by Szilard Nemeth. 
(cherry picked from commit 46045c5cb3ab06a35df27879afbd1bc3c2a384dd)
 2019-03-06 16:02:39 +05:30
bibinchundatt
e663a6af89 Revert "YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph" 
This reverts commit 7db50ffcebf5413bbd1a80ee2cd8e2ffc11befe1.
 2019-03-04 17:03:45 +05:30
Sunil G
80d507d1a4 YARN-9139. Simplify initializer code of GpuDiscoverer. Contributed by Szilard Nemeth. 2019-03-01 19:28:33 +05:30
Eric Yang
72eacb3e10 YARN-9334. Allow YARN Service client to send SPNEGO challenge header when authentication type is not simple. 
Contributed by Billie Rinaldi

(cherry picked from commit 04b228e43b728d574d7ad97330aa4218cb7f8bf8)
 2019-02-28 10:12:09 -08:00
Sunil G
817028364a YARN-9121. Replace GpuDiscoverer.getInstance() to a readable object for easy access control. Contributed by Szilard Nemeth. 2019-02-27 17:46:43 +05:30
Weiwei Yang
10d4a9a7fb YARN-9248. RMContainerImpl:Invalid event: ACQUIRED at KILLED. Contributed by lujie. 
(cherry picked from commit 8c30114b0055bdb44608d4a6d1fa838a04821ff6)
 2019-02-27 17:39:37 +08:00
Sunil G
51b010b19f YARN-9087. Improve logging for initialization of Resource plugins. Contributed by Szilard Nemeth. 2019-02-27 11:57:32 +05:30
Sunil G
84928ba3d1 YARN-9168. DistributedShell client timeout should be -1 by default. Contributed by Zhankun Tang. 
(cherry picked from commit 6cec90653dc2ae60a10262745997ea91dd872a8b)
 2019-02-25 15:29:53 +05:30
Sunil G
cb0f45b75b YARN-9213. RM Web UI v1 does not show custom resource allocations for containers page. Contributed by Szilard Nemeth. 
(cherry picked from commit f282f9c362dc5a2b7a86b7e2f78d53835c2a6188)
 2019-02-25 11:39:05 +05:30
Weiwei Yang
dab22e74a4 YARN-9316. TestPlacementConstraintsUtil#testInterAppConstraintsByAppID fails intermittently. Contributed by Prabhu Joseph. 
(cherry picked from commit 9cd5c5447f008b38be43b21ca596d8a36ad0cd55)
 2019-02-24 22:57:40 +08:00
bibinchundatt
6d1cf7b395 YARN-9317. Avoid repeated YarnConfiguration#timelineServiceV2Enabled check. Contributed by Prabhu Joseph 2019-02-23 08:03:28 +05:30
bibinchundatt
7db50ffceb YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph 2019-02-22 20:52:40 +05:30
Sunil G
d6377c8b68 YARN-9118. Handle exceptions with parsing user defined GPU devices in GpuDiscoverer. Contributed by Szilard Nemeth. 
(cherry picked from commit 95fbbfed75dd309b5d56032ece64996165572287)
 2019-02-22 20:23:51 +05:30
Weiwei Yang
040d475030 YARN-9238. Avoid allocating opportunistic containers to previous/removed/non-exist application attempt. Contributed by lujie. 
(cherry picked from commit 9c88695bcda0ffe4c7f49d643c649dfa1dce9bde)
 2019-02-22 21:43:53 +08:00
Weiwei Yang
1ffa7f8349 YARN-9315. TestCapacitySchedulerMetrics fails intermittently. Contributed by Prabhu Joseph. 2019-02-21 18:16:22 +08:00
bibinchundatt
77c7e8492e YARN-9286. [Timeline Server] Sorting based on FinalStatus shows pop-up message. Contributed by Bilwa S T. 
(cherry picked from commit b8de78c570babe4f802d951957c495ea0a4b07da)
 2019-02-20 01:21:12 +05:30
Sunil G
2576aea729 YARN-7824. [UI2] Yarn Component Instance page should include link to container logs. Contributed by Akhil PB. 
(cherry picked from commit a060e8cb518445994d1ace4c0dd625f18ae11cd9)
 2019-02-17 20:20:40 +05:30
Adam Antal
511ffb5f70
YARN-9283. Javadoc of LinuxContainerExecutor#addSchedPriorityCommand has a wrong property name as reference 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9385ec45d75109a2e6565faa10527cc56637bf5f)
 2019-02-15 18:50:42 +09:00
Sunil G
8551831ac2 YARN-8295. [UI2] Improve Resource Usage tab error message when there are no data available. Contributed by Charan Hebri. 
(cherry picked from commit 5b55f3538cb27baf8ac08568a5f752423e7b29a4)
 2019-02-15 12:43:26 +05:30
Akira Ajisaka
3b10c07946
YARN-9284. Fix the unit of yarn.service.am-resource.memory in the document. Contributed by Masahiro Tanaka. 
(cherry picked from commit 3a39d9a2d28943316dd32196bba10ef326218d6e)
 2019-02-15 15:44:02 +09:00
bibinchundatt
8b074e6be8 YARN-9295. [UI2] Fix label typo in Cluster Overview page. Contributed by Charan Hebri. 
(cherry picked from commit b66d5ae9e26447113b146be0ffd81ed7d663a778)
 2019-02-14 23:11:59 +05:30
Sunil G
ec08eed542 YARN-7761. [UI2] Clicking 'master container log' or 'Link' next to 'log' under application's appAttempt goes to Old UI's Log link. Contributed by Akhil PB. 
(cherry picked from commit d321d0e747ec91f6b66fe8eb73ea3b9ef99825ae)
 2019-02-14 20:56:52 +05:30
Weiwei Yang
f3c1e456b0 YARN-9253. Add UT to verify Placement Constraint in Distributed Shell. Contributed by Prabhu Joseph. 
(cherry picked from commit 711d22f166b4bb76005ae876a9d7ff7238991f90)
 2019-02-12 16:26:38 +08:00
Giovanni Matteo Fumarola
c6582cc04c YARN-9191. Add cli option in DS to support enforceExecutionType in resource requests. Contributed by Abhishek Modi. 
(cherry picked from commit f738b397ae021c9be900e4ec51ab55cd69b075e0)
 2019-02-12 14:21:57 +08:00
Eric Yang
102db40870 YARN-8761. Service AM support for decommissioning component instances. 
Contributed by Billie Rinaldi

(cherry picked from commit 4c465f5535054dad2ef0b18128fb115129f6939e)
 2019-02-08 08:38:43 -08:00
Masatake Iwasaki
11ebdaab48 YARN-9282. Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'. Contributed by Charan Hebri. 
(cherry picked from commit e0ab1bdecec3d7ba7ddc0849781d7f71714f8687)
 2019-02-09 00:29:58 +09:00
Sunil G
73956d5de9 YARN-9257. Distributed Shell client throws a NPE for a non-existent queue. Contributed by Charan Hebri. 
(cherry picked from commit fbc08145cfb6a81395448c4b3463bf6d28a6272b)
 2019-02-08 11:23:21 +05:30
Eric E Payne
834a862bd0 YARN-7171: RM UI should sort memory / cores numerically. Contributed by Ahmed Hussein 
(cherry picked from commit d1ca9432dd0b3e9b46b4903e8c9d33f5c28fcc1b)
 2019-02-07 20:36:54 +00:00
Sunil G
6ffe6ea899 YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla. 2019-02-07 19:08:41 +05:30
Wangda Tan
a1e09b4c0c Make upstream aware of 3.1.2 release 
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
(cherry picked from commit 308f3168fa221a336ef0dbb1a17358b1ad6f9356)
(cherry picked from commit 649da5af04d57ee910dbe7fb0f4a7d9cb4c40eb0)
 2019-02-05 14:09:33 -08:00
Weiwei Yang
41bdcf4110 YARN-9262. TestRMAppAttemptTransitions is failing with an NPE. Contributed by lujie. 
(cherry picked from commit 28ad20a711735243bc10b10e33866dc525f415eb)
 2019-02-04 14:00:54 +05:30
Sunil G
3b03ff6fdd YARN-9099. GpuResourceAllocator#getReleasingGpus calculates number of GPUs in a wrong way. Contributed by Szilard Nemeth. 
(cherry picked from commit 71c49fa60faad2504b0411979a6e46e595b97a85)
 2019-01-31 09:26:38 +05:30
Eric E Payne
0cb05a9fe3 YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph 
(cherry picked from commit 04105bbfdb041a41062c856632641140de84fba8)
 2019-01-29 18:04:01 +00:00
Weiwei Yang
4257043232 YARN-9237. NM should ignore sending finished apps to RM during RM fail-over. Contributed by Jiandan Yang. 
(cherry picked from commit 4f63ffe444286dac91ed36fd647d8ce69e75b0f0)
 2019-01-29 11:03:26 +08:00
Eric Yang
29ccb8689f YARN-8901. Fixed restart policy NEVER/ON_FAILURE with component dependency. 
Contributed by Suma Shivaprasad

(cherry picked from commit f5a95f7998e110cab81e52acd99b07e13ea9653d)
 2019-01-28 18:12:39 -05:00
Rohith Sharma K S
6e059c7930 Revert "YARN-8270 Adding JMX Metrics for Timeline Collector and Reader. Contributed by Sushil Ks." 
This reverts commit 5b72aa04e104242d1761abf56822fb38e9915def.
 2019-01-28 10:55:12 +05:30
Jonathan Hung
6092d913b1 YARN-9222. Print launchTime in ApplicationSummary 
(cherry picked from commit 6cace58e212d3ee0aec988926a5a17c9cc58e645)
(cherry picked from commit bf760e7e81f8e02ad413c470fccf78aaa9cb9f86)
 2019-01-25 13:50:44 -08:00
Haibo Chen
61a6cc8d23 YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 
(cherry picked from commit bb92bfb4ef96baa234966b60e464d1773fbf3f22)
 2019-01-24 15:58:37 -08:00
Sunil G
45c4df152f YARN-8961. [UI2] Flow Run End Time shows 'Invalid date'. Contributed by Akhil PB 
(cherry picked from commit c7264459901e59021e180228cea3a6ae334a339f)
 2019-01-24 15:02:53 +05:30
Weiwei Yang
2471d8a6e7 YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang. 
(cherry picked from commit bc6374f282dbff3b9ed91fb5d7825d57e6720f5e)
 2019-01-23 18:18:38 +08:00
Weiwei Yang
b61754b1bd YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang. 
(cherry picked from commit d43df31751bcadab77d42b31e3e1dd5748b471b5)
 2019-01-22 11:01:00 +08:00
Weiwei Yang
4edd883d48 YARN-9204. RM fails to start if absolute resource is specified for partition capacity in CS queues. Contributed by Jiandan Yang. 
(cherry picked from commit abde1e1f58d5b699e4b8e460cff68e154738169b)
 2019-01-21 21:27:40 +08:00
Wangda Tan
a685ffe9a9 YARN-9194. Invalid event: REGISTERED and LAUNCH_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM. (lujie via wangda) 
Change-Id: I4359f59a73a278a941f4bb9d106dd38c9cb471fe
(cherry picked from commit 6d7eedfd28cc1712690db2f6ca8a281b0901ee28)
(cherry picked from commit fe7cb2d84ac160c5fed00640d85e2c5c4c6d2412)
 2019-01-17 15:17:34 -08:00
rahul3
8dea94d071
YARN-9203. Fix typos in yarn-default.xml. 
This closes #437

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 0a46baecd31c485d1ea4e567c29c47bfba0b092e)
 2019-01-17 16:20:15 +09:00
Akira Ajisaka
d07d275f94
YARN-8747. [UI2] YARN UI2 page loading failed due to js error under some time zone configuration. Contributed by collinma. 
(cherry picked from commit 104ef5df36fd45f1c2d07a2f3d441263aa85e22e)
 2019-01-16 14:38:29 +09:00
Weiwei Yang
91e9c9f96e YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed Wilfred Spiegelenburg. 2019-01-08 13:56:21 +08:00
Wangda Tan
3570713a69 YARN-8822. Nvidia-docker v2 support for YARN GPU feature. (Charo Zhang via wangda) 
Change-Id: I416268888a7b6f097d218d84e8497dd70b4b6d8f
 2019-01-07 12:30:30 -08:00
Eric Yang
a446949f56 HADOOP-16031. Fixed TestSecureLogins unit test. Contributed by Akira Ajisaka 
(cherry picked from commit bba76b6f31d5452fe048dcc6853c12a1894024fe)
 2019-01-07 13:24:50 -05:00