Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,145 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

5051 Commits

Author SHA1 Message Date
Sunil G 84928ba3d1 YARN-9168. DistributedShell client timeout should be -1 by default. Contributed by Zhankun Tang. 
(cherry picked from commit 6cec90653d)
 2019-02-25 15:29:53 +05:30
Sunil G cb0f45b75b YARN-9213. RM Web UI v1 does not show custom resource allocations for containers page. Contributed by Szilard Nemeth. 
(cherry picked from commit f282f9c362)
 2019-02-25 11:39:05 +05:30
Weiwei Yang dab22e74a4 YARN-9316. TestPlacementConstraintsUtil#testInterAppConstraintsByAppID fails intermittently. Contributed by Prabhu Joseph. 
(cherry picked from commit 9cd5c5447f)
 2019-02-24 22:57:40 +08:00
bibinchundatt 6d1cf7b395 YARN-9317. Avoid repeated YarnConfiguration#timelineServiceV2Enabled check. Contributed by Prabhu Joseph 2019-02-23 08:03:28 +05:30
bibinchundatt 7db50ffceb YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph 2019-02-22 20:52:40 +05:30
Sunil G d6377c8b68 YARN-9118. Handle exceptions with parsing user defined GPU devices in GpuDiscoverer. Contributed by Szilard Nemeth. 
(cherry picked from commit 95fbbfed75)
 2019-02-22 20:23:51 +05:30
Weiwei Yang 040d475030 YARN-9238. Avoid allocating opportunistic containers to previous/removed/non-exist application attempt. Contributed by lujie. 
(cherry picked from commit 9c88695bcd)
 2019-02-22 21:43:53 +08:00
Weiwei Yang 1ffa7f8349 YARN-9315. TestCapacitySchedulerMetrics fails intermittently. Contributed by Prabhu Joseph. 2019-02-21 18:16:22 +08:00
bibinchundatt 77c7e8492e YARN-9286. [Timeline Server] Sorting based on FinalStatus shows pop-up message. Contributed by Bilwa S T. 
(cherry picked from commit b8de78c570)
 2019-02-20 01:21:12 +05:30
Sunil G 2576aea729 YARN-7824. [UI2] Yarn Component Instance page should include link to container logs. Contributed by Akhil PB. 
(cherry picked from commit a060e8cb51)
 2019-02-17 20:20:40 +05:30
Adam Antal 511ffb5f70
YARN-9283. Javadoc of LinuxContainerExecutor#addSchedPriorityCommand has a wrong property name as reference 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9385ec45d7)
 2019-02-15 18:50:42 +09:00
Sunil G 8551831ac2 YARN-8295. [UI2] Improve Resource Usage tab error message when there are no data available. Contributed by Charan Hebri. 
(cherry picked from commit 5b55f3538c)
 2019-02-15 12:43:26 +05:30
Akira Ajisaka 3b10c07946
YARN-9284. Fix the unit of yarn.service.am-resource.memory in the document. Contributed by Masahiro Tanaka. 
(cherry picked from commit 3a39d9a2d2)
 2019-02-15 15:44:02 +09:00
bibinchundatt 8b074e6be8 YARN-9295. [UI2] Fix label typo in Cluster Overview page. Contributed by Charan Hebri. 
(cherry picked from commit b66d5ae9e2)
 2019-02-14 23:11:59 +05:30
Sunil G ec08eed542 YARN-7761. [UI2] Clicking 'master container log' or 'Link' next to 'log' under application's appAttempt goes to Old UI's Log link. Contributed by Akhil PB. 
(cherry picked from commit d321d0e747)
 2019-02-14 20:56:52 +05:30
Weiwei Yang f3c1e456b0 YARN-9253. Add UT to verify Placement Constraint in Distributed Shell. Contributed by Prabhu Joseph. 
(cherry picked from commit 711d22f166)
 2019-02-12 16:26:38 +08:00
Giovanni Matteo Fumarola c6582cc04c YARN-9191. Add cli option in DS to support enforceExecutionType in resource requests. Contributed by Abhishek Modi. 
(cherry picked from commit f738b397ae)
 2019-02-12 14:21:57 +08:00
Eric Yang 102db40870 YARN-8761. Service AM support for decommissioning component instances. 
Contributed by Billie Rinaldi

(cherry picked from commit 4c465f5535)
 2019-02-08 08:38:43 -08:00
Masatake Iwasaki 11ebdaab48 YARN-9282. Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'. Contributed by Charan Hebri. 
(cherry picked from commit e0ab1bdece)
 2019-02-09 00:29:58 +09:00
Sunil G 73956d5de9 YARN-9257. Distributed Shell client throws a NPE for a non-existent queue. Contributed by Charan Hebri. 
(cherry picked from commit fbc08145cf)
 2019-02-08 11:23:21 +05:30
Eric E Payne 834a862bd0 YARN-7171: RM UI should sort memory / cores numerically. Contributed by Ahmed Hussein 
(cherry picked from commit d1ca9432dd)
 2019-02-07 20:36:54 +00:00
Sunil G 6ffe6ea899 YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla. 2019-02-07 19:08:41 +05:30
Wangda Tan a1e09b4c0c Make upstream aware of 3.1.2 release 
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
(cherry picked from commit 308f3168fa)
(cherry picked from commit 649da5af04)
 2019-02-05 14:09:33 -08:00
Weiwei Yang 41bdcf4110 YARN-9262. TestRMAppAttemptTransitions is failing with an NPE. Contributed by lujie. 
(cherry picked from commit 28ad20a711)
 2019-02-04 14:00:54 +05:30
Sunil G 3b03ff6fdd YARN-9099. GpuResourceAllocator#getReleasingGpus calculates number of GPUs in a wrong way. Contributed by Szilard Nemeth. 
(cherry picked from commit 71c49fa60f)
 2019-01-31 09:26:38 +05:30
Eric E Payne 0cb05a9fe3 YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph 
(cherry picked from commit 04105bbfdb)
 2019-01-29 18:04:01 +00:00
Weiwei Yang 4257043232 YARN-9237. NM should ignore sending finished apps to RM during RM fail-over. Contributed by Jiandan Yang. 
(cherry picked from commit 4f63ffe444)
 2019-01-29 11:03:26 +08:00
Eric Yang 29ccb8689f YARN-8901. Fixed restart policy NEVER/ON_FAILURE with component dependency. 
Contributed by Suma Shivaprasad

(cherry picked from commit f5a95f7998)
 2019-01-28 18:12:39 -05:00
Rohith Sharma K S 6e059c7930 Revert "YARN-8270 Adding JMX Metrics for Timeline Collector and Reader. Contributed by Sushil Ks." 
This reverts commit 5b72aa04e1.
 2019-01-28 10:55:12 +05:30
Jonathan Hung 6092d913b1 YARN-9222. Print launchTime in ApplicationSummary 
(cherry picked from commit 6cace58e21)
(cherry picked from commit bf760e7e81)
 2019-01-25 13:50:44 -08:00
Haibo Chen 61a6cc8d23 YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 
(cherry picked from commit bb92bfb4ef)
 2019-01-24 15:58:37 -08:00
Sunil G 45c4df152f YARN-8961. [UI2] Flow Run End Time shows 'Invalid date'. Contributed by Akhil PB 
(cherry picked from commit c726445990)
 2019-01-24 15:02:53 +05:30
Weiwei Yang 2471d8a6e7 YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang. 
(cherry picked from commit bc6374f282)
 2019-01-23 18:18:38 +08:00
Weiwei Yang b61754b1bd YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang. 
(cherry picked from commit d43df31751)
 2019-01-22 11:01:00 +08:00
Weiwei Yang 4edd883d48 YARN-9204. RM fails to start if absolute resource is specified for partition capacity in CS queues. Contributed by Jiandan Yang. 
(cherry picked from commit abde1e1f58)
 2019-01-21 21:27:40 +08:00
Wangda Tan a685ffe9a9 YARN-9194. Invalid event: REGISTERED and LAUNCH_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM. (lujie via wangda) 
Change-Id: I4359f59a73a278a941f4bb9d106dd38c9cb471fe
(cherry picked from commit 6d7eedfd28)
(cherry picked from commit fe7cb2d84a)
 2019-01-17 15:17:34 -08:00
rahul3 8dea94d071
YARN-9203. Fix typos in yarn-default.xml. 
This closes #437

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 0a46baecd3)
 2019-01-17 16:20:15 +09:00
Akira Ajisaka d07d275f94
YARN-8747. [UI2] YARN UI2 page loading failed due to js error under some time zone configuration. Contributed by collinma. 
(cherry picked from commit 104ef5df36)
 2019-01-16 14:38:29 +09:00
Weiwei Yang 91e9c9f96e YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed Wilfred Spiegelenburg. 2019-01-08 13:56:21 +08:00
Wangda Tan 3570713a69 YARN-8822. Nvidia-docker v2 support for YARN GPU feature. (Charo Zhang via wangda) 
Change-Id: I416268888a7b6f097d218d84e8497dd70b4b6d8f
 2019-01-07 12:30:30 -08:00
Eric Yang a446949f56 HADOOP-16031. Fixed TestSecureLogins unit test. Contributed by Akira Ajisaka 
(cherry picked from commit bba76b6f31)
 2019-01-07 13:24:50 -05:00
Wangda Tan 31ea2f7806 Preparing for 3.1.3 development 
Change-Id: I3c3d3ee47dc4fef239127b4452ff14676fa26e3d
 2019-01-07 10:04:58 -08:00
Weiwei Yang d6464629ca YARN-9164. Shutdown NM may cause NPE when opportunistic container scheduling is enabled. Contributed by lujie. 
(cherry picked from commit cfe89e6f96)
 2019-01-04 01:37:47 +08:00
Akira Ajisaka 4026cc78a2
YARN-9154. Fix itemization in YARN service quickstart document. Contributed by Ayush Saxena. 
(cherry picked from commit e98e963e51)
 2018-12-22 13:27:01 +09:00
Eric Yang 956e6cfee6 YARN-9084. Reset container state and defer readiness check for upgrade. 
Contributed by Chandni Singh

(cherry picked from commit ccdd982e51)
 2018-12-18 18:06:04 -05:00
Eric Yang b4fa1830a8 YARN-9040. Fixed memory leak in LevelDBCacheTimelineStore and DBIterator. 
Contributed by Tarun Parimi

(cherry picked from commit 71e0b0d800)
 2018-12-17 12:08:35 -05:00
Akira Ajisaka 0ccfee31a5
YARN-9122. Add table of contents to YARN Service API document. Contributed by Zhankun Tang. 
(cherry picked from commit 26e55d4ceb)
 2018-12-15 10:31:01 +09:00
Eric Yang 690d760174 YARN-9125. Fixed Carriage Return detection in Docker container launch command. 
Contributed by Billie Rinaldi

(cherry picked from commit b2d7204ed0)
 2018-12-14 17:55:38 -05:00
Weiwei Yang 14ecdb62b6 YARN-9009. Fix flaky test TestEntityGroupFSTimelineStore.testCleanLogs. Contributed by OrDTesters. 
(cherry picked from commit 1c09a10e96)
 2018-12-10 12:17:00 +08:00
Jonathan Hung 7b523e6a77 YARN-9085. Add Guaranteed and MaxCapacity to CSQueueMetrics 
(cherry picked from commit 978ab3e958227220cb6f1a08ae6e7cdb8a46628b)
(cherry picked from commit dca69d178dba21c41fd1293187f29143f7e81e19)
 2018-12-07 10:45:57 -08:00
Eric Yang 7ef4ff1905 YARN-9071. Improved status update for reinitialized containers. 
Contributed by Chandni Singh

(cherry picked from commit 1b790f4dd1)
 2018-12-05 19:05:26 -05:00
Jonathan Hung 2cb9479bfc YARN-9036. Escape newlines in health report in YARN UI. Contributed by Keqiu Hu 2018-11-30 10:16:39 -08:00
Eric Yang 1b731de944 YARN-9067. Fixed Resource Manager resource leak via YARN service. 
Contributed by Eric Yang
 2018-11-29 13:51:39 -05:00
bibinchundatt 8be2d16b94 YARN-9069. Fix SchedulerInfo#getSchedulerType for custom schedulers. Contributed by Bilwa S T. 
(cherry picked from commit 07142f54a8)
 2018-11-29 22:08:35 +05:30
Jason Lowe d9457df989 YARN-8812. Containers fail during creating a symlink which started with hyphen for a resource file. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 3ce99e32f7)
 2018-11-28 08:54:04 -06:00
Eric Yang fc74a3f803 YARN-9056. Improved YARN service upgrade state logic for readiness check. 
Contributed by Chandni Singh
 2018-11-27 19:43:24 -05:00
Eric Yang bec5036397 YARN-8665. Added Yarn service cancel upgrade option. 
Contributed by Chandni Singh
 2018-11-27 16:29:08 -05:00
Eric Yang 463de48f04 YARN-8986. Added port publish for Docker container running with bridge. 
Contributed by Charo Zhang
 2018-11-27 14:28:37 -05:00
Eric Yang 7741106fd0 YARN-8488. Added SUCCEEDED/FAILED states to YARN service for branch-3.1. 
Contributed by Chandni Singh
 2018-11-27 13:36:33 -05:00
Eric Yang eb43772896 YARN-8298. Added express upgrade for YARN service. 
Contributed by Chandni Singh
 2018-11-26 18:55:28 -05:00
Weiwei Yang 17a41f5d86 YARN-8833. Avoid potential integer overflow when computing fair shares. Contributed by liyakun. 
(cherry picked from commit d027a24f03)
 2018-11-18 23:24:37 +08:00
Eric Yang fce0350289 YARN-8160. Support upgrade of service that use docker containers. 
Contributed by Chandni Singh
 2018-11-16 16:01:25 -05:00
Wangda Tan 077a7ab37b YARN-8299. Yarn Service Upgrade: Add GET APIs that returns instances matching query params. (Chandni Singh via wangda) 
Change-Id: I7f2aef89c6053d5dc5955f79fdfcd4cd74f74737
 2018-11-16 10:32:10 -08:00
Rohith Sharma K S 095635d984 YARN-8303. YarnClient should contact TimelineReader for application/attempt/container report. 
(cherry picked from commit ee3355be3c)
 2018-11-16 18:38:11 +05:30
Vrushali C 1dd2cf296d YARN-8834 Provide Java client for fetching Yarn specific entities from TimelineReader. Contributed by Abhishek Modi 
(cherry picked from commit a3edfddcf7)
 2018-11-16 18:38:11 +05:30
Eric Yang 5ec0a34151 YARN-9002. Improve keytab loading for YARN Service. 
Contributed by Gour Saha
 2018-11-10 01:56:15 -05:00
Akira Ajisaka daad077121
YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang. 2018-11-10 14:37:35 +09:00
Weiwei Yang a3b61baf94 YARN-8977. Remove unnecessary type casting when calling AbstractYarnScheduler#getSchedulerNode. Contributed by Wanqiang Ji. 
(cherry picked from commit c96cbe8659)
 2018-11-07 22:50:05 +08:00
Takanobu Asanuma eb426db206 YARN-8866. Fix a parsing error for crossdomain.xml. 
(cherry picked from commit 8dc1f6dbf7)
 2018-11-07 18:28:57 +09:00
Akira Ajisaka 52af95fdce
Revert "YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang." 
This reverts commit dd8479e80d.
 2018-11-07 11:33:31 +09:00
Akira Ajisaka dd8479e80d
YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang. 
(cherry picked from commit 951c98f890)
 2018-11-07 11:19:11 +09:00
Jason Lowe 7335d940de YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit ab6aa4c726)
 2018-11-06 08:52:29 -06:00
Weiwei Yang 631b31110c YARN-8970. Improve the debug message in CS#allocateContainerOnSingleNode. Contributed by Zhankun Tang. 
(cherry picked from commit 5d6554c722)
 2018-11-06 14:53:28 +08:00
Weiwei Yang 71999f4464 YARN-8969. AbstractYarnScheduler#getNodeTracker should return generic type to avoid type casting. Contributed by Wanqiang Ji. 
(cherry picked from commit c7fcca0d7e)
 2018-11-06 13:23:42 +08:00
Jonathan Hung 221494a75c YARN-7225. Add queue and partition info to RM audit log. Contributed by Eric Payne 
(cherry picked from commit 2ab611d48b)
 2018-11-01 14:31:22 -07:00
Rohith Sharma K S d9a494b1e0 YARN-8950. Fix compilation issue due to dependency convergence error for hbase.profile=2.0. 
(cherry picked from commit 4ec4ec6971)
 2018-10-30 11:50:56 +05:30
Weiwei Yang 7dd8eafe33 YARN-8915. Update the doc about the default value of maximum-container-assignments for capacity scheduler. Contributed by Zhankun Tang. 
(cherry picked from commit 78ea897b67)
 2018-10-29 12:06:52 +08:00
Weiwei Yang 70efe253f3 YARN-8944. TestContainerAllocation.testUserLimitAllocationMultipleContainers failure after YARN-8896. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 1d90a0dd23)
 2018-10-29 11:56:31 +08:00
Jason Lowe 3be72b7aa2 YARN-8904. TestRMDelegationTokens can fail in testRMDTMasterKeyStateOnRollingMasterKey. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 93fb3b4b9c)
 2018-10-23 12:55:48 -05:00
Rohith Sharma K S 3e3b088856 YARN-8826. Fix lingering timeline collector after serviceStop in TimelineCollectorManager. Contributed by Prabha Manepalli. 
(cherry picked from commit 0b62983c5a)
 2018-10-23 14:08:06 +05:30
Sunil G 0cb184d6e9 YARN-8868. Set HTTPOnly attribute to Cookie. Contributed by Chandni Singh. 
(cherry picked from commit 2202e00ba8)
 2018-10-23 09:58:05 +05:30
Sunil G 450f231a53 YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. 
(cherry picked from commit e50334513c)
 2018-10-22 14:16:54 +05:30
Eric Yang e86efa8712 YARN-8910. Fixed misleading log statement when container max retries is infinite. 
Contributed by Chandni Singh

(cherry picked from commit 47ad98b2e1)
 2018-10-19 13:50:32 -04:00
Wangda Tan c11d2341af YARN-8908. Fix errors in yarn-default.xml related to GPU/FPGA. (Zhankun Tang via wangda) 
Change-Id: I46f511a9b51b65b069e15d5b827530110c41edab
(cherry picked from commit 071b9e19aa)
 2018-10-19 09:55:27 -07:00
Weiwei Yang beca90ece8 YARN-8907. Fix incorrect logging message in TestCapacityScheduler. Contributed by Zhankun Tang. 
(cherry picked from commit 13cc0f50ea)
 2018-10-19 10:02:46 +08:00
Wangda Tan d381f32219 YARN-6098. Add documentation for Delete Queue. (Suma Shivaprasad via wangda) 
Change-Id: I23bf7ca0acbe1168a7f805ceccdd62fe41bfeb35
(cherry picked from commit bfb88b10f4)
 2018-10-18 12:29:28 -07:00
Wangda Tan 46baafedf1 YARN-8896. Limit the maximum number of container assignments per heartbeat. (Zhankun Tang via wangda) 
Change-Id: I6e72f8362bd7f5c2a844cb9e3c4732492314e9f1
(cherry picked from commit 780be14f07)
 2018-10-18 12:29:19 -07:00
Eric Yang 3cb52958b2 YARN-8687. Update YARN service file type in documentation. 
Contributed by Suma Shivaprasad
 2018-10-18 12:04:27 -04:00
Eric Yang 7c40f9b37c YARN-8810. Fixed a YARN service bug in comparing ConfigFile object. 
Contributed by Chandni Singh
 2018-10-16 18:58:28 -04:00
Wangda Tan 3e1894c376 YARN-8892. YARN UI2 doc changes to update security status (verified under security environment). (Sunil G via wangda) 
Change-Id: I8bc8622936861b8d6de3e42a0b75af86ad8a3961
(cherry picked from commit 538250db26)
 2018-10-16 14:36:36 -07:00
Sunil G 38e187384b YARN-8869. YARN Service Client might not work correctly with RM REST API for Kerberos authentication. Contributed by Eric Yang. 
(cherry picked from commit fa94d370b6)
 2018-10-15 21:23:07 +05:30
Sunil G 52d270ef8c YARN-8753. [UI2] Lost nodes representation missing from Nodemanagers Chart. Contributed by Yesha Vora. 
(cherry picked from commit 8a37983990)
 2018-10-11 16:16:26 +05:30
Sunil G fd9cd5f3c1 YARN-8666. [UI2] Remove application tab from YARN Queue Page. Contributed by Yesha Vora. 
(cherry picked from commit d3da741fa2)
 2018-10-11 16:07:30 +05:30
Weiwei Yang a0060cf8ee Revert "YARN-8468. Enable the use of queue based maximum container allocation limit and implement it in FairScheduler. Contributed by Antal Bálint Steinbach." 
This reverts commit ce4a0898df.
 2018-10-10 21:41:00 +08:00
Weiwei Yang 3968ce1073 YARN-8858. CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used. Contributed by Wangda Tan. 
(cherry picked from commit edce866489)
 2018-10-10 09:48:56 +08:00
Eric Yang 5813c1de56 YARN-8845. Removed unused hadoop.registry.rm reference. 
Contributed by Íñigo Goiri

(cherry picked from commit bf04f19456)
 2018-10-09 15:56:38 -04:00
Sunil G be0ce74c71 YARN-8853. [UI2] Application Attempts tab is not shown correctly when there are no attempts. Contributed by Akhil PB. 
(cherry picked from commit 6f11919beb)
 2018-10-09 22:06:21 +05:30
Weiwei Yang ce4a0898df YARN-8468. Enable the use of queue based maximum container allocation limit and implement it in FairScheduler. Contributed by Antal Bálint Steinbach. 2018-10-09 22:30:42 +08:00
Sunil G dd445e0368 YARN-7825. [UI2] Maintain constant horizontal application info bar for Application Attempt page. Contributed by Akhil PB. 
(cherry picked from commit 228dc192f1)
 2018-10-08 11:56:28 +05:30
Sunil G f3f0a9a2ca YARN-8797. [UI2] Improve error pages in new YARN UI. Contributed by Akhil PB. 
(cherry picked from commit 94d82f4162)
 2018-10-05 14:34:55 +05:30