Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,936 Commits 326 Branches 371 Tags
Commit Graph

4979 Commits

Author SHA1 Message Date
Wangda Tan
a1e09b4c0c Make upstream aware of 3.1.2 release 
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
(cherry picked from commit 308f3168fa221a336ef0dbb1a17358b1ad6f9356)
(cherry picked from commit 649da5af04d57ee910dbe7fb0f4a7d9cb4c40eb0)
 2019-02-05 14:09:33 -08:00
Weiwei Yang
41bdcf4110 YARN-9262. TestRMAppAttemptTransitions is failing with an NPE. Contributed by lujie. 
(cherry picked from commit 28ad20a711735243bc10b10e33866dc525f415eb)
 2019-02-04 14:00:54 +05:30
Sunil G
3b03ff6fdd YARN-9099. GpuResourceAllocator#getReleasingGpus calculates number of GPUs in a wrong way. Contributed by Szilard Nemeth. 
(cherry picked from commit 71c49fa60faad2504b0411979a6e46e595b97a85)
 2019-01-31 09:26:38 +05:30
Eric E Payne
0cb05a9fe3 YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph 
(cherry picked from commit 04105bbfdb041a41062c856632641140de84fba8)
 2019-01-29 18:04:01 +00:00
Weiwei Yang
4257043232 YARN-9237. NM should ignore sending finished apps to RM during RM fail-over. Contributed by Jiandan Yang. 
(cherry picked from commit 4f63ffe444286dac91ed36fd647d8ce69e75b0f0)
 2019-01-29 11:03:26 +08:00
Eric Yang
29ccb8689f YARN-8901. Fixed restart policy NEVER/ON_FAILURE with component dependency. 
Contributed by Suma Shivaprasad

(cherry picked from commit f5a95f7998e110cab81e52acd99b07e13ea9653d)
 2019-01-28 18:12:39 -05:00
Rohith Sharma K S
6e059c7930 Revert "YARN-8270 Adding JMX Metrics for Timeline Collector and Reader. Contributed by Sushil Ks." 
This reverts commit 5b72aa04e104242d1761abf56822fb38e9915def.
 2019-01-28 10:55:12 +05:30
Jonathan Hung
6092d913b1 YARN-9222. Print launchTime in ApplicationSummary 
(cherry picked from commit 6cace58e212d3ee0aec988926a5a17c9cc58e645)
(cherry picked from commit bf760e7e81f8e02ad413c470fccf78aaa9cb9f86)
 2019-01-25 13:50:44 -08:00
Haibo Chen
61a6cc8d23 YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 
(cherry picked from commit bb92bfb4ef96baa234966b60e464d1773fbf3f22)
 2019-01-24 15:58:37 -08:00
Sunil G
45c4df152f YARN-8961. [UI2] Flow Run End Time shows 'Invalid date'. Contributed by Akhil PB 
(cherry picked from commit c7264459901e59021e180228cea3a6ae334a339f)
 2019-01-24 15:02:53 +05:30
Weiwei Yang
2471d8a6e7 YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang. 
(cherry picked from commit bc6374f282dbff3b9ed91fb5d7825d57e6720f5e)
 2019-01-23 18:18:38 +08:00
Weiwei Yang
b61754b1bd YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang. 
(cherry picked from commit d43df31751bcadab77d42b31e3e1dd5748b471b5)
 2019-01-22 11:01:00 +08:00
Weiwei Yang
4edd883d48 YARN-9204. RM fails to start if absolute resource is specified for partition capacity in CS queues. Contributed by Jiandan Yang. 
(cherry picked from commit abde1e1f58d5b699e4b8e460cff68e154738169b)
 2019-01-21 21:27:40 +08:00
Wangda Tan
a685ffe9a9 YARN-9194. Invalid event: REGISTERED and LAUNCH_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM. (lujie via wangda) 
Change-Id: I4359f59a73a278a941f4bb9d106dd38c9cb471fe
(cherry picked from commit 6d7eedfd28cc1712690db2f6ca8a281b0901ee28)
(cherry picked from commit fe7cb2d84ac160c5fed00640d85e2c5c4c6d2412)
 2019-01-17 15:17:34 -08:00
rahul3
8dea94d071
YARN-9203. Fix typos in yarn-default.xml. 
This closes #437

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 0a46baecd31c485d1ea4e567c29c47bfba0b092e)
 2019-01-17 16:20:15 +09:00
Akira Ajisaka
d07d275f94
YARN-8747. [UI2] YARN UI2 page loading failed due to js error under some time zone configuration. Contributed by collinma. 
(cherry picked from commit 104ef5df36fd45f1c2d07a2f3d441263aa85e22e)
 2019-01-16 14:38:29 +09:00
Weiwei Yang
91e9c9f96e YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed Wilfred Spiegelenburg. 2019-01-08 13:56:21 +08:00
Wangda Tan
3570713a69 YARN-8822. Nvidia-docker v2 support for YARN GPU feature. (Charo Zhang via wangda) 
Change-Id: I416268888a7b6f097d218d84e8497dd70b4b6d8f
 2019-01-07 12:30:30 -08:00
Eric Yang
a446949f56 HADOOP-16031. Fixed TestSecureLogins unit test. Contributed by Akira Ajisaka 
(cherry picked from commit bba76b6f31d5452fe048dcc6853c12a1894024fe)
 2019-01-07 13:24:50 -05:00
Wangda Tan
31ea2f7806 Preparing for 3.1.3 development 
Change-Id: I3c3d3ee47dc4fef239127b4452ff14676fa26e3d
 2019-01-07 10:04:58 -08:00
Weiwei Yang
d6464629ca YARN-9164. Shutdown NM may cause NPE when opportunistic container scheduling is enabled. Contributed by lujie. 
(cherry picked from commit cfe89e6f963ba25b5fff1ce48cad36d74b3c789c)
 2019-01-04 01:37:47 +08:00
Akira Ajisaka
4026cc78a2
YARN-9154. Fix itemization in YARN service quickstart document. Contributed by Ayush Saxena. 
(cherry picked from commit e98e963e5143600017d2d227d504140b8802439d)
 2018-12-22 13:27:01 +09:00
Eric Yang
956e6cfee6 YARN-9084. Reset container state and defer readiness check for upgrade. 
Contributed by Chandni Singh

(cherry picked from commit ccdd982e51f4413bc2e98d03e0a05b1133042a31)
 2018-12-18 18:06:04 -05:00
Eric Yang
b4fa1830a8 YARN-9040. Fixed memory leak in LevelDBCacheTimelineStore and DBIterator. 
Contributed by Tarun Parimi

(cherry picked from commit 71e0b0d8005ea1952dc7e582db15c2ac09df7c91)
 2018-12-17 12:08:35 -05:00
Akira Ajisaka
0ccfee31a5
YARN-9122. Add table of contents to YARN Service API document. Contributed by Zhankun Tang. 
(cherry picked from commit 26e55d4ceb691f47801fa5f0ba1147a5bc413f96)
 2018-12-15 10:31:01 +09:00
Eric Yang
690d760174 YARN-9125. Fixed Carriage Return detection in Docker container launch command. 
Contributed by Billie Rinaldi

(cherry picked from commit b2d7204ed0b64405a0adbe07e3eaf385e046efa1)
 2018-12-14 17:55:38 -05:00
Weiwei Yang
14ecdb62b6 YARN-9009. Fix flaky test TestEntityGroupFSTimelineStore.testCleanLogs. Contributed by OrDTesters. 
(cherry picked from commit 1c09a10e9601bd628f1e887f3bf92c5a4ac286ab)
 2018-12-10 12:17:00 +08:00
Jonathan Hung
7b523e6a77 YARN-9085. Add Guaranteed and MaxCapacity to CSQueueMetrics 
(cherry picked from commit 978ab3e958227220cb6f1a08ae6e7cdb8a46628b)
(cherry picked from commit dca69d178dba21c41fd1293187f29143f7e81e19)
 2018-12-07 10:45:57 -08:00
Eric Yang
7ef4ff1905 YARN-9071. Improved status update for reinitialized containers. 
Contributed by Chandni Singh

(cherry picked from commit 1b790f4dd1f682423d5dbb8e70c6225cbddce989)
 2018-12-05 19:05:26 -05:00
Jonathan Hung
2cb9479bfc YARN-9036. Escape newlines in health report in YARN UI. Contributed by Keqiu Hu 2018-11-30 10:16:39 -08:00
Eric Yang
1b731de944 YARN-9067. Fixed Resource Manager resource leak via YARN service. 
Contributed by Eric Yang
 2018-11-29 13:51:39 -05:00
bibinchundatt
8be2d16b94 YARN-9069. Fix SchedulerInfo#getSchedulerType for custom schedulers. Contributed by Bilwa S T. 
(cherry picked from commit 07142f54a8c7f70857e99c041f3a2a5189c809b5)
 2018-11-29 22:08:35 +05:30
Jason Lowe
d9457df989 YARN-8812. Containers fail during creating a symlink which started with hyphen for a resource file. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 3ce99e32f7d7887412cae8337cd4ebeb3b2ee308)
 2018-11-28 08:54:04 -06:00
Eric Yang
fc74a3f803 YARN-9056. Improved YARN service upgrade state logic for readiness check. 
Contributed by Chandni Singh
 2018-11-27 19:43:24 -05:00
Eric Yang
bec5036397 YARN-8665. Added Yarn service cancel upgrade option. 
Contributed by Chandni Singh
 2018-11-27 16:29:08 -05:00
Eric Yang
463de48f04 YARN-8986. Added port publish for Docker container running with bridge. 
Contributed by Charo Zhang
 2018-11-27 14:28:37 -05:00
Eric Yang
7741106fd0 YARN-8488. Added SUCCEEDED/FAILED states to YARN service for branch-3.1. 
Contributed by Chandni Singh
 2018-11-27 13:36:33 -05:00
Eric Yang
eb43772896 YARN-8298. Added express upgrade for YARN service. 
Contributed by Chandni Singh
 2018-11-26 18:55:28 -05:00
Weiwei Yang
17a41f5d86 YARN-8833. Avoid potential integer overflow when computing fair shares. Contributed by liyakun. 
(cherry picked from commit d027a24f0349b60efa5125c330058f123771748f)
 2018-11-18 23:24:37 +08:00
Eric Yang
fce0350289 YARN-8160. Support upgrade of service that use docker containers. 
Contributed by Chandni Singh
 2018-11-16 16:01:25 -05:00
Wangda Tan
077a7ab37b YARN-8299. Yarn Service Upgrade: Add GET APIs that returns instances matching query params. (Chandni Singh via wangda) 
Change-Id: I7f2aef89c6053d5dc5955f79fdfcd4cd74f74737
 2018-11-16 10:32:10 -08:00
Rohith Sharma K S
095635d984 YARN-8303. YarnClient should contact TimelineReader for application/attempt/container report. 
(cherry picked from commit ee3355be3cd1cd956296a1b79b264177ff139ce6)
 2018-11-16 18:38:11 +05:30
Vrushali C
1dd2cf296d YARN-8834 Provide Java client for fetching Yarn specific entities from TimelineReader. Contributed by Abhishek Modi 
(cherry picked from commit a3edfddcf7822ea13bdf4858672eb82cea5e0b5f)
 2018-11-16 18:38:11 +05:30
Eric Yang
5ec0a34151 YARN-9002. Improve keytab loading for YARN Service. 
Contributed by Gour Saha
 2018-11-10 01:56:15 -05:00
Akira Ajisaka
daad077121
YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang. 2018-11-10 14:37:35 +09:00
Weiwei Yang
a3b61baf94 YARN-8977. Remove unnecessary type casting when calling AbstractYarnScheduler#getSchedulerNode. Contributed by Wanqiang Ji. 
(cherry picked from commit c96cbe8659587cfc114a96aab1be5cc85029fe44)
 2018-11-07 22:50:05 +08:00
Takanobu Asanuma
eb426db206 YARN-8866. Fix a parsing error for crossdomain.xml. 
(cherry picked from commit 8dc1f6dbf712a65390a9a6859f62fec0481af31b)
 2018-11-07 18:28:57 +09:00
Akira Ajisaka
52af95fdce
Revert "YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang." 
This reverts commit dd8479e80d3f0fe87a6edb099e7f617bff42106a.
 2018-11-07 11:33:31 +09:00
Akira Ajisaka
dd8479e80d
YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang. 
(cherry picked from commit 951c98f89059d64fda8456366f680eff4a7a6785)
 2018-11-07 11:19:11 +09:00
Jason Lowe
7335d940de YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit ab6aa4c7265db5bcbb446c2f779289023d454b81)
 2018-11-06 08:52:29 -06:00