Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,005 Commits 326 Branches 371 Tags
Commit Graph

5137 Commits

Author SHA1 Message Date
Jonathan Hung
bf760e7e81 YARN-9222. Print launchTime in ApplicationSummary 
(cherry picked from commit 6cace58e212d3ee0aec988926a5a17c9cc58e645)
 2019-01-25 13:23:37 -08:00
Sunil G
4aea4b7c9c YARN-8961. [UI2] Flow Run End Time shows 'Invalid date'. Contributed by Akhil PB 
(cherry picked from commit c7264459901e59021e180228cea3a6ae334a339f)
 2019-01-24 15:02:21 +05:30
Weiwei Yang
bc6374f282 YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang. 2019-01-23 18:10:28 +08:00
Weiwei Yang
8ad7711605 YARN-8101. Add UT to verify node-attributes in RM nodes rest API. Contributed by Prabhu Joseph. 
(cherry picked from commit 721d5c2a5f43b29c9cc4439980f21854d9e8ae27)
 2019-01-23 18:07:45 +08:00
Weiwei Yang
9114489566 YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang. 
(cherry picked from commit d43df31751bcadab77d42b31e3e1dd5748b471b5)
 2019-01-22 10:46:37 +08:00
Weiwei Yang
ac2f4b64f9 YARN-9204. RM fails to start if absolute resource is specified for partition capacity in CS queues. Contributed by Jiandan Yang. 
(cherry picked from commit abde1e1f58d5b699e4b8e460cff68e154738169b)
 2019-01-21 21:20:01 +08:00
Wangda Tan
fe7cb2d84a YARN-9194. Invalid event: REGISTERED and LAUNCH_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM. (lujie via wangda) 
Change-Id: I4359f59a73a278a941f4bb9d106dd38c9cb471fe
(cherry picked from commit 6d7eedfd28cc1712690db2f6ca8a281b0901ee28)
 2019-01-17 15:13:42 -08:00
rahul3
4aee7946dc
YARN-9203. Fix typos in yarn-default.xml. 
This closes #437

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 0a46baecd31c485d1ea4e567c29c47bfba0b092e)
 2019-01-17 16:19:49 +09:00
Akira Ajisaka
a415446705
YARN-8747. [UI2] YARN UI2 page loading failed due to js error under some time zone configuration. Contributed by collinma. 
(cherry picked from commit 104ef5df36fd45f1c2d07a2f3d441263aa85e22e)
 2019-01-16 14:37:43 +09:00
Wangda Tan
1dc2b49bfd YARN-8822. Nvidia-docker v2 support for YARN GPU feature. (Charo Zhang via wangda) 
Change-Id: Ib8044307a4241f6b1b7b9b8266b9256f39b16384
 2019-01-07 12:21:33 -08:00
Eric Yang
bba76b6f31 HADOOP-16031. Fixed TestSecureLogins unit test. Contributed by Akira Ajisaka 2019-01-07 13:24:01 -05:00
Weiwei Yang
2b549e32e1 YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 944cf872232d1a8899df0f0bbe90aa29f225cb90)
 2019-01-07 16:05:57 +08:00
Wangda Tan
cffe5c1ba0 YARN-9160. [Submarine] Document 'PYTHONPATH' environment variable setting when using -localization options. (Zhankun Tang via wangda) 
Change-Id: I115b6710eb11501e21ab3e9e7cd1a87f322e5cf8
(cherry picked from commit 2c02aa6ec259128934cc5468cf66104a624d88a7)
 2019-01-06 11:10:47 -08:00
Wangda Tan
1ad1ab5d7a YARN-9141. [Submarine] JobStatus outputs with system UTC clock, not local clock. (Zac Zhou via wangda) 
Change-Id: Ib2b17cf7860ddc9188f06498086de3326bc09519
(cherry picked from commit 8f004fe4e0d479e26d0063427566edc1606bced5)
 2019-01-06 11:10:41 -08:00
Weiwei Yang
a24cca11f2 YARN-9164. Shutdown NM may cause NPE when opportunistic container scheduling is enabled. Contributed by lujie. 
(cherry picked from commit cfe89e6f963ba25b5fff1ce48cad36d74b3c789c)
 2019-01-04 01:04:39 +08:00
Sunil G
9bfea98746 YARN-9090. [Submarine] Update submarine installation script document. Contributed by Xun Liu. 
(cherry picked from commit 742fd978b5b585fc14fc3fea7ec37f80c5d7dc97)
 2018-12-26 08:27:50 +05:30
Akira Ajisaka
d6baf9d5bf
YARN-9154. Fix itemization in YARN service quickstart document. Contributed by Ayush Saxena. 
(cherry picked from commit e98e963e5143600017d2d227d504140b8802439d)
 2018-12-22 13:26:41 +09:00
Weiwei Yang
7deef08eb8 YARN-8925. Updating distributed node attributes only when necessary. Contributed by Tao Yang. 2018-12-21 16:31:03 +08:00
Eric Yang
29c9c8a893 YARN-9126. Fix container clean up for reinitialization. 
Contributed by Chandni Singh

(cherry picked from commit e815fd9c49e80b9200dd8852abe74fe219ad9110)
 2018-12-19 14:58:19 -05:00
Eric Yang
4e1cef3625 YARN-9084. Reset container state and defer readiness check for upgrade. 
Contributed by Chandni Singh

(cherry picked from commit ccdd982e51f4413bc2e98d03e0a05b1133042a31)
 2018-12-18 18:05:40 -05:00
Eric Yang
28ca14e71b YARN-9040. Fixed memory leak in LevelDBCacheTimelineStore and DBIterator. 
Contributed by Tarun Parimi

(cherry picked from commit 71e0b0d8005ea1952dc7e582db15c2ac09df7c91)
 2018-12-17 12:08:09 -05:00
Akira Ajisaka
71d8ffb551
YARN-9122. Add table of contents to YARN Service API document. Contributed by Zhankun Tang. 
(cherry picked from commit 26e55d4ceb691f47801fa5f0ba1147a5bc413f96)
 2018-12-15 10:30:37 +09:00
Eric Yang
52aafb9789 YARN-9125. Fixed Carriage Return detection in Docker container launch command. 
Contributed by Billie Rinaldi

(cherry picked from commit b2d7204ed0b64405a0adbe07e3eaf385e046efa1)
 2018-12-14 17:55:10 -05:00
Wangda Tan
29e4e5f62c YARN-9112. [Submarine] Support polling applicationId when it's not ready in cluster. (Zhankun Tang via wangda) 
Change-Id: I73d73f3d631b28fb9866faa56571839b13824a97
(cherry picked from commit 9fba6cc2471db8df9ca6da7b9df378e3072dfeb2)
 2018-12-12 11:52:16 -08:00
Wangda Tan
ef0f835537 YARN-8714. [Submarine] Support files/tarballs to be localized for a training job. (Zhankun Tang via wangda) 
Change-Id: I845131273e52a9d81dbc813ea6d4af06b205e334
(cherry picked from commit c771fe6e10bd3f8533a01026de3c33db2a983cf4)
 2018-12-12 11:52:07 -08:00
Weiwei Yang
2b3c3d2a32 YARN-9009. Fix flaky test TestEntityGroupFSTimelineStore.testCleanLogs. Contributed by OrDTesters. 
(cherry picked from commit 1c09a10e9601bd628f1e887f3bf92c5a4ac286ab)
 2018-12-10 12:07:23 +08:00
Jonathan Hung
3ab6ea7aca YARN-9085. Add Guaranteed and MaxCapacity to CSQueueMetrics 
(cherry picked from commit 978ab3e958227220cb6f1a08ae6e7cdb8a46628b)
 2018-12-07 10:45:47 -08:00
Eric Yang
8c70728f7f YARN-9071. Improved status update for reinitialized containers. 
Contributed by Chandni Singh

(cherry picked from commit 1b790f4dd1f682423d5dbb8e70c6225cbddce989)
 2018-12-05 19:04:55 -05:00
Wangda Tan
559cb99bf6 Revert "YARN-8870. [Submarine] Add submarine installation scripts. (Xun Liu via wangda)" 
This reverts commit 920401b2966c2dcd7cd1b3dfd7a764f08412323f.
 2018-12-04 14:13:39 -08:00
Jonathan Hung
6b01e4d2a8 YARN-9036. Escape newlines in health report in YARN UI. Contributed by Keqiu Hu 
(cherry picked from commit 1c8bd7128c99d8215ef16438bd2ce6b1f025a966)
 2018-11-30 10:16:00 -08:00
Eric Yang
a8f67ad7c9 YARN-9067. Fixed Resource Manager resource leak via YARN service. 
Contributed by Eric Yang
 2018-11-29 13:50:06 -05:00
bibinchundatt
183ec39c4b YARN-9069. Fix SchedulerInfo#getSchedulerType for custom schedulers. Contributed by Bilwa S T. 
(cherry picked from commit 07142f54a8c7f70857e99c041f3a2a5189c809b5)
 2018-11-29 22:16:32 +05:30
Wangda Tan
ee9deb6e9c YARN-9030. Log aggregation changes to handle filesystems which do not support setting permissions. (Suma Shivaprasad via wangda) 
Change-Id: I80f1e8196b8624e24d74494719fdedfd7061dced
(cherry picked from commit 9de8e8d0496a2628b63cc841b1fdee80e2912f7a)
 2018-11-28 15:06:35 -08:00
Jason Lowe
df0e7766e4 YARN-8812. Containers fail during creating a symlink which started with hyphen for a resource file. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 3ce99e32f7d7887412cae8337cd4ebeb3b2ee308)
 2018-11-28 08:50:18 -06:00
Eric Yang
db8b2a130c YARN-9056. Improved YARN service upgrade state logic for readiness check. 
Contributed by Chandni Singh

(cherry picked from commit f657a2a661ed7470915fa33ea4bbda2f4980c8b2)
 2018-11-27 19:45:23 -05:00
Eric Yang
838190482d YARN-8986. Added port publish for Docker container running with bridge. 
Contributed by Charo Zhang
 2018-11-27 14:27:13 -05:00
Weiwei Yang
01477108d9 YARN-8984. AMRMClient#OutstandingSchedRequests leaks when AllocationTags is null or empty. Contributed by Yang Wang. 
(cherry picked from commit 176bb3f812e49b0fe3abddf54eebfc7219b5d718)
 2018-11-22 17:04:16 +08:00
Wangda Tan
34387599c4 YARN-8960. [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment. (Zac Zhou via wangda) 
Change-Id: I21b1addc9c32817650ea744a8f2e6b5602f2f4d4
(cherry picked from commit 8b2381441558cd49b4c940b0760c8accbb2a5567)
 2018-11-19 09:01:21 -08:00
Weiwei Yang
650581a19d YARN-8833. Avoid potential integer overflow when computing fair shares. Contributed by liyakun. 
(cherry picked from commit d027a24f0349b60efa5125c330058f123771748f)
 2018-11-18 23:23:09 +08:00
Wangda Tan
bee5bf867b YARN-9001. [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs. (Zac Zhou via wangda) 
Change-Id: I7e8d1c27ebd37e0907ca570c4f3d56fe7a859635
 2018-11-16 10:26:50 -08:00
Rohith Sharma K S
13e3670e7f YARN-8303. YarnClient should contact TimelineReader for application/attempt/container report. 
(cherry picked from commit ee3355be3cd1cd956296a1b79b264177ff139ce6)
 2018-11-16 18:37:20 +05:30
Vrushali C
da05328824 YARN-8834 Provide Java client for fetching Yarn specific entities from TimelineReader. Contributed by Abhishek Modi 
(cherry picked from commit a3edfddcf7822ea13bdf4858672eb82cea5e0b5f)
 2018-11-16 18:37:01 +05:30
Wangda Tan
ea753046b0 YARN-8918. [Submarine] Correct method usage of str.subString in CliUtils. (Zhankun Tang via wangda) 
Change-Id: Id1f11dbab3aa838dee3a0ec8b8fd5dc32f5dd946
(cherry picked from commit 076b795b2e82d7f89cc91e0a8513c7081ee8b930)
 2018-11-13 11:35:06 -08:00
Weiwei Yang
2415f8a5be YARN-8987. Usability improvements node-attributes CLI. Contributed by Bibin A Chundatt. 
(cherry picked from commit c741109522d2913b87638957c64b94dee6b51029)
 2018-11-12 18:21:11 +08:00
Eric Yang
5399ff8177 YARN-9002. Improve keytab loading for YARN Service. 
Contributed by Gour Saha

(cherry picked from commit 2664248797365761089a86d5bd59aa9ac3ebcc28)
 2018-11-10 01:54:59 -05:00
Weiwei Yang
b3321d9933 YARN-8988. Reduce the verbose log on RM heartbeat path when distributed node-attributes is enabled. Contributed by Tao Yang. 
(cherry picked from commit e1bbf7dcdfc30a61a2b10bef09c59ff17d290488)
 2018-11-08 17:50:16 +08:00
Weiwei Yang
b10ec0aa14 YARN-8977. Remove unnecessary type casting when calling AbstractYarnScheduler#getSchedulerNode. Contributed by Wanqiang Ji. 
(cherry picked from commit c96cbe8659587cfc114a96aab1be5cc85029fe44)
 2018-11-07 22:44:01 +08:00
Takanobu Asanuma
02abd0f80a YARN-8866. Fix a parsing error for crossdomain.xml. 
(cherry picked from commit 8dc1f6dbf712a65390a9a6859f62fec0481af31b)
 2018-11-07 18:29:28 +09:00
Weiwei Yang
4ff7255d6e YARN-8976. Remove redundant modifiers in interface ApplicationConstants. Contributed by Zhankun Tang. 
(cherry picked from commit 482716e5a4d1edfd3aa6a1ae65a58652f89375f1)
 2018-11-07 10:49:09 +08:00
Akira Ajisaka
41c4e9583d
YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang. 
(cherry picked from commit 951c98f89059d64fda8456366f680eff4a7a6785)
 2018-11-07 11:18:48 +09:00