Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,380 Commits 326 Branches 371 Tags
Commit Graph

5135 Commits

Author SHA1 Message Date
Sunil G
9bfea98746 YARN-9090. [Submarine] Update submarine installation script document. Contributed by Xun Liu. 
(cherry picked from commit 742fd978b5b585fc14fc3fea7ec37f80c5d7dc97)
 2018-12-26 08:27:50 +05:30
Akira Ajisaka
d6baf9d5bf
YARN-9154. Fix itemization in YARN service quickstart document. Contributed by Ayush Saxena. 
(cherry picked from commit e98e963e5143600017d2d227d504140b8802439d)
 2018-12-22 13:26:41 +09:00
Weiwei Yang
7deef08eb8 YARN-8925. Updating distributed node attributes only when necessary. Contributed by Tao Yang. 2018-12-21 16:31:03 +08:00
Eric Yang
29c9c8a893 YARN-9126. Fix container clean up for reinitialization. 
Contributed by Chandni Singh

(cherry picked from commit e815fd9c49e80b9200dd8852abe74fe219ad9110)
 2018-12-19 14:58:19 -05:00
Eric Yang
4e1cef3625 YARN-9084. Reset container state and defer readiness check for upgrade. 
Contributed by Chandni Singh

(cherry picked from commit ccdd982e51f4413bc2e98d03e0a05b1133042a31)
 2018-12-18 18:05:40 -05:00
Eric Yang
28ca14e71b YARN-9040. Fixed memory leak in LevelDBCacheTimelineStore and DBIterator. 
Contributed by Tarun Parimi

(cherry picked from commit 71e0b0d8005ea1952dc7e582db15c2ac09df7c91)
 2018-12-17 12:08:09 -05:00
Akira Ajisaka
71d8ffb551
YARN-9122. Add table of contents to YARN Service API document. Contributed by Zhankun Tang. 
(cherry picked from commit 26e55d4ceb691f47801fa5f0ba1147a5bc413f96)
 2018-12-15 10:30:37 +09:00
Eric Yang
52aafb9789 YARN-9125. Fixed Carriage Return detection in Docker container launch command. 
Contributed by Billie Rinaldi

(cherry picked from commit b2d7204ed0b64405a0adbe07e3eaf385e046efa1)
 2018-12-14 17:55:10 -05:00
Wangda Tan
29e4e5f62c YARN-9112. [Submarine] Support polling applicationId when it's not ready in cluster. (Zhankun Tang via wangda) 
Change-Id: I73d73f3d631b28fb9866faa56571839b13824a97
(cherry picked from commit 9fba6cc2471db8df9ca6da7b9df378e3072dfeb2)
 2018-12-12 11:52:16 -08:00
Wangda Tan
ef0f835537 YARN-8714. [Submarine] Support files/tarballs to be localized for a training job. (Zhankun Tang via wangda) 
Change-Id: I845131273e52a9d81dbc813ea6d4af06b205e334
(cherry picked from commit c771fe6e10bd3f8533a01026de3c33db2a983cf4)
 2018-12-12 11:52:07 -08:00
Weiwei Yang
2b3c3d2a32 YARN-9009. Fix flaky test TestEntityGroupFSTimelineStore.testCleanLogs. Contributed by OrDTesters. 
(cherry picked from commit 1c09a10e9601bd628f1e887f3bf92c5a4ac286ab)
 2018-12-10 12:07:23 +08:00
Jonathan Hung
3ab6ea7aca YARN-9085. Add Guaranteed and MaxCapacity to CSQueueMetrics 
(cherry picked from commit 978ab3e958227220cb6f1a08ae6e7cdb8a46628b)
 2018-12-07 10:45:47 -08:00
Eric Yang
8c70728f7f YARN-9071. Improved status update for reinitialized containers. 
Contributed by Chandni Singh

(cherry picked from commit 1b790f4dd1f682423d5dbb8e70c6225cbddce989)
 2018-12-05 19:04:55 -05:00
Wangda Tan
559cb99bf6 Revert "YARN-8870. [Submarine] Add submarine installation scripts. (Xun Liu via wangda)" 
This reverts commit 920401b2966c2dcd7cd1b3dfd7a764f08412323f.
 2018-12-04 14:13:39 -08:00
Jonathan Hung
6b01e4d2a8 YARN-9036. Escape newlines in health report in YARN UI. Contributed by Keqiu Hu 
(cherry picked from commit 1c8bd7128c99d8215ef16438bd2ce6b1f025a966)
 2018-11-30 10:16:00 -08:00
Eric Yang
a8f67ad7c9 YARN-9067. Fixed Resource Manager resource leak via YARN service. 
Contributed by Eric Yang
 2018-11-29 13:50:06 -05:00
bibinchundatt
183ec39c4b YARN-9069. Fix SchedulerInfo#getSchedulerType for custom schedulers. Contributed by Bilwa S T. 
(cherry picked from commit 07142f54a8c7f70857e99c041f3a2a5189c809b5)
 2018-11-29 22:16:32 +05:30
Wangda Tan
ee9deb6e9c YARN-9030. Log aggregation changes to handle filesystems which do not support setting permissions. (Suma Shivaprasad via wangda) 
Change-Id: I80f1e8196b8624e24d74494719fdedfd7061dced
(cherry picked from commit 9de8e8d0496a2628b63cc841b1fdee80e2912f7a)
 2018-11-28 15:06:35 -08:00
Jason Lowe
df0e7766e4 YARN-8812. Containers fail during creating a symlink which started with hyphen for a resource file. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 3ce99e32f7d7887412cae8337cd4ebeb3b2ee308)
 2018-11-28 08:50:18 -06:00
Eric Yang
db8b2a130c YARN-9056. Improved YARN service upgrade state logic for readiness check. 
Contributed by Chandni Singh

(cherry picked from commit f657a2a661ed7470915fa33ea4bbda2f4980c8b2)
 2018-11-27 19:45:23 -05:00
Eric Yang
838190482d YARN-8986. Added port publish for Docker container running with bridge. 
Contributed by Charo Zhang
 2018-11-27 14:27:13 -05:00
Weiwei Yang
01477108d9 YARN-8984. AMRMClient#OutstandingSchedRequests leaks when AllocationTags is null or empty. Contributed by Yang Wang. 
(cherry picked from commit 176bb3f812e49b0fe3abddf54eebfc7219b5d718)
 2018-11-22 17:04:16 +08:00
Wangda Tan
34387599c4 YARN-8960. [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment. (Zac Zhou via wangda) 
Change-Id: I21b1addc9c32817650ea744a8f2e6b5602f2f4d4
(cherry picked from commit 8b2381441558cd49b4c940b0760c8accbb2a5567)
 2018-11-19 09:01:21 -08:00
Weiwei Yang
650581a19d YARN-8833. Avoid potential integer overflow when computing fair shares. Contributed by liyakun. 
(cherry picked from commit d027a24f0349b60efa5125c330058f123771748f)
 2018-11-18 23:23:09 +08:00
Wangda Tan
bee5bf867b YARN-9001. [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs. (Zac Zhou via wangda) 
Change-Id: I7e8d1c27ebd37e0907ca570c4f3d56fe7a859635
 2018-11-16 10:26:50 -08:00
Rohith Sharma K S
13e3670e7f YARN-8303. YarnClient should contact TimelineReader for application/attempt/container report. 
(cherry picked from commit ee3355be3cd1cd956296a1b79b264177ff139ce6)
 2018-11-16 18:37:20 +05:30
Vrushali C
da05328824 YARN-8834 Provide Java client for fetching Yarn specific entities from TimelineReader. Contributed by Abhishek Modi 
(cherry picked from commit a3edfddcf7822ea13bdf4858672eb82cea5e0b5f)
 2018-11-16 18:37:01 +05:30
Wangda Tan
ea753046b0 YARN-8918. [Submarine] Correct method usage of str.subString in CliUtils. (Zhankun Tang via wangda) 
Change-Id: Id1f11dbab3aa838dee3a0ec8b8fd5dc32f5dd946
(cherry picked from commit 076b795b2e82d7f89cc91e0a8513c7081ee8b930)
 2018-11-13 11:35:06 -08:00
Weiwei Yang
2415f8a5be YARN-8987. Usability improvements node-attributes CLI. Contributed by Bibin A Chundatt. 
(cherry picked from commit c741109522d2913b87638957c64b94dee6b51029)
 2018-11-12 18:21:11 +08:00
Eric Yang
5399ff8177 YARN-9002. Improve keytab loading for YARN Service. 
Contributed by Gour Saha

(cherry picked from commit 2664248797365761089a86d5bd59aa9ac3ebcc28)
 2018-11-10 01:54:59 -05:00
Weiwei Yang
b3321d9933 YARN-8988. Reduce the verbose log on RM heartbeat path when distributed node-attributes is enabled. Contributed by Tao Yang. 
(cherry picked from commit e1bbf7dcdfc30a61a2b10bef09c59ff17d290488)
 2018-11-08 17:50:16 +08:00
Weiwei Yang
b10ec0aa14 YARN-8977. Remove unnecessary type casting when calling AbstractYarnScheduler#getSchedulerNode. Contributed by Wanqiang Ji. 
(cherry picked from commit c96cbe8659587cfc114a96aab1be5cc85029fe44)
 2018-11-07 22:44:01 +08:00
Takanobu Asanuma
02abd0f80a YARN-8866. Fix a parsing error for crossdomain.xml. 
(cherry picked from commit 8dc1f6dbf712a65390a9a6859f62fec0481af31b)
 2018-11-07 18:29:28 +09:00
Weiwei Yang
4ff7255d6e YARN-8976. Remove redundant modifiers in interface ApplicationConstants. Contributed by Zhankun Tang. 
(cherry picked from commit 482716e5a4d1edfd3aa6a1ae65a58652f89375f1)
 2018-11-07 10:49:09 +08:00
Akira Ajisaka
41c4e9583d
YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang. 
(cherry picked from commit 951c98f89059d64fda8456366f680eff4a7a6785)
 2018-11-07 11:18:48 +09:00
Jason Lowe
9265934201 YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit ab6aa4c7265db5bcbb446c2f779289023d454b81)
 2018-11-06 08:47:30 -06:00
Weiwei Yang
f00125e2d9 YARN-8970. Improve the debug message in CS#allocateContainerOnSingleNode. Contributed by Zhankun Tang. 
(cherry picked from commit 5d6554c722f08f79bce904e021243605ee75bae3)
 2018-11-06 14:52:10 +08:00
Weiwei Yang
b474239e0b YARN-8969. AbstractYarnScheduler#getNodeTracker should return generic type to avoid type casting. Contributed by Wanqiang Ji. 
(cherry picked from commit c7fcca0d7ec9e31d43ef3040ecd576ec808f1f8b)
 2018-11-06 13:17:19 +08:00
Jonathan Hung
b9a3c988c0 YARN-7225. Add queue and partition info to RM audit log. Contributed by Eric Payne 
(cherry picked from commit 2ab611d48b7669b31bd2c9b918f47251da77d0f6)
 2018-11-01 14:31:09 -07:00
Rohith Sharma K S
7920f79049 YARN-8950. Fix compilation issue due to dependency convergence error for hbase.profile=2.0. 
(cherry picked from commit 4ec4ec69711180d642c5b56cd3d3dbdf44d3c61f)
 2018-10-30 11:50:33 +05:30
Weiwei Yang
288bc3f1eb YARN-8915. Update the doc about the default value of maximum-container-assignments for capacity scheduler. Contributed by Zhankun Tang. 
(cherry picked from commit 78ea897b6701c82799a6231103ca7ee1f8b4efde)
 2018-10-29 12:05:45 +08:00
Weiwei Yang
5db4272d91 YARN-8944. TestContainerAllocation.testUserLimitAllocationMultipleContainers failure after YARN-8896. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 1d90a0dd23c5d1bd52d04c303506806d0dc61cd2)
 2018-10-29 11:55:22 +08:00
Eric Yang
3665c46176 YARN-8938. Updated YARN service upgrade document. 
Contributed by Chandni Singh

(cherry picked from commit bbc6dcd3d0976932a49d8650804fb0a4018b3a02)
 2018-10-24 11:52:50 -04:00
Jason Lowe
7097755925 YARN-8904. TestRMDelegationTokens can fail in testRMDTMasterKeyStateOnRollingMasterKey. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 93fb3b4b9cbc63e65d590676a5a318d555a25904)
 2018-10-23 12:54:04 -05:00
Rohith Sharma K S
660fff3138 YARN-8826. Fix lingering timeline collector after serviceStop in TimelineCollectorManager. Contributed by Prabha Manepalli. 
(cherry picked from commit 0b62983c5a9361eb832784f134f140f9926c9ec6)
 2018-10-23 14:06:13 +05:30
Sunil G
30998fea28 YARN-8868. Set HTTPOnly attribute to Cookie. Contributed by Chandni Singh. 
(cherry picked from commit 2202e00ba8a44ad70f0a90e6c519257e3ae56a36)
 2018-10-23 09:57:26 +05:30
Sunil G
8724c2900f YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. 
(cherry picked from commit e50334513c781a10368e7353bec6674f89eb0a7b)
 2018-10-22 14:16:24 +05:30
Eric Yang
04600cf5da YARN-8910. Fixed misleading log statement when container max retries is infinite. 
Contributed by Chandni Singh

(cherry picked from commit 47ad98b2e1b4e9ad9304aca4d452da428778e7ee)
 2018-10-19 13:49:51 -04:00
Wangda Tan
9ed9e185d7 YARN-8916. Define a constant docker string in ContainerRuntimeConstants.java for better maintainability. (Zhankun Tang via wangda) 
Change-Id: I1349e740037f81afdbe30edbe741f20e88fd0a90
(cherry picked from commit 5e02b4915b876eab943b30cf728cfc26ba832188)
 2018-10-19 09:50:18 -07:00
Wangda Tan
bd08554aa4 YARN-8908. Fix errors in yarn-default.xml related to GPU/FPGA. (Zhankun Tang via wangda) 
Change-Id: I46f511a9b51b65b069e15d5b827530110c41edab
(cherry picked from commit 071b9e19aa2c17bb9fa046c6ddabd6b8f0cc6d14)
 2018-10-19 09:50:12 -07:00