Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,434 Commits 326 Branches 371 Tags
Commit Graph

3545 Commits

Author SHA1 Message Date
Sunil G
99876a5ab8 YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla. 
(cherry picked from commit 604b2489a9e168a0fd702343f6f8844df9e86d17)
 2019-02-04 12:49:06 +05:30
Weiwei Yang
a0fafbc3ef YARN-9263. TestConfigurationNodeAttributesProvider fails after Mockito updated. Contributed by Weiwei Yang. 
(cherry picked from commit f20b043a02bd5b20f68123b2a7ab86f7cb9fe32a)
 2019-02-04 12:45:40 +05:30
Sunil G
0e7060a1d5 YARN-9099. GpuResourceAllocator#getReleasingGpus calculates number of GPUs in a wrong way. Contributed by Szilard Nemeth. 
(cherry picked from commit 71c49fa60faad2504b0411979a6e46e595b97a85)
 2019-01-31 09:26:07 +05:30
Eric E Payne
4052b7ee60 YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph 
(cherry picked from commit 04105bbfdb041a41062c856632641140de84fba8)
 2019-01-29 17:52:54 +00:00
Weiwei Yang
6b8dd8d113 YARN-9237. NM should ignore sending finished apps to RM during RM fail-over. Contributed by Jiandan Yang. 
(cherry picked from commit 4f63ffe444286dac91ed36fd647d8ce69e75b0f0)
 2019-01-29 10:42:09 +08:00
Jonathan Hung
bf760e7e81 YARN-9222. Print launchTime in ApplicationSummary 
(cherry picked from commit 6cace58e212d3ee0aec988926a5a17c9cc58e645)
 2019-01-25 13:23:37 -08:00
Weiwei Yang
bc6374f282 YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang. 2019-01-23 18:10:28 +08:00
Weiwei Yang
8ad7711605 YARN-8101. Add UT to verify node-attributes in RM nodes rest API. Contributed by Prabhu Joseph. 
(cherry picked from commit 721d5c2a5f43b29c9cc4439980f21854d9e8ae27)
 2019-01-23 18:07:45 +08:00
Weiwei Yang
9114489566 YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang. 
(cherry picked from commit d43df31751bcadab77d42b31e3e1dd5748b471b5)
 2019-01-22 10:46:37 +08:00
Weiwei Yang
ac2f4b64f9 YARN-9204. RM fails to start if absolute resource is specified for partition capacity in CS queues. Contributed by Jiandan Yang. 
(cherry picked from commit abde1e1f58d5b699e4b8e460cff68e154738169b)
 2019-01-21 21:20:01 +08:00
Wangda Tan
fe7cb2d84a YARN-9194. Invalid event: REGISTERED and LAUNCH_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM. (lujie via wangda) 
Change-Id: I4359f59a73a278a941f4bb9d106dd38c9cb471fe
(cherry picked from commit 6d7eedfd28cc1712690db2f6ca8a281b0901ee28)
 2019-01-17 15:13:42 -08:00
Wangda Tan
1dc2b49bfd YARN-8822. Nvidia-docker v2 support for YARN GPU feature. (Charo Zhang via wangda) 
Change-Id: Ib8044307a4241f6b1b7b9b8266b9256f39b16384
 2019-01-07 12:21:33 -08:00
Weiwei Yang
2b549e32e1 YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 944cf872232d1a8899df0f0bbe90aa29f225cb90)
 2019-01-07 16:05:57 +08:00
Weiwei Yang
a24cca11f2 YARN-9164. Shutdown NM may cause NPE when opportunistic container scheduling is enabled. Contributed by lujie. 
(cherry picked from commit cfe89e6f963ba25b5fff1ce48cad36d74b3c789c)
 2019-01-04 01:04:39 +08:00
Weiwei Yang
7deef08eb8 YARN-8925. Updating distributed node attributes only when necessary. Contributed by Tao Yang. 2018-12-21 16:31:03 +08:00
Eric Yang
29c9c8a893 YARN-9126. Fix container clean up for reinitialization. 
Contributed by Chandni Singh

(cherry picked from commit e815fd9c49e80b9200dd8852abe74fe219ad9110)
 2018-12-19 14:58:19 -05:00
Eric Yang
28ca14e71b YARN-9040. Fixed memory leak in LevelDBCacheTimelineStore and DBIterator. 
Contributed by Tarun Parimi

(cherry picked from commit 71e0b0d8005ea1952dc7e582db15c2ac09df7c91)
 2018-12-17 12:08:09 -05:00
Eric Yang
52aafb9789 YARN-9125. Fixed Carriage Return detection in Docker container launch command. 
Contributed by Billie Rinaldi

(cherry picked from commit b2d7204ed0b64405a0adbe07e3eaf385e046efa1)
 2018-12-14 17:55:10 -05:00
Weiwei Yang
2b3c3d2a32 YARN-9009. Fix flaky test TestEntityGroupFSTimelineStore.testCleanLogs. Contributed by OrDTesters. 
(cherry picked from commit 1c09a10e9601bd628f1e887f3bf92c5a4ac286ab)
 2018-12-10 12:07:23 +08:00
Jonathan Hung
3ab6ea7aca YARN-9085. Add Guaranteed and MaxCapacity to CSQueueMetrics 
(cherry picked from commit 978ab3e958227220cb6f1a08ae6e7cdb8a46628b)
 2018-12-07 10:45:47 -08:00
Eric Yang
8c70728f7f YARN-9071. Improved status update for reinitialized containers. 
Contributed by Chandni Singh

(cherry picked from commit 1b790f4dd1f682423d5dbb8e70c6225cbddce989)
 2018-12-05 19:04:55 -05:00
Jonathan Hung
6b01e4d2a8 YARN-9036. Escape newlines in health report in YARN UI. Contributed by Keqiu Hu 
(cherry picked from commit 1c8bd7128c99d8215ef16438bd2ce6b1f025a966)
 2018-11-30 10:16:00 -08:00
bibinchundatt
183ec39c4b YARN-9069. Fix SchedulerInfo#getSchedulerType for custom schedulers. Contributed by Bilwa S T. 
(cherry picked from commit 07142f54a8c7f70857e99c041f3a2a5189c809b5)
 2018-11-29 22:16:32 +05:30
Jason Lowe
df0e7766e4 YARN-8812. Containers fail during creating a symlink which started with hyphen for a resource file. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 3ce99e32f7d7887412cae8337cd4ebeb3b2ee308)
 2018-11-28 08:50:18 -06:00
Eric Yang
838190482d YARN-8986. Added port publish for Docker container running with bridge. 
Contributed by Charo Zhang
 2018-11-27 14:27:13 -05:00
Weiwei Yang
650581a19d YARN-8833. Avoid potential integer overflow when computing fair shares. Contributed by liyakun. 
(cherry picked from commit d027a24f0349b60efa5125c330058f123771748f)
 2018-11-18 23:23:09 +08:00
Rohith Sharma K S
13e3670e7f YARN-8303. YarnClient should contact TimelineReader for application/attempt/container report. 
(cherry picked from commit ee3355be3cd1cd956296a1b79b264177ff139ce6)
 2018-11-16 18:37:20 +05:30
Weiwei Yang
2415f8a5be YARN-8987. Usability improvements node-attributes CLI. Contributed by Bibin A Chundatt. 
(cherry picked from commit c741109522d2913b87638957c64b94dee6b51029)
 2018-11-12 18:21:11 +08:00
Weiwei Yang
b3321d9933 YARN-8988. Reduce the verbose log on RM heartbeat path when distributed node-attributes is enabled. Contributed by Tao Yang. 
(cherry picked from commit e1bbf7dcdfc30a61a2b10bef09c59ff17d290488)
 2018-11-08 17:50:16 +08:00
Weiwei Yang
b10ec0aa14 YARN-8977. Remove unnecessary type casting when calling AbstractYarnScheduler#getSchedulerNode. Contributed by Wanqiang Ji. 
(cherry picked from commit c96cbe8659587cfc114a96aab1be5cc85029fe44)
 2018-11-07 22:44:01 +08:00
Akira Ajisaka
41c4e9583d
YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang. 
(cherry picked from commit 951c98f89059d64fda8456366f680eff4a7a6785)
 2018-11-07 11:18:48 +09:00
Jason Lowe
9265934201 YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit ab6aa4c7265db5bcbb446c2f779289023d454b81)
 2018-11-06 08:47:30 -06:00
Weiwei Yang
f00125e2d9 YARN-8970. Improve the debug message in CS#allocateContainerOnSingleNode. Contributed by Zhankun Tang. 
(cherry picked from commit 5d6554c722f08f79bce904e021243605ee75bae3)
 2018-11-06 14:52:10 +08:00
Weiwei Yang
b474239e0b YARN-8969. AbstractYarnScheduler#getNodeTracker should return generic type to avoid type casting. Contributed by Wanqiang Ji. 
(cherry picked from commit c7fcca0d7ec9e31d43ef3040ecd576ec808f1f8b)
 2018-11-06 13:17:19 +08:00
Jonathan Hung
b9a3c988c0 YARN-7225. Add queue and partition info to RM audit log. Contributed by Eric Payne 
(cherry picked from commit 2ab611d48b7669b31bd2c9b918f47251da77d0f6)
 2018-11-01 14:31:09 -07:00
Rohith Sharma K S
7920f79049 YARN-8950. Fix compilation issue due to dependency convergence error for hbase.profile=2.0. 
(cherry picked from commit 4ec4ec69711180d642c5b56cd3d3dbdf44d3c61f)
 2018-10-30 11:50:33 +05:30
Weiwei Yang
5db4272d91 YARN-8944. TestContainerAllocation.testUserLimitAllocationMultipleContainers failure after YARN-8896. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 1d90a0dd23c5d1bd52d04c303506806d0dc61cd2)
 2018-10-29 11:55:22 +08:00
Jason Lowe
7097755925 YARN-8904. TestRMDelegationTokens can fail in testRMDTMasterKeyStateOnRollingMasterKey. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 93fb3b4b9cbc63e65d590676a5a318d555a25904)
 2018-10-23 12:54:04 -05:00
Rohith Sharma K S
660fff3138 YARN-8826. Fix lingering timeline collector after serviceStop in TimelineCollectorManager. Contributed by Prabha Manepalli. 
(cherry picked from commit 0b62983c5a9361eb832784f134f140f9926c9ec6)
 2018-10-23 14:06:13 +05:30
Sunil G
30998fea28 YARN-8868. Set HTTPOnly attribute to Cookie. Contributed by Chandni Singh. 
(cherry picked from commit 2202e00ba8a44ad70f0a90e6c519257e3ae56a36)
 2018-10-23 09:57:26 +05:30
Eric Yang
04600cf5da YARN-8910. Fixed misleading log statement when container max retries is infinite. 
Contributed by Chandni Singh

(cherry picked from commit 47ad98b2e1b4e9ad9304aca4d452da428778e7ee)
 2018-10-19 13:49:51 -04:00
Wangda Tan
9ed9e185d7 YARN-8916. Define a constant docker string in ContainerRuntimeConstants.java for better maintainability. (Zhankun Tang via wangda) 
Change-Id: I1349e740037f81afdbe30edbe741f20e88fd0a90
(cherry picked from commit 5e02b4915b876eab943b30cf728cfc26ba832188)
 2018-10-19 09:50:18 -07:00
Weiwei Yang
042f2df19b YARN-8907. Fix incorrect logging message in TestCapacityScheduler. Contributed by Zhankun Tang. 
(cherry picked from commit 13cc0f50ea1c5f8978455e34f49716ddb0e3a143)
 2018-10-19 10:01:45 +08:00
Wangda Tan
7252f8e117 YARN-8896. Limit the maximum number of container assignments per heartbeat. (Zhankun Tang via wangda) 
Change-Id: I6e72f8362bd7f5c2a844cb9e3c4732492314e9f1
(cherry picked from commit 780be14f07df2a3ed6273b96ae857c278fd72718)
 2018-10-18 12:12:19 -07:00
Wangda Tan
5f7ed043c8 YARN-8456. Fix a configuration handling bug when user leave FPGA discover executable path configuration default but set OpenCL SDK path environment variable. (Zhankun Tang via wangda) 
Change-Id: Iff150ea98ba0c60d448474fd940eb121afce6965
(cherry picked from commit a457a8951a1b35f06811c40443ca44bb9c698c30)
 2018-10-18 12:12:14 -07:00
Sunil G
bde4fd5ed9 Preparing for 3.2.0 release 2018-10-18 17:07:45 +05:30
Sunil G
6380ee5512 YARN-8759. Copy of resource-types.xml is not deleted if test fails, causes other test failures. Contributed by Antal Bálint Steinbach. 
(cherry picked from commit 5085e5fa9e8c7e489a8518e8541c12b14f3651df)
 2018-10-17 16:06:07 +05:30
Vrushali C
62d329cac0 YARN-5742 Serve aggregated logs of historical apps from timeline service. Contributed by Rohith Sharma KS 
(cherry picked from commit 8d1981806feb8278966c02a9eff42d72541bb35e)
 2018-10-12 07:18:13 +05:30
Jason Lowe
cdbca8b133 YARN-8861. executorLock is misleading in ContainerLaunch. Contributed by Chandni Singh 
(cherry picked from commit e787d65a08f5d5245d2313fc34f2dde518bfaa5b)
 2018-10-11 10:58:48 -05:00
Jason Lowe
145c7aa663 YARN-7644. NM gets backed up deleting docker containers. Contributed by Chandni Singh 
(cherry picked from commit 5ce70e1211e624d58e8bb1181aec00729ebdc085)
 2018-10-10 10:01:52 -05:00