Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,627 Commits 326 Branches 371 Tags
Commit Graph

2200 Commits

Author SHA1 Message Date
Junping Du
861e275646 YARN-6011. Add a new web service to list the files on a container in AHSWebService. Contributed by Xuan Gong. 
(cherry picked from commit cf695577aa6d9715b77ab7309ecd792ef226c439)
 2017-01-16 16:41:16 -08:00
Sunil G
f166bb8f09 YARN-6081. LeafQueue#getTotalPendingResourcesConsideringUserLimit should deduct reserved from pending to avoid unnecessary preemption of reserved container. Contributed by Wangda Tan. 
(cherry picked from commit d3170f9eba9bc5c38b5fa50c24e37ca2bd5636c2)
 2017-01-13 18:30:31 +05:30
Naganarasimha
4bd05840fc YARN-6072. RM unable to start in secure mode. Contributed by Ajith S. 
(cherry picked from commit a6b06f71797ad1ed9edbcef279bcf7d9e569f955)
 2017-01-12 06:42:18 +05:30
Daniel Templeton
718ae2d477 YARN-5849. Automatically create YARN control group for pre-mounted cgroups (Contributed by Miklos Szegedi via Daniel Templeton) 
(cherry picked from commit 0fffebe51a91023c4949ab560f5c828f1b568072)
 2017-01-11 15:36:18 -08:00
Jason Lowe
cc09b1a49a YARN-5416. TestRMRestart#testRMRestartWaitForPreviousAMToFinish failed intermittently due to not wait SchedulerApplicationAttempt to be stopped. Contributed by Junping Du 
(cherry picked from commit 357eab95668dbc419239857ac5ce763d76fd40e7)
 2017-01-11 17:15:14 +00:00
Junping Du
ba6a013341 YARN-6079. Fix simple spelling errors in yarn test code. Contributed by vijay. 
(cherry picked from commit 4db119b7b55213929e5b86f2abb0ed84a21719b5)
 2017-01-10 18:22:45 -08:00
Wangda Tan
c00a468b0b YARN-6022. Revert changes of AbstractResourceRequest. (Contributed by Wangda Tan / Daniel Templeton) 2017-01-10 10:17:33 -08:00
Naganarasimha
a7cdd39234 YARN-6054. TimelineServer fails to start when some LevelDb state files are missing. Contributed by Ravi Prakash. 
(cherry picked from commit 4c431a694059e40e78365b02a1497a6c7e479a70)
 2017-01-10 15:55:46 +05:30
Naganarasimha
ea38256fab YARN-6015. AsyncDispatcher thread name can be set to improved debugging. Contributed by Ajith S. 2017-01-10 15:10:54 +05:30
Sunil G
3190a4ba44 YARN-5899. Debug log in AbstractCSQueue#canAssignToThisQueue needs improvement. Contributed by Ying Zhang. 
(cherry picked from commit 9594c35dcb655add1991d8fd15897b40c4ad6205)
 2017-01-10 14:10:25 +05:30
Junping Du
b600577095 YARN-4148. When killing app, RM releases app's resource before they are released by NM. Contributed by Jason Lowe. 
(cherry picked from commit 945db55f2e6521d33d4f90bbb09179b0feba5e7a)
 2017-01-09 18:15:21 -08:00
Wangda Tan
447f71326b YARN-3955. Support for application priority ACLs in queues of CapacityScheduler. (Sunil G via wangda) 
(cherry picked from commit 287d3d6804a869723ae36605a3c2d2b3eae3941e)
 2017-01-09 08:42:38 -08:00
Varun Saxena
4fab250fc0 YARN-6068. Log aggregation get failed when NM restart even with recovery (Junping Du via Varun Saxena) 
(cherry picked from commit f59e36b4ce71d3019ab91b136b6d7646316954e7)
 2017-01-09 11:31:39 +05:30
Arun Suresh
4985217de4 YARN-6066. Opportunistic containers Minor fixes : API annotations, parameter name changes, checkstyles. (asuresh) 2017-01-08 08:27:10 -08:00
Junping Du
f615c1fbbc Addendum patch to YARN-3866: fix incompatible API change. Contributed by Wangda Tan. 2017-01-07 19:58:42 -08:00
Jian He
ba5d482366 YARN-6009. Skip validating app timeout value on recovery. Contributed by Rohith Sharma K S 
(cherry picked from commit 020316458dfe6059b700f8d93a9791e4cb817b3f)
 2017-01-06 18:27:36 -08:00
Arun Suresh
81da7d1d30 YARN-5938. Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes (asuresh) 
(cherry picked from commit ac1e5d4f77e3b9df8dcacb0b1f72eecc27931eb8)
 2017-01-06 11:25:49 -08:00
Arun Suresh
2b4d3e8506 YARN-5646. Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 2273a74c1f3895163046cca09ff5e983df301d22)
(cherry picked from commit 9e17ffe599e12c59c48f4355de645c31a4735d04)
 2017-01-06 11:15:10 -08:00
Arun Suresh
3d3bb30a25 YARN-5982. Simplify opportunistic container parameters and metrics. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit b0aace21b1ef3436ba9d516186208fee9a9ceef2)
(cherry picked from commit 3faa0b0d682687099b253feee9b41340edef7111)
 2017-01-06 11:15:09 -08:00
Arun Suresh
44774eb21c YARN-5918. Handle Opportunistic scheduling allocate request failure when NM is lost. (Bibin A Chundatt via asuresh) 
(cherry picked from commit 005850b28feb2f7bb8c2844d11e3f9d21b45d754)
(cherry picked from commit cbff10b4147f98a89b393519b17e16385294af07)
 2017-01-06 11:15:09 -08:00
Arun Suresh
368565f0ce YARN-4597. Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle. (asuresh) 
(cherry picked from commit 3219b7b4ac7d12aee343f6ab2980b3357fc618b6)
(cherry picked from commit 49921678858a17ca58ca2ae704ced1ff584e0859)
 2017-01-06 11:15:09 -08:00
Arun Suresh
047772f15f YARN-5823. Update NMTokens in case of requests with only opportunistic containers. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 283fa33febe043bd7b4fa87546be26c9c5a8f8b5)
(cherry picked from commit 1c4cc88a754ac9f557cdc8c859b8aadec19a5067)
 2017-01-06 11:15:09 -08:00
Arun Suresh
0e60c7cd33 YARN-5377. Fix TestQueuingContainerManager.testKillMultipleOpportunisticContainers. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit f38a6d03a11ca6de93a225563ddf55ec99d5063c)
(cherry picked from commit 83ccab8a653a4ef943276651c9604bd434675980)
 2017-01-06 11:15:09 -08:00
Arun Suresh
7437252102 YARN-2995. Enhance UI to show cluster resource utilization of various container Execution types. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 0aafc122d401a2f49d0f9ca556343f9ba871e979)
(cherry picked from commit 25598b6e7118bacd33a167fb496e9cda5e9f9ad0)
 2017-01-06 11:15:09 -08:00
Arun Suresh
28f3bb38bf YARN-5799. Fix Opportunistic Allocation to set the correct value of Node Http Address. (asuresh) 
(cherry picked from commit aa3cab1eb29c56368d15882d7260a994e615e8d8)
(cherry picked from commit ce10e98826884245e733431297d0275177965f17)
 2017-01-06 11:15:09 -08:00
Arun Suresh
f9ea753414 YARN-5486. Update OpportunisticContainerAllocatorAMService::allocate method to handle OPPORTUNISTIC container requests. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 10be45986cdf86a89055065b752959bd6369d54f)
(cherry picked from commit e3baa0988b4dfb592afe426ffeba3c7091b271c5)
 2017-01-06 11:15:09 -08:00
Sunil G
65e7ae5dcf YARN-5906. Update AppSchedulingInfo to use SchedulingPlacementSet. Contributed by Wangda Tan. 2017-01-06 21:30:52 +05:30
Rohith Sharma K S
3207762192 YARN-5988. RM unable to start in secure setup. Contributed by Ajith S. 
(cherry picked from commit e49e0a6e37f4a32535d7d4a07015fbf9eb33c74a)
 2017-01-04 12:10:05 +05:30
Junping Du
dc2e44e8f8 YARN-5923. Unable to access logs for a running application if YARN_ACL_ENABLE is enabled. Contributed by Xuan Gong. 
(cherry picked from commit 8fadd69047143c9c389cc09ca24100b5f90f79d2)
 2017-01-03 15:19:36 -08:00
Wangda Tan
23f6a33156 YARN-6025. Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda) 
(cherry picked from commit e0f2379312c48e26b0cb2c1e1e803ef71d1839cf)
(cherry picked from commit 554e6656bc9cf898d14d1fe2da5be5bff8fc4284)
 2017-01-03 14:52:31 -08:00
Wangda Tan
cb64e8eb19 Revert "Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda)" because forgot to add JIRA Number 
This reverts commit 554e6656bc9cf898d14d1fe2da5be5bff8fc4284.
 2017-01-03 14:51:59 -08:00
Wangda Tan
554e6656bc Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda) 
(cherry picked from commit e0f2379312c48e26b0cb2c1e1e803ef71d1839cf)
 2017-01-03 14:50:19 -08:00
Daniel Templeton
8cb7aa2b52 YARN-5931. Document timeout interfaces CLI and REST APIs (Contributed by Rohith Sharma K S via Daniel Templeton) 
(cherry picked from commit 165d01a73e5f328108a9e876c7a751a3c2cec4a8)
 2016-12-30 08:56:03 -08:00
Robert Kanter
f9b48206dd YARN-4882. Change the log level to DEBUG for recovering completed applications (templedf via rkanter) 
(cherry picked from commit f216276d2164c6564632c571fd3adbb03bc8b3e4)
 2016-12-28 15:29:06 -08:00
Daniel Templeton
47f0a0e15b HADOOP-13932. Fix indefinite article in comments (Contributed by LiXin Ge via Daniel Templeton) 2016-12-28 07:03:51 -08:00
Sunil G
b3231b480d YARN-6024. Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to max. Contributed by Wangda Tan. 
(cherry picked from commit f6715b26b65fc127b35368d0460b786ece88b5b8)
 2016-12-28 16:00:52 +05:30
Wangda Tan
42f571728c YARN-5756. Add state-machine implementation for scheduler queues. (Xuan Gong via wangda) 
(cherry picked from commit 0840b4329b2428b20b862f70d72cbdcd6d1618ed)
 2016-12-27 21:19:10 -08:00
Robert Kanter
88597991b0 YARN-5257. Fix unreleased resources and null dereferences (yufeigu via rkanter) 
(cherry picked from commit 9262797e86453fc04b7ca3710b73b21fcdf9e6b4)
 2016-12-27 14:14:41 -08:00
Robert Kanter
8fbd36c60f YARN-5962. Spelling errors in logging and exceptions for resource manager code (gsohn via rkanter) 
(cherry picked from commit 1bbd023275db535ab80fcb60e022151e9679d468)
 2016-12-27 13:35:55 -08:00
Karthik Kambatla
10a154cedd YARN-5969. FairShareComparator: Cache value of getResourceUsage for better performance. (zhangshilong via kasha) 
(cherry picked from commit c3973e7080bf71b57ace4a6adf4bb43f3c5d35b5)
 2016-12-27 12:30:50 -08:00
Akira Ajisaka
522bc98026 YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor. 
(cherry picked from commit ae401539eaf7745ec8690f9281726fb4cdcdbe94)
 2016-12-23 04:06:53 +09:00
Junping Du
cc42dbb703 YARN-4990. Re-direction of a particular log file within in a container in NM UI does not redirect properly. Contributed by Xuan Gong. 
(cherry picked from commit 736f54b727c3f0ecc8fb9a594f2281c240c89cb8)
 2016-12-21 14:16:19 -08:00
Sunil G
238863c11b YARN-5650. Render Application Timeout value in web UI. Contributed by Akhil PB. 2016-12-20 11:03:57 +05:30
Sunil G
23bd68a4a4 YARN-5970. Validate application update timeout request parameters. Contributed by Rohith Sharma K S. 2016-12-12 11:59:14 +05:30
Jian He
71fdca4600 YARN-5709. Cleanup leader election configs and pluggability. Contribtued by Karthik Kambatla 
(cherry picked from commit b817c565c8be1d4a682d119bfac6f43ee09e87f0)
 2016-12-09 16:59:52 -08:00
Robert Kanter
9cba7c6d71 YARN-5963. Spelling errors in logging and exceptions for node manager, client, web-proxy, common, and app history code (gsohn via rkanter) 
(cherry picked from commit 72fe54684198b7df5c5fb2114616dff6d17a4402)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/impl/TimelineClientImpl.java
 2016-12-07 15:09:37 -08:00
Sunil
a636a87270 YARN-5965. Retrospect ApplicationReport#getApplicationTimeouts. Contributed by Rohith Sharma K S 
(cherry picked from commit ab923a53fcf55d4d75aa027d46e3c4a659015325)
 2016-12-08 00:34:45 +05:30
Daniel Templeton
d8b4961515 YARN-5136. Error in handling event type APP_ATTEMPT_REMOVED to the scheduler 
(Contributed by Wilfred Spiegelenburg via Daniel Templeton)
 2016-12-07 11:01:09 -08:00
Rohith Sharma K S
602c998443 YARN-5932. Retrospect moveApplicationToQueue in align with YARN-5611. Contributed by Sunil G. 2016-12-07 22:45:25 +05:30
Naganarasimha
54c5880cf7 YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena 
(cherry picked from commit f3b8ff54ab08545d7093bf8861b44ec9912e8dc3)
 2016-12-06 11:01:14 +05:30