Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,281 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1418 Commits

Author SHA1 Message Date
Wangda Tan 0a502c665b YARN-3272. Surface container locality info in RM web UI (Jian He via wangda) 
(cherry picked from commit e17e5ba9d7)
 2015-03-03 11:49:45 -08:00
Vinod Kumar Vavilapalli 253c78548b YARN-3265. Fixed a deadlock in CapacityScheduler by always passing a queue's available resource-limit from the parent queue. Contributed by Wangda Tan. 
(cherry picked from commit 14dd647c55)
 2015-03-02 17:54:01 -08:00
Jian He 76b7283fa1 YARN-3281. Added RMStateStore to StateMachine visualization list. Contributed by Chengbing Liu 
(cherry picked from commit 5d0bae550f)
 2015-03-02 14:43:12 -08:00
Wangda Tan 0b0be0056b YARN-3262. Surface application outstanding resource requests table in RM web UI. (Jian He via wangda) 
(cherry picked from commit edcecedc1c)
 2015-02-27 16:14:35 -08:00
Tsuyoshi Ozawa 79f73f4613 YARN-2820. Retry in FileSystemRMStateStore when FS's operations fail due to IOException. Contributed by Zhihai Xu. 
(cherry picked from commit 01a1621930)
 2015-02-28 00:57:01 +09:00
Konstantin V Shvachko 9e67f2cb0d YARN-3255. RM, NM, JobHistoryServer, and WebAppProxyServer's main() should support generic options. Contributed by Konstantin Shvachko. 2015-02-26 17:26:08 -08:00
Tsuyoshi Ozawa dc348f4894 YARN-3217. Remove httpclient dependency from hadoop-yarn-server-web-proxy. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 773b6515ac)
 2015-02-27 00:25:12 +09:00
Devaraj K f83d2e4410 YARN-3256. TestClientToAMTokens#testClientTokenRace is not running against 
all Schedulers even when using ParameterizedSchedulerTestBase. Contributed
by Anubhav Dhoot.

(cherry picked from commit 0d4296f0e0)
 2015-02-26 15:47:32 +05:30
Jason Lowe 257087417e YARN-3239. WebAppProxy does not support a final tracking url which has query fragments and params. Contributed by Jian He 
(cherry picked from commit 1a68fc4346)
 2015-02-25 16:15:35 +00:00
Tsuyoshi Ozawa 13e213c41a YARN-3247. TestQueueMappings should use CapacityScheduler explicitly. Contributed by Zhihai Xu. 
(cherry picked from commit 6cbd9f1113)
 2015-02-25 11:23:34 +09:00
Xuan 0ca974d0b0 YARN-2797. TestWorkPreservingRMRestart should use 
ParametrizedSchedulerTestBase. Contributed by Karthik Kambatla

(cherry picked from commit fe7a302473)
 2015-02-21 19:19:08 -08:00
Xuan f8435c5d3d YARN-3236. Cleanup RMAuthenticationFilter#AUTH_HANDLER_PROPERTY. 
Contributed by zhihai xu

(cherry picked from commit e3d290244c)
 2015-02-21 16:20:33 -08:00
Xuan 9cbe9bcbaa YARN-3237. AppLogAggregatorImpl fails to log error cause. Contributed by 
Rushabh S Shah

(cherry picked from commit f56c65bb3e)
 2015-02-20 14:03:46 -08:00
Wangda Tan a1963968d2 YARN-3230. Clarify application states on the web UI. (Jian He via wangda) 
(cherry picked from commit ce5bf927c3)
 2015-02-20 10:40:04 -08:00
Junping Du 4a87a61fe9 YARN-2799. Cleanup TestLogAggregationService based on the change in YARN-90. Contributed by Zhihai Xu 
(cherry picked from commit c33ae271c2)
 2015-02-20 09:44:31 -08:00
Jason Lowe 3e8dfd1299 YARN-3194. RM should handle NMContainerStatuses sent by NM while registering if NM is Reconnected node. Contributed by Rohith 
(cherry picked from commit a64dd3d24b)
 2015-02-20 15:12:08 +00:00
Jian He eaeaf80d39 YARN-933. Fixed InvalidStateTransitonException at FINAL_SAVING state in RMApp. Contributed by Rohith Sharmaks 
(cherry picked from commit c0d9b93953)
 2015-02-19 15:43:14 -08:00
Wangda Tan cd5eb9c1de YARN-3076. Add API/Implementation to YarnClient to retrieve label-to-node mapping (Varun Saxena via wangda) 
(cherry picked from commit d49ae725d5)
 2015-02-19 11:01:47 -08:00
Tsuyoshi Ozawa fef5efa0cb YARN-1615. Fix typos in delay scheduler's description. Contributed by Akira Ajisaka. 
(cherry picked from commit b8a14efdf5)
 2015-02-19 10:24:40 +09:00
Jian He b8fe1a747e YARN-1514. Utility to benchmark ZKRMStateStore#loadState for RM HA. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 1c03376300)
 2015-02-18 16:34:42 -08:00
Jian He 8f06eaa251 YARN-3132. RMNodeLabelsManager should remove node from node-to-label mapping when node becomes deactivated. Contributed by Wangda Tan 
(cherry picked from commit f5da5566d9)
 2015-02-18 11:52:28 -08:00
Xuan ba18adbb27 YARN-3207. Secondary filter matches entites which do not have the key 
being filtered for. Contributed by Zhijie Shen

(cherry picked from commit 57db50cbe3)
 2015-02-17 18:18:56 -08:00
Tsuyoshi Ozawa fabac2ff30 YARN-1299. Improve a log message in AppSchedulingInfo by adding application id. Contributed by Ashutosh Jindal and Devaraj K. 
(cherry picked from commit 556386a070)
 2015-02-17 01:54:09 +09:00
Tsuyoshi Ozawa 42368b91c4 Revert "YARN-1299. Improve a log message in AppSchedulingInfo by adding application id. Contributed by Ashutosh Jindal and devaraj." 
This reverts commit 67e3a77720.
 2015-02-17 01:53:58 +09:00
Tsuyoshi Ozawa 67e3a77720 YARN-1299. Improve a log message in AppSchedulingInfo by adding application id. Contributed by Ashutosh Jindal and devaraj. 
(cherry picked from commit 9aae81c934)
 2015-02-17 01:48:53 +09:00
Tsuyoshi Ozawa fd8f7e2c7d YARN-3203. Correct a log message in AuxServices. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 447bd7b5a6)
 2015-02-16 23:56:32 +09:00
Junping Du eee89bd18c YARN-2749. Fix some testcases from TestLogAggregationService fails in trunk. (Contributed by Xuan Gong) 
(cherry picked from commit ab0b958a52)
 2015-02-15 06:48:29 -08:00
cnauroth 48302e687a YARN-2899. Run TestDockerContainerExecutorWithMocks on Linux only. Contributed by Ming Ma. 
(cherry picked from commit 6804d68901)
 2015-02-13 21:59:14 -08:00
Jason Lowe c9cd58d711 YARN-2847. Linux native container executor segfaults if default banned user detected. Contributed by Olaf Flebbe 
(cherry picked from commit 1a0f508b63)
 2015-02-13 20:22:12 +00:00
Tsuyoshi Ozawa 613065d75a YARN-3182. Cleanup switch statement in ApplicationMasterLauncher#handle(). Contributed by Ray Chiang. 
(cherry picked from commit 110cf6b9a3)
 2015-02-13 14:21:43 +09:00
Akira Ajisaka 9d8dbf8ea4 YARN-3191. Log object should be initialized with its own class. Contributed by Rohith. 
(cherry picked from commit 99c7f2b8fa417303e8f42ea3641c302135879280)
 2015-02-12 17:59:04 -08:00
Xuan 5f0495689f YARN-3158. Correct log messages in ResourceTrackerService. Contributed 
by Varun Saxena

(cherry picked from commit 99f6bd4f7a)
 2015-02-12 16:43:28 -08:00
Jian He b5d6f76c6a YARN-3104. Fixed RM to not generate new AMRM tokens on every heartbeat between rolling and activation. Contributed by Jason Lowe 
(cherry picked from commit 18297e0972)
 2015-02-12 16:03:12 -08:00
Karthik Kambatla fcfbdf6440 YARN-3181. FairScheduler: Fix up outdated findbugs issues. (kasha) 
(cherry picked from commit c2b185def8)
 2015-02-12 15:39:50 -08:00
Jian He 1be2d64ddd YARN-3124. Fixed CS LeafQueue/ParentQueue to use QueueCapacities to track capacities-by-label. Contributed by Wangda Tan 
(cherry picked from commit 18a594257e)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-02-12 15:01:14 -08:00
Junping Du 380cc4dbed YARN-2079. Recover NonAggregatingLogHandler state upon nodemanager restart. (Contributed by Jason Lowe) 
(cherry picked from commit 04f5ef18f7)
 2015-02-12 11:48:24 -08:00
Xuan 58d9ce2ea1 YARN-3147. Clean up RM web proxy code. Contributed by Steve Loughran 
(cherry picked from commit 83be450acc)
 2015-02-12 10:59:18 -08:00
Xuan cac6fe217a YARN-3151. On Failover tracking url wrong in application cli for KILLED 
application. Contributed by Rohith

(cherry picked from commit 65c69e296e)
 2015-02-11 21:21:06 -08:00
Jason Lowe 38333c8f29 YARN-3074. Nodemanager dies when localizer runner tries to write to a full disk. Contributed by Varun Saxena 
(cherry picked from commit b379972ab3)
 2015-02-11 16:34:42 +00:00
Junping Du f2c3058b8d YARN-3160. Fix non-atomic operation on nodeUpdateQueue in RMNodeImpl. (Contributed by Chengbing Liu) 
(cherry picked from commit c541a374d8)
 2015-02-11 03:46:40 -08:00
Zhijie Shen fd75b8c9ca YARN-2246. Made the proxy tracking URL always be http(s)://proxy addr:port/proxy/<appId> to avoid duplicate sections. Contributed by Devaraj K. 
(cherry picked from commit d5855c0e46)
 2015-02-10 15:29:09 -08:00
Jason Lowe ca11ffa5de YARN-2809. Implement workaround for linux kernel panic when removing cgroup. Contributed by Nathan Roberts 
(cherry picked from commit 3f5431a22f)
 2015-02-10 17:28:18 +00:00
Jason Lowe 9ffd985dc0 YARN-3090. DeletionService can silently ignore deletion task failures. Contributed by Varun Saxena 
(cherry picked from commit 4eb5f7fa32)
 2015-02-10 16:55:21 +00:00
Zhijie Shen 24d4d933f7 YARN-3100. Made YARN authorization pluggable. Contributed by Jian He. 
(cherry picked from commit 23bf6c7207)
 2015-02-09 20:37:58 -08:00
Jian He 6146680955 YARN-3094. Reset timer for liveness monitors after RM recovery. Contributed by Jun Gong 
(cherry picked from commit 0af6a99a3f)
 2015-02-09 13:47:33 -08:00
Karthik Kambatla 1cb1c8b615 YARN-2990. FairScheduler's delay-scheduling always waits for node-local and rack-local delays, even for off-rack-only requests. (kasha) 
(cherry picked from commit 7e42088abf)
 2015-02-08 22:48:29 -08:00
Jason Lowe 61fdd862be YARN-3143. RM Apps REST API can return NPE or entries missing id and other fields. Contributed by Jason Lowe 
(cherry picked from commit da2fb2bc46)
 2015-02-06 21:51:49 +00:00
Jason Lowe 83449a4e4d YARN-3089. LinuxContainerExecutor does not handle file arguments to deleteAsUser. Contributed by Eric Payne 
(cherry picked from commit 4c484320b4)
 2015-02-06 20:40:33 +00:00
Jian He 3ddafaa7c8 YARN-2694. Ensure only single node label specified in ResourceRequest. Contributed by Wangda Tan 
(cherry picked from commit c1957fef29)
 2015-02-06 11:34:56 -08:00
Arun C. Murthy 92ff524182 YARN-1537. Fix race condition in TestLocalResourcesTrackerImpl.testLocalResourceCache. Contributed by Xuan Gong. 2015-02-05 23:59:34 -08:00
Arun C. Murthy 29c77de297 YARN-1904. Ensure exceptions thrown in ClientRMService & ApplicationHistoryClientService are uniform when application-attempt is not found. Contributed by Zhijie Shen. 2015-02-05 23:49:43 -08:00
Jian He b109051145 YARN-3145. Fixed ConcurrentModificationException on CapacityScheduler ParentQueue#getQueueUserAclInfo. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 4641196fe0)
 2015-02-05 16:13:23 -08:00
Jason Lowe ebdd88ec81 YARN-1582. Capacity Scheduler: add a maximum-allocation-mb setting per queue. Contributed by Thomas Graves 
(cherry picked from commit 69c8a7f45b)
 2015-02-05 19:30:06 +00:00
Sandy Ryza 84df660af4 YARN-3101. In Fair Scheduler, fix canceling of reservations for exceeding max share (Anubhav Dhoot via Sandy Ryza) 2015-02-05 09:45:17 -08:00
Tsuyoshi Ozawa cdd76d5922 YARN-3058. Fix error message of tokens' activation delay configuration. Contributed by Yi Liu. 
(cherry picked from commit 26dee1486b)
 2015-02-04 11:41:48 +09:00
Zhijie Shen 3ab3caa9ea YARN-2543. Made resource usage be published to the timeline server too. Contributed by Naganarasimha G R. 
(cherry picked from commit e5b56e2af6)
 2015-02-03 17:35:40 -08:00
Xuan c22dcdd191 YARN-3056. Add verification for containerLaunchDuration in 
TestNodeManagerMetrics. Contributed by zhihai Xu

(cherry picked from commit b73e776abc)
 2015-02-03 15:14:23 -08:00
Wangda Tan 205e15c4a3 YARN-3075. NodeLabelsManager implementation to retrieve label to node mapping (Varun Saxena via wangda) 
(cherry picked from commit 5bd984691b)
 2015-02-03 12:55:25 -08:00
Jian He c0b1311a93 YARN-3098. Created common QueueCapacities class in Capacity Scheduler to track capacities-by-labels of queues. Contributed by Wangda Tan 
(cherry picked from commit 21d80b3dd9)
 2015-02-03 11:43:40 -08:00
Robert Kanter 410830fe8c YARN-3022. Expose Container resource information from NodeManager for monitoring (adhoot via ranter) 
(cherry picked from commit f7a77819a1)
 2015-02-03 10:39:51 -08:00
Jason Lowe 3828fb5fab YARN-3085. Application summary should include the application type. Contributed by Rohith 
(cherry picked from commit a761bf8726)
 2015-02-03 14:57:25 +00:00
Jian He 4941b8f30d YARN-3077. Fixed RM to create zk root path recursively. Contributed by Chun Chen 
(cherry picked from commit 054a947989)
 2015-01-30 17:35:26 -08:00
Jian He cabf97ae4f YARN-3099. Capacity Scheduler LeafQueue/ParentQueue should use ResourceUsage to track used-resources-by-label. Contributed by Wangda Tan 
(cherry picked from commit 86358221fc)
 2015-01-30 15:16:22 -08:00
Jonathan Eagles d4ffca41d7 YARN-3108. ApplicationHistoryServer doesn't process -D arguments (Chang Li via jeagles) 
(cherry picked from commit 30a8778c63)
 2015-01-29 16:52:37 -06:00
Akira Ajisaka 1081b4c4a7 HADOOP-9907. Webapp http://hostname:port/metrics link is not working. (aajisaka) 
(cherry picked from commit 342efa110a)
 2015-01-30 02:50:46 +09:00
Wangda Tan 58d7d1efc6 YARN-3079. Scheduler should also update maximumAllocation when updateNodeResource. (Zhihai Xu via wangda) 
(cherry picked from commit 7882bc0f14)
 2015-01-28 21:56:17 -08:00
Wangda Tan 4718af4352 YARN-2932. Add entry for preemptable status (enabled/disabled) to scheduler web UI and queue initialize/refresh logging. (Eric Payne via wangda) 
(cherry picked from commit 18741adf97)
 2015-01-27 15:37:02 -08:00
Jonathan Eagles c95e8951f8 YARN-2897. CrossOriginFilter needs more log statements (Mit Desai via jeagles) 
(cherry picked from commit a8ad1e8089)
 2015-01-27 16:07:27 -06:00
Jian He 8100c8a68c YARN-3011. Possible IllegalArgumentException in ResourceLocalizationService might lead NM to crash. Contributed by Varun Saxena 
(cherry picked from commit 4e15fc0841)
 2015-01-27 13:31:48 -08:00
Tsuyoshi Ozawa 04ad500fac YARN-3086. Make NodeManager memory configurable in MiniYARNCluster. Contributed by Robert Metzger. 
(cherry picked from commit f56da3ce04)
 2015-01-28 00:16:07 +09:00
Jian He 61b4116b4b YARN-3092. Created a common ResourceUsage class to track labeled resource usages in Capacity Scheduler. Contributed by Wangda Tan 
(cherry picked from commit 6f9fe76918)
 2015-01-26 15:38:27 -08:00
Jason Lowe 07fe6a36cb YARN-3088. LinuxContainerExecutor.deleteAsUser can throw NPE if native executor returns an error. Contributed by Eric Payne 
(cherry picked from commit 902c6ea7e4)
 2015-01-26 15:41:23 +00:00
Xuan a7696b3fbf YARN-3024. LocalizerRunner should give DIE action when all resources are 
localized. Contributed by Chengbing Liu

(cherry picked from commit 0d6bd62102)
 2015-01-25 19:39:52 -08:00
Tsuyoshi Ozawa 25f2cf8a4e YARN-2800. Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature. Contributed by Wangda Tan. 
(cherry picked from commit 24aa462673)
 2015-01-23 20:37:26 +09:00
Tsuyoshi Ozawa ff627d94e7 YARN-3082. Non thread safe access to systemCredentials in NodeHeartbeatResponse processing. Contributed by Anubhav Dhoot. 
(cherry picked from commit 3aab354e66)
 2015-01-23 16:05:05 +09:00
Wangda Tan ef6fc24dfb YARN-2933. Capacity Scheduler preemption policy should only consider capacity without labels temporarily. Contributed by Mayank Bansal 
(cherry picked from commit 0a2d3e717d)
 2015-01-19 16:57:17 -08:00
Karthik Kambatla 4d8fa9615f YARN-2984. Metrics for container's actual memory usage. (kasha) 
(cherry picked from commit 84198564ba)
 2015-01-17 06:26:47 +05:30
Zhijie Shen dcd6eed13a YARN-2815. Excluded transitive dependency of JLine in hadoop-yarn-server-common. Contributed by Ferdinand Xu. 
(cherry picked from commit 43302f6f44)
 2015-01-16 16:38:36 -08:00
Junping Du 7cddec31d7 YARN-3064. TestRMRestart/TestContainerResourceUsage/TestNodeManagerResync failure with allocation timeout. (Contributed by Jian He) 
(cherry picked from commit 5d1cca34fa)
 2015-01-16 00:10:36 -08:00
Jian He ed5a0d0aa8 YARN-2861. Fixed Timeline DT secret manager to not reuse RM's configs. Contributed by Zhijie Shen 
(cherry picked from commit 9e33116d1d)
 2015-01-15 11:08:19 -08:00
Jian He 4931600030 YARN-2637. Fixed max-am-resource-percent calculation in CapacityScheduler when activating applications. Contributed by Craig Welch 
(cherry picked from commit c53420f583)
 2015-01-13 17:32:36 -08:00
Robert Kanter 13cdcf28e0 YARN-2643. Don't create a new DominantResourceCalculator on every FairScheduler.allocate call. (kasha via rkanter) 
(cherry picked from commit 51881535e6)
 2015-01-12 17:51:45 -08:00
Robert Kanter e5059b91bb YARN-3027. Scheduler should use totalAvailable resource from node instead of availableResource for maxAllocation. (adhoot via rkanter) 
(cherry picked from commit ae7bf31fe1)
 2015-01-12 10:48:45 -08:00
Jian He 36b3dcaab2 YARN-3014. Replaces labels on a host should update all NM's labels on that host. Contributed by Wangda Tan 
(cherry picked from commit a260406268)
 2015-01-09 17:50:18 -08:00
Jian He e7e6173049 YARN-2997. Fixed NodeStatusUpdater to not send alreay-sent completed container statuses on heartbeat. Contributed by Chengbing Liu 
(cherry picked from commit cc2a745f7e)
 2015-01-08 11:28:24 -08:00
Zhijie Shen d2fbba790a YARN-2996. Improved synchronization and I/O operations of FS- and Mem- RMStateStore. Contributed by Yi Liu. 
(cherry picked from commit dc2eaa26b2)
 2015-01-08 09:49:27 -08:00
Jian He d3b26454e4 YARN-3010. Fixed findbugs warning in AbstractYarnScheduler. Contributed by Yi Liu 
(cherry picked from commit e13a484a2b)
 2015-01-07 14:12:54 -08:00
Zhijie Shen 8a81312062 YARN-2427. Added the API of moving apps between queues in RM web services. Contributed by Varun Vasudev. 
(cherry picked from commit 60103fca04)
 2015-01-06 14:39:44 -08:00
Jian He c61e8a7bfa YARN-2978. Fixed potential NPE while getting queue info. Contributed by Varun Saxena 
(cherry picked from commit dd57c2047b)
 2015-01-06 14:16:43 -08:00
Karthik Kambatla 7adffad2bb YARN-2881. [YARN-2574] Implement PlanFollower for FairScheduler. (Anubhav Dhoot via kasha) 
(cherry picked from commit 0c4b112677)
 2015-01-06 04:42:55 +05:30
Karthik Kambatla d070597a0e YARN-2360. Fair Scheduler: Display dynamic fair share for queues on the scheduler page. (Ashwin Shankar and Wei Yan via kasha) 
(cherry picked from commit 270a271f53)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-01-06 03:28:48 +05:30
Zhijie Shen c6cf748985 YARN-2958. Made RMStateStore not update the last sequence number when updating the delegation token. Contributed by Varun Saxena. 
(cherry picked from commit 562a701945)
 2015-01-05 13:35:09 -08:00
Tsuyoshi Ozawa c116743bdd YARN-2922. ConcurrentModificationException in CapacityScheduler's LeafQueue. Contributed by Rohith Sharmaks. 
(cherry picked from commit ddc5be48fc)
 2015-01-05 00:11:01 +09:00
Karthik Kambatla 798ab51289 YARN-2998. Abstract out scheduler independent PlanFollower components. (Anubhav Dhoot via kasha) 
(cherry picked from commit e7257acd8a)
 2014-12-30 19:55:50 -08:00
Jian He cde5bfe3ec YARN-2987. Fixed ClientRMService#getQueueInfo to check against queue and app ACLs. Contributed by Varun Saxena 
(cherry picked from commit e2351c7ae2)
 2014-12-30 17:16:05 -08:00
Jian He 143e48c25a YARN-2493. Added node-labels page on RM web UI. Contributed by Wangda Tan 
(cherry picked from commit b7442bf92e)
 2014-12-30 16:49:59 -08:00
Zhijie Shen e5cb52ec1f YARN-2938. Fixed new findbugs warnings in hadoop-yarn-resourcemanager and hadoop-yarn-applicationhistoryservice. Contributed by Varun Saxena. 
(cherry picked from commit 241d3b3a50)
 2014-12-29 10:07:54 -08:00
Jian He ca0349b87a YARN-2992. ZKRMStateStore crashes due to session expiry. Contributed by Karthik Kambatla 
(cherry picked from commit 1454efe5d4)
 2014-12-26 18:00:32 -08:00
Junping Du fc3e810a19 YARN-2993. Several fixes (missing acl check, error log msg ...) and some refinement in AdminService. (Contributed by Yi Liu) 
(cherry picked from commit 40ee4bff65)
 2014-12-26 07:01:44 -08:00
Jian He 60530a6c4f YARN-2946. Fixed potential deadlock in RMStateStore. Contributed by Rohith Sharmaks 
(cherry picked from commit 4f18018b7a)
 2014-12-23 22:15:28 -08:00
Zhijie Shen f02bd6683a YARN-2937. Fixed new findbugs warnings in hadoop-yarn-nodemanager. Contributed by Varun Saxena. 
(cherry picked from commit 41a548a916)
 2014-12-23 20:35:32 -08:00
Junping Du 361fab14be YARN-2940. Fix new findbugs warnings in rest of the hadoop-yarn components. (Contributed by Li Lu) 
(cherry picked from commit d468c9aaf1)
 2014-12-23 20:06:53 -08:00
Jian He e3a54b1b8b YARN-2837. Support TimeLine server to recover delegation token when restarting. Contributed by Zhijie Shen 
(cherry picked from commit 149512a837)
 2014-12-23 18:26:37 -08:00
Jian He 8b398a66ca YARN-2340. Fixed NPE when queue is stopped during RM restart. Contributed by Rohith Sharmaks 
(cherry picked from commit 0d89859b51)
 2014-12-22 21:55:06 -08:00
Jian He 411836b74c YARN-2920. Changed CapacityScheduler to kill containers on nodes where node labels are changed. Contributed by Wangda Tan 
(cherry picked from commit fdf042dfff)
 2014-12-22 16:52:15 -08:00
Junping Du 6f252f7f36 YARN-2939. Fix new findbugs warnings in hadoop-yarn-common. (Li Lu via junping_du) 
(cherry picked from commit a696fbb001)
 2014-12-22 03:08:25 -08:00
Karthik Kambatla 2abec14ec6 YARN-2975. FSLeafQueue app lists are accessed without required locks. (kasha) 
(cherry picked from commit 24ee9e3431)
 2014-12-20 12:20:29 -08:00
Jian He 9180d11b3b YARN-2952. Fixed incorrect version check in StateStore. Contributed by Rohith Sharmaks 
(cherry picked from commit 808cba3821)
 2014-12-19 16:56:55 -08:00
Karthik Kambatla b4e8ae591d YARN-2675. containersKilled metrics is not updated when the container is killed during localization. (Zhihai Xu via kasha) 
(cherry picked from commit 954fb8581e)
 2014-12-19 16:03:02 -08:00
Karthik Kambatla 8ee40a1580 YARN-2738. [YARN-2574] Add FairReservationSystem for FairScheduler. (Anubhav Dhoot via kasha) 
(cherry picked from commit a22ffc3188)
 2014-12-19 15:38:27 -08:00
Jason Lowe 173664d70f YARN-2964. RM prematurely cancels tokens for jobs that submit jobs (oozie). Contributed by Jian He 
(cherry picked from commit 0402bada19)
 2014-12-18 23:31:11 +00:00
Junping Du 11e4ec52a1 YARN-2972. DelegationTokenRenewer thread pool never expands. Contributed by Jason Lowe 
(cherry picked from commit 2b4b0e8847)
 2014-12-17 17:41:21 -08:00
Karthik Kambatla d0d42c4e32 YARN-2203. [YARN-1492] Web UI for cache manager. (Chris Trezzo via kasha) 
(cherry picked from commit b7f64823e1)
 2014-12-17 16:34:33 -08:00
Karthik Kambatla 375a1e5971 YARN-2944. InMemorySCMStore can not be instantiated with ReflectionUtils#newInstance. (Chris Trezzo via kasha) 
(cherry picked from commit a1bd140964)
 2014-12-17 16:25:02 -08:00
Karthik Kambatla 2d832ad2eb YARN-2964. FSLeafQueue#assignContainer - document the reason for using both write and read locks. (Tsuyoshi Ozawa via kasha) 
(cherry picked from commit f2d150ea12)
 2014-12-17 16:24:17 -08:00
cnauroth 36068768d8 HADOOP-11321. copyToLocal cannot save a file to an SMB share unless the user has Full Control permissions. Contributed by Chris Nauroth. 
(cherry picked from commit e996a1bfd4)
 2014-12-16 15:32:23 -08:00
Karthik Kambatla b1ccf68db3 YARN-2914. [YARN-1492] Potential race condition in Singleton implementation of SharedCacheUploaderMetrics, CleanerMetrics, ClientSCMMetrics. (Varun Saxena via kasha) 
(cherry picked from commit e597249d36)
 2014-12-15 12:00:46 -08:00
Jian He d2356a552a Revert "YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai." 
This reverts commit 51af8d367d.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
(cherry picked from commit a4f2995b9e)
 2014-12-15 11:08:44 -08:00
Steve Loughran a858d726c8 YARN-2912 Jersey Tests failing with port in use. (varun saxena via stevel) 2014-12-12 17:10:54 +00:00
Devaraj K 10ff600b35 YARN-2243. Order of arguments for Preconditions.checkNotNull() is wrong in 
SchedulerApplicationAttempt ctor. Contributed by Devaraj K.

(cherry picked from commit bda748ac3a)
 2014-12-12 12:37:13 +05:30
Jian He 0f47e4e745 YARN-2930. Fixed TestRMRestart#testRMRestartRecoveringNodeLabelManager intermittent failure. Contributed by Wangda Tan 
(cherry picked from commit 2ed90a57fd)
 2014-12-09 16:48:46 -08:00
Karthik Kambatla 1986ea8dd2 YARN-2910. FSLeafQueue can throw ConcurrentModificationException. (Wilfred Spiegelenburg via kasha) 
(cherry picked from commit a2e07a5456)
 2014-12-09 14:00:52 -08:00
Karthik Kambatla 9d72b0282f YARN-2931. PublicLocalizer may fail until directory is initialized by LocalizeRunner. (Anubhav Dhoot via kasha) 
(cherry picked from commit db73cc9124)
 2014-12-08 22:26:44 -08:00
Jian He 4cc0abe4fe YARN-2869. CapacityScheduler should trim sub queue names when parse configuration. Contributed by Wangda Tan 
(cherry picked from commit e69af836f3)
 2014-12-05 17:34:15 -08:00
Jason Lowe b72fb6c774 YARN-2056. Disable preemption at Queue level. Contributed by Eric Payne 
(cherry picked from commit 4b13082199)
 2014-12-05 21:08:14 +00:00
Karthik Kambatla aadd0c392b YARN-2189. [YARN-1492] Admin service for cache manager. (Chris Trezzo via kasha) 
(cherry picked from commit 78968155d7)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/bin/yarn
 2014-12-04 19:48:18 -08:00
Jian He 1d1e7682c9 YARN-2301. Improved yarn container command. Contributed by Naganarasimha G R 
(cherry picked from commit 258623ff8b)
 2014-12-04 12:53:52 -08:00
Jian He 58c971164c YARN-2880. Added a test to make sure node labels will be recovered if RM restart is enabled. Contributed by Rohith Sharmaks 
(cherry picked from commit 73fbb3c66b)
 2014-12-03 17:17:40 -08:00
Harsh J ee25c0d890 YARN-2891. Failed Container Executor does not provide a clear error message. Contributed by Dustin Cote. (harsh) 
(cherry picked from commit 4b13318dea7a1cbbbfc1f84207af829cbe2f720e)
 2014-12-04 03:19:01 +05:30
Karthik Kambatla 25be97808b YARN-2874. Dead lock in DelegationTokenRenewer which blocks RM to execute any further apps. (Naganarasimha G R via kasha) 
(cherry picked from commit 799353e2c7)
 2014-12-03 13:45:03 -08:00
Jason Lowe db723a8499 MAPREDUCE-5932. Provide an option to use a dedicated reduce-side shuffle log. Contributed by Gera Shegalov 
(cherry picked from commit 03ab24aa01)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2014-12-03 17:12:47 +00:00
Junping Du 86535ff65f YARN-1156. Enhance NodeManager AllocatedGB and AvailableGB metrics for aggregation of decimal values. (Contributed by Tsuyoshi OZAWA) 
(cherry picked from commit e65b7c5ff6)
 2014-12-03 04:12:35 -08:00
Jian He d6f3d4893d YARN-2894. Fixed a bug regarding application view acl when RM fails over. Contributed by Rohith Sharmaks 
(cherry picked from commit 392c3aaea8)
 2014-12-02 17:17:16 -08:00
Jian He 5067ac098b YARN-2136. Changed RMStateStore to ignore store opearations when fenced. Contributed by Varun Saxena 
(cherry picked from commit 52bcefca8b)
 2014-12-02 11:02:00 -08:00
Zhijie Shen d21ef79707 YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai. 
(cherry picked from commit 51af8d367d)
 2014-12-01 19:08:31 -08:00
Jian He d208c90148 YARN-2765. Added leveldb-based implementation for RMStateStore. Contributed by Jason Lowe 
(cherry picked from commit a7fba0bc28)
 2014-12-01 16:45:17 -08:00
Junping Du c7bd22974a YARN-2907. SchedulerNode#toString should print all resource detail instead of only memory. (Contributed by Rohith) 
(cherry picked from commit c732ed760e)
 2014-12-01 05:41:56 -08:00
Zhijie Shen a9adc1295c YARN-2165. Added the sanity check for the numeric configuration values of the timeline service. Contributed by Vasanth kumar RJ. 
(cherry picked from commit 8f1454cc6d)
 2014-11-26 12:12:42 -08:00
Karthik Kambatla 3d4536af82 YARN-2188. [YARN-1492] Client service for cache manager. (Chris Trezzo and Sangjin Lee via kasha) 
(cherry picked from commit 834b3b98e7ecc4bfe93c7dd1a9e928e9653cf138)
 2014-11-25 16:22:14 -08:00
Jian He ae35b0e14d YARN-2906. CapacitySchedulerPage shows HTML tags for a queue's Active Users. Contributed by Jason Lowe 
(cherry picked from commit 8a7ca13b13)
 2014-11-25 16:18:02 -08:00
Jian He 2863056530 YARN-2404. Removed ApplicationAttemptState and ApplicationState class in RMStateStore. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 5805a81efb)
 2014-11-25 12:51:03 -08:00
Zhijie Shen 57d62d4ded YARN-2697. Remove useless RMAuthenticationHandler. Contributed by Haosong Huang. 
(cherry picked from commit e37a4ff0c1)
 2014-11-24 15:06:44 -08:00
Jason Lowe 89ef49fb08 YARN-1984. LeveldbTimelineStore does not handle db exceptions properly. Contributed by Varun Saxena 
(cherry picked from commit 1ce4d33c2d)
 2014-11-24 22:38:09 +00:00
Sandy Ryza 3e2e8eac22 YARN-2669. FairScheduler: queue names shouldn't allow periods (Wei Yan via Sandy Ryza) 2014-11-21 16:08:22 -08:00
Karthik Kambatla af0b54a4ee YARN-2679. Add metric for container launch duration. (Zhihai Xu via kasha) 
(cherry picked from commit 233b61e495)
 2014-11-21 14:22:53 -08:00
Karthik Kambatla e9db0aa35c YARN-2604. Scheduler should consider max-allocation-* in conjunction with the largest node. (Robert Kanter via kasha) 
(cherry picked from commit 3114d4731d)
 2014-11-21 10:42:47 -08:00
Karthik Kambatla 7ba5bb0c5b YARN-2315. FairScheduler: Set current capacity in addition to capacity. (Zhihai Xu via kasha) 
(cherry picked from commit a9a0cc3679)
 2014-11-19 20:19:54 -08:00
Karthik Kambatla 564deb75a8 YARN-2802. ClusterMetrics to include AM launch and register delays. (Zhihai Xu via kasha) 
(cherry picked from commit f9f8e750edbe6db453f0a845e2ed49ede66e0e8a)
 2014-11-19 19:50:35 -08:00
Jian He db31ef7e7f YARN-2865. Fixed RM to always create a new RMContext when transtions from StandBy to Active. Contributed by Rohith Sharmaks 
(cherry picked from commit 9cb8b75ba5)
 2014-11-19 19:49:44 -08:00
Karthik Kambatla 8df4c04c3a YARN-2690. [YARN-2574] Make ReservationSystem and its dependent classes independent of Scheduler type. (Anubhav Dhoot via kasha) 
(cherry picked from commit 2fce6d6141)
 2014-11-17 16:46:18 -08:00
Jason Lowe 242fd0e39a YARN-2414. RM web UI: app page will crash if app is failed before any attempt has been created. Contributed by Wangda Tan 
(cherry picked from commit 81c9d17af8)
 2014-11-17 21:16:54 +00:00
Vinod Kumar Vavilapalli 41275755c7 YARN-1703. Fixed ResourceManager web-proxy to close connections correctly. Contributed by Rohith Sharma. 
(cherry picked from commit 198fb58776)
 2014-11-15 12:06:53 -08:00
Jian He 21054c5d14 YARN-2432. RMStateStore should process the pending events before close. Contributed by Varun Saxena 
(cherry picked from commit 9b860661a5)
 2014-11-14 17:25:21 -08:00
Sandy Ryza c4c77669f0 YARN-2811. In Fair Scheduler, reservation fulfillments shouldn't ignore max share (Siqi Li via Sandy Ryza) 2014-11-14 15:20:37 -08:00
Jason Lowe ad140d1fc8 YARN-2816. NM fail to start with NPE during container recovery. Contributed by Zhihai Xu 
(cherry picked from commit 49c38898b0)
 2014-11-14 21:27:16 +00:00
Jian He beb184ac58 YARN-2856. Fixed RMAppImpl to handle ATTEMPT_KILLED event at ACCEPTED state on app recovery. Contributed by Rohith Sharmaks 
(cherry picked from commit d005404ef7)
 2014-11-13 15:34:56 -08:00
Zhijie Shen 53d6c91df9 YARN-2766. Made ApplicationHistoryManager return a sorted list of apps, attempts and containers. Contributed by Robert Kanter. 
(cherry picked from commit 3648cb57c9)
 2014-11-13 13:54:30 -08:00
Jason Lowe 6fd547da8f YARN-2846. Incorrect persist exit code for running containers in reacquireContainer() that interrupted by NodeManager restart. Contributed by Junping Du 
(cherry picked from commit 33ea5ae92b)
 2014-11-13 16:17:08 +00:00
Vinod Kumar Vavilapalli ed3e5cb164 YARN-2853. Fixed a bug in ResourceManager causing apps to hang when the user kill request races with ApplicationMaster finish. Contributed by Jian He. 
(cherry picked from commit 3651fe1b08)
 2014-11-13 08:13:57 -08:00
Zhijie Shen 8405ec0895 YARN-2794. Fixed log messages about distributing system-credentials. Contributed by Jian He. 
(cherry picked from commit be7bf956e9)
 2014-11-12 11:09:34 -08:00
Karthik Kambatla ff1b13ded5 YARN-2236. [YARN-1492] Shared Cache uploader service on the Node Manager. (Chris Trezzo and Sanjin Lee via kasha) 
(cherry picked from commit a04143039e)
 2014-11-12 09:31:30 -08:00
Jason Lowe 6984d899e3 YARN-2780. Log aggregated resource allocation in rm-appsummary.log. Contributed by Eric Payne 
(cherry picked from commit f8aefa5e9c)
 2014-11-12 17:02:30 +00:00
Ravi Prakash d863f54f57 YARN-1964. Create Docker analog of the LinuxContainerExecutor in YARN 2014-11-11 21:29:27 -08:00
Karthik Kambatla f030959849 YARN-2735. diskUtilizationPercentageCutoff and diskUtilizationSpaceCutoff are initialized twice in DirectoryCollection. (Zhihai Xu via kasha) 
(cherry picked from commit 061bc293c8)
 2014-11-11 10:31:56 -08:00
Xuan b3badf935a YARN-2841. RMProxy should retry EOFException. Contributed by Jian He 
(cherry picked from commit 5c9a51f140)
 2014-11-10 18:26:32 -08:00
Arun C. Murthy adfb830a2b Preparing to release hadoop-2.6.0: Set version in branch-2 to 2.7.0-SNAPSHOT. 2014-11-09 19:19:02 -08:00
Vinod Kumar Vavilapalli d76fc94b21 YARN-2834. Fixed ResourceManager to ignore token-renewal failures on recovery consistent with the (somewhat incorrect) behaviour in the non-recovery case. Contributed by Jian He. 
(cherry picked from commit e76faebc95)
 2014-11-09 18:57:48 -08:00
Arun C. Murthy 175d222bfc YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles. 
(cherry picked from commit 43cd07b408)
 2014-11-09 15:03:59 -08:00
Zhijie Shen 1880a5a7c3 YARN-2505. Supported get/add/remove/change labels in RM REST API. Contributed by Craig Welch. 
(cherry picked from commit 9a4e0d343e)
 2014-11-07 20:39:53 -08:00
Vinod Kumar Vavilapalli b865a95f06 YARN-2826. Fixed user-groups mappings' refresh bug caused by YARN-2826. Contributed by Wangda Tan. 
(cherry picked from commit df36edf751)
 2014-11-07 19:44:52 -08:00
Xuan 7d26cffb0c YARN-2819. NPE in ATS Timeline Domains when upgrading from 2.4 to 2.6. Contributed by Zhijie Shen 
(cherry picked from commit 4a114dd67a)
 2014-11-07 16:12:45 -08:00
Jason Lowe a5764cb783 YARN-2825. Container leak on NM. Contributed by Jian He 
(cherry picked from commit c3d475070a)
 2014-11-07 23:17:34 +00:00
Vinod Kumar Vavilapalli a50345f654 YARN-2753. Fixed a bunch of bugs in the NodeLabelsManager classes. Contributed by Zhihai xu. 
(cherry picked from commit 4cfd5bc7c1)
 2014-11-07 14:16:54 -08:00
cnauroth c16f718293 YARN-2803. MR distributed cache not working correctly on Windows after NodeManager privileged account changes. Contributed by Craig Welch. 
(cherry picked from commit 06b797947c)
 2014-11-07 12:36:42 -08:00
Vinod Kumar Vavilapalli d5d2905262 YARN-2824. Fixed Capacity Scheduler to not crash when some node-labels are not mapped to queues by making default capacities per label to be zero. Contributed by Wangda Tan. 
(cherry picked from commit 2ac1be7dec)
 2014-11-07 10:44:25 -08:00
Xuan bf79541868 YARN-2810. TestRMProxyUsersConf fails on Windows VMs. Contributed by Varun Vasudev 
(cherry picked from commit 1e97f2f094)
 2014-11-07 09:46:04 -08:00
Vinod Kumar Vavilapalli eb01c602ed YARN-2823. Fixed ResourceManager app-attempt state machine to inform schedulers about previous finished attempts of a running appliation to avoid expectation mismatch w.r.t transferred containers. Contributed by Jian He. 
(cherry picked from commit a5657182a7)
 2014-11-07 09:30:04 -08:00
Vinod Kumar Vavilapalli 52e57a95d9 YARN-2744. Fixed CapacityScheduler to validate node-labels correctly against queues. Contributed by Wangda Tan. 
(cherry picked from commit a3839a9fbf)
 2014-11-06 17:29:12 -08:00
Vinod Kumar Vavilapalli 0f9199fb07 YARN-2818. Removed the now unnecessary user entity injection from Timeline service given we now have domains. Contributed by Zhijie Shen. 
(cherry picked from commit f5b19bed7d)
 2014-11-06 11:49:44 -08:00
Xuan 4aa98d5991 YARN-2812. TestApplicationHistoryServer is likely to fail on less powerful machine. Contributed by Zhijie Shen 
(cherry picked from commit b0b52c4e11)
 2014-11-05 20:44:07 -08:00
Xuan e26be1d85c YARN-2813. Fixed NPE from MemoryTimelineStore.getDomains. Contributed by Zhijie Shen 2014-11-05 18:26:07 -08:00
Jian He f92ff24f5e YARN-2579. Fixed a deadlock issue when EmbeddedElectorService and FatalEventDispatcher try to transition RM to StandBy at the same time. Contributed by Rohith Sharmaks 
(cherry picked from commit 395275af86)
 2014-11-05 17:03:01 -08:00
Vinod Kumar Vavilapalli 3bc31e35a6 YARN-2805. Fixed ResourceManager to load HA configs correctly before kerberos login. Contributed by Wangda Tan. 
(cherry picked from commit 834e931d8e)
 2014-11-05 15:31:41 -08:00
Zhijie Shen f3ed97de3a YARN-2767. Added a test case to verify that http static user cannot kill or submit apps in the secure mode. Contributed by Varun Vasudev. 
(cherry picked from commit 7a4c92a9d55fcecef066053ac30dff0fcd4ec90c)
 2014-11-05 10:59:05 -08:00
Vinod Kumar Vavilapalli 56ab333161 YARN-2804. Fixed Timeline service to not fill the logs with JAXB bindings exceptions. Contributed by Zhijie Shen. 
(cherry picked from commit b76179895d)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-11-04 18:03:17 -08:00
Haohui Mai fa92b40f09 HADOOP-11230. Add missing dependency of bouncycastle for kms, httpfs, hdfs, MR and YARN. Contributed by Robert Kanter. 2014-11-04 17:53:00 -08:00
Karthik Kambatla 3e4b280de7 YARN-2010. Handle app-recovery failures gracefully. (Jian He and Karthik Kambatla via kasha) 
(cherry picked from commit b2cd269802)
 2014-11-04 17:48:49 -08:00
Zhijie Shen e06c23a6c9 YARN-2752. Made ContainerExecutor append "nice -n" arg only when priority adjustment flag is set. Contributed by Xuan Gong. 2014-11-04 15:50:10 -08:00
Haohui Mai aa09c4c342 HADOOP-10563. Remove the dependency of jsp in trunk. Contributed by Haohui Mai. 2014-11-04 14:52:51 -08:00
Vinod Kumar Vavilapalli 9c76dcadaf YARN-1922. Fixed NodeManager to kill process-trees correctly in the presence of races between the launch and the stop-container call and when root processes crash. Contributed by Billie Rinaldi. 
(cherry picked from commit c5a46d4c8c)
 2014-11-03 16:40:37 -08:00
Vinod Kumar Vavilapalli 277141b82d YARN-2795. Fixed ResourceManager to not crash loading node-label data from HDFS in secure mode. Contributed by Wangda Tan. 
(cherry picked from commit ec6cbece8e)
 2014-11-03 13:45:31 -08:00
Vinod Kumar Vavilapalli 715c81ef6d YARN-2788. Fixed backwards compatiblity issues with log-aggregation feature that were caused when adding log-upload-time via YARN-2703. Contributed by Xuan Gong. 
(cherry picked from commit 58e9f24e0f)
 2014-11-03 13:19:34 -08:00
Jason Lowe 8091ea54d5 YARN-2730. DefaultContainerExecutor runs only one localizer at a time. Contributed by Siqi Li 
(cherry picked from commit 6157ace547)
 2014-11-03 20:39:14 +00:00
Zhijie Shen caae0a0a08 YARN-2785. Fixed intermittent TestContainerResourceUsage failure. Contributed by Varun Vasudev. 
(cherry picked from commit 27715ec63b)
 2014-11-02 15:22:54 -08:00
Vinod Kumar Vavilapalli 6627f67bf5 YARN-2790. Fixed a NodeManager bug that was causing log-aggregation to fail beyond HFDS delegation-token expiry even when RM is a proxy-user (YARN-2704). Contributed by Jian He. 
(cherry picked from commit 5c0381c96a)
 2014-11-01 16:33:35 -07:00
Zhijie Shen d9ac25454c YARN-2711. Fixed TestDefaultContainerExecutor#testContainerLaunchError failure on Windows. Contributed by Varun Vasudev. 
(cherry picked from commit 1cd088fd9d)
 2014-10-31 17:45:05 -07:00
Vinod Kumar Vavilapalli 6d4a04a033 YARN-2789. Re-instated the NodeReport.newInstance private unstable API modified in YARN-2698 so that tests in YARN frameworks don't break. Contributed by Wangda Tan. 
(cherry picked from commit 6ce32f593b)
 2014-10-31 15:33:41 -07:00
Xuan 902566741e YARN-2701. Addendum patch. Potential race condition in startLocalizer when using LinuxContainerExecutor. Contributed by Xuan Gong 2014-10-31 14:39:06 -07:00
Jian He a87f827ae4 YARN-2770. Added functionality to renew/cancel TimeLineDelegationToken. Contributed by Zhijie Shen 
(cherry picked from commit 1b4be91866)
 2014-10-31 13:17:19 -07:00
Karthik Kambatla 268af259b5 YARN-2186. [YARN-1492] Node Manager uploader service for cache manager. (Chris Trezzo and Sangjin Lee via kasha) 
(cherry picked from commit 256697acd5)
 2014-10-31 13:01:25 -07:00
Vinod Kumar Vavilapalli 97694e78ad YARN-2779. Fixed ResourceManager to not require delegation tokens for communicating with Timeline Service. Contributed by Zhijie Shen. 
(cherry picked from commit d1828d9443)
 2014-10-30 23:17:16 -07:00
Vinod Kumar Vavilapalli 0ba8d8c19c YARN-2698. Moved some node label APIs to be correctly placed in client protocol. Contributed by Wangda Tan. 
(cherry picked from commit e0233c16eb)
 2014-10-30 23:02:06 -07:00
Jason Lowe 3e8544c5f2 YARN-2755. NM fails to clean up usercache_DEL_<timestamp> dirs after YARN-661. Contributed by Siqi Li 
(cherry picked from commit 73e626ad91)
 2014-10-30 15:11:57 +00:00
Karthik Kambatla d2ba115f06 YARN-2712. TestWorkPreservingRMRestart: Augment FS tests with queue and headroom checks. (Tsuyoshi Ozawa via kasha) 
(cherry picked from commit 179cab81e0)
 2014-10-30 00:39:34 -07:00
Karthik Kambatla e88832dfb3 YARN-2742. FairSchedulerConfiguration should allow extra spaces between value and unit. (Wei Yan via kasha) 
(cherry picked from commit 782971ae7a)
 2014-10-29 10:25:29 -07:00
Jian He 720de7eb4c YARN-2503. Added node lablels in web UI. Contributed by Wangda Tan 
(cherry picked from commit d5e0a09721)
 2014-10-28 18:00:59 -07:00
Zhijie Shen f40389ae08 YARN-2741. Made NM web UI serve logs on the drive other than C: on Windows. Contributed by Craig Welch. 
(cherry picked from commit 8984e9b177)
 2014-10-28 14:12:09 -07:00
Xuan 43b1878f86 YARN-2758. Update TestApplicationHistoryClientService to use the new generic history store. Contributed by Zhijie Shen 2014-10-28 13:30:55 -07:00
Xuan 43a53f2b65 YARN-2279. Add UTs to cover timeline server authentication. Contributed by Zhijie Shen 2014-10-28 11:18:35 -07:00
Vinod Kumar Vavilapalli 0ad33e1483 YARN-2704. Changed ResourceManager to optionally obtain tokens itself for the sake of localization and log-aggregation for long-running services. Contributed by Jian He. 
(cherry picked from commit a16d022ca4)
 2014-10-27 15:50:51 -07:00
Jian He 2d3c64943d YARN-2591. Fixed AHSWebServices to return FORBIDDEN(403) if the request user doesn't have access to the history data. Contributed by Zhijie Shen 
(cherry picked from commit c05b581a55)
 2014-10-27 10:46:37 -07:00
Xuan 3a77838535 YARN-2726. CapacityScheduler should explicitly log when an accessible label has no capacity. Contributed by Wangda Tan 2014-10-27 10:28:45 -07:00
Vinod Kumar Vavilapalli 36d3f1fcca YARN-2743. Fixed a bug in ResourceManager that was causing RMDelegationToken identifiers to be tampered and thus causing app submission failures in secure mode. Contributed by Jian He. 
(cherry picked from commit 0186645505)
 2014-10-26 11:15:45 -07:00
Karthik Kambatla acc6f622e0 YARN-2183. [YARN-1492] Cleaner service for cache manager. (Chris Trezzo and Sangjin Lee via kasha) 
(cherry picked from commit c51e53d7aa)
 2014-10-25 11:50:06 -07:00
Jian He 96a6e02d16 YARN-1915. Fixed a race condition that client could use the ClientToAMToken to contact with AM before AM actually receives the ClientToAMTokenMasterKey. Contributed by Jason Lowe 
(cherry picked from commit 5864dd99a4)
 2014-10-24 22:48:48 -07:00
Zhijie Shen 1b81105143 YARN-2703. Added logUploadedTime into LogValue for better display. Contributed by Xuan Gong. 
(cherry picked from commit f81dc3f995)
 2014-10-24 14:12:17 -07:00
Zhijie Shen 3b03ea6b50 YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He. 
(cherry picked from commit 0f3b6900be)
 2014-10-23 22:03:30 -07:00
Jian He 1c235a4448 YARN-2198. Remove the need to run NodeManager as privileged account for Windows Secure Container Executor. Contributed by Remus Rusanu 
(cherry picked from commit 3b12fd6cfb)
 2014-10-22 15:58:26 -07:00
Zhijie Shen 4b9de2cdfe YARN-2721. Suppress NodeExist exception thrown by ZKRMStateStore when it retries creating znode. Contributed by Jian He. 
(cherry picked from commit 7e3b5e6f5c)
 2014-10-21 21:52:01 -07:00
Vinod Kumar Vavilapalli 77dc9afe86 YARN-2715. Fixed ResourceManager to respect common configurations for proxy users/groups beyond just the YARN level config. Contributed by Zhijie Shen. 
(cherry picked from commit c0e034336c)
 2014-10-21 20:10:45 -07:00
cnauroth a066134277 YARN-2720. Windows: Wildcard classpath variables not expanded against resources contained in archives. Contributed by Craig Welch. 
(cherry picked from commit 6637e3cf95)
 2014-10-21 12:34:58 -07:00
Jason Lowe 3820bf055e YARN-90. NodeManager should identify failed disks becoming good again. Contributed by Varun Vasudev 
(cherry picked from commit 6f2028bd15)
 2014-10-21 17:33:34 +00:00
Jian He e9564e729f Missing file for YARN-2701 
(cherry picked from commit 4fa1fb3193)
 2014-10-20 19:58:21 -07:00
Jian He 3c8ae89050 YARN-2701. Potential race condition in startLocalizer when using LinuxContainerExecutor. Contributed by Xuan Gong 
(cherry picked from commit 2839365f23)
 2014-10-20 19:54:10 -07:00
Vinod Kumar Vavilapalli 9915d52185 YARN-2504. Enhanced RM Admin CLI to support management of node-labels. Contribyted by Wangda Tan. 
(cherry picked from commit 8256766498)
 2014-10-18 12:08:28 -07:00
Vinod Kumar Vavilapalli ea841eb25d YARN-2676. Enhanced Timeline auth-filter to support proxy users. Contributed by Zhijie Shen. 
(cherry picked from commit 39063cd36f)
 2014-10-17 22:03:43 -07:00
Vinod Kumar Vavilapalli 1c52b6551b YARN-2705. Fixed bugs in ResourceManager node-label manager that were causing test-failures: added a dummy in-memory labels-manager. Contributed by Wangda Tan. 
(cherry picked from commit e9c66e8fd2)
 2014-10-17 18:27:04 -07:00
Jian He 78a57087bf YARN-1879. Marked Idempotent/AtMostOnce annotations to ApplicationMasterProtocol for RM fail over. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit c3de2412eb)
 2014-10-17 16:36:15 -07:00
Jian He 79428e021f YARN-2588. Standby RM fails to transitionToActive if previous transitionToActive failed with ZK exception. Contributed by Rohith Sharmaks 
(cherry picked from commit a6aa6e42ca)
 2014-10-17 10:55:04 -07:00
Vinod Kumar Vavilapalli 99ce4277a8 YARN-2699. Fixed a bug in CommonNodeLabelsManager that caused tests to fail when using ephemeral ports on NodeIDs. Contributed by Wangda Tan. 
(cherry picked from commit abae63caf9)
 2014-10-17 08:59:22 -07:00
Jian He a78f1f8da8 YARN-2682. Updated WindowsSecureContainerExecutor to not use DefaultContainerExecutor#getFirstApplicationDir and use getWorkingDir() instead. Contributed by Zhihai Xu 
(cherry picked from commit 0fd0ebae64)
 2014-10-16 18:15:43 -07:00
Jian He caba212f12 YARN-2621. Simplify the output when the user doesn't have the access for getDomain(s). Contributed by Zhijie Shen 
(cherry picked from commit 233d446be1)
 2014-10-16 16:38:00 -07:00
Vinod Kumar Vavilapalli 76cf2250de YARN-2685. Fixed a bug in CommonNodeLabelsManager that caused wrong resource tracking per label when a host runs multiple node-managers. Contributed by Wangda Tan. 
(cherry picked from commit b3056c266a)
 2014-10-15 18:48:36 -07:00
Vinod Kumar Vavilapalli e8e3a36213 YARN-2496. Enhanced Capacity Scheduler to have basic support for allocating resources based on node-labels. Contributed by Wangda Tan. 
YARN-2500. Ehnaced ResourceManager to support schedulers allocating resources based on node-labels. Contributed by Wangda Tan.

(cherry picked from commit f2ea555ac6)
 2014-10-15 18:34:19 -07:00
Jian He f93d2ea27e YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 2014-10-15 15:28:26 -07:00
Zhijie Shen bec165e163 YARN-2656. Made RM web services authentication filter support proxy user. Contributed by Varun Vasudev and Zhijie Shen. 
(cherry picked from commit 1220bb72d4)
 2014-10-14 21:56:44 -07:00
Zhijie Shen 8aeda20b25 HADOOP-11181. Generalized o.a.h.s.t.d.DelegationTokenManager to handle all sub-classes of AbstractDelegationTokenIdentifier. Contributed by Zhijie Shen. 
(cherry picked from commit cdce88376a)
 2014-10-14 11:37:28 -07:00
Karthik Kambatla 88455173e8 YARN-2566. DefaultContainerExecutor should pick a working directory randomly. (Zhihai Xu via kasha) 
(cherry picked from commit cc93e7e683)
 2014-10-13 16:32:42 -07:00
Karthik Kambatla a1116b56a4 YARN-2641. Decommission nodes on -refreshNodes instead of next NM-RM heartbeat. (Zhihai Xu via kasha) 
(cherry picked from commit da709a2eac)
 2014-10-13 16:24:00 -07:00
Jian He 536254f5e7 YARN-2308. Changed CapacityScheduler to explicitly throw exception if the queue 
to which the apps were submitted is changed across RM restart. Contributed by Craig Welch & Chang Li
(cherry picked from commit f9680d9a16)
 2014-10-13 14:16:12 -07:00
Jason Lowe 6b5741d423 YARN-2377. Localization exception stack traces are not passed as diagnostic info. Contributed by Gera Shegalov 
(cherry picked from commit a56ea01002)
 2014-10-13 18:32:38 +00:00
Zhijie Shen e51ae64761 YARN-2651. Spun off LogRollingInterval from LogAggregationContext. Contributed by Xuan Gong. 
(cherry picked from commit 4aed2d8e91)
 2014-10-13 10:55:09 -07:00
Vinod Kumar Vavilapalli 8ef83a9480 YARN-2494. Added NodeLabels Manager internal API and implementation. Contributed by Wangda Tan. 
(cherry picked from commit db7f165319)
 2014-10-10 11:45:15 -07:00
Zhijie Shen 1e6d81a886 YARN-2583. Modified AggregatedLogDeletionService to be able to delete rolling aggregated logs. Contributed by Xuan Gong. 
(cherry picked from commit cb81bac002)
 2014-10-10 00:16:34 -07:00
cnauroth b81641a310 YARN-2662. TestCgroupsLCEResourcesHandler leaks file descriptors. Contributed by Chris Nauroth. 
(cherry picked from commit d3afd730ac)
 2014-10-09 22:47:04 -07:00
Karthik Kambatla 7e20187f91 YARN-2180. [YARN-1492] In-memory backing store for cache manager. (Chris Trezzo via kasha) 
(cherry picked from commit 4f426fe223)
 2014-10-09 17:16:45 -07:00
Mayank Bansal c1a86d105e YARN-2598 GHS should show N/A instead of null for the inaccessible information (Zhijie Shen via mayank) 2014-10-09 13:43:36 -07:00
Jian He 5e6f86e3aa YARN-2649. Fixed TestAMRMRPCNodeUpdates test failure. Contributed by Ming Ma 
(cherry picked from commit e16e25ab1b)
 2014-10-08 11:03:00 -07:00
Jian He 9ff7c0c96c YARN-1857. CapacityScheduler headroom doesn't account for other AM's running. Contributed by Chen He and Craig Welch 
(cherry picked from commit 30d56fdbb4)
 2014-10-07 13:48:03 -07:00
Jian He 5c33e91229 YARN-2644. Fixed CapacityScheduler to return up-to-date headroom when AM allocates. Contributed by Craig Welch 
(cherry picked from commit 519e5a7dd2)
 2014-10-06 15:51:38 -07:00