7adffad2bb
YARN-2881. [YARN-2574] Implement PlanFollower for FairScheduler. (Anubhav Dhoot via kasha)
...
(cherry picked from commit 0c4b112677
2015-01-06 04:42:55 +05:30
d070597a0e
YARN-2360. Fair Scheduler: Display dynamic fair share for queues on the scheduler page. (Ashwin Shankar and Wei Yan via kasha)
...
(cherry picked from commit 270a271f53
2015-01-06 03:28:48 +05:30
c6cf748985
YARN-2958. Made RMStateStore not update the last sequence number when updating the delegation token. Contributed by Varun Saxena.
...
(cherry picked from commit 562a701945
2015-01-05 13:35:09 -08:00
c116743bdd
YARN-2922. ConcurrentModificationException in CapacityScheduler's LeafQueue. Contributed by Rohith Sharmaks.
...
(cherry picked from commit ddc5be48fc
2015-01-05 00:11:01 +09:00
798ab51289
YARN-2998. Abstract out scheduler independent PlanFollower components. (Anubhav Dhoot via kasha)
...
(cherry picked from commit e7257acd8a
2014-12-30 19:55:50 -08:00
cde5bfe3ec
YARN-2987. Fixed ClientRMService#getQueueInfo to check against queue and app ACLs. Contributed by Varun Saxena
...
(cherry picked from commit e2351c7ae2
2014-12-30 17:16:05 -08:00
143e48c25a
YARN-2493. Added node-labels page on RM web UI. Contributed by Wangda Tan
...
(cherry picked from commit b7442bf92e
2014-12-30 16:49:59 -08:00
e5cb52ec1f
YARN-2938. Fixed new findbugs warnings in hadoop-yarn-resourcemanager and hadoop-yarn-applicationhistoryservice. Contributed by Varun Saxena.
...
(cherry picked from commit 241d3b3a50
2014-12-29 10:07:54 -08:00
ca0349b87a
YARN-2992. ZKRMStateStore crashes due to session expiry. Contributed by Karthik Kambatla
...
(cherry picked from commit 1454efe5d4
2014-12-26 18:00:32 -08:00
fc3e810a19
YARN-2993. Several fixes (missing acl check, error log msg ...) and some refinement in AdminService. (Contributed by Yi Liu)
...
(cherry picked from commit 40ee4bff65
2014-12-26 07:01:44 -08:00
60530a6c4f
YARN-2946. Fixed potential deadlock in RMStateStore. Contributed by Rohith Sharmaks
...
(cherry picked from commit 4f18018b7a
2014-12-23 22:15:28 -08:00
e3a54b1b8b
YARN-2837. Support TimeLine server to recover delegation token when restarting. Contributed by Zhijie Shen
...
(cherry picked from commit 149512a837
2014-12-23 18:26:37 -08:00
8b398a66ca
YARN-2340. Fixed NPE when queue is stopped during RM restart. Contributed by Rohith Sharmaks
...
(cherry picked from commit 0d89859b51
2014-12-22 21:55:06 -08:00
411836b74c
YARN-2920. Changed CapacityScheduler to kill containers on nodes where node labels are changed. Contributed by Wangda Tan
...
(cherry picked from commit fdf042dfff
2014-12-22 16:52:15 -08:00
6f252f7f36
YARN-2939. Fix new findbugs warnings in hadoop-yarn-common. (Li Lu via junping_du)
...
(cherry picked from commit a696fbb001
2014-12-22 03:08:25 -08:00
2abec14ec6
YARN-2975. FSLeafQueue app lists are accessed without required locks. (kasha)
...
(cherry picked from commit 24ee9e3431
2014-12-20 12:20:29 -08:00
9180d11b3b
YARN-2952. Fixed incorrect version check in StateStore. Contributed by Rohith Sharmaks
...
(cherry picked from commit 808cba3821
2014-12-19 16:56:55 -08:00
8ee40a1580
YARN-2738. [YARN-2574] Add FairReservationSystem for FairScheduler. (Anubhav Dhoot via kasha)
...
(cherry picked from commit a22ffc3188
2014-12-19 15:38:27 -08:00
173664d70f
YARN-2964. RM prematurely cancels tokens for jobs that submit jobs (oozie). Contributed by Jian He
...
(cherry picked from commit 0402bada19
2014-12-18 23:31:11 +00:00
11e4ec52a1
YARN-2972. DelegationTokenRenewer thread pool never expands. Contributed by Jason Lowe
...
(cherry picked from commit 2b4b0e8847
2014-12-17 17:41:21 -08:00
2d832ad2eb
YARN-2964. FSLeafQueue#assignContainer - document the reason for using both write and read locks. (Tsuyoshi Ozawa via kasha)
...
(cherry picked from commit f2d150ea12
2014-12-17 16:24:17 -08:00
a858d726c8
YARN-2912 Jersey Tests failing with port in use. (varun saxena via stevel)
2014-12-12 17:10:54 +00:00
10ff600b35
YARN-2243. Order of arguments for Preconditions.checkNotNull() is wrong in
...
SchedulerApplicationAttempt ctor. Contributed by Devaraj K.
(cherry picked from commit bda748ac3a
2014-12-12 12:37:13 +05:30
0f47e4e745
YARN-2930. Fixed TestRMRestart#testRMRestartRecoveringNodeLabelManager intermittent failure. Contributed by Wangda Tan
...
(cherry picked from commit 2ed90a57fd
2014-12-09 16:48:46 -08:00
1986ea8dd2
YARN-2910. FSLeafQueue can throw ConcurrentModificationException. (Wilfred Spiegelenburg via kasha)
...
(cherry picked from commit a2e07a5456
2014-12-09 14:00:52 -08:00
4cc0abe4fe
YARN-2869. CapacityScheduler should trim sub queue names when parse configuration. Contributed by Wangda Tan
...
(cherry picked from commit e69af836f3
2014-12-05 17:34:15 -08:00
b72fb6c774
YARN-2056. Disable preemption at Queue level. Contributed by Eric Payne
...
(cherry picked from commit 4b13082199
2014-12-05 21:08:14 +00:00
1d1e7682c9
YARN-2301. Improved yarn container command. Contributed by Naganarasimha G R
...
(cherry picked from commit 258623ff8b
2014-12-04 12:53:52 -08:00
58c971164c
YARN-2880. Added a test to make sure node labels will be recovered if RM restart is enabled. Contributed by Rohith Sharmaks
...
(cherry picked from commit 73fbb3c66b
2014-12-03 17:17:40 -08:00
25be97808b
YARN-2874. Dead lock in DelegationTokenRenewer which blocks RM to execute any further apps. (Naganarasimha G R via kasha)
...
(cherry picked from commit 799353e2c7
2014-12-03 13:45:03 -08:00
d6f3d4893d
YARN-2894. Fixed a bug regarding application view acl when RM fails over. Contributed by Rohith Sharmaks
...
(cherry picked from commit 392c3aaea8
2014-12-02 17:17:16 -08:00
5067ac098b
YARN-2136. Changed RMStateStore to ignore store opearations when fenced. Contributed by Varun Saxena
...
(cherry picked from commit 52bcefca8b
2014-12-02 11:02:00 -08:00
d208c90148
YARN-2765. Added leveldb-based implementation for RMStateStore. Contributed by Jason Lowe
...
(cherry picked from commit a7fba0bc28
2014-12-01 16:45:17 -08:00
c7bd22974a
YARN-2907. SchedulerNode#toString should print all resource detail instead of only memory. (Contributed by Rohith)
...
(cherry picked from commit c732ed760e
2014-12-01 05:41:56 -08:00
ae35b0e14d
YARN-2906. CapacitySchedulerPage shows HTML tags for a queue's Active Users. Contributed by Jason Lowe
...
(cherry picked from commit 8a7ca13b13
2014-11-25 16:18:02 -08:00
2863056530
YARN-2404. Removed ApplicationAttemptState and ApplicationState class in RMStateStore. Contributed by Tsuyoshi OZAWA
...
(cherry picked from commit 5805a81efb
2014-11-25 12:51:03 -08:00
57d62d4ded
YARN-2697. Remove useless RMAuthenticationHandler. Contributed by Haosong Huang.
...
(cherry picked from commit e37a4ff0c1
2014-11-24 15:06:44 -08:00
3e2e8eac22
YARN-2669. FairScheduler: queue names shouldn't allow periods (Wei Yan via Sandy Ryza)
2014-11-21 16:08:22 -08:00
e9db0aa35c
YARN-2604. Scheduler should consider max-allocation-* in conjunction with the largest node. (Robert Kanter via kasha)
...
(cherry picked from commit 3114d4731d
2014-11-21 10:42:47 -08:00
7ba5bb0c5b
YARN-2315. FairScheduler: Set current capacity in addition to capacity. (Zhihai Xu via kasha)
...
(cherry picked from commit a9a0cc3679
2014-11-19 20:19:54 -08:00
564deb75a8
YARN-2802. ClusterMetrics to include AM launch and register delays. (Zhihai Xu via kasha)
...
(cherry picked from commit f9f8e750edbe6db453f0a845e2ed49ede66e0e8a)
2014-11-19 19:50:35 -08:00
db31ef7e7f
YARN-2865. Fixed RM to always create a new RMContext when transtions from StandBy to Active. Contributed by Rohith Sharmaks
...
(cherry picked from commit 9cb8b75ba5
2014-11-19 19:49:44 -08:00
8df4c04c3a
YARN-2690. [YARN-2574] Make ReservationSystem and its dependent classes independent of Scheduler type. (Anubhav Dhoot via kasha)
...
(cherry picked from commit 2fce6d6141
2014-11-17 16:46:18 -08:00
242fd0e39a
YARN-2414. RM web UI: app page will crash if app is failed before any attempt has been created. Contributed by Wangda Tan
...
(cherry picked from commit 81c9d17af8
2014-11-17 21:16:54 +00:00
21054c5d14
YARN-2432. RMStateStore should process the pending events before close. Contributed by Varun Saxena
...
(cherry picked from commit 9b860661a5
2014-11-14 17:25:21 -08:00
c4c77669f0
YARN-2811. In Fair Scheduler, reservation fulfillments shouldn't ignore max share (Siqi Li via Sandy Ryza)
2014-11-14 15:20:37 -08:00
beb184ac58
YARN-2856. Fixed RMAppImpl to handle ATTEMPT_KILLED event at ACCEPTED state on app recovery. Contributed by Rohith Sharmaks
...
(cherry picked from commit d005404ef7
2014-11-13 15:34:56 -08:00
ed3e5cb164
YARN-2853. Fixed a bug in ResourceManager causing apps to hang when the user kill request races with ApplicationMaster finish. Contributed by Jian He.
...
(cherry picked from commit 3651fe1b08
2014-11-13 08:13:57 -08:00
6984d899e3
YARN-2780. Log aggregated resource allocation in rm-appsummary.log. Contributed by Eric Payne
...
(cherry picked from commit f8aefa5e9c
2014-11-12 17:02:30 +00:00
adfb830a2b
Preparing to release hadoop-2.6.0: Set version in branch-2 to 2.7.0-SNAPSHOT.
2014-11-09 19:19:02 -08:00
d76fc94b21
YARN-2834. Fixed ResourceManager to ignore token-renewal failures on recovery consistent with the (somewhat incorrect) behaviour in the non-recovery case. Contributed by Jian He.
...
(cherry picked from commit e76faebc95
2014-11-09 18:57:48 -08:00
175d222bfc
YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles.
...
(cherry picked from commit 43cd07b408
2014-11-09 15:03:59 -08:00
1880a5a7c3
YARN-2505. Supported get/add/remove/change labels in RM REST API. Contributed by Craig Welch.
...
(cherry picked from commit 9a4e0d343e
2014-11-07 20:39:53 -08:00
b865a95f06
YARN-2826. Fixed user-groups mappings' refresh bug caused by YARN-2826. Contributed by Wangda Tan.
...
(cherry picked from commit df36edf751
2014-11-07 19:44:52 -08:00
a50345f654
YARN-2753. Fixed a bunch of bugs in the NodeLabelsManager classes. Contributed by Zhihai xu.
...
(cherry picked from commit 4cfd5bc7c1
2014-11-07 14:16:54 -08:00
d5d2905262
YARN-2824. Fixed Capacity Scheduler to not crash when some node-labels are not mapped to queues by making default capacities per label to be zero. Contributed by Wangda Tan.
...
(cherry picked from commit 2ac1be7dec
2014-11-07 10:44:25 -08:00
bf79541868
YARN-2810. TestRMProxyUsersConf fails on Windows VMs. Contributed by Varun Vasudev
...
(cherry picked from commit 1e97f2f094
2014-11-07 09:46:04 -08:00
eb01c602ed
YARN-2823. Fixed ResourceManager app-attempt state machine to inform schedulers about previous finished attempts of a running appliation to avoid expectation mismatch w.r.t transferred containers. Contributed by Jian He.
...
(cherry picked from commit a5657182a7
2014-11-07 09:30:04 -08:00
52e57a95d9
YARN-2744. Fixed CapacityScheduler to validate node-labels correctly against queues. Contributed by Wangda Tan.
...
(cherry picked from commit a3839a9fbf
2014-11-06 17:29:12 -08:00
f92ff24f5e
YARN-2579. Fixed a deadlock issue when EmbeddedElectorService and FatalEventDispatcher try to transition RM to StandBy at the same time. Contributed by Rohith Sharmaks
...
(cherry picked from commit 395275af86
2014-11-05 17:03:01 -08:00
3bc31e35a6
YARN-2805. Fixed ResourceManager to load HA configs correctly before kerberos login. Contributed by Wangda Tan.
...
(cherry picked from commit 834e931d8e
2014-11-05 15:31:41 -08:00
f3ed97de3a
YARN-2767. Added a test case to verify that http static user cannot kill or submit apps in the secure mode. Contributed by Varun Vasudev.
...
(cherry picked from commit 7a4c92a9d55fcecef066053ac30dff0fcd4ec90c)
2014-11-05 10:59:05 -08:00
3e4b280de7
YARN-2010. Handle app-recovery failures gracefully. (Jian He and Karthik Kambatla via kasha)
...
(cherry picked from commit b2cd269802
2014-11-04 17:48:49 -08:00
aa09c4c342
HADOOP-10563. Remove the dependency of jsp in trunk. Contributed by Haohui Mai.
2014-11-04 14:52:51 -08:00
277141b82d
YARN-2795. Fixed ResourceManager to not crash loading node-label data from HDFS in secure mode. Contributed by Wangda Tan.
...
(cherry picked from commit ec6cbece8e
2014-11-03 13:45:31 -08:00
caae0a0a08
YARN-2785. Fixed intermittent TestContainerResourceUsage failure. Contributed by Varun Vasudev.
...
(cherry picked from commit 27715ec63b
2014-11-02 15:22:54 -08:00
97694e78ad
YARN-2779. Fixed ResourceManager to not require delegation tokens for communicating with Timeline Service. Contributed by Zhijie Shen.
...
(cherry picked from commit d1828d9443
2014-10-30 23:17:16 -07:00
0ba8d8c19c
YARN-2698. Moved some node label APIs to be correctly placed in client protocol. Contributed by Wangda Tan.
...
(cherry picked from commit e0233c16eb
2014-10-30 23:02:06 -07:00
d2ba115f06
YARN-2712. TestWorkPreservingRMRestart: Augment FS tests with queue and headroom checks. (Tsuyoshi Ozawa via kasha)
...
(cherry picked from commit 179cab81e0
2014-10-30 00:39:34 -07:00
e88832dfb3
YARN-2742. FairSchedulerConfiguration should allow extra spaces between value and unit. (Wei Yan via kasha)
...
(cherry picked from commit 782971ae7a
2014-10-29 10:25:29 -07:00
720de7eb4c
YARN-2503. Added node lablels in web UI. Contributed by Wangda Tan
...
(cherry picked from commit d5e0a09721
2014-10-28 18:00:59 -07:00
0ad33e1483
YARN-2704. Changed ResourceManager to optionally obtain tokens itself for the sake of localization and log-aggregation for long-running services. Contributed by Jian He.
...
(cherry picked from commit a16d022ca4
2014-10-27 15:50:51 -07:00
3a77838535
YARN-2726. CapacityScheduler should explicitly log when an accessible label has no capacity. Contributed by Wangda Tan
2014-10-27 10:28:45 -07:00
36d3f1fcca
YARN-2743. Fixed a bug in ResourceManager that was causing RMDelegationToken identifiers to be tampered and thus causing app submission failures in secure mode. Contributed by Jian He.
...
(cherry picked from commit 0186645505
2014-10-26 11:15:45 -07:00
96a6e02d16
YARN-1915. Fixed a race condition that client could use the ClientToAMToken to contact with AM before AM actually receives the ClientToAMTokenMasterKey. Contributed by Jason Lowe
...
(cherry picked from commit 5864dd99a4
2014-10-24 22:48:48 -07:00
3b03ea6b50
YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He.
...
(cherry picked from commit 0f3b6900be
2014-10-23 22:03:30 -07:00
4b9de2cdfe
YARN-2721. Suppress NodeExist exception thrown by ZKRMStateStore when it retries creating znode. Contributed by Jian He.
...
(cherry picked from commit 7e3b5e6f5c
2014-10-21 21:52:01 -07:00
77dc9afe86
YARN-2715. Fixed ResourceManager to respect common configurations for proxy users/groups beyond just the YARN level config. Contributed by Zhijie Shen.
...
(cherry picked from commit c0e034336c
2014-10-21 20:10:45 -07:00
9915d52185
YARN-2504. Enhanced RM Admin CLI to support management of node-labels. Contribyted by Wangda Tan.
...
(cherry picked from commit 8256766498
2014-10-18 12:08:28 -07:00
ea841eb25d
YARN-2676. Enhanced Timeline auth-filter to support proxy users. Contributed by Zhijie Shen.
...
(cherry picked from commit 39063cd36f
2014-10-17 22:03:43 -07:00
1c52b6551b
YARN-2705. Fixed bugs in ResourceManager node-label manager that were causing test-failures: added a dummy in-memory labels-manager. Contributed by Wangda Tan.
...
(cherry picked from commit e9c66e8fd2
2014-10-17 18:27:04 -07:00
78a57087bf
YARN-1879. Marked Idempotent/AtMostOnce annotations to ApplicationMasterProtocol for RM fail over. Contributed by Tsuyoshi OZAWA
...
(cherry picked from commit c3de2412eb
2014-10-17 16:36:15 -07:00
79428e021f
YARN-2588. Standby RM fails to transitionToActive if previous transitionToActive failed with ZK exception. Contributed by Rohith Sharmaks
...
(cherry picked from commit a6aa6e42ca
2014-10-17 10:55:04 -07:00
99ce4277a8
YARN-2699. Fixed a bug in CommonNodeLabelsManager that caused tests to fail when using ephemeral ports on NodeIDs. Contributed by Wangda Tan.
...
(cherry picked from commit abae63caf9
2014-10-17 08:59:22 -07:00
76cf2250de
YARN-2685. Fixed a bug in CommonNodeLabelsManager that caused wrong resource tracking per label when a host runs multiple node-managers. Contributed by Wangda Tan.
...
(cherry picked from commit b3056c266a
2014-10-15 18:48:36 -07:00
e8e3a36213
YARN-2496. Enhanced Capacity Scheduler to have basic support for allocating resources based on node-labels. Contributed by Wangda Tan.
...
YARN-2500. Ehnaced ResourceManager to support schedulers allocating resources based on node-labels. Contributed by Wangda Tan.
(cherry picked from commit f2ea555ac6
2014-10-15 18:34:19 -07:00
f93d2ea27e
YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA
2014-10-15 15:28:26 -07:00
bec165e163
YARN-2656. Made RM web services authentication filter support proxy user. Contributed by Varun Vasudev and Zhijie Shen.
...
(cherry picked from commit 1220bb72d4
2014-10-14 21:56:44 -07:00
8aeda20b25
HADOOP-11181. Generalized o.a.h.s.t.d.DelegationTokenManager to handle all sub-classes of AbstractDelegationTokenIdentifier. Contributed by Zhijie Shen.
...
(cherry picked from commit cdce88376a
2014-10-14 11:37:28 -07:00
a1116b56a4
YARN-2641. Decommission nodes on -refreshNodes instead of next NM-RM heartbeat. (Zhihai Xu via kasha)
...
(cherry picked from commit da709a2eac
2014-10-13 16:24:00 -07:00
536254f5e7
YARN-2308. Changed CapacityScheduler to explicitly throw exception if the queue
...
to which the apps were submitted is changed across RM restart. Contributed by Craig Welch & Chang Li
(cherry picked from commit f9680d9a16
2014-10-13 14:16:12 -07:00
e51ae64761
YARN-2651. Spun off LogRollingInterval from LogAggregationContext. Contributed by Xuan Gong.
...
(cherry picked from commit 4aed2d8e91
2014-10-13 10:55:09 -07:00
8ef83a9480
YARN-2494. Added NodeLabels Manager internal API and implementation. Contributed by Wangda Tan.
...
(cherry picked from commit db7f165319
2014-10-10 11:45:15 -07:00
5e6f86e3aa
YARN-2649. Fixed TestAMRMRPCNodeUpdates test failure. Contributed by Ming Ma
...
(cherry picked from commit e16e25ab1b
2014-10-08 11:03:00 -07:00
9ff7c0c96c
YARN-1857. CapacityScheduler headroom doesn't account for other AM's running. Contributed by Chen He and Craig Welch
...
(cherry picked from commit 30d56fdbb4
2014-10-07 13:48:03 -07:00
5c33e91229
YARN-2644. Fixed CapacityScheduler to return up-to-date headroom when AM allocates. Contributed by Craig Welch
...
(cherry picked from commit 519e5a7dd2
2014-10-06 15:51:38 -07:00
43358be60b
YARN-2615. Changed ClientToAMTokenIdentifier/RM(Timeline)DelegationTokenIdentifier to use protobuf as payload. Contributed by Junping Du
...
(cherry picked from commit ea26cc0b4a
2014-10-06 10:48:27 -07:00
4ba102bdc3
YARN-1051. Add a system for creating reservations of cluster capacity.
...
Contributed by Subru Krishnan and Carlo Curino.
(cherry picked from commit c8212bacb1
2014-10-06 10:30:53 -07:00
fb5e9df7fd
YARN-2611. Fixing jenkins findbugs warning and TestRMWebServicesCapacitySched for branch YARN-1051. Contributed by Subru Krishnan and Carlo Curino.
...
(cherry picked from commit c47464aba4a2986234be
2014-10-06 10:21:07 -07:00
ae0f16ccc8
YARN-2576. Making test patch pass in branch. Contributed by Subru Krishnan and Carlo Curino.
...
(cherry picked from commit 90ac0be86b5e10a13bb4
2014-10-06 10:21:06 -07:00
Karthik Kambatla
Zhijie Shen
Tsuyoshi Ozawa
Jian He
Junping Du
Jason Lowe
Steve Loughran
Devaraj K
Sandy Ryza
Vinod Kumar Vavilapalli
Arun C. Murthy
Xuan
Haohui Mai
Chris Douglas
subru