da456ffd62
Preparing for 3.0.0-alpha2 development
2016-07-15 19:04:17 -07:00
f5f1c81e7d
YARN-5272. Handle queue names consistently in FairScheduler. (Wilfred Spiegelenburg via rchiang)
2016-07-15 14:38:50 -07:00
e905a42a2c
YARN=5181. ClusterNodeTracker: add method to get list of nodes matching a specific resourceName. (kasha via asuresh)
2016-07-15 14:35:12 -07:00
24db9167f1
YARN-4484. Available Resource calculation for a queue is not correct when used with labels. (Sunil G via wangda)
2016-07-15 11:40:12 -07:00
d6d41e820a
YARN-5362. TestRMRestart#testFinishedAppRemovalAfterRMRestart can fail. Contributed by sandflee.
2016-07-13 19:12:35 +05:30
06c56ff79b
YARN-5353. ResourceManager can leak delegation tokens when they are shared across apps. (Jason Lowe via Varun Saxena).
2016-07-13 07:55:34 +05:30
10b704c594
YARN-5317. testAMRestartNotLostContainerCompleteMsg may fail. Contributed by sandflee
2016-07-12 20:27:41 +00:00
819224dcf9
YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan
2016-07-11 22:36:20 -07:00
0fd3980a1f
YARN-5037. Fix random failure of TestRMRestart#testQueueMetricsOnRMRestart (sandflee via Varun Saxena).
2016-07-10 21:28:52 +05:30
6cf6ab7b78
Made a number of miscellaneous fixes for javac, javadoc, and checstyle warnings.
2016-07-10 08:46:05 -07:00
6d943038f6
Cleanup changes during rebase with trunk (Vrushali C)
2016-07-10 08:46:04 -07:00
1ff6833bba
YARN-5243. fix several rebase and other miscellaneous issues before merge. (Sangjin Lee via Varun Saxena)
2016-07-10 08:46:03 -07:00
0a9b085f05
YARN-5189. Make HBaseTimeline[Reader|Writer]Impl default and move FileSystemTimeline*Impl. (Joep Rottinghuis and Sangjin Lee via gtcarrera9)
2016-07-10 08:46:01 -07:00
702236129b
YARN-5095. flow activities and flow runs are populated with wrong timestamp when RM restarts w/ recovery enabled (Varun Saxena via sjlee)
2016-07-10 08:46:00 -07:00
a1b6d7456f
YARN-5018. Online aggregation logic should not run immediately after collectors got started (Li Lu via sjlee)
2016-07-10 08:45:59 -07:00
c2055a97d5
YARN-3150. Documenting the timeline service v2. (Sangjin Lee and Vrushali C via gtcarrera9)
2016-07-10 08:45:57 -07:00
a3cf40e532
YARN-3461. Consolidate flow name/version/run defaults. (Sangjin Lee via Varun Saxena)
2016-07-10 08:45:55 -07:00
960af7d471
YARN-4409. Fix javadoc and checkstyle issues in timelineservice code (Varun Saxena via sjlee)
2016-07-10 08:45:53 -07:00
06f0b50a28
YARN-4644. TestRMRestart fails and findbugs issue in YARN-2928 branch (Varun Saxena via Naganarasimha G R)
2016-07-10 08:45:52 -07:00
6934b05c71
YARN-4238. createdTime and modifiedTime is not reported while publishing entities to ATSv2. (Varun Saxena via Naganarasimha G R)
2016-07-10 08:45:52 -07:00
34f02f07d5
Rebase to latest trunk
2016-07-10 08:45:51 -07:00
829cceebc0
YARN-3586. RM to only get back addresses of Collectors that NM needs to know.
...
(Junping Du via Varun Saxena).
2016-07-10 08:45:50 -07:00
8ef546c1ee
YARN-4445. Unify the term flowId and flowName in timeline v2 codebase.
...
Contributed by Zhan Zhang.
2016-07-10 08:45:49 -07:00
c4d7bbda5c
YARN-4460. [Bug fix] RM fails to start when SMP is enabled. (Li Lu via Varun Saxena)
2016-07-10 08:45:49 -07:00
2e2dbf59d1
YARN-4392. ApplicationCreatedEvent event time resets after RM
...
restart/failover. Contributed by Naganarasimha G R and Xuan Gong
(cherry picked from commit 4546c7582b
2016-07-10 08:45:49 -07:00
89e5c44f9e
YARN-4356. Ensure the timeline service v.2 is disabled cleanly and has no
...
impact when it's turned off. Contributed by Sangjin Lee.
2016-07-10 08:45:48 -07:00
10ec5586fb
YARN-4129. Refactor the SystemMetricPublisher in RM to better support newer events (Naganarasimha G R via sjlee)
2016-07-10 08:45:46 -07:00
8d9476ec5f
YARN-4058. Miscellaneous issues in NodeManager project (Naganarasimha G R via sjlee)
2016-07-10 08:45:43 -07:00
22e7ae5771
YARN-3792. Test case failures in TestDistributedShell and some issue fixes related to ATSV2 (Naganarasimha G R via sjlee)
...
(cherry picked from commit 84f37f1c7eefec6d139cbf091c50d6c06f734323)
2016-07-10 08:45:38 -07:00
f3c661e8dd
YARN-3044. Made RM write app, attempt and optional container lifecycle events to timeline service v2. Contributed by Naganarasimha G R.
2016-07-10 08:45:37 -07:00
dc1f306fdc
YARN-3562. unit tests failures and issues found from findbug from earlier ATS checkins (Naganarasimha G R via sjlee)
2016-07-10 08:45:35 -07:00
11e8905d8d
YARN-3390. Reuse TimelineCollectorManager for RM (Zhijie Shen via sjlee)
...
(cherry picked from commit 58221188811e0f61d842dac89e1f4ad4fd8aa182)
2016-07-10 08:45:33 -07:00
47f35a30bb
YARN-3391. Clearly define flow ID/ flow run / flow version in API and storage. Contributed by Zhijie Shen
...
(cherry picked from commit 68c6232f8423e55b4d152ef3d1d66aeb2d6a555e)
2016-07-10 08:45:33 -07:00
5712b8f9fd
YARN-3334. NM uses timeline client to publish container metrics to new timeline service. Contributed by Junping Du.
2016-07-10 08:45:33 -07:00
d67c9bdb4d
YARN-3040. Make putEntities operation be aware of the app's context. Contributed by Zhijie Shen
2016-07-10 08:45:32 -07:00
5e3d9a477b
YARN-3034. Implement RM starting its timeline collector. Contributed by Naganarasimha G R
2016-07-10 08:45:32 -07:00
2188a07e5b
YARN-3333. Rename TimelineAggregator etc. to TimelineCollector. Contributed by Sangjin Lee
2016-07-10 08:45:31 -07:00
9b56364080
YARN-3039. Implemented the app-level timeline aggregator discovery service. Contributed by Junping Du.
2016-07-10 08:45:31 -07:00
c04c5ec501
YARN-5318. Fix intermittent test failure of TestRMAdminService#testRefreshNodesResourceWithFileSystemBasedConfigurationProvider. (Jun Gong via Varun Saxena).
2016-07-09 01:13:18 +05:30
5252562edf
YARN-5297. Avoid printing a stack trace when recovering an app after the RM restarts. (Junping Du via Varun Saxena).
2016-07-09 00:09:25 +05:30
30ee57ceb1
YARN-4939. The decommissioning Node should keep alive during NM restart. Contributed by sandflee.
2016-07-08 04:14:53 -07:00
04f6ebb66a
YARN-5294. Pass remote ip address down to YarnAuthorizationProvider. (Jian He via wangda)
2016-07-06 10:36:48 -07:00
8e672e3c71
YARN-5286. Add RPC port info in RM web service's response when getting app status. (Jun Gong via Varun Saxena).
2016-07-05 22:56:07 +05:30
c35a5a7a8d
YARN-5023. TestAMRestart#testShouldNotCountFailureToMaxAttemptRetry fails. Contributed by sandflee
2016-07-01 14:29:03 -07:00
abe7fc22c1
YARN-5182. MockNodes.newNodes creates one more node per rack than requested. (Karthik Kambatla via Varun Saxena).
2016-06-30 00:13:28 +05:30
26b5e6116f
YARN-5262. Optimize sending RMNodeFinishedContainersPulledByAMEvent for every AM heartbeat.
2016-06-29 10:08:30 +05:30
a8a48c9125
YARN-5278. Remove unused argument in TestRMWebServicesForCSWithPartitions#setupQueueConfiguration. Contributed by Tao Jie.
2016-06-23 14:28:12 +09:00
99e5dd68d0
YARN-5171. Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node. (Inigo Goiri via asuresh)
2016-06-22 19:04:54 -07:00
5d58858bb6
HADOOP-9613. [JDK8] Update jersey version to latest 1.x release.
2016-06-21 08:05:32 +09:00
d0162f2040
YARN-5246. NMWebAppFilter web redirects drop query parameters. Contributed by Varun Vasudev.
2016-06-19 17:44:54 -07:00
20f2799938
YARN-5077. Fix FSLeafQueue#getFairShare() for queues with zero fairshare. (Yufei Gu via kasha)
2016-06-17 22:24:42 -07:00
fbbe0bb627
YARN-5082. Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit. Addendum to fix javac warning. (Arun Suresh via kasha)
2016-06-17 22:12:50 -07:00
c77a1095dc
YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda)
2016-06-14 15:06:38 -07:00
28b66ae919
YARN-4989. TestWorkPreservingRMRestart#testCapacitySchedulerRecovery fails intermittently. Contributed by Ajith S.
2016-06-13 11:09:32 +05:30
5279af7cd4
YARN-5082. Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit (sandflee via asuresh)
2016-06-10 22:33:42 -07:00
e0f4620cc7
YARN-5197. RM leaks containers if running container disappears from node update. Contributed by Jason Lowe.
2016-06-11 10:22:27 +05:30
244506f9c8
YARN-5208. Run TestAMRMClient TestNMClient TestYarnClient TestClientRMTokens TestAMAuthorization tests with hadoop.security.token.service.use_ip enabled. (Rohith Sharma K S via wangda)
2016-06-10 09:34:32 -07:00
620325e816
YARN-4837. User facing aspects of 'AM blacklisting' feature need fixing. (vinodkv via wangda)
2016-06-07 15:06:42 -07:00
3a154f75ed
YARN-4525. Fix bug in RLESparseResourceAllocation.getRangeOverlapping(). (Ishai Menache and Carlo Curino via asuresh)
2016-06-06 21:18:32 -07:00
7a9b7372a1
YARN-5185. StageAllocaterGreedyRLE: Fix NPE in corner case. (Carlo Curino via asuresh)
2016-06-06 21:06:52 -07:00
4a1cedc010
MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma)
2016-06-06 14:30:51 -07:00
db54670e83
YARN-5165. Fix NoOvercommitPolicy to take advantage of RLE representation of plan. (Carlo Curino via asuresh)
2016-06-03 14:49:32 -07:00
f10ebc67f5
YARN-5098. Fixed ResourceManager's DelegationTokenRenewer to replace expiring system-tokens if RM stops and only restarts after a long time. Contributed by Jian He.
2016-06-03 13:00:07 -07:00
097baaaeba
YARN-1815. Work preserving recovery of Unmanged AMs. Contributed by Subru Krishnan
2016-06-03 10:49:30 -07:00
dc26601d8f
YARN-5180. Allow ResourceRequest to specify an enforceExecutionType flag. (asuresh)
2016-06-02 09:01:02 -07:00
42f90ab885
YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan.
2016-05-29 21:24:16 +05:30
aa975bc781
YARN-5127. Expose ExecutionType in Container api record. (Hitesh Sharma via asuresh)
2016-05-27 14:06:32 -07:00
916140604f
HADOOP-12911. Upgrade Hadoop MiniKDC with Kerby. Contributed by Jiajia Li
2016-05-28 14:23:39 +08:00
0a544f8a3e
YARN-5005. TestRMWebServices#testDumpingSchedulerLogs fails randomly. Contributed by Bibin A Chundatt.
2016-05-27 10:44:35 +05:30
5b41b288d0
YARN-5162. Fix Exceptions thrown during in registerAM call when Distributed Scheduling is Enabled (Hitesh Sharma via asuresh)
2016-05-26 14:56:37 -07:00
04ded558b0
YARN-5035. FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on. (kasha)
2016-05-26 14:41:07 -07:00
4f513a4a8e
YARN-4866. FairScheduler: AMs can consume all vcores leading to a livelock when using FAIR policy. (Yufei Gu via kasha)
2016-05-25 22:13:27 -07:00
013532a95e
YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino)
2016-05-25 16:55:49 -07:00
28bd63e92b
YARN-5024. TestContainerResourceUsage#testUsageAfterAMRestartWithMultipleContainers random failure. Contributed by Bibin A Chundatt
2016-05-25 10:15:50 +05:30
edd716e99c
YARN-5114. Add additional tests in TestRMWebServicesApps and rectify testInvalidAppAttempts failure in 2.8. Contributed by Bibin A Chundatt
2016-05-25 06:11:38 +08:00
f979d779e1
YARN-4878. Expose scheduling policy and max running apps over JMX for Yarn queues. (Yufei Gu via kasha)
2016-05-24 10:54:11 -07:00
b4078bd17b
YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (addendum patch). Contributed by Bibin A chundatt
2016-05-24 08:06:53 +08:00
6d043aa4cf
YARN-4979. FSAppAttempt demand calculation considers demands at multiple locality levels different. (Zhihai Xu via kasha)
2016-05-23 14:29:28 -07:00
ac954486c5
YARN-5055. max apps per user can be larger than max per queue. Contributed by Eric Badger
2016-05-23 15:54:42 +00:00
22fcd819f0
YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron.
...
(cherry picked from commit 2703ec6871
2016-05-19 14:15:21 -07:00
feb90ffcca
YARN-4002. Make ResourceTrackerService#nodeHeartbeat more concurrent. Contributed by Rohith Sharma K S & Zhiguo Hong
2016-05-19 13:01:36 -07:00
1597630681
YARN-5110. Fix OpportunisticContainerAllocator to insert complete HostAddress in issued ContainerTokenIds. (Konstantinos Karanasos via asuresh)
2016-05-18 18:46:00 -07:00
8a9ecb7584
YARN-5090. Add End-to-End test-cases for DistributedScheduling using MiniYarnCluster. (asuresh)
2016-05-17 19:01:29 -07:00
fa3bc3405d
YARN-4832. NM side resource value should get updated if change applied in RM side. Contributed by Junping Du
2016-05-17 12:52:19 -07:00
ccc93e7812
YARN-5075. Fix findbugs warnings in hadoop-yarn-common module. (asuresh)
2016-05-16 23:22:01 -07:00
1217c8f6b4
YARN-5069. TestFifoScheduler.testResourceOverCommit race condition. Contributed by Eric Badger.
2016-05-16 20:28:04 +00:00
f45bc5a83e
YARN-4738. Notify the RM about the status of OPPORTUNISTIC containers (Konstantinos Karanasos via asuresh)
2016-05-15 17:54:34 -07:00
f0ac18d001
YARN-2888. Corrective mechanisms for rebalancing NM container queues. (asuresh)
2016-05-13 13:38:36 -07:00
3c5c57af28
HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1.
2016-05-12 18:27:28 -07:00
ca5613af91
Revert "Update project version to 3.0.0-alpha1-SNAPSHOT."
...
This reverts commit 6b53802cba
2016-05-12 15:32:45 -07:00
013000fbc2
YARN-5053. More informative diagnostics when applications killed by a user. Contributed by Eric Badger
2016-05-12 20:28:36 +00:00
6b53802cba
Update project version to 3.0.0-alpha1-SNAPSHOT.
2016-05-12 11:05:05 -07:00
b7ac85259c
YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks)
2016-05-12 15:17:49 +05:30
4b4e4c6ba8
YARN-4995. FairScheduler: Display per-queue demand on the scheduler page. (xupeng via kasha)
2016-05-11 17:36:21 -07:00
39f2bac38b
YARN-5029. RM needs to send update event with YarnApplicationState as Running to ATS/AHS. Contributed by Xuan Gong.
2016-05-11 09:28:35 -07:00
2750fb900f
YARN-4926. Change nodelabel rest API invalid reponse status to 400. Contributed by Bibin A Chundatt
2016-05-08 22:49:25 +05:30
47c41e7ac7
YARN-5048. DelegationTokenRenewer#skipTokenRenewal may throw NPE (Jian He via Yongjun Zhang)
2016-05-06 21:50:09 -07:00
b2ed6ae731
YARN-4747. AHS error 500 due to NPE when container start event is missing. Contributed by Varun Saxena
2016-05-06 22:59:39 +00:00
23248f63aa
getApplicationReport call may raise NPE for removed queues. (Jian He via wangda)
2016-05-06 15:30:45 -07:00
bb62e05925
YARN-4390. Do surgical preemption based on reserved container in CapacityScheduler. Contributed by Wangda Tan
2016-05-05 12:56:21 -07:00
d0da13229c
YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla
2016-05-05 14:07:54 +00:00
75e0450593
YARN-4947. Test timeout is happening for TestRMWebServicesNodes. Contributed by Bibin A Chundatt
2016-05-04 09:58:26 +05:30
ed54f5f1ff
YARN-5003. Add container resource to RM audit log. Contributed by Nathan Roberts
2016-05-03 20:03:41 +00:00
dd80042c42
YARN-5008. LeveldbRMStateStore database can grow substantially leading to long recovery times. Contributed by Jason Lowe
2016-04-28 21:27:25 -07:00
185c3d4de1
YARN-4807. MockAM#waitForState sleep duration is too long. (Yufei Gu via kasha)
2016-04-27 09:43:23 -07:00
4beff01354
YARN-4983. JVM and UGI metrics disappear after RM transitioned to standby mode
2016-04-26 21:00:17 -07:00
341888a0aa
YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh)
2016-04-26 20:12:12 -07:00
4b1dcbbe0c
YARN-1297. FairScheduler: Move some logs to debug and check if debug logging is enabled
2016-04-26 05:10:09 -07:00
c282a08f38
YARN-2885. Create AMRMProxy request interceptor and ContainerAllocator to distribute OPPORTUNISTIC containers to appropriate Nodes (asuresh)
...
(cherry picked from commit 2bf025278a318b0452fdc9ece4427b4c42124e39)
2016-04-24 22:38:33 -07:00
7cb3a3da96
YARN-4846. Fix random failures for TestCapacitySchedulerPreemption#testPreemptionPolicyShouldRespectAlreadyMarkedKillableContainers. (Bibin A Chundatt via wangda)
2016-04-22 11:40:32 -07:00
3dce486d88
YARN-4556. TestFifoScheduler.testResourceOverCommit fails. Contributed by Akihiro Suda
2016-04-21 21:16:47 +00:00
7c6339f66a
YARN-4968. A couple of AM retry unit tests need to wait SchedulerApplicationAttempt stopped. (Wangda Tan via gtcarrera9)
2016-04-21 13:25:33 -07:00
170c4fd4cd
YARN-4784. Fairscheduler: defaultQueueSchedulingPolicy should not accept FIFO. (Yufei Gu via kasha)
2016-04-20 23:58:12 -07:00
33fd95a99c
YARN-4890. Unit test intermittent failure: TestNodeLabelContainerAllocation#testQueueUsedCapacitiesUpdate. (Sunil G via wangda)
2016-04-20 17:37:38 -07:00
fdc46bfb37
YARN-4934. Reserved Resource for QueueMetrics needs to be handled correctly in few cases. (Sunil G via wangda)
2016-04-16 22:47:41 -07:00
69f3d428d5
YARN-4940. yarn node -list -all failed if RM start with decommissioned node. Contributed by sandflee
2016-04-15 20:36:45 +00:00
2a5da97f81
Revert "YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla"
...
This reverts commit 1cbcd4a491
2016-04-11 15:51:01 +00:00
1ff27f9d12
YARN-4630. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta.
2016-04-11 14:55:03 +09:00
ff95fd547b
YARN-4927. TestRMHA#testTransitionedToActiveRefreshFail fails with FairScheduler. (Bibin A Chundatt via kasha)
2016-04-09 10:31:02 -07:00
ec06957941
YARN-3215. Respect labels in CapacityScheduler when computing headroom. (Naganarasimha G R via wangda)
2016-04-08 15:33:04 -07:00
9cb0c963d2
YARN-4740. AM may not receive the container complete msg when it restarts. Contributed by Jun Gong
2016-04-08 11:20:35 -07:00
93bacda08b
YARN-4769. Add support for CSRF header in the dump capacity scheduler logs and kill app buttons in RM web UI. Contributed by Varun Vasudev
2016-04-06 16:13:47 -07:00
21eb428448
YARN-4699. Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node. (Sunil G via wangda)
2016-04-05 16:24:11 -07:00
6be28bcc46
YARN-4893. Fix some intermittent test failures in TestRMAdminService. Contributed by Brahma Reddy Battula.
2016-04-05 06:57:54 -07:00
1cbcd4a491
YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla
2016-04-05 13:40:19 +00:00
776b549e2a
YARN-4609. RM Nodes list page takes too much time to load. Contributed by Bibin A Chundatt
2016-04-05 14:47:25 +05:30
552237d4a3
YARN-4880. Running TestZKRMStateStorePerf with real zookeeper cluster throws NPE. Contributed by Sunil G
2016-04-05 14:26:19 +05:30
5092c94195
YARN-4746. yarn web services should convert parse failures of appId, appAttemptId and containerId to 400. Contributed by Bibin A Chundatt
2016-04-04 16:25:03 +05:30
1e6f92977d
YARN-4607. Pagination support for AppAttempt page TotalOutstandingResource Requests table. Contributed by Bibin A Chundatt
2016-04-04 08:09:29 +05:30
12b11e2e68
YARN-4634. Scheduler UI/Metrics need to consider cases like non-queue label mappings. (Sunil G via wangda)
2016-03-31 14:35:18 -07:00
7a021471c3
YARN-4639. Remove dead code in TestDelegationTokenRenewer added in YARN-3055 (templedf via rkanter)
2016-03-31 13:09:09 -07:00
60e4116bf1
YARN-4822. Refactor existing Preemption Policy of CS for easier adding new approach to select preemption candidates. Contributed by Wangda Tan
2016-03-30 12:43:52 -07:00
fc055a3cbe
YARN-4865. Track Reserved resources in ResourceUsage and QueueCapacities. (Sunil G via wangda)
2016-03-29 17:07:55 -07:00
524bc3c33a
YARN-998. Keep NM resource updated through dynamic resource config for RM/NM restart. Contributed by Junping Du
2016-03-28 11:12:33 -07:00
49ff54c860
YARN-4805. Don't go through all schedulers in ParameterizedTestBase. (kasha)
2016-03-26 21:45:13 -07:00
00bebb7e58
YARN-4823. Refactor the nested reservation id field in listReservation to simple string field. (subru via asuresh)
2016-03-25 15:54:38 -07:00
d82e797b65
YARN-4825. Remove redundant code in ClientRMService::listReservations. (subru via asuresh)
2016-03-24 09:59:55 -07:00
b1394d6307
YARN-4850. test-fair-scheduler.xml isn't valid xml (Yufei Gu via aw)
2016-03-24 08:15:58 -07:00
19b645c938
YARN-4820. ResourceManager web redirects in HA mode drops query parameters. Contributed by Varun Vasudev.
2016-03-23 19:34:30 -07:00
ca8106d2dd
YARN-4785. inconsistent value type of the type field for LeafQueueInfo in response of RM REST API.
2016-03-17 09:04:41 -07:00
f84af8bd58
YARN-4812. TestFairScheduler#testContinuousScheduling fails intermittently. (kasha)
2016-03-17 05:54:06 -07:00
ae14e5d07f
YARN-4108. CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request. (Wangda Tan)
...
(cherry picked from commit 7e8c9beb41
2016-03-16 17:02:33 -07:00
fa7a43529d
Revert "CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request. (Wangda Tan)"
...
This reverts commit 7e8c9beb41
2016-03-16 17:02:10 -07:00
7e8c9beb41
CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request. (Wangda Tan)
2016-03-16 16:59:59 -07:00
3ef5500783
YARN-4560. Make scheduler error checking message more user friendly. (Ray Chiang via kasha)
2016-03-15 23:45:01 -07:00
20d389ce61
YARN-4719. Add a helper library to maintain node state and allows common queries. (kasha)
2016-03-14 14:19:05 -07:00
0233d4e0ee
YARN-4465. SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled. (Bibin A Chundatt via wangda)
2016-03-08 14:27:03 -08:00
3c33158d1c
YARN-4764. Application submission fails when submitted queue is not available in scheduler xml. Contributed by Bibin A Chundatt
2016-03-08 13:07:57 -08:00
e51a8c1056
YARN-4737. Add CSRF filter support in YARN. Contributed by Jonathan Maron.
2016-03-07 15:26:44 +05:30
e1ccc9622b
YARN-4761. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations on fair scheduler. Contributed by Sangjin Lee
2016-03-06 19:46:09 -08:00
19ee185907
YARN-4763. RMApps Page crashes with NPE. (Bibin A Chundatt via rohithsharmaks)
2016-03-05 13:02:57 +05:30
5c465df904
YARN-4671. There is no need to acquire CS lock when completing a container. Contributed by Meng Ding
2016-03-01 13:14:12 -08:00
9dafaaaf0d
YARN-4704. TestResourceManager#testResourceAllocation() fails when using FairScheduler. (Yufei Gu via kasha)
2016-02-29 16:10:12 -08:00
0fa54d45b1
HADOOP-12813. Migrate TestRPC and related codes to rebase on ProtobufRpcEngine. Contributed by Kai Zheng.
2016-02-29 11:41:00 -08:00
f9692770a5
YARN-4718. Rename variables in SchedulerNode to reduce ambiguity post YARN-1011. (Inigo Goiri via kasha)
2016-02-28 09:35:59 -08:00
6b0f813e89
YARN-4723. NodesListManager$UnknownNodeId ClassCastException. Contributed by Kuhu Shukla
2016-02-26 20:24:50 +00:00
c684f2b007
YARN-4729. SchedulerApplicationAttempt#getTotalRequiredResources can throw an NPE. (kasha)
2016-02-24 18:33:57 -08:00
553b591ba0
YARN-4722. AsyncDispatcher logs redundant event queue sizes (Jason Lowe via sjlee)
2016-02-24 09:29:41 -08:00
9ed17f181d
YARN-3223. Resource update during NM graceful decommission. Contributed by Brook Zhou.
2016-02-23 03:30:26 -08:00
0e12114c9c
YARN-4648. Move preemption related tests from TestFairScheduler to TestFairSchedulerPreemption. Contributed by Kai Sasaki.
2016-02-23 19:50:08 +09:00
3fab88540f
YARN-4386. refreshNodesGracefully() should send recommission event to active RMNodes only. Contributed by Kuhu Shukla.
2016-02-22 07:04:19 -08:00
7de70680fe
YARN-4690. Skip object allocation in FSAppAttempt#getResourceUsage when possible (Ming Ma via sjlee)
2016-02-17 20:55:21 -08:00
2ab4c476ed
YARN-4689. FairScheduler: Cleanup preemptContainer to be more readable. (Kai Sasaki via kasha)
2016-02-17 18:16:15 -08:00
23f937e3b7
YARN-2575. Create separate ACLs for Reservation create/update/delete/list ops (Sean Po via asuresh)
2016-02-11 10:47:43 -08:00
fa00d3e205
YARN-4655. Log uncaught exceptions/errors in various thread pools in YARN. Contributed by Sidharta Seethana.
2016-02-11 12:06:42 +05:30
d16b17b4d2
YARN-4138. Roll back container resource allocation after resource increase token expires. Contributed by Meng Ding
2016-02-11 10:06:27 +08:00
b706cbc1bc
YARN-4420. Add REST API for List Reservations (Sean Po via curino)
2016-02-10 10:19:26 -08:00
5cf5c41a89
YARN-4360. Improve GreedyReservationAgent to support "early" allocations, and performance improvements (curino via asuresh)
2016-02-10 09:11:15 -08:00
565af873d5
YARN-4667. RM Admin CLI for refreshNodesResources throws NPE when nothing
...
is configured. Contributed by Naganarasimha G R.
2016-02-08 15:01:54 +05:30
22a2b2231d
YARN-4669. Fix logging statements in resource manager's Application class. Contributed by Sidharta Seethana.
2016-02-04 13:51:25 +05:30
308d63f382
YARN-4307. Display blacklisted nodes for AM container in the RM web UI. Contributed by Naganarasimha G R.
2016-02-04 13:32:54 +05:30
1adb64e09b
YARN-4625. Make ApplicationSubmissionContext and ApplicationSubmissionContextInfo more consistent. Contributed by Xuan Gong.
2016-02-03 16:26:28 +05:30
9875325d5c
YARN-4340. Add list API to reservation system. (Sean Po via wangda)
2016-02-02 10:17:33 +08:00
ed55950164
YARN-3102. Decommisioned Nodes not listed in Web UI. Contributed by Kuhu Shukla
2016-02-01 23:15:26 +00:00
2673cbaf55
YARN-4615. Fix random test failure in TestAbstractYarnScheduler#testResourceRequestRecoveryToTheRightAppAttempt. (Sunil G via rohithsharmaks)
2016-02-01 10:43:56 +05:30
772ea7b41b
YARN-4428. Redirect RM page to AHS page when AHS turned on and RM page is not available. Contributed by Chang Li
2016-01-29 21:48:54 +00:00
f4a57d4a53
YARN-4617. LeafQueue#pendingOrderingPolicy should always use fixed ordering policy instead of using same as active applications ordering policy. Contributed by Rohith Sharma K S
2016-01-29 12:22:23 -08:00
a277bdc9ed
YARN-4411. RMAppAttemptImpl#createApplicationAttemptReport throws
...
IllegalArgumentException. Contributed by Bibin A Chundatt and yarntime.
2016-01-29 13:51:37 +05:30
7f46636495
YARN-4519. Potential deadlock of CapacityScheduler between decrease container and assign containers. Contributed by Meng Ding
2016-01-28 14:51:00 -08:00
ef343be82b
YARN-4633. Fix random test failure in TestRMRestart#testRMRestartAfterPreemption. (Bibin A Chundatt via rohithsharmaks)
2016-01-28 21:53:45 +05:30
fb238d7e5d
YARN-4462. FairScheduler: Disallow preemption from a queue. (Tao Jie via kasha)
2016-01-27 12:29:06 -08:00
c01bee0108
YARN-4573. Fix test failure in TestRMAppTransitions#testAppRunningKill and testAppKilledKilled. (Takashi Ohnishi via rohithsharmaks)
2016-01-27 08:23:02 +05:30
10dc2c0493
YARN-4613. Fix test failure in TestClientRMService#testGetClusterNodes. (Takashi Ohnishi via rohithsharmaks)
2016-01-24 23:36:15 +05:30
99829eb221
YARN-4614. Fix random failure in TestApplicationPriority#testApplicationPriorityAllocationWithChangeInPriority. (Sunil G via rohithsharmaks)
2016-01-23 07:56:57 +05:30
d6258b33a7
YARN-4497. RM might fail to restart when recovering apps whose attempts are missing. (Jun Gong via rohithsharmaks)
2016-01-22 20:27:38 +05:30
8f58f742ae
YARN-4605. Spelling mistake in the help message of "yarn applicationattempt" command. Contributed by Weiwei Yang.
2016-01-22 19:43:06 +09:00
e30668106d
YARN-4584. RM startup failure when AM attempts greater than max-attempts. (Bibin A Chundatt via rohithsharmaks)
2016-01-22 10:14:46 +05:30
468a53b22f
YARN-4610. Reservations continue looking for one app causes other apps to starve. Contributed by Jason Lowe
2016-01-21 18:31:29 +00:00
4992398aee
YARN-4603. FairScheduler should mention user requested queuename in error message when failed in queue ACL check. (Tao Jie via kasha)
2016-01-21 17:40:59 +01:00
5ff5f67332
YARN-4557. Fix improper Queues sorting in PartitionedQueueComparator when accessible-node-labels=*. (Naganarasimha G R via wangda)
2016-01-21 11:21:06 +08:00
890a2ebd1a
YARN-4559. Make leader elector and zk store share the same curator
...
client. Contributed by Jian He
2016-01-20 14:48:10 -08:00
edc43a9097
YARN-4565. Fix a bug that leads to AM resource limit not hornored when sizeBasedWeight enabled for FairOrderingPolicy. Contributed by Wangda Tan
2016-01-18 21:04:36 -08:00
a44ce3f14f
YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)
2016-01-19 09:30:04 +08:00
150f5ae034
Revert "YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)"
...
This reverts commit 3fe5728563
2016-01-19 09:27:36 +08:00
f385851141
YARN-4596. SystemMetricPublisher should not swallow error messages from TimelineClient#putEntities. Contributed by Li Lu
2016-01-18 16:58:39 -08:00
d40859fab1
YARN-4526. Make SystemClock singleton so AppSchedulingInfo could use it. (kasha)
2016-01-18 10:58:14 +01:00
3fe5728563
YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)
...
(cherry picked from commit 805a9ed85e
2016-01-18 17:06:05 +08:00
adf260a728
Revert "YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)"
...
This reverts commit 805a9ed85e
2016-01-18 16:50:45 +08:00
b08ecf5c75
YARN-4304. AM max resource configuration per partition to be displayed/updated correctly in UI and in various partition related metrics. (Sunil G via wangda)
2016-01-18 11:11:32 +08:00
805a9ed85e
YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)
2016-01-18 11:04:25 +08:00
9523648d57
YARN-4538. QueueMetrics pending cores and memory metrics wrong. (Bibin A Chundatt via wangda)
2016-01-18 10:57:14 +08:00
f7736f464f
YARN-4389. Allow application to enable or disable am blacklisting. (Sunil G via rohithsharmaks)
2016-01-15 21:38:26 +05:30
9d04f26d4c
YARN-3446. FairScheduler headroom calculation should exclude nodes in the blacklist. (Zhihai Xu via kasha)
2016-01-14 08:33:23 -08:00
321072ba81
YARN-4551. Address the duplication between StatusUpdateWhenHealthy and StatusUpdateWhenUnhealthy transitions. (Sunil G via kasha)
2016-01-13 12:09:34 -08:00
c0537bcd2c
YARN-4571. Make app id/name available to the yarn authorizer provider for better auditing. (Jian He via wangda)
2016-01-13 13:18:31 +08:00
da1e3e3c57
YARN-4567. javadoc failing on java 8. Contributed by Steve Loughran. This closes #67 .
2016-01-12 15:12:17 +09:00
9e792da014
YARN-4582. Label-related invalid resource request exception should be able to properly handled by application. (Bibin A Chundatt via wangda)
2016-01-12 12:53:31 +08:00
5fab4ec31c
Missing file for YARN-4580
2016-01-11 17:00:44 -08:00
b8942be888
YARN-4537. Pull out priority comparison from fifocomparator and use compound comparator for FifoOrdering policy. Contributed by Rohith Sharma K S
2016-01-11 16:44:28 -08:00
109e528ef5
YARN-4479. Change CS LeafQueue pendingOrderingPolicy to hornor recovered apps. Contributed by Rohith Sharma K S
2016-01-08 15:51:10 -08:00
89022f8d4b
YARN-4438. Implement RM leader election with curator. Contributed by Jian He
2016-01-07 14:33:06 -08:00
c1462a67ff
YARN-4546. ResourceManager crash due to scheduling opportunity overflow. Contributed by Jason Lowe.
2016-01-06 05:49:24 -08:00
6da6d87872
YARN-4535. Fix checkstyle error in CapacityScheduler.java (Naganarasimha G R via rohithsharmaks)
2016-01-05 12:09:57 +05:30
4e4b3a8465
YARN-4524. Cleanup AppSchedulingInfo. (Karthik Kambatla via wangda)
...
(cherry picked from commit 05fa852d75
2015-12-30 15:39:34 -08:00
8310b2e9ff
YARN-4522. Queue acl can be checked at app submission. (Jian He via wangda)
2015-12-30 15:30:12 -08:00
223ce323bb
YARN-1382. Remove unusableRMNodesConcurrentSet (never used) in NodeListManager to get rid of memory leak. Contributed by Rohith Sharma K S.
2015-12-30 07:52:07 -08:00
5273413411
YARN-3480. Remove attempts that are beyond max-attempt limit from state store. Contributed by Jun Gong
2015-12-29 15:58:39 -08:00
561abb9fee
YARN-4315. NaN in Queue percentage for cluster apps page. (Bibin A Chundatt via wangda)
2015-12-29 13:28:00 -08:00
d0a22bae9b
YARN-4417. Make RM and Timeline-server REST APIs more consistent. Contributed by Wangda Tan
2015-12-28 15:52:45 -08:00
0af492b4bd
YARN-4156. TestAMRestart#testAMBlacklistPreventsRestartOnSameNode assumes CapacityScheduler. (Anubhav Dhoot via kasha)
2015-12-23 17:52:36 -08:00
8c180a13c8
YARN-4109. Exception on RM scheduler page loading with labels. (Mohammad Shahid Khan via rohithsharmaks)
2015-12-23 09:12:32 +05:30
e88422df45
YARN-4477. FairScheduler: Handle condition which can result in an infinite loop in attemptScheduling. (Tao Jie via asuresh)
2015-12-21 22:41:09 -08:00
bc038b382c
YARN-4454. NM to nodelabel mapping going wrong after RM restart. (Bibin A Chundatt via wangda)
2015-12-21 11:30:13 -08:00
85c2466048
YARN-4164. Changed updateApplicationPriority API to return the updated application priority. Contributed by Rohith Sharma K S
2015-12-18 14:13:48 -08:00
1de56b0448
YARN-3226. UI changes for decommissioning node. Contributed by Sunil G.
2015-12-17 15:20:17 -08:00
91828fef6b
YARN-4461. Redundant nodeLocalityDelay log in LeafQueue. Contributed by Eric Payne
2015-12-16 23:22:31 +00:00
9b856d9787
YARN-4416. Deadlock due to synchronised get Methods in AbstractCSQueue. (Naganarasimha G R via wangda)
2015-12-16 13:22:37 -08:00
7faa406f27
YARN-4225. Add preemption status to yarn queue -status for capacity scheduler. (Eric Payne via wangda)
2015-12-16 13:19:40 -08:00
79c41b1d83
YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda)
2015-12-16 13:18:19 -08:00
50bd067e1d
YARN-4452. NPE when submit Unmanaged application. Contributed by Naganarasimha G R.
2015-12-16 10:57:39 -08:00
2aaed10327
YARN-4440. FSAppAttempt#getAllowedLocalityLevelByTime should init the lastScheduler time. Contributed by Lin Yiqun
2015-12-15 00:17:21 -08:00
1cb3299b48
YARN-4403. (AM/NM/Container)LivelinessMonitor should use monotonic time when calculating period. Contributed by Junping Du
2015-12-14 13:51:23 -08:00
07b0fb996a
YARN-4418. AM Resource Limit per partition can be updated to ResourceUsage as well. (Sunil G via wangda)
2015-12-14 11:24:30 -08:00
6cb0af3c39
YARN-3946. Update exact reason as to why a submitted app is in ACCEPTED state to app's diagnostic message. (Naganarasimha G R via wangda)
2015-12-14 10:52:46 -08:00
7fb212e5e6
YARN-4358 addendum patch to fix javadoc error
2015-12-12 22:22:55 -08:00
a5e2e1ecb0
YARN-4421. Remove dead code in RmAppImpl.RMAppRecoveredTransition. (Daniel Templeton via rohithsharmaks)
2015-12-09 11:31:51 +05:30
7e4715186d
YARN-4424. Fix deadlock in RMAppImpl. (Jian he via wangda)
2015-12-08 14:25:16 -08:00
9f50e13d5d
YARN-4248. Followup patch adding asf-licence exclusions for json test files
2015-12-08 12:08:04 -08:00
c25a635459
YARN-4248. REST API for submit/update/delete Reservations. (curino)
2015-12-07 13:33:28 -08:00
4ff973f96a
YARN-4422. Generic AHS sometimes doesn't show started, node, or logs on App page (Eric Payne via jeagles)
2015-12-07 15:04:48 -06:00
4546c7582b
YARN-4392. ApplicationCreatedEvent event time resets after RM
...
restart/failover. Contributed by Naganarasimha G R and Xuan Gong
2015-12-07 12:24:55 -08:00
65f395226b
HADOOP-12321. Make JvmPauseMonitor an AbstractService. (Sunil G via Stevel) [includes HDFS-8947 MAPREDUCE-6462 and YARN-4072]
2015-12-06 17:43:35 +00:00
742632e346
YARN-4358. Reservation System: Improve relationship between SharingPolicy and ReservationAgent. (Carlo Curino via asuresh)
2015-12-05 21:26:16 -08:00
755dda8dd8
YARN-4405. Support node label store in non-appendable file system. Contributed by Wangda Tan
2015-12-03 17:45:31 -08:00
a2c3bfc8c1
YARN-4292. ResourceUtilization should be a part of NodeInfo REST API. (Sunil G via wangda)
2015-12-03 14:28:32 -08:00
9f77ccad73
YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena
2015-12-03 12:48:50 -08:00
6b9a5beb2b
YARN-4398. Remove unnecessary synchronization in RMStateStore. Contributed by Ning Ding
2015-12-02 11:07:18 -08:00
28dfe721b8
YARN-4387. Fix typo in FairScheduler log message. Contributed by Xin Wang.
2015-11-24 19:24:01 +09:00
52948bb20b
YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha)
2015-11-24 13:47:17 +05:30
8676a118a1
YARN-4349. Support CallerContext in YARN. Contributed by Wangda Tan
2015-11-23 17:19:48 -08:00
d36b6e045f
YARN-4344. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations. Contributed by Varun Vasudev
2015-11-23 20:30:26 +00:00
da1016365a
YARN-3454. Add efficient merge operation to RLESparseResourceAllocation (Carlo Curino via asuresh)
2015-11-21 09:59:41 -08:00
2346fa3141
YARN-3769. Consider user limit when calculating total pending resource for preemption policy in Capacity Scheduler. (Eric Payne via wangda)
2015-11-20 15:55:50 -08:00
060cdcbe5d
YARN-4374. RM capacity scheduler UI rounds user limit factor. Contributed by Chang Li
2015-11-20 23:12:29 +00:00
6a61928fb7
YARN-4184. Remove update reservation state api from state store as its not used by ReservationSystem (Sean Po via asuresh)
2015-11-17 15:50:34 -08:00
fcd7888029
Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan"
...
This reverts commit 8fbea531d7
2015-11-16 20:18:44 -08:00
7f55a18071
YARN-4347. Resource manager fails with Null pointer exception. (Jian He via wangda)
2015-11-12 11:23:40 -08:00
796638d9bc
YARN-4287. Capacity Scheduler: Rack Locality improvement (Nathan Roberts via wangda)
2015-11-12 11:09:37 -08:00
6351d3fa63
YARN-4183. Reverting the patch to fix behaviour change.
...
Revert "YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles)"
This reverts commit c293c58954
2015-11-11 10:40:43 -08:00
8fbea531d7
YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan
2015-11-09 10:43:45 -08:00
e5b1733e04
YARN-4127. RM fail with noAuth error if switched from failover to non-failover. Contributed by Varun Saxena
2015-10-29 15:42:57 -07:00
c293c58954
YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles)
2015-10-29 16:41:10 -05:00
58d1df585c
YARN-4310. FairScheduler: Log skipping reservation messages at DEBUG level (asuresh)
2015-10-29 13:42:09 -07:00
656c8f9527
YARN-4130. Duplicate declaration of ApplicationId in RMAppManager#submitApplication method. (Kai Sasaki via rohithsharmaks)
2015-10-29 12:22:44 +05:30
56e4f6237a
YARN-3216. Max-AM-Resource-Percentage should respect node labels. (Sunil G via wangda)
2015-10-26 16:44:39 -07:00
6f606214e7
YARN-4169. Fix racing condition of TestNodeStatusUpdaterForLabels. (Naganarasimha G R via wangda)
2015-10-26 16:36:34 -07:00
3cc73773eb
YARN-4285. Display resource usage as percentage of queue and cluster in the RM UI (Varun Vasudev via wangda)
2015-10-26 13:07:39 -07:00
33a03af3c3
YARN-4284. condition for AM blacklisting is too narrow. Contributed by Sangjin Lee
2015-10-26 19:53:03 +00:00
5acdde4744
YARN-2729. Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup. (Naganarasimha G R via rohithsharmaks)
2015-10-26 15:42:42 +05:30
ab8eb8770c
YARN-3738. Add support for recovery of reserved apps running under dynamic queues (subru via asuresh)
2015-10-24 22:53:10 -07:00
7781fe1b9e
YARN-4294. [JDK8] Fix javadoc errors caused by wrong reference and illegal tag. (aajisaka)
2015-10-24 11:54:42 +09:00
d3a34a4f38
YARN-4041. Slow delegation token renewal can severely prolong RM recovery. Contributed by Sunil G
2015-10-23 20:57:01 +00:00
934d96a334
YARN-2913. Fair scheduler should have ability to set MaxResourceDefault for each queue. (Siqi Li via mingma)
2015-10-23 08:36:33 -07:00
f8adeb712d
YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles)
2015-10-23 10:34:08 -05:00
0fce5f9a49
YARN-4243. Add retry on establishing Zookeeper conenction in EmbeddedElectorService#serviceInit. Contributed by Xuan Gong.
2015-10-22 13:41:09 -07:00
960201b79b
YARN-4256. YARN fair scheduler vcores with decimal values. Contributed by Jun Gong
2015-10-22 12:28:03 -07:00
2798723a54
YARN-3739. Add reservation system recovery to RM recovery process. Contributed by Subru Krishnan.
2015-10-22 06:51:00 -07:00
506d1b1dbc
YARN-3985. Make ReservationSystem persist state using RMStateStore reservation APIs. (adhoot via asuresh)
2015-10-20 16:46:14 -07:00
7e2837f830
YARN-4270. Limit application resource reservation on nodes for non-node/rack specific requests (asuresh)
2015-10-19 20:00:38 -07:00
f9da5cdb2b
YARN-4170. AM need to be notified with priority in AllocateResponse. Contributed by Sunil G
2015-10-16 15:26:27 -07:00
4337b263aa
YARN-4162. CapacityScheduler: Add resource usage by partition and queue capacity by partition to REST API. (Naganarasimha G R via wangda)
2015-10-16 15:06:28 -07:00
cf23f2c2b5
YARN-4000. RM crashes with NPE if leaf queue becomes parent queue during restart. Contributed by Varun Saxena
2015-10-15 17:12:46 -07:00
d6c8bad869
YARN-4250. NPE in AppSchedulingInfo#isRequestLabelChanged. (Brahma Reddy Battula via rohithsharmaks)
2015-10-14 16:11:34 +05:30
9849c8b386
YARN-4230. RM crashes with NPE when increasing container resource if there is no headroom left. Contributed by Meng Ding
2015-10-12 11:51:33 -07:00
049c6e8dc0
YARN-4201. AMBlacklist does not work for minicluster. Contributed by Jun Gong.
2015-10-12 00:14:25 -07:00
db93047881
YARN-3964. Support NodeLabelsProvider at Resource Manager side.
...
Contributed by Dian Fu.
2015-10-11 11:21:29 +05:30
def374e666
YARN-4140. RM container allocation delayed incase of app submitted to Nodelabel partition. (Bibin A Chundatt via wangda)
2015-10-09 16:38:59 -07:00
4aa9b3e75c
MAPREDUCE-6302. Incorrect headroom can lead to a deadlock between map and reduce allocations. (kasha)
2015-10-09 07:37:39 -07:00
a0bca2b5ad
YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S
2015-10-09 14:17:38 +00:00
8f195387a4
YARN-4235. FairScheduler PrimaryGroup does not handle empty groups returned for a user. (Anubhav Dhoot via rohithsharmaks)
2015-10-09 10:09:26 +05:30
3793cbe4c3
YARN-4228. FileSystemRMStateStore use IOUtils#close instead of fs#close. (Bibin A Chundatt via rohithsharmaks)
2015-10-07 10:12:14 +05:30
9156fc60c6
YARN-4209. RMStateStore FENCED state doesn’t work due to updateFencedState called by stateMachine.doTransition. (Zhihai Xu via rohithsharmaks)
2015-10-07 09:34:59 +05:30
29a582ada0
YARN-4215. RMNodeLabels Manager Need to verify and replace node labels for the only modified Node Label Mappings in the request. (Naganarasimha G R via wangda)
2015-10-06 11:56:04 -07:00
c918f7be5e
HADOOP-12458. Retries is typoed to spell Retires in parts of hadoop-yarn and hadoop-common. Contributed by Neelesh Srinivas Salian.
2015-10-03 18:37:58 +05:30
8f08532bde
YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma
2015-10-02 18:50:47 -07:00
a0b5a0a419
YARN-4066. Large number of queues choke fair scheduler. (Johan Gustavsson via kasha)
2015-09-29 07:55:34 -07:00
9735afe967
YARN-4180. AMLauncher does not retry on failures when talking to NM. (adhoot)
2015-09-28 16:13:41 -07:00
9f53a95ff6
YARN-4141. Runtime Application Priority change should not throw exception for applications at finishing states. Contributed by Sunil G
2015-09-28 22:55:20 +00:00
fb2e525c07
YARN-4204. ConcurrentModificationException in FairSchedulerQueueInfo. (adhoot)
2015-09-28 09:05:45 -07:00
a9aafad12b
YARN-4044. Running applications information changes such as movequeue is not published to TimeLine server. (Sunil G via rohithsharmaks)
2015-09-24 12:13:22 +05:30
b3f6b641dc
YARN-4171. Fix findbugs warnings in YARN-1197 branch. Contributed by Wangda Tan
2015-09-23 13:29:38 -07:00
89cab1ba5f
YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan
2015-09-23 13:29:38 -07:00
5f5a968d65
YARN-3867. ContainerImpl changes to support container resizing. Contributed by Meng Ding
2015-09-23 13:29:37 -07:00
83a18add10
YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan)
2015-09-23 13:29:36 -07:00
dfd807afab
HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang.
2015-09-22 12:54:29 +09:00
c9cb6a5960
YARN-4167. NPE on RMActiveServices#serviceStop when store is null. (Bibin A Chundatt via rohithsharmaks)
2015-09-21 09:59:30 +05:30
94dec5a916
YARN-3920. FairScheduler container reservation on a node should be configurable to limit it to large containers (adhoot via asuresh)
2015-09-18 14:02:55 -07:00
9bc913a35c
YARN-3212. RMNode State Transition Update with DECOMMISSIONING state. (Junping Du via wangda)
2015-09-18 10:04:17 -07:00
723c31d45b
YARN-4135. Improve the assertion message in MockRM while failing after waiting for the state.(Nijel S F via rohithsharmaks)
2015-09-18 08:44:10 +05:30
6c6e734f0b
YARN-4034. Render cluster Max Priority in scheduler metrics in RM web UI. Contributed by Rohith Sharma K S
2015-09-17 14:55:50 +08:00
452079af8b
YARN-4078. Add getPendingResourceRequestForAttempt in YarnScheduler interface. Contributed by Naganarasimha G R
2015-09-16 14:59:20 +08:00
ae5308fe1d
YARN-3717. Expose app/am/queue's node-label-expression to RM web UI / CLI / REST-API. (Naganarasimha G R via wangda)
2015-09-15 11:40:50 -07:00
73e3a49eb0
YARN-313. Add Admin API for supporting node resource configuration in command line. (Contributed by Inigo Goiri, Kenji Kikushima and Junping Du)
2015-09-15 07:56:47 -07:00
5468baa80a
YARN-3635. Refactored current queue mapping implementation in CapacityScheduler to use a generic PlacementManager framework. Contributed by Wangda Tan
2015-09-15 15:39:20 +08:00
e1b1d7e4ae
YARN-4126. RM should not issue delegation tokens in unsecure mode. Contributed by Bibin A Chundatt
2015-09-14 14:09:19 +08:00
332b520a48
YARN-3697. FairScheduler: ContinuousSchedulingThread can fail to shutdown. (Zhihai Xu via kasha)
2015-09-13 18:07:43 -07:00
81df7b586a
YARN-2005. Blacklisting support for scheduling AMs. (Anubhav Dhoot via kasha)
2015-09-13 17:03:15 -07:00
7269906254
HADOOP-12087. [JDK8] Fix javadoc errors caused by incorrect or illegal tags. (Akira AJISAKA via stevel).
2015-09-13 14:25:26 +01:00
ea4bb2749f
YARN-4145. Make RMHATestBase abstract so its not run when running all tests under that namespace (adhoot via rkanter)
2015-09-11 11:46:10 -07:00
6f72f1e600
YARN-2884. Added a proxy service in NM to proxy the the communication between AM and RM. Contributed by Kishore Chaliparambil
2015-09-08 09:35:46 +08:00
9b78e6e33d
YARN-4087. Followup fixes after YARN-2019 regarding RM behavior when
...
state-store error occurs. Contributed by Jian He
2015-09-07 17:45:47 -07:00
bcc85e3bab
YARN-4024. YARN RM should avoid unnecessary resolving IP when NMs doing heartbeat. (Hong Zhiguo via wangda)
2015-09-04 15:13:53 -07:00
6eaca2e363
YARN-4105. Capacity Scheduler headroom for DRF is wrong. Contributed by Chang Li
2015-09-04 15:30:53 +00:00
40d222e862
YARN-4103. RM WebServices missing scheme for appattempts logLinks. Contributed by Jonathan Eagles.
2015-09-04 14:31:51 +05:30
b469ac531a
YARN-3970. Add REST api support for Application Priority. Contributed by Naganarasimha G R.
2015-09-03 16:40:10 +05:30
09c64ba1ba
YARN-4101. RM should print alert messages if Zookeeper and Resourcemanager gets connection issue. Contributed by Xuan Gong
2015-09-02 17:45:23 -07:00
7d6687fe76
YARN-3893. Both RM in active state when Admin#transitionToActive failure from refeshAll() (Bibin A Chundatt via rohithsharmaks)
2015-09-02 15:22:48 +05:30
bf669b6d9f
YARN-4082. Container shouldn't be killed when node's label updated. Contributed by Wangda Tan.
2015-09-01 14:19:11 +05:30
a3fd2ccc86
YARN-4092. Fixed UI redirection to print useful messages when both RMs are in standby mode. Contributed by Xuan Gong
2015-08-31 17:33:24 -07:00
beb65c9465
YARN-1556. NPE getting application report with a null appId. Contributed by Weiwei Yang.
2015-08-28 05:57:34 -07:00
a9c8ea71aa
YARN-3250. Support admin cli interface in for Application Priority. Contributed by Rohith Sharma K S
2015-08-27 13:25:53 -07:00
57c7ae1aff
YARN-4014. Support user cli interface in for Application Priority. Contributed by Rohith Sharma K S
2015-08-24 20:36:44 -07:00
feaf034994
YARN-3896. RMNode transitioned from RUNNING to REBOOTED because its response id has not been reset synchronously. (Jun Gong via rohithsharmaks)
2015-08-24 11:25:07 +05:30
37e1c3d82a
YARN-221. NM should provide a way for AM to tell it not to aggregate
...
logs. Contributed by Ming Ma
2015-08-22 16:25:24 -07:00
22de7c1dca
YARN-3986. getTransferredContainers in AbstractYarnScheduler should be present in YarnScheduler interface
2015-08-21 10:51:11 +05:30
22dc5fc209
YARN-4028. AppBlock page key update and diagnostics value null on
...
recovery. Contributed by Bibin A Chundatt
2015-08-18 22:53:03 -07:00
3a76a010b8
YARN-3857: Memory leak in ResourceManager with SIMPLE mode. Contributed by mujunchao.
2015-08-18 10:36:40 -07:00
0a030546e2
YARN-3987. Send AM container completed msg to NM once AM finishes. Contributed by sandflee
2015-08-13 16:22:53 -07:00
7a445fcfab
YARN-4047. ClientRMService getApplications has high scheduler lock contention. Contributed by Jason Lowe
2015-08-13 16:02:57 -07:00
e5003be907
YARN-4026. Refactored ContainerAllocator to accept a list of priorites rather than a single priority. Contributed by Wangda Tan
2015-08-12 15:07:50 -07:00
1c12adb71f
YARN-4023. Publish Application Priority to TimelineServer. (Sunil G via rohithsharmaks)
2015-08-12 14:45:41 +05:30
3ae716fa69
YARN-3999. RM hangs on draing events. Contributed by Jian He
2015-08-11 18:25:11 -07:00
fa1d84ae27
YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G
2015-08-10 20:51:54 -07:00
cf9d3c9256
YARN-3873. PendingApplications in LeafQueue should also use OrderingPolicy. (Sunil G via wangda)
2015-08-10 14:54:55 -07:00
4bc42d76e7
YARN-3966. Fix excessive loggings in CapacityScheduler. (Jian He via wangda)
2015-08-07 09:46:57 -07:00
b6265d39c5
YARN-3948. Display Application Priority in RM Web UI.(Sunil G via rohithsharmaks)
2015-08-07 10:43:41 +05:30
8572a5a14b
YARN-3974. Refactor the reservation system test cases to use parameterized base test. (subru via curino)
2015-08-02 01:55:31 -07:00
cfee02b3bd
YARN-4019. Add JvmPauseMonitor to ResourceManager and NodeManager. Contributed by Robert Kanter.
2015-08-06 06:49:45 -07:00
154c9d2e42
YARN-3961. Expose pending, running and reserved containers of a queue in REST api and yarn top (adhoot via asuresh)
2015-08-05 23:14:14 -07:00
df9e7280db
YARN-3992. TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G)
2015-08-06 10:43:37 +05:30
Andrew Wang
Ray Chiang
Arun Suresh
Wangda Tan
Rohith Sharma K S
Varun Saxena
Jason Lowe
Jian He
Sangjin Lee
Vrushali
Li Lu
Naganarasimha
Xuan
Zhijie Shen
Junping Du
Akira Ajisaka
Tsuyoshi Ozawa
Karthik Kambatla
Ming Ma
Vinod Kumar Vavilapalli
Varun Vasudev
Kai Zheng
Karthik Kambatla
Carlo Curino
Naganarasimha
Eric Payne
Yongjun Zhang
Robert Kanter
Allen Wittenauer
Zhihai Xu
Haohui Mai
Devaraj K
Chris Douglas
Jonathan Eagles
Steve Loughran
Anubhav Dhoot
Harsh J
Carlo Curino