a33ce45e35
YARN-5547. NMLeveldbStateStore should be more tolerant of unknown keys. Contributed by Ajith S
2017-01-24 16:17:36 +00:00
0101267d9d
YARN-6082. Invalid REST api response for getApps since queueUsagePercentage is coming as INF. Contributed by Sunil G.
2017-01-24 10:59:55 +05:30
ce832059db
YARN-5864. Capacity Scheduler - Queue Priorities. (wangda)
2017-01-23 10:52:14 -08:00
69fa81679f
YARN-5910. Support for multi-cluster delegation tokens. Contributed by Jian He
2017-01-23 17:12:51 +00:00
3fa0d540df
YARN-6031. Application recovery has failed when node label feature is turned off during RM recovery. Contributed by Ying Zhang.
2017-01-23 12:19:35 +05:30
b01514f65b
YARN-5928. Move ATSv2 HBase backend code into a new module that is only dependent at runtime by yarn servers. Contributed by Haibo Chen.
2017-01-19 20:52:55 -08:00
5d8b80ea9b
Preparing for 3.0.0-alpha3 development
2017-01-19 15:50:07 -08:00
72054a817d
YARN-5556. CapacityScheduler: Support deleting queues without requiring a RM restart. (Naganarasimha G R via wangda)
2017-01-18 13:31:33 -08:00
e224c96234
YARN-5831. FairScheduler: Propagate allowPreemptionFrom flag all the way down to the app. (Yufei Gu via kasha)
2017-01-17 17:01:31 -08:00
4d1f3d9020
YARN-6016. Fix minor bugs in handling of local AMRMToken in AMRMProxy. (Botong Huang via Subru).
2017-01-17 14:48:03 -08:00
695f110f93
YARN-6071. Fix incompatible API change on AM-RM protocol due to YARN-3866 (trunk only)
...
(Contributed by Wangda Tan via Daniel Templeton)
2017-01-17 12:55:47 -08:00
cf695577aa
YARN-6011. Add a new web service to list the files on a container in AHSWebService. Contributed by Xuan Gong.
2017-01-16 16:20:24 -08:00
d3170f9eba
YARN-6081. LeafQueue#getTotalPendingResourcesConsideringUserLimit should deduct reserved from pending to avoid unnecessary preemption of reserved container. Contributed by Wangda Tan.
2017-01-13 18:22:29 +05:30
a6b06f7179
YARN-6072. RM unable to start in secure mode. Contributed by Ajith S.
2017-01-12 06:40:41 +05:30
e6f13fe5d1
YARN-5849. Automatically create YARN control group for pre-mounted cgroups (Contributed by Miklos Szegedi via Daniel Templeton)
2017-01-11 16:00:01 -08:00
7979939428
YARN-5554. MoveApplicationAcrossQueues does not check user permission on the target queue
...
(Contributed by Wilfred Spiegelenburg via Daniel Templeton)
2017-01-11 14:34:10 -08:00
357eab9566
YARN-5416. TestRMRestart#testRMRestartWaitForPreviousAMToFinish failed intermittently due to not wait SchedulerApplicationAttempt to be stopped. Contributed by Junping Du
2017-01-11 17:13:37 +00:00
4db119b7b5
YARN-6079. Fix simple spelling errors in yarn test code. Contributed by vijay.
2017-01-10 18:20:27 -08:00
4c431a6940
YARN-6054. TimelineServer fails to start when some LevelDb state files are missing. Contributed by Ravi Prakash.
2017-01-10 15:54:16 +05:30
9594c35dcb
YARN-5899. Debug log in AbstractCSQueue#canAssignToThisQueue needs improvement. Contributed by Ying Zhang.
2017-01-10 14:08:01 +05:30
945db55f2e
YARN-4148. When killing app, RM releases app's resource before they are released by NM. Contributed by Jason Lowe.
2017-01-09 18:14:46 -08:00
287d3d6804
YARN-3955. Support for application priority ACLs in queues of CapacityScheduler. (Sunil G via wangda)
2017-01-09 08:40:39 -08:00
f59e36b4ce
YARN-6068. Log aggregation get failed when NM restart even with recovery (Junping Du via Varun Saxena)
2017-01-09 11:17:19 +05:30
ac16400e1f
YARN-6022. Revert changes of AbstractResourceRequest (Contributed by Wangda Tan via Daniel Templeton)
2017-01-07 08:24:30 -08:00
a59df15757
YARN-6015. AsyncDispatcher thread name can be set to improved debugging. Contributed by Ajith S.
2017-01-07 08:11:58 +05:30
020316458d
YARN-6009. Skip validating app timeout value on recovery. Contributed by Rohith Sharma K S
2017-01-06 18:19:09 -08:00
2977bc6a14
YARN-6040. Introduce api independent PendingAsk to replace usage of ResourceRequest within Scheduler classes. (Wangda Tan via asuresh)
2017-01-06 09:59:57 -08:00
0a55bd841e
YARN-5959. RM changes to support change of container ExecutionType. (Arun Suresh via wangda)
2017-01-05 10:31:05 -08:00
e49e0a6e37
YARN-5988. RM unable to start in secure setup. Contributed by Ajith S.
2017-01-04 12:09:29 +05:30
8fadd69047
YARN-5923. Unable to access logs for a running application if YARN_ACL_ENABLE is enabled. Contributed by Xuan Gong.
2017-01-03 15:03:38 -08:00
f69a107aec
YARN-6025. Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda)
...
(cherry picked from commit e0f2379312
2017-01-03 14:53:36 -08:00
ab1faa4ba8
Revert "Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda)" to add correct JIRA number
...
This reverts commit e0f2379312
2017-01-03 14:53:13 -08:00
e0f2379312
Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda)
2017-01-03 14:46:41 -08:00
165d01a73e
YARN-5931. Document timeout interfaces CLI and REST APIs (Contributed by Rohith Sharma K S via Daniel Templeton)
2016-12-30 08:41:52 -08:00
f216276d21
YARN-4882. Change the log level to DEBUG for recovering completed applications (templedf via rkanter)
2016-12-28 15:21:52 -08:00
9ca54f4810
YARN-5906. Update AppSchedulingInfo to use SchedulingPlacementSet. Contributed by Wangda Tan.
2016-12-28 22:48:19 +05:30
972da46cb4
YARN-5719. Enforce a C standard for native container-executor. Contributed by Chris Douglas.
2016-12-28 14:59:57 +05:30
f6715b26b6
YARN-6024. Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to max. Contributed by Wangda Tan.
2016-12-28 12:33:02 +05:30
0840b4329b
YARN-5756. Add state-machine implementation for scheduler queues. (Xuan Gong via wangda)
2016-12-27 21:18:33 -08:00
9262797e86
YARN-5257. Fix unreleased resources and null dereferences (yufeigu via rkanter)
2016-12-27 14:14:08 -08:00
1bbd023275
YARN-5962. Spelling errors in logging and exceptions for resource manager code (gsohn via rkanter)
2016-12-27 13:35:34 -08:00
e216e8e233
HADOOP-13932. Fix indefinite article in comments (Contributed by LiXin Ge via Daniel Templeton)
2016-12-27 13:03:16 -08:00
ac1e5d4f77
YARN-5938. Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes (asuresh)
2016-12-27 12:40:44 -08:00
c3973e7080
YARN-5969. FairShareComparator: Cache value of getResourceUsage for better performance. (zhangshilong via kasha)
2016-12-27 12:30:35 -08:00
4d3f73acc0
YARN-6000. Make AllocationFileLoaderService.Listener public. (Tao Jie via kasha)
2016-12-23 11:40:56 -08:00
ae401539ea
YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor.
2016-12-22 14:32:24 +09:00
736f54b727
YARN-4990. Re-direction of a particular log file within in a container in NM UI does not redirect properly. Contributed by Xuan Gong.
2016-12-21 14:14:42 -08:00
8b042bc1e6
YARN-5976. Update hbase version to 1.2. Contributed by Vrushali C.
2016-12-21 09:43:17 -08:00
575773a357
YARN-5877. Allow all env's from yarn.nodemanager.env-whitelist to get overridden during launch. Contributed by Bibin A Chundatt.
2016-12-20 11:39:06 +05:30
ef2dd7b78c
YARN-5650. Render Application Timeout value in web UI. Contributed by Akhil PB.
2016-12-19 11:44:03 +05:30
2273a74c1f
YARN-5646. Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers. (Konstantinos Karanasos via asuresh)
2016-12-16 08:14:34 -08:00
ada876cd1d
Revert YARN-4126. RM should not issue delegation tokens in unsecure mode.
2016-12-13 20:49:54 -08:00
4b149a1e77
YARN-4457. Cleanup unchecked types for EventHandler (templedf via rkanter)
2016-12-09 17:28:25 -08:00
a6410a542e
YARN-5709. Cleanup leader election configs and pluggability. Contribtued by Karthik Kambatla
2016-12-09 17:00:37 -08:00
b0aace21b1
YARN-5982. Simplify opportunistic container parameters and metrics. (Konstantinos Karanasos via asuresh)
2016-12-09 16:41:25 -08:00
55f5886ea2
YARN-5925. Extract hbase-backend-exclusive utility methods from TimelineStorageUtil. Contributed by Haibo Chen.
2016-12-09 16:17:24 -08:00
a5a55a54ab
YARN-5922. Remove direct references of HBaseTimelineWriter/Reader in core ATS classes. Contributed by Haibo Chen.
2016-12-08 12:31:12 -08:00
74d0066d33
YARN-5970. Validate application update timeout request parameters. Contributed by Rohith Sharma K S.
2016-12-08 15:53:56 +05:30
72fe546841
YARN-5963. Spelling errors in logging and exceptions for node manager, client, web-proxy, common, and app history code (gsohn via rkanter)
2016-12-07 15:07:25 -08:00
9f5d2c4fff
YARN-5136. Error in handling event type APP_ATTEMPT_REMOVED to the scheduler
...
(Contributed by Wilfred Spiegelenburg via Daniel Templeton)
2016-12-07 11:12:14 -08:00
ab923a53fc
YARN-5965. Retrospect ApplicationReport#getApplicationTimeouts. Contributed by Rohith Sharma K S
2016-12-08 00:27:25 +05:30
563480dccd
YARN-5932. Retrospect moveApplicationToQueue in align with YARN-5611. Contributed by Sunil G.
2016-12-07 10:39:14 +05:30
f3b8ff54ab
YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena
2016-12-06 06:53:38 +05:30
43ebff2e35
YARN-5559. Analyse 2.8.0/3.0.0 jdiff reports and fix any issues. Contributed by Akira Ajisaka & Wangda Tan
2016-12-05 11:39:34 -08:00
f885160f4a
YARN-5746. The state of the parentQueue and its childQueues should be synchronized. Contributed by Xuan Gong
2016-12-02 16:17:31 -08:00
5bd18c49bd
YARN-5929. Missing scheduling policy in the FS queue metric. (Contributed by Yufei Gu via Daniel Templeton)
2016-12-02 13:55:42 -08:00
69fb70c31a
YARN-5761. Separate QueueManager from Scheduler. (Xuan Gong via gtcarrera9)
2016-11-30 13:38:42 -08:00
b3befc021b
YARN-4997. Update fair scheduler to use pluggable auth provider (Contributed by Tao Jie via Daniel Templeton)
2016-11-30 09:50:33 -08:00
cfd8076f81
Revert due to an error "HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen"
...
This reverts commit 5614f847b2ef2a5b70bd9a06edc4eba06174c6.
2016-11-30 15:44:52 +08:00
dd9a96cb17
YARN-5851. TestContainerManagerSecurity testContainerManager[1] failed (Contributed by Haibo Chen via Daniel Templeton)
2016-11-29 14:22:19 -08:00
3b9d3acd20
YARN-5890. FairScheduler should log information about AM-resource-usage and max-AM-share for queues
...
(Contributed by Yufei Gu via Daniel Templeton)
2016-11-29 12:46:05 -08:00
25f9872be6
YARN-5774. MR Job stuck in ACCEPTED status without any progress in Fair Scheduler
...
if set yarn.scheduler.minimum-allocation-mb to 0. (Contributed by Yufei Gu via Daniel Templeton)
2016-11-29 09:40:49 -08:00
62b42ef5dd
YARN-5725. Test uncaught exception in TestContainersMonitorResourceChange.testContainersResourceChange
...
when setting IP and host (Contributed by Miklos Szegedi via Daniel Templeton)
2016-11-28 15:19:52 -08:00
5d5614f847
HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen
2016-11-28 14:34:44 +08:00
209e805430
HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki.
2016-11-28 14:34:57 +09:00
c06114d6a3
YARN-5572. HBaseTimelineWriterImpl appears to reference a bad property name. Contributed by Varun Saxena.
2016-11-27 23:35:53 +05:30
e2f77ca77b
Revert "YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena."
...
This reverts commit 9f2960be19
2016-11-27 23:32:59 +05:30
9f2960be19
YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena.
2016-11-27 23:31:27 +05:30
eb0a483ed0
YARN-4206. Add Application timeouts in Application report and CLI. Contributed by Rohith Sharma K S.
2016-11-24 18:18:42 +05:30
e15c20edba
YARN-5920. Fix deadlock in TestRMHA.testTransitionedToStandbyShouldNotHang. Contributed by Varun Saxena.
2016-11-24 12:18:38 +05:30
10468529a9
YARN-4752. Improved preemption in FairScheduler. (kasha)
...
Contains:
YARN-5605. Preempt containers (all on one node) to meet the requirement of starved applications
YARN-5821. Drop left-over preemption-related code and clean up method visibilities in the Schedulable hierarchy
YARN-5783. Verify identification of starved applications.
YARN-5819. Verify fairshare and minshare preemption
YARN-5885. Cleanup YARN-4752 branch for merge
Change-Id: Iee0962377d019dd64dc69a020725d2eaf360858c
2016-11-23 19:48:59 -10:00
1f12867a69
YARN-5649. Add REST endpoints for updating application timeouts. Contributed by Rohith Sharma K S
2016-11-23 16:25:39 -08:00
005850b28f
YARN-5918. Handle Opportunistic scheduling allocate request failure when NM is lost. (Bibin A Chundatt via asuresh)
2016-11-23 09:53:31 -08:00
3541ed8068
YARN-4330. MiniYARNCluster is showing multiple Failed to instantiate default resource calculator warning messages. Contributed by Varun Saxena
2016-11-23 14:12:23 +05:30
83cc7263af
YARN-5722. FairScheduler hides group resolution exceptions when assigning queue (Contributed by Wilfred Spiegelenburg via Daniel Templeton)
2016-11-22 11:16:00 -08:00
a926f895c1
YARN-5865. Retrospect updateApplicationPriority api to handle state store exception in align with YARN-5611. Contributed by Sunil G.
2016-11-22 14:49:15 +05:30
6f8074298d
YARN-5713. Update jackson from 1.9.13 to 2.x in hadoop-yarn.
2016-11-22 10:20:31 +09:00
009452bb6d
YARN-5859. TestResourceLocalizationService#testParallelDownloadAttemptsForPublicResource sometimes fails. Contributed by Eric Badger
2016-11-21 16:36:08 +00:00
f05a9ceb4a
YARN-3538. TimelineWebService doesn't catch runtime exception. Contributed by Steve Loughran and Tsuyoshi Ozawa.
2016-11-17 16:55:39 -08:00
59bfcbf357
YARN-5836. Malicious AM can kill containers of other apps running in any node its containers are running. Contributed by Botong Huang
2016-11-16 22:21:03 +00:00
d65603517e
YARN-5375. invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures. Contributed by sandflee.
2016-11-16 15:14:00 +05:30
f121d0b036
YARN-5875. TestTokenClientRMService#testTokenRenewalWrongUser fails. Contributed by Gergely Novák.
2016-11-15 13:58:11 -08:00
264ddb13ff
YARN-5736 Addendum. Fixes segfault due to unterminated string. (Contributed by Miklos Szegedi via Daniel Templeton)
2016-11-15 13:01:57 -08:00
5af572b644
HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu
2016-11-15 10:57:00 -08:00
3219b7b4ac
YARN-4597. Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle. (asuresh)
2016-11-15 07:56:25 -08:00
7ffb9943b8
YARN-4355. NPE while processing localizer heartbeat. Contributed by Varun Saxena & Jonathan Hung.
2016-11-15 15:41:56 +05:30
43aef303bf
Reverted due to issue YARN-5765. Revert "YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang"
...
This reverts commit 131d58a24e
2016-11-15 13:27:37 +05:30
b7070f3308
YARN-5874. RM -format-state-store and -remove-application-from-state-store commands fail with NPE. Contributed by Varun Saxena.
2016-11-15 10:58:25 +05:30
296c5de0cf
YARN-5873. RM crashes with NPE if generic application history is enabled. Contributed by Varun Saxena.
2016-11-15 10:56:28 +05:30
79448d4ab1
YARN-4033. In FairScheduler, parent queues should also display queue status. (Siqi Li via kasha)
2016-11-13 21:38:36 -08:00
Jason Lowe
Rohith Sharma K S
Wangda Tan
Sunil G
Sangjin Lee
Andrew Wang
Karthik Kambatla
Subru Krishnan
Daniel Templeton
Junping Du
Naganarasimha
Varun Saxena
Jian He
Arun Suresh
Junping Du
Robert Kanter
Varun Vasudev
Karthik Kambatla
Akira Ajisaka
Li Lu
Kai Zheng
Xiao Chen
Mingliang Liu