9dbfab1284
YARN-6108. Improve AHS webservice to accept NM address as a parameter to get container logs. Contributed by Xuan Gong.
2017-02-07 02:13:14 -08:00
cce35c3815
HADOOP-12097. Allow port range to be specified while starting webapp. Contributed by Varun Saxena.
2017-02-05 19:42:11 -08:00
327c9980aa
YARN-6100. Improve YARN webservice to output aggregated container logs. Contributed by Xuan Gong.
2017-02-02 00:41:18 -08:00
3e06475307
YARN-4658. Typo in o.a.h.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler comment (Contributed by Udai Potluri via Daniel Templeton)
2017-01-31 16:25:33 -08:00
4c6bae5a22
YARN-5866. Fix few issues reported by jshint in new YARN UI. Contributed by Akhil P B.
2017-01-31 14:18:31 +05:30
87852b6ef4
YARN-6103. Log updates for ZKRMStateStore (Contributed by Daniel Sturman via Daniel Templeton)
2017-01-30 21:44:48 -08:00
11e44bdda7
YARN-6131. FairScheduler: Lower update interval for faster tests. (kasha)
2017-01-30 13:35:23 -08:00
1a16431bd0
YARN-6126. Obtaining app logs for Running application fails with json parse error. Contributed by Xuan Gong.
2017-01-27 06:31:10 -08:00
165f07f51a
YARN-6123. [YARN-5864] Add a test to make sure queues of orderingPolicy will be updated when childQueues is added or removed. Contributed by Wangda Tan.
2017-01-27 18:37:51 +05:30
f85b74ccf9
YARN-4975. Fair Scheduler: exception thrown when a parent queue marked 'parent' has configured child queues
...
(Contributed by Yufei Gu via Daniel Templeton)
2017-01-26 10:31:09 -08:00
425a7e5028
YARN-3637. Handle localization sym-linking correctly at the YARN level. Contributed by Chris Trezzo.
2017-01-25 15:51:36 -08:00
9e19f758c1
YARN-5641. Localizer leaves behind tarballs after container is complete. Contributed by Eric Badger
2017-01-25 21:41:43 +00:00
abedb8a9d8
YARN-5830. FairScheduler: Avoid preempting AM containers. (Yufei Gu via kasha)
2017-01-25 12:17:28 -08:00
8528d85a68
YARN-6099. Improve webservice to list aggregated log files. Contributed by Xuan Gong.
2017-01-24 15:26:53 -08:00
a33ce45e35
YARN-5547. NMLeveldbStateStore should be more tolerant of unknown keys. Contributed by Ajith S
2017-01-24 16:17:36 +00:00
0101267d9d
YARN-6082. Invalid REST api response for getApps since queueUsagePercentage is coming as INF. Contributed by Sunil G.
2017-01-24 10:59:55 +05:30
dc6ec97048
YARN-6117. SharedCacheManager does not start up. Contributed by Chris Trezzo.
2017-01-23 21:07:25 -08:00
a2c50127d1
YARN-6012. Remove node label (removeFromClusterNodeLabels) document is missing (Contributed by Ying Zhang via Daniel Templeton)
2017-01-23 17:42:54 -08:00
ce832059db
YARN-5864. Capacity Scheduler - Queue Priorities. (wangda)
2017-01-23 10:52:14 -08:00
69fa81679f
YARN-5910. Support for multi-cluster delegation tokens. Contributed by Jian He
2017-01-23 17:12:51 +00:00
3fa0d540df
YARN-6031. Application recovery has failed when node label feature is turned off during RM recovery. Contributed by Ying Zhang.
2017-01-23 12:19:35 +05:30
9bab85cadf
YARN-6106. Document FairScheduler 'allowPreemptionFrom' queue property. (Yufei Gu via rchiang)
2017-01-20 14:08:25 -08:00
b01514f65b
YARN-5928. Move ATSv2 HBase backend code into a new module that is only dependent at runtime by yarn servers. Contributed by Haibo Chen.
2017-01-19 20:52:55 -08:00
5d8b80ea9b
Preparing for 3.0.0-alpha3 development
2017-01-19 15:50:07 -08:00
efc8faa1ba
YARN-6104. RegistrySecurity overrides zookeeper sasl system properties. Contributed by Billie Rinaldi
2017-01-19 10:18:59 -08:00
7e8d32147c
YARN-6110. Fix opportunistic containers documentation. (Akira Ajisaka via asuresh)
2017-01-19 06:14:28 -08:00
0eb4b513b7
HADOOP-13673. Update scripts to be smarter when running with privilege
...
Signed-off-by: Andrew Wang <wang@apache.org>
Signed-off-by: Ravi Prakash <raviprak@apache.org>
2017-01-18 14:39:05 -08:00
72054a817d
YARN-5556. CapacityScheduler: Support deleting queues without requiring a RM restart. (Naganarasimha G R via wangda)
2017-01-18 13:31:33 -08:00
e224c96234
YARN-5831. FairScheduler: Propagate allowPreemptionFrom flag all the way down to the app. (Yufei Gu via kasha)
2017-01-17 17:01:31 -08:00
4d1f3d9020
YARN-6016. Fix minor bugs in handling of local AMRMToken in AMRMProxy. (Botong Huang via Subru).
2017-01-17 14:48:03 -08:00
695f110f93
YARN-6071. Fix incompatible API change on AM-RM protocol due to YARN-3866 (trunk only)
...
(Contributed by Wangda Tan via Daniel Templeton)
2017-01-17 12:55:47 -08:00
a9d946d4df
YARN-6057. yarn.scheduler.minimum-allocation-* descriptions are incorrect about behavior when a request is out of bounds. Contributed by Julia Sommer.
2017-01-17 14:16:00 +05:30
e407449ddb
HADOOP-13933. Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers. Contributed by Surendra Singh Lilhore.
2017-01-17 10:10:23 +09:00
cf695577aa
YARN-6011. Add a new web service to list the files on a container in AHSWebService. Contributed by Xuan Gong.
2017-01-16 16:20:24 -08:00
d3170f9eba
YARN-6081. LeafQueue#getTotalPendingResourcesConsideringUserLimit should deduct reserved from pending to avoid unnecessary preemption of reserved container. Contributed by Wangda Tan.
2017-01-13 18:22:29 +05:30
a6b06f7179
YARN-6072. RM unable to start in secure mode. Contributed by Ajith S.
2017-01-12 06:40:41 +05:30
e6f13fe5d1
YARN-5849. Automatically create YARN control group for pre-mounted cgroups (Contributed by Miklos Szegedi via Daniel Templeton)
2017-01-11 16:00:01 -08:00
7979939428
YARN-5554. MoveApplicationAcrossQueues does not check user permission on the target queue
...
(Contributed by Wilfred Spiegelenburg via Daniel Templeton)
2017-01-11 14:34:10 -08:00
357eab9566
YARN-5416. TestRMRestart#testRMRestartWaitForPreviousAMToFinish failed intermittently due to not wait SchedulerApplicationAttempt to be stopped. Contributed by Junping Du
2017-01-11 17:13:37 +00:00
4db119b7b5
YARN-6079. Fix simple spelling errors in yarn test code. Contributed by vijay.
2017-01-10 18:20:27 -08:00
4c431a6940
YARN-6054. TimelineServer fails to start when some LevelDb state files are missing. Contributed by Ravi Prakash.
2017-01-10 15:54:16 +05:30
41db07d532
YARN-5937. stop-yarn.sh is not able to gracefully stop node managers. Contributed by Weiwei Yang.
2017-01-10 15:35:01 +05:30
9594c35dcb
YARN-5899. Debug log in AbstractCSQueue#canAssignToThisQueue needs improvement. Contributed by Ying Zhang.
2017-01-10 14:08:01 +05:30
6332a318bc
YARN-6073. Misuse of format specifier in Preconditions.checkArgument (Contributed by Yuanbo Liu via Daniel Templeton)
2017-01-09 22:12:58 -08:00
945db55f2e
YARN-4148. When killing app, RM releases app's resource before they are released by NM. Contributed by Jason Lowe.
2017-01-09 18:14:46 -08:00
287d3d6804
YARN-3955. Support for application priority ACLs in queues of CapacityScheduler. (Sunil G via wangda)
2017-01-09 08:40:39 -08:00
db490eccce
YARN-6074. FlowRunEntity does not deserialize long values correctly (Rohith Sharma K S via Varun Saxena)
2017-01-09 18:54:22 +05:30
f59e36b4ce
YARN-6068. Log aggregation get failed when NM restart even with recovery (Junping Du via Varun Saxena)
2017-01-09 11:17:19 +05:30
85826f6ca5
YARN-6066. Opportunistic containers Minor fixes : API annotations, parameter name changes, checkstyles. (asuresh)
...
(cherry picked from commit 4985217de4
2017-01-08 09:04:20 -08:00
ac16400e1f
YARN-6022. Revert changes of AbstractResourceRequest (Contributed by Wangda Tan via Daniel Templeton)
2017-01-07 08:24:30 -08:00
a59df15757
YARN-6015. AsyncDispatcher thread name can be set to improved debugging. Contributed by Ajith S.
2017-01-07 08:11:58 +05:30
020316458d
YARN-6009. Skip validating app timeout value on recovery. Contributed by Rohith Sharma K S
2017-01-06 18:19:09 -08:00
71a4acf74b
YARN-5991. Yarn Distributed Shell does not print throwable t to App Master When failed to start container
...
(Contributed by Jim Frankola via Daniel Templeton)
2017-01-06 14:26:08 -08:00
2977bc6a14
YARN-6040. Introduce api independent PendingAsk to replace usage of ResourceRequest within Scheduler classes. (Wangda Tan via asuresh)
2017-01-06 09:59:57 -08:00
0a55bd841e
YARN-5959. RM changes to support change of container ExecutionType. (Arun Suresh via wangda)
2017-01-05 10:31:05 -08:00
e49e0a6e37
YARN-5988. RM unable to start in secure setup. Contributed by Ajith S.
2017-01-04 12:09:29 +05:30
8fadd69047
YARN-5923. Unable to access logs for a running application if YARN_ACL_ENABLE is enabled. Contributed by Xuan Gong.
2017-01-03 15:03:38 -08:00
f69a107aec
YARN-6025. Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda)
...
(cherry picked from commit e0f2379312
2017-01-03 14:53:36 -08:00
ab1faa4ba8
Revert "Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda)" to add correct JIRA number
...
This reverts commit e0f2379312
2017-01-03 14:53:13 -08:00
e0f2379312
Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda)
2017-01-03 14:46:41 -08:00
165d01a73e
YARN-5931. Document timeout interfaces CLI and REST APIs (Contributed by Rohith Sharma K S via Daniel Templeton)
2016-12-30 08:41:52 -08:00
f216276d21
YARN-4882. Change the log level to DEBUG for recovering completed applications (templedf via rkanter)
2016-12-28 15:21:52 -08:00
9ca54f4810
YARN-5906. Update AppSchedulingInfo to use SchedulingPlacementSet. Contributed by Wangda Tan.
2016-12-28 22:48:19 +05:30
972da46cb4
YARN-5719. Enforce a C standard for native container-executor. Contributed by Chris Douglas.
2016-12-28 14:59:57 +05:30
f6715b26b6
YARN-6024. Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to max. Contributed by Wangda Tan.
2016-12-28 12:33:02 +05:30
e297be74ed
YARN-6001. Improve moveApplicationQueues command line. Contributed by Sunil G.
2016-12-28 12:16:11 +05:30
0840b4329b
YARN-5756. Add state-machine implementation for scheduler queues. (Xuan Gong via wangda)
2016-12-27 21:18:33 -08:00
9262797e86
YARN-5257. Fix unreleased resources and null dereferences (yufeigu via rkanter)
2016-12-27 14:14:08 -08:00
1bbd023275
YARN-5962. Spelling errors in logging and exceptions for resource manager code (gsohn via rkanter)
2016-12-27 13:35:34 -08:00
e216e8e233
HADOOP-13932. Fix indefinite article in comments (Contributed by LiXin Ge via Daniel Templeton)
2016-12-27 13:03:16 -08:00
ac1e5d4f77
YARN-5938. Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes (asuresh)
2016-12-27 12:40:44 -08:00
c3973e7080
YARN-5969. FairShareComparator: Cache value of getResourceUsage for better performance. (zhangshilong via kasha)
2016-12-27 12:30:35 -08:00
ded2d08f33
HADOOP-13940. Document the missing envvars commands (Contributed by Yiqun Lin via Daniel Templeton)
2016-12-27 07:18:08 -08:00
c721f78a3c
YARN-6026. A couple of spelling errors in the docs. Contributed by Grant Sohn.
2016-12-24 05:38:09 +05:30
4d3f73acc0
YARN-6000. Make AllocationFileLoaderService.Listener public. (Tao Jie via kasha)
2016-12-23 11:40:56 -08:00
38e66d4d64
YARN-5903. Fix race condition in TestResourceManagerAdministrationProtocolPBClientImpl beforeclass setup method (Haibo Chen via Varun Saxena)
2016-12-22 23:08:33 +05:30
ae401539ea
YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor.
2016-12-22 14:32:24 +09:00
736f54b727
YARN-4990. Re-direction of a particular log file within in a container in NM UI does not redirect properly. Contributed by Xuan Gong.
2016-12-21 14:14:42 -08:00
8b042bc1e6
YARN-5976. Update hbase version to 1.2. Contributed by Vrushali C.
2016-12-21 09:43:17 -08:00
575773a357
YARN-5877. Allow all env's from yarn.nodemanager.env-whitelist to get overridden during launch. Contributed by Bibin A Chundatt.
2016-12-20 11:39:06 +05:30
358341398a
YARN-5524. Yarn live log aggregation does not throw if command line arg is wrong. Contributed by Xuan Gong.
2016-12-20 06:36:25 +05:30
ef2dd7b78c
YARN-5650. Render Application Timeout value in web UI. Contributed by Akhil PB.
2016-12-19 11:44:03 +05:30
2273a74c1f
YARN-5646. Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers. (Konstantinos Karanasos via asuresh)
2016-12-16 08:14:34 -08:00
64a2d5be91
YARN-5999. AMRMClientAsync will stop if any exceptions thrown on allocate call. Contributed by Jian He
2016-12-14 14:33:23 -08:00
ada876cd1d
Revert YARN-4126. RM should not issue delegation tokens in unsecure mode.
2016-12-13 20:49:54 -08:00
f66f61892a
HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran
2016-12-12 13:58:27 +00:00
4b149a1e77
YARN-4457. Cleanup unchecked types for EventHandler (templedf via rkanter)
2016-12-09 17:28:25 -08:00
a6410a542e
YARN-5709. Cleanup leader election configs and pluggability. Contribtued by Karthik Kambatla
2016-12-09 17:00:37 -08:00
b0aace21b1
YARN-5982. Simplify opportunistic container parameters and metrics. (Konstantinos Karanasos via asuresh)
2016-12-09 16:41:25 -08:00
55f5886ea2
YARN-5925. Extract hbase-backend-exclusive utility methods from TimelineStorageUtil. Contributed by Haibo Chen.
2016-12-09 16:17:24 -08:00
7d8e440eee
Revert "HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran"
...
This reverts commit c265515725
2016-12-09 12:49:27 +09:00
a5a55a54ab
YARN-5922. Remove direct references of HBaseTimelineWriter/Reader in core ATS classes. Contributed by Haibo Chen.
2016-12-08 12:31:12 -08:00
c265515725
HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran
2016-12-08 17:57:15 +00:00
74d0066d33
YARN-5970. Validate application update timeout request parameters. Contributed by Rohith Sharma K S.
2016-12-08 15:53:56 +05:30
72fe546841
YARN-5963. Spelling errors in logging and exceptions for node manager, client, web-proxy, common, and app history code (gsohn via rkanter)
2016-12-07 15:07:25 -08:00
9f5d2c4fff
YARN-5136. Error in handling event type APP_ATTEMPT_REMOVED to the scheduler
...
(Contributed by Wilfred Spiegelenburg via Daniel Templeton)
2016-12-07 11:12:14 -08:00
ab923a53fc
YARN-5965. Retrospect ApplicationReport#getApplicationTimeouts. Contributed by Rohith Sharma K S
2016-12-08 00:27:25 +05:30
563480dccd
YARN-5932. Retrospect moveApplicationToQueue in align with YARN-5611. Contributed by Sunil G.
2016-12-07 10:39:14 +05:30
a7288da595
YARN-5184. Fix up incompatible changes introduced on ContainerStatus and NodeReport. Contributed by Sangjin Lee.
2016-12-06 14:36:41 -08:00
c8d0a049b0
HADOOP-13859. TestConfigurationFieldsBase fails for fields that are DEFAULT values of skipped properties. (Haibo Chen via kasha)
2016-12-06 14:10:58 -08:00
f3b8ff54ab
YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena
2016-12-06 06:53:38 +05:30
43ebff2e35
YARN-5559. Analyse 2.8.0/3.0.0 jdiff reports and fix any issues. Contributed by Akira Ajisaka & Wangda Tan
2016-12-05 11:39:34 -08:00
f885160f4a
YARN-5746. The state of the parentQueue and its childQueues should be synchronized. Contributed by Xuan Gong
2016-12-02 16:17:31 -08:00
5bd18c49bd
YARN-5929. Missing scheduling policy in the FS queue metric. (Contributed by Yufei Gu via Daniel Templeton)
2016-12-02 13:55:42 -08:00
f304ccae3c
YARN-5915. ATS 1.5 FileSystemTimelineWriter causes flush() to be called after every event write. Contributed by Atul Sikaria
2016-12-02 16:54:15 +00:00
c87b3a448a
MAPREDUCE-6787. Allow job_conf.xml to be downloadable on the job overview page in JHS (haibochen via rkanter)
2016-12-01 17:29:38 -08:00
2d77dc727d
YARN-5901. Fix race condition in TestGetGroups beforeclass setup() (Contributed by Haibo Chen via Daniel Templeton)
2016-12-01 15:57:39 -08:00
69fb70c31a
YARN-5761. Separate QueueManager from Scheduler. (Xuan Gong via gtcarrera9)
2016-11-30 13:38:42 -08:00
4fca94fbda
YARN-5942. "Overridden" is misspelled as "overriden" in FairScheduler.md
...
(Contributed by Heather Sutherland via Daniel Templeton)
2016-11-30 11:23:51 -08:00
b3befc021b
YARN-4997. Update fair scheduler to use pluggable auth provider (Contributed by Tao Jie via Daniel Templeton)
2016-11-30 09:50:33 -08:00
cfd8076f81
Revert due to an error "HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen"
...
This reverts commit 5614f847b2ef2a5b70bd9a06edc4eba06174c6.
2016-11-30 15:44:52 +08:00
dd9a96cb17
YARN-5851. TestContainerManagerSecurity testContainerManager[1] failed (Contributed by Haibo Chen via Daniel Templeton)
2016-11-29 14:22:19 -08:00
3b9d3acd20
YARN-5890. FairScheduler should log information about AM-resource-usage and max-AM-share for queues
...
(Contributed by Yufei Gu via Daniel Templeton)
2016-11-29 12:46:05 -08:00
25f9872be6
YARN-5774. MR Job stuck in ACCEPTED status without any progress in Fair Scheduler
...
if set yarn.scheduler.minimum-allocation-mb to 0. (Contributed by Yufei Gu via Daniel Templeton)
2016-11-29 09:40:49 -08:00
62b42ef5dd
YARN-5725. Test uncaught exception in TestContainersMonitorResourceChange.testContainersResourceChange
...
when setting IP and host (Contributed by Miklos Szegedi via Daniel Templeton)
2016-11-28 15:19:52 -08:00
5d5614f847
HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen
2016-11-28 14:34:44 +08:00
209e805430
HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki.
2016-11-28 14:34:57 +09:00
c06114d6a3
YARN-5572. HBaseTimelineWriterImpl appears to reference a bad property name. Contributed by Varun Saxena.
2016-11-27 23:35:53 +05:30
e2f77ca77b
Revert "YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena."
...
This reverts commit 9f2960be19
2016-11-27 23:32:59 +05:30
9f2960be19
YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena.
2016-11-27 23:31:27 +05:30
eb0a483ed0
YARN-4206. Add Application timeouts in Application report and CLI. Contributed by Rohith Sharma K S.
2016-11-24 18:18:42 +05:30
e15c20edba
YARN-5920. Fix deadlock in TestRMHA.testTransitionedToStandbyShouldNotHang. Contributed by Varun Saxena.
2016-11-24 12:18:38 +05:30
10468529a9
YARN-4752. Improved preemption in FairScheduler. (kasha)
...
Contains:
YARN-5605. Preempt containers (all on one node) to meet the requirement of starved applications
YARN-5821. Drop left-over preemption-related code and clean up method visibilities in the Schedulable hierarchy
YARN-5783. Verify identification of starved applications.
YARN-5819. Verify fairshare and minshare preemption
YARN-5885. Cleanup YARN-4752 branch for merge
Change-Id: Iee0962377d019dd64dc69a020725d2eaf360858c
2016-11-23 19:48:59 -10:00
1f12867a69
YARN-5649. Add REST endpoints for updating application timeouts. Contributed by Rohith Sharma K S
2016-11-23 16:25:39 -08:00
005850b28f
YARN-5918. Handle Opportunistic scheduling allocate request failure when NM is lost. (Bibin A Chundatt via asuresh)
2016-11-23 09:53:31 -08:00
3541ed8068
YARN-4330. MiniYARNCluster is showing multiple Failed to instantiate default resource calculator warning messages. Contributed by Varun Saxena
2016-11-23 14:12:23 +05:30
4667564162
YARN-5911. DrainDispatcher does not drain all events on stop even if setDrainEventsOnStop is true. Contributed by Varun Saxena.
2016-11-23 08:49:48 +05:30
83cc7263af
YARN-5722. FairScheduler hides group resolution exceptions when assigning queue (Contributed by Wilfred Spiegelenburg via Daniel Templeton)
2016-11-22 11:16:00 -08:00
a926f895c1
YARN-5865. Retrospect updateApplicationPriority api to handle state store exception in align with YARN-5611. Contributed by Sunil G.
2016-11-22 14:49:15 +05:30
6f8074298d
YARN-5713. Update jackson from 1.9.13 to 2.x in hadoop-yarn.
2016-11-22 10:20:31 +09:00
009452bb6d
YARN-5859. TestResourceLocalizationService#testParallelDownloadAttemptsForPublicResource sometimes fails. Contributed by Eric Badger
2016-11-21 16:36:08 +00:00
09520cb439
YARN-5271. ATS client doesn't work with Jersey 2 on the classpath. Contributed by Weiwei Yang.
2016-11-17 22:17:23 -06:00
140b9939da
YARN-5904. Reduce the number of default server threads for AMRMProxyService.
2016-11-17 18:26:25 -08:00
f05a9ceb4a
YARN-3538. TimelineWebService doesn't catch runtime exception. Contributed by Steve Loughran and Tsuyoshi Ozawa.
2016-11-17 16:55:39 -08:00
ce2847e794
YARN-5870. Expose getApplications API in YarnClient with GetApplicationsRequest parameter. Contributed by Jian He.
2016-11-17 08:48:35 +05:30
59bfcbf357
YARN-5836. Malicious AM can kill containers of other apps running in any node its containers are running. Contributed by Botong Huang
2016-11-16 22:21:03 +00:00
6a11877ab5
YARN-5891. yarn rmadmin -help contains a misspelled ResourceManager (Grant Sohn via Varun Saxena)
2016-11-17 03:26:13 +05:30
d65603517e
YARN-5375. invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures. Contributed by sandflee.
2016-11-16 15:14:00 +05:30
f121d0b036
YARN-5875. TestTokenClientRMService#testTokenRenewalWrongUser fails. Contributed by Gergely Novák.
2016-11-15 13:58:11 -08:00
264ddb13ff
YARN-5736 Addendum. Fixes segfault due to unterminated string. (Contributed by Miklos Szegedi via Daniel Templeton)
2016-11-15 13:01:57 -08:00
5af572b644
HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu
2016-11-15 10:57:00 -08:00
3219b7b4ac
YARN-4597. Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle. (asuresh)
2016-11-15 07:56:25 -08:00
7ffb9943b8
YARN-4355. NPE while processing localizer heartbeat. Contributed by Varun Saxena & Jonathan Hung.
2016-11-15 15:41:56 +05:30
43aef303bf
Reverted due to issue YARN-5765. Revert "YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang"
...
This reverts commit 131d58a24e
2016-11-15 13:27:37 +05:30
b7070f3308
YARN-5874. RM -format-state-store and -remove-application-from-state-store commands fail with NPE. Contributed by Varun Saxena.
2016-11-15 10:58:25 +05:30
296c5de0cf
YARN-5873. RM crashes with NPE if generic application history is enabled. Contributed by Varun Saxena.
2016-11-15 10:56:28 +05:30
79448d4ab1
YARN-4033. In FairScheduler, parent queues should also display queue status. (Siqi Li via kasha)
2016-11-13 21:38:36 -08:00
fad9609d13
YARN-5825. ProportionalPreemptionalPolicy should use readLock over LeafQueue instead of synchronized block. Contributed by Sunil G
2016-11-11 15:16:21 -08:00
ede1a473f5
YARN-5552. Add Builder methods for common yarn API records. (Tao Jie via wangda)
2016-11-11 13:34:56 -08:00
503e73e849
YARN-5545. Fix issues related to Max App in capacity scheduler. Contributed by Bibin A Chundatt
2016-11-11 20:48:31 +05:30
93eeb13164
YARN-4218. Metric for resource*time that was preempted. Contributed by Chang Li.
2016-11-10 22:35:12 +00:00
3a98419532
YARN-5834. TestNodeStatusUpdater.testNMRMConnectionConf compares nodemanager wait time to the incorrect value. (Chang Li via kasha)
2016-11-10 14:08:51 -08:00
86ac1ad9fd
YARN-5453. FairScheduler#update may skip update demand resource of child queue/app if current demand reached maxResource. (sandflee via kasha)
2016-11-09 23:44:02 -08:00
c8bc7a8475
YARN-5843. Incorrect documentation for timeline service entityType/events REST end points (Bibin A Chundatt via Varun Saxena)
2016-11-10 12:10:03 +05:30
c202a10923
YARN-5862. TestDiskFailures.testLocalDirsFailures failed (Yufei Gu via Varun Saxena)
2016-11-10 11:41:34 +05:30
de3a5f8d08
YARN-5856. Unnecessary duplicate start container request sent to NM State store. Contributed by Varun Saxena.
2016-11-10 05:42:30 +05:30
bcc15c6290
YARN-5611. Provide an API to update lifetime of an application. Contributed by Rohith Sharma K S
2016-11-09 16:08:05 -08:00
edbee9e609
YARN-4498. Application level node labels stats to be available in REST (addendum patch). Contributed by Bibin A Chundatt.
2016-11-10 05:00:05 +05:30
59ee8b7a88
YARN-4329. [YARN-5437] Allow fetching exact reason as to why a submitted app
...
is in ACCEPTED state in Fair Scheduler (Contributed by Yufei Gu)
2016-11-09 13:11:37 -08:00
280357c29f
YARN-5833. Addendum patch to include missing changes to yarn-default.xml
2016-11-09 07:15:11 -08:00
09f43fa9c0
YARN-5736. YARN container executor config does not handle white space (miklos.szegedi@cloudera.com via rkanter)
2016-11-09 13:34:40 +01:00
283fa33feb
YARN-5823. Update NMTokens in case of requests with only opportunistic containers. (Konstantinos Karanasos via asuresh)
2016-11-09 00:11:25 -08:00
e1c6ef2efa
HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey.
2016-11-08 16:33:55 -08:00
29e3b3417c
YARN-5833. Add validation to ensure default ports are unique in Configuration. (Konstantinos Karanasos via Subru).
2016-11-08 14:38:18 -08:00
3f93ac0733
YARN-5356. NodeManager should communicate physical resource capability to ResourceManager. Contributed by Inigo Goiri
2016-11-08 22:01:26 +00:00
f38a6d03a1
YARN-5377. Fix TestQueuingContainerManager.testKillMultipleOpportunisticContainers. (Konstantinos Karanasos via asuresh)
2016-11-07 22:10:03 -08:00
de3b4aac56
YARN-5716. Add global scheduler interface definition and update CapacityScheduler to use it. Contributed by Wangda Tan
2016-11-07 10:14:39 -08:00
59bc84aadd
YARN-4849. Addendum patch to improve pom for yarn-ui. (Wangda Tan via Sunil G)
2016-11-06 13:13:31 -08:00
c00b5d1e51
YARN-5804. New UI2 is not able to launch with jetty 9 upgrade post HADOOP-10075. (Sunil G via wangda)
...
(cherry picked from commit ddebaa6b7ae1031899139d153d57df5387d60542)
2016-11-06 13:13:31 -08:00
013ff07bc6
YARN-5785. [YARN-3368] Accessing applications and containers list from Node page is throwing few exceptions in console. (Akhil P B and Sreenath Somarajapuram via Sunil G)
2016-11-06 13:13:31 -08:00
fad392a22f
YARN-5779. [YARN-3368] Addendum patch to document limits/notes of the new YARN UI . (Sunil G via Sreenath Somarajapuram)
2016-11-06 13:13:31 -08:00
64c7cda7e5
YARN-5490. [YARN-3368] Fix various alignment issues and broken breadcrumb link in Node page. (Akhil P B Tan via Sunil G)
2016-11-06 13:13:31 -08:00
12ddbbc61d
YARN-5497. [YARN-3368] Use different color for Undefined and Succeeded for Final State in applications page. (Akhil P B Tan via Sunil G)
2016-11-06 13:13:31 -08:00
bc273c43ae
YARN-5500. [YARN-3368] ‘Master node' link under application tab is broken. (Akhil P B Tan via Sunil G)
2016-11-06 13:13:31 -08:00
e93f900b55
YARN-5772. [YARN-3368] Replace old Hadoop logo with new one (Akhil P B Tan via Sunil G)
2016-11-06 13:13:31 -08:00
825de90b96
YARN-5779. [YARN-3368] Document limits/notes of the new YARN UI (Wangda Tan via Sunil G)
2016-11-06 13:13:31 -08:00
cb77e3eb40
YARN-5741. [YARN-3368] Update UI2 documentation for new UI2 path (Kai Sasaki and Wangda Tan via Sunil G)
2016-11-06 13:13:31 -08:00
d8be766759
YARN-5145. [YARN-3368] Move new YARN UI configuration to HADOOP_CONF_DIR. (Sunil G and Kai Sasaki via wangda)
2016-11-06 13:13:31 -08:00
1be2fd662d
YARN-4849. Addendum patch to remove unwanted files from rat exclusions. (Wangda Tan via Sunil G)
2016-11-06 13:13:31 -08:00
3de0da2a76
YARN-5698. [YARN-3368] Launch new YARN UI under hadoop web app port. (Sunil G via wangda)
2016-11-06 13:13:31 -08:00
98b2ad7208
YARN-5682. [YARN-3368] Fix maven build to keep all generated or downloaded files in target folder (Wangda Tan via Sunil G)
2016-11-06 13:13:31 -08:00
817d5f4c96
YARN-4849. Addendum patch to fix javadocs. (Sunil G via wangda)
2016-11-06 13:13:31 -08:00
a5be75974f
YARN-4849. Addendum patch to fix ASF warnings. (Wangda Tan via Sunil G)
2016-11-06 13:13:31 -08:00
f6574d9ff6
YARN-5503. [YARN-3368] Add missing hidden files in webapp folder for deployment (Sreenath Somarajapuram via Sunil G)
2016-11-06 13:13:31 -08:00
7005580752
YARN-5504. [YARN-3368] Fix YARN UI build pom.xml (Sreenath Somarajapuram via Sunil G)
2016-11-06 13:13:31 -08:00
ce2361a941
YARN-4849. Addendum patch to fix document. (Wangda Tan via Sunil G)
2016-11-06 13:13:31 -08:00
23b0287f62
YARN-5488. [YARN-3368] Applications table overflows beyond the page boundary(Harish Jaiprakash via Sunil G)
2016-11-06 13:13:31 -08:00
b61e60fb92
YARN-5509. Build error due to preparing 3.0.0-alpha2 deployment. (Kai Sasaki via wangda)
2016-11-06 13:13:31 -08:00
561839cb5f
YARN-3334. [YARN-3368] Introduce REFRESH button in various UI pages (Sreenath Somarajapuram via Sunil G)
2016-11-06 13:13:31 -08:00
8f584a561e
YARN-5321. [YARN-3368] Add resource usage for application by node managers (Wangda Tan via Sunil G)
...
YARN-5320. [YARN-3368] Add resource usage by applications and queues to cluster overview page (Wangda Tan via Sunil G)
YARN-5322. [YARN-3368] Add a node heat chart map (Wangda Tan via Sunil G)
YARN-5347. [YARN-3368] Applications page improvements (Sreenath Somarajapuram via Sunil G)
YARN-5348. [YARN-3368] Node details page improvements (Sreenath Somarajapuram via Sunil G)
YARN-5346. [YARN-3368] Queues page improvements (Sreenath Somarajapuram via Sunil G)
YARN-5345. [YARN-3368] Cluster overview page improvements (Sreenath Somarajapuram via Sunil G)
YARN-5344. [YARN-3368] Generic UI improvements (Sreenath Somarajapuram via Sunil G)
2016-11-06 13:13:31 -08:00
35f08122e2
YARN-5161. [YARN-3368] Add Apache Hadoop logo in YarnUI home page. (Kai Sasaki via Sunil G)
2016-11-06 13:13:31 -08:00
14aa58c594
YARN-5183. [YARN-3368] Support for responsive navbar when window is resized. (Kai Sasaki via Sunil G)
2016-11-06 13:13:31 -08:00
c85cc3b56e
YARN-4515. [YARN-3368] Support hosting web UI framework inside YARN RM. (Sunil G via wangda)
...
YARN-5000. [YARN-3368] App attempt page is not loading when timeline server is not started (Sunil G via wangda)
YARN-5038. [YARN-3368] Application and Container pages shows wrong values when RM is stopped. (Sunil G via wangda)
2016-11-06 13:13:31 -08:00
2c4f164e16
YARN-5019. [YARN-3368] Change urls in new YARN ui from camel casing to hyphens. (Sunil G via wangda)
2016-11-06 13:13:31 -08:00
dea4a296e5
YARN-4514. [YARN-3368] Cleanup hardcoded configurations, such as RM/ATS addresses. (Sunil G via wangda)
2016-11-06 13:13:31 -08:00
266784b849
YARN-4849. [YARN-3368] cleanup code base, integrate web UI related build to mvn, and fix licenses. (wangda)
2016-11-06 13:13:31 -08:00
0a5f652071
YARN-4517. Add nodes page and fix bunch of license issues. (Varun Saxena via wangda)
2016-11-06 13:13:31 -08:00
53e661f68e
YARN-4733. [YARN-3368] Initial commit of new YARN web UI. (wangda)
2016-11-06 13:13:31 -08:00
6bb741ff0e
YARN-5837. NPE when getting node status of a decommissioned node after an RM restart. Contributed by Robert Kanter
2016-11-04 22:20:21 +00:00
0aafc122d4
YARN-2995. Enhance UI to show cluster resource utilization of various container Execution types. (Konstantinos Karanasos via asuresh)
2016-11-04 07:31:54 -07:00
19b3779ae7
YARN-5802. updateApplicationPriority api in scheduler should ensure to re-insert app to correct ordering policy. Contributred by Bibin A Chundatt
2016-11-04 16:07:28 +05:30
9ee0e3172e
YARN-5822. Log ContainerRuntime initialization error in LinuxContainerExecutor. Contributed by Sidharta Seethana.
2016-11-03 22:33:43 +05:30
352cbaa7a5
YARN-4862. Handle duplicate completed containers in RMNodeImpl. Contributed by Rohith Sharma K S
2016-11-03 13:54:31 +00:00
0e75496049
YARN-5720. Update document for "rmadmin -replaceLabelOnNode". Contributred by Tao Jie
2016-11-03 12:32:22 +05:30
377919010b
YARN-5815. Random failure of TestApplicationPriority.testOrderOfActivatingThePriorityApplicationOnRMRestart (Bibin A Chundatt via Varun Saxena)
2016-11-03 00:37:09 +05:30
aacf214a27
YARN-5697. Use CliParser to parse options in RMAdminCLI. Contributed by Tao jie
2016-11-02 04:44:33 +05:30
76893a4100
YARN-5001. Aggregated Logs root directory is created with wrong group if nonexistent. Contributed by Haibo Chen
2016-11-01 20:23:24 +00:00
9d0c2ad5a5
YARN-3732. Change NodeHeartbeatResponse.java and RegisterNodeManagerResponse.java as abstract classes. Contributed by Devaraj K.
2016-11-01 16:25:45 +05:30
7d2d8d25ba
YARN-5788. Apps not activiated and AM limit resource in UI and REST not updated after -replaceLabelsOnNode (Bibin A Chundatt via Varun Saxena)
2016-11-01 15:32:04 +05:30
310aa4686e
YARN-5805. Add isDebugEnabled check for debug logs in nodemanager (Bibin A Chundatt via Varun Saxena)
2016-11-01 14:51:31 +05:30
733aa99313
YARN-4998. Minor cleanup to UGI use in AdminService. (Daniel Templeton via kasha)
2016-10-31 16:26:01 -07:00
07ab89e8bb
YARN-5809. AsyncDispatcher possibly invokes multiple shutdown threads when handling exception (Jian He via Varun Saxena)
2016-11-01 04:37:03 +05:30
90dd3a8148
YARN-2009. CapacityScheduler: Add intra-queue preemption for app priority support. (Sunil G via wangda)
2016-10-31 15:18:31 -07:00
773c60bd7b
YARN-5800. Delete LinuxContainerExecutor comment from yarn-default.xml (Contributed by Jan Hentschel via Daniel Templeton)
2016-10-31 14:47:48 -07:00
f3eb4c3c73
YARN-5793. Trim configuration values in DockerLinuxContainerRuntime (Contributed by Tianyin Xu via Daniel Templeton)
2016-10-31 13:51:07 -07:00
2528bea67f
YARN-4396. Log the trace information on FSAppAttempt#assignContainer (Contributed by Yiqun Li via Daniel Templeton)
2016-10-31 13:34:25 -07:00
cc2c993a8a
YARN-4907. Make all MockRM#waitForState consistent. (Contributed by Yufei Gu via Daniel Templeton)
2016-10-31 13:20:56 -07:00
e0bebbbcdd
YARN-4498. Application level node labels stats to be available in REST. Contributed by Bibin A Chundatt
2016-10-31 04:38:20 +05:30
aa3cab1eb2
YARN-5799. Fix Opportunistic Allocation to set the correct value of Node Http Address. (asuresh)
2016-10-29 02:03:57 -07:00
1c8ab41e8b
YARN-5773. RM recovery too slow due to LeafQueue#activateApplications (Bibin A Chundatt via Varun Saxena)
2016-10-29 13:47:39 +05:30
76a1042980
YARN-3460. TestSecureRMRegistryOperations fails with IBM_JAVA. Contributed by Pascal Oliva
2016-10-28 16:49:36 -07:00
1eae719bce
YARN-4963. capacity scheduler: Make number of OFF_SWITCH assignments per heartbeat configurable. Contributed by Nathan Roberts
2016-10-28 17:30:15 +00:00
1b79c417dc
YARN-5767. Fix the order that resources are cleaned up from the local Public/Private caches. Contributed by Chris Trezzo
2016-10-28 15:58:04 +00:00
7146359bfd
YARN-5027. NM should clean up app log dirs after NM restart. Contributed by sandflee
2016-10-28 15:48:58 +00:00
c017171da0
HADOOP-13770. Shell.checkIsBashSupported swallowed an interrupted exception. Contributed by Wei-Chiu Chuang
2016-10-28 14:57:50 +00:00
3de130252d
YARN-5794. Fix the asflicense warnings. Contributed by Yiqun Lin
2016-10-28 12:37:37 +05:30
4df8ed63ed
YARN-4743. FairSharePolicy breaks TimSort assumption. (Zephyr Guo and Yufei Gu via kasha)
2016-10-27 17:45:48 -07:00
b2c4f24c31
YARN-2306. Add test for leakage of reservation metrics in fair scheduler. (Hong Zhiguo and Yufei Gu via subru).
2016-10-27 17:43:13 -07:00
5877f20f9c
HADOOP-10075. Update jetty dependency to version 9 (rkanter)
2016-10-27 16:09:00 -07:00
022bf783aa
YARN-4765 Split TestHBaseTimelineStorage into multiple test classes (Varun Saxena via Vrushali C)
...
(cherry picked from commit 513dcf6817dd76fde8096ff04cd888d7c908461d)
2016-10-27 15:37:36 -07:00
221582c4ab
YARN-4668. Reuse objectMapper instance in Yarn. (Yiqun Lin via gtcarrera9)
2016-10-27 15:20:17 -07:00
892a8348fc
YARN-3432. Cluster metrics have wrong Total Memory when there is reserved memory on CS. (Brahma Reddy Battula via curino)
2016-10-27 15:15:49 -07:00
79ae78dcbe
YARN-3568. TestAMRMTokens should use some random port. (Takashi Ohnishi via Subru).
2016-10-27 15:11:12 -07:00
b98fc8249f
YARN-4710. Reduce logging application reserved debug info in FSAppAttempt#assignContainer (Contributed by Yiqun Lin via Daniel Templeton)
2016-10-27 14:42:19 -07:00
9449519a25
YARN-5776. Checkstyle: MonitoringThread.Run method length is too long (miklos.szegedi@cloudera.com via rkanter)
2016-10-27 14:36:38 -07:00
dd4ed6a587
YARN-4388. Cleanup mapreduce.job.hdfs-servers from yarn-default.xml (Junping Du via Varun Saxena)
2016-10-28 02:22:25 +05:30
7e3c327d31
YARN-4831. Recovered containers will be killed after NM stateful restart. Contributed by Siqi Li
2016-10-27 20:42:52 +00:00
b4a8fbcbbc
YARN-5172. Update yarn daemonlog documentation due to HADOOP-12847. Contributed by Wei-Chiu Chuang
2016-10-27 19:43:02 +00:00
6fbfb501f2
YARN-4456. Clean up Lint warnings in nodemanager (templedf via rkanter)
2016-10-27 12:37:01 -07:00
79aeddc88f
YARN-5308. FairScheduler: Move continuous scheduling related tests to TestContinuousScheduling (Kai Sasaki via Varun Saxena)
2016-10-28 00:34:50 +05:30
b3c15e4ef7
YARN-5420. Delete org.apache.hadoop.yarn.server.resourcemanager.resource.Priority as its not necessary. Contributed by Sunil G.
2016-10-27 18:22:07 +05:30
8bd6d77a41
YARN-5757. RM REST API documentation is not up to date (Miklos Szegedi via Varun Saxena)
2016-10-27 17:21:01 +05:30
6c8830992c
YARN-3848. TestNodeLabelContainerAllocation is timing out. Contributed by Varun Saxena
2016-10-27 17:10:02 +05:30
7b4e9ec3b0
YARN-5686. DefaultContainerExecutor random working dir algorigthm skews results (Vrushali C via Varun Saxena)
2016-10-27 14:02:29 +05:30
6cc7c43866
YARN-5710. Fix inconsistent naming in class ResourceRequest (Yufei Gu via Varun Saxena)
2016-10-27 13:55:17 +05:30
5c39a419a6
YARN-5752. TestLocalResourcesTrackerImpl#testLocalResourceCache times out (Eric Badger via Varun Saxena)
2016-10-27 13:41:49 +05:30
e29cba61a0
YARN-4363. In TestFairScheduler, testcase should not create FairScheduler redundantly. Conntributed by Tao Jie.
2016-10-27 11:57:17 +05:30
b110c4b5e8
YARN-4555. TestDefaultContainerExecutor#testContainerLaunchError fails on non-english locale environment. Contributed by Takashi Ohnishi.
2016-10-27 11:34:07 +05:30
f511cc89b6
YARN-5433. Audit dependencies for Category-X. Contributed by Sangjin Lee.
2016-10-26 11:31:00 -07:00
44fdf00964
YARN-5753. fix NPE in AMRMClientImpl.getMatchingRequests() (haibochen via rkanter)
2016-10-25 23:59:39 -07:00
d3bb69a667
YARN-5575. Many classes use bare yarn. properties instead of the defined constants. Contributed by Daniel Templeton.
2016-10-26 15:32:07 +09:00
de6faae97c
YARN-5388. Deprecate and remove DockerContainerExecutor. (Daniel Templeton via kasha)
2016-10-25 13:35:47 -07:00
c88c1dc50c
YARN-5777. TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails. Contributed by Akira Ajisaka.
2016-10-25 12:15:09 -07:00
a71fc81655
YARN-5754. Null check missing for earliest in FifoPolicy. (Yufei Gu via kasha)
2016-10-24 22:19:23 -07:00
0a166b1347
YARN-5711. Propogate exceptions back to client when using hedging RM failover provider.
2016-10-24 18:59:51 -07:00
23d7d53a41
YARN-5679. TestAHSWebServices is failing (ajisakaa via rkanter)
2016-10-21 13:31:03 -07:00
44eb2bd7ae
YARN-5747. Application timeline metric aggregation in timeline v2 will lose last round aggregation when an application finishes (Li Lu via Varun Saxena)
2016-10-22 01:14:49 +05:30
754cb4e30f
YARN-5047. Refactor nodeUpdate across schedulers. (Ray Chiang via kasha)
2016-10-20 21:17:48 -07:00
a064865abf
YARN-4911. Bad placement policy in FairScheduler causes the RM to crash
2016-10-20 20:57:04 -07:00
e9c4616b5e
YARN-5561. [Atsv2] : Support for ability to retrieve apps/app-attempt/containers and entities via REST. Contributed by Rohith Sharma K S.
2016-10-19 09:45:23 -07:00
b733a6f862
YARN-5718. TimelineClient (and other places in YARN) shouldn't over-write HDFS client retry settings which could cause unexpected behavior. Contributed by Junping Du.
2016-10-18 11:06:47 -07:00
b154d3edce
YARN-5743. [Atsv2] Publish queue name and RMAppMetrics to ATS (Rohith Sharma K S via Varun Saxena)
2016-10-18 23:32:52 +05:30
f5d9235914
YARN-5466. DefaultContainerExecutor needs JavaDocs (templedf via rkanter)
2016-10-17 14:29:09 -07:00
1f304b0c7f
YARN-5699. Retrospect yarn entity fields which are publishing in events info fields. Contributed by Rohith Sharma K S.
2016-10-15 13:54:40 -07:00
cf3f43e95b
YARN-5717. Add tests for container-executor is_feature_enabled. Contributed by Sidharta Seethana
2016-10-13 20:49:07 -07:00
6476934ae5
YARN-5677. RM should transition to standby when connection is lost for an extended period. (Daniel Templeton via kasha)
2016-10-11 22:07:10 -07:00
6378845f9e
YARN-4464. Lower the default max applications stored in the RM and store. (Daniel Templeton via kasha)
2016-10-11 21:42:08 -07:00
ecb51b857a
YARN-5551. Ignore file backed pages from memory computation when smaps is enabled. Contributed by Rajesh Balamohan
2016-10-11 15:12:43 +00:00
0773ffd0f8
YARN-5057. Resourcemanager.security.TestDelegationTokenRenewer fails in trunk. Contributed by Jason Lowe.
2016-10-10 18:04:47 -04:00
459a4833a9
YARN-5659. getPathFromYarnURL should use standard methods. Contributed by Sergey Shelukhin.
2016-10-07 07:46:08 -07:00
4d2f380d78
YARN-5101. YARN_APPLICATION_UPDATED event is parsed in ApplicationHistoryManagerOnTimelineStore#convertToApplicationReport with reversed order. Contributed by Sunil G.
2016-10-06 20:42:36 +05:30
272a21747e
MAPREDUCE-6789. Fix TestAMWebApp failure. Contributed by Daniel Templeton.
2016-10-06 15:57:15 +09:00
d65b957776
YARN-5702. Refactor TestPBImplRecords for reuse in other YARN modules. Contributed by Subru Krishnan
2016-10-05 12:16:49 -07:00
0992708d79
YARN-5704. Provide config knobs to control enabling/disabling new/work in progress features in container-executor. Contributed by Sidharta Seethana.
2016-10-05 15:31:38 +05:30
31f8da22d0
YARN-3139. Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler. Contributed by Wangda Tan
2016-10-04 17:23:13 -07:00
736d33cddd
YARN-4767. Network issues can cause persistent RM UI outage. (Daniel Templeton via kasha)
2016-10-03 14:35:57 -07:00
0da54e8848
YARN-5672. FairScheduler: Wrong queue name in log when adding application. (Wilfred Spiegelenburg via kasha)
2016-10-03 06:03:46 -07:00
6e130c308c
YARN-4855. Should check if node exists when replace nodelabels. Contributeed by Tao Jie
2016-10-03 02:02:26 -04:00
82857037b6
YARN-5678. Log demand as demand in FSLeafQueue and FSParentQueue. (Yufei Gu via kasha)
2016-10-02 22:09:43 -07:00
3a3697deab
YARN-5384. Expose priority in ReservationSystem submission APIs. (Sean Po via Subru).
2016-09-30 19:41:43 -07:00
2549ee9d4c
YARN-5693. Reduce loglevel to Debug in ContainerManagementProtocolProxy and AMRMClientImpl (yufeigu via rkanter)
2016-09-30 13:30:55 -07:00
10be45986c
YARN-5486. Update OpportunisticContainerAllocatorAMService::allocate method to handle OPPORTUNISTIC container requests. (Konstantinos Karanasos via asuresh)
2016-09-29 15:11:41 -07:00
2ae5a3a5bf
YARN-4205. Add a service for monitoring application life time out. Contributed by Rohith Sharma K S
2016-09-29 22:00:31 +08:00
bcb2528a51
YARN-5400. Light cleanup in ZKRMStateStore (templedf via rkanter)
2016-09-28 14:56:41 -07:00
9b0fd01d2e
YARN-5599. Publish AM launch command to ATS (Rohith Sharma K S via Varun Saxena)
2016-09-28 16:10:10 +05:30
bc2656f09f
YARN-5662. Provide an option to enable ContainerMonitor. Contributed by Jian He.
2016-09-28 15:18:18 +05:30
1831be8e73
YARN-3142. Improve locks in AppSchedulingInfo. (Varun Saxena via wangda)
2016-09-27 11:54:55 -07:00
875062b5bc
HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs. (vinodkv via wangda)
2016-09-27 11:26:45 -07:00
ebf528c443
YARN-5660. Wrong audit constants are used in Get/Put of priority in RMWebService. Contributed by Rohith Sharma K S
2016-09-27 16:37:59 +05:30
4815d024c5
YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh
2016-09-26 08:46:54 -07:00
2f163cd5cf
Revert "YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh"
...
This reverts commit fe644bafe7
2016-09-26 08:36:59 -07:00
fe644bafe7
YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh
2016-09-26 22:41:16 +08:00
14a696f369
YARN-5663. Small refactor in ZKRMStateStore. Contributed by Oleksii Dymytrov.
2016-09-26 15:00:01 +09:00
e4e72db5f9
YARN-3877. YarnClientImpl.submitApplication swallows exceptions. Contributed by Varun Saxena
2016-09-25 17:36:30 +05:30
4d16d2d72f
YARN-5664. Fix Yarn documentation to link to correct versions. Contributed by Xiao Chen
2016-09-24 23:21:03 +05:30
6e849cb658
YARN-5622. TestYarnCLI.testGetContainers fails due to mismatched date formats. Contributed by Eric Badger.
2016-09-23 14:09:25 -05:00
b8a2d7b8fc
YARN-5539. TimelineClient failed to retry on java.net.SocketTimeoutException: Read timed out (Junping Du via Varun Saxena)
2016-09-23 13:27:31 +05:30
3293a7d92d
Revert "TimelineClient failed to retry on java.net.SocketTimeoutException: Read timed out"
...
This reverts commit 2e6ee95716
2016-09-23 13:25:46 +05:30
2e6ee95716
TimelineClient failed to retry on java.net.SocketTimeoutException: Read timed out
2016-09-23 13:23:54 +05:30
d0372dc613
YARN-3692. Allow REST API to set a user generated message when killing an application. Contributed by Rohith Sharma K S
2016-09-23 06:30:49 +05:30
5ffd4b7c1e
YARN-4973. YarnWebParams next.fresh.interval should be next.refresh.interval (templedf via rkanter)
2016-09-22 16:45:34 -07:00
5a58bfee30
YARN-4591. YARN Web UIs should provide a robots.txt. (Sidharta Seethana via wangda)
2016-09-20 17:20:50 -07:00
9f03b403ec
YARN-5656. Fix ReservationACLsTestBase. (Sean Po via asuresh)
2016-09-20 12:27:17 -07:00
e45307c9a0
Addendum patch for fix javadocs failure which is caused by YARN-3141. (wangda)
2016-09-20 11:21:01 -07:00
c6d1d742e7
YARN-5655. TestContainerManagerSecurity#testNMTokens is asserting. Contributed by Robert Kanter
2016-09-20 14:15:06 +00:00
2b66d9ec5b
YARN-3140. Improve locks in AbstractCSQueue/LeafQueue/ParentQueue. Contributed by Wangda Tan
2016-09-20 15:03:31 +08:00
7558dbbb48
YARN-5540. Scheduler spends too much time looking at empty priorities. Contributed by Jason Lowe
2016-09-19 20:31:35 +00:00
b8a30f2f17
YARN-3141. Improve locks in SchedulerApplicationAttempt/FSAppAttempt/FiCaSchedulerApp. Contributed by Wangda Tan
2016-09-19 17:08:01 +08:00
ea29e3bc27
YARN-5577. [Atsv2] Document object passing in infofilters with an example (Rohith Sharma K S via Varun Saxena)
2016-09-19 14:33:06 +05:30
3552c2b99d
YARN-5637. Changes in NodeManager to support Container rollback and commit. (asuresh)
2016-09-18 10:55:18 -07:00
f67237cbe7
YARN-5657. Fix TestDefaultContainerExecutor. (asuresh)
2016-09-17 09:32:05 -07:00
4174b9756c
YARN-5642. Typos in 9 log messages. Contributed by Mehran Hassani
2016-09-17 10:35:39 +05:30
ade7c2bc9c
YARN-4232. TopCLI console support for HA mode. Contributed by Bibin A Chundatt
2016-09-17 09:52:39 +05:30
58bae35447
YARN-5163. Migrate TestClientToAMTokens and TestClientRMTokens tests from the old RPC engine. Contributed by Wei Zhou and Kai Zheng
2016-09-18 08:43:36 +08:00
40b5a59b72
YARN-5620. Core changes in NodeManager to support re-initialization of Containers with new launchContext. (asuresh)
2016-09-15 07:15:11 -07:00
e793309735
YARN-5630. NM fails to start after downgrade from 2.8 to 2.7. Contributed by Jason Lowe
2016-09-13 14:41:27 +00:00
729de3e6b6
YARN-5631. Missing refreshClusterMaxPriority usage in rmadmin help message. Contributed by Kai Sasaki.
2016-09-13 15:23:24 +05:30
58ed4fa544
HADOOP-13341. Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTS
...
This commit includes the following changes:
HADOOP-13356. Add a function to handle command_subcommand_OPTS
HADOOP-13355. Handle HADOOP_CLIENT_OPTS in a function
HADOOP-13554. Add an equivalent of hadoop_subcmd_opts for secure opts
HADOOP-13562. Change hadoop_subcommand_opts to use only uppercase
HADOOP-13358. Modify HDFS to use hadoop_subcommand_opts
HADOOP-13357. Modify common to use hadoop_subcommand_opts
HADOOP-13359. Modify YARN to use hadoop_subcommand_opts
HADOOP-13361. Modify hadoop_verify_user to be consistent with hadoop_subcommand_opts (ie more granularity)
HADOOP-13564. modify mapred to use hadoop_subcommand_opts
HADOOP-13563. hadoop_subcommand_opts should print name not actual content during debug
HADOOP-13360. Documentation for HADOOP_subcommand_OPTS
This closes apache/hadoop#126
2016-09-12 11:10:00 -07:00
c0e492e50f
YARN-5616. Clean up WeightAdjuster. (Yufei Gu via kasha)
2016-09-06 22:40:20 -07:00
e6fcfe28e3
YARN-5576. Allow resource localization while container is running. Contributed by Jian He.
2016-09-06 20:01:45 +05:30
f9557127b3
YARN-5608. TestAMRMClient.setup() fails with ArrayOutOfBoundsException. Contributed by Daniel Templeton.
2016-09-06 14:09:45 +05:30
f6ea9be547
YARN-5264. Store all queue-specific information in FSQueue. (Yufei Gu via kasha)
2016-09-02 14:56:29 -07:00
378f624a39
YARN-5549. AMLauncher#createAMContainerLaunchContext() should not log the command to be launched indiscriminately. (Daniel Templeton via rchiang)
2016-09-02 11:14:35 -07:00
05f5c0f631
YARN-5555. Scheduler UI: "% of Queue" is inaccurate if leaf queue is hierarchically nested. Contributed by Eric Payne.
2016-09-02 16:02:01 +05:30
74f4bae455
YARN-5566. Client-side NM graceful decom is not triggered when jobs finish. (Robert Kanter via kasha)
2016-09-01 14:44:01 -07:00
08f55ccbb0
YARN-5596. Fix failing unit test in TestDockerContainerRuntime. Contributed by Sidharta Seethana.
2016-09-01 14:08:51 +05:30
d6d9cff21b
YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh)
2016-08-30 15:52:29 -07:00
ed6ff5cd78
YARN-5550. TestYarnCLI#testGetContainers should format according to CONTAINER_PATTERN. Contributed by Jonathan Hung.
2016-08-29 14:46:00 -07:00
4cbe61407d
YARN-5560. Clean up bad exception catching practices in TestYarnClient. Contributed by Sean Po
2016-08-29 16:14:55 +00:00
19c743c1bb
YARN-4889. Changes in AMRMClient for identifying resource-requests explicitly. (Arun Suresh via wangda)
2016-08-26 17:14:12 -07:00
b930dc3ec0
YARN-5327. API changes required to support recurring reservations in the YARN ReservationSystem. (Sangeetha Abdu Jyothi via Subru).
2016-08-26 16:58:47 -07:00
cde3a00526
YARN-5373. NPE listing wildcard directory in containerLaunch. (Daniel Templeton via kasha)
2016-08-26 11:04:33 -07:00
9ef632f3b0
YARN-5557. Add localize API to the ContainerManagementProtocol. Contributed by Jian He.
2016-08-26 09:04:44 -07:00
46e02ab719
YARN-3940. Application moveToQueue should check NodeLabel permission. Contributed by Bibin A Chundatt
2016-08-26 20:19:11 +05:30
bfb4d95059
YARN-5430. Return container's ip and host from NM ContainerStatus call. Contributed by Jian He.
2016-08-26 16:30:18 +05:30
27c3b86252
YARN-5564. Fix typo in RM_SCHEDULER_RESERVATION_THRESHOLD_INCREMENT_MULTIPLE. Contributed by Ray Chiang
2016-08-26 08:47:21 +05:30
3d86110a5c
YARN-5389. TestYarnClient#testReservationDelete fails. Contributed by Sean Po
2016-08-25 14:42:06 +00:00
525d52bf7c
YARN-5042. Mount /sys/fs/cgroup into Docker containers as read only mount. Contributed by luhuichun.
2016-08-25 14:18:26 +05:30
79603f5882
YARN-5537. Fix intermittent failure of TestAMRMClient#testAMRMClientWithContainerResourceChange (Bibin A Chundatt via Varun Saxena)
2016-08-25 14:10:34 +05:30
143c59e4c5
YARN-4491. yarn list command to support filtering by tags. Contributed by Varun Saxena
2016-08-24 01:53:02 +05:30
0d5997d2b9
YARN-5544. TestNodeBlacklistingOnAMFailures fails on trunk. Contributed by Sunil G.
2016-08-23 14:37:39 +05:30
2da32a6ef9
YARN-4685. Disable AM blacklisting by default to mitigate situations that application get hanged. (Rohith Sharma K S via wangda)
2016-08-19 16:31:16 -07:00
444b2ea7af
YARN-3388. Allocation in LeafQueue could get stuck because DRF calculator isn't well supported when computing user-limit. (Nathan Roberts via wangda)
2016-08-19 16:28:32 -07:00
091dd19e86
YARN-5533. JMX AM Used metrics for queue wrong when app submited to nodelabel partition (Bibin A Chundatt via Varun Saxena)
2016-08-19 17:30:17 +05:30
8aed374182
Revert "YARN-5533. JMX AM Used metrics for queue wrong when app submited to nodelabel partition (Bibin A Chundatt via Varun Saxena)"
...
This reverts commit 59557e85a4
2016-08-19 16:14:16 +05:30
59557e85a4
YARN-5533. JMX AM Used metrics for queue wrong when app submited to nodelabel partition (Bibin A Chundatt via Varun Saxena)
2016-08-19 15:01:48 +05:30
0da69c324d
YARN-4676. Automatic and Asynchronous Decommissioning Nodes Status Tracking. Contributed by Diniel Zhi.
...
(cherry picked from commit d464483bf7
2016-08-18 07:27:23 -07:00
913a895136
YARN=5526. DrainDispacher#serviceStop blocked if setDrainEventsOnStop invoked (sandflee via Varun Saxena)
2016-08-18 11:38:13 +05:30
20f0eb871c
YARN-4702. FairScheduler: Allow setting maxResources for ad hoc queues. (Daniel Templeton via kasha)
2016-08-17 17:40:20 -07:00
e3037c5641
YARN-5523. Yarn running container log fetching causes OutOfMemoryError (Xuan Gong via Varun Saxena)
2016-08-18 01:45:33 +05:30
7f05ff7a4e
YARN-5455. Update Javadocs for LinuxContainerExecutor. Contributed by Daniel Templeton.
2016-08-17 15:34:58 +05:30
b427ce12bc
YARN-5475. Fix test failure of TestAggregatedLogFormat#testReadAcontainerLogs1 (Jun Gong via Varun Saxena)
2016-08-16 20:24:53 +05:30
ffe1fff526
YARN-5514. Clarify DecommissionType.FORCEFUL comment (Vrushali C via Varun Saxena)
2016-08-16 14:05:41 +05:30
24249115bf
YARN-5521. Fix random failure of TestCapacityScheduler#testKillAllAppsInQueue (sandflee via Varun Saxena)
2016-08-16 00:03:29 +05:30
d677b68c25
YARN-5491. Fix random failure of TestCapacityScheduler#testCSQueueBlocked (Bibin A Chundatt via Varun Saxena)
2016-08-15 03:31:21 +05:30
23c6e3c4e4
YARN-5476. Non existent application reported as ACCEPTED by YarnClientImpl (Junping Du via Varun Saxena)
2016-08-12 20:37:58 +05:30
874577a67d
YARN-4833. For Queue AccessControlException client retries multiple times on both RM. Contributed by Bibin A Chundatt
2016-08-12 01:09:41 +05:30
5199db387d
YARN-5492. TestSubmitApplicationWithRMHA is failing sporadically during precommit builds. Contributed by Vrushali C.
2016-08-11 11:50:46 +05:30
666ad0b67b
YARN-2398. TestResourceTrackerOnHA crashes. Contributed by Ajith S.
2016-08-11 11:27:00 +05:30
aea3e65749
YARN-5137. Make DiskChecker pluggable in NodeManager. (Yufei Gu via rchiang)
2016-08-10 19:23:29 -07:00
89c0bff719
YARN-5495. Remove import wildcard in CapacityScheduler. Contributed by Ray Chiang.
2016-08-11 00:35:19 +05:30
5c95bb315b
YARN-5382. RM does not audit log kill request for active applications. Contributed by Vrushali C
2016-08-10 18:25:54 +00:00
e0b570dffb
YARN-5483. Optimize RMAppAttempt#pullJustFinishedContainers. Contributed by sandflee
2016-08-10 18:04:18 +00:00
7992c0b42c
YARN-5343. TestContinuousScheduling#testSortedNodes fails intermittently. (Yufei Gu via kasha)
2016-08-09 16:51:03 -07:00
3cd386bd97
YARN-5474. Typo mistake in AMRMClient#getRegisteredTimeineClient API. Contributed by Naganarasimha G R.
2016-08-09 22:28:54 +05:30
522ddbde79
YARN-5394. Remove bind-mount /etc/passwd for Docker containers. Contributed by Zhankun Tang.
2016-08-09 16:04:09 +05:30
82c9e06101
YARN-5457. Refactor DistributedScheduling framework to pull out common functionality. (asuresh)
2016-08-09 00:42:29 -07:00
83a2ffec40
YARN-5470. Addedum to differentiate exactly matching of log-files with regex in yarn log CLI. Contributed by Xuan Gong.
2016-08-08 14:28:07 -07:00
37d939a453
YARN-4910. Fix incomplete log info in ResourceLocalizationService (Jun Gong via Varun Saxena)
2016-08-08 22:12:53 +05:30
131d58a24e
YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang
2016-08-07 01:08:13 +05:30
d81b8163b4
YARN-4624. NPE in PartitionQueueCapacitiesInfo while accessing Schduler UI. Contributed by Brahma Reddy Battula
2016-08-06 01:13:36 +05:30
e605d47df0
YARN-5470. Differentiate exactly match with regex in yarn log CLI. Contributed by Xuan Gong.
2016-08-05 10:55:29 -07:00
3f100d76ff
YARN-4888. Changes in scheduler to identify resource-requests explicitly by allocation-id. (Subru Krishnan via wangda)
2016-08-05 10:43:35 -07:00
e0d131f055
YARN-4091. Add REST API to retrieve scheduler activity. (Chen Ge and Sunil G via wangda)
2016-08-05 10:27:34 -07:00
d9a354c2f3
YARN-5333. Some recovered apps are put into default queue when RM HA. Contributed by Jun Gong.
2016-08-05 21:35:49 +05:30
4a26221021
YARN-5429. Fix return related javadoc warnings in yarn-api (Vrushali C via Varun Saxena)
2016-08-05 16:27:53 +05:30
08e33381f5
YARN-5459. Add support for docker rm. Contributed by Shane Kumpf.
2016-08-04 14:08:34 +05:30
db4a61dc61
YARN-5469. Increase timeout of TestAmFilter.testFilter. Contributed by Eric Badger
2016-08-03 19:51:44 +00:00
db646540f0
YARN-5462. TestNodeStatusUpdater.testNodeStatusUpdaterRetryAndNMShutdown fails intermittently. Contributed by Eric Badger
2016-08-03 19:17:25 +00:00
4d92aefd35
YARN-4280. CapacityScheduler reservations may not prevent indefinite postponement on a busy cluster. Contributed by Kuhu Shukla
2016-08-03 18:53:14 +00:00
d848184e90
YARN-5460. Change container runtime type logging in DelegatingLinuxContainerRuntime to debug. Contributed by Shane Kumpf.
2016-08-03 12:02:39 +05:30
b913677365
YARN-5456. container-executor support for FreeBSD, NetBSD, and others if conf path is absolute. Contributed by Allen Wittenauer.
2016-08-02 22:24:34 -07:00
3818393297
YARN-5226. Remove AHS enable check from LogsCLI#fetchAMContainerLogs. Contributed by Xuan Gong.
2016-08-02 10:41:46 -07:00
7fc70c6422
YARN-5458. Rename DockerStopCommandTest to TestDockerStopCommand. Contributed by Shane Kumpf.
2016-08-02 16:56:45 +05:30
2e7c2a13a8
YARN-5443. Add support for docker inspect command. Contributed by Shane Kumpf.
2016-08-02 16:55:10 +05:30
95694b70cd
YARN-5444. Fix failing unit tests in TestLinuxContainerExecutorWithMocks. Contributed by Yufei Gu.
2016-08-01 16:08:38 +05:30
e5766b1dbe
YARN-5113. Refactoring and other clean-up for distributed scheduling. (Konstantinos Karanasos via asuresh)
2016-07-31 11:48:25 -07:00
ef501b1a0b
YARN-5121. fix some container-executor portability issues. Contributed by Allen Wittenauer.
2016-07-30 08:26:19 -07:00
95f2b98597
YARN-5434. Add -client|server argument for graceful decommmission. Contributed by Robert Kanter.
2016-07-29 10:26:11 -07:00
7086fc72ee
YARN-5436. Race in AsyncDispatcher can cause random test failures in Tez (probably YARN also). (Zhiyuan Yang via gtcarrera9)
2016-07-28 16:50:57 -07:00
4e756d7271
YARN-5203.Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API. Contributed by Ellen Hui.
2016-07-28 16:03:24 -07:00
a1890c32c5
YARN-5438. TimelineClientImpl leaking FileSystem Instances causing Long running services like HiverServer2 daemon going OOM. Contributed by Rohith Sharma K S
2016-07-28 21:47:13 +00:00
26de4f0de7
YARN-5440. Use AHSClient in YarnClient when TimelineServer is running. (Xuan Gong via gtcarrera9)
2016-07-28 10:23:18 -07:00
7f3c306e2e
YARN-5432. Lock already held by another process while LevelDB cache store creation for dag. Contributed by Li Lu.
2016-07-28 06:35:24 -07:00
8d06bda337
YARN-5431. TimelineReader daemon start should allow to pass its own reader opts (Rohith Sharma K S via Varun Saxena)
2016-07-28 08:36:09 +05:30
d2cbfd7de3
YARN-5441. Fixing minor Scheduler test case failures
2016-07-27 18:21:58 -07:00
54fe17a607
HADOOP-9427. Use JUnit assumptions to skip platform-specific tests. Contributed by Gergely Novák.
2016-07-27 19:41:09 +09:00
d62e121ffc
YARN-5195. RM intermittently crashed with NPE while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler. (sandflee via wangda)
2016-07-26 21:22:59 -07:00
2d8d183b19
YARN-5351. ResourceRequest should take ExecutionType into account during comparison. (Konstantinos Karanasos via asuresh)
2016-07-26 19:08:30 -07:00
49969b16cd
YARN-5342. Improve non-exclusive node partition resource allocation in Capacity Scheduler. (Sunil G via wangda)
2016-07-26 18:14:09 -07:00
Junping Du
Daniel Templeton
Sunil G
Karthik Kambatla
Sangjin Lee
Jason Lowe
Karthik Kambatla
Rohith Sharma K S
Wangda Tan
Ray Chiang
Andrew Wang
Jian He
Arun Suresh
Allen Wittenauer
Subru Krishnan
bibinchundatt
Akira Ajisaka
Naganarasimha
Varun Saxena
Junping Du
Robert Kanter
Varun Vasudev
Xuan
Steve Loughran
Li Lu
Kai Zheng
Wei-Chiu Chuang
Xiao Chen
Mingliang Liu
Eric Payne
Sreenath Somarajapuram
Chris Douglas
Subru Krishnan
Vrushali Channapattan
Carlo Curino
Kihwal Lee
Zhe Zhang
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
Chris Nauroth