Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,765 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2237 Commits

Author SHA1 Message Date
Junping Du cfee02b3bd YARN-4019. Add JvmPauseMonitor to ResourceManager and NodeManager. Contributed by Robert Kanter. 2015-08-06 06:49:45 -07:00
Arun Suresh 154c9d2e42 YARN-3961. Expose pending, running and reserved containers of a queue in REST api and yarn top (adhoot via asuresh) 2015-08-05 23:14:14 -07:00
rohithsharmaks df9e7280db YARN-3992. TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) 2015-08-06 10:43:37 +05:30
Jian He ba2313d614 YARN-3983. Refactored CapacityScheduleri#FiCaSchedulerApp to easier extend container allocation logic. Contributed by Wangda Tan 2015-08-05 13:47:40 -07:00
Arun Suresh f271d37735 YARN-3736. Add RMStateStore apis to store and load accepted reservations for failover (adhoot via asuresh) 2015-08-05 12:57:12 -07:00
Xuan c3364ca8e7 YARN-4004. container-executor should print output of docker logs if the 
docker container exits with non-0 exit status. Contributed by Varun
Vasudev
 2015-08-03 18:10:11 -07:00
Xuan 0306d902f5 YARN-3543. ApplicationReport should be able to tell whether the 
Application is AM managed or not. Contributed by Rohith Sharma K S
 2015-08-03 15:46:00 -07:00
Jason Lowe 469cfcd695 YARN-3965. Add startup timestamp to nodemanager UI. Contributed by Hong Zhiguo 2015-08-03 15:53:32 +00:00
Jonathan Eagles 3cd02b9522 YARN-3978. Configurably turn off the saving of container info in Generic AHS (Eric Payne via jeagles) 2015-08-03 10:38:05 -05:00
Jason Lowe 32e490b6c0 YARN-3990. AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected. Contributed by Bibin A Chundatt 2015-07-31 17:37:24 +00:00
Zhihai Xu ab80e27703 YARN-433. When RM is catching up with node updates then it should not expire acquired containers. Contributed by Xuan Gong 2015-07-30 21:57:11 -07:00
Wangda Tan 91b42e7d6e YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) 2015-07-30 10:00:31 -07:00
Wangda Tan 8acb30b016 YARN-3963. AddNodeLabel on duplicate label addition shows success. (Bibin A Chundatt via wangda) 2015-07-30 09:42:55 -07:00
rohithsharmaks c020b62cf8 YARN-3919. NPEs' while stopping service after exception during CommonNodeLabelsManager#start. (varun saxena via rohithsharmaks) 2015-07-29 23:30:47 +05:30
Karthik Kambatla 5205a330b3 YARN-2768. Avoid cloning Resource in FSAppAttempt#updateDemand. (Hong Zhiguo via kasha) 2015-07-29 09:42:32 -07:00
Jason Lowe 2b2bd92146 YARN-3950. Add unique SHELL_ID environment variable to DistributedShell. Contributed by Robert Kanter 2015-07-29 15:16:40 +00:00
Xuan f170934215 YARN-3982. container-executor parsing of container-executor.cfg broken 
in trunk and branch-2. Contributed by Varun Vasudev
 2015-07-27 23:45:58 -07:00
Jian He 3572ebd738 YARN-3846. RM Web UI queue filter is not working for sub queue. Contributed by Mohammad Shahid Khan 2015-07-27 17:12:05 -07:00
Varun Vasudev 3e6fce91a4 YARN-3853. Add docker container runtime support to LinuxContainterExecutor. Contributed by Sidharta Seethana. 2015-07-27 11:57:40 -07:00
Varun Vasudev f36835ff9b YARN-3852. Add docker container support to container-executor. Contributed by Abin Shahab. 2015-07-27 10:14:51 -07:00
Akira Ajisaka 42d4e0ae99 YARN-3958. TestYarnConfigurationFields should be moved to hadoop-yarn-api module. Contributed by Varun Saxena. 2015-07-27 11:43:25 +09:00
ccurino 156f24ead0 YARN-3656. LowCost: A Cost-Based Placement Agent for YARN Reservations. (Jonathan Yaniv and Ishai Menache via curino) 2015-07-25 07:39:47 -07:00
Wangda Tan a3bd7b4a59 YARN-3973. Recent changes to application priority management break reservation system from YARN-1051 (Carlo Curino via wangda) 2015-07-24 16:44:18 -07:00
Jason Lowe ff9c13e0a7 YARN-3925. ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks. Contributed by zhihai xu 2015-07-24 22:14:39 +00:00
Jian He 83fe34ac08 YARN-3026. Move application-specific container allocation logic from LeafQueue to FiCaSchedulerApp. Contributed by Wangda Tan 2015-07-24 14:00:25 -07:00
carlo curino fc42fa8ae3 YARN-3969. Updating CHANGES.txt to reflect the correct set of branches where this is committed 2015-07-24 13:38:44 -07:00
Karthik Kambatla d19d187753 YARN-3957. FairScheduler NPE In FairSchedulerQueueInfo causing scheduler page to return 500. (Anubhav Dhoot via kasha) 2015-07-24 11:44:37 -07:00
Xuan fbd6063269 YARN-3967. Fetch the application report from the AHS if the RM does not 
know about it. Contributed by Mit Desai
 2015-07-24 10:15:54 -07:00
carlo curino 0fcb4a8cf2 YARN-3969. Allow jobs to be submitted to reservation that is active but does not have any allocations. (subru via curino) 2015-07-23 19:33:59 -07:00
Rohith Sharma K S e202efaf93 YARN-3845. Scheduler page does not render RGBA color combinations in IE11. (Contributed by Mohammad Shahid Khan) 2015-07-24 12:43:06 +05:30
Robert Kanter 1d3026e7b3 YARN-3900. Protobuf layout of yarn_security_token causes errors in other protos that include it (adhoot via rkanter) 2015-07-23 14:46:54 -07:00
Wangda Tan 3bba180051 YARN-3941. Proportional Preemption policy should try to avoid sending duplicate PREEMPT_CONTAINER event to scheduler. (Sunil G via wangda) 2015-07-23 10:07:57 -07:00
Junping Du ee98d6354b YARN-2019. Retrospect on decision of making RM crashed if any exception throw in ZKRMStateStore. Contributed by Jian He. 2015-07-22 17:52:35 -07:00
Wangda Tan 06e5dd2c84 YARN-3956. Fix TestNodeManagerHardwareUtils fails on Mac (Varun Vasudev via wangda) 2015-07-22 12:01:41 -07:00
rohithsharmaks 8376ea3297 YARN-3954. Fix TestYarnConfigurationFields#testCompareConfigurationClassAgainstXml. (varun saxena via rohithsharmaks) 2015-07-23 00:28:24 +05:30
Wangda Tan 76ec26de80 YARN-3932. SchedulerApplicationAttempt#getResourceUsageReport and UserInfo should based on total-used-resources. (Bibin A Chundatt via wangda) 2015-07-22 11:54:02 -07:00
Jian He 393fe71771 YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. Contributed by Varun Saxena 2015-07-21 15:05:41 -07:00
Allen Wittenauer 3b7ffc4f3f YARN-3261. rewrite resourcemanager restart doc to remove roadmap bits (Gururaj Shetty via aw) 2015-07-21 10:00:34 -07:00
Wangda Tan c39ca541f4 YARN-2003. Support for Application priority : Changes in RM and Capacity Scheduler. (Sunil G via wangda) 2015-07-21 09:57:23 -07:00
Allen Wittenauer da2d1ac4bc YARN-3915. scmadmin help message correction (Bibin A Chundatt via aw) 2015-07-21 09:44:45 -07:00
Arun Suresh 176131f12b Pulling in YARN-3535 to branch 2.7.x 2015-07-18 10:05:54 -07:00
Colin Patrick Mccabe 419c51d233 YARN-3844. Make hadoop-yarn-project Native code -Wall-clean (Alan Burlison via Colin P. McCabe) 2015-07-17 11:38:59 -07:00
Jonathan Eagles 7faae0e6fe YARN-3905. Application History Server UI NPEs when accessing apps run after RM restart (Eric Payne via jeagles) 2015-07-17 11:02:11 -05:00
Arun Suresh 9b272ccae7 YARN-3535. Scheduler must re-request container resources when RMContainer transitions from ALLOCATED to KILLED (rohithsharma and peng.zhang via asuresh) 2015-07-17 04:31:34 -07:00
Wangda Tan 3540d5fe4b YARN-3885. ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2 level. (Ajith S via wangda) 2015-07-16 16:13:32 -07:00
Wangda Tan fa2b63ed16 YARN-3930. FileSystemNodeLabelsStore should make sure edit log file closed when exception is thrown. (Dian Fu via wangda) 2015-07-16 16:06:20 -07:00
Tsuyoshi Ozawa 1ba2986dee YARN-3805. Update the documentation of Disk Checker based on YARN-90. Contributed by Masatake Iwasaki. 2015-07-16 17:52:38 +09:00
Tsuyoshi Ozawa f02dd146f5 YARN-3174. Consolidate the NodeManager and NodeManagerRestart documentation into one. Contributed by Masatake Iwasaki. 2015-07-16 15:22:30 +09:00
Tsuyoshi Ozawa edcaae44c1 YARN-3170. YARN architecture document needs updating. Contirubted by Brahma Reddy Battula. 2015-07-15 15:42:41 +09:00
Arun Suresh ac94ba3e18 YARN-3453. Ensure preemption logic in FairScheduler uses DominantResourceCalculator in DRF queues to prevent unnecessary thrashing. (asuresh) 2015-07-14 00:23:55 -07:00
Jian He 2466460d4c Revert "YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. (Varun Saxena via kasha)" 
This reverts commit aa067c6aa4.
 2015-07-13 14:30:35 -07:00
Akira Ajisaka 19295b36d9 YARN-3381. Fix typo InvalidStateTransitonException. Contributed by Brahma Reddy Battula. 2015-07-13 17:52:13 +09:00
Wangda Tan 5ed1fead6b YARN-3894. RM startup should fail for wrong CS xml NodeLabel capacity configuration. (Bibin A Chundatt via wangda) 2015-07-12 21:52:11 -07:00
Akira Ajisaka d6675606dc YARN-3069. Document missing properties in yarn-default.xml. Contributed by Ray Chiang. 2015-07-13 12:57:01 +09:00
Gera Shegalov d7319dee37 YARN-3917. getResourceCalculatorPlugin for the default should intercept all exceptions. (gera) 2015-07-11 22:29:36 -07:00
Wangda Tan 1df39c1efc YARN-3849. Too much of preemption activity causing continuos killing of containers across queues. (Sunil G via wangda) 2015-07-11 10:26:46 -07:00
Zhijie Shen 1ea36299a4 YARN-3116. RM notifies NM whether a container is an AM container or normal task container. Contributed by Giovanni Matteo Fumarola. 2015-07-10 18:58:10 -07:00
Ming Ma 08244264c0 YARN-3445. Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) 2015-07-10 08:30:10 -07:00
Xuan 5214876792 YARN-3888. ApplicationMaster link is broken in RM WebUI when appstate is 
NEW. Contributed by Bibin A Chundatt
 2015-07-09 21:37:33 -07:00
carlo curino 0e602fa3a1 YARN-3800. Reduce storage footprint for ReservationAllocation. Contributed by Anubhav Dhoot. 2015-07-09 16:51:59 -07:00
Karthik Kambatla aa067c6aa4 YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. (Varun Saxena via kasha) 2015-07-09 09:48:29 -07:00
Karthik Kambatla 527c40e4d6 YARN-1012. Report NM aggregated container resource utilization in heartbeat. (Inigo Goiri via kasha) 2015-07-09 09:35:14 -07:00
Akira Ajisaka d6325745e2 YARN-3690. [JDK8] 'mvn site' fails. Contributed by Brahma Reddy Battula. 2015-07-08 15:42:31 +09:00
Jian He c9dd2cada0 YARN-3892. Fixed NPE on RMStateStore#serviceStop when CapacityScheduler#serviceInit fails. Contributed by Bibin A Chundatt 2015-07-07 14:16:21 -07:00
Varun Vasudev c40bdb56a7 YARN-2194. Fix bug causing CGroups functionality to fail on RHEL7. Contributed by Wei Yan. 2015-07-07 16:59:29 +05:30
Devaraj K af63427c6d YARN-3837. javadocs of TimelineAuthenticationFilterInitializer give wrong 
prefix for auth options. Contributed by Bibin A Chundatt.
 2015-07-07 12:06:30 +05:30
Vinod Kumar Vavilapalli (I am also known as @tshooter.) bf89ddb9b8 Release process for 2.7.1: Set the release date for 2.7.1. 2015-07-06 16:39:59 -07:00
Xuan 688617d6d7 YARN-3882. AggregatedLogFormat should close aclScanner and ownerScanner 
after create them. Contributed by zhihai xu
 2015-07-04 21:51:58 -07:00
Devaraj K 37d7395773 YARN-3875. FSSchedulerNode#reserveResource() doesn't print Application Id 
properly in log. Contributed by Bibin A Chundatt.
 2015-07-02 10:20:31 +05:30
Wangda Tan 0e4b06690f YARN-3508. Prevent processing preemption events on the main RM dispatcher. (Varun Saxena via wangda) 2015-07-01 17:32:22 -07:00
Jason Lowe b5cdf78e8e YARN-3793. Several NPEs when deleting local files on NM recovery. Contributed by Varun Saxena 2015-07-01 21:13:32 +00:00
Devaraj K 80a68d6056 YARN-3830. AbstractYarnScheduler.createReleaseCache may try to clean a 
null attempt. Contributed by nijel.
 2015-07-01 19:03:44 +05:30
Devaraj K 7405c59799 YARN-3823. Fix mismatch in default values for 
yarn.scheduler.maximum-allocation-vcores property. Contributed by Ray
Chiang.
 2015-07-01 11:11:26 +05:30
Colin Patrick Mccabe d0cc0380b5 YARN-3827. Migrate YARN native build to new CMake framework (Alan Burlison via Colin P. McCabe) 2015-06-30 16:24:19 -07:00
Gera Shegalov 6f2a41e37d YARN-3768. ArrayIndexOutOfBoundsException with empty environment variables. (Zhihai Xu via gera) 2015-06-30 14:59:33 -07:00
Jian He 4672315e2d YARN-3770. SerializedException should also handle java.lang.Error on de-serialization. Contributed by Lavkesh Lahngir 2015-06-29 14:31:49 -07:00
Jian He 62e583c7dc YARN-3695. ServerProxy (NMProxy, etc.) shouldn't retry forever for non network exception. Contributed by Raju Bairishetti 2015-06-29 13:37:32 -07:00
Junping Du a95d39f9d0 YARN-3860. rmadmin -transitionToActive should check the state of non-target node. (Contributed by Masatake Iwasaki) 2015-06-28 22:26:47 -07:00
Vinod Kumar Vavilapalli aad6a7d5db Adding release 2.7.2 to CHANGES.txt. 2015-06-28 16:29:12 -07:00
Devaraj K b543d1a390 YARN-3859. LeafQueue doesn't print user properly for application add. 
Contributed by Varun Saxena.
 2015-06-28 10:04:50 +05:30
Xuan fe6c1bd73a YARN-2871. TestRMRestart#testRMRestartGetApplicationList sometime fails 
in trunk. Contributed by zhihai xu
 2015-06-26 19:43:59 -07:00
Jason Lowe 40b256949a YARN-3850. NM fails to read files from full disks which can lead to container logs being lost and other issues. Contributed by Varun Saxena 2015-06-26 15:47:07 +00:00
Devaraj K b381f88c71 YARN-3745. SerializedException should also try to instantiate internal 
exception with the default constructor. Contributed by Lavkesh Lahngir.
 2015-06-25 17:38:11 +05:30
Devaraj K 57f1a01eda YARN-3826. Race condition in ResourceTrackerService leads to wrong 
diagnostics messages. Contributed by Chengbing Liu.
 2015-06-25 16:13:59 +05:30
Jonathan Eagles 4c659ddbf7 YARN-3360. Add JMX metrics to TimelineDataManager (Jason Lowe via jeagles) 2015-06-24 16:16:52 -05:00
rohithsharmaks 2df00d53d1 Move YARN-3790 from 2.7.1 to 2.8 in CHANGES.txt 2015-06-24 23:31:03 +05:30
rohithsharmaks dd4b387d96 YARN-3790. usedResource from rootQueue metrics may get stale data for FS scheduler after recovering the container (Zhihai Xu via rohithsharmaks) 2015-06-24 23:00:14 +05:30
Jason Lowe 8d58512d6e YARN-3832. Resource Localization fails on a cluster due to existing cache directories. Contributed by Brahma Reddy Battula 2015-06-24 16:37:39 +00:00
Jason Lowe 2a20dd9b61 YARN-3809. Failed to launch new attempts because ApplicationMasterLauncher's threads all hang. Contributed by Jun Gong 2015-06-24 16:23:48 +00:00
Robert Kanter 99271b7621 YARN-3835. hadoop-yarn-server-resourcemanager test package bundles core-site.xml, yarn-site.xml (vamsee via rkanter) 2015-06-22 18:02:27 -07:00
Karthik Kambatla 5ebf2817e5 YARN-3842. NMProxy should retry on NMNotYetReadyException. (Robert Kanter via kasha) 2015-06-22 17:47:47 -07:00
Xuan 6c7a9d502a YARN-3834. Scrub debug logging of tokens during resource localization. Contributed by Chris Nauroth 2015-06-21 17:13:44 -07:00
Xuan 5b5bb8dcdc YARN-3802. Two RMNodes for the same NodeId are used in RM sometimes 
after NM is reconnected. Contributed by zhihai xu
 2015-06-18 14:37:49 -07:00
Devaraj K 2ad668748e YARN-3824. Fix two minor nits in member variable properties of 
YarnConfiguration. Contributed by Ray Chiang.
 2015-06-18 16:44:08 +05:30
Xuan a826d432f9 YARN-3804. Both RM are on standBy state when kerberos user not in yarn.admin.acl. Contributed by Varun Saxena 2015-06-17 16:23:27 -07:00
Devaraj K ebb9a82519 YARN-3148. Allow CORS related headers to passthrough in 
WebAppProxyServlet. Contributed by Varun Saxena.
 2015-06-17 15:56:18 +05:30
Devaraj K 318d2cde7c YARN-3617. Fix WindowsResourceCalculatorPlugin.getCpuFrequency() returning 
always -1. Contributed by J.Andreina.
 2015-06-17 13:54:09 +05:30
Xuan e27d5a13b0 YARN-3714. AM proxy filter can not get RM webapp address from 
yarn.resourcemanager.hostname.rm-id. Contributed by Masatake Iwasaki
 2015-06-16 14:06:16 -07:00
Devaraj K b039e69bb0 YARN-3789. Improve logs for LeafQueue#activateApplications(). Contributed 
by Bibin A Chundatt.
 2015-06-16 14:03:22 +05:30
Tsuyoshi Ozawa e8c514373f YARN-3711. Documentation of ResourceManager HA should explain configurations about listen addresses. Contributed by Masatake Iwasaki. 2015-06-16 10:12:38 +09:00
Devaraj K d8dcfa98e3 YARN-3794. TestRMEmbeddedElector fails because of ambiguous LOG reference. 
Contributed by Chengbing Liu.
 2015-06-12 13:42:49 +05:30
Xuan 5583f88bf7 YARN-3785. Support for Resource as an argument during submitApp call in 
MockRM test class. Contributed by Sunil G
 2015-06-10 21:40:48 -07:00
Zhijie Shen 8d0ef31632 YARN-3787. Allowed generic history service to load a number of applications whose started time is within the given range. Contributed by Xuan Gong. 2015-06-08 23:14:58 -07:00
cnauroth a531b058ae YARN-3786. Document yarn class path options. Contributed by Brahma Reddy Battula. 2015-06-08 16:18:36 -07:00
Xuan 2b2465dfac YARN-3778. Fix Yarn resourcemanger CLI usage. Contributed by Brahma Reddy Battula 2015-06-08 15:43:03 -07:00
Jian He 960b8f19ca YARN-2716. Refactor ZKRMStateStore retry code with Apache Curator. Contributed by Karthik Kambatla 2015-06-08 14:50:58 -07:00
Devaraj K 126321eded YARN-3747. TestLocalDirsHandlerService should delete the created test 
directory logDir2. Contributed by David Moore.
 2015-06-08 15:32:13 +05:30
Devaraj K c7ee6c151c YARN-3780. Should use equals when compare Resource in 
RMNodeImpl#ReconnectNodeTransition. Contributed by zhihai xu.
 2015-06-08 11:54:55 +05:30
Karthik Kambatla bd69ea408f YARN-3655. FairScheduler: potential livelock due to maxAMShare limitation and container reservation. (Zhihai Xu via kasha) 2015-06-07 11:37:52 -07:00
Xuan 3e000a919f YARN-1462. AHS API and other AHS changes to handle tags for completed MR jobs. Contributed by Xuan Gong 2015-06-05 12:48:52 -07:00
Karthik Kambatla 75885852cc YARN-3259. FairScheduler: Trigger fairShare updates on node events. (Anubhav Dhoot via kasha) 2015-06-05 09:39:41 -07:00
Zhijie Shen 18dd01d6bf YARN-3766. Fixed the apps table column error of generic history web UI. Contributed by Xuan Gong. 2015-06-04 14:47:08 -07:00
Jian He 1970ca7cbc YARN-2392. Add more diags about app retry limits on AM failures. Contributed by Steve Loughran 2015-06-04 11:14:09 -07:00
Jian He 6ad4e59cfc YARN-3764. CapacityScheduler should forbid moving LeafQueue from one parent to another. Contributed by Wangda Tan 2015-06-04 10:52:59 -07:00
Wangda Tan ebd797c48f YARN-3733. Fix DominantRC#compare() does not work as expected if cluster resource is empty. (Rohith Sharmaks via wangda) 2015-06-04 10:22:57 -07:00
Junping Du d7e7f6aa03 YARN-41. The RM should handle the graceful shutdown of the NM. Contributed by Devaraj K. 2015-06-04 04:59:27 -07:00
Xuan 5766a04428 YARN-3749. We should make a copy of configuration when init 
MiniYARNCluster with multiple RMs. Contributed by Chun Chen
 2015-06-03 17:20:15 -07:00
Zhijie Shen bc85959edd Revert "YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong." 
This reverts commit 4a9ec1a824.
 2015-06-03 14:15:56 -07:00
Zhijie Shen 4eec2fd132 Revert "YARN-1462. Correct fix version from branch-2.7.1 to branch-2.8 in" 
This reverts commit 0b5cfacde6.
 2015-06-03 14:15:56 -07:00
Karthik Kambatla edb9cd0f7a YARN-3762. FairScheduler: CME on FSParentQueue#getQueueUserAclInfo. (kasha) 2015-06-03 13:47:24 -07:00
Jason Lowe e13b671aa5 YARN-3585. NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled. Contributed by Rohith Sharmaks 2015-06-03 19:44:07 +00:00
Zhijie Shen dbc4f64937 YARN-3751. Fixed AppInfo to check if used resources are null. Contributed by Sunil G. 2015-06-03 11:51:41 -07:00
Xuan 0b5cfacde6 YARN-1462. Correct fix version from branch-2.7.1 to branch-2.8 in 
CHANGES.txt
 2015-06-01 11:39:00 -07:00
Wangda Tan 5cc3fced95 YARN-3725. App submission via REST API is broken in secure mode due to Timeline DT service address is empty. (Zhijie Shen via wangda) 2015-05-31 16:31:09 -07:00
Xuan 9686261ecb YARN-2900. Application (Attempt and Container) Not Found in AHS results 
in Internal Server Error (500). Contributed by Zhijie Shen and Mit Desai
 2015-05-31 15:37:05 -07:00
Xuan da05361c3b Revert "YARN-2900. Application (Attempt and Container) Not Found in AHS results" 
This reverts commit 06f8e9caba.
 2015-05-31 00:26:04 -07:00
Xuan 06f8e9caba YARN-2900. Application (Attempt and Container) Not Found in AHS results 
in Internal Server Error (500). Contributed by Zhijie Shen and Mit Desai
 2015-05-30 21:56:19 -07:00
Zhijie Shen 4a9ec1a824 YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong. 2015-05-30 21:05:36 -07:00
Karthik Kambatla a8acdd65b3 YARN-3467. Expose allocatedMB, allocatedVCores, and runningContainers metrics on running Applications in RM Web UI. (Anubhav Dhoot via kasha) 2015-05-30 13:36:24 -07:00
Zhijie Shen eb6bf91eea YARN-3740. Fixed the typo in the configuration name: APPLICATION_HISTORY_PREFIX_MAX_APPS. Contributed by Xuan Gong. 2015-05-29 18:17:48 -07:00
Robert Kanter 6aec13cb33 YARN-3713. Remove duplicate function call storeContainerDiagnostics in ContainerDiagnosticsUpdateTransition (zxu via rkanter) 2015-05-29 15:34:37 -07:00
Karthik Kambatla 3ae2a62501 YARN-3547. FairScheduler: Apps that have no resource demand should not participate scheduling. (Xianyin Xin via kasha) 2015-05-29 15:17:22 -07:00
Wangda Tan 788bfa0359 YARN-3716. Node-label-expression should be included by ResourceRequestPBImpl.toString. (Xianyin Xin via wangda) 2015-05-28 16:34:18 -07:00
Xuan 3077c299da YARN-3723. Need to clearly document primaryFilter and otherInfo value 
type. Contributed by Zhijie Shen
 2015-05-28 10:19:36 -07:00
Devaraj K 7e509f5843 YARN-3722. Merge multiple TestWebAppUtils into 
o.a.h.yarn.webapp.util.TestWebAppUtils. Contributed by Masatake Iwasaki.
 2015-05-28 15:35:06 +05:30
Zhijie Shen 54504133f4 YARN-3700. Made generic history service load a number of latest applications according to the parameter or the configuration. Contributed by Xuan Gong. 2015-05-27 16:52:22 -07:00
Allen Wittenauer d6e3164d4a YARN-2355. MAX_APP_ATTEMPTS_ENV may no longer be a useful env var for a container (Darrell Taylor via aw) 2015-05-27 16:40:56 -07:00
Wangda Tan cab7674e54 YARN-3581. Deprecate -directlyAccessNodeLabelStore in RMAdminCLI. (Naganarasimha G R via wangda) 2015-05-27 15:34:38 -07:00
Wangda Tan ec0a852a37 YARN-3647. RMWebServices api's should use updated api from CommonNodeLabelsManager to get NodeLabel object. (Sunil G via wangda) 2015-05-27 12:51:20 -07:00
Wangda Tan cdbd66be11 YARN-3686. CapacityScheduler should trim default_node_label_expression. (Sunil G via wangda) 2015-05-26 15:58:47 -07:00
Jian He 10732d515f YARN-3632. Ordering policy should be allowed to reorder an application when demand changes. Contributed by Craig Welch 2015-05-26 12:00:51 -07:00
Vinod Kumar Vavilapalli 500a1d9c76 YARN-160. Enhanced NodeManager to automatically obtain cpu/memory values from underlying OS when configured to do so. Contributed by Varun Vasudev. 2015-05-26 11:38:35 -07:00
Tsuyoshi Ozawa 9a3d617b63 YARN-2336. Fair scheduler's REST API returns a missing '[' bracket JSON for deep queue tree. Contributed by Kenji Kikushima and Akira Ajisaka. 2015-05-26 19:07:40 +09:00
Xuan 39077dba2e YARN-2238. Filtering on UI sticks even if I move away from the page. 
Contributed by Jian He
 2015-05-25 22:39:16 -07:00
Jian He 446d51591e YARN-3707. RM Web UI queue filter doesn't work. Contributed by Wangda Tan 2015-05-22 15:58:22 -07:00
Xuan 455b3acf0e YARN-3701. Isolating the error of generating a single app report when 
getting all apps from generic history service. Contributed by Zhijie
Shen
 2015-05-22 14:34:39 -07:00
Junping Du 132d909d4a YARN-3594. WintuilsProcessStubExecutor.startStreamReader leaks streams. Contributed by Lars Francke. 2015-05-22 04:23:25 -07:00
Vinod Kumar Vavilapalli 53fafcf061 YARN-3684. Changed ContainerExecutor's primary lifecycle methods to use a more extensible mechanism of context objects. Contributed by Sidharta Seethana. 2015-05-21 15:50:23 -07:00
Karthik Kambatla 4513761869 YARN-3675. FairScheduler: RM quits when node removal races with continuous-scheduling on the same node. (Anubhav Dhoot via kasha) 2015-05-21 13:44:42 -07:00
Devaraj K 0305316d69 YARN-3646. Applications are getting stuck some times in case of retry 
policy forever. Contributed by Raju Bairishetti.
 2015-05-21 20:14:44 +05:30
Akira Ajisaka a5def58087 YARN-3694. Fix dead link for TimelineServer REST API. Contributed by Jagadesh Kiran N. 2015-05-21 23:14:44 +09:00
Jian He 6329bd00fa YARN-3654. ContainerLogsPage web UI should not have meta-refresh. Contributed by Xuan Gong 2015-05-20 17:20:21 -07:00
Jian He 8966d42179 YARN-3609. Load node labels from storage inside RM serviceStart. Contributed by Wangda Tan 2015-05-20 16:30:07 -07:00
Xuan 5774f6b1e5 YARN-3681. yarn cmd says "could not find main class 'queue'" in windows. 
Contributed by Craig Welch and Varun Saxena
 2015-05-20 14:21:12 -07:00
Wangda Tan 03f897fd1a Move YARN-2918 from 2.8.0 to 2.7.1 2015-05-20 13:49:10 -07:00
Tsuyoshi Ozawa 7401e5b5e8 YARN-3677. Fix findbugs warnings in yarn-server-resourcemanager. Contributed by Vinod Kumar Vavilapalli. 2015-05-20 08:59:26 +09:00
Wangda Tan 563eb1ad2a YARN-3583. Support of NodeLabel object instead of plain String in YarnClient side. (Sunil G via wangda) 2015-05-19 16:54:38 -07:00
Wangda Tan b37da52a1c YARN-3565. NodeHeartbeatRequest/RegisterNodeManagerRequest should use NodeLabel object instead of String. (Naganarasimha G R via wangda) 2015-05-19 16:34:17 -07:00
Jian He 7438966586 YARN-2821. Fixed a problem that DistributedShell AM may hang if restarted. Contributed by Varun Vasudev 2015-05-19 14:20:31 -07:00
Ravi Prakash c97f32e7b9 YARN-3302. TestDockerContainerExecutor should run automatically if it can detect docker in the usual place (Ravindra Kumar Naik via raviprak) 2015-05-19 10:28:11 -07:00
Xuan 5009ad4a7f YARN-3601. Fix UT TestRMFailover.testRMWebAppRedirect. Contributed by Weiwei Yang 2015-05-19 09:56:01 -07:00
Xuan 76afd28862 YARN-3541. Add version info on timeline service / generic history web UI and REST API. Contributed by Zhijie Shen 2015-05-18 13:17:16 -07:00
Xuan b0ad644083 YARN-3526. ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster. Contributed by Weiwei Yang 2015-05-15 22:39:49 -07:00
Jason Lowe f7e051c431 YARN-2421. RM still allocates containers to an app in the FINISHING state. Contributed by Chang Li 2015-05-15 22:09:30 +00:00
Ravi Prakash 53fe4eff09 YARN-1519. Check in container-executor if sysconf is implemented before using it (Radim Kolar and Eric Payne via raviprak) 2015-05-14 15:55:37 -07:00
Junping Du 15ccd967ee YARN-3505. Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. 2015-05-14 10:58:12 -07:00
Wangda Tan 0e85044e26 YARN-3362. Add node label usage in RM CapacityScheduler web UI. (Naganarasimha G R via wangda) 2015-05-13 17:00:36 -07:00
Jason Lowe 711d77cc54 YARN-3641. NodeManager: stopRecoveryStore() shouldn't be skipped when exceptions happen in stopping NM's sub-services. Contributed by Junping Du 2015-05-13 21:06:47 +00:00
Jason Lowe f7de6198da Update fix version for YARN-3457 and YARN-3537. 2015-05-13 20:53:16 +00:00
Wangda Tan 7f19e7a254 YARN-3521. Support return structured NodeLabel objects in REST API (Sunil G via wangda) 2015-05-13 13:43:17 -07:00
Wangda Tan d4f53fc963 YARN-3579. CommonNodeLabelsManager should support NodeLabel instead of string label name when getting node-to-label/label-to-label mappings. (Sunil G via wangda) 2015-05-13 13:29:09 -07:00
Xuan 0f95921447 YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch 2015-05-13 13:10:53 -07:00
Wangda Tan 341a476812 YARN-2921. Fix MockRM/MockAM#waitForState sleep too long. (Tsuyoshi Ozawa via wangda) 2015-05-13 13:06:07 -07:00
Zhijie Shen fcd0702c10 YARN-3539. Updated timeline server documentation and marked REST APIs evolving. Contributed by Steve Loughran. 2015-05-12 21:13:22 -07:00
Karthik Kambatla fe0df59627 YARN-3613. TestContainerManagerSecurity should init and start Yarn cluster in setup instead of individual methods. (nijel via kasha) 2015-05-12 10:45:33 -07:00
Devaraj K 5c2f05cd9b YARN-3629. NodeID is always printed as "null" in node manager 
initialization log. Contributed by nijel.
 2015-05-12 22:20:25 +05:30
Devaraj K 8badd82ce2 YARN-3513. Remove unused variables in ContainersMonitorImpl and add debug 
log for overall resource usage by all containers.  Contributed by
Naganarasimha G R.
 2015-05-12 16:54:38 +05:30
Wangda Tan 3d28611cc6 Move YARN-3493 in CHANGES.txt from 2.8 to 2.7.1 2015-05-11 18:06:54 -07:00
Wangda Tan d6f6741296 YARN-3489. RMServerUtils.validateResourceRequests should only obtain queue info once. (Varun Saxena via wangda) 2015-05-11 17:31:15 -07:00
Wangda Tan 1952f88889 Moved YARN-3434. (Interaction between reservations and userlimit can result in significant ULF violation.) From 2.8.0 to 2.7.1 2015-05-11 15:21:35 -07:00
Junping Du 7e543c27fa YARN-3587. Fix the javadoc of DelegationTokenSecretManager in yarn, etc. projects. Contributed by Gabor Liptak. 2015-05-11 05:27:07 -07:00
Karthik Kambatla a60f78e98e YARN-3395. FairScheduler: Trim whitespaces when using username for queuename. (Zhihai Xu via kasha) 2015-05-09 15:41:20 -07:00
Karthik Kambatla 70fb37cd79 YARN-1287. Consolidate MockClocks. (Sebastian Wong and Anubhav Dhoot via kasha) 2015-05-09 14:34:54 -07:00
Xuan 6471d18bc7 YARN-1912. ResourceLocalizer started without any jvm memory control. 
Contributed by Masatake Iwasaki
 2015-05-08 20:01:21 -07:00
Xuan 333f9a896d YARN-3602. TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IOException from cleanup. Contributed by zhihai xu 2015-05-08 17:52:54 -07:00
Zhijie Shen 08f0ae403c YARN-2206. Updated document for applications REST API response examples. Contributed by Kenji Kikushima and Brahma Reddy Battula. 2015-05-08 16:40:20 -07:00
Karthik Kambatla 2fb44c8aaf YARN-3271. FairScheduler: Move tests related to max-runnable-apps from TestFairScheduler to TestAppRunnability. (nijel via kasha) 2015-05-08 16:39:10 -07:00
Karthik Kambatla 96473bdc2b YARN-1050. Document the Fair Scheduler REST API. (Kenji Kikushima and Roman Shaposhnik via kasha) 2015-05-08 16:31:14 -07:00
Robert Kanter 5658998845 YARN-3473. Fix RM Web UI configuration for some properties (rchiang via rkanter) 2015-05-08 16:23:10 -07:00
Jason Lowe 25e2b02122 YARN-3476. Nodemanager can fail to delete local logs if log aggregation fails. Contributed by Rohith 2015-05-08 22:45:52 +00:00
Jian He d0b73287c2 YARN-3604. Fixed ZKRMStateStore#removeApplication to also disable watch. Contributed Zhihai Xu 2015-05-08 15:12:33 -07:00
Xuan 088156de43 YARN-2331. Distinguish shutdown during supervision vs. shutdown for 
rolling upgrade. Contributed by Jason Lowe
 2015-05-08 15:10:43 -07:00
Vinod Kumar Vavilapalli f7a74d2491 YARN-3018. Unified the default value for the configuration property yarn.scheduler.capacity.node-locality-delay in code and default xml file. Contributed by Nijel SF. 2015-05-08 14:45:06 -07:00
Wangda Tan f55d21d894 YARN-3593. Add label-type and Improve "DEFAULT_PARTITION" in Node Labels Page. (Naganarasimha G R via wangda) 2015-05-08 12:04:50 -07:00
Thomas Graves f0f5e3c075 YARN-20. More information for yarn.resourcemanager.webapp.address in yarn-default.xml (Bartosz Ługowski vai tgraves) 2015-05-08 17:21:32 +00:00
Thomas Graves 5d708a4725 YARN-3600. AM container link is broken (Naganarasimha G R via tgraves 2015-05-08 16:35:40 +00:00
Robert (Bobby) Evans bcf2890502 YARN-644: Basic null check is not performed on passed in arguments before using them in ContainerManagerImpl.startContainer 2015-05-08 11:11:01 -05:00
Devaraj K 241a72af0d YARN-2784. Make POM project names consistent. Contributed by Rohith. 2015-05-08 19:23:53 +05:30
Jason Lowe 9757864fd6 YARN-3554. Default value for maximum nodemanager connect wait time is too high. Contributed by Naganarasimha G R 2015-05-08 13:45:35 +00:00
Tsuyoshi Ozawa b419c1b2ec YARN-3169. Drop YARN's overview document. Contributed by Brahma Reddy Battula. 2015-05-08 19:58:33 +09:00
Tsuyoshi Ozawa f26700f287 YARN-3589. RM and AH web UI display DOCTYPE wrongly. Contbituted by Rohith. 2015-05-08 17:13:40 +09:00
Devaraj K ef3d66d462 YARN-3358. Audit log not present while refreshing Service ACLs. 
Contributed by Varun Saxena.
 2015-05-08 12:14:10 +05:30
Devaraj K a28cd02ba0 YARN-3592. Fix typos in RMNodeLabelsManager. Contributed by Sunil G. 2015-05-08 10:54:36 +05:30
Akira Ajisaka a521b50955 YARN-3572. Correct typos in WritingYarnApplications.md. Contributed by Gabor Liptak. 2015-05-08 13:24:29 +09:00
Akira Ajisaka b167fe7605 YARN-1832. Fix wrong MockLocalizerStatus#equals implementation. Contributed by Hong Zhiguo. 2015-05-08 11:14:45 +09:00
Jian He f489a4ec96 YARN-2918. RM should not fail on startup if queue's configured labels do not exist in cluster-node-labels. Contributed by Wangda Tan 2015-05-07 17:35:41 -07:00
Jian He b88700dcd0 YARN-3584. Fixed attempt diagnostics format shown on the UI. Contributed by nijel 2015-05-07 13:54:07 -07:00
Zhijie Shen daf3e4ef8b YARN-3448. Added a rolling time-to-live LevelDB timeline store implementation. Contributed by Jonathan Eagles. 2015-05-07 10:01:51 -07:00
Junping Du 8e991f4b1d YARN-3523. Cleanup ResourceManagerAdministrationProtocol interface audience. Contributed by Naganarasimha G R 2015-05-07 05:39:04 -07:00
Akira Ajisaka 918af8efff YARN-3577. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 2015-05-07 13:33:03 +09:00
Vinod Kumar Vavilapalli 4c7b9b6abe YARN-3385. Fixed a race-condition in ResourceManager's ZooKeeper based state-store to avoid crashing on duplicate deletes. Contributed by Zhihai Xu. 2015-05-06 17:51:17 -07:00
Junping Du 31b627b2a8 YARN-3580. [JDK8] TestClientRMService.testGetLabelsToNodes fails. Contributed by Robert Kanter. 2015-05-06 16:51:05 -07:00
Robert Kanter b72507810a YARN-3491. PublicLocalizer#addResource is too slow. (zxu via rkanter) 2015-05-06 14:19:06 -07:00
Jian He 0d3188fd25 YARN-3301. Fixed the format issue of the new RM attempt web page. Contributed by Xuan Gong 2015-05-06 13:45:00 -07:00
Vinod Kumar Vavilapalli 185e63a726 YARN-3243. Moving CHANGES.txt entry to the right release. 2015-05-06 10:42:50 -07:00
Xuan a583a40693 YARN-3582. NPE in WebAppProxyServlet. Contributed by Jian He 2015-05-05 21:42:19 -07:00
Jian He e4c3b52c89 YARN-3343. Increased TestCapacitySchedulerNodeLabelUpdate#testNodeUpdate timeout. Contributed by Rohith Sharmaks 2015-05-05 11:33:47 -07:00
Xuan b7dd3a4f04 YARN-2123. Progress bars in Web UI always at 100% due to non-US locale. 
Contributed by Akira AJISAKA
 2015-05-05 10:40:16 -07:00
Junping Du 3810242062 YARN-3396. Handle URISyntaxException in ResourceLocalizationService. (Contributed by Brahma Reddy Battula) 2015-05-05 10:18:23 -07:00
Jason Lowe 9356cf8676 YARN-3552. RM Web UI shows -1 running containers for completed apps. Contributed by Rohith 2015-05-05 13:42:32 +00:00
Jian He d701acc9c6 YARN-2725. Added test cases of retrying creating znode in ZKRMStateStore. Contributed by Tsuyoshi Ozawa 2015-05-04 16:13:29 -07:00
Wangda Tan 71f4de220c YARN-3375. NodeHealthScriptRunner.shouldRun() check is performing 3 times for starting NodeHealthScriptRunner (Devaraj K via wangda) 2015-05-04 15:49:19 -07:00
Jason Lowe 8f65c793f2 YARN-3097. Logging of resource recovery on NM restart has redundancies. Contributed by Eric Payne 2015-05-04 15:31:15 +00:00
Tsuyoshi Ozawa e8d0ee5fc9 YARN-1993. Cross-site scripting vulnerability in TextView.java. Contributed byKenji Kikushima. 2015-05-03 10:51:17 +09:00
Junping Du 57d9a972cb YARN-2454. Fix compareTo of variable UNBOUNDED in o.a.h.y.util.resource.Resources. Contributed by Xu Yang. 2015-05-02 10:18:44 -07:00
Gera Shegalov f8204e241d YARN-2893. AMLaucher: sporadic job failures due to EOFException in readTokenStorageStream. (Zhihai Xu via gera) 2015-05-01 18:18:55 -07:00
Wangda Tan 7d46a806e7 YARN-3006. Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda) 2015-05-01 16:40:55 -07:00
Robert Kanter ac7d152901 YARN-3363. add localization and container launch time to ContainerMetrics at NM to show these timing information for each active container. (zxu via rkanter) 2015-05-01 16:39:21 -07:00
Vinod Kumar Vavilapalli 1b3b9e5c31 YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. 2015-04-30 21:41:07 -07:00
Zhijie Shen 7e8639fda4 YARN-3544. Got back AM logs link on the RM web UI for a completed app. Contributed by Xuan Gong. 2015-04-30 13:05:55 -07:00
Wangda Tan e2e8f77118 YARN-3564. Fix TestContainerAllocation.testAMContainerAllocationWhenDNSUnavailable fails randomly. (Jian He via wangda) 2015-04-30 11:03:19 -07:00
Jian He 4c1af156ae YARN-3533. Test: Fix launchAM in MockRM to wait for attempt to be scheduled. Contributed by Anubhav Dhoot 2015-04-29 14:50:01 -07:00
tgraves 2e215484bd YARN-3517. RM web ui for dumping scheduler logs should be for admins only (Varun Vasudev via tgraves) 2015-04-29 21:25:42 +00:00
Karthik Kambatla 8f82970e0c YARN-3485. FairScheduler headroom calculation doesn't consider maxResources for Fifo and FairShare policies. (kasha) 2015-04-28 21:00:35 -07:00
Wangda Tan db1b674b50 YARN-2740. Fix NodeLabelsManager to properly handle node label modifications when distributed node label configuration enabled. (Naganarasimha G R via wangda) 2015-04-27 16:24:38 -07:00
Karthik Kambatla 32cd2c8d42 Update CHANGES.txt - Pulled in YARN-3465, YARN-3516, and YARN-3464 to branch-2.7 (for 2.7.1) 2015-04-27 13:43:43 -07:00
Xuan 7f07c4d810 YARN-3530. ATS throws exception on trying to filter results without 
otherinfo. Contributed by zhijie shen
 2015-04-27 10:36:42 -07:00
Karthik Kambatla 47279c3228 YARN-3464. Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) 2015-04-26 09:13:46 -07:00
Vinod Kumar Vavilapalli 2f82ae042a Moving YARN-3351, YARN-3382, YARN-3472, MAPREDUCE-6238 to the 2.7.1 CHANGES.txt 
sections given the recent merge into branch-2.7.
 2015-04-24 17:18:46 -07:00
Jian He d497f6ea2b YARN-2498. Respect labels in preemption policy of capacity scheduler for inter-queue preemption. Contributed by Wangda Tan 2015-04-24 17:03:13 -07:00
Tsuyoshi Ozawa 4a3dabd94f YARN-3406. Display count of running containers in the RM's Web UI. Contributed by Ryu Kobayashi. 2015-04-25 07:17:11 +09:00
Jason Lowe 78fe6e57c7 Fix commit version for YARN-3537 2015-04-24 22:07:53 +00:00
Jason Lowe 5e093f0d40 YARN-3537. NPE when NodeManager.serviceInit fails and stopRecoveryStore invoked. Contributed by Brahma Reddy Battula 2015-04-24 22:02:53 +00:00
Akira Ajisaka 5ce3a77f3c YARN-3444. Fix typo capabililty. Contributed by Gabor Liptak. 2015-04-25 06:08:16 +09:00
Jian He d03dcb9635 YARN-3387. Previous AM's container completed status couldn't pass to current AM if AM and RM restarted during the same time. Contributed by Sandflee 2015-04-24 12:13:29 -07:00
Xuan eee9facbba YARN-3511. Add errors and warnings page to ATS. Contributed by Varun Vasudev 2015-04-24 09:41:59 -07:00
Xuan 0b3f8957a8 YARN-3516. killing ContainerLocalizer action doesn't take effect when 
private localizer receives FETCH_FAILURE status. Contributed by zhihai
xu
 2015-04-23 16:40:40 -07:00
Jian He aa4a192feb YARN-3522. Fixed DistributedShell to instantiate TimeLineClient as the correct user. Contributed by Zhijie Shen 2015-04-23 11:07:26 -07:00
Wangda Tan 395205444e YARN-3319. Implement a FairOrderingPolicy. (Craig Welch via wangda) 2015-04-23 10:47:15 -07:00
tgraves 189a63a719 YARN-3434. Interaction between reservations and userlimit can result in significant ULF violation 2015-04-23 14:39:25 +00:00
Steve Loughran d9bcf99aef YARN-2605. [RM HA] Rest api endpoints doing redirect incorrectly. (Xuan Gong via stevel) 2015-04-23 09:22:48 +01:00
Vinod Kumar Vavilapalli a100be685c YARN-3366. Enhanced NodeManager to support classifying/shaping outgoing network bandwidth traffic originating from YARN containers Contributed by Sidharta Seethana. 2015-04-22 17:26:13 -07:00
Junping Du fad9d7e85b New parameter of CLI for decommissioning node gracefully in RMAdmin CLI. Contributed by Devaraj K 2015-04-22 10:07:20 -07:00
Jian He 674c7ef649 YARN-3503. Expose disk utilization percentage and bad local and log dir counts in NM metrics. Contributed by Varun Vasudev 2015-04-21 20:57:02 -07:00
Jian He bdd90110e6 YARN-3494. Expose AM resource limit and usage in CS QueueMetrics. Contributed by Rohith Sharmaks 2015-04-21 20:06:20 -07:00
Wangda Tan e71d0d87d9 YARN-3410. YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) 2015-04-21 17:51:22 -07:00
Tsuyoshi Ozawa 105afd5477 YARN-3495. Confusing log generated by FairScheduler. Contributed by Brahma Reddy Battula. 2015-04-22 05:47:59 +09:00
Vinod Kumar Vavilapalli d52de61544 Set the release date for 2.7.0 2015-04-20 20:16:58 -07:00
Wangda Tan 44872b76fc YARN-3463. Integrate OrderingPolicy Framework with CapacityScheduler. (Craig Welch via wangda) 2015-04-20 17:12:32 -07:00
Jian He f967fd2f21 YARN-3497. ContainerManagementProtocolProxy modifies IPC timeout conf without making a copy. Contributed by Jason Lowe 2015-04-20 10:38:27 -07:00
Jian He 497c86b485 YARN-3136. Fixed a synchronization problem of AbstractYarnScheduler#getTransferredContainers. Contributed by Sunil G 2015-04-18 12:46:37 -07:00
Wangda Tan f47a5763ac YARN-3487. CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) 2015-04-17 17:22:27 -07:00
Wangda Tan f65eeb412d YARN-3493. RM fails to come up with error "Failed to load/recover state" when mem settings are changed. (Jian He via wangda) 2015-04-17 17:11:22 -07:00
Jian He 6779467ab6 YARN-3451. Display attempt start time and elapsed time on the web UI. Contributed by Rohith Sharmaks 2015-04-17 15:55:34 -07:00
Jian He d573f09fb9 YARN-2696. Queue sorting in CapacityScheduler should consider node label. Contributed by Wangda Tan 2015-04-17 13:36:59 -07:00
Junping Du 1db355a875 YARN-1402. Update related Web UI and CLI with exposing client API to check log aggregation status. Contributed by Xuan Gong. 2015-04-17 13:18:59 -07:00
Jian He bb6dde68f1 YARN-3021. YARN's delegation-token handling disallows certain trust setups to operate properly over DistCp. Contributed by Yongjun Zhang 2015-04-16 19:43:37 -07:00
Jian He 1b89a3e173 YARN-3354. Add node label expression in ContainerTokenIdentifier to support RM recovery. Contributed by Wangda Tan 2015-04-15 13:57:06 -07:00
Jian He b2e6cf607f YARN-3404. Display queue name on application page. Contributed by Ryu Kobayashi 2015-04-15 13:52:50 -07:00
Tsuyoshi Ozawa e48cedc663 YARN-3326. Support RESTful API for getLabelsToNodes. Contributed by Naganarasimha G R. 2015-04-15 14:03:55 -05:00
Wangda Tan 5004e75332 YARN-3318. Create Initial OrderingPolicy Framework and FifoOrderingPolicy. (Craig Welch via wangda) 2015-04-15 09:56:32 -07:00
Tsuyoshi Ozawa 05007b45e5 YARN-3436. Fix URIs in documantion of YARN web service REST APIs. Contributed by Bibin A Chundatt. 2015-04-14 14:13:55 -05:00
Jian He 0fefda645b YARN-3361. CapacityScheduler side changes to support non-exclusive node labels. Contributed by Wangda Tan 2015-04-14 11:45:58 -07:00
Jian He b46ee1e7a3 YARN-3266. RMContext#inactiveNodes should have NodeId as map key. Contributed by Chengbing Liu 2015-04-14 10:54:22 -07:00
Junping Du 838b06ac87 YARN-3443. Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. 2015-04-13 18:35:56 -07:00
Junping Du a77d628339 YARN-3347. Improve YARN log command to get AMContainer logs as well as running containers logs. Contributed by Xuan Gong. 2015-04-13 17:29:06 -07:00
Jian He 7b4671490a YARN-3394. Enrich WebApplication proxy documentation. Contributed by Naganarasimha G R 2015-04-13 17:05:38 -07:00
Jian He a1afbc48b5 YARN-3472. Fixed possible leak in DelegationTokenRenewer#allTokens. Contributed by Rohith Sharmaks 2015-04-13 14:07:17 -07:00
Karthik Kambatla 5cda6fffd3 Fix up author name to Jun Gong in CHANGES.txt for YARN-3469 2015-04-13 13:52:28 -07:00
Jian He 944a16579f YARN-3382. Some of UserMetricsInfo metrics are incorrectly set to root queue metrics. Contributed by Rohit Agarwal 2015-04-13 10:53:34 -07:00
Karthik Kambatla e516706b89 YARN-3469. ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) 2015-04-10 11:20:34 -07:00
Jian He 577d755e4b YARN-3348. Add a 'yarn top' tool to help understand cluster usage. Contributed by Varun Vasudev 2015-04-10 09:57:17 -07:00
Junping Du 92431c9617 YARN-1376. NM need to notify the log aggregation status to RM through Node heartbeat. Contributed by Xuan Gong. 2015-04-10 08:56:18 -07:00
Xuan afa5d4715a YARN-3293. Track and display capacity scheduler health metrics in web 
UI. Contributed by Varun Vasudev
 2015-04-09 23:38:04 -07:00
Vinod Kumar Vavilapalli 623fd46c1e Adding release 2.7.1 to CHANGES.txt 2015-04-09 13:54:20 -07:00
Vinod Kumar Vavilapalli 9c5911294e YARN-3055. Fixed ResourceManager's DelegationTokenRenewer to not stop token renewal of applications part of a bigger workflow. Contributed by Daryn Sharp. 2015-04-09 13:08:53 -07:00
Wangda Tan 1885141e90 YARN-3466. Fix RM nodes web page to sort by node HTTP-address, #containers and node-label column. (Jason Lowe via wangda) 2015-04-09 10:35:12 -07:00
Karthik Kambatla 6495940eae YARN-3465. Use LinkedHashMap to preserve order of resource requests. (Zhihai Xu via kasha) 2015-04-09 00:07:49 -07:00
Hitesh Shah 265ed1fe80 YARN-2890. MiniYarnCluster should turn on timeline service if configured to do so. Contributed by Mit Desai. 2015-04-08 14:13:10 -07:00
Wangda Tan 7af086a515 YARN-3459. Fix failiure of TestLog4jWarningErrorMetricsAppender. (Varun Vasudev via wangda) 2015-04-08 10:57:48 -07:00
Tsuyoshi Ozawa dd852f5b8c YARN-3457. NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. 2015-04-08 15:56:18 +09:00
Robert Kanter 5b8a3ae366 YARN-3429. Fix incorrect CHANGES.txt 2015-04-07 16:15:42 -07:00
Xuan d27e9241e8 YARN-3294. Allow dumping of Capacity Scheduler debug logs via web UI for 
a fixed time period. Contributed by Varun Vasudev
 2015-04-07 09:52:36 -07:00
Xuan 19a4feaf6f YARN-3110. Few issues in ApplicationHistory web ui. Contributed by Naganarasimha G R 2015-04-07 08:22:39 -07:00
Zhijie Shen 3fb5abfc87 Move YARN-3273 from 2.8 to 2.7. 2015-04-06 14:14:53 -07:00
Robert Kanter 99b08a748e YARN-2429. TestAMRMTokens.testTokenExpiry fails Intermittently with error message:Invalid AMRMToken (zxu via rkanter) 2015-04-06 14:11:20 -07:00
Tsuyoshi Ozawa 53959e69f7 TestFairScheduler.testContinuousScheduling fails Intermittently. Contributed by Zhihai Xu. 2015-04-06 20:19:13 +09:00
Xuan 96d72118f5 YARN-3435. AM container to be allocated Appattempt AM container shown as 
null. Contributed by Bibin A Chundatt
 2015-04-05 00:40:57 -07:00
Wangda Tan bad070fe15 YARN-2901. Add errors and warning metrics page to RM, NM web UI. (Varun Vasudev via wangda) 2015-04-02 17:23:20 -07:00
Vinod Kumar Vavilapalli b21c72777a YARN-3365. Enhanced NodeManager to support using the 'tc' tool via container-executor for outbound network traffic control. Contributed by Sidharta Seethana. 2015-04-02 16:53:59 -07:00
Sandy Ryza 6a6a59db7f YARN-3415. Non-AM containers can be counted towards amResourceUsage of a fairscheduler queue (Zhihai Xu via Sandy Ryza) 2015-04-02 13:56:08 -07:00
Zhijie Shen 8366a36ad3 YARN-3430. Made headroom data available on app attempt page of RM WebUI. Contributed by Xuan Gong. 2015-04-01 13:47:54 -07:00
Wangda Tan 492239424a YARN-3425. NPE from RMNodeLabelsManager.serviceStop when NodeLabelsManager.serviceInit failed. (Bibin A Chundatt via wangda) 2015-04-01 10:14:48 -07:00
Xuan 2e79f1c212 YARN-3248. Correct fix version from branch-2.7 to branch-2.8 in the change log. 2015-04-01 04:32:11 -07:00
Xuan 4728bdfa15 YARN-3248. Display count of nodes blacklisted by apps in the web UI. 
Contributed by Varun Vasudev
 2015-04-01 04:19:18 -07:00
Tsuyoshi Ozawa c69ba81497 YARN-3424. Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. 2015-04-01 17:44:25 +09:00
Karthik Kambatla 2daa478a64 YARN-3428. Debug log resources to be localized for a container. (kasha) 2015-03-31 17:34:47 -07:00
Karthik Kambatla 79f7f2aabf YARN-3412. RM tests should use MockRM where possible. (kasha) 2015-03-31 09:14:15 -07:00
Tsuyoshi Ozawa b5a22e9838 YARN-3258. FairScheduler: Need to add more logging to investigate allocations. Contributed by Anubhav Dhoot. 2015-03-31 17:42:44 +09:00
Wangda Tan 2a945d24f7 YARN-2495. Allow admin specify labels from each NM (Distributed configuration for node label). (Naganarasimha G R via wangda) 2015-03-30 12:05:21 -07:00
Vinod Kumar Vavilapalli c358368f51 YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla. 2015-03-30 10:09:40 -07:00
Ravi Prakash e0ccea33c9 YARN-3288. Document and fix indentation in the DockerContainerExecutor code 2015-03-28 08:00:41 -07:00
Robert Kanter 87130bf6b2 YARN-3400. [JDK 8] Build Failure due to unreported exceptions in RPCUtil (rkanter) 2015-03-26 11:00:20 -07:00
Karthik Kambatla c906a1de72 YARN-3397. yarn rmadmin should skip -failover. (J.Andreina via kasha) 2015-03-25 07:42:27 -07:00
Xuan e556198e71 YARN-2213. Change proxy-user cookie log in AmIpFilter to DEBUG. 
Contributed by Varun Saxena
 2015-03-25 04:49:43 -07:00
Wangda Tan 97a7277a2d YARN-3383. AdminService should use warn instead of info to log exception when operation fails. (Li Lu via wangda) 2015-03-24 10:38:54 -07:00
Harsh J fbceb3b418 YARN-1880. Cleanup TestApplicationClientProtocolOnHA. Contributed by ozawa. 2015-03-24 11:57:58 +05:30
Xuan 9fae455e26 YARN-3393. Getting application(s) goes wrong when app finishes before 
starting the attempt. Contributed by Zhijie Shen
 2015-03-23 20:33:16 -07:00
Karthik Kambatla 972f1f1ab9 YARN-2868. FairScheduler: Metric for latency to allocate first container for an application. (Ray Chiang via kasha) 2015-03-23 14:07:05 -07:00
Karthik Kambatla 2bc097cd14 YARN-3241. FairScheduler handles invalid queue names inconsistently. (Zhihai Xu via kasha) 2015-03-23 13:22:03 -07:00
cnauroth 6ca1f12024 YARN-3336. FileSystem memory leak in DelegationTokenRenewer. 2015-03-23 10:45:50 -07:00
Tsuyoshi Ozawa 82eda771e0 YARN-3384. TestLogAggregationService.verifyContainerLogs fails after YARN-2777. Contributed by Naganarasimha G R. 2015-03-24 00:25:30 +09:00
Jian He e1feb4ea1a YARN-3345. Add non-exclusive node label API. Contributed by Wangda Tan 2015-03-20 19:04:38 -07:00
Junping Du 7f1e2f9969 YARN-3350. YARN RackResolver spams logs with messages at info level. Contributed by Wilfred Spiegelenburg 2015-03-20 18:21:33 -07:00
Jian He 586348e4cb YARN-3356. Capacity Scheduler FiCaSchedulerApp should use ResourceUsage to track used-resources-by-label. Contributed by Wangda Tan 2015-03-20 13:54:01 -07:00
Junping Du d81109e588 YARN-3269. Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path. Contributed by Xuan Gong 2015-03-20 13:41:22 -07:00
Wangda Tan 6bc7710ec7 YARN-3369. Missing NullPointer check in AppSchedulingInfo causes RM to die. (Brahma Reddy Battula via wangda) 2015-03-20 10:42:05 -07:00
Xuan 1a4b528691 YARN-2777. Mark the end of individual log in aggregated log. Contributed 
by Varun Saxena
 2015-03-20 10:24:13 -07:00
Jian He 4e886eb9cb YARN-3379. Fixed missing data in localityTable and ResourceRequests table in RM WebUI. Contributed by Xuan Gong 2015-03-19 22:27:21 -07:00
Devaraj K 93d0f4acc8 YARN-3357. Move TestFifoScheduler to FIFO package. Contributed by Rohith 
Sharmaks.
 2015-03-19 12:16:52 +05:30
Karthik Kambatla 20b49224eb YARN-3351. AppMaster tracking URL is broken in HA. (Anubhav Dhoot via kasha) 2015-03-18 16:30:33 -07:00
Jian He 658097d6da YARN-3273. Improve scheduler UI to facilitate scheduling analysis and debugging. Contributed Rohith Sharmaks 2015-03-17 21:30:23 -07:00
Tsuyoshi Ozawa 3bc72cc16d YARN-3205. FileSystemRMStateStore should disable FileSystem Cache to avoid get a Filesystem with an old configuration. Contributed by Zhihai Xu. 2015-03-18 11:53:19 +09:00
Jian He 968425e9f7 YARN-3305. Normalize AM resource request on app submission. Contributed by Rohith Sharmaks 2015-03-17 13:49:59 -07:00
Karthik Kambatla 32b4330456 Revert "YARN-3181. FairScheduler: Fix up outdated findbugs issues. (kasha)" 
This reverts commit c2b185def8.
 2015-03-17 12:31:15 -07:00
Jian He 487374b7fe YARN-3243. CapacityScheduler should pass headroom from parent to children to make sure ParentQueue obey its capacity limits. Contributed by Wangda Tan. 2015-03-17 10:24:23 -07:00
Devaraj K 7179f94f9d YARN-3197. Confusing log generated by CapacityScheduler. Contributed by 
Varun Saxena.
 2015-03-17 15:57:57 +05:30
Ravi Prakash 56085203c4 YARN-3339. TestDockerContainerExecutor should pull a single image and not the entire centos repository. (Ravindra Kumar Naik via raviprak) 2015-03-16 16:17:58 -07:00
Tsuyoshi Ozawa 7522a643fa YARN-3349. Treat all exceptions as failure in TestFSRMStateStore#testFSRMStateStoreClientRetry. Contributed by Zhihai Xu. 2015-03-17 08:09:55 +09:00
Tsuyoshi Ozawa 3da9a97cfb YARN-1453. [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments. Contributed by Akira AJISAKA, Andrew Purtell, and Allen Wittenauer. 2015-03-16 23:19:05 +09:00
Xuan 3ff1ba2a7b YARN-3171. Sort by Application id, AppAttempt and ContainerID doesn't 
work in ATS / RM web ui. Contributed by Naganarasimha G R
 2015-03-15 20:26:10 -07:00
Zhijie Shen 6fdef76cc3 YARN-2854. Updated the documentation of the timeline service and the generic history service. Contributed by Naganarasimha G R. 2015-03-13 14:00:09 -07:00
Jonathan Eagles 8180e676ab YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles) 2015-03-13 12:04:30 -05:00
Vinod Kumar Vavilapalli 863079bb87 YARN-3154. Added additional APIs in LogAggregationContext to avoid aggregating running logs of application when rolling is enabled. Contributed by Xuan Gong. 2015-03-12 13:32:29 -07:00
Xuan 06ce1d9a6c YARN-3338. Exclude jline dependency from YARN. Contributed by Zhijie 
Shen
 2015-03-12 10:25:00 -07:00
Zhijie Shen 85f6d67fa7 YARN-1884. Added nodeHttpAddress into ContainerReport and fixed the link to NM web page. Contributed by Xuan Gong. 2015-03-11 19:35:19 -07:00
Tsuyoshi Ozawa 30c428a858 YARN-3295. Fix documentation nits found in markdown conversion. Contributed by Masatake Iwasaki. 2015-03-11 15:02:24 +09:00
Allen Wittenauer a5cf985bf5 YARN-2280. Resource manager web service fields are not accessible (Krisztian Horvath via aw) 2015-03-10 16:32:20 -07:00
Jian He a380643d20 YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user or group. Contributed by Gururaj Shetty 2015-03-10 11:15:57 -07:00
Jian He c3003eba6f YARN-3300. Outstanding_resource_requests table should not be shown in AHS. Contributed by Xuan Gong 2015-03-09 20:46:48 -07:00