Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,418 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2119 Commits

Author SHA1 Message Date
Varun Vasudev 64c2e8011e YARN-4103. RM WebServices missing scheme for appattempts logLinks. Contributed by Jonathan Eagles. 
(cherry picked from commit 1b2e2ca9104dfc05e9aa7f93954ae2e00b3e62c4)
 2015-09-04 14:32:07 +05:30
Varun Vasudev a53054c853 YARN-3970. Add REST api support for Application Priority. Contributed by Naganarasimha G R. 
(cherry picked from commit 223a306da925ad031bc398f2ea5e56f0e6521edf)
 2015-09-03 16:40:42 +05:30
Jian He cd82fa2f83 YARN-4101. RM should print alert messages if Zookeeper and Resourcemanager gets connection issue. Contributed by Xuan Gong 
(cherry picked from commit 09c64ba1ba)
 2015-09-02 17:46:04 -07:00
Rohith Sharma K S 6e41aa64a5 YARN-3893. Both RM in active state when Admin#transitionToActive failure from refeshAll() (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit 7d6687fe76)
 2015-09-02 15:27:33 +05:30
Rohith Sharma K S 7ecdf51441 YARN-4073. Removed unused ApplicationACLsManager in ContainerManagerImpl constructor. (Naganarasimha G R via rohithsharmaks) 
(cherry picked from commit 095ab9ab5f)
 2015-09-02 14:28:50 +05:30
Vinod Kumar Vavilapalli acb3000187 YARN-3222. Added the missing CHANGES.txt entry. 
(cherry picked from commit 4620767156)
 2015-09-01 13:44:10 -07:00
Tsuyoshi Ozawa d3d65f62b3 YARN-2801. Add documentation for node labels feature. Contributed by Wangda Tan and Naganarasimha G R. 
(cherry picked from commit faa38e1aa4)
 2015-09-01 17:55:53 +09:00
Varun Vasudev 855e0f8b00 YARN-4082. Container shouldn't be killed when node's label updated. Contributed by Wangda Tan. 
(cherry picked from commit bf669b6d9f)
 2015-09-01 14:20:54 +05:30
Jian He 21b478e1bf Move YARN-4092 to 2.7.2 
(cherry picked from commit 4eaa7fd3ea)
 2015-08-31 17:44:04 -07:00
Jian He 48f5161cd5 YARN-4092. Fixed UI redirection to print useful messages when both RMs are in standby mode. Contributed by Xuan Gong 
(cherry picked from commit a3fd2ccc86)
 2015-08-31 17:34:49 -07:00
Vinod Kumar Vavilapalli 0191dae2ec YARN-2945. Fixing the CHANGES.txt to have the right JIRA number. 
(cherry picked from commit cf831565e8)
 2015-08-30 20:02:36 -07:00
Junping Du 792b9c0eaf YARN-1556. NPE getting application report with a null appId. Contributed by Weiwei Yang. 
(cherry picked from commit beb65c9465)
 2015-08-28 05:58:45 -07:00
Jian He 306c535395 YARN-3250. Support admin cli interface in for Application Priority. Contributed by Rohith Sharma K S 
(cherry picked from commit a9c8ea71aa)
 2015-08-27 13:27:34 -07:00
Jian He 387076894f YARN-4014. Support user cli interface in for Application Priority. Contributed by Rohith Sharma K S 
(cherry picked from commit 57c7ae1aff)
 2015-08-24 20:37:18 -07:00
Rohith Sharma K S 6dc732f2f7 YARN-3896. RMNode transitioned from RUNNING to REBOOTED because its response id has not been reset synchronously. (Jun Gong via rohithsharmaks) 
(cherry picked from commit feaf034994)
 2015-08-24 11:30:46 +05:30
Xuan 205b98c059 YARN-221. NM should provide a way for AM to tell it not to aggregate 
logs. Contributed by Ming Ma

(cherry picked from commit 37e1c3d82a)
 2015-08-22 16:26:39 -07:00
Rohith Sharma K S 27ced8253b YARN-3986. getTransferredContainers in AbstractYarnScheduler should be present in YarnScheduler interface 
(cherry picked from commit 22de7c1dca)
 2015-08-21 10:56:45 +05:30
Vinod Kumar Vavilapalli 45be1240d9 Creating 2.6.2 entries in CHANGES.txt files. 
(cherry picked from commit 33474939a0)
 2015-08-20 17:23:54 -07:00
Wangda Tan 6452b31760 YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) 
(cherry picked from commit fc07464d1a)
 2015-08-20 16:52:57 -07:00
Karthik Kambatla b567aa2b4f YARN-4055. Report node resource utilization in heartbeat. (Inigo Goiri via kasha) 
(cherry picked from commit 13604bd5f1)
 2015-08-20 16:52:33 -07:00
Xuan fc6b176e70 YARN-4028. AppBlock page key update and diagnostics value null on 
recovery. Contributed by Bibin A Chundatt

(cherry picked from commit 22dc5fc209)
 2015-08-18 22:54:01 -07:00
Zhihai Xu 051a622b57 YARN-4057. If ContainersMonitor is not enabled, only print related log info one time. Contributed by Jun Gong. 
(cherry picked from commit 14215c8ef8)
 2015-08-18 11:47:41 -07:00
Zhihai Xu 8f2eb99d71 YARN-3857: Memory leak in ResourceManager with SIMPLE mode. Contributed by mujunchao. 
(cherry picked from commit 3a76a010b8)
 2015-08-18 10:53:16 -07:00
Karthik Kambatla d2ff763533 YARN-3534. Collect memory/cpu usage on the node. (Inigo Goiri via kasha) 
(cherry picked from commit def12933b3)
 2015-08-16 06:26:21 -07:00
Jian He b2e869f2a8 YARN-3987. Send AM container completed msg to NM once AM finishes. Contributed by sandflee 
(cherry picked from commit 0a030546e2)
 2015-08-13 16:23:30 -07:00
Jian He 703fa1b141 YARN-4047. ClientRMService getApplications has high scheduler lock contention. Contributed by Jason Lowe 
(cherry picked from commit 7a445fcfab)
 2015-08-13 16:13:28 -07:00
Jian He 4a5fdb7c23 YARN-4005. Completed container whose app is finished is possibly not removed from NMStateStore. Contributed by Jun Gong 
(cherry picked from commit 38aed1a94e)
 2015-08-13 14:46:34 -07:00
Robert Kanter fbcfca6f91 YARN-4031. Add JvmPauseMonitor to ApplicationHistoryServer and WebAppProxyServer (djp via rkanter) 
(cherry picked from commit dc2340c60e)
 2015-08-12 15:48:14 -07:00
Jian He 65d22b3686 YARN-4026. Refactored ContainerAllocator to accept a list of priorites rather than a single priority. Contributed by Wangda Tan 
(cherry picked from commit e5003be907)
 2015-08-12 15:09:46 -07:00
rohithsharmaks c5b20e0db6 YARN-4023. Publish Application Priority to TimelineServer. (Sunil G via rohithsharmaks) 
(cherry picked from commit 1c12adb71f)
 2015-08-12 14:47:23 +05:30
Xuan 2ebdf5bfce YARN-3999. RM hangs on draing events. Contributed by Jian He 
(cherry picked from commit 3ae716fa69)
 2015-08-11 18:26:00 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 3dcff42437 Adding release 2.6.1 to CHANGES.txt 
(cherry picked from commit 7c796fd12c)
 2015-08-11 13:21:41 -07:00
Jian He df97e3265b YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G 
(cherry picked from commit fa1d84ae27)
 2015-08-10 20:52:18 -07:00
Wangda Tan 380d401da5 YARN-3873. PendingApplications in LeafQueue should also use OrderingPolicy. (Sunil G via wangda) 
(cherry picked from commit cf9d3c9256)
 2015-08-10 14:58:19 -07:00
Wangda Tan cdb61b5fb2 YARN-3966. Fix excessive loggings in CapacityScheduler. (Jian He via wangda) 
(cherry picked from commit 4bc42d76e7)
 2015-08-07 09:53:42 -07:00
Rohith Sharma K S a0da1ec010 YARN-3948. Display Application Priority in RM Web UI.(Sunil G via rohithsharmaks) 
(cherry picked from commit b6265d39c5)
 2015-08-07 10:47:08 +05:30
Carlo Curino 8e23a7997f YARN-3974. Refactor the reservation system test cases to use parameterized base test. (subru via curino) 
(cherry picked from commit 8572a5a14b)
 2015-08-02 02:06:06 -07:00
Junping Du 7af531d2e3 YARN-4019. Add JvmPauseMonitor to ResourceManager and NodeManager. Contributed by Robert Kanter. 
(cherry picked from commit cfee02b3bd)
 2015-08-06 06:50:40 -07:00
Arun Suresh 84ea7cd408 YARN-3961. Expose pending, running and reserved containers of a queue in REST api and yarn top (adhoot via asuresh) 
(cherry picked from commit 154c9d2e42)
 2015-08-05 23:15:33 -07:00
rohithsharmaks dc76c4b035 YARN-3992. TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) 
(cherry picked from commit df9e7280db)
 2015-08-06 10:49:37 +05:30
Jian He 1466772827 YARN-3983. Refactored CapacityScheduleri#FiCaSchedulerApp to easier extend container allocation logic. Contributed by Wangda Tan 
(cherry picked from commit ba2313d614)
 2015-08-05 13:51:08 -07:00
Arun Suresh 707b96fa58 YARN-3736. Add RMStateStore apis to store and load accepted reservations for failover (adhoot via asuresh) 
(cherry picked from commit f271d37735)
 2015-08-05 12:58:37 -07:00
Xuan 89ccfd9fcc YARN-4004. container-executor should print output of docker logs if the 
docker container exits with non-0 exit status. Contributed by Varun
Vasudev

(cherry picked from commit c3364ca8e7)
 2015-08-03 18:11:16 -07:00
Xuan 8126fde884 YARN-3543. ApplicationReport should be able to tell whether the 
Application is AM managed or not. Contributed by Rohith Sharma K S

(cherry picked from commit 0306d902f5)
 2015-08-03 15:47:21 -07:00
Jason Lowe b264217bc3 YARN-3965. Add startup timestamp to nodemanager UI. Contributed by Hong Zhiguo 
(cherry picked from commit 469cfcd695)
 2015-08-03 15:55:04 +00:00
Jonathan Eagles 899df5bce0 YARN-3978. Configurably turn off the saving of container info in Generic AHS (Eric Payne via jeagles) 
(cherry picked from commit 3cd02b9522)
 2015-08-03 10:40:04 -05:00
Jason Lowe c31e3ba921 YARN-3990. AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected. Contributed by Bibin A Chundatt 
(cherry picked from commit 32e490b6c0)
 2015-07-31 17:38:49 +00:00
Zhihai Xu 0e2019fa30 YARN-433. When RM is catching up with node updates then it should not expire acquired containers. Contributed by Xuan Gong 
(cherry picked from commit ab80e27703)
 2015-07-30 22:03:04 -07:00
Wangda Tan 53a0653826 YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) 
(cherry picked from commit 91b42e7d6e)
 2015-07-30 10:01:01 -07:00
Wangda Tan a28e23d049 YARN-3963. AddNodeLabel on duplicate label addition shows success. (Bibin A Chundatt via wangda) 
(cherry picked from commit 8acb30b016)
 2015-07-30 09:43:45 -07:00
rohithsharmaks 9c00ae0009 YARN-3919. NPEs' while stopping service after exception during CommonNodeLabelsManager#start. (varun saxena via rohithsharmaks) 2015-07-29 23:44:15 +05:30
Karthik Kambatla 8659315062 YARN-2768. Avoid cloning Resource in FSAppAttempt#updateDemand. (Hong Zhiguo via kasha) 
(cherry picked from commit 5205a330b3)
 2015-07-29 09:42:57 -07:00
Jason Lowe 6f50a6a456 YARN-3950. Add unique SHELL_ID environment variable to DistributedShell. Contributed by Robert Kanter 
(cherry picked from commit 2b2bd92146)
 2015-07-29 15:18:45 +00:00
Xuan 2ec008eb38 YARN-3982. container-executor parsing of container-executor.cfg broken 
in trunk and branch-2. Contributed by Varun Vasudev

(cherry picked from commit f170934215)
 2015-07-27 23:47:31 -07:00
Jian He 7c123accda YARN-3846. RM Web UI queue filter is not working for sub queue. Contributed by Mohammad Shahid Khan 
(cherry picked from commit 3572ebd738)
 2015-07-27 17:12:24 -07:00
Varun Vasudev 9da487e0fd YARN-3853. Add docker container runtime support to LinuxContainterExecutor. Contributed by Sidharta Seethana. 
(cherry picked from commit 3e6fce91a4)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
 2015-07-27 12:03:36 -07:00
Varun Vasudev ec0f801f52 YARN-3852. Add docker container support to container-executor. Contributed by Abin Shahab. 
(cherry picked from commit f36835ff9b)
 2015-07-27 10:18:06 -07:00
Akira Ajisaka 0e7c88347e YARN-3958. TestYarnConfigurationFields should be moved to hadoop-yarn-api module. Contributed by Varun Saxena. 
(cherry picked from commit 42d4e0ae99)
 2015-07-27 11:45:41 +09:00
ccurino 26ea045814 YARN-3656. LowCost: A Cost-Based Placement Agent for YARN Reservations. (Jonathan Yaniv and Ishai Menache via curino) 
(cherry picked from commit 156f24ead0)
 2015-07-25 07:47:11 -07:00
Wangda Tan 4ba2e1ae95 YARN-3973. Recent changes to application priority management break reservation system from YARN-1051 (Carlo Curino via wangda) 
(cherry picked from commit a3bd7b4a59)
 2015-07-24 16:45:06 -07:00
Jason Lowe 2cf714123a YARN-3925. ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks. Contributed by zhihai xu 
(cherry picked from commit ff9c13e0a7)
 2015-07-24 22:15:44 +00:00
Jian He 3233284e87 YARN-3026. Move application-specific container allocation logic from LeafQueue to FiCaSchedulerApp. Contributed by Wangda Tan 
(cherry picked from commit 83fe34ac08)
 2015-07-24 14:01:36 -07:00
carlo curino bddc48ed86 YARN-3969. Updating CHANGES.txt to reflect the correct set of branches where this is committed 
(cherry picked from commit fc42fa8ae3)
 2015-07-24 13:40:14 -07:00
Karthik Kambatla d2ee02940a YARN-3957. FairScheduler NPE In FairSchedulerQueueInfo causing scheduler page to return 500. (Anubhav Dhoot via kasha) 
(cherry picked from commit d19d187753)
 2015-07-24 11:49:46 -07:00
Xuan 98ed4bca3b YARN-3967. Fetch the application report from the AHS if the RM does not 
know about it. Contributed by Mit Desai

(cherry picked from commit fbd6063269)
 2015-07-24 10:17:05 -07:00
carlo curino 6a71a0a230 YARN-3969. Allow jobs to be submitted to reservation that is active but does not have any allocations. (subru via curino) 
(cherry picked from commit 0fcb4a8cf2)
 2015-07-23 19:36:54 -07:00
Rohith Sharma K S 9d15860e6d YARN-3845. Scheduler page does not render RGBA color combinations in IE11. (Contributed by Mohammad Shahid Khan) 2015-07-24 12:54:07 +05:30
Robert Kanter ba3b40ae79 YARN-3900. Protobuf layout of yarn_security_token causes errors in other protos that include it (adhoot via rkanter) 
(cherry picked from commit 1d3026e7b3)
 2015-07-23 14:47:28 -07:00
Wangda Tan 6baebcaedc YARN-3941. Proportional Preemption policy should try to avoid sending duplicate PREEMPT_CONTAINER event to scheduler. (Sunil G via wangda) 
(cherry picked from commit 3bba180051)
 2015-07-23 10:08:30 -07:00
Junping Du 6772c3f4dd YARN-2019. Retrospect on decision of making RM crashed if any exception throw in ZKRMStateStore. Contributed by Jian He. 
(cherry picked from commit ee98d6354b)
 2015-07-22 17:57:16 -07:00
rohithsharmaks be2334ba3a YARN-3954. Fix TestYarnConfigurationFields#testCompareConfigurationClassAgainstXml. (varun saxena via rohithsharmaks) 2015-07-23 02:10:45 +05:30
Wangda Tan 3800e25d6e YARN-3956. Fix TestNodeManagerHardwareUtils fails on Mac (Varun Vasudev via wangda) 
(cherry picked from commit 06e5dd2c84)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-07-22 12:03:14 -07:00
Wangda Tan 3576b0ac5e YARN-3932. SchedulerApplicationAttempt#getResourceUsageReport and UserInfo should based on total-used-resources. (Bibin A Chundatt via wangda) 
(cherry picked from commit 76ec26de80)
 2015-07-22 11:54:38 -07:00
Jian He 11cb00f7a0 YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. Contributed by Varun Saxena 
(cherry picked from commit 393fe71771)
 2015-07-21 15:06:16 -07:00
Wangda Tan 3dd113fa7c YARN-2003. Support for Application priority : Changes in RM and Capacity Scheduler. (Sunil G via wangda) 
(cherry picked from commit c39ca541f4)
 2015-07-21 09:57:53 -07:00
Arun Suresh eddb326669 Pulling in YARN-3535 to branch 2.7.x 
(cherry picked from commit 176131f12b)
 2015-07-18 10:11:29 -07:00
Colin Patrick Mccabe aff955a14a YARN-3844. Make hadoop-yarn-project Native code -Wall-clean (Alan Burlison via Colin P. McCabe) 
(cherry picked from commit 419c51d233)
 2015-07-17 12:05:24 -07:00
Jonathan Eagles b289939514 YARN-3905. Application History Server UI NPEs when accessing apps run after RM restart (Eric Payne via jeagles) 
(cherry picked from commit 7faae0e6fe)
 2015-07-17 11:03:34 -05:00
Arun Suresh 8018041b49 YARN-3535. Scheduler must re-request container resources when RMContainer transitions from ALLOCATED to KILLED (rohithsharma and peng.zhang via asuresh) 
(cherry picked from commit 9b272ccae7)
 2015-07-17 04:33:04 -07:00
Wangda Tan a7de3cde13 YARN-3885. ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2 level. (Ajith S via wangda) 
(cherry picked from commit 3540d5fe4b)
 2015-07-16 16:14:07 -07:00
Wangda Tan 7b2540770e YARN-3930. FileSystemNodeLabelsStore should make sure edit log file closed when exception is thrown. (Dian Fu via wangda) 
(cherry picked from commit fa2b63ed16)
 2015-07-16 16:08:04 -07:00
Tsuyoshi Ozawa e97a14ae6f YARN-3805. Update the documentation of Disk Checker based on YARN-90. Contributed by Masatake Iwasaki. 
(cherry picked from commit 1ba2986dee)
 2015-07-16 17:52:58 +09:00
Tsuyoshi Ozawa f95e3c3091 YARN-3174. Consolidate the NodeManager and NodeManagerRestart documentation into one. Contributed by Masatake Iwasaki. 
(cherry picked from commit f02dd146f5)
 2015-07-16 15:22:57 +09:00
Tsuyoshi Ozawa bfbc805a13 YARN-3170. YARN architecture document needs updating. Contirubted by Brahma Reddy Battula. 
(cherry picked from commit edcaae44c1)
 2015-07-15 15:43:09 +09:00
Arun Suresh 5f58be7dd4 YARN-3453. Ensure preemption logic in FairScheduler uses DominantResourceCalculator in DRF queues to prevent unnecessary thrashing. (asuresh) 
(cherry picked from commit ac94ba3e18)
 2015-07-14 00:25:57 -07:00
Jian He 27366dd7c3 Revert "YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. (Varun Saxena via kasha)" 
This reverts commit aa067c6aa4.
(cherry picked from commit 2466460d4c)
 2015-07-13 14:31:55 -07:00
Akira Ajisaka 0ba2defb9e YARN-3381. Fix typo InvalidStateTransitonException. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 19295b36d9)
 2015-07-13 17:53:27 +09:00
Wangda Tan 87360be089 YARN-3894. RM startup should fail for wrong CS xml NodeLabel capacity configuration. (Bibin A Chundatt via wangda) 
(cherry picked from commit 5ed1fead6b)
 2015-07-12 21:53:08 -07:00
Akira Ajisaka ad6787aa43 YARN-3069. Document missing properties in yarn-default.xml. Contributed by Ray Chiang. 
(cherry picked from commit d6675606dc)
 2015-07-13 12:57:29 +09:00
Gera Shegalov 896b8c6551 YARN-3917. getResourceCalculatorPlugin for the default should intercept all exceptions. (gera) 
(cherry picked from commit d7319dee37)
 2015-07-11 22:41:04 -07:00
Wangda Tan c36090fd3f YARN-3849. Too much of preemption activity causing continuos killing of containers across queues. (Sunil G via wangda) 
(cherry picked from commit 1df39c1efc)
 2015-07-11 10:27:58 -07:00
Zhijie Shen 37a93c2d78 YARN-3116. RM notifies NM whether a container is an AM container or normal task container. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 1ea36299a4)
 2015-07-10 19:02:13 -07:00
Ming Ma b169889f01 YARN-3445. Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) 
(cherry picked from commit 08244264c0)
 2015-07-10 08:34:01 -07:00
Xuan 2a01495544 YARN-3888. ApplicationMaster link is broken in RM WebUI when appstate is 
NEW. Contributed by Bibin A Chundatt

(cherry picked from commit 5214876792)
 2015-07-09 21:38:46 -07:00
carlo curino 753d87066f YARN-3800. Reduce storage footprint for ReservationAllocation. Contributed by Anubhav Dhoot. 
(cherry picked from commit 0e602fa3a1)
 2015-07-09 16:52:47 -07:00
Karthik Kambatla ccf18705f7 YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. (Varun Saxena via kasha) 
(cherry picked from commit aa067c6aa4)
 2015-07-09 09:48:46 -07:00
Karthik Kambatla d15f69457b YARN-1012. Report NM aggregated container resource utilization in heartbeat. (Inigo Goiri via kasha) 
(cherry picked from commit 527c40e4d6)
 2015-07-09 09:35:39 -07:00
Akira Ajisaka d260478d3a YARN-3690. [JDK8] 'mvn site' fails. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d6325745e2)
 2015-07-08 15:43:04 +09:00
Jian He fa59b224b8 YARN-3892. Fixed NPE on RMStateStore#serviceStop when CapacityScheduler#serviceInit fails. Contributed by Bibin A Chundatt 
(cherry picked from commit c9dd2cada0)
 2015-07-07 14:18:16 -07:00
Varun Vasudev 16ca0fcf99 YARN-2194. Fix bug causing CGroups functionality to fail on RHEL7. Contributed by Wei Yan. 
(cherry-picked from commit 7e3ebc0fc59dbd1d7768b5a6c05af18b8c8fea4d)
 2015-07-07 17:05:06 +05:30
Devaraj K f558429efe YARN-3837. javadocs of TimelineAuthenticationFilterInitializer give wrong 
prefix for auth options. Contributed by Bibin A Chundatt.

(cherry picked from commit af63427c6d)
 2015-07-07 12:08:03 +05:30
Vinod Kumar Vavilapalli (I am also known as @tshooter.) a8bf34cad8 Release process for 2.7.1: Set the release date for 2.7.1. 
(cherry picked from commit bf89ddb9b8)
 2015-07-06 16:41:00 -07:00
Xuan 09639ac6b4 YARN-3882. AggregatedLogFormat should close aclScanner and ownerScanner 
after create them. Contributed by zhihai xu

(cherry picked from commit 688617d6d7)
 2015-07-04 21:54:22 -07:00
Devaraj K e1ca886419 YARN-3875. FSSchedulerNode#reserveResource() doesn't print Application Id 
properly in log. Contributed by Bibin A Chundatt.

(cherry picked from commit 37d7395773)
 2015-07-02 10:21:32 +05:30
Wangda Tan d61dd10b50 YARN-3508. Prevent processing preemption events on the main RM dispatcher. (Varun Saxena via wangda) 
(cherry picked from commit 0e4b06690f)
 2015-07-01 17:32:57 -07:00
Jason Lowe a721b9d812 YARN-3793. Several NPEs when deleting local files on NM recovery. Contributed by Varun Saxena 
(cherry picked from commit b5cdf78e8e)
 2015-07-01 21:14:35 +00:00
Devaraj K c343250657 YARN-3830. AbstractYarnScheduler.createReleaseCache may try to clean a 
null attempt. Contributed by nijel.

(cherry picked from commit 80a68d6056)
 2015-07-01 19:05:27 +05:30
Devaraj K 3432017de0 YARN-3823. Fix mismatch in default values for 
yarn.scheduler.maximum-allocation-vcores property. Contributed by Ray
Chiang.

(cherry picked from commit 7405c59799)
 2015-07-01 11:12:43 +05:30
Colin Patrick Mccabe b1da58d23d YARN-3827. Migrate YARN native build to new CMake framework (Alan Burlison via Colin P. McCabe) 
(cherry picked from commit d0cc0380b5)
 2015-06-30 16:24:44 -07:00
Gera Shegalov f9b98082b8 YARN-3768. ArrayIndexOutOfBoundsException with empty environment variables. (Zhihai Xu via gera) 
(cherry picked from commit 6f2a41e37d)
 2015-06-30 15:05:01 -07:00
Jian He 484a6c0f01 YARN-3770. SerializedException should also handle java.lang.Error on de-serialization. Contributed by Lavkesh Lahngir 
(cherry picked from commit 4672315e2d)
 2015-06-29 14:32:22 -07:00
Jian He 0100995c5e YARN-3695. ServerProxy (NMProxy, etc.) shouldn't retry forever for non network exception. Contributed by Raju Bairishetti 
(cherry picked from commit 62e583c7dc)
 2015-06-29 13:38:04 -07:00
Junping Du dbd2b18205 YARN-3860. rmadmin -transitionToActive should check the state of non-target node. (Contributed by Masatake Iwasaki) 
(cherry picked from commit a95d39f9d0)
 2015-06-28 22:28:35 -07:00
Vinod Kumar Vavilapalli 4155bb565e Adding release 2.7.2 to CHANGES.txt. 
(cherry picked from commit aad6a7d5db)
 2015-06-28 16:30:27 -07:00
Devaraj K e163c1e0da YARN-3859. LeafQueue doesn't print user properly for application add. 
Contributed by Varun Saxena.

(cherry picked from commit b543d1a390)
 2015-06-28 10:06:08 +05:30
Xuan 1a8d162bc4 YARN-2871. TestRMRestart#testRMRestartGetApplicationList sometime fails 
in trunk. Contributed by zhihai xu

(cherry picked from commit fe6c1bd73a)
 2015-06-26 19:45:00 -07:00
Jason Lowe 0221d19f4e YARN-3850. NM fails to read files from full disks which can lead to container logs being lost and other issues. Contributed by Varun Saxena 
(cherry picked from commit 40b256949a)
 2015-06-26 15:48:05 +00:00
Devaraj K bb8a2cff2a YARN-3745. SerializedException should also try to instantiate internal 
exception with the default constructor. Contributed by Lavkesh Lahngir.

(cherry picked from commit b381f88c71)
 2015-06-25 17:39:40 +05:30
Devaraj K 3e20f595d4 YARN-3826. Race condition in ResourceTrackerService leads to wrong 
diagnostics messages. Contributed by Chengbing Liu.

(cherry picked from commit 57f1a01eda)
 2015-06-25 16:15:24 +05:30
Jonathan Eagles d1f0ff165b YARN-3360. Add JMX metrics to TimelineDataManager (Jason Lowe via jeagles) 
(cherry picked from commit 4c659ddbf7)
 2015-06-24 16:17:32 -05:00
rohithsharmaks 1619d8d62a Move YARN-3790 from 2.7.1 to 2.8 in CHANGES.txt 2015-06-24 23:31:09 +05:30
rohithsharmaks 1392d1ad94 YARN-3790. usedResource from rootQueue metrics may get stale data for FS scheduler after recovering the container (Zhihai Xu via rohithsharmaks) 2015-06-24 23:00:25 +05:30
Jason Lowe 15b1800b12 YARN-3832. Resource Localization fails on a cluster due to existing cache directories. Contributed by Brahma Reddy Battula 
(cherry picked from commit 8d58512d6e)
 2015-06-24 16:38:22 +00:00
Jason Lowe 14afa5d53d YARN-3809. Failed to launch new attempts because ApplicationMasterLauncher's threads all hang. Contributed by Jun Gong 
(cherry picked from commit 2a20dd9b61)
 2015-06-24 16:24:43 +00:00
Robert Kanter 5c1e809af5 YARN-3835. hadoop-yarn-server-resourcemanager test package bundles core-site.xml, yarn-site.xml (vamsee via rkanter) 
(cherry picked from commit 99271b7621)
 2015-06-22 18:03:00 -07:00
Karthik Kambatla 9656ee4ee7 YARN-3842. NMProxy should retry on NMNotYetReadyException. (Robert Kanter via kasha) 
(cherry picked from commit 5ebf2817e5)
 2015-06-22 17:48:07 -07:00
Xuan c3254a63a9 YARN-3834. Scrub debug logging of tokens during resource localization. Contributed by Chris Nauroth 
(cherry picked from commit 6c7a9d502a)
 2015-06-21 17:14:59 -07:00
Xuan 86b75ac544 YARN-3802. Two RMNodes for the same NodeId are used in RM sometimes 
after NM is reconnected. Contributed by zhihai xu

(cherry picked from commit 5b5bb8dcdc)
 2015-06-18 14:38:52 -07:00
Devaraj K 3b9698ecac YARN-3824. Fix two minor nits in member variable properties of 
YarnConfiguration. Contributed by Ray Chiang.

(cherry picked from commit 2ad668748e)
 2015-06-18 16:46:33 +05:30
Xuan 16d2412a25 YARN-3804. Both RM are on standBy state when kerberos user not in yarn.admin.acl. Contributed by Varun Saxena 
(cherry picked from commit a826d432f9)
 2015-06-17 16:24:50 -07:00
Devaraj K f0901e703b YARN-3148. Allow CORS related headers to passthrough in 
WebAppProxyServlet. Contributed by Varun Saxena.

(cherry picked from commit ebb9a82519)
 2015-06-17 15:57:45 +05:30
Devaraj K 2802c63ac3 YARN-3617. Fix WindowsResourceCalculatorPlugin.getCpuFrequency() returning 
always -1. Contributed by J.Andreina.

(cherry picked from commit 318d2cde7c)
 2015-06-17 13:55:25 +05:30
Xuan fe7f2d40fe YARN-3714. AM proxy filter can not get RM webapp address from 
yarn.resourcemanager.hostname.rm-id. Contributed by Masatake Iwasaki

(cherry picked from commit e27d5a13b0)
 2015-06-16 14:06:56 -07:00
Devaraj K eb8e2c5448 YARN-3789. Improve logs for LeafQueue#activateApplications(). Contributed 
by Bibin A Chundatt.

(cherry picked from commit b039e69bb0)
 2015-06-16 14:04:46 +05:30
Tsuyoshi Ozawa ff6d5f2ee7 YARN-3711. Documentation of ResourceManager HA should explain configurations about listen addresses. Contributed by Masatake Iwasaki. 
(cherry picked from commit e8c514373f)
 2015-06-16 10:15:19 +09:00
Devaraj K 3863342ca9 YARN-3794. TestRMEmbeddedElector fails because of ambiguous LOG reference. 
Contributed by Chengbing Liu.

(cherry picked from commit d8dcfa98e3)
 2015-06-12 13:45:05 +05:30
Xuan 8a6c7d9973 YARN-3785. Support for Resource as an argument during submitApp call in 
MockRM test class. Contributed by Sunil G

(cherry picked from commit 5583f88bf7)
 2015-06-10 21:41:58 -07:00
Zhijie Shen 0dad10d845 YARN-3787. Allowed generic history service to load a number of applications whose started time is within the given range. Contributed by Xuan Gong. 
(cherry picked from commit 8d0ef31632)
 2015-06-08 23:21:35 -07:00
cnauroth d9016b0286 YARN-3786. Document yarn class path options. Contributed by Brahma Reddy Battula. 
(cherry picked from commit a531b058ae)
 2015-06-08 16:18:43 -07:00
Xuan 869304dc83 YARN-3778. Fix Yarn resourcemanger CLI usage. Contributed by Brahma Reddy Battula 
(cherry picked from commit 2b2465dfac)
 2015-06-08 15:44:25 -07:00
Jian He a24ead8c6d YARN-2716. Refactor ZKRMStateStore retry code with Apache Curator. Contributed by Karthik Kambatla 
(cherry picked from commit 960b8f19ca)
 2015-06-08 14:51:25 -07:00
Devaraj K 8c643e3bf9 YARN-3747. TestLocalDirsHandlerService should delete the created test 
directory logDir2. Contributed by David Moore.

(cherry picked from commit 126321eded)
 2015-06-08 15:33:38 +05:30
Devaraj K 2a01c01af7 YARN-3780. Should use equals when compare Resource in 
RMNodeImpl#ReconnectNodeTransition. Contributed by zhihai xu.

(cherry picked from commit c7ee6c151c)
 2015-06-08 11:56:58 +05:30
Karthik Kambatla 322e7d0d5e YARN-3655. FairScheduler: potential livelock due to maxAMShare limitation and container reservation. (Zhihai Xu via kasha) 
(cherry picked from commit bd69ea408f)
 2015-06-07 11:38:14 -07:00
Xuan 4d385b48f0 YARN-1462. AHS API and other AHS changes to handle tags for completed MR jobs. Contributed by Xuan Gong 
(cherry picked from commit 3e000a919f)
 2015-06-05 12:49:51 -07:00
Karthik Kambatla 429754de09 YARN-3259. FairScheduler: Trigger fairShare updates on node events. (Anubhav Dhoot via kasha) 
(cherry picked from commit 75885852cc)
 2015-06-05 09:42:12 -07:00
Zhijie Shen 8f04ac9c34 YARN-3766. Fixed the apps table column error of generic history web UI. Contributed by Xuan Gong. 
(cherry picked from commit 18dd01d6bf)
 2015-06-04 14:48:05 -07:00
Jian He 2e99210e4e YARN-2392. Add more diags about app retry limits on AM failures. Contributed by Steve Loughran 
(cherry picked from commit 1970ca7cbc)
 2015-06-04 11:14:36 -07:00
Jian He 6325e4b7dd YARN-3764. CapacityScheduler should forbid moving LeafQueue from one parent to another. Contributed by Wangda Tan 
(cherry picked from commit 6ad4e59cfc)
 2015-06-04 10:53:24 -07:00
Wangda Tan 78d626fa89 YARN-3733. Fix DominantRC#compare() does not work as expected if cluster resource is empty. (Rohith Sharmaks via wangda) 
(cherry picked from commit ebd797c48f)
 2015-06-04 10:23:33 -07:00
Junping Du d8c7ee1b53 YARN-41. The RM should handle the graceful shutdown of the NM. Contributed by Devaraj K. 
(cherry picked from commit d7e7f6aa03)
 2015-06-04 05:00:32 -07:00
Xuan 3f0573b059 YARN-3749. We should make a copy of configuration when init 
MiniYARNCluster with multiple RMs. Contributed by Chun Chen

(cherry picked from commit 5766a04428)
 2015-06-03 17:21:48 -07:00
Zhijie Shen 2fcd3fe6ba Revert "YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong." 
This reverts commit 4e2ed34d20.
 2015-06-03 14:16:51 -07:00
Zhijie Shen 5e976fffe3 Revert "YARN-1462. Correct fix version from branch-2.7.1 to branch-2.8 in" 
This reverts commit 26ad3293ec.
 2015-06-03 14:16:51 -07:00
Karthik Kambatla 62d51b889a YARN-3762. FairScheduler: CME on FSParentQueue#getQueueUserAclInfo. (kasha) 
(cherry picked from commit edb9cd0f7a)
 2015-06-03 13:50:02 -07:00
Jason Lowe 752caa95a4 YARN-3585. NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled. Contributed by Rohith Sharmaks 
(cherry picked from commit e13b671aa5)
 2015-06-03 19:45:18 +00:00
Zhijie Shen aaa567514a YARN-3751. Fixed AppInfo to check if used resources are null. Contributed by Sunil G. 
(cherry picked from commit dbc4f64937)
 2015-06-03 11:52:39 -07:00
Xuan 26ad3293ec YARN-1462. Correct fix version from branch-2.7.1 to branch-2.8 in 
CHANGES.txt

(cherry picked from commit 0b5cfacde6)
 2015-06-01 11:40:24 -07:00
Wangda Tan a3734f67d3 YARN-3725. App submission via REST API is broken in secure mode due to Timeline DT service address is empty. (Zhijie Shen via wangda) 
(cherry picked from commit 5cc3fced95)
 2015-05-31 16:33:50 -07:00
Xuan 4563411e04 YARN-2900. Application (Attempt and Container) Not Found in AHS results 
in Internal Server Error (500). Contributed by Zhijie Shen and Mit Desai

(cherry picked from commit 9686261ecb)
 2015-05-31 15:38:39 -07:00
Xuan 1ebaff0b86 Revert "YARN-2900. Application (Attempt and Container) Not Found in AHS results" 
This reverts commit 4fee8b3202.
 2015-05-31 00:24:08 -07:00
Xuan 4fee8b3202 YARN-2900. Application (Attempt and Container) Not Found in AHS results 
in Internal Server Error (500). Contributed by Zhijie Shen and Mit Desai

(cherry picked from commit 06f8e9caba)
 2015-05-30 21:58:04 -07:00
Zhijie Shen 4e2ed34d20 YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong. 
(cherry picked from commit a9f986bc5a41ab74171da1076d10e83c5adc284f)
 2015-05-30 21:06:32 -07:00
Karthik Kambatla 6c80e599d1 YARN-3467. Expose allocatedMB, allocatedVCores, and runningContainers metrics on running Applications in RM Web UI. (Anubhav Dhoot via kasha) 
(cherry picked from commit a8acdd65b3)
 2015-05-30 13:36:49 -07:00
Zhijie Shen 68cddb894a YARN-3740. Fixed the typo in the configuration name: APPLICATION_HISTORY_PREFIX_MAX_APPS. Contributed by Xuan Gong. 
(cherry picked from commit eb6bf91eea)
 2015-05-29 18:18:47 -07:00
Robert Kanter bde4031a02 YARN-3713. Remove duplicate function call storeContainerDiagnostics in ContainerDiagnosticsUpdateTransition (zxu via rkanter) 
(cherry picked from commit 6aec13cb33)
 2015-05-29 15:35:08 -07:00
Karthik Kambatla b9c5c5de68 YARN-3547. FairScheduler: Apps that have no resource demand should not participate scheduling. (Xianyin Xin via kasha) 
(cherry picked from commit 3ae2a62501)
 2015-05-29 15:17:43 -07:00
Wangda Tan 2fb794ecaf YARN-3716. Node-label-expression should be included by ResourceRequestPBImpl.toString. (Xianyin Xin via wangda) 
(cherry picked from commit 788bfa0359)
 2015-05-28 16:35:17 -07:00
Xuan 550b55146d YARN-3723. Need to clearly document primaryFilter and otherInfo value 
type. Contributed by Zhijie Shen

(cherry picked from commit 3077c299da)
 2015-05-28 10:20:33 -07:00
Devaraj K 7116141d7d YARN-3722. Merge multiple TestWebAppUtils into 
o.a.h.yarn.webapp.util.TestWebAppUtils. Contributed by Masatake Iwasaki.

(cherry picked from commit 7e509f5843)
 2015-05-28 15:36:13 +05:30
Zhijie Shen 839f81a632 YARN-3700. Made generic history service load a number of latest applications according to the parameter or the configuration. Contributed by Xuan Gong. 
(cherry picked from commit 54504133f4)
 2015-05-27 16:53:47 -07:00
Wangda Tan 4790495538 YARN-3581. Deprecate -directlyAccessNodeLabelStore in RMAdminCLI. (Naganarasimha G R via wangda) 
(cherry picked from commit cab7674e54)
 2015-05-27 15:37:14 -07:00
Wangda Tan 8857cba72e YARN-3647. RMWebServices api's should use updated api from CommonNodeLabelsManager to get NodeLabel object. (Sunil G via wangda) 
(cherry picked from commit ec0a852a37)
 2015-05-27 13:15:55 -07:00
Wangda Tan 0728a86b3a YARN-3686. CapacityScheduler should trim default_node_label_expression. (Sunil G via wangda) 
(cherry picked from commit cdbd66be11)
 2015-05-26 16:00:17 -07:00
Jian He 5e7be094ec YARN-3632. Ordering policy should be allowed to reorder an application when demand changes. Contributed by Craig Welch 
(cherry picked from commit 10732d515f)
 2015-05-26 12:02:36 -07:00
Vinod Kumar Vavilapalli a67cb4826b YARN-160. Enhanced NodeManager to automatically obtain cpu/memory values from underlying OS when configured to do so. Contributed by Varun Vasudev. 
(cherry picked from commit 500a1d9c76)
 2015-05-26 11:39:40 -07:00
Tsuyoshi Ozawa cf5c1adf4f YARN-2336. Fair scheduler's REST API returns a missing '[' bracket JSON for deep queue tree. Contributed by Kenji Kikushima and Akira Ajisaka. 
(cherry picked from commit 9a3d617b63)
 2015-05-26 19:08:01 +09:00
Xuan 84245ff3b2 YARN-2238. Filtering on UI sticks even if I move away from the page. 
Contributed by Jian He

(cherry picked from commit 39077dba2e)
 2015-05-25 22:41:02 -07:00
Jian He 03af057a47 YARN-3707. RM Web UI queue filter doesn't work. Contributed by Wangda Tan 
(cherry picked from commit 446d51591e)
 2015-05-22 15:59:00 -07:00
Xuan 33be070a5e YARN-3701. Isolating the error of generating a single app report when 
getting all apps from generic history service. Contributed by Zhijie
Shen

(cherry picked from commit 455b3acf0e)
 2015-05-22 14:35:51 -07:00
Junping Du 5fb61d71b1 YARN-3594. WintuilsProcessStubExecutor.startStreamReader leaks streams. Contributed by Lars Francke. 
(cherry picked from commit 132d909d4a)
 2015-05-22 04:25:18 -07:00
Vinod Kumar Vavilapalli 673bd9708f YARN-3684. Changed ContainerExecutor's primary lifecycle methods to use a more extensible mechanism of context objects. Contributed by Sidharta Seethana. 
(cherry picked from commit 53fafcf061)
 2015-05-21 15:51:19 -07:00
Karthik Kambatla e8ac88d4fe YARN-3675. FairScheduler: RM quits when node removal races with continuous-scheduling on the same node. (Anubhav Dhoot via kasha) 
(cherry picked from commit a8b50e46737c11936ba72c427da69b2365a07aac)
 2015-05-21 13:39:40 -07:00
Devaraj K 157ecb2241 YARN-3646. Applications are getting stuck some times in case of retry 
policy forever. Contributed by Raju Bairishetti.

(cherry picked from commit 0305316d69)
 2015-05-21 20:16:09 +05:30
Akira Ajisaka 6d7e7ef1c4 YARN-3694. Fix dead link for TimelineServer REST API. Contributed by Jagadesh Kiran N. 
(cherry picked from commit a5def58087)
 2015-05-21 23:15:17 +09:00
Jian He 7c3895c8f6 YARN-3654. ContainerLogsPage web UI should not have meta-refresh. Contributed by Xuan Gong 
(cherry picked from commit 6329bd00fa)
 2015-05-20 17:21:19 -07:00
Jian He 521378ffa3 YARN-3609. Load node labels from storage inside RM serviceStart. Contributed by Wangda Tan 
(cherry picked from commit 8966d42179)
 2015-05-20 16:30:44 -07:00
Xuan a665b22cfa YARN-3681. yarn cmd says "could not find main class 'queue'" in windows. Contributed by Craig Welch and Varun Saxena 2015-05-20 14:42:59 -07:00
Wangda Tan d3193fd1d7 Move YARN-2918 from 2.8.0 to 2.7.1 
(cherry picked from commit 03f897fd1a)
 2015-05-20 13:50:11 -07:00
Wangda Tan b5ba6979b7 YARN-3583. Support of NodeLabel object instead of plain String in YarnClient side. (Sunil G via wangda) 
(cherry picked from commit 563eb1ad2a)
(cherry picked from commit b0d22b0c606fad6b4ab5443c0aed07c829b46726)
 2015-05-19 17:37:24 -07:00
Tsuyoshi Ozawa d022c5aba8 YARN-3677. Fix findbugs warnings in yarn-server-resourcemanager. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit 7401e5b5e8)
 2015-05-20 09:01:42 +09:00
Wangda Tan 7b97f049ee YARN-3565. NodeHeartbeatRequest/RegisterNodeManagerRequest should use NodeLabel object instead of String. (Naganarasimha G R via wangda) 
(cherry picked from commit b37da52a1c)
 2015-05-19 16:35:23 -07:00
Jian He adb90c7f52 YARN-2821. Fixed a problem that DistributedShell AM may hang if restarted. Contributed by Varun Vasudev 
(cherry picked from commit 7438966586)
 2015-05-19 14:21:48 -07:00
Ravi Prakash 084e453629 YARN-3302. TestDockerContainerExecutor should run automatically if it can detect docker in the usual place (Ravindra Kumar Naik via raviprak) 
(cherry picked from commit c97f32e7b9)
 2015-05-19 10:30:24 -07:00
Xuan d39039d54d YARN-3601. Fix UT TestRMFailover.testRMWebAppRedirect. Contributed by Weiwei Yang 
(cherry picked from commit 5009ad4a7f)
 2015-05-19 09:57:13 -07:00
Xuan 3ceb2ffe54 YARN-3541. Add version info on timeline service / generic history web UI and REST API. Contributed by Zhijie Shen 
(cherry picked from commit 76afd28862)
 2015-05-18 13:19:01 -07:00
Xuan 802676e1be YARN-3526. ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster. Contributed by Weiwei Yang 
(cherry picked from commit b0ad644083)
 2015-05-15 22:41:13 -07:00
Jason Lowe ce64720516 YARN-2421. RM still allocates containers to an app in the FINISHING state. Contributed by Chang Li 
(cherry picked from commit f7e051c431)
 2015-05-15 22:10:16 +00:00
Ravi Prakash 4caadaa8bb YARN-1519. Check in container-executor if sysconf is implemented before using it (Radim Kolar and Eric Payne via raviprak) 
(cherry picked from commit 53fe4eff09)
 2015-05-14 15:56:36 -07:00
Junping Du bc13c7d84b YARN-3505. Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. 
(cherry picked from commit 15ccd967ee)
 2015-05-14 10:59:48 -07:00