Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,914 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2462 Commits

Author SHA1 Message Date
cnauroth d9016b0286 YARN-3786. Document yarn class path options. Contributed by Brahma Reddy Battula. 
(cherry picked from commit a531b058ae)
 2015-06-08 16:18:43 -07:00
Xuan 869304dc83 YARN-3778. Fix Yarn resourcemanger CLI usage. Contributed by Brahma Reddy Battula 
(cherry picked from commit 2b2465dfac)
 2015-06-08 15:44:25 -07:00
Jian He a24ead8c6d YARN-2716. Refactor ZKRMStateStore retry code with Apache Curator. Contributed by Karthik Kambatla 
(cherry picked from commit 960b8f19ca)
 2015-06-08 14:51:25 -07:00
Devaraj K 8c643e3bf9 YARN-3747. TestLocalDirsHandlerService should delete the created test 
directory logDir2. Contributed by David Moore.

(cherry picked from commit 126321eded)
 2015-06-08 15:33:38 +05:30
Devaraj K 2a01c01af7 YARN-3780. Should use equals when compare Resource in 
RMNodeImpl#ReconnectNodeTransition. Contributed by zhihai xu.

(cherry picked from commit c7ee6c151c)
 2015-06-08 11:56:58 +05:30
Karthik Kambatla 322e7d0d5e YARN-3655. FairScheduler: potential livelock due to maxAMShare limitation and container reservation. (Zhihai Xu via kasha) 
(cherry picked from commit bd69ea408f)
 2015-06-07 11:38:14 -07:00
Xuan 4d385b48f0 YARN-1462. AHS API and other AHS changes to handle tags for completed MR jobs. Contributed by Xuan Gong 
(cherry picked from commit 3e000a919f)
 2015-06-05 12:49:51 -07:00
Karthik Kambatla 429754de09 YARN-3259. FairScheduler: Trigger fairShare updates on node events. (Anubhav Dhoot via kasha) 
(cherry picked from commit 75885852cc)
 2015-06-05 09:42:12 -07:00
Zhijie Shen 8f04ac9c34 YARN-3766. Fixed the apps table column error of generic history web UI. Contributed by Xuan Gong. 
(cherry picked from commit 18dd01d6bf)
 2015-06-04 14:48:05 -07:00
Wangda Tan 344b750915 Add missing test file of YARN-3733 
(cherry picked from commit 405bbcf68c)
 2015-06-04 13:20:49 -07:00
Jian He 2e99210e4e YARN-2392. Add more diags about app retry limits on AM failures. Contributed by Steve Loughran 
(cherry picked from commit 1970ca7cbc)
 2015-06-04 11:14:36 -07:00
Jian He 6325e4b7dd YARN-3764. CapacityScheduler should forbid moving LeafQueue from one parent to another. Contributed by Wangda Tan 
(cherry picked from commit 6ad4e59cfc)
 2015-06-04 10:53:24 -07:00
Wangda Tan 78d626fa89 YARN-3733. Fix DominantRC#compare() does not work as expected if cluster resource is empty. (Rohith Sharmaks via wangda) 
(cherry picked from commit ebd797c48f)
 2015-06-04 10:23:33 -07:00
Junping Du d8c7ee1b53 YARN-41. The RM should handle the graceful shutdown of the NM. Contributed by Devaraj K. 
(cherry picked from commit d7e7f6aa03)
 2015-06-04 05:00:32 -07:00
Xuan 3f0573b059 YARN-3749. We should make a copy of configuration when init 
MiniYARNCluster with multiple RMs. Contributed by Chun Chen

(cherry picked from commit 5766a04428)
 2015-06-03 17:21:48 -07:00
Zhijie Shen 2fcd3fe6ba Revert "YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong." 
This reverts commit 4e2ed34d20.
 2015-06-03 14:16:51 -07:00
Zhijie Shen 5e976fffe3 Revert "YARN-1462. Correct fix version from branch-2.7.1 to branch-2.8 in" 
This reverts commit 26ad3293ec.
 2015-06-03 14:16:51 -07:00
Karthik Kambatla 62d51b889a YARN-3762. FairScheduler: CME on FSParentQueue#getQueueUserAclInfo. (kasha) 
(cherry picked from commit edb9cd0f7a)
 2015-06-03 13:50:02 -07:00
Jason Lowe 752caa95a4 YARN-3585. NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled. Contributed by Rohith Sharmaks 
(cherry picked from commit e13b671aa5)
 2015-06-03 19:45:18 +00:00
Zhijie Shen aaa567514a YARN-3751. Fixed AppInfo to check if used resources are null. Contributed by Sunil G. 
(cherry picked from commit dbc4f64937)
 2015-06-03 11:52:39 -07:00
Xuan 26ad3293ec YARN-1462. Correct fix version from branch-2.7.1 to branch-2.8 in 
CHANGES.txt

(cherry picked from commit 0b5cfacde6)
 2015-06-01 11:40:24 -07:00
Wangda Tan a3734f67d3 YARN-3725. App submission via REST API is broken in secure mode due to Timeline DT service address is empty. (Zhijie Shen via wangda) 
(cherry picked from commit 5cc3fced95)
 2015-05-31 16:33:50 -07:00
Xuan 4563411e04 YARN-2900. Application (Attempt and Container) Not Found in AHS results 
in Internal Server Error (500). Contributed by Zhijie Shen and Mit Desai

(cherry picked from commit 9686261ecb)
 2015-05-31 15:38:39 -07:00
Xuan 1ebaff0b86 Revert "YARN-2900. Application (Attempt and Container) Not Found in AHS results" 
This reverts commit 4fee8b3202.
 2015-05-31 00:24:08 -07:00
Xuan 4fee8b3202 YARN-2900. Application (Attempt and Container) Not Found in AHS results 
in Internal Server Error (500). Contributed by Zhijie Shen and Mit Desai

(cherry picked from commit 06f8e9caba)
 2015-05-30 21:58:04 -07:00
Zhijie Shen 4e2ed34d20 YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong. 
(cherry picked from commit a9f986bc5a41ab74171da1076d10e83c5adc284f)
 2015-05-30 21:06:32 -07:00
Karthik Kambatla 6c80e599d1 YARN-3467. Expose allocatedMB, allocatedVCores, and runningContainers metrics on running Applications in RM Web UI. (Anubhav Dhoot via kasha) 
(cherry picked from commit a8acdd65b3)
 2015-05-30 13:36:49 -07:00
Zhijie Shen 68cddb894a YARN-3740. Fixed the typo in the configuration name: APPLICATION_HISTORY_PREFIX_MAX_APPS. Contributed by Xuan Gong. 
(cherry picked from commit eb6bf91eea)
 2015-05-29 18:18:47 -07:00
Robert Kanter bde4031a02 YARN-3713. Remove duplicate function call storeContainerDiagnostics in ContainerDiagnosticsUpdateTransition (zxu via rkanter) 
(cherry picked from commit 6aec13cb33)
 2015-05-29 15:35:08 -07:00
Karthik Kambatla b9c5c5de68 YARN-3547. FairScheduler: Apps that have no resource demand should not participate scheduling. (Xianyin Xin via kasha) 
(cherry picked from commit 3ae2a62501)
 2015-05-29 15:17:43 -07:00
Andrew Wang bdaa0fed13 HADOOP-11885. hadoop-dist dist-layout-stitching.sh does not work with dash. (wang) 
(cherry picked from commit 7673d4f205)

 Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml
	hadoop-mapreduce-project/pom.xml
 2015-05-29 14:55:52 -07:00
Wangda Tan 2fb794ecaf YARN-3716. Node-label-expression should be included by ResourceRequestPBImpl.toString. (Xianyin Xin via wangda) 
(cherry picked from commit 788bfa0359)
 2015-05-28 16:35:17 -07:00
Vinod Kumar Vavilapalli 148412bb30 Fixed more FilesSystemRMStateStore issues. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit 9acd24fec4)
 2015-05-28 15:27:01 -07:00
Xuan 550b55146d YARN-3723. Need to clearly document primaryFilter and otherInfo value 
type. Contributed by Zhijie Shen

(cherry picked from commit 3077c299da)
 2015-05-28 10:20:33 -07:00
Devaraj K 7116141d7d YARN-3722. Merge multiple TestWebAppUtils into 
o.a.h.yarn.webapp.util.TestWebAppUtils. Contributed by Masatake Iwasaki.

(cherry picked from commit 7e509f5843)
 2015-05-28 15:36:13 +05:30
Zhijie Shen 839f81a632 YARN-3700. Made generic history service load a number of latest applications according to the parameter or the configuration. Contributed by Xuan Gong. 
(cherry picked from commit 54504133f4)
 2015-05-27 16:53:47 -07:00
Wangda Tan 4790495538 YARN-3581. Deprecate -directlyAccessNodeLabelStore in RMAdminCLI. (Naganarasimha G R via wangda) 
(cherry picked from commit cab7674e54)
 2015-05-27 15:37:14 -07:00
cnauroth 6ed8989a6f YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch. 
(cherry picked from commit 7bc33b63c78c6ee9a04c0c1511ef16e63f7f3074)
 2015-05-27 14:21:15 -07:00
Wangda Tan 8857cba72e YARN-3647. RMWebServices api's should use updated api from CommonNodeLabelsManager to get NodeLabel object. (Sunil G via wangda) 
(cherry picked from commit ec0a852a37)
 2015-05-27 13:15:55 -07:00
Wangda Tan 0728a86b3a YARN-3686. CapacityScheduler should trim default_node_label_expression. (Sunil G via wangda) 
(cherry picked from commit cdbd66be11)
 2015-05-26 16:00:17 -07:00
Jian He 5e7be094ec YARN-3632. Ordering policy should be allowed to reorder an application when demand changes. Contributed by Craig Welch 
(cherry picked from commit 10732d515f)
 2015-05-26 12:02:36 -07:00
Vinod Kumar Vavilapalli a67cb4826b YARN-160. Enhanced NodeManager to automatically obtain cpu/memory values from underlying OS when configured to do so. Contributed by Varun Vasudev. 
(cherry picked from commit 500a1d9c76)
 2015-05-26 11:39:40 -07:00
Tsuyoshi Ozawa cf5c1adf4f YARN-2336. Fair scheduler's REST API returns a missing '[' bracket JSON for deep queue tree. Contributed by Kenji Kikushima and Akira Ajisaka. 
(cherry picked from commit 9a3d617b63)
 2015-05-26 19:08:01 +09:00
Xuan 84245ff3b2 YARN-2238. Filtering on UI sticks even if I move away from the page. 
Contributed by Jian He

(cherry picked from commit 39077dba2e)
 2015-05-25 22:41:02 -07:00
Jian He 03af057a47 YARN-3707. RM Web UI queue filter doesn't work. Contributed by Wangda Tan 
(cherry picked from commit 446d51591e)
 2015-05-22 15:59:00 -07:00
Xuan 33be070a5e YARN-3701. Isolating the error of generating a single app report when 
getting all apps from generic history service. Contributed by Zhijie
Shen

(cherry picked from commit 455b3acf0e)
 2015-05-22 14:35:51 -07:00
Vinayakumar B 19895a41ad HADOOP-11743. maven doesn't clean all the site files (Contributed by ramtin) 
(cherry picked from commit 55ed6558ba)

Conflicts:
	hadoop-common-project/hadoop-common/pom.xml
 2015-05-22 16:46:50 +05:30
Junping Du 5fb61d71b1 YARN-3594. WintuilsProcessStubExecutor.startStreamReader leaks streams. Contributed by Lars Francke. 
(cherry picked from commit 132d909d4a)
 2015-05-22 04:25:18 -07:00
Vinod Kumar Vavilapalli 673bd9708f YARN-3684. Changed ContainerExecutor's primary lifecycle methods to use a more extensible mechanism of context objects. Contributed by Sidharta Seethana. 
(cherry picked from commit 53fafcf061)
 2015-05-21 15:51:19 -07:00
Karthik Kambatla e8ac88d4fe YARN-3675. FairScheduler: RM quits when node removal races with continuous-scheduling on the same node. (Anubhav Dhoot via kasha) 
(cherry picked from commit a8b50e46737c11936ba72c427da69b2365a07aac)
 2015-05-21 13:39:40 -07:00
Devaraj K 157ecb2241 YARN-3646. Applications are getting stuck some times in case of retry 
policy forever. Contributed by Raju Bairishetti.

(cherry picked from commit 0305316d69)
 2015-05-21 20:16:09 +05:30
Akira Ajisaka 6d7e7ef1c4 YARN-3694. Fix dead link for TimelineServer REST API. Contributed by Jagadesh Kiran N. 
(cherry picked from commit a5def58087)
 2015-05-21 23:15:17 +09:00
Jian He 7c3895c8f6 YARN-3654. ContainerLogsPage web UI should not have meta-refresh. Contributed by Xuan Gong 
(cherry picked from commit 6329bd00fa)
 2015-05-20 17:21:19 -07:00
Jian He 521378ffa3 YARN-3609. Load node labels from storage inside RM serviceStart. Contributed by Wangda Tan 
(cherry picked from commit 8966d42179)
 2015-05-20 16:30:44 -07:00
Xuan a665b22cfa YARN-3681. yarn cmd says "could not find main class 'queue'" in windows. Contributed by Craig Welch and Varun Saxena 2015-05-20 14:42:59 -07:00
Wangda Tan d3193fd1d7 Move YARN-2918 from 2.8.0 to 2.7.1 
(cherry picked from commit 03f897fd1a)
 2015-05-20 13:50:11 -07:00
Wangda Tan b5ba6979b7 YARN-3583. Support of NodeLabel object instead of plain String in YarnClient side. (Sunil G via wangda) 
(cherry picked from commit 563eb1ad2a)
(cherry picked from commit b0d22b0c606fad6b4ab5443c0aed07c829b46726)
 2015-05-19 17:37:24 -07:00
Tsuyoshi Ozawa d022c5aba8 YARN-3677. Fix findbugs warnings in yarn-server-resourcemanager. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit 7401e5b5e8)
 2015-05-20 09:01:42 +09:00
Wangda Tan 7b97f049ee YARN-3565. NodeHeartbeatRequest/RegisterNodeManagerRequest should use NodeLabel object instead of String. (Naganarasimha G R via wangda) 
(cherry picked from commit b37da52a1c)
 2015-05-19 16:35:23 -07:00
Jian He adb90c7f52 YARN-2821. Fixed a problem that DistributedShell AM may hang if restarted. Contributed by Varun Vasudev 
(cherry picked from commit 7438966586)
 2015-05-19 14:21:48 -07:00
Colin Patrick Mccabe 9534e533dc HADOOP-11970. Replace uses of ThreadLocal<Random> with JDK7 ThreadLocalRandom (Sean Busbey via Colin P. McCabe) 
(cherry picked from commit 470c87dbc6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
        hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java
 2015-05-19 11:15:04 -07:00
Ravi Prakash 084e453629 YARN-3302. TestDockerContainerExecutor should run automatically if it can detect docker in the usual place (Ravindra Kumar Naik via raviprak) 
(cherry picked from commit c97f32e7b9)
 2015-05-19 10:30:24 -07:00
Xuan d39039d54d YARN-3601. Fix UT TestRMFailover.testRMWebAppRedirect. Contributed by Weiwei Yang 
(cherry picked from commit 5009ad4a7f)
 2015-05-19 09:57:13 -07:00
Xuan 3ceb2ffe54 YARN-3541. Add version info on timeline service / generic history web UI and REST API. Contributed by Zhijie Shen 
(cherry picked from commit 76afd28862)
 2015-05-18 13:19:01 -07:00
Xuan 802676e1be YARN-3526. ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster. Contributed by Weiwei Yang 
(cherry picked from commit b0ad644083)
 2015-05-15 22:41:13 -07:00
Jason Lowe ce64720516 YARN-2421. RM still allocates containers to an app in the FINISHING state. Contributed by Chang Li 
(cherry picked from commit f7e051c431)
 2015-05-15 22:10:16 +00:00
Junping Du cc671349bc YARN-3505 addendum: fix an issue in previous patch. 
(cherry picked from commit 03a293aed6)
 2015-05-15 06:40:28 -07:00
Vinod Kumar Vavilapalli ad3196e016 Fixing HDFS state-store. Contributed by Arun Suresh. 
(cherry picked from commit 9a2a9553ee)
 2015-05-14 16:57:03 -07:00
Ravi Prakash 4caadaa8bb YARN-1519. Check in container-executor if sysconf is implemented before using it (Radim Kolar and Eric Payne via raviprak) 
(cherry picked from commit 53fe4eff09)
 2015-05-14 15:56:36 -07:00
Junping Du bc13c7d84b YARN-3505. Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. 
(cherry picked from commit 15ccd967ee)
 2015-05-14 10:59:48 -07:00
Wangda Tan 3a11baa15f YARN-3362. Add node label usage in RM CapacityScheduler web UI. (Naganarasimha G R via wangda) 
(cherry picked from commit 0e85044e26)
 2015-05-13 17:01:23 -07:00
Jason Lowe a81ad81461 YARN-3641. NodeManager: stopRecoveryStore() shouldn't be skipped when exceptions happen in stopping NM's sub-services. Contributed by Junping Du 
(cherry picked from commit 711d77cc54)
 2015-05-13 21:08:45 +00:00
Jason Lowe aefc2bae9e Update fix version for YARN-3457 and YARN-3537. 
(cherry picked from commit f7de6198da)
 2015-05-13 20:54:21 +00:00
Wangda Tan d49262a172 YARN-3521. Support return structured NodeLabel objects in REST API (Sunil G via wangda) 
(cherry picked from commit 7f19e7a254)
 2015-05-13 13:43:57 -07:00
Wangda Tan 8555a5146d YARN-3579. CommonNodeLabelsManager should support NodeLabel instead of string label name when getting node-to-label/label-to-label mappings. (Sunil G via wangda) 
(cherry picked from commit d4f53fc963)
 2015-05-13 13:33:38 -07:00
Xuan 487d9b0f3f YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch 
(cherry picked from commit 0f95921447)
 2015-05-13 13:12:04 -07:00
Wangda Tan 3df8880bbe YARN-2921. Fix MockRM/MockAM#waitForState sleep too long. (Tsuyoshi Ozawa via wangda) 
(cherry picked from commit 341a476812)
 2015-05-13 13:07:05 -07:00
cnauroth ddc4afeff2 HADOOP-11966. Variable cygwin is undefined in hadoop-config.sh when executed through hadoop-daemon.sh. Contributed by Chris Nauroth. 2015-05-13 12:25:49 -07:00
Zhijie Shen 6043224458 YARN-3539. Updated timeline server documentation and marked REST APIs evolving. Contributed by Steve Loughran. 
(cherry picked from commit fcd0702c10)
 2015-05-12 21:13:47 -07:00
Karthik Kambatla 9d6c63f8be YARN-3613. TestContainerManagerSecurity should init and start Yarn cluster in setup instead of individual methods. (nijel via kasha) 
(cherry picked from commit fe0df59627)
 2015-05-12 10:45:51 -07:00
Devaraj K 5fceebd696 YARN-3629. NodeID is always printed as "null" in node manager 
initialization log. Contributed by nijel.

(cherry picked from commit 5c2f05cd9b)
 2015-05-12 22:22:40 +05:30
Devaraj K f887243f88 YARN-3513. Remove unused variables in ContainersMonitorImpl and add debug 
log for overall resource usage by all containers.  Contributed by
Naganarasimha G R.

(cherry picked from commit 8badd82ce2)
 2015-05-12 16:56:26 +05:30
Wangda Tan 12584ac10e Move YARN-3493 from 2.8 to 2.7.1 in CHANGES.txt 2015-05-11 18:08:25 -07:00
Wangda Tan d06d2a96f5 YARN-3489. RMServerUtils.validateResourceRequests should only obtain queue info once. (Varun Saxena via wangda) 
(cherry picked from commit d6f6741296)
 2015-05-11 17:35:01 -07:00
Wangda Tan 2c80a3fe19 Moved YARN-3434. (Interaction between reservations and userlimit can result in significant ULF violation.) From 2.8.0 to 2.7.1 
(cherry picked from commit 1952f88889)
 2015-05-11 15:22:25 -07:00
Junping Du a1a7deebf8 YARN-3587. Fix the javadoc of DelegationTokenSecretManager in yarn, etc. projects. Contributed by Gabor Liptak. 
(cherry picked from commit 7e543c27fa)
 2015-05-11 05:28:25 -07:00
Karthik Kambatla 3cf4f24a9f YARN-3395. FairScheduler: Trim whitespaces when using username for queuename. (Zhihai Xu via kasha) 
(cherry picked from commit a60f78e98e)
 2015-05-09 15:41:49 -07:00
Karthik Kambatla e8dd0bc4fd YARN-1287. Consolidate MockClocks. (Sebastian Wong and Anubhav Dhoot via kasha) 
(cherry picked from commit 70fb37cd79)
 2015-05-09 14:35:13 -07:00
Xuan acb342b77c YARN-1912. ResourceLocalizer started without any jvm memory control. 
Contributed by Masatake Iwasaki

(cherry picked from commit 6471d18bc7)
 2015-05-08 20:02:23 -07:00
Xuan fc980247a6 YARN-3602. TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IOException from cleanup. Contributed by zhihai xu 
(cherry picked from commit 333f9a896d)
 2015-05-08 17:54:46 -07:00
Zhijie Shen ef096599c3 YARN-2206. Updated document for applications REST API response examples. Contributed by Kenji Kikushima and Brahma Reddy Battula. 
(cherry picked from commit 08f0ae403c)
 2015-05-08 16:41:29 -07:00
Karthik Kambatla a7d8f21630 YARN-3271. FairScheduler: Move tests related to max-runnable-apps from TestFairScheduler to TestAppRunnability. (nijel via kasha) 
(cherry picked from commit 2fb44c8aaf)
 2015-05-08 16:40:36 -07:00
Karthik Kambatla 21a07769d4 YARN-1050. Document the Fair Scheduler REST API. (Kenji Kikushima and Roman Shaposhnik via kasha) 
(cherry picked from commit 96473bdc2b)
 2015-05-08 16:31:30 -07:00
Robert Kanter 453488cff2 YARN-3473. Fix RM Web UI configuration for some properties (rchiang via rkanter) 
(cherry picked from commit 5658998845)
 2015-05-08 16:23:20 -07:00
Jason Lowe 743ca1a056 YARN-3476. Nodemanager can fail to delete local logs if log aggregation fails. Contributed by Rohith 
(cherry picked from commit 25e2b02122)
 2015-05-08 22:46:34 +00:00
Jian He 5a1c2fed34 YARN-3604. Fixed ZKRMStateStore#removeApplication to also disable watch. Contributed Zhihai Xu 
(cherry picked from commit d0b73287c2)
 2015-05-08 15:13:43 -07:00
Xuan 8d07116266 YARN-2331. Distinguish shutdown during supervision vs. shutdown for 
rolling upgrade. Contributed by Jason Lowe

(cherry picked from commit 088156de43)
 2015-05-08 15:11:55 -07:00
Vinod Kumar Vavilapalli 24c8d3e54b YARN-3018. Unified the default value for the configuration property yarn.scheduler.capacity.node-locality-delay in code and default xml file. Contributed by Nijel SF. 
(cherry picked from commit f7a74d2491)
 2015-05-08 14:45:55 -07:00
Wangda Tan f8807839c6 YARN-3593. Add label-type and Improve "DEFAULT_PARTITION" in Node Labels Page. (Naganarasimha G R via wangda) 
(cherry picked from commit f55d21d894)
 2015-05-08 12:05:25 -07:00
Thomas Graves 638feaaa35 YARN-20. More information for yarn.resourcemanager.webapp.address in yarn-default.xml (Bartosz Ługowski vai tgraves) 
(cherry picked from commit f0f5e3c075)
 2015-05-08 17:22:33 +00:00
Thomas Graves 547b069887 YARN-3600. AM container link is broken (Naganarasimha G R via tgraves 
(cherry picked from commit 5d708a4725)
 2015-05-08 16:37:20 +00:00
Robert (Bobby) Evans 28e0593b96 YARN-644: Basic null check is not performed on passed in arguments before using them in ContainerManagerImpl.startContainer 
(cherry picked from commit bcf2890502)
 2015-05-08 11:11:53 -05:00
Devaraj K bf8be56292 YARN-2784. Make POM project names consistent. Contributed by Rohith. 2015-05-08 19:26:11 +05:30
Jason Lowe bcb863b156 YARN-3554. Default value for maximum nodemanager connect wait time is too high. Contributed by Naganarasimha G R 
(cherry picked from commit 9757864fd6)
 2015-05-08 13:47:48 +00:00
Tsuyoshi Ozawa 5ddc3adb0d YARN-3169. Drop YARN's overview document. Contributed by Brahma Reddy Battula. 
(cherry picked from commit b419c1b2ec)
 2015-05-08 19:58:47 +09:00
Tsuyoshi Ozawa b1f3590b22 YARN-3589. RM and AH web UI display DOCTYPE wrongly. Contbituted by Rohith. 
(cherry picked from commit f26700f287)
 2015-05-08 17:13:56 +09:00
Tsuyoshi Ozawa 1ae8bf5b21 MAPREDUCE-6284. Add Task Attempt State API to MapReduce Application Master REST API. Contributed by Ryu Kobayashi. 
(cherry picked from commit d18f10ad1b)
 2015-05-08 16:00:41 +09:00
Devaraj K 0d286d1041 YARN-3358. Audit log not present while refreshing Service ACLs. 
Contributed by Varun Saxena.

(cherry picked from commit ef3d66d462)
 2015-05-08 12:15:45 +05:30
Devaraj K dce2381dc4 YARN-3592. Fix typos in RMNodeLabelsManager. Contributed by Sunil G. 
(cherry picked from commit a28cd02ba0)
 2015-05-08 10:55:53 +05:30
Akira Ajisaka c7f3dd1eb3 YARN-3572. Correct typos in WritingYarnApplications.md. Contributed by Gabor Liptak. 
(cherry picked from commit a521b50955)
 2015-05-08 13:25:07 +09:00
Akira Ajisaka 170f1b0afd YARN-1832. Fix wrong MockLocalizerStatus#equals implementation. Contributed by Hong Zhiguo. 
(cherry picked from commit b167fe7605)
 2015-05-08 11:15:28 +09:00
Jian He d817fbb34d YARN-2918. RM should not fail on startup if queue's configured labels do not exist in cluster-node-labels. Contributed by Wangda Tan 
(cherry picked from commit f489a4ec96)
 2015-05-07 17:36:24 -07:00
Jian He c35114a1e6 YARN-3584. Fixed attempt diagnostics format shown on the UI. Contributed by nijel 
(cherry picked from commit b88700dcd0)
 2015-05-07 13:56:25 -07:00
Zhijie Shen bb035ff087 YARN-3448. Added a rolling time-to-live LevelDB timeline store implementation. Contributed by Jonathan Eagles. 
(cherry picked from commit daf3e4ef8b)
 2015-05-07 10:10:13 -07:00
Junping Du 684a5a6aeb YARN-3523. Cleanup ResourceManagerAdministrationProtocol interface audience. Contributed by Naganarasimha G R 
(cherry picked from commit 8e991f4b1d)
 2015-05-07 05:40:13 -07:00
Akira Ajisaka 67bebabb21 YARN-3577. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 995f1b5831fc33042de90267a689333dc110c4c3)
 2015-05-07 13:33:17 +09:00
Vinod Kumar Vavilapalli d127dd64a7 YARN-3385. Fixed a race-condition in ResourceManager's ZooKeeper based state-store to avoid crashing on duplicate deletes. Contributed by Zhihai Xu. 
(cherry picked from commit 4c7b9b6abe)
 2015-05-06 17:52:13 -07:00
Junping Du 79babc4bd2 YARN-3580. [JDK8] TestClientRMService.testGetLabelsToNodes fails. Contributed by Robert Kanter. 
(cherry picked from commit 31b627b2a8)
 2015-05-06 16:51:56 -07:00
Robert Kanter 28b40aca98 YARN-3491. PublicLocalizer#addResource is too slow. (zxu via rkanter) 
(cherry picked from commit b72507810a)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-05-06 14:20:08 -07:00
Jian He f8488b5aeb YARN-3301. Fixed the format issue of the new RM attempt web page. Contributed by Xuan Gong 
(cherry picked from commit 0d3188fd25)
 2015-05-06 13:51:44 -07:00
Vinod Kumar Vavilapalli da410ea02a YARN-3243. Moving CHANGES.txt entry to the right release. 
(cherry picked from commit 185e63a726)
 2015-05-06 10:44:19 -07:00
Xuan a0445eae5c YARN-3582. NPE in WebAppProxyServlet. Contributed by Jian He 
(cherry picked from commit a583a40693)
 2015-05-05 21:43:35 -07:00
Jian He eb0c6d2ee7 YARN-3343. Increased TestCapacitySchedulerNodeLabelUpdate#testNodeUpdate timeout. Contributed by Rohith Sharmaks 
(cherry picked from commit e4c3b52c89)
 2015-05-05 11:34:11 -07:00
Xuan 02cdcaec58 YARN-2123. Progress bars in Web UI always at 100% due to non-US locale. 
Contributed by Akira AJISAKA

(cherry picked from commit b7dd3a4f04)
 2015-05-05 10:45:18 -07:00
Junping Du 6ea007ba91 YARN-3396. Handle URISyntaxException in ResourceLocalizationService. (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 3810242062)
 2015-05-05 10:19:16 -07:00
Jason Lowe 0f30913ea8 YARN-3552. RM Web UI shows -1 running containers for completed apps. Contributed by Rohith 
(cherry picked from commit 9356cf8676)
 2015-05-05 13:43:57 +00:00
Jian He f71c49c5e3 YARN-2725. Added test cases of retrying creating znode in ZKRMStateStore. Contributed by Tsuyoshi Ozawa 
(cherry picked from commit d701acc9c6)
 2015-05-04 16:17:01 -07:00
Wangda Tan 4e77ee2f6f YARN-3375. NodeHealthScriptRunner.shouldRun() check is performing 3 times for starting NodeHealthScriptRunner (Devaraj K via wangda) 
(cherry picked from commit 71f4de220c)
 2015-05-04 16:02:57 -07:00
Allen Wittenauer 02e650248d YARN-2980. Move health check script related functionality to hadoop-common (Varun Saxena via aw) 
(cherry picked from commit d4ac6822e1)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
 2015-05-04 16:00:20 -07:00
Jason Lowe 99b49a8181 YARN-3097. Logging of resource recovery on NM restart has redundancies. Contributed by Eric Payne 
(cherry picked from commit 8f65c793f2)
 2015-05-04 10:32:33 -05:00
Tsuyoshi Ozawa f382602de1 YARN-1993. Cross-site scripting vulnerability in TextView.java. Contributed byKenji Kikushima. 
(cherry picked from commit e8d0ee5fc9)
 2015-05-03 10:51:39 +09:00
Junping Du 8068e79989 YARN-2454. Fix compareTo of variable UNBOUNDED in o.a.h.y.util.resource.Resources. Contributed by Xu Yang. 
(cherry picked from commit 57d9a972cb)
 2015-05-02 10:20:30 -07:00
Gera Shegalov 9325dd863c YARN-2893. AMLaucher: sporadic job failures due to EOFException in readTokenStorageStream. (Zhihai Xu via gera) 
(cherry picked from commit f8204e241d)
 2015-05-01 19:04:48 -07:00
Wangda Tan 4151e5729a YARN-3006. Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda) 
(cherry picked from commit 7d46a806e7)
 2015-05-01 16:44:49 -07:00
Robert Kanter 32b3b8e2ef YARN-3363. add localization and container launch time to ContainerMetrics at NM to show these timing information for each active container. (zxu via rkanter) 
(cherry picked from commit ac7d152901)
 2015-05-01 16:39:59 -07:00
Vinod Kumar Vavilapalli 90f6e8c90f YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. 
(cherry picked from commit 1b3b9e5c31)
 2015-04-30 21:43:22 -07:00
Zhijie Shen c9ee316045 YARN-3544. Got back AM logs link on the RM web UI for a completed app. Contributed by Xuan Gong. 
(cherry picked from commit 21bf2cdcb77f69abc906e6cd401a8fb221f250e9)
 2015-04-30 13:06:46 -07:00
Wangda Tan 100b41add6 YARN-3564. Fix TestContainerAllocation.testAMContainerAllocationWhenDNSUnavailable fails randomly. (Jian He via wangda) 
(cherry picked from commit e2e8f77118)
 2015-04-30 11:06:47 -07:00
Jian He a7a5737bc4 YARN-3533. Test: Fix launchAM in MockRM to wait for attempt to be scheduled. Contributed by Anubhav Dhoot 
(cherry picked from commit 4c1af156ae)
 2015-04-29 14:51:04 -07:00
tgraves 2e13183f60 YARN-3517. RM web ui for dumping scheduler logs should be for admins only (Varun Vasudev via tgraves) 
(cherry picked from commit 2e215484bd)
 2015-04-29 21:27:16 +00:00
Karthik Kambatla 4cc38df7de YARN-3485. FairScheduler headroom calculation doesn't consider maxResources for Fifo and FairShare policies. (kasha) 
(cherry picked from commit 8f82970e0c)
 2015-04-28 21:03:12 -07:00
Wangda Tan 8ee632caa7 YARN-2740. Fix NodeLabelsManager to properly handle node label modifications when distributed node label configuration enabled. (Naganarasimha G R via wangda) 
(cherry picked from commit db1b674b50)
 2015-04-27 16:42:36 -07:00
Karthik Kambatla d2d8bc243b Update CHANGES.txt - Pulled in YARN-3465, YARN-3516, and YARN-3464 to branch-2.7 (for 2.7.1) 
(cherry picked from commit 32cd2c8d42)
 2015-04-27 13:44:06 -07:00
Xuan 0e8af401d7 YARN-3530. ATS throws exception on trying to filter results without 
otherinfo. Contributed by zhijie shen

(cherry picked from commit 7f07c4d810)
 2015-04-27 10:37:40 -07:00
Karthik Kambatla 4045c41afe YARN-3464. Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) 
(cherry picked from commit 47279c3228)
 2015-04-26 09:14:12 -07:00
Vinod Kumar Vavilapalli db1a33e956 Moving YARN-3351, YARN-3382, YARN-3472, MAPREDUCE-6238 to the 2.7.1 CHANGES.txt 
sections given the recent merge into branch-2.7.

(cherry picked from commit 2f82ae042a)
 2015-04-24 17:20:02 -07:00
Jian He 9bf09b334d YARN-2498. Respect labels in preemption policy of capacity scheduler for inter-queue preemption. Contributed by Wangda Tan 
(cherry picked from commit d497f6ea2b)
 2015-04-24 17:03:57 -07:00
Tsuyoshi Ozawa 3884948d6c YARN-3406. Display count of running containers in the RM's Web UI. Contributed by Ryu Kobayashi. 
(cherry picked from commit 4a3dabd94f)
 2015-04-25 07:17:42 +09:00
Jason Lowe bd750f160b Fix commit version for YARN-3537 
(cherry picked from commit 78fe6e57c7)
 2015-04-24 22:09:01 +00:00
Jason Lowe cf4154676b YARN-3537. NPE when NodeManager.serviceInit fails and stopRecoveryStore invoked. Contributed by Brahma Reddy Battula 
(cherry picked from commit 5e093f0d40)
 2015-04-24 22:04:03 +00:00
Akira Ajisaka 73ba3ebe7c YARN-3444. Fix typo capabililty. Contributed by Gabor Liptak. 
(cherry picked from commit 5ce3a77f3c)
 2015-04-25 06:09:03 +09:00
Jian He 0583c27fb1 YARN-3387. Previous AM's container completed status couldn't pass to current AM if AM and RM restarted during the same time. Contributed by Sandflee 
(cherry picked from commit d03dcb9635)
 2015-04-24 12:14:17 -07:00
Xuan c184466939 YARN-3511. Add errors and warnings page to ATS. Contributed by Varun Vasudev 
(cherry picked from commit eee9facbba)
 2015-04-24 09:43:35 -07:00
Xuan 8f6053ae51 YARN-3516. killing ContainerLocalizer action doesn't take effect when 
private localizer receives FETCH_FAILURE status. Contributed by zhihai
xu

(cherry picked from commit 0b3f8957a8)
 2015-04-23 16:41:52 -07:00
Vinod Kumar Vavilapalli 0ec6e7e7f5 YARN-3413. Changed Nodelabel attributes (like exclusivity) to be settable only via addToClusterNodeLabels but not changeable at runtime. (Wangda Tan via vinodkv) 
(cherry picked from commit f5fe35e297)
 2015-04-23 11:25:30 -07:00
Jian He 48d8d88245 YARN-3522. Fixed DistributedShell to instantiate TimeLineClient as the correct user. Contributed by Zhijie Shen 
(cherry picked from commit aa4a192feb)
 2015-04-23 11:07:54 -07:00
Wangda Tan 9692bcc2c8 YARN-3319. Implement a FairOrderingPolicy. (Craig Welch via wangda) 
(cherry picked from commit 395205444e)
 2015-04-23 10:48:00 -07:00
tgraves 1cd2fcf25d YARN-3434. Interaction between reservations and userlimit can result in significant ULF violation 
(cherry picked from commit 189a63a719)
 2015-04-23 14:49:24 +00:00
Steve Loughran 63ccb9167e YARN-2605. [RM HA] Rest api endpoints doing redirect incorrectly. (Xuan Gong via stevel) 2015-04-23 09:22:23 +01:00
Vinod Kumar Vavilapalli 04783b0402 YARN-3366. Enhanced NodeManager to support classifying/shaping outgoing network bandwidth traffic originating from YARN containers Contributed by Sidharta Seethana. 
(cherry picked from commit a100be685c)
 2015-04-22 17:36:55 -07:00
Junping Du 8de6045842 New parameter of CLI for decommissioning node gracefully in RMAdmin CLI. Contributed by Devaraj K 
(cherry picked from commit fad9d7e85b)
 2015-04-22 10:08:16 -07:00
Akira Ajisaka f469f9af2d YARN-3410. Addendum fix for compilation error. Contributed by Rohith. 
(cherry picked from commit b08908ae5e)
 2015-04-22 17:48:55 +09:00
Jian He 613a783380 YARN-3503. Expose disk utilization percentage and bad local and log dir counts in NM metrics. Contributed by Varun Vasudev 
(cherry picked from commit 674c7ef649)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
 2015-04-21 21:06:06 -07:00
Jian He dc4698bb33 YARN-3494. Expose AM resource limit and usage in CS QueueMetrics. Contributed by Rohith Sharmaks 
(cherry picked from commit bdd90110e6)
 2015-04-21 20:34:30 -07:00
Wangda Tan 8a9599fb9e YARN-3410. YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) 2015-04-21 17:47:27 -07:00
Tsuyoshi Ozawa 0b87ae27aa YARN-3495. Confusing log generated by FairScheduler. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 105afd5477)
 2015-04-22 05:48:16 +09:00
Vinod Kumar Vavilapalli 27fc4fb998 Set the release date for 2.7.0 
(cherry picked from commit d52de61544)
 2015-04-20 20:18:01 -07:00
Wangda Tan d0ea982e64 YARN-3463. Integrate OrderingPolicy Framework with CapacityScheduler. (Craig Welch via wangda) 
(cherry picked from commit 44872b76fc)
 2015-04-20 17:13:01 -07:00
Jian He 6b454aacc3 YARN-3497. ContainerManagementProtocolProxy modifies IPC timeout conf without making a copy. Contributed by Jason Lowe 
(cherry picked from commit f967fd2f21)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-04-20 10:40:09 -07:00
Jian He 6f70f1b1af YARN-3136. Fixed a synchronization problem of AbstractYarnScheduler#getTransferredContainers. Contributed by Sunil G 
(cherry picked from commit 497c86b485)
 2015-04-18 12:47:30 -07:00
Wangda Tan 3316cd4357 YARN-3487. CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) 
(cherry picked from commit f47a5763ac)
 2015-04-17 17:23:00 -07:00
Wangda Tan e7cbecddc3 YARN-3493. RM fails to come up with error "Failed to load/recover state" when mem settings are changed. (Jian He via wangda) 
(cherry picked from commit f65eeb412d)
 2015-04-17 17:15:16 -07:00
Jian He 9827da95bc YARN-3451. Display attempt start time and elapsed time on the web UI. Contributed by Rohith Sharmaks 
(cherry picked from commit 6779467ab6)
 2015-04-17 15:57:36 -07:00
Jian He 57eb07d34b YARN-2696. Queue sorting in CapacityScheduler should consider node label. Contributed by Wangda Tan 
(cherry picked from commit d573f09fb9)
 2015-04-17 13:37:34 -07:00
Junping Du b244701fad YARN-1402. Update related Web UI and CLI with exposing client API to check log aggregation status. Contributed by Xuan Gong. 
(cherry picked from commit 1db355a875)
 2015-04-17 13:20:27 -07:00
Jian He 42cf8e0554 YARN-3021. YARN's delegation-token handling disallows certain trust setups to operate properly over DistCp. Contributed by Yongjun Zhang 
(cherry picked from commit bb6dde68f1)
 2015-04-16 19:44:32 -07:00
Akira Ajisaka 51c458a5c4 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 1fa8075794)
 2015-04-16 10:06:03 -05:00
Jian He 6fed2c2a79 YARN-3354. Add node label expression in ContainerTokenIdentifier to support RM recovery. Contributed by Wangda Tan 
(cherry picked from commit 1b89a3e173)
 2015-04-15 14:03:29 -07:00
Jian He 58f99b740a YARN-3404. Display queue name on application page. Contributed by Ryu Kobayashi 
(cherry picked from commit b2e6cf607f)
 2015-04-15 13:53:24 -07:00
Tsuyoshi Ozawa e977247f3e YARN-3326. Support RESTful API for getLabelsToNodes. Contributed by Naganarasimha G R. 
(cherry picked from commit e48cedc663)
 2015-04-15 14:04:24 -05:00
Wangda Tan 89a7c98436 YARN-3318. Create Initial OrderingPolicy Framework and FifoOrderingPolicy. (Craig Welch via wangda) 
(cherry picked from commit 5004e75332)
 2015-04-15 09:57:52 -07:00
Harsh J 01af29106a YARN-3462. Patches applied for YARN-2424 are inconsistent between trunk and branch-2. Contributed by Naganarasimha G R. 2015-04-15 09:41:43 +05:30
Tsuyoshi Ozawa 3daca4f040 YARN-3436. Fix URIs in documantion of YARN web service REST APIs. Contributed by Bibin A Chundatt. 
(cherry picked from commit 05007b45e5)
 2015-04-14 14:14:17 -05:00
Jian He 9ebbf1bfce YARN-3361. CapacityScheduler side changes to support non-exclusive node labels. Contributed by Wangda Tan 
(cherry picked from commit 0fefda645b)
 2015-04-14 11:46:35 -07:00
Jian He 81bbee6852 YARN-3266. RMContext#inactiveNodes should have NodeId as map key. Contributed by Chengbing Liu 
(cherry picked from commit b46ee1e7a3)
 2015-04-14 11:31:42 -07:00
Junping Du 7c072bf092 YARN-3443. Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. 
(cherry picked from commit 838b06ac87)
 2015-04-13 18:37:39 -07:00
Junping Du a64b8915de YARN-3347. Improve YARN log command to get AMContainer logs as well as running containers logs. Contributed by Xuan Gong. 
(cherry picked from commit a77d628339)
 2015-04-13 17:41:02 -07:00
Jian He 1afea8a15a YARN-3394. Enrich WebApplication proxy documentation. Contributed by Naganarasimha G R 
(cherry picked from commit 7b4671490a)
 2015-04-13 17:05:58 -07:00
Jian He d58f5c8894 YARN-3472. Fixed possible leak in DelegationTokenRenewer#allTokens. Contributed by Rohith Sharmaks 
(cherry picked from commit a1afbc48b5)
 2015-04-13 14:07:51 -07:00
Karthik Kambatla fa12700379 Fix up author name to Jun Gong in CHANGES.txt for YARN-3469 
(cherry picked from commit 5cda6fffd3)
 2015-04-13 13:52:44 -07:00
Jian He 32b01263d3 YARN-3382. Some of UserMetricsInfo metrics are incorrectly set to root queue metrics. Contributed by Rohit Agarwal 
(cherry picked from commit 944a16579f)
 2015-04-13 10:54:15 -07:00
Karthik Kambatla ad99d268e5 YARN-3469. ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) 
(cherry picked from commit e516706b89)
 2015-04-10 11:21:30 -07:00
Jian He d3daf9665c YARN-3348. Add a 'yarn top' tool to help understand cluster usage. Contributed by Varun Vasudev 2015-04-10 10:40:31 -07:00
Junping Du 0891de0712 YARN-1376. NM need to notify the log aggregation status to RM through Node heartbeat. Contributed by Xuan Gong. 
(cherry picked from commit 92431c9617)
 2015-04-10 08:58:07 -07:00
Xuan f5b49160d9 YARN-3293. Track and display capacity scheduler health metrics in web 
UI. Contributed by Varun Vasudev

(cherry picked from commit afa5d4715a)
 2015-04-09 23:40:02 -07:00
Vinod Kumar Vavilapalli aec1c5ec7d Adding release 2.7.1 to CHANGES.txt 
(cherry picked from commit 623fd46c1e)
 2015-04-09 13:56:14 -07:00
Vinod Kumar Vavilapalli 1ff3fd33ed YARN-3055. Fixed ResourceManager's DelegationTokenRenewer to not stop token renewal of applications part of a bigger workflow. Contributed by Daryn Sharp. 
(cherry picked from commit 9c5911294e)
 2015-04-09 13:10:06 -07:00
Wangda Tan 73d9e4c77b YARN-3466. Fix RM nodes web page to sort by node HTTP-address, #containers and node-label column. (Jason Lowe via wangda) 
(cherry picked from commit 1885141e90)
 2015-04-09 10:36:10 -07:00
Karthik Kambatla 53e0bf5c17 YARN-3465. Use LinkedHashMap to preserve order of resource requests. (Zhihai Xu via kasha) 
(cherry picked from commit 6495940eae)
 2015-04-09 00:10:23 -07:00
Hitesh Shah 55b794e7fa YARN-2890. MiniYarnCluster should turn on timeline service if configured to do so. Contributed by Mit Desai. 
(cherry picked from commit 265ed1fe80)
 2015-04-08 14:14:06 -07:00
Wangda Tan 5f59e621ba YARN-2901 addendum: Fixed findbugs warning caused by previously patch 
(cherry picked from commit ba9ee22ca4)
 2015-04-08 11:02:54 -07:00
Wangda Tan 97244b143c YARN-3459. Fix failiure of TestLog4jWarningErrorMetricsAppender. (Varun Vasudev via wangda) 
(cherry picked from commit 7af086a515)
 2015-04-08 10:58:38 -07:00
Tsuyoshi Ozawa ac32fa187c YARN-3457. NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. 
(cherry picked from commit dd852f5b8c)
 2015-04-08 15:56:33 +09:00
Robert Kanter b584ce6c0a YARN-3429. Fix incorrect CHANGES.txt 
(cherry picked from commit 5b8a3ae366)
 2015-04-07 16:16:08 -07:00
Xuan 0522d6970d YARN-3294. Allow dumping of Capacity Scheduler debug logs via web UI for 
a fixed time period. Contributed by Varun Vasudev

(cherry picked from commit d27e9241e8)
 2015-04-07 09:54:29 -07:00
Xuan 7941cc554b YARN-3110. Few issues in ApplicationHistory web ui. Contributed by Naganarasimha G R 
(cherry picked from commit 19a4feaf6f)
 2015-04-07 08:24:09 -07:00
Zhijie Shen 735e4f15ab Move YARN-3273 from 2.8 to 2.7. 
(cherry picked from commit 5e9f4099de4b949c838a6224442f335c67d80936)
 2015-04-06 14:15:23 -07:00
Robert Kanter 354d2c75cf YARN-2429. TestAMRMTokens.testTokenExpiry fails Intermittently with error message:Invalid AMRMToken (zxu via rkanter) 
(cherry picked from commit 99b08a748e)
 2015-04-06 14:11:50 -07:00
Tsuyoshi Ozawa d2b2d76cce TestFairScheduler.testContinuousScheduling fails Intermittently. Contributed by Zhihai Xu. 
(cherry picked from commit 53959e69f7)
 2015-04-06 20:19:47 +09:00
Xuan 1eb6d74967 YARN-3435. AM container to be allocated Appattempt AM container shown as 
null. Contributed by Bibin A Chundatt

(cherry picked from commit 96d72118f5)
 2015-04-05 00:43:22 -07:00
Wangda Tan 405aead879 YARN-2901. Add errors and warning metrics page to RM, NM web UI. (Varun Vasudev via wangda) 
(cherry picked from commit bad070fe15)
 2015-04-02 17:23:44 -07:00
Vinod Kumar Vavilapalli d8e17c58bc YARN-3365. Enhanced NodeManager to support using the 'tc' tool via container-executor for outbound network traffic control. Contributed by Sidharta Seethana. 
(cherry picked from commit b21c72777a)
 2015-04-02 16:55:00 -07:00
Sandy Ryza 6286cfd711 YARN-3415. Non-AM containers can be counted towards amResourceUsage of a fairscheduler queue (Zhihai Xu via Sandy Ryza) 2015-04-02 13:57:53 -07:00
Zhijie Shen 6d0bebc912 YARN-3430. Made headroom data available on app attempt page of RM WebUI. Contributed by Xuan Gong. 
(cherry picked from commit 8366a36ad3)
 2015-04-01 13:49:00 -07:00
Wangda Tan 865be70b02 YARN-3425. NPE from RMNodeLabelsManager.serviceStop when NodeLabelsManager.serviceInit failed. (Bibin A Chundatt via wangda) 
(cherry picked from commit 492239424a)
 2015-04-01 10:15:47 -07:00
Xuan 5e4d457674 YARN-3248. Correct fix version from branch-2.7 to branch-2.8 in the change log. 
(cherry picked from commit 2e79f1c212)
 2015-04-01 04:33:32 -07:00
Xuan e26b6e55e9 YARN-3248. Display count of nodes blacklisted by apps in the web UI. 
Contributed by Varun Vasudev

(cherry picked from commit 4728bdfa15)
 2015-04-01 04:20:37 -07:00
Tsuyoshi Ozawa 80755edb7a YARN-3424. Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. 
(cherry picked from commit c69ba81497)
 2015-04-01 17:44:40 +09:00
Karthik Kambatla 93b29d962d YARN-3428. Debug log resources to be localized for a container. (kasha) 
(cherry picked from commit 2daa478a64)
 2015-03-31 17:35:11 -07:00
Vinod Kumar Vavilapalli c980e34bc5 YARN-3304. Addendum patch. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. (Junping Du and Karthik Kambatla via vinodkv) 
(cherry picked from commit 7610925e90)
 2015-03-31 17:28:48 -07:00
Karthik Kambatla 85c420a4d4 YARN-3412. RM tests should use MockRM where possible. (kasha) 
(cherry picked from commit 79f7f2aabf)
 2015-03-31 09:22:16 -07:00
Tsuyoshi Ozawa 99b8255693 YARN-3258. FairScheduler: Need to add more logging to investigate allocations. Contributed by Anubhav Dhoot. 
(cherry picked from commit b5a22e9838)
 2015-03-31 17:43:07 +09:00
Wangda Tan cba4ed1678 YARN-2495. Allow admin specify labels from each NM (Distributed configuration for node label). (Naganarasimha G R via wangda) 
(cherry picked from commit 2a945d24f7)
 2015-03-30 12:05:54 -07:00
Vinod Kumar Vavilapalli c5bc48946d YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla. 
(cherry picked from commit c358368f51)
 2015-03-30 10:11:12 -07:00
Ravi Prakash b1b4951452 YARN-3288. Document and fix indentation in the DockerContainerExecutor code 
(cherry picked from commit e0ccea33c9)
 2015-03-28 08:01:26 -07:00
Robert Kanter 6f0e864d61 YARN-3400. [JDK 8] Build Failure due to unreported exceptions in RPCUtil (rkanter) 
(cherry picked from commit 87130bf6b2)
 2015-03-26 11:00:37 -07:00
Haohui Mai 17f17dac4e HADOOP-10670. Allow AuthenticationFilters to load secret from signature secret files. Contributed by Kai Zheng. 2015-03-25 11:13:04 -07:00
Karthik Kambatla 2158cd2bef YARN-3397. yarn rmadmin should skip -failover. (J.Andreina via kasha) 
(cherry picked from commit c906a1de72)
 2015-03-25 07:42:45 -07:00
Xuan 0c4fa872be YARN-2213. Change proxy-user cookie log in AmIpFilter to DEBUG. 
Contributed by Varun Saxena

(cherry picked from commit e556198e71)
 2015-03-25 04:51:55 -07:00
Wangda Tan 29629020c1 YARN-3383. AdminService should use warn instead of info to log exception when operation fails. (Li Lu via wangda) 
(cherry picked from commit 97a7277a2d)
 2015-03-24 10:39:45 -07:00
Harsh J e7702ac6a1 YARN-1880. Cleanup TestApplicationClientProtocolOnHA. Contributed by ozawa. 
(cherry picked from commit 8a9bbf548dad82598354c24fcd4dc80a715da7ac)
 2015-03-24 11:58:16 +05:30
Xuan cbdcdfad6d YARN-3393. Getting application(s) goes wrong when app finishes before 
starting the attempt. Contributed by Zhijie Shen

(cherry picked from commit 9fae455e26)
 2015-03-23 20:34:29 -07:00
Karthik Kambatla 4e0c48703e YARN-2868. FairScheduler: Metric for latency to allocate first container for an application. (Ray Chiang via kasha) 
(cherry picked from commit 972f1f1ab9)
 2015-03-23 14:10:23 -07:00
Karthik Kambatla 75591e4131 YARN-3241. FairScheduler handles invalid queue names inconsistently. (Zhihai Xu via kasha) 
(cherry picked from commit 2bc097cd14)
 2015-03-23 13:24:22 -07:00
cnauroth 342c525eaa YARN-3336. FileSystem memory leak in DelegationTokenRenewer. 
(cherry picked from commit 6ca1f12024)
 2015-03-23 10:46:06 -07:00
Tsuyoshi Ozawa cbacf20755 YARN-3384. TestLogAggregationService.verifyContainerLogs fails after YARN-2777. Contributed by Naganarasimha G R. 
(cherry picked from commit 82eda771e0)
 2015-03-24 00:25:52 +09:00
Jian He 9c494cedaf YARN-3345. Add non-exclusive node label API. Contributed by Wangda Tan 
(cherry picked from commit e1feb4ea1a)
 2015-03-20 19:05:14 -07:00
Junping Du 9aedb3d688 YARN-3350. YARN RackResolver spams logs with messages at info level. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 7f1e2f9969)
 2015-03-20 18:22:23 -07:00
Jian He a3a155a34e YARN-3356. Capacity Scheduler FiCaSchedulerApp should use ResourceUsage to track used-resources-by-label. Contributed by Wangda Tan 
(cherry picked from commit 586348e4cb)
 2015-03-20 13:54:46 -07:00
Junping Du f40f17489c YARN-3269. Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path. Contributed by Xuan Gong 
(cherry picked from commit d81109e588)
 2015-03-20 13:42:31 -07:00
Wangda Tan 8e142d27cb YARN-3369. Missing NullPointer check in AppSchedulingInfo causes RM to die. (Brahma Reddy Battula via wangda) 
(cherry picked from commit 6bc7710ec7)
 2015-03-20 10:42:51 -07:00
Xuan 7990e1d80d YARN-2777. Mark the end of individual log in aggregated log. Contributed 
by Varun Saxena

(cherry picked from commit 1a4b528691)
 2015-03-20 10:25:26 -07:00
Jian He 3f0c9e5fe3 YARN-3379. Fixed missing data in localityTable and ResourceRequests table in RM WebUI. Contributed by Xuan Gong 
(cherry picked from commit 4e886eb9cb)
 2015-03-19 22:27:57 -07:00
Devaraj K a888dac344 YARN-3357. Move TestFifoScheduler to FIFO package. Contributed by Rohith 
Sharmaks.

(cherry picked from commit 93d0f4acc8)
 2015-03-19 12:18:08 +05:30
Karthik Kambatla 05627dd96d YARN-3351. AppMaster tracking URL is broken in HA. (Anubhav Dhoot via kasha) 
(cherry picked from commit 20b49224eb)
 2015-03-18 16:33:57 -07:00
Jian He 15ebacf03a YARN-3273. Improve scheduler UI to facilitate scheduling analysis and debugging. Contributed Rohith Sharmaks 
(cherry picked from commit 658097d6da)
 2015-03-17 21:31:20 -07:00
Tsuyoshi Ozawa b00b216a91 YARN-3205. FileSystemRMStateStore should disable FileSystem Cache to avoid get a Filesystem with an old configuration. Contributed by Zhihai Xu. 
(cherry picked from commit 3bc72cc16d)
 2015-03-18 11:53:40 +09:00
Jian He ab34e6975b YARN-3305. Normalize AM resource request on app submission. Contributed by Rohith Sharmaks 
(cherry picked from commit 968425e9f7)
 2015-03-17 13:50:44 -07:00
Karthik Kambatla 1e77d92d62 Revert "YARN-3181. FairScheduler: Fix up outdated findbugs issues. (kasha)" 
This reverts commit c2b185def8.

(cherry picked from commit 32b4330456)
 2015-03-17 12:31:44 -07:00
Jian He 1c601e492f YARN-3243. CapacityScheduler should pass headroom from parent to children to make sure ParentQueue obey its capacity limits. Contributed by Wangda Tan. 
(cherry picked from commit 487374b7fe)
 2015-03-17 10:25:07 -07:00
Devaraj K 895588b439 YARN-3197. Confusing log generated by CapacityScheduler. Contributed by 
Varun Saxena.

(cherry picked from commit 7179f94f9d)
 2015-03-17 15:59:19 +05:30
Ravi Prakash 9f227ad696 YARN-3339. TestDockerContainerExecutor should pull a single image and not the entire centos repository. (Ravindra Kumar Naik via raviprak) 
(cherry picked from commit 56085203c4)
 2015-03-16 16:18:42 -07:00
Tsuyoshi Ozawa a20427d339 YARN-3349. Treat all exceptions as failure in TestFSRMStateStore#testFSRMStateStoreClientRetry. Contributed by Zhihai Xu. 
(cherry picked from commit 7522a643fa)
 2015-03-17 08:10:12 +09:00
Harsh J 2684b0b440 MAPREDUCE-4414. Add main methods to JobConf and YarnConfiguration, for debug purposes. Contributed by Plamen Jeliazkov. 
(cherry picked from commit 571f75b468a4412be2178466cb5d3888b05cd076)
 2015-03-17 01:02:21 +05:30
Zhijie Shen fbe811d904 YARN-2854. Updated the documentation of the timeline service and the generic history service. Contributed by Naganarasimha G R. 2015-03-16 10:52:32 -07:00
Tsuyoshi Ozawa d6e3ef2506 YARN-1453. [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments. Contributed by Akira AJISAKA, Andrew Purtell, and Allen Wittenauer. 
(cherry picked from commit 3da9a97cfb)
 2015-03-16 23:20:17 +09:00
Xuan b6eb36dbdc YARN-3171. Sort by Application id, AppAttempt and ContainerID doesn't 
work in ATS / RM web ui. Contributed by Naganarasimha G R

(cherry picked from commit 3ff1ba2a7b)
 2015-03-15 20:27:44 -07:00
Jonathan Eagles 44aedad5dd YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles) 
(cherry picked from commit 8180e676ab)
 2015-03-13 12:05:21 -05:00
Vinod Kumar Vavilapalli 53aa3a4d1f YARN-3154. Added additional APIs in LogAggregationContext to avoid aggregating running logs of application when rolling is enabled. Contributed by Xuan Gong. 
(cherry picked from commit 863079bb87)
 2015-03-12 13:33:42 -07:00
Xuan 498d775c9b YARN-3338. Exclude jline dependency from YARN. Contributed by Zhijie 
Shen

(cherry picked from commit 06ce1d9a6c)
 2015-03-12 10:25:55 -07:00
Zhijie Shen 426535007b YARN-1884. Added nodeHttpAddress into ContainerReport and fixed the link to NM web page. Contributed by Xuan Gong. 
(cherry picked from commit 85f6d67fa7)
 2015-03-11 19:39:46 -07:00
Tsuyoshi Ozawa d29cf296cb YARN-3295. Fix documentation nits found in markdown conversion. Contributed by Masatake Iwasaki. 
(cherry picked from commit 30c428a858)
 2015-03-11 15:02:45 +09:00
Jian He 2690c72522 YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user or group. Contributed by Gururaj Shetty 
(cherry picked from commit a380643d20)
 2015-03-10 11:16:51 -07:00
Jian He 4021451746 YARN-3300. Outstanding_resource_requests table should not be shown in AHS. Contributed by Xuan Gong 
(cherry picked from commit c3003eba6f)
 2015-03-09 20:47:33 -07:00
Zhijie Shen a94d23762e YARN-3287. Made TimelineClient put methods do as the correct login context. Contributed by Daryn Sharp and Jonathan Eagles. 
(cherry picked from commit d6e05c5ee2)
 2015-03-09 13:55:51 -07:00
Tsuyoshi Ozawa b46f9e72db HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 2015-03-09 19:53:55 +09:00
Vinod Kumar Vavilapalli d9416317a9 Preparing for 2.8.0 development. Changing SNAPSHOT version to be 2.8 on branch-2. 2015-03-08 20:32:51 -07:00
Vinod Kumar Vavilapalli 8e6f8e9a04 Adding 2.8 section in CHANGES.txt 2015-03-08 20:26:57 -07:00
Junping Du b91e1b0feb YARN-3296. Mark ResourceCalculatorProcessTree class as Public for configurable resource monitoring. Contributed by Hitesh Shah 
(cherry picked from commit 7ce3c76353)
 2015-03-08 14:48:42 -07:00
Jason Lowe ef3d9bdf6b YARN-3275. CapacityScheduler: Preemption happening on non-preemptable queues. Contributed by Eric Payne 
(cherry picked from commit 27e8ea820f)
 2015-03-06 22:45:09 +00:00
Jian He 6cef2c16de YARN-2190. Added CPU and memory limit options to the default container executor for Windows containers. Contributed by Chuan Liu 
(cherry picked from commit 21101c01f2)
 2015-03-06 14:18:56 -08:00
Xuan 56c2050ab7 YARN-3227. Timeline renew delegation token fails when RM user's TGT is 
expired. Contributed by Zhijie Shen

(cherry picked from commit d1abc5d4fc)
 2015-03-06 13:33:18 -08:00
Jian He a5f3fb4dc1 YARN-1809. Synchronize RM and TimeLineServer Web-UIs. Contributed by Zhijie Shen and Xuan Gong 
(cherry picked from commit 95bfd087dc)
 2015-03-05 21:27:03 -08:00
Vinod Kumar Vavilapalli 9c8882546a YARN-2786. Created a yarn cluster CLI and seeded with one command for listing node-labels collection. Contributed by Wangda Tan. 2015-03-05 10:56:28 -08:00
Tsuyoshi Ozawa 5680b41597 Update CHANGES.txt for YARN-2616 to fix indentation. 
(cherry picked from commit 28b85a2116)
 2015-03-06 00:53:20 +09:00
Tsuyoshi Ozawa 6660c2f83b YARN-3249. Add a 'kill application' button to Resource Manager's Web UI. Contributed by Ryu Kobayashi. 
(cherry picked from commit 1b67209612)
 2015-03-05 19:56:59 +09:00
Karthik Kambatla 0d62e94887 YARN-3242. Asynchrony in ZK-close can lead to ZKRMStateStore watcher receiving events for old client. (Zhihai Xu via kasha) 
(cherry picked from commit 8d88691d16)
 2015-03-04 19:49:21 -08:00
Karthik Kambatla 721d7b5741 YARN-3231. FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) 
(cherry picked from commit 22426a1c9f)
 2015-03-04 18:07:33 -08:00
Karthik Kambatla 1749094fab YARN-3122. Metrics for container's actual CPU usage. (Anubhav Dhoot via kasha) 
(cherry picked from commit 53947f37c7)
 2015-03-04 17:39:39 -08:00
Jason Lowe ab397194c3 YARN-3131. YarnClientImpl should check FAILED and KILLED state in submitApplication. Contributed by Chang Li 
(cherry picked from commit 03cc22945e)
 2015-03-04 18:05:20 +00:00
Jian He aafe571398 YARN-3285. (Backport YARN-3168) Convert branch-2 .apt.vm files of YARN to markdown. Contributed by Masatake Iwasaki 2015-03-03 16:44:59 -08:00
Jian He 888a445638 YARN-3222. Fixed NPE on RMNodeImpl#ReconnectNodeTransition when a node is reconnected with a different port. Contributed by Rohith Sharmaks 
(cherry picked from commit b2f1ec312e)
 2015-03-03 16:28:55 -08:00
Wangda Tan 0a502c665b YARN-3272. Surface container locality info in RM web UI (Jian He via wangda) 
(cherry picked from commit e17e5ba9d7)
 2015-03-03 11:49:45 -08:00
Vinod Kumar Vavilapalli 253c78548b YARN-3265. Fixed a deadlock in CapacityScheduler by always passing a queue's available resource-limit from the parent queue. Contributed by Wangda Tan. 
(cherry picked from commit 14dd647c55)
 2015-03-02 17:54:01 -08:00
Wangda Tan 19a8e48230 YARN-3270. Fix node label expression not getting set in ApplicationSubmissionContext (Rohit Agarwal via wangda) 
(cherry picked from commit abac6eb9d5)
 2015-03-02 17:24:55 -08:00
Jian He 76b7283fa1 YARN-3281. Added RMStateStore to StateMachine visualization list. Contributed by Chengbing Liu 
(cherry picked from commit 5d0bae550f)
 2015-03-02 14:43:12 -08:00
Wangda Tan 0b0be0056b YARN-3262. Surface application outstanding resource requests table in RM web UI. (Jian He via wangda) 
(cherry picked from commit edcecedc1c)
 2015-02-27 16:14:35 -08:00
Tsuyoshi Ozawa 79f73f4613 YARN-2820. Retry in FileSystemRMStateStore when FS's operations fail due to IOException. Contributed by Zhihai Xu. 
(cherry picked from commit 01a1621930)
 2015-02-28 00:57:01 +09:00
Konstantin V Shvachko 9e67f2cb0d YARN-3255. RM, NM, JobHistoryServer, and WebAppProxyServer's main() should support generic options. Contributed by Konstantin Shvachko. 2015-02-26 17:26:08 -08:00
Tsuyoshi Ozawa dc348f4894 YARN-3217. Remove httpclient dependency from hadoop-yarn-server-web-proxy. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 773b6515ac)
 2015-02-27 00:25:12 +09:00
Devaraj K f83d2e4410 YARN-3256. TestClientToAMTokens#testClientTokenRace is not running against 
all Schedulers even when using ParameterizedSchedulerTestBase. Contributed
by Anubhav Dhoot.

(cherry picked from commit 0d4296f0e0)
 2015-02-26 15:47:32 +05:30
Jason Lowe 257087417e YARN-3239. WebAppProxy does not support a final tracking url which has query fragments and params. Contributed by Jian He 
(cherry picked from commit 1a68fc4346)
 2015-02-25 16:15:35 +00:00
Tsuyoshi Ozawa 13e213c41a YARN-3247. TestQueueMappings should use CapacityScheduler explicitly. Contributed by Zhihai Xu. 
(cherry picked from commit 6cbd9f1113)
 2015-02-25 11:23:34 +09:00
Devaraj K b9e82e27f5 YARN-2797. Add -help to yarn logs and nodes CLI command. Contributed by 
Jagadesh Kiran N.

(cherry picked from commit b610c68d44)
 2015-02-24 12:08:50 +05:30
Xuan 0ca974d0b0 YARN-2797. TestWorkPreservingRMRestart should use 
ParametrizedSchedulerTestBase. Contributed by Karthik Kambatla

(cherry picked from commit fe7a302473)
 2015-02-21 19:19:08 -08:00
Xuan f8435c5d3d YARN-3236. Cleanup RMAuthenticationFilter#AUTH_HANDLER_PROPERTY. 
Contributed by zhihai xu

(cherry picked from commit e3d290244c)
 2015-02-21 16:20:33 -08:00
Xuan fefeba4ac8 YARN-3238. Connection timeouts to nodemanagers are retried at multiple 
levels. Contributed by Jason Lowe

(cherry picked from commit 92d67ace32)
 2015-02-21 16:07:28 -08:00
Xuan 9cbe9bcbaa YARN-3237. AppLogAggregatorImpl fails to log error cause. Contributed by 
Rushabh S Shah

(cherry picked from commit f56c65bb3e)
 2015-02-20 14:03:46 -08:00
Wangda Tan a1963968d2 YARN-3230. Clarify application states on the web UI. (Jian He via wangda) 
(cherry picked from commit ce5bf927c3)
 2015-02-20 10:40:04 -08:00
Junping Du 4a87a61fe9 YARN-2799. Cleanup TestLogAggregationService based on the change in YARN-90. Contributed by Zhihai Xu 
(cherry picked from commit c33ae271c2)
 2015-02-20 09:44:31 -08:00
Jason Lowe 3e8dfd1299 YARN-3194. RM should handle NMContainerStatuses sent by NM while registering if NM is Reconnected node. Contributed by Rohith 
(cherry picked from commit a64dd3d24b)
 2015-02-20 15:12:08 +00:00
Jian He eaeaf80d39 YARN-933. Fixed InvalidStateTransitonException at FINAL_SAVING state in RMApp. Contributed by Rohith Sharmaks 
(cherry picked from commit c0d9b93953)
 2015-02-19 15:43:14 -08:00
Wangda Tan cd5eb9c1de YARN-3076. Add API/Implementation to YarnClient to retrieve label-to-node mapping (Varun Saxena via wangda) 
(cherry picked from commit d49ae725d5)
 2015-02-19 11:01:47 -08:00
Tsuyoshi Ozawa fef5efa0cb YARN-1615. Fix typos in delay scheduler's description. Contributed by Akira Ajisaka. 
(cherry picked from commit b8a14efdf5)
 2015-02-19 10:24:40 +09:00
Jian He b8fe1a747e YARN-1514. Utility to benchmark ZKRMStateStore#loadState for RM HA. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 1c03376300)
 2015-02-18 16:34:42 -08:00
Jian He 8f06eaa251 YARN-3132. RMNodeLabelsManager should remove node from node-to-label mapping when node becomes deactivated. Contributed by Wangda Tan 
(cherry picked from commit f5da5566d9)
 2015-02-18 11:52:28 -08:00
Xuan ba18adbb27 YARN-3207. Secondary filter matches entites which do not have the key 
being filtered for. Contributed by Zhijie Shen

(cherry picked from commit 57db50cbe3)
 2015-02-17 18:18:56 -08:00
Tsuyoshi Ozawa fabac2ff30 YARN-1299. Improve a log message in AppSchedulingInfo by adding application id. Contributed by Ashutosh Jindal and Devaraj K. 
(cherry picked from commit 556386a070)
 2015-02-17 01:54:09 +09:00
Tsuyoshi Ozawa 42368b91c4 Revert "YARN-1299. Improve a log message in AppSchedulingInfo by adding application id. Contributed by Ashutosh Jindal and devaraj." 
This reverts commit 67e3a77720.
 2015-02-17 01:53:58 +09:00
Tsuyoshi Ozawa 67e3a77720 YARN-1299. Improve a log message in AppSchedulingInfo by adding application id. Contributed by Ashutosh Jindal and devaraj. 
(cherry picked from commit 9aae81c934)
 2015-02-17 01:48:53 +09:00
Tsuyoshi Ozawa fd8f7e2c7d YARN-3203. Correct a log message in AuxServices. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 447bd7b5a6)
 2015-02-16 23:56:32 +09:00
Junping Du eee89bd18c YARN-2749. Fix some testcases from TestLogAggregationService fails in trunk. (Contributed by Xuan Gong) 
(cherry picked from commit ab0b958a52)
 2015-02-15 06:48:29 -08:00
cnauroth 48302e687a YARN-2899. Run TestDockerContainerExecutorWithMocks on Linux only. Contributed by Ming Ma. 
(cherry picked from commit 6804d68901)
 2015-02-13 21:59:14 -08:00
Jason Lowe c9cd58d711 YARN-2847. Linux native container executor segfaults if default banned user detected. Contributed by Olaf Flebbe 
(cherry picked from commit 1a0f508b63)
 2015-02-13 20:22:12 +00:00
Xuan 3c4b5606dc YARN-3164. RMAdmin command usage prints incorrect command name. 
Contributed by Bibin A Chundatt

(cherry picked from commit 253035491c)
 2015-02-12 21:40:42 -08:00
Tsuyoshi Ozawa 613065d75a YARN-3182. Cleanup switch statement in ApplicationMasterLauncher#handle(). Contributed by Ray Chiang. 
(cherry picked from commit 110cf6b9a3)
 2015-02-13 14:21:43 +09:00
Tsuyoshi Ozawa 43429cd33a YARN-2994. Document work-preserving RM restart. Contributed by Jian He. 
(cherry picked from commit b0d81e05ab)
 2015-02-13 13:08:37 +09:00
Akira Ajisaka 9d8dbf8ea4 YARN-3191. Log object should be initialized with its own class. Contributed by Rohith. 
(cherry picked from commit 99c7f2b8fa417303e8f42ea3641c302135879280)
 2015-02-12 17:59:04 -08:00
Xuan df6bee5a75 YARN-3179. Update use of Iterator to Iterable in RMAdminCLI and 
CommonNodeLabelsManager. Contributed by Ray Chiang

(cherry picked from commit 2586915bb3)
 2015-02-12 17:32:58 -08:00
Xuan 5f0495689f YARN-3158. Correct log messages in ResourceTrackerService. Contributed 
by Varun Saxena

(cherry picked from commit 99f6bd4f7a)
 2015-02-12 16:43:28 -08:00
Jian He b5d6f76c6a YARN-3104. Fixed RM to not generate new AMRM tokens on every heartbeat between rolling and activation. Contributed by Jason Lowe 
(cherry picked from commit 18297e0972)
 2015-02-12 16:03:12 -08:00
Karthik Kambatla fcfbdf6440 YARN-3181. FairScheduler: Fix up outdated findbugs issues. (kasha) 
(cherry picked from commit c2b185def8)
 2015-02-12 15:39:50 -08:00
Jian He 1be2d64ddd YARN-3124. Fixed CS LeafQueue/ParentQueue to use QueueCapacities to track capacities-by-label. Contributed by Wangda Tan 
(cherry picked from commit 18a594257e)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-02-12 15:01:14 -08:00
Junping Du 380cc4dbed YARN-2079. Recover NonAggregatingLogHandler state upon nodemanager restart. (Contributed by Jason Lowe) 
(cherry picked from commit 04f5ef18f7)
 2015-02-12 11:48:24 -08:00
Xuan 58d9ce2ea1 YARN-3147. Clean up RM web proxy code. Contributed by Steve Loughran 
(cherry picked from commit 83be450acc)
 2015-02-12 10:59:18 -08:00
Tsuyoshi Ozawa dc14f2772f YARN-3157. Refactor the exception handling in ConverterUtils#to*Id. Contributed by Bibin A Chundatt. 
(cherry picked from commit 7bc492adab)
 2015-02-12 23:49:27 +09:00
Junping Du 634cc297ba YARN-1580. Documentation error regarding container-allocation.expiry-interval-ms (Brahma Reddy Battula via junping_du) 
(cherry picked from commit 46c7577b98)
 2015-02-12 03:40:57 -08:00
Tsuyoshi Ozawa c9c1d9a5a6 HADOOP-11543. Improve help message for hadoop/yarn command. Contributed by Brahma Reddy Battula. 2015-02-12 19:03:24 +09:00
Tsuyoshi Ozawa fd047702e4 YARN-1237. Description for yarn.nodemanager.aux-services in yarn-default.xml is misleading. Contributed by Brahma Reddy Battula. 
(cherry picked from commit b3bcbaf277)
 2015-02-12 15:19:54 +09:00
Xuan cac6fe217a YARN-3151. On Failover tracking url wrong in application cli for KILLED 
application. Contributed by Rohith

(cherry picked from commit 65c69e296e)
 2015-02-11 21:21:06 -08:00
Jason Lowe 38333c8f29 YARN-3074. Nodemanager dies when localizer runner tries to write to a full disk. Contributed by Varun Saxena 
(cherry picked from commit b379972ab3)
 2015-02-11 16:34:42 +00:00
Steve Loughran a8eaec396f YARN-2683. [YARN-913] registry config options: document and move to core-default. (stevel) 2015-02-11 14:26:22 +00:00
Steve Loughran 4241c2be3d YARN-2616 [YARN-913] Add CLI client to the registry to list, view and manipulate entries. (Akshay Radia via stevel) 2015-02-11 14:18:46 +00:00
Junping Du f2c3058b8d YARN-3160. Fix non-atomic operation on nodeUpdateQueue in RMNodeImpl. (Contributed by Chengbing Liu) 
(cherry picked from commit c541a374d8)
 2015-02-11 03:46:40 -08:00
Zhijie Shen fd75b8c9ca YARN-2246. Made the proxy tracking URL always be http(s)://proxy addr:port/proxy/<appId> to avoid duplicate sections. Contributed by Devaraj K. 
(cherry picked from commit d5855c0e46)
 2015-02-10 15:29:09 -08:00
Jason Lowe ca11ffa5de YARN-2809. Implement workaround for linux kernel panic when removing cgroup. Contributed by Nathan Roberts 
(cherry picked from commit 3f5431a22f)
 2015-02-10 17:28:18 +00:00
Jason Lowe 9ffd985dc0 YARN-3090. DeletionService can silently ignore deletion task failures. Contributed by Varun Saxena 
(cherry picked from commit 4eb5f7fa32)
 2015-02-10 16:55:21 +00:00
Zhijie Shen 24d4d933f7 YARN-3100. Made YARN authorization pluggable. Contributed by Jian He. 
(cherry picked from commit 23bf6c7207)
 2015-02-09 20:37:58 -08:00
Wangda Tan 4d4442cb39 YARN-3155. Refactor the exception handling code for TimelineClientImpl's retryOn method (Li Lu via wangda) 
(cherry picked from commit 00a748d24a)
 2015-02-09 17:30:59 -08:00
Jonathan Eagles c0e14d546e YARN-2971. RM uses conf instead of token service address to renew timeline delegation tokens (jeagles) 
(cherry picked from commit af08425893)
 2015-02-09 17:56:59 -06:00
Jian He 6146680955 YARN-3094. Reset timer for liveness monitors after RM recovery. Contributed by Jun Gong 
(cherry picked from commit 0af6a99a3f)
 2015-02-09 13:47:33 -08:00
Karthik Kambatla 1cb1c8b615 YARN-2990. FairScheduler's delay-scheduling always waits for node-local and rack-local delays, even for off-rack-only requests. (kasha) 
(cherry picked from commit 7e42088abf)
 2015-02-08 22:48:29 -08:00
Jason Lowe 61fdd862be YARN-3143. RM Apps REST API can return NPE or entries missing id and other fields. Contributed by Jason Lowe 
(cherry picked from commit da2fb2bc46)
 2015-02-06 21:51:49 +00:00
Jason Lowe 9361fc8f70 YARN-3144. Configuration for making delegation token failures to timeline server not-fatal. Contributed by Jonathan Eagles 
(cherry picked from commit 6f10434a5a)
 2015-02-06 20:55:56 +00:00
Jason Lowe 83449a4e4d YARN-3089. LinuxContainerExecutor does not handle file arguments to deleteAsUser. Contributed by Eric Payne 
(cherry picked from commit 4c484320b4)
 2015-02-06 20:40:33 +00:00
Jian He 3ddafaa7c8 YARN-2694. Ensure only single node label specified in ResourceRequest. Contributed by Wangda Tan 
(cherry picked from commit c1957fef29)
 2015-02-06 11:34:56 -08:00
Arun C. Murthy 92ff524182 YARN-1537. Fix race condition in TestLocalResourcesTrackerImpl.testLocalResourceCache. Contributed by Xuan Gong. 2015-02-05 23:59:34 -08:00
Arun C. Murthy 29c77de297 YARN-1904. Ensure exceptions thrown in ClientRMService & ApplicationHistoryClientService are uniform when application-attempt is not found. Contributed by Zhijie Shen. 2015-02-05 23:49:43 -08:00
Tsuyoshi Ozawa 995f9809c0 Revert "HADOOP-11543. Improve help message for hadoop/yarn command. Contributed by Brahma Reddy Battula." 
This reverts commit 366a12c3b9.
 2015-02-06 13:49:58 +09:00
Tsuyoshi Ozawa 366a12c3b9 HADOOP-11543. Improve help message for hadoop/yarn command. Contributed by Brahma Reddy Battula. 2015-02-06 13:34:48 +09:00
Jian He b109051145 YARN-3145. Fixed ConcurrentModificationException on CapacityScheduler ParentQueue#getQueueUserAclInfo. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 4641196fe0)
 2015-02-05 16:13:23 -08:00
Xuan 096ba583fa YARN-3149. Fix typo in message for invalid application id. Contributed 
by Bibin A Chundatt.

(cherry picked from commit b77ff37686)
 2015-02-05 15:15:29 -08:00
Jason Lowe ebdd88ec81 YARN-1582. Capacity Scheduler: add a maximum-allocation-mb setting per queue. Contributed by Thomas Graves 
(cherry picked from commit 69c8a7f45b)
 2015-02-05 19:30:06 +00:00
Sandy Ryza 84df660af4 YARN-3101. In Fair Scheduler, fix canceling of reservations for exceeding max share (Anubhav Dhoot via Sandy Ryza) 2015-02-05 09:45:17 -08:00
Zhijie Shen 2db56a4932 YARN-3123. Made YARN CLI show a single completed container even if the app is running. Contributed by Naganarasimha G R. 
(cherry picked from commit 30510cff75)
 2015-02-04 16:07:24 -08:00
Siddharth Seth c330bca833 YARN-1723. AMRMClientAsync missing blacklist addition and removal 
functionality. Contributed by Bartosz Ługowski.
(cherry picked from commit e04e8fa810)
 2015-02-04 12:53:31 -08:00
Tsuyoshi Ozawa cdd76d5922 YARN-3058. Fix error message of tokens' activation delay configuration. Contributed by Yi Liu. 
(cherry picked from commit 26dee1486b)
 2015-02-04 11:41:48 +09:00
Zhijie Shen 3ab3caa9ea YARN-2543. Made resource usage be published to the timeline server too. Contributed by Naganarasimha G R. 
(cherry picked from commit e5b56e2af6)
 2015-02-03 17:35:40 -08:00
Xuan c22dcdd191 YARN-3056. Add verification for containerLaunchDuration in 
TestNodeManagerMetrics. Contributed by zhihai Xu

(cherry picked from commit b73e776abc)
 2015-02-03 15:14:23 -08:00
Karthik Kambatla c981b040ba YARN-1393. SLS: Add how-to-use instructions. (Wei Yan via kasha) 
(cherry picked from commit c559df2219)
 2015-02-03 13:20:57 -08:00
Wangda Tan 205e15c4a3 YARN-3075. NodeLabelsManager implementation to retrieve label to node mapping (Varun Saxena via wangda) 
(cherry picked from commit 5bd984691b)
 2015-02-03 12:55:25 -08:00
Jian He c0b1311a93 YARN-3098. Created common QueueCapacities class in Capacity Scheduler to track capacities-by-labels of queues. Contributed by Wangda Tan 
(cherry picked from commit 21d80b3dd9)
 2015-02-03 11:43:40 -08:00
Robert Kanter 410830fe8c YARN-3022. Expose Container resource information from NodeManager for monitoring (adhoot via ranter) 
(cherry picked from commit f7a77819a1)
 2015-02-03 10:39:51 -08:00
Jason Lowe 3828fb5fab YARN-3085. Application summary should include the application type. Contributed by Rohith 
(cherry picked from commit a761bf8726)
 2015-02-03 14:57:25 +00:00
Zhijie Shen ebaf6220fe YARN-2216. Fixed the change log. 
(cherry picked from commit 5f9a0dd8d9)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-02-02 12:16:07 -08:00
Zhijie Shen b51c0288de YARN-2808. Made YARN CLI list attempt’s finished containers of a running application. Contributed by Naganarasimha G R. 
(cherry picked from commit c0ec4f4cc12208342d45236b8841887a20f7f88a)
 2015-02-02 12:05:05 -08:00
Kihwal Lee 05d831c4b7 YARN-3113. Release audit warning for Sorting icons.psd. Contributed by Steve Loughran. 
(cherry picked from commit f33c99ba39)
 2015-02-02 12:59:45 -06:00
Jian He 4941b8f30d YARN-3077. Fixed RM to create zk root path recursively. Contributed by Chun Chen 
(cherry picked from commit 054a947989)
 2015-01-30 17:35:26 -08:00
Jian He cabf97ae4f YARN-3099. Capacity Scheduler LeafQueue/ParentQueue should use ResourceUsage to track used-resources-by-label. Contributed by Wangda Tan 
(cherry picked from commit 86358221fc)
 2015-01-30 15:16:22 -08:00
Jonathan Eagles d4ffca41d7 YARN-3108. ApplicationHistoryServer doesn't process -D arguments (Chang Li via jeagles) 
(cherry picked from commit 30a8778c63)
 2015-01-29 16:52:37 -06:00
Tsuyoshi Ozawa 6efffc9cd8 YARN-3029. FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere. Contributed by Varun Saxena. 
(cherry picked from commit 7acce7d364)
 2015-01-30 04:42:25 +09:00
Akira Ajisaka 1081b4c4a7 HADOOP-9907. Webapp http://hostname:port/metrics link is not working. (aajisaka) 
(cherry picked from commit 342efa110a)
 2015-01-30 02:50:46 +09:00
Wangda Tan 58d7d1efc6 YARN-3079. Scheduler should also update maximumAllocation when updateNodeResource. (Zhihai Xu via wangda) 
(cherry picked from commit 7882bc0f14)
 2015-01-28 21:56:17 -08:00
Jian He 12522fd9cb YARN-3103. AMRMClientImpl does not update AMRM token properly. Contributed by Jason Lowe 
(cherry picked from commit 6d2bdbd7da)
 2015-01-28 15:24:10 -08:00
Wangda Tan 4718af4352 YARN-2932. Add entry for preemptable status (enabled/disabled) to scheduler web UI and queue initialize/refresh logging. (Eric Payne via wangda) 
(cherry picked from commit 18741adf97)
 2015-01-27 15:37:02 -08:00
Wangda Tan 12060cb9a6 YARN-3028. Better syntax for replaceLabelsOnNode in RMAdmin CLI. Contributed by Rohith Sharmaks 
(cherry picked from commit fd93e5387b)
 2015-01-27 15:25:47 -08:00
Jonathan Eagles c95e8951f8 YARN-2897. CrossOriginFilter needs more log statements (Mit Desai via jeagles) 
(cherry picked from commit a8ad1e8089)
 2015-01-27 16:07:27 -06:00
Jian He 8100c8a68c YARN-3011. Possible IllegalArgumentException in ResourceLocalizationService might lead NM to crash. Contributed by Varun Saxena 
(cherry picked from commit 4e15fc0841)
 2015-01-27 13:31:48 -08:00
Tsuyoshi Ozawa 04ad500fac YARN-3086. Make NodeManager memory configurable in MiniYARNCluster. Contributed by Robert Metzger. 
(cherry picked from commit f56da3ce04)
 2015-01-28 00:16:07 +09:00
Jian He 61b4116b4b YARN-3092. Created a common ResourceUsage class to track labeled resource usages in Capacity Scheduler. Contributed by Wangda Tan 
(cherry picked from commit 6f9fe76918)
 2015-01-26 15:38:27 -08:00
Jason Lowe 07fe6a36cb YARN-3088. LinuxContainerExecutor.deleteAsUser can throw NPE if native executor returns an error. Contributed by Eric Payne 
(cherry picked from commit 902c6ea7e4)
 2015-01-26 15:41:23 +00:00
Xuan a7696b3fbf YARN-3024. LocalizerRunner should give DIE action when all resources are 
localized. Contributed by Chengbing Liu

(cherry picked from commit 0d6bd62102)
 2015-01-25 19:39:52 -08:00
Tsuyoshi Ozawa 25f2cf8a4e YARN-2800. Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature. Contributed by Wangda Tan. 
(cherry picked from commit 24aa462673)
 2015-01-23 20:37:26 +09:00
Tsuyoshi Ozawa ff627d94e7 YARN-3082. Non thread safe access to systemCredentials in NodeHeartbeatResponse processing. Contributed by Anubhav Dhoot. 
(cherry picked from commit 3aab354e66)
 2015-01-23 16:05:05 +09:00
Tsuyoshi Ozawa 316eedb55f YARN-3078. LogCLIHelpers lacks of a blank space before string 'does not exist'. Contributed by Sam Liu. 
(cherry picked from commit 5712c9f96a)
 2015-01-22 12:25:29 +09:00
Wangda Tan 610aa71a10 YARN-2731. Fixed RegisterApplicationMasterResponsePBImpl to properly invoke maybeInitBuilder. (Contributed by Carlo Curino) 
(cherry picked from commit f250ad1773)
 2015-01-20 14:49:06 -08:00
Wangda Tan ef6fc24dfb YARN-2933. Capacity Scheduler preemption policy should only consider capacity without labels temporarily. Contributed by Mayank Bansal 
(cherry picked from commit 0a2d3e717d)
 2015-01-19 16:57:17 -08:00
cnauroth 2c4abe9208 YARN-3015. yarn classpath command should support same options as hadoop classpath. Contributed by Varun Saxena. 2015-01-19 15:54:41 -08:00
Akira Ajisaka 401c7d72f5 YARN-3071. Remove invalid char from sample conf in doc of FairScheduler. (Contributed by Masatake Iwasaki) 
(cherry picked from commit 4a5c3a4cfe)
 2015-01-19 21:22:48 +09:00
Junping Du ef4d7b73b9 YARN-3070. TestRMAdminCLI#testHelp fails for transitionToActive command. (Contributed by Junping Du) 
(cherry picked from commit 19cbce3898)
 2015-01-18 23:09:24 -08:00
Karthik Kambatla 4d8fa9615f YARN-2984. Metrics for container's actual memory usage. (kasha) 
(cherry picked from commit 84198564ba)
 2015-01-17 06:26:47 +05:30
Zhijie Shen dcd6eed13a YARN-2815. Excluded transitive dependency of JLine in hadoop-yarn-server-common. Contributed by Ferdinand Xu. 
(cherry picked from commit 43302f6f44)
 2015-01-16 16:38:36 -08:00
Junping Du 7cddec31d7 YARN-3064. TestRMRestart/TestContainerResourceUsage/TestNodeManagerResync failure with allocation timeout. (Contributed by Jian He) 
(cherry picked from commit 5d1cca34fa)
 2015-01-16 00:10:36 -08:00
Jian He ed5a0d0aa8 YARN-2861. Fixed Timeline DT secret manager to not reuse RM's configs. Contributed by Zhijie Shen 
(cherry picked from commit 9e33116d1d)
 2015-01-15 11:08:19 -08:00
Akira Ajisaka b349cd9d0b YARN-3005. [JDK7] Use switch statement for String instead of if-else statement in RegistrySecurity.java (Contributed by Kengo Seki) 
(cherry picked from commit 533e551eb4)
 2015-01-15 21:30:26 +09:00
Karthik Kambatla d98b8f4bd3 YARN-2217. [YARN-1492] Shared cache client side changes. (Chris Trezzo via kasha) 
(cherry picked from commit ba5116ec8e)
 2015-01-15 14:32:35 +05:30
Xuan 79c8f4dcba YARN-2807. Option "--forceactive" not works as described in usage of 
"yarn rmadmin -transitionToActive". Contributed by Masatake Iwasaki

(cherry picked from commit d15cbae73c)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/YarnCommands.apt.vm
 2015-01-14 11:18:10 -08:00
Junping Du 0119e717ae YARN-3019. Make work-preserving-recovery the default mechanism for RM recovery. (Contributed by Jian He) 
(cherry picked from commit f92e503800)
 2015-01-13 18:23:32 -08:00
Jian He 4931600030 YARN-2637. Fixed max-am-resource-percent calculation in CapacityScheduler when activating applications. Contributed by Craig Welch 
(cherry picked from commit c53420f583)
 2015-01-13 17:32:36 -08:00
Robert Kanter 13cdcf28e0 YARN-2643. Don't create a new DominantResourceCalculator on every FairScheduler.allocate call. (kasha via rkanter) 
(cherry picked from commit 51881535e6)
 2015-01-12 17:51:45 -08:00
Robert Kanter 7c4e5ff660 YARN-2957. Create unit test to automatically compare YarnConfiguration and yarn-default.xml. (rchiang via rkanter) 
(cherry picked from commit f451631915)
 2015-01-12 12:00:52 -08:00
Robert Kanter e5059b91bb YARN-3027. Scheduler should use totalAvailable resource from node instead of availableResource for maxAllocation. (adhoot via rkanter) 
(cherry picked from commit ae7bf31fe1)
 2015-01-12 10:48:45 -08:00
Jian He 36b3dcaab2 YARN-3014. Replaces labels on a host should update all NM's labels on that host. Contributed by Wangda Tan 
(cherry picked from commit a260406268)
 2015-01-09 17:50:18 -08:00
Jian He 68709cb9be YARN-2956. Added missing links in YARN documentation. Contributed by Masatake Iwasaki 
(cherry picked from commit 27a489cba6)
 2015-01-09 17:47:00 -08:00
cnauroth 9a7c763b3f HADOOP-11464. Reinstate support for launching Hadoop processes on Windows using Cygwin. Contributed by Chris Nauroth. 2015-01-09 15:01:33 -08:00
Jian He e7e6173049 YARN-2997. Fixed NodeStatusUpdater to not send alreay-sent completed container statuses on heartbeat. Contributed by Chengbing Liu 
(cherry picked from commit cc2a745f7e)
 2015-01-08 11:28:24 -08:00
Zhijie Shen d2fbba790a YARN-2996. Improved synchronization and I/O operations of FS- and Mem- RMStateStore. Contributed by Yi Liu. 
(cherry picked from commit dc2eaa26b2)
 2015-01-08 09:49:27 -08:00
Jian He 9eb7611e81 Moved YARN-2880 to improvement section in CHANGES.txt 
(cherry picked from commit ef237bd52f)
 2015-01-07 16:21:12 -08:00
Jian He fe664b8b8d YARN-2936. Changed YARNDelegationTokenIdentifier to set proto fields on getProto method. Contributed by Varun Saxena 
(cherry picked from commit 2638f4d0f0)
 2015-01-07 16:15:03 -08:00
Jian He d3b26454e4 YARN-3010. Fixed findbugs warning in AbstractYarnScheduler. Contributed by Yi Liu 
(cherry picked from commit e13a484a2b)
 2015-01-07 14:12:54 -08:00
Jian He 492717cbf5 YARN-2230. Fixed few configs description in yarn-default.xml. Contributed by Vijay Bhat 
(cherry picked from commit fe8d2bd741)
 2015-01-07 10:57:41 -08:00
Zhijie Shen 8a81312062 YARN-2427. Added the API of moving apps between queues in RM web services. Contributed by Varun Vasudev. 
(cherry picked from commit 60103fca04)
 2015-01-06 14:39:44 -08:00
Jian He c61e8a7bfa YARN-2978. Fixed potential NPE while getting queue info. Contributed by Varun Saxena 
(cherry picked from commit dd57c2047b)
 2015-01-06 14:16:43 -08:00
Karthik Kambatla 7adffad2bb YARN-2881. [YARN-2574] Implement PlanFollower for FairScheduler. (Anubhav Dhoot via kasha) 
(cherry picked from commit 0c4b112677)
 2015-01-06 04:42:55 +05:30
Karthik Kambatla d070597a0e YARN-2360. Fair Scheduler: Display dynamic fair share for queues on the scheduler page. (Ashwin Shankar and Wei Yan via kasha) 
(cherry picked from commit 270a271f53)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-01-06 03:28:48 +05:30
Zhijie Shen c6cf748985 YARN-2958. Made RMStateStore not update the last sequence number when updating the delegation token. Contributed by Varun Saxena. 
(cherry picked from commit 562a701945)
 2015-01-05 13:35:09 -08:00
Tsuyoshi Ozawa c116743bdd YARN-2922. ConcurrentModificationException in CapacityScheduler's LeafQueue. Contributed by Rohith Sharmaks. 
(cherry picked from commit ddc5be48fc)
 2015-01-05 00:11:01 +09:00
Zhijie Shen efe6260762 YARN-2991. Fixed DrainDispatcher to reuse the draining code path in AsyncDispatcher. Contributed by Rohith Sharmaks. 
(cherry picked from commit 947578c1c1)
 2015-01-02 11:10:46 -08:00
Karthik Kambatla 798ab51289 YARN-2998. Abstract out scheduler independent PlanFollower components. (Anubhav Dhoot via kasha) 
(cherry picked from commit e7257acd8a)
 2014-12-30 19:55:50 -08:00
Jian He cde5bfe3ec YARN-2987. Fixed ClientRMService#getQueueInfo to check against queue and app ACLs. Contributed by Varun Saxena 
(cherry picked from commit e2351c7ae2)
 2014-12-30 17:16:05 -08:00
Jian He 143e48c25a YARN-2493. Added node-labels page on RM web UI. Contributed by Wangda Tan 
(cherry picked from commit b7442bf92e)
 2014-12-30 16:49:59 -08:00
Zhijie Shen e5cb52ec1f YARN-2938. Fixed new findbugs warnings in hadoop-yarn-resourcemanager and hadoop-yarn-applicationhistoryservice. Contributed by Varun Saxena. 
(cherry picked from commit 241d3b3a50)
 2014-12-29 10:07:54 -08:00
Jian He ca0349b87a YARN-2992. ZKRMStateStore crashes due to session expiry. Contributed by Karthik Kambatla 
(cherry picked from commit 1454efe5d4)
 2014-12-26 18:00:32 -08:00
Junping Du fc3e810a19 YARN-2993. Several fixes (missing acl check, error log msg ...) and some refinement in AdminService. (Contributed by Yi Liu) 
(cherry picked from commit 40ee4bff65)
 2014-12-26 07:01:44 -08:00
Tsuyoshi Ozawa 4ca49c8634 YARN-2988. Graph#save() may leak file descriptors. (Ted Yu via ozawa) 2014-12-25 13:09:51 +09:00
Jian He 60530a6c4f YARN-2946. Fixed potential deadlock in RMStateStore. Contributed by Rohith Sharmaks 
(cherry picked from commit 4f18018b7a)
 2014-12-23 22:15:28 -08:00
Zhijie Shen f02bd6683a YARN-2937. Fixed new findbugs warnings in hadoop-yarn-nodemanager. Contributed by Varun Saxena. 
(cherry picked from commit 41a548a916)
 2014-12-23 20:35:32 -08:00
Junping Du 361fab14be YARN-2940. Fix new findbugs warnings in rest of the hadoop-yarn components. (Contributed by Li Lu) 
(cherry picked from commit d468c9aaf1)
 2014-12-23 20:06:53 -08:00
Jian He e3a54b1b8b YARN-2837. Support TimeLine server to recover delegation token when restarting. Contributed by Zhijie Shen 
(cherry picked from commit 149512a837)
 2014-12-23 18:26:37 -08:00
Junping Du ae3faab13e YARN-2970. NodeLabel operations in RMAdmin CLI get missing in help command. (Contributed by Varun Saxena) 
(cherry picked from commit e1ee0d45ea)
 2014-12-23 18:24:45 -08:00
Jian He 8b398a66ca YARN-2340. Fixed NPE when queue is stopped during RM restart. Contributed by Rohith Sharmaks 
(cherry picked from commit 0d89859b51)
 2014-12-22 21:55:06 -08:00
Jian He 411836b74c YARN-2920. Changed CapacityScheduler to kill containers on nodes where node labels are changed. Contributed by Wangda Tan 
(cherry picked from commit fdf042dfff)
 2014-12-22 16:52:15 -08:00
Junping Du 6f252f7f36 YARN-2939. Fix new findbugs warnings in hadoop-yarn-common. (Li Lu via junping_du) 
(cherry picked from commit a696fbb001)
 2014-12-22 03:08:25 -08:00
Tsuyoshi Ozawa d5d13bbe0f CHANGES.txt: move YARN-2977 from 2.6.1 to 2.7.0 2014-12-21 12:53:25 +09:00
Karthik Kambatla 2abec14ec6 YARN-2975. FSLeafQueue app lists are accessed without required locks. (kasha) 
(cherry picked from commit 24ee9e3431)
 2014-12-20 12:20:29 -08:00
Tsuyoshi Ozawa 7b6a0badc2 CHANGES.txt: add YARN-2977 2014-12-20 22:57:58 +09:00
Tsuyoshi Ozawa 31753da961 YARN-2977. Fixed intermittent TestNMClient failure. (Contributed by Junping Du) 2014-12-20 22:43:47 +09:00
Jian He 9180d11b3b YARN-2952. Fixed incorrect version check in StateStore. Contributed by Rohith Sharmaks 
(cherry picked from commit 808cba3821)
 2014-12-19 16:56:55 -08:00
Karthik Kambatla b4e8ae591d YARN-2675. containersKilled metrics is not updated when the container is killed during localization. (Zhihai Xu via kasha) 
(cherry picked from commit 954fb8581e)
 2014-12-19 16:03:02 -08:00
Karthik Kambatla 8ee40a1580 YARN-2738. [YARN-2574] Add FairReservationSystem for FairScheduler. (Anubhav Dhoot via kasha) 
(cherry picked from commit a22ffc3188)
 2014-12-19 15:38:27 -08:00
Jason Lowe 173664d70f YARN-2964. RM prematurely cancels tokens for jobs that submit jobs (oozie). Contributed by Jian He 
(cherry picked from commit 0402bada19)
 2014-12-18 23:31:11 +00:00
Junping Du cec9300201 YARN-2949. Add documentation for CGroups. (Contributed by Varun Vasudev) 
(cherry picked from commit 389f881d42)
 2014-12-18 06:26:53 -08:00
Junping Du 11e4ec52a1 YARN-2972. DelegationTokenRenewer thread pool never expands. Contributed by Jason Lowe 
(cherry picked from commit 2b4b0e8847)
 2014-12-17 17:41:21 -08:00
Karthik Kambatla d0d42c4e32 YARN-2203. [YARN-1492] Web UI for cache manager. (Chris Trezzo via kasha) 
(cherry picked from commit b7f64823e1)
 2014-12-17 16:34:33 -08:00
Karthik Kambatla 375a1e5971 YARN-2944. InMemorySCMStore can not be instantiated with ReflectionUtils#newInstance. (Chris Trezzo via kasha) 
(cherry picked from commit a1bd140964)
 2014-12-17 16:25:02 -08:00
Karthik Kambatla 2d832ad2eb YARN-2964. FSLeafQueue#assignContainer - document the reason for using both write and read locks. (Tsuyoshi Ozawa via kasha) 
(cherry picked from commit f2d150ea12)
 2014-12-17 16:24:17 -08:00
Karthik Kambatla 300a665307 YARN-2189. Admin service for cache manager. Addendum to fix merge conflicts 2014-12-17 11:47:24 -08:00
cnauroth 36068768d8 HADOOP-11321. copyToLocal cannot save a file to an SMB share unless the user has Full Control permissions. Contributed by Chris Nauroth. 
(cherry picked from commit e996a1bfd4)
 2014-12-16 15:32:23 -08:00
Jian He 878350a0e8 YARN-2762. Fixed RMAdminCLI to trim and check node-label related arguments before sending to RM. Contributed by Rohith Sharmaks 
(cherry picked from commit c65f1b382e)
 2014-12-16 11:01:24 -08:00
Karthik Kambatla b1ccf68db3 YARN-2914. [YARN-1492] Potential race condition in Singleton implementation of SharedCacheUploaderMetrics, CleanerMetrics, ClientSCMMetrics. (Varun Saxena via kasha) 
(cherry picked from commit e597249d36)
 2014-12-15 12:00:46 -08:00
Jian He d2356a552a Revert "YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai." 
This reverts commit 51af8d367d.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
(cherry picked from commit a4f2995b9e)
 2014-12-15 11:08:44 -08:00
Devaraj K 4c6760bc6b YARN-2356. yarn status command for non-existent application/application 
attempt/container is too verbose. Contributed by Sunil G.

(cherry picked from commit fae3e8614f)
 2014-12-15 14:46:27 +05:30
Harsh J da36252077 YARN-2950. Change message to mandate, not suggest JS requirement on UI. Contributed by Dustin Cote. 
(cherry picked from commit 0e37bbc8e3)
 2014-12-13 07:27:37 +05:30
Steve Loughran a858d726c8 YARN-2912 Jersey Tests failing with port in use. (varun saxena via stevel) 2014-12-12 17:10:54 +00:00
Devaraj K 10ff600b35 YARN-2243. Order of arguments for Preconditions.checkNotNull() is wrong in 
SchedulerApplicationAttempt ctor. Contributed by Devaraj K.

(cherry picked from commit bda748ac3a)
 2014-12-12 12:37:13 +05:30
Jian He 41f0d20fcb YARN-2917. Fixed potential deadlock when system.exit is called in AsyncDispatcher. Contributed by Rohith Sharmaks 
(cherry picked from commit 614b6afea4)
 2014-12-11 11:18:45 -08:00
Jian He b31dfb17d8 YARN-2924. Fixed RMAdminCLI to not convert node labels to lower case. Contributed by Wangda Tan 
(cherry picked from commit 437322afca)
 2014-12-09 17:56:41 -08:00
Jian He 0f47e4e745 YARN-2930. Fixed TestRMRestart#testRMRestartRecoveringNodeLabelManager intermittent failure. Contributed by Wangda Tan 
(cherry picked from commit 2ed90a57fd)
 2014-12-09 16:48:46 -08:00
Karthik Kambatla 1986ea8dd2 YARN-2910. FSLeafQueue can throw ConcurrentModificationException. (Wilfred Spiegelenburg via kasha) 
(cherry picked from commit a2e07a5456)
 2014-12-09 14:00:52 -08:00
Karthik Kambatla 9d72b0282f YARN-2931. PublicLocalizer may fail until directory is initialized by LocalizeRunner. (Anubhav Dhoot via kasha) 
(cherry picked from commit db73cc9124)
 2014-12-08 22:26:44 -08:00
Karthik Kambatla d02cb9c512 YARN-2927. [YARN-1492] InMemorySCMStore properties are inconsistent. (Ray Chiang via kasha) 
(cherry picked from commit 120e1decd7)
 2014-12-07 22:28:57 -08:00
Jian He 4cc0abe4fe YARN-2869. CapacityScheduler should trim sub queue names when parse configuration. Contributed by Wangda Tan 
(cherry picked from commit e69af836f3)
 2014-12-05 17:34:15 -08:00
Jason Lowe b72fb6c774 YARN-2056. Disable preemption at Queue level. Contributed by Eric Payne 
(cherry picked from commit 4b13082199)
 2014-12-05 21:08:14 +00:00
Robert Kanter 2ebd1bc654 YARN-2461. Fix PROCFS_USE_SMAPS_BASED_RSS_ENABLED property in YarnConfiguration. (rchiang via rkanter) 
(cherry picked from commit 3c72f54ef5)
 2014-12-05 12:10:20 -08:00
Karthik Kambatla aadd0c392b YARN-2189. [YARN-1492] Admin service for cache manager. (Chris Trezzo via kasha) 
(cherry picked from commit 78968155d7)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/bin/yarn
 2014-12-04 19:48:18 -08:00
Jian He 1d1e7682c9 YARN-2301. Improved yarn container command. Contributed by Naganarasimha G R 
(cherry picked from commit 258623ff8b)
 2014-12-04 12:53:52 -08:00
Jian He 58c971164c YARN-2880. Added a test to make sure node labels will be recovered if RM restart is enabled. Contributed by Rohith Sharmaks 
(cherry picked from commit 73fbb3c66b)
 2014-12-03 17:17:40 -08:00
Harsh J ee25c0d890 YARN-2891. Failed Container Executor does not provide a clear error message. Contributed by Dustin Cote. (harsh) 
(cherry picked from commit 4b13318dea7a1cbbbfc1f84207af829cbe2f720e)
 2014-12-04 03:19:01 +05:30
Karthik Kambatla 25be97808b YARN-2874. Dead lock in DelegationTokenRenewer which blocks RM to execute any further apps. (Naganarasimha G R via kasha) 
(cherry picked from commit 799353e2c7)
 2014-12-03 13:45:03 -08:00
Jason Lowe db723a8499 MAPREDUCE-5932. Provide an option to use a dedicated reduce-side shuffle log. Contributed by Gera Shegalov 
(cherry picked from commit 03ab24aa01)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2014-12-03 17:12:47 +00:00
Junping Du 86535ff65f YARN-1156. Enhance NodeManager AllocatedGB and AvailableGB metrics for aggregation of decimal values. (Contributed by Tsuyoshi OZAWA) 
(cherry picked from commit e65b7c5ff6)
 2014-12-03 04:12:35 -08:00
Jian He d6f3d4893d YARN-2894. Fixed a bug regarding application view acl when RM fails over. Contributed by Rohith Sharmaks 
(cherry picked from commit 392c3aaea8)
 2014-12-02 17:17:16 -08:00
Jian He 5067ac098b YARN-2136. Changed RMStateStore to ignore store opearations when fenced. Contributed by Varun Saxena 
(cherry picked from commit 52bcefca8b)
 2014-12-02 11:02:00 -08:00
Zhijie Shen d21ef79707 YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai. 
(cherry picked from commit 51af8d367d)
 2014-12-01 19:08:31 -08:00
Jian He d208c90148 YARN-2765. Added leveldb-based implementation for RMStateStore. Contributed by Jason Lowe 
(cherry picked from commit a7fba0bc28)
 2014-12-01 16:45:17 -08:00
Jason Lowe 38ea1419f6 YARN-2905. AggregatedLogsBlock page can infinitely loop if the aggregated log file is corrupted. Contributed by Varun Saxena 
(cherry picked from commit 0f9528b99a)
 2014-12-01 22:30:20 +00:00
Junping Du c7bd22974a YARN-2907. SchedulerNode#toString should print all resource detail instead of only memory. (Contributed by Rohith) 
(cherry picked from commit c732ed760e)
 2014-12-01 05:41:56 -08:00
Zhijie Shen a9adc1295c YARN-2165. Added the sanity check for the numeric configuration values of the timeline service. Contributed by Vasanth kumar RJ. 
(cherry picked from commit 8f1454cc6d)
 2014-11-26 12:12:42 -08:00
Karthik Kambatla 3d4536af82 YARN-2188. [YARN-1492] Client service for cache manager. (Chris Trezzo and Sangjin Lee via kasha) 
(cherry picked from commit 834b3b98e7ecc4bfe93c7dd1a9e928e9653cf138)
 2014-11-25 16:22:14 -08:00
Jian He ae35b0e14d YARN-2906. CapacitySchedulerPage shows HTML tags for a queue's Active Users. Contributed by Jason Lowe 
(cherry picked from commit 8a7ca13b13)
 2014-11-25 16:18:02 -08:00
Jian He 2863056530 YARN-2404. Removed ApplicationAttemptState and ApplicationState class in RMStateStore. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 5805a81efb)
 2014-11-25 12:51:03 -08:00
Zhijie Shen 57d62d4ded YARN-2697. Remove useless RMAuthenticationHandler. Contributed by Haosong Huang. 
(cherry picked from commit e37a4ff0c1)
 2014-11-24 15:06:44 -08:00
Jason Lowe 89ef49fb08 YARN-1984. LeveldbTimelineStore does not handle db exceptions properly. Contributed by Varun Saxena 
(cherry picked from commit 1ce4d33c2d)
 2014-11-24 22:38:09 +00:00
Sandy Ryza 3e2e8eac22 YARN-2669. FairScheduler: queue names shouldn't allow periods (Wei Yan via Sandy Ryza) 2014-11-21 16:08:22 -08:00
Karthik Kambatla e0df934ed1 Update release date for 2.5.2 
(cherry picked from commit 1e9a3f42ff)
 2014-11-21 14:34:47 -08:00
Karthik Kambatla af0b54a4ee YARN-2679. Add metric for container launch duration. (Zhihai Xu via kasha) 
(cherry picked from commit 233b61e495)
 2014-11-21 14:22:53 -08:00
Karthik Kambatla e9db0aa35c YARN-2604. Scheduler should consider max-allocation-* in conjunction with the largest node. (Robert Kanter via kasha) 
(cherry picked from commit 3114d4731d)
 2014-11-21 10:42:47 -08:00
Jonathan Eagles 6570f7fd72 YARN-2375. Allow enabling/disabling timeline server per framework. (Mit Desai via jeagles) 
(cherry picked from commit c298a9a845)
 2014-11-20 23:40:26 -06:00
Karthik Kambatla 7ba5bb0c5b YARN-2315. FairScheduler: Set current capacity in addition to capacity. (Zhihai Xu via kasha) 
(cherry picked from commit a9a0cc3679)
 2014-11-19 20:19:54 -08:00
Karthik Kambatla 564deb75a8 YARN-2802. ClusterMetrics to include AM launch and register delays. (Zhihai Xu via kasha) 
(cherry picked from commit f9f8e750edbe6db453f0a845e2ed49ede66e0e8a)
 2014-11-19 19:50:35 -08:00
Jian He db31ef7e7f YARN-2865. Fixed RM to always create a new RMContext when transtions from StandBy to Active. Contributed by Rohith Sharmaks 
(cherry picked from commit 9cb8b75ba5)
 2014-11-19 19:49:44 -08:00
Jian He c359e2a6a4 YARN-2878. Fix DockerContainerExecutor.apt.vm formatting. Contributed by Abin Shahab 
(cherry picked from commit bc4ee5e06f)
 2014-11-19 14:45:01 -08:00
Zhijie Shen 610e123772 YARN-2870. Updated the command to run the timeline server it the document. Contributed by Masatake Iwasaki. 
(cherry picked from commit ef38fb9758)
 2014-11-18 16:17:07 -08:00
Jian He 1412e712b4 YARN-2157. Added YARN metrics in the documentaion. Contributed by Akira AJISAKA 
(cherry picked from commit 90a968d675)
 2014-11-18 16:13:08 -08:00
Karthik Kambatla 8df4c04c3a YARN-2690. [YARN-2574] Make ReservationSystem and its dependent classes independent of Scheduler type. (Anubhav Dhoot via kasha) 
(cherry picked from commit 2fce6d6141)
 2014-11-17 16:46:18 -08:00
Jason Lowe 242fd0e39a YARN-2414. RM web UI: app page will crash if app is failed before any attempt has been created. Contributed by Wangda Tan 
(cherry picked from commit 81c9d17af8)
 2014-11-17 21:16:54 +00:00
Vinod Kumar Vavilapalli 41275755c7 YARN-1703. Fixed ResourceManager web-proxy to close connections correctly. Contributed by Rohith Sharma. 
(cherry picked from commit 198fb58776)
 2014-11-15 12:06:53 -08:00
Jian He 21054c5d14 YARN-2432. RMStateStore should process the pending events before close. Contributed by Varun Saxena 
(cherry picked from commit 9b860661a5)
 2014-11-14 17:25:21 -08:00
Sandy Ryza c4c77669f0 YARN-2811. In Fair Scheduler, reservation fulfillments shouldn't ignore max share (Siqi Li via Sandy Ryza) 2014-11-14 15:20:37 -08:00
Jason Lowe ad140d1fc8 YARN-2816. NM fail to start with NPE during container recovery. Contributed by Zhihai Xu 
(cherry picked from commit 49c38898b0)
 2014-11-14 21:27:16 +00:00