Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,344 Commits 326 Branches 371 Tags
Commit Graph

1868 Commits

Author SHA1 Message Date
Karthik Kambatla
429754de09 YARN-3259. FairScheduler: Trigger fairShare updates on node events. (Anubhav Dhoot via kasha) 
(cherry picked from commit 75885852cc19dd6de12e62498b112d5d70ce87f4)
 2015-06-05 09:42:12 -07:00
Zhijie Shen
8f04ac9c34 YARN-3766. Fixed the apps table column error of generic history web UI. Contributed by Xuan Gong. 
(cherry picked from commit 18dd01d6bf67f4d522b947454c1f4347d1cbbc19)
 2015-06-04 14:48:05 -07:00
Wangda Tan
344b750915 Add missing test file of YARN-3733 
(cherry picked from commit 405bbcf68c32d8fd8a83e46e686eacd14e5a533c)
 2015-06-04 13:20:49 -07:00
Jian He
2e99210e4e YARN-2392. Add more diags about app retry limits on AM failures. Contributed by Steve Loughran 
(cherry picked from commit 1970ca7cbcdb7efa160d0cedc2e3e22c1401fad6)
 2015-06-04 11:14:36 -07:00
Jian He
6325e4b7dd YARN-3764. CapacityScheduler should forbid moving LeafQueue from one parent to another. Contributed by Wangda Tan 
(cherry picked from commit 6ad4e59cfc111a92747fdb1fb99cc6378044832a)
 2015-06-04 10:53:24 -07:00
Wangda Tan
78d626fa89 YARN-3733. Fix DominantRC#compare() does not work as expected if cluster resource is empty. (Rohith Sharmaks via wangda) 
(cherry picked from commit ebd797c48fe236b404cf3a125ac9d1f7714e291e)
 2015-06-04 10:23:33 -07:00
Junping Du
d8c7ee1b53 YARN-41. The RM should handle the graceful shutdown of the NM. Contributed by Devaraj K. 
(cherry picked from commit d7e7f6aa03c67b6a6ccf664adcb06d90bc963e58)
 2015-06-04 05:00:32 -07:00
Xuan
3f0573b059 YARN-3749. We should make a copy of configuration when init 
MiniYARNCluster with multiple RMs. Contributed by Chun Chen

(cherry picked from commit 5766a04428f65bb008b5c451f6f09e61e1000300)
 2015-06-03 17:21:48 -07:00
Zhijie Shen
2fcd3fe6ba Revert "YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong." 
This reverts commit 4e2ed34d2023a7cd3842e41aba1995f5d0f53cf2.
 2015-06-03 14:16:51 -07:00
Karthik Kambatla
62d51b889a YARN-3762. FairScheduler: CME on FSParentQueue#getQueueUserAclInfo. (kasha) 
(cherry picked from commit edb9cd0f7aa1ecaf34afaa120e3d79583e0ec689)
 2015-06-03 13:50:02 -07:00
Jason Lowe
752caa95a4 YARN-3585. NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled. Contributed by Rohith Sharmaks 
(cherry picked from commit e13b671aa510f553f4a6a232b4694b6a4cce88ae)
 2015-06-03 19:45:18 +00:00
Zhijie Shen
aaa567514a YARN-3751. Fixed AppInfo to check if used resources are null. Contributed by Sunil G. 
(cherry picked from commit dbc4f64937ea2b4c941a3ac49afc4eeba3f5b763)
 2015-06-03 11:52:39 -07:00
Wangda Tan
a3734f67d3 YARN-3725. App submission via REST API is broken in secure mode due to Timeline DT service address is empty. (Zhijie Shen via wangda) 
(cherry picked from commit 5cc3fced957a8471733e0e9490878bd68429fe24)
 2015-05-31 16:33:50 -07:00
Xuan
4563411e04 YARN-2900. Application (Attempt and Container) Not Found in AHS results 
in Internal Server Error (500). Contributed by Zhijie Shen and Mit Desai

(cherry picked from commit 9686261ecb872ad159fac3ca44f1792143c6d7db)
 2015-05-31 15:38:39 -07:00
Xuan
1ebaff0b86 Revert "YARN-2900. Application (Attempt and Container) Not Found in AHS results" 
This reverts commit 4fee8b320276bac86278e1ae0a3397592a78aa18.
 2015-05-31 00:24:08 -07:00
Xuan
4fee8b3202 YARN-2900. Application (Attempt and Container) Not Found in AHS results 
in Internal Server Error (500). Contributed by Zhijie Shen and Mit Desai

(cherry picked from commit 06f8e9cabaf3c05cd7d16215cff47265ea773f39)
 2015-05-30 21:58:04 -07:00
Zhijie Shen
4e2ed34d20 YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong. 
(cherry picked from commit a9f986bc5a41ab74171da1076d10e83c5adc284f)
 2015-05-30 21:06:32 -07:00
Karthik Kambatla
6c80e599d1 YARN-3467. Expose allocatedMB, allocatedVCores, and runningContainers metrics on running Applications in RM Web UI. (Anubhav Dhoot via kasha) 
(cherry picked from commit a8acdd65b3f0e8633050a1100136fd5e02ebdcfa)
 2015-05-30 13:36:49 -07:00
Zhijie Shen
68cddb894a YARN-3740. Fixed the typo in the configuration name: APPLICATION_HISTORY_PREFIX_MAX_APPS. Contributed by Xuan Gong. 
(cherry picked from commit eb6bf91eeacf97afb4cefe590f75ba94f3187d2b)
 2015-05-29 18:18:47 -07:00
Robert Kanter
bde4031a02 YARN-3713. Remove duplicate function call storeContainerDiagnostics in ContainerDiagnosticsUpdateTransition (zxu via rkanter) 
(cherry picked from commit 6aec13cb338b0fe62ca915f78aa729c9b0b86fba)
 2015-05-29 15:35:08 -07:00
Karthik Kambatla
b9c5c5de68 YARN-3547. FairScheduler: Apps that have no resource demand should not participate scheduling. (Xianyin Xin via kasha) 
(cherry picked from commit 3ae2a625018bc8cf431aa19da5bf8fe4ef8c1ad4)
 2015-05-29 15:17:43 -07:00
Andrew Wang
bdaa0fed13 HADOOP-11885. hadoop-dist dist-layout-stitching.sh does not work with dash. (wang) 
(cherry picked from commit 7673d4f205b26a6a26cfc47d999ece96f3c42782)

 Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml
	hadoop-mapreduce-project/pom.xml
 2015-05-29 14:55:52 -07:00
Wangda Tan
2fb794ecaf YARN-3716. Node-label-expression should be included by ResourceRequestPBImpl.toString. (Xianyin Xin via wangda) 
(cherry picked from commit 788bfa0359c1789fa48f21724a8117fe3fd9e532)
 2015-05-28 16:35:17 -07:00
Vinod Kumar Vavilapalli
148412bb30 Fixed more FilesSystemRMStateStore issues. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit 9acd24fec408c182683968ae2c75ca2c38866a60)
 2015-05-28 15:27:01 -07:00
Xuan
550b55146d YARN-3723. Need to clearly document primaryFilter and otherInfo value 
type. Contributed by Zhijie Shen

(cherry picked from commit 3077c299da4c5142503c9f92aad4b82349b2fde2)
 2015-05-28 10:20:33 -07:00
Devaraj K
7116141d7d YARN-3722. Merge multiple TestWebAppUtils into 
o.a.h.yarn.webapp.util.TestWebAppUtils. Contributed by Masatake Iwasaki.

(cherry picked from commit 7e509f58439b1089462f51f9a0f9782faec7d198)
 2015-05-28 15:36:13 +05:30
Zhijie Shen
839f81a632 YARN-3700. Made generic history service load a number of latest applications according to the parameter or the configuration. Contributed by Xuan Gong. 
(cherry picked from commit 54504133f41e36eaea6bb06c7b9ddb249468ecd7)
 2015-05-27 16:53:47 -07:00
Wangda Tan
4790495538 YARN-3581. Deprecate -directlyAccessNodeLabelStore in RMAdminCLI. (Naganarasimha G R via wangda) 
(cherry picked from commit cab7674e54c4fe56838668462de99a6787841309)
 2015-05-27 15:37:14 -07:00
cnauroth
6ed8989a6f YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch. 
(cherry picked from commit 7bc33b63c78c6ee9a04c0c1511ef16e63f7f3074)
 2015-05-27 14:21:15 -07:00
Wangda Tan
8857cba72e YARN-3647. RMWebServices api's should use updated api from CommonNodeLabelsManager to get NodeLabel object. (Sunil G via wangda) 
(cherry picked from commit ec0a852a37d5c91a62d3d0ff3ddbd9d58235b312)
 2015-05-27 13:15:55 -07:00
Wangda Tan
0728a86b3a YARN-3686. CapacityScheduler should trim default_node_label_expression. (Sunil G via wangda) 
(cherry picked from commit cdbd66be111c93c85a409d47284e588c453ecae9)
 2015-05-26 16:00:17 -07:00
Jian He
5e7be094ec YARN-3632. Ordering policy should be allowed to reorder an application when demand changes. Contributed by Craig Welch 
(cherry picked from commit 10732d515f62258309f98e4d7d23249f80b1847d)
 2015-05-26 12:02:36 -07:00
Vinod Kumar Vavilapalli
a67cb4826b YARN-160. Enhanced NodeManager to automatically obtain cpu/memory values from underlying OS when configured to do so. Contributed by Varun Vasudev. 
(cherry picked from commit 500a1d9c76ec612b4e737888f4be79951c11591d)
 2015-05-26 11:39:40 -07:00
Tsuyoshi Ozawa
cf5c1adf4f YARN-2336. Fair scheduler's REST API returns a missing '[' bracket JSON for deep queue tree. Contributed by Kenji Kikushima and Akira Ajisaka. 
(cherry picked from commit 9a3d617b6325d8918f2833c3e9ce329ecada9242)
 2015-05-26 19:08:01 +09:00
Xuan
84245ff3b2 YARN-2238. Filtering on UI sticks even if I move away from the page. 
Contributed by Jian He

(cherry picked from commit 39077dba2e877420e7470df253f6154f6ecc64ec)
 2015-05-25 22:41:02 -07:00
Jian He
03af057a47 YARN-3707. RM Web UI queue filter doesn't work. Contributed by Wangda Tan 
(cherry picked from commit 446d51591e6e99cc60a85c4b9fbac379a8caa49d)
 2015-05-22 15:59:00 -07:00
Xuan
33be070a5e YARN-3701. Isolating the error of generating a single app report when 
getting all apps from generic history service. Contributed by Zhijie
Shen

(cherry picked from commit 455b3acf0e82b214e06bd7b538968252945cd3c4)
 2015-05-22 14:35:51 -07:00
Vinayakumar B
19895a41ad HADOOP-11743. maven doesn't clean all the site files (Contributed by ramtin) 
(cherry picked from commit 55ed6558bad5d76ce4ebf958fc101a48032a0cd1)

Conflicts:
	hadoop-common-project/hadoop-common/pom.xml
 2015-05-22 16:46:50 +05:30
Junping Du
5fb61d71b1 YARN-3594. WintuilsProcessStubExecutor.startStreamReader leaks streams. Contributed by Lars Francke. 
(cherry picked from commit 132d909d4a6509af9e63e24cbb719be10006b6cd)
 2015-05-22 04:25:18 -07:00
Vinod Kumar Vavilapalli
673bd9708f YARN-3684. Changed ContainerExecutor's primary lifecycle methods to use a more extensible mechanism of context objects. Contributed by Sidharta Seethana. 
(cherry picked from commit 53fafcf061616516c24e2e2007a66a93d23d3e25)
 2015-05-21 15:51:19 -07:00
Karthik Kambatla
e8ac88d4fe YARN-3675. FairScheduler: RM quits when node removal races with continuous-scheduling on the same node. (Anubhav Dhoot via kasha) 
(cherry picked from commit a8b50e46737c11936ba72c427da69b2365a07aac)
 2015-05-21 13:39:40 -07:00
Devaraj K
157ecb2241 YARN-3646. Applications are getting stuck some times in case of retry 
policy forever. Contributed by Raju Bairishetti.

(cherry picked from commit 0305316d6932e6f1a05021354d77b6934e57e171)
 2015-05-21 20:16:09 +05:30
Jian He
7c3895c8f6 YARN-3654. ContainerLogsPage web UI should not have meta-refresh. Contributed by Xuan Gong 
(cherry picked from commit 6329bd00fa1f17cc9555efa496ea7607ad93e0ce)
 2015-05-20 17:21:19 -07:00
Jian He
521378ffa3 YARN-3609. Load node labels from storage inside RM serviceStart. Contributed by Wangda Tan 
(cherry picked from commit 8966d4217969eb71767ba83a3ff2b5bb38189b19)
 2015-05-20 16:30:44 -07:00
Xuan
a665b22cfa YARN-3681. yarn cmd says "could not find main class 'queue'" in windows. Contributed by Craig Welch and Varun Saxena 2015-05-20 14:42:59 -07:00
Wangda Tan
b5ba6979b7 YARN-3583. Support of NodeLabel object instead of plain String in YarnClient side. (Sunil G via wangda) 
(cherry picked from commit 563eb1ad2ae848a23bbbf32ebfaf107e8fa14e87)
(cherry picked from commit b0d22b0c606fad6b4ab5443c0aed07c829b46726)
 2015-05-19 17:37:24 -07:00
Tsuyoshi Ozawa
d022c5aba8 YARN-3677. Fix findbugs warnings in yarn-server-resourcemanager. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit 7401e5b5e8060b6b027d714b5ceb641fcfe5b598)
 2015-05-20 09:01:42 +09:00
Wangda Tan
7b97f049ee YARN-3565. NodeHeartbeatRequest/RegisterNodeManagerRequest should use NodeLabel object instead of String. (Naganarasimha G R via wangda) 
(cherry picked from commit b37da52a1c4fb3da2bd21bfadc5ec61c5f953a59)
 2015-05-19 16:35:23 -07:00
Jian He
adb90c7f52 YARN-2821. Fixed a problem that DistributedShell AM may hang if restarted. Contributed by Varun Vasudev 
(cherry picked from commit 7438966586f1896ab3e8b067d47a4af28a894106)
 2015-05-19 14:21:48 -07:00
Colin Patrick Mccabe
9534e533dc HADOOP-11970. Replace uses of ThreadLocal<Random> with JDK7 ThreadLocalRandom (Sean Busbey via Colin P. McCabe) 
(cherry picked from commit 470c87dbc6c24dd3b370f1ad9e7ab1f6dabd2080)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
        hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java
 2015-05-19 11:15:04 -07:00