Commit Graph

325 Commits

Author SHA1 Message Date
Xuan 53bddc410f YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma
(cherry picked from commit 8f08532bde)
2015-10-02 18:53:04 -07:00
Jian He d458805690 YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan
(cherry picked from commit 89cab1ba5f)

Conflicts:
	hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SLSCapacityScheduler.java
2015-09-24 10:30:04 -07:00
Jian He b6c594c6ea YARN-1644. RM-NM protocol changes and NodeStatusUpdater implementation to support container resizing. Contributed by Meng Ding
(cherry picked from commit c3dc1af072)
2015-09-24 10:24:04 -07:00
Jason Lowe b7111449a1 YARN-3975. WebAppProxyServlet should not redirect to RM page if AHS is enabled. Contributed by Mit Desai
(cherry picked from commit 692d51c09d)
2015-09-23 16:39:50 +00:00
Tsuyoshi Ozawa 52cc2ca488 HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang.
(cherry picked from commit dfd807afab)
2015-09-22 12:54:44 +09:00
Xuan f3c0a210b6 YARN-4149. yarn logs -am should provide an option to fetch all the log
files. Contributed by Varun Vasudev

(cherry picked from commit 34ef1a092b)
2015-09-15 14:37:59 -07:00
Wangda Tan c31fe79979 YARN-3717. Expose app/am/queue's node-label-expression to RM web UI / CLI / REST-API. (Naganarasimha G R via wangda)
(cherry picked from commit ae5308fe1d)
2015-09-15 11:41:57 -07:00
Junping Du df714e25aa YARN-313. Add Admin API for supporting node resource configuration in command line. (Contributed by Inigo Goiri, Kenji Kikushima and Junping Du)
(cherry picked from commit 73e3a49eb0)
2015-09-15 07:58:47 -07:00
Robert Kanter d07eef8c87 YARN-4115. Reduce loglevel of ContainerManagementProtocolProxy to Debug (adhoot via rkanter)
(cherry picked from commit b84fb41bb6)
2015-09-11 11:46:35 -07:00
Robert Kanter ea64a9b277 YARN-4086. Allow Aggregated Log readers to handle HAR files (rkanter)
(cherry picked from commit 6dd6ca442a)
2015-09-09 18:03:47 -07:00
Varun Vasudev a53054c853 YARN-3970. Add REST api support for Application Priority. Contributed by Naganarasimha G R.
(cherry picked from commit 223a306da925ad031bc398f2ea5e56f0e6521edf)
2015-09-03 16:40:42 +05:30
Jian He 48f5161cd5 YARN-4092. Fixed UI redirection to print useful messages when both RMs are in standby mode. Contributed by Xuan Gong
(cherry picked from commit a3fd2ccc86)
2015-08-31 17:34:49 -07:00
Jian He 306c535395 YARN-3250. Support admin cli interface in for Application Priority. Contributed by Rohith Sharma K S
(cherry picked from commit a9c8ea71aa)
2015-08-27 13:27:34 -07:00
Jian He 387076894f YARN-4014. Support user cli interface in for Application Priority. Contributed by Rohith Sharma K S
(cherry picked from commit 57c7ae1aff)
2015-08-24 20:37:18 -07:00
Karthik Kambatla b567aa2b4f YARN-4055. Report node resource utilization in heartbeat. (Inigo Goiri via kasha)
(cherry picked from commit 13604bd5f1)
2015-08-20 16:52:33 -07:00
rohithsharmaks c5b20e0db6 YARN-4023. Publish Application Priority to TimelineServer. (Sunil G via rohithsharmaks)
(cherry picked from commit 1c12adb71f)
2015-08-12 14:47:23 +05:30
Rohith Sharma K S a0da1ec010 YARN-3948. Display Application Priority in RM Web UI.(Sunil G via rohithsharmaks)
(cherry picked from commit b6265d39c5)
2015-08-07 10:47:08 +05:30
Arun Suresh 84ea7cd408 YARN-3961. Expose pending, running and reserved containers of a queue in REST api and yarn top (adhoot via asuresh)
(cherry picked from commit 154c9d2e42)
2015-08-05 23:15:33 -07:00
Xuan 8126fde884 YARN-3543. ApplicationReport should be able to tell whether the
Application is AM managed or not. Contributed by Rohith Sharma K S

(cherry picked from commit 0306d902f5)
2015-08-03 15:47:21 -07:00
Akira Ajisaka 0ba2defb9e YARN-3381. Fix typo InvalidStateTransitonException. Contributed by Brahma Reddy Battula.
(cherry picked from commit 19295b36d9)
2015-07-13 17:53:27 +09:00
Karthik Kambatla d15f69457b YARN-1012. Report NM aggregated container resource utilization in heartbeat. (Inigo Goiri via kasha)
(cherry picked from commit 527c40e4d6)
2015-07-09 09:35:39 -07:00
Junping Du dbd2b18205 YARN-3860. rmadmin -transitionToActive should check the state of non-target node. (Contributed by Masatake Iwasaki)
(cherry picked from commit a95d39f9d0)
2015-06-28 22:28:35 -07:00
Xuan 3f0573b059 YARN-3749. We should make a copy of configuration when init
MiniYARNCluster with multiple RMs. Contributed by Chun Chen

(cherry picked from commit 5766a04428)
2015-06-03 17:21:48 -07:00
Zhijie Shen 2fcd3fe6ba Revert "YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong."
This reverts commit 4e2ed34d20.
2015-06-03 14:16:51 -07:00
Zhijie Shen 4e2ed34d20 YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong.
(cherry picked from commit a9f986bc5a41ab74171da1076d10e83c5adc284f)
2015-05-30 21:06:32 -07:00
Wangda Tan 4790495538 YARN-3581. Deprecate -directlyAccessNodeLabelStore in RMAdminCLI. (Naganarasimha G R via wangda)
(cherry picked from commit cab7674e54)
2015-05-27 15:37:14 -07:00
Devaraj K 157ecb2241 YARN-3646. Applications are getting stuck some times in case of retry
policy forever. Contributed by Raju Bairishetti.

(cherry picked from commit 0305316d69)
2015-05-21 20:16:09 +05:30
Wangda Tan b5ba6979b7 YARN-3583. Support of NodeLabel object instead of plain String in YarnClient side. (Sunil G via wangda)
(cherry picked from commit 563eb1ad2a)
(cherry picked from commit b0d22b0c606fad6b4ab5443c0aed07c829b46726)
2015-05-19 17:37:24 -07:00
Xuan d39039d54d YARN-3601. Fix UT TestRMFailover.testRMWebAppRedirect. Contributed by Weiwei Yang
(cherry picked from commit 5009ad4a7f)
2015-05-19 09:57:13 -07:00
Xuan 802676e1be YARN-3526. ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster. Contributed by Weiwei Yang
(cherry picked from commit b0ad644083)
2015-05-15 22:41:13 -07:00
Devaraj K bf8be56292 YARN-2784. Make POM project names consistent. Contributed by Rohith. 2015-05-08 19:26:11 +05:30
Vinod Kumar Vavilapalli 0ec6e7e7f5 YARN-3413. Changed Nodelabel attributes (like exclusivity) to be settable only via addToClusterNodeLabels but not changeable at runtime. (Wangda Tan via vinodkv)
(cherry picked from commit f5fe35e297)
2015-04-23 11:25:30 -07:00
Steve Loughran 63ccb9167e YARN-2605. [RM HA] Rest api endpoints doing redirect incorrectly. (Xuan Gong via stevel) 2015-04-23 09:22:23 +01:00
Junping Du 8de6045842 New parameter of CLI for decommissioning node gracefully in RMAdmin CLI. Contributed by Devaraj K
(cherry picked from commit fad9d7e85b)
2015-04-22 10:08:16 -07:00
Jian He 6b454aacc3 YARN-3497. ContainerManagementProtocolProxy modifies IPC timeout conf without making a copy. Contributed by Jason Lowe
(cherry picked from commit f967fd2f21)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
2015-04-20 10:40:09 -07:00
Jian He 9827da95bc YARN-3451. Display attempt start time and elapsed time on the web UI. Contributed by Rohith Sharmaks
(cherry picked from commit 6779467ab6)
2015-04-17 15:57:36 -07:00
Junping Du b244701fad YARN-1402. Update related Web UI and CLI with exposing client API to check log aggregation status. Contributed by Xuan Gong.
(cherry picked from commit 1db355a875)
2015-04-17 13:20:27 -07:00
Junping Du a64b8915de YARN-3347. Improve YARN log command to get AMContainer logs as well as running containers logs. Contributed by Xuan Gong.
(cherry picked from commit a77d628339)
2015-04-13 17:41:02 -07:00
Jian He d3daf9665c YARN-3348. Add a 'yarn top' tool to help understand cluster usage. Contributed by Varun Vasudev 2015-04-10 10:40:31 -07:00
Wangda Tan cba4ed1678 YARN-2495. Allow admin specify labels from each NM (Distributed configuration for node label). (Naganarasimha G R via wangda)
(cherry picked from commit 2a945d24f7)
2015-03-30 12:05:54 -07:00
Karthik Kambatla 2158cd2bef YARN-3397. yarn rmadmin should skip -failover. (J.Andreina via kasha)
(cherry picked from commit c906a1de72)
2015-03-25 07:42:45 -07:00
Harsh J e7702ac6a1 YARN-1880. Cleanup TestApplicationClientProtocolOnHA. Contributed by ozawa.
(cherry picked from commit 8a9bbf548dad82598354c24fcd4dc80a715da7ac)
2015-03-24 11:58:16 +05:30
Tsuyoshi Ozawa d6e3ef2506 YARN-1453. [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments. Contributed by Akira AJISAKA, Andrew Purtell, and Allen Wittenauer.
(cherry picked from commit 3da9a97cfb)
2015-03-16 23:20:17 +09:00
Zhijie Shen 426535007b YARN-1884. Added nodeHttpAddress into ContainerReport and fixed the link to NM web page. Contributed by Xuan Gong.
(cherry picked from commit 85f6d67fa7)
2015-03-11 19:39:46 -07:00
Tsuyoshi Ozawa b46f9e72db HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 2015-03-09 19:53:55 +09:00
Vinod Kumar Vavilapalli d9416317a9 Preparing for 2.8.0 development. Changing SNAPSHOT version to be 2.8 on branch-2. 2015-03-08 20:32:51 -07:00
Vinod Kumar Vavilapalli 9c8882546a YARN-2786. Created a yarn cluster CLI and seeded with one command for listing node-labels collection. Contributed by Wangda Tan. 2015-03-05 10:56:28 -08:00
Jason Lowe ab397194c3 YARN-3131. YarnClientImpl should check FAILED and KILLED state in submitApplication. Contributed by Chang Li
(cherry picked from commit 03cc22945e)
2015-03-04 18:05:20 +00:00
Devaraj K b9e82e27f5 YARN-2797. Add -help to yarn logs and nodes CLI command. Contributed by
Jagadesh Kiran N.

(cherry picked from commit b610c68d44)
2015-02-24 12:08:50 +05:30
Wangda Tan cd5eb9c1de YARN-3076. Add API/Implementation to YarnClient to retrieve label-to-node mapping (Varun Saxena via wangda)
(cherry picked from commit d49ae725d5)
2015-02-19 11:01:47 -08:00
Xuan 3c4b5606dc YARN-3164. RMAdmin command usage prints incorrect command name.
Contributed by Bibin A Chundatt

(cherry picked from commit 253035491c)
2015-02-12 21:40:42 -08:00
Xuan df6bee5a75 YARN-3179. Update use of Iterator to Iterable in RMAdminCLI and
CommonNodeLabelsManager. Contributed by Ray Chiang

(cherry picked from commit 2586915bb3)
2015-02-12 17:32:58 -08:00
Jason Lowe 9361fc8f70 YARN-3144. Configuration for making delegation token failures to timeline server not-fatal. Contributed by Jonathan Eagles
(cherry picked from commit 6f10434a5a)
2015-02-06 20:55:56 +00:00
Jian He 3ddafaa7c8 YARN-2694. Ensure only single node label specified in ResourceRequest. Contributed by Wangda Tan
(cherry picked from commit c1957fef29)
2015-02-06 11:34:56 -08:00
Zhijie Shen 2db56a4932 YARN-3123. Made YARN CLI show a single completed container even if the app is running. Contributed by Naganarasimha G R.
(cherry picked from commit 30510cff75)
2015-02-04 16:07:24 -08:00
Siddharth Seth c330bca833 YARN-1723. AMRMClientAsync missing blacklist addition and removal
functionality. Contributed by Bartosz Ługowski.
(cherry picked from commit e04e8fa810)
2015-02-04 12:53:31 -08:00
Zhijie Shen b51c0288de YARN-2808. Made YARN CLI list attempt’s finished containers of a running application. Contributed by Naganarasimha G R.
(cherry picked from commit c0ec4f4cc12208342d45236b8841887a20f7f88a)
2015-02-02 12:05:05 -08:00
Jian He 12522fd9cb YARN-3103. AMRMClientImpl does not update AMRM token properly. Contributed by Jason Lowe
(cherry picked from commit 6d2bdbd7da)
2015-01-28 15:24:10 -08:00
Wangda Tan 12060cb9a6 YARN-3028. Better syntax for replaceLabelsOnNode in RMAdmin CLI. Contributed by Rohith Sharmaks
(cherry picked from commit fd93e5387b)
2015-01-27 15:25:47 -08:00
Junping Du ef4d7b73b9 YARN-3070. TestRMAdminCLI#testHelp fails for transitionToActive command. (Contributed by Junping Du)
(cherry picked from commit 19cbce3898)
2015-01-18 23:09:24 -08:00
Karthik Kambatla d98b8f4bd3 YARN-2217. [YARN-1492] Shared cache client side changes. (Chris Trezzo via kasha)
(cherry picked from commit ba5116ec8e)
2015-01-15 14:32:35 +05:30
Junping Du 361fab14be YARN-2940. Fix new findbugs warnings in rest of the hadoop-yarn components. (Contributed by Li Lu)
(cherry picked from commit d468c9aaf1)
2014-12-23 20:06:53 -08:00
Junping Du ae3faab13e YARN-2970. NodeLabel operations in RMAdmin CLI get missing in help command. (Contributed by Varun Saxena)
(cherry picked from commit e1ee0d45ea)
2014-12-23 18:24:45 -08:00
Tsuyoshi Ozawa 31753da961 YARN-2977. Fixed intermittent TestNMClient failure. (Contributed by Junping Du) 2014-12-20 22:43:47 +09:00
Jian He 878350a0e8 YARN-2762. Fixed RMAdminCLI to trim and check node-label related arguments before sending to RM. Contributed by Rohith Sharmaks
(cherry picked from commit c65f1b382e)
2014-12-16 11:01:24 -08:00
Jian He d2356a552a Revert "YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai."
This reverts commit 51af8d367d.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
(cherry picked from commit a4f2995b9e)
2014-12-15 11:08:44 -08:00
Devaraj K 4c6760bc6b YARN-2356. yarn status command for non-existent application/application
attempt/container is too verbose. Contributed by Sunil G.

(cherry picked from commit fae3e8614f)
2014-12-15 14:46:27 +05:30
Jian He b31dfb17d8 YARN-2924. Fixed RMAdminCLI to not convert node labels to lower case. Contributed by Wangda Tan
(cherry picked from commit 437322afca)
2014-12-09 17:56:41 -08:00
Karthik Kambatla aadd0c392b YARN-2189. [YARN-1492] Admin service for cache manager. (Chris Trezzo via kasha)
(cherry picked from commit 78968155d7)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/bin/yarn
2014-12-04 19:48:18 -08:00
Jian He 1d1e7682c9 YARN-2301. Improved yarn container command. Contributed by Naganarasimha G R
(cherry picked from commit 258623ff8b)
2014-12-04 12:53:52 -08:00
Zhijie Shen d21ef79707 YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai.
(cherry picked from commit 51af8d367d)
2014-12-01 19:08:31 -08:00
Vinod Kumar Vavilapalli 9d6c0d7c9f YARN-2843. Fixed NodeLabelsManager to trim inputs for hosts and labels so as to make them work correctly. Contributed by Wangda Tan.
(cherry picked from commit 0fd97f9c19)
2014-11-11 12:45:59 -08:00
Arun C. Murthy adfb830a2b Preparing to release hadoop-2.6.0: Set version in branch-2 to 2.7.0-SNAPSHOT. 2014-11-09 19:19:02 -08:00
Arun C. Murthy 175d222bfc YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles.
(cherry picked from commit 43cd07b408)
2014-11-09 15:03:59 -08:00
Vinod Kumar Vavilapalli 7b19079657 YARN-2827. Fixed bugs in "yarn queue" CLI. Contributed by Wangda Tan. 2014-11-07 12:00:49 -08:00
Vinod Kumar Vavilapalli b1521c7c3c YARN-2647. Added a queue CLI for getting queue information. Contributed by Sunil Govind.
(cherry picked from commit 61effcbebc)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/bin/yarn
2014-11-06 20:57:55 -08:00
Jian He f92ff24f5e YARN-2579. Fixed a deadlock issue when EmbeddedElectorService and FatalEventDispatcher try to transition RM to StandBy at the same time. Contributed by Rohith Sharmaks
(cherry picked from commit 395275af86)
2014-11-05 17:03:01 -08:00
Haohui Mai aa09c4c342 HADOOP-10563. Remove the dependency of jsp in trunk. Contributed by Haohui Mai. 2014-11-04 14:52:51 -08:00
Vinod Kumar Vavilapalli 715c81ef6d YARN-2788. Fixed backwards compatiblity issues with log-aggregation feature that were caused when adding log-upload-time via YARN-2703. Contributed by Xuan Gong.
(cherry picked from commit 58e9f24e0f)
2014-11-03 13:19:34 -08:00
Vinod Kumar Vavilapalli 52c58115d2 YARN-2798. Fixed YarnClient to populate the renewer correctly for Timeline delegation tokens. Contributed by Zhijie Shen.
(cherry picked from commit 71fbb474f5)
2014-11-03 12:50:39 -08:00
Jian He a87f827ae4 YARN-2770. Added functionality to renew/cancel TimeLineDelegationToken. Contributed by Zhijie Shen
(cherry picked from commit 1b4be91866)
2014-10-31 13:17:19 -07:00
Vinod Kumar Vavilapalli 9b879d713a YARN-2778. Moved node-lables' reports to the yarn nodes CLI from the admin CLI. Contributed by Wangda Tan.
(cherry picked from commit b6c1188b85)
2014-10-31 10:19:14 -07:00
Vinod Kumar Vavilapalli 0ba8d8c19c YARN-2698. Moved some node label APIs to be correctly placed in client protocol. Contributed by Wangda Tan.
(cherry picked from commit e0233c16eb)
2014-10-30 23:02:06 -07:00
Xuan f862999a7e YARN-2723. Fix rmadmin -replaceLabelsOnNode does not correctly parse
port. Contributed by Naganarasimha G R
2014-10-25 12:47:09 -07:00
Jian He 9bd149978d YARN-2314. Disable ContainerManagementProtocolProxy cache by default to prevent creating thousands of threads in a large cluster. Contributed by Jason Lowe
(cherry picked from commit f44cf99599)
2014-10-24 23:06:00 -07:00
Zhijie Shen 1b81105143 YARN-2703. Added logUploadedTime into LogValue for better display. Contributed by Xuan Gong.
(cherry picked from commit f81dc3f995)
2014-10-24 14:12:17 -07:00
Zhijie Shen 3b03ea6b50 YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He.
(cherry picked from commit 0f3b6900be)
2014-10-23 22:03:30 -07:00
Jian He b52928a6ea Missing file for YARN-1879
(cherry picked from commit 4a78a75228)
2014-10-20 21:14:57 -07:00
Zhijie Shen f75570734b YARN-2582. Fixed Log CLI and Web UI for showing aggregated logs of LRS. Contributed Xuan Gong.
(cherry picked from commit e90718fa5a)
2014-10-20 13:42:22 -07:00
Vinod Kumar Vavilapalli 9915d52185 YARN-2504. Enhanced RM Admin CLI to support management of node-labels. Contribyted by Wangda Tan.
(cherry picked from commit 8256766498)
2014-10-18 12:08:28 -07:00
Jian He 78a57087bf YARN-1879. Marked Idempotent/AtMostOnce annotations to ApplicationMasterProtocol for RM fail over. Contributed by Tsuyoshi OZAWA
(cherry picked from commit c3de2412eb)
2014-10-17 16:36:15 -07:00
Jian He f93d2ea27e YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 2014-10-15 15:28:26 -07:00
Vinod Kumar Vavilapalli fff6a1f77e YARN-2501. Enhanced AMRMClient library to support requests against node labels. Contributed by Wangda Tan.
(cherry picked from commit a5ec3d0809)
2014-10-10 19:58:42 -07:00
Vinod Kumar Vavilapalli d3125f3de5 YARN-2544. Added admin-API objects for using node-labels. Contributed by Wangda Tan.
(cherry picked from commit 596702a025)
2014-10-09 15:47:20 -07:00
subru ae0f16ccc8 YARN-2576. Making test patch pass in branch. Contributed by Subru Krishnan and Carlo Curino.
(cherry picked from commit 90ac0be86b)
(cherry picked from commit 5e10a13bb4)
2014-10-06 10:21:06 -07:00
subru cbfbdf60d6 YARN-2080. Integrating reservation system with ResourceManager and client-RM protocol. Contributed by Subru Krishnan and Carlo Curino.
(cherry picked from commit 8baeaead85)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
(cherry picked from commit 6261f7cc69)
2014-10-06 10:21:06 -07:00
junping_du 625456746c YARN-2613. Support retry in NMClient for rolling-upgrades. (Contributed by Jian He) 2014-10-01 17:08:55 -07:00
Jian He cb08ed1484 YARN-668. Changed NMTokenIdentifier/AMRMTokenIdentifier/ContainerTokenIdentifier to use protobuf object as the payload. Contributed by Junping Du.
(cherry picked from commit 5391919b09)
2014-09-26 17:53:35 -07:00
Zhijie Shen 9333ee3068 YARN-2568. Fixed the potential test failures due to race conditions when RM work-preserving recovery is enabled. Contributed by Jian He.
(cherry picked from commit 6fe5c6b746)
2014-09-18 22:02:53 -07:00
Jian He 4cabeef931 YARN-2563. Fixed YarnClient to call getTimeLineDelegationToken only if the Token is not present. Contributed by Zhijie Shen
(cherry picked from commit eb92cc67df)
2014-09-18 14:41:14 -07:00