Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,273 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

234 Commits

Author SHA1 Message Date
Karthik Kambatla a2cdf208dd YARN-1757. NM Recovery. Auxiliary service support. (Jason Lowe via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1585784 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-08 17:17:59 +00:00
Jian He 39871241a0 Merge r1583203 from trunk. YARN-1893. Mark AtMostOnce annotation to ApplicationMasterProtocol#allocate. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1583205 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-30 21:38:27 +00:00
Jian He 859f27040d Merge r1581678 from trunk. YARN-1521. Mark Idempotent/AtMostOnce annotations to the APIs in ApplicationClientProtcol, ResourceManagerAdministrationProtocol and ResourceTrackerProtocol so that they work in HA scenario. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1581679 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-26 03:48:46 +00:00
Vinod Kumar Vavilapalli 45d6277ec2 YARN-1850. Introduced the ability to optionally disable sending out timeline-events in the TimelineClient. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1581189 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1581190 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-25 02:23:08 +00:00
Vinod Kumar Vavilapalli 08a194fb55 YARN-1775. Enhanced ProcfsBasedProcessTree to optionally add the ability to use smaps for obtaining used memory information. Contributed by Rajesh Balamohan. 
svn merge --ignore-ancestry -c 1580087 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580088 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-22 00:02:26 +00:00
Vinod Kumar Vavilapalli e1bda693dd YARN-1811. Fixed AMFilters in YARN to correctly accept requests from either web-app proxy or the RMs when HA is enabled. Contributed by Robert Kanter. 
svn merge --ignore-ancestry -c 1579877 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1579878 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-21 04:21:14 +00:00
Vinod Kumar Vavilapalli 6aae750762 YARN-1685. Fixed few bugs related to handling of containers' log-URLs on ResourceManager and history-service. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1578602 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578603 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 21:36:51 +00:00
Vinod Kumar Vavilapalli d470c7b71a YARN-1824. Improved NodeManager and clients to be able to handle cross platform application submissions. Contributed by Jian He. 
MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to handle cross platform application submissions. Contributed by Jian He.
svn merge --ignore-ancestry -c 1578135 ../../trunk/ with a couple of minor edits for working in branch-2.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578139 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-16 19:13:16 +00:00
Zhijie Shen f1947fcb28 YARN-1717. Enabled periodically discarding old data in LeveldbTimelineStore. Contributed by Billie Rinaldi. 
svn merge --ignore-ancestry -c 1577693 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577695 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 20:37:10 +00:00
Zhijie Shen 7834c7f193 YARN-1389. Made ApplicationClientProtocol and ApplicationHistoryProtocol expose analogous getApplication(s)/Attempt(s)/Container(s) APIs. Contributed by Mayank Bansal. 
svn merge --ignore-ancestry -c 1577052 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577053 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-13 07:45:30 +00:00
Vinod Kumar Vavilapalli cbe653f9a2 YARN-1764. Modified YarnClient to correctly handle failover of ResourceManager after the submitApplication call goes through. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1576160 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576162 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-11 00:52:33 +00:00
Vinod Kumar Vavilapalli 71a8953ae9 YARN-1410. Added tests to validate that clients can fail-over to a new RM 
after getting an application-ID but before submission and can still submit
to the newly active RM with no issues. Contributed by Xuan Gong.
svn merge --ignore-ancestry -c 1575478 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1575479 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 04:44:02 +00:00
Vinod Kumar Vavilapalli 2fbec50fed YARN-1781. Modified NodeManagers to allow admins to specify max disk utilization for local disks so as to be able to offline full disks. Contributed by Varun Vasudev. 
svn merge --ignore-ancestry -c 1575463 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1575464 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 00:52:32 +00:00
Jian He 296ad40766 Merge r1574623 from trunk to branch-2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1574625 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 19:09:01 +00:00
Vinod Kumar Vavilapalli 7c17cff55c YARN-986. Changed client side to be able to figure out the right RM Delegation token for the right ResourceManager when HA is enabled. Contributed by Karthik Kambatla. 
svn merge --ignore-ancestry -c 1574190 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1574191 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 20:39:33 +00:00
Vinod Kumar Vavilapalli f2d48add16 YARN-1730. Implemented simple write-locking in the LevelDB based timeline-store. Contributed by Billie Rinaldi. 
svn merge --ignore-ancestry -c 1574145 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1574146 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 17:34:01 +00:00
Karthik Kambatla 052e6e7fa6 YARN-1528. Allow setting auth for ZK connections. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1573017 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-28 17:36:04 +00:00
Vinod Kumar Vavilapalli 3406b14755 YARN-1588. Enhanced RM and the scheduling protocol to also send NMTokens of transferred containers from previous app-attempts to new AMs after YARN-1490. Contributed by Jian He. 
svn merge --ignore-ancestry -c 1572230 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1572232 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 20:20:39 +00:00
Vinod Kumar Vavilapalli e23022e096 YARN-1742. Fixed javadoc of configuration parameter DEFAULT_NM_MIN_HEALTHY_DISKS_FRACTION. Contributed by Akira Ajisaka. 
svn merge --ignore-ancestry -c 1570956 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1570957 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-23 02:33:15 +00:00
Vinod Kumar Vavilapalli ece585643c YARN-1749. Updated application-history related configs to reflect the latest reality and to be consistently named. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1570948 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1570949 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-23 02:05:28 +00:00
Vinod Kumar Vavilapalli 507ddad47f YARN-1687. Renamed user-facing records for the timeline-service to be simply named after 'timeline' instead of 'apptimeline'. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1570922 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1570924 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-22 20:55:42 +00:00
Vinod Kumar Vavilapalli bd58e30fd6 YARN-1732. Changed types of related-entities and primary-filters in the timeline-service to be sets instead of maps. Contributed by Billie Rinaldi. 
svn merge --ignore-ancestry -c 1570914 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1570915 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-22 20:47:24 +00:00
Vinod Kumar Vavilapalli 0bbc1f4d95 YARN-1666. Modified RM HA handling of include/exclude node-lists to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1569856 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1569857 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 17:53:45 +00:00
Vinod Kumar Vavilapalli eeae0cd829 YARN-1590. Fixed ResourceManager, web-app proxy and MR JobHistoryServer to expand _HOST properly in their kerberos principles. Contributed by Mohammad Kamrul Islam. 
svn merge --ignore-ancestry -c 1569537 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1569539 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-18 21:51:06 +00:00
Arun Murthy 1ccccb8534 Branching for hadoop-2.4 releases, also set version to 2.5.0-SNAPSHOT in branch-2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1569186 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-18 04:41:01 +00:00
Vinod Kumar Vavilapalli 7e252cdc9d YARN-1553. Modified YARN and MR to stop using HttpConfig.isSecure() and 
instead rely on the http policy framework. And also fix some bugs related
to https handling in YARN web-apps. Contributed by Haohui Mai.
svn merge --ignore-ancestry -c 1568501 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1568503 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 20:01:32 +00:00
Jian He d05f300504 Merge r1567820 from trunk. YARN-1345. Remove FINAL_SAVING state from YarnApplicationAttemptState. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1567821 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 01:01:40 +00:00
Sanford Ryza 81bff204c3 YARN-1496. Protocol additions to allow moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1567479 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-12 01:00:08 +00:00
Vinod Kumar Vavilapalli 3cd8219519 YARN-1459. Changed ResourceManager to depend its service initialization on the configuration-provider mechanism during startup too. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1566791 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1566793 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-10 22:51:30 +00:00
Zhijie Shen c9b6c3c36c YARN-1635. Implemented a Leveldb based ApplicationTimelineStore. Contributed by Billie Rinaldi. 
svn merge --ignore-ancestry -c 1565868 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1565869 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-08 02:18:06 +00:00
Karthik Kambatla 49389403d6 YARN-1672. YarnConfiguration is missing a default for yarn.nodemanager.log.retain-seconds (Naren Koneru via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1565867 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-08 01:56:10 +00:00
Vinod Kumar Vavilapalli 2ad6a71880 YARN-1660. Simplified the RM HA configuration to accept and be able to simply depend just on 
configuration properties of the form yarn.resourcemanager.hostname.RMID and use the default
ports for all service addresses. Contributed by Xuan Gong.
svn merge --ignore-ancestry -c 1565523 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1565524 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 03:19:04 +00:00
Vinod Kumar Vavilapalli c16df11991 YARN-1665. Simplify the configuration of RM HA by having better default values. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1565517 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1565518 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 02:40:02 +00:00
Vinod Kumar Vavilapalli a30e1889a6 YARN-1636. Augmented Application-history server's web-services to also expose new APIs for retrieving and storing timeline information. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1564829 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1564830 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 17:03:01 +00:00
Zhijie Shen 2017bac9e4 YARN-1461. Added tags for YARN applications and changed RM to handle them. Contributed by Karthik Kambatla. 
svn merge --ignore-ancestry -c 1564633 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1564634 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 04:58:50 +00:00
Vinod Kumar Vavilapalli cc582a1e45 YARN-1634. Added a testable in-memory implementation of ApplicationTimelineStore. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1564583 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1564585 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 00:32:40 +00:00
Vinod Kumar Vavilapalli afa5cebbc1 YARN-1669. Modified RM HA handling of protocol level service-ACLS to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1564549 ../../trunk/
Ran into minor import related conflicts that I merged manually.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1564552 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-04 22:57:37 +00:00
Vinod Kumar Vavilapalli 615716010e YARN-1667. Modified RM HA handling of super users (with proxying ability) to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1564100 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1564101 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 22:11:22 +00:00
Vinod Kumar Vavilapalli 884958ce31 YARN-1668. Modified RM HA handling of admin-acls to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1564043 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1564044 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 20:16:56 +00:00
Vinod Kumar Vavilapalli fb892d8f84 YARN-1659. Defined the ApplicationTimelineStore store as an abstraction for implementing different storage impls for storing timeline information. Contributed by Billie Rinaldi. 
svn merge --ignore-ancestry -c 1564037 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1564038 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 20:07:55 +00:00
Vinod Kumar Vavilapalli bd84052ff6 YARN-1639. Modified RM HA configuration handling to have a way of not requiring separate configuration files for each RM. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1564032 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1564033 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:55:58 +00:00
Vinod Kumar Vavilapalli d083be9bc5 YARN-1611. Introduced the concept of a configuration provider which can be used by ResourceManager to read configuration locally or from remote systems so as to help RM failover. Contributed by Xuan Gong. 
svn merge -c 1564002 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1564003 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:08:26 +00:00
Vinod Kumar Vavilapalli a420b40e00 YARN-1633. Defined user-facing entity, entity-info and event objects related to Application Timeline feature. Contributed by Zhijie Shen. 
svn merge -c 1563356 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1563357 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-01 00:10:26 +00:00
Sanford Ryza a01eba048b YARN-1630. Introduce timeout for async polling operations in YarnClientImpl (Aditya Acharya via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562290 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 22:30:45 +00:00
Vinod Kumar Vavilapalli 060411928c YARN-1613. Fixed the typo with the configuration name YARN_HISTORY_SERVICE_ENABLED. Contributed by Akira Ajisaka. 
svn merge --ignore-ancestry -c 1561461 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562226 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 20:51:58 +00:00
Vinod Kumar Vavilapalli 1c8e403fdd YARN-1605. Fixed formatting issues in the new module on branch YARN-321. Contributed by Vinod Kumar Vavilapalli. 
svn merge --ignore-ancestry -c 1559004 ../YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562219 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 20:19:53 +00:00
Vinod Kumar Vavilapalli 4a3b6b7367 YARN-1595. Made enabling history service configurable and fixed test failures on branch YARN-321. Contributed by Vinod Kumar Vavilapalli. 
svn merge --ignore-ancestry -c 1559001 ../YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562218 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 20:18:42 +00:00
Vinod Kumar Vavilapalli ff83e97655 YARN-1596. Fixed Javadoc warnings on branch YARN-321. Contributed by Vinod Kumar Vavilapalli. 
svn merge --ignore-ancestry -c 1557957 ../YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562215 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 20:14:54 +00:00
Vinod Kumar Vavilapalli b7a3d0ae93 YARN-967. Added the client and CLI interfaces for obtaining ApplicationHistory data. Contributed by Mayank Bansal. 
svn merge --ignore-ancestry -c 1556747  ../YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562202 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 20:00:38 +00:00
Vinod Kumar Vavilapalli 20664c0460 YARN-954. Implemented web UI for the ApplicationHistoryServer and wired it into the HistoryStorage. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1556745 ../YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562200 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 19:59:17 +00:00