Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,019 Commits 326 Branches 371 Tags
Commit Graph

230 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli
08a194fb55 YARN-1775. Enhanced ProcfsBasedProcessTree to optionally add the ability to use smaps for obtaining used memory information. Contributed by Rajesh Balamohan. 
svn merge --ignore-ancestry -c 1580087 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580088 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-22 00:02:26 +00:00
Vinod Kumar Vavilapalli
e1bda693dd YARN-1811. Fixed AMFilters in YARN to correctly accept requests from either web-app proxy or the RMs when HA is enabled. Contributed by Robert Kanter. 
svn merge --ignore-ancestry -c 1579877 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1579878 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-21 04:21:14 +00:00
Vinod Kumar Vavilapalli
6aae750762 YARN-1685. Fixed few bugs related to handling of containers' log-URLs on ResourceManager and history-service. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1578602 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578603 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 21:36:51 +00:00
Vinod Kumar Vavilapalli
d470c7b71a YARN-1824. Improved NodeManager and clients to be able to handle cross platform application submissions. Contributed by Jian He. 
MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to handle cross platform application submissions. Contributed by Jian He.
svn merge --ignore-ancestry -c 1578135 ../../trunk/ with a couple of minor edits for working in branch-2.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578139 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-16 19:13:16 +00:00
Zhijie Shen
f1947fcb28 YARN-1717. Enabled periodically discarding old data in LeveldbTimelineStore. Contributed by Billie Rinaldi. 
svn merge --ignore-ancestry -c 1577693 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577695 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 20:37:10 +00:00
Zhijie Shen
7834c7f193 YARN-1389. Made ApplicationClientProtocol and ApplicationHistoryProtocol expose analogous getApplication(s)/Attempt(s)/Container(s) APIs. Contributed by Mayank Bansal. 
svn merge --ignore-ancestry -c 1577052 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577053 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-13 07:45:30 +00:00
Vinod Kumar Vavilapalli
cbe653f9a2 YARN-1764. Modified YarnClient to correctly handle failover of ResourceManager after the submitApplication call goes through. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1576160 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576162 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-11 00:52:33 +00:00
Vinod Kumar Vavilapalli
71a8953ae9 YARN-1410. Added tests to validate that clients can fail-over to a new RM 
after getting an application-ID but before submission and can still submit
to the newly active RM with no issues. Contributed by Xuan Gong.
svn merge --ignore-ancestry -c 1575478 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1575479 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 04:44:02 +00:00
Vinod Kumar Vavilapalli
2fbec50fed YARN-1781. Modified NodeManagers to allow admins to specify max disk utilization for local disks so as to be able to offline full disks. Contributed by Varun Vasudev. 
svn merge --ignore-ancestry -c 1575463 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1575464 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 00:52:32 +00:00
Jian He
296ad40766 Merge r1574623 from trunk to branch-2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1574625 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 19:09:01 +00:00
Vinod Kumar Vavilapalli
7c17cff55c YARN-986. Changed client side to be able to figure out the right RM Delegation token for the right ResourceManager when HA is enabled. Contributed by Karthik Kambatla. 
svn merge --ignore-ancestry -c 1574190 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1574191 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 20:39:33 +00:00
Vinod Kumar Vavilapalli
f2d48add16 YARN-1730. Implemented simple write-locking in the LevelDB based timeline-store. Contributed by Billie Rinaldi. 
svn merge --ignore-ancestry -c 1574145 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1574146 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 17:34:01 +00:00
Karthik Kambatla
052e6e7fa6 YARN-1528. Allow setting auth for ZK connections. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1573017 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-28 17:36:04 +00:00
Vinod Kumar Vavilapalli
3406b14755 YARN-1588. Enhanced RM and the scheduling protocol to also send NMTokens of transferred containers from previous app-attempts to new AMs after YARN-1490. Contributed by Jian He. 
svn merge --ignore-ancestry -c 1572230 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1572232 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 20:20:39 +00:00
Vinod Kumar Vavilapalli
e23022e096 YARN-1742. Fixed javadoc of configuration parameter DEFAULT_NM_MIN_HEALTHY_DISKS_FRACTION. Contributed by Akira Ajisaka. 
svn merge --ignore-ancestry -c 1570956 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1570957 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-23 02:33:15 +00:00
Vinod Kumar Vavilapalli
ece585643c YARN-1749. Updated application-history related configs to reflect the latest reality and to be consistently named. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1570948 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1570949 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-23 02:05:28 +00:00
Vinod Kumar Vavilapalli
507ddad47f YARN-1687. Renamed user-facing records for the timeline-service to be simply named after 'timeline' instead of 'apptimeline'. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1570922 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1570924 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-22 20:55:42 +00:00
Vinod Kumar Vavilapalli
bd58e30fd6 YARN-1732. Changed types of related-entities and primary-filters in the timeline-service to be sets instead of maps. Contributed by Billie Rinaldi. 
svn merge --ignore-ancestry -c 1570914 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1570915 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-22 20:47:24 +00:00
Vinod Kumar Vavilapalli
0bbc1f4d95 YARN-1666. Modified RM HA handling of include/exclude node-lists to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1569856 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1569857 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 17:53:45 +00:00
Vinod Kumar Vavilapalli
eeae0cd829 YARN-1590. Fixed ResourceManager, web-app proxy and MR JobHistoryServer to expand _HOST properly in their kerberos principles. Contributed by Mohammad Kamrul Islam. 
svn merge --ignore-ancestry -c 1569537 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1569539 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-18 21:51:06 +00:00
Arun Murthy
1ccccb8534 Branching for hadoop-2.4 releases, also set version to 2.5.0-SNAPSHOT in branch-2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1569186 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-18 04:41:01 +00:00
Vinod Kumar Vavilapalli
7e252cdc9d YARN-1553. Modified YARN and MR to stop using HttpConfig.isSecure() and 
instead rely on the http policy framework. And also fix some bugs related
to https handling in YARN web-apps. Contributed by Haohui Mai.
svn merge --ignore-ancestry -c 1568501 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1568503 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 20:01:32 +00:00
Jian He
d05f300504 Merge r1567820 from trunk. YARN-1345. Remove FINAL_SAVING state from YarnApplicationAttemptState. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1567821 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 01:01:40 +00:00
Sanford Ryza
81bff204c3 YARN-1496. Protocol additions to allow moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1567479 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-12 01:00:08 +00:00
Vinod Kumar Vavilapalli
3cd8219519 YARN-1459. Changed ResourceManager to depend its service initialization on the configuration-provider mechanism during startup too. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1566791 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1566793 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-10 22:51:30 +00:00
Zhijie Shen
c9b6c3c36c YARN-1635. Implemented a Leveldb based ApplicationTimelineStore. Contributed by Billie Rinaldi. 
svn merge --ignore-ancestry -c 1565868 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1565869 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-08 02:18:06 +00:00
Karthik Kambatla
49389403d6 YARN-1672. YarnConfiguration is missing a default for yarn.nodemanager.log.retain-seconds (Naren Koneru via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1565867 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-08 01:56:10 +00:00
Vinod Kumar Vavilapalli
2ad6a71880 YARN-1660. Simplified the RM HA configuration to accept and be able to simply depend just on 
configuration properties of the form yarn.resourcemanager.hostname.RMID and use the default
ports for all service addresses. Contributed by Xuan Gong.
svn merge --ignore-ancestry -c 1565523 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1565524 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 03:19:04 +00:00
Vinod Kumar Vavilapalli
c16df11991 YARN-1665. Simplify the configuration of RM HA by having better default values. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1565517 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1565518 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 02:40:02 +00:00
Vinod Kumar Vavilapalli
a30e1889a6 YARN-1636. Augmented Application-history server's web-services to also expose new APIs for retrieving and storing timeline information. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1564829 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1564830 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 17:03:01 +00:00
Zhijie Shen
2017bac9e4 YARN-1461. Added tags for YARN applications and changed RM to handle them. Contributed by Karthik Kambatla. 
svn merge --ignore-ancestry -c 1564633 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1564634 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 04:58:50 +00:00
Vinod Kumar Vavilapalli
cc582a1e45 YARN-1634. Added a testable in-memory implementation of ApplicationTimelineStore. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1564583 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1564585 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 00:32:40 +00:00
Vinod Kumar Vavilapalli
afa5cebbc1 YARN-1669. Modified RM HA handling of protocol level service-ACLS to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1564549 ../../trunk/
Ran into minor import related conflicts that I merged manually.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1564552 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-04 22:57:37 +00:00
Vinod Kumar Vavilapalli
615716010e YARN-1667. Modified RM HA handling of super users (with proxying ability) to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1564100 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1564101 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 22:11:22 +00:00
Vinod Kumar Vavilapalli
884958ce31 YARN-1668. Modified RM HA handling of admin-acls to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1564043 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1564044 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 20:16:56 +00:00
Vinod Kumar Vavilapalli
fb892d8f84 YARN-1659. Defined the ApplicationTimelineStore store as an abstraction for implementing different storage impls for storing timeline information. Contributed by Billie Rinaldi. 
svn merge --ignore-ancestry -c 1564037 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1564038 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 20:07:55 +00:00
Vinod Kumar Vavilapalli
bd84052ff6 YARN-1639. Modified RM HA configuration handling to have a way of not requiring separate configuration files for each RM. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1564032 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1564033 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:55:58 +00:00
Vinod Kumar Vavilapalli
d083be9bc5 YARN-1611. Introduced the concept of a configuration provider which can be used by ResourceManager to read configuration locally or from remote systems so as to help RM failover. Contributed by Xuan Gong. 
svn merge -c 1564002 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1564003 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:08:26 +00:00
Vinod Kumar Vavilapalli
a420b40e00 YARN-1633. Defined user-facing entity, entity-info and event objects related to Application Timeline feature. Contributed by Zhijie Shen. 
svn merge -c 1563356 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1563357 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-01 00:10:26 +00:00
Sanford Ryza
a01eba048b YARN-1630. Introduce timeout for async polling operations in YarnClientImpl (Aditya Acharya via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562290 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 22:30:45 +00:00
Vinod Kumar Vavilapalli
060411928c YARN-1613. Fixed the typo with the configuration name YARN_HISTORY_SERVICE_ENABLED. Contributed by Akira Ajisaka. 
svn merge --ignore-ancestry -c 1561461 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562226 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 20:51:58 +00:00
Vinod Kumar Vavilapalli
1c8e403fdd YARN-1605. Fixed formatting issues in the new module on branch YARN-321. Contributed by Vinod Kumar Vavilapalli. 
svn merge --ignore-ancestry -c 1559004 ../YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562219 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 20:19:53 +00:00
Vinod Kumar Vavilapalli
4a3b6b7367 YARN-1595. Made enabling history service configurable and fixed test failures on branch YARN-321. Contributed by Vinod Kumar Vavilapalli. 
svn merge --ignore-ancestry -c 1559001 ../YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562218 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 20:18:42 +00:00
Vinod Kumar Vavilapalli
ff83e97655 YARN-1596. Fixed Javadoc warnings on branch YARN-321. Contributed by Vinod Kumar Vavilapalli. 
svn merge --ignore-ancestry -c 1557957 ../YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562215 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 20:14:54 +00:00
Vinod Kumar Vavilapalli
b7a3d0ae93 YARN-967. Added the client and CLI interfaces for obtaining ApplicationHistory data. Contributed by Mayank Bansal. 
svn merge --ignore-ancestry -c 1556747  ../YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562202 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 20:00:38 +00:00
Vinod Kumar Vavilapalli
20664c0460 YARN-954. Implemented web UI for the ApplicationHistoryServer and wired it into the HistoryStorage. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1556745 ../YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562200 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 19:59:17 +00:00
Vinod Kumar Vavilapalli
3d227e9544 YARN-955. Implemented ApplicationHistoryProtocol handler. Contributed by Mayank Bansal. 
svn merge --ignore-ancestry -c 1556743 ../YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562198 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 19:58:07 +00:00
Vinod Kumar Vavilapalli
cee917ace9 YARN-1266. Implemented PB service and client wrappers for ApplicationHistoryProtocol. Contributed by Mayank Bansal. 
svn merge --ignore-ancestry -c 1556739 ../YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562193 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 19:57:00 +00:00
Vinod Kumar Vavilapalli
0f87e4928b YARN-953. Changed ResourceManager to start writing history data. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1556738 ../YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562192 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 19:56:28 +00:00
Vinod Kumar Vavilapalli
0d2e62ae04 YARN-987. Added ApplicationHistoryManager responsible for exposing reports to all clients. Contributed by Mayank Bansal. 
svn merge --ignore-ancestry -c 1556736 ../YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562189 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 19:53:54 +00:00