Commit Graph

719 Commits

Author SHA1 Message Date
Zhijie Shen befba101f3 YARN-1924. Made ZKRMStateStore updateApplication(Attempt)StateInternal work when Application(Attempt) state hasn't been stored before. Contributed by Jian He.
svn merge --ignore-ancestry -c 1586547 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1586548 13f79535-47bb-0310-9956-ffa450edef68
2014-04-11 03:38:27 +00:00
Jian He 9df6ddd282 Merge r1586522 from trunk. YARN-1903. Set exit code and diagnostics when container is killed at NEW/LOCALIZING state. Contributed by Zhijie Shen
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1586523 13f79535-47bb-0310-9956-ffa450edef68
2014-04-11 01:28:22 +00:00
Zhijie Shen 874b4979d0 YARN-1920. Fixed TestFileSystemApplicationHistoryStore failure on windows. Contributed by Vinod Kumar Vavilapalli.
svn merge --ignore-ancestry -c 1586414 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1586420 13f79535-47bb-0310-9956-ffa450edef68
2014-04-10 18:59:44 +00:00
Vinod Kumar Vavilapalli 614faccf1d YARN-1910. Fixed a race condition in TestAMRMTokens that causes the test to fail more often on Windows. Contributed by Xuan Gong.
svn merge --ignore-ancestry -c 1586192 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1586193 13f79535-47bb-0310-9956-ffa450edef68
2014-04-10 01:37:22 +00:00
Kihwal Lee f10bf59e6e svn merge -c 1585992 merging from trunk to branch-2 to fix:YARN-1907. TestRMApplicationHistoryWriter#testRMWritingMassiveHistory intermittently fails. Contributed by Mit Desai.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1585993 13f79535-47bb-0310-9956-ffa450edef68
2014-04-09 15:35:13 +00:00
Karthik Kambatla fa3c632fb2 YARN-1784. TestContainerAllocation assumes CapacityScheduler. (Robert Kanter via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1585895 13f79535-47bb-0310-9956-ffa450edef68
2014-04-09 05:55:11 +00:00
Karthik Kambatla a2cdf208dd YARN-1757. NM Recovery. Auxiliary service support. (Jason Lowe via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1585784 13f79535-47bb-0310-9956-ffa450edef68
2014-04-08 17:17:59 +00:00
Arun Murthy 243bb224a2 Merge -c 1584954 from trunk to branch-2 to fix YARN-1898. Addendum patch to ensure /jmx and /metrics are re-directed to Active RM.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1584955 13f79535-47bb-0310-9956-ffa450edef68
2014-04-05 02:52:35 +00:00
Jian He 85c733feee Merge r1584862 from trunk. YARN-1837. Fixed TestMoveApplication#testMoveRejectedByScheduler failure. Contributed by Hong Zhiguo
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1584863 13f79535-47bb-0310-9956-ffa450edef68
2014-04-04 18:43:59 +00:00
Zhijie Shen d5f1c9a044 YARN-1898. Made Standby RM links conf, stacks, logLevel, metrics, jmx, logs and static not be redirected to Active RM. Contributed by Xuan Gong.
svn merge --ignore-ancestry -c 1583833 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1583835 13f79535-47bb-0310-9956-ffa450edef68
2014-04-02 00:05:15 +00:00
Sanford Ryza 06d62b1982 YARN-1889. In Fair Scheduler, avoid creating objects on each call to AppSchedulable comparator (Hong Zhiguo via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1583494 13f79535-47bb-0310-9956-ffa450edef68
2014-03-31 23:53:58 +00:00
Jian He 39871241a0 Merge r1583203 from trunk. YARN-1893. Mark AtMostOnce annotation to ApplicationMasterProtocol#allocate. Contributed by Xuan Gong
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1583205 13f79535-47bb-0310-9956-ffa450edef68
2014-03-30 21:38:27 +00:00
Jonathan Turner Eagles a467e88ae4 YARN-1883. TestRMAdminService fails due to inconsistent entries in UserGroups (Mit Desai via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1582865 13f79535-47bb-0310-9956-ffa450edef68
2014-03-28 19:45:05 +00:00
Tsz-wo Sze a6c405b99f svn merge -c 1582856 from trunk for HDFS-6168. Remove a deprecated constructor and the deprecated methods reportChecksumFailure, getDelegationToken(Text), renewDelegationToken and cancelDelegationToken from DistributedFileSystem.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1582857 13f79535-47bb-0310-9956-ffa450edef68
2014-03-28 18:36:20 +00:00
Jian He 859f27040d Merge r1581678 from trunk. YARN-1521. Mark Idempotent/AtMostOnce annotations to the APIs in ApplicationClientProtcol, ResourceManagerAdministrationProtocol and ResourceTrackerProtocol so that they work in HA scenario. Contributed by Xuan Gong
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1581679 13f79535-47bb-0310-9956-ffa450edef68
2014-03-26 03:48:46 +00:00
Vinod Kumar Vavilapalli 7bd77e23d7 YARN-1867. Fixed a bug in ResourceManager that was causing invalid ACL checks in the web-services after fail-over. Contributed by Vinod Kumar Vavilapalli.
svn merge --ignore-ancestry -c 1581662 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1581664 13f79535-47bb-0310-9956-ffa450edef68
2014-03-26 03:04:02 +00:00
Vinod Kumar Vavilapalli dd8a9bbadf YARN-1866. Fixed an issue with renewal of RM-delegation tokens on restart or fail-over. Contributed by Jian He.
svn merge --ignore-ancestry -c 1581448 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1581450 13f79535-47bb-0310-9956-ffa450edef68
2014-03-25 18:22:59 +00:00
Jian He cb9fcd7f03 Merge r1580997 from trunk. YARN-1852. Fixed RMAppAttempt to not resend AttemptFailed/AttemptKilled events to already recovered Failed/Killed RMApps. Contributed by Rohith Sharmaks
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1581001 13f79535-47bb-0310-9956-ffa450edef68
2014-03-24 19:06:50 +00:00
Zhijie Shen 5f0f16c0f6 YARN-1838. Enhanced timeline service getEntities API to get entities from a given entity ID or insertion timestamp. Contributed by Billie Rinaldi.
svn merge --ignore-ancestry -c 1580960 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580961 13f79535-47bb-0310-9956-ffa450edef68
2014-03-24 18:16:27 +00:00
Zhijie Shen ebd254117d YARN-1577. Made UnmanagedAMLauncher do launchAM after the attempt reaches the LAUNCHED state. Contributed by Jian He.
svn merge --ignore-ancestry -c 1580164 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580165 13f79535-47bb-0310-9956-ffa450edef68
2014-03-22 04:37:46 +00:00
Jian He 21b5ec140a Merge r1580154 from trunk. YARN-1776. Fixed DelegationToken renewal to survive RM failover. Contributed by Zhijie Shen
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580156 13f79535-47bb-0310-9956-ffa450edef68
2014-03-22 04:14:42 +00:00
Karthik Kambatla 1bb24f2c18 YARN-1536. Cleanup: Get rid of ResourceManager#get*SecretManager() methods and use the RMContext methods instead. (Anubhav Dhoot via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580129 13f79535-47bb-0310-9956-ffa450edef68
2014-03-22 02:38:23 +00:00
Vinod Kumar Vavilapalli 8f3f0caee1 YARN-1854. Fixed test failure in TestRMHA#testStartAndTransitions. Contributed by Rohith Sharma KS.
svn merge --ignore-ancestry -c 1580097 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580098 13f79535-47bb-0310-9956-ffa450edef68
2014-03-22 00:30:11 +00:00
Vinod Kumar Vavilapalli 08a194fb55 YARN-1775. Enhanced ProcfsBasedProcessTree to optionally add the ability to use smaps for obtaining used memory information. Contributed by Rajesh Balamohan.
svn merge --ignore-ancestry -c 1580087 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580088 13f79535-47bb-0310-9956-ffa450edef68
2014-03-22 00:02:26 +00:00
Jian He b701a84d37 Merge r1580077 from trunk. YARN-1849. Fixed NPE in ResourceTrackerService#registerNodeManager for UAM. Contributed by Karthik Kambatla
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580078 13f79535-47bb-0310-9956-ffa450edef68
2014-03-21 22:57:09 +00:00
Vinod Kumar Vavilapalli e1bda693dd YARN-1811. Fixed AMFilters in YARN to correctly accept requests from either web-app proxy or the RMs when HA is enabled. Contributed by Robert Kanter.
svn merge --ignore-ancestry -c 1579877 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1579878 13f79535-47bb-0310-9956-ffa450edef68
2014-03-21 04:21:14 +00:00
Vinod Kumar Vavilapalli c7b39f75ba YARN-1859. Fixed WebAppProxyServlet to correctly handle applications absent on the ResourceManager. Contributed by Zhijie Shen.
svn merge --ignore-ancestry -c 1579866 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1579867 13f79535-47bb-0310-9956-ffa450edef68
2014-03-21 02:03:41 +00:00
Vinod Kumar Vavilapalli 2f8d99f642 YARN-1855. Made Application-history server to be optional in MiniYARNCluster and thus avoid the failure of TestRMFailover#testRMWebAppRedirect. Contributed by Zhijie Shen.
svn merge --ignore-ancestry -c 1579838 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1579839 13f79535-47bb-0310-9956-ffa450edef68
2014-03-21 00:18:56 +00:00
Vinod Kumar Vavilapalli af6b5b8e9a YARN-1640. Fixed manual failover of ResourceManagers to work correctly in secure clusters. Contributed by Xuan Gong.
svn merge --ignore-ancestry -c 1579510 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1579511 13f79535-47bb-0310-9956-ffa450edef68
2014-03-20 02:40:54 +00:00
Zhijie Shen e39ccf3b24 YARN-1690. Made DistributedShell send timeline entities+events. Contributed by Mayank Bansal.
svn merge --ignore-ancestry -c 1579123 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1579124 13f79535-47bb-0310-9956-ffa450edef68
2014-03-19 03:51:27 +00:00
Karthik Kambatla d33b320214 YARN-1705. Reset cluster-metrics on transition to standby. (Rohith via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1579015 13f79535-47bb-0310-9956-ffa450edef68
2014-03-18 19:03:24 +00:00
Karthik Kambatla de68e1d6de YARN-1846. TestRM#testNMTokenSentForNormalContainer assumes CapacityScheduler. (Robert Kanter via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578767 13f79535-47bb-0310-9956-ffa450edef68
2014-03-18 07:00:29 +00:00
Vinod Kumar Vavilapalli 0c6db6af6e YARN-1512. Enhanced CapacityScheduler to be able to decouple scheduling from node-heartbeats. Contributed by Arun C Murthy.
svn merge --ignore-ancestry -c 1578722 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578724 13f79535-47bb-0310-9956-ffa450edef68
2014-03-18 02:53:50 +00:00
Vinod Kumar Vavilapalli 80f3726024 YARN-1839. Fixed handling of NMTokens in ResourceManager such that containers launched by AMs running on the same machine as the AM are correctly propagated. Contributed by Jian He.
svn merge --ignore-ancestry -c 1578631 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578632 13f79535-47bb-0310-9956-ffa450edef68
2014-03-17 22:22:58 +00:00
Vinod Kumar Vavilapalli 597c7d054f YARN-1591. Fixed AsyncDispatcher to handle interrupts on shutdown in a sane manner and thus fix failure of TestResourceTrackerService. Contributed by Tsuyoshi Ozawa.
svn merge --ignore-ancestry -c 1578628 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578629 13f79535-47bb-0310-9956-ffa450edef68
2014-03-17 22:18:09 +00:00
Jian He 69835b9651 Merge r1578614 from trunk. Fixed AM container log to show on NM web page after application finishes if log-aggregation is disabled. Contributed by Rohith Sharmaks
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578618 13f79535-47bb-0310-9956-ffa450edef68
2014-03-17 21:52:41 +00:00
Vinod Kumar Vavilapalli 6aae750762 YARN-1685. Fixed few bugs related to handling of containers' log-URLs on ResourceManager and history-service. Contributed by Zhijie Shen.
svn merge --ignore-ancestry -c 1578602 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578603 13f79535-47bb-0310-9956-ffa450edef68
2014-03-17 21:36:51 +00:00
Jonathan Turner Eagles 8e38068076 YARN-1136. Replace junit.framework.Assert with org.junit.Assert (Chen He via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578546 13f79535-47bb-0310-9956-ffa450edef68
2014-03-17 20:20:22 +00:00
Jian He ed83fe61e2 Merge r1578486 from trunk. YARN-1830. Fixed TestRMRestart#testQueueMetricsOnRMRestart failure due to race condition when app is submitted. Contributed by Zhijie Shen
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578487 13f79535-47bb-0310-9956-ffa450edef68
2014-03-17 18:06:17 +00:00
Vinod Kumar Vavilapalli d470c7b71a YARN-1824. Improved NodeManager and clients to be able to handle cross platform application submissions. Contributed by Jian He.
MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to handle cross platform application submissions. Contributed by Jian He.
svn merge --ignore-ancestry -c 1578135 ../../trunk/ with a couple of minor edits for working in branch-2.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578139 13f79535-47bb-0310-9956-ffa450edef68
2014-03-16 19:13:16 +00:00
Jonathan Turner Eagles b3b33de0d2 YARN-1833. TestRMAdminService Fails in trunk and branch-2 (Mit Desais via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577743 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 22:15:02 +00:00
Zhijie Shen f1947fcb28 YARN-1717. Enabled periodically discarding old data in LeveldbTimelineStore. Contributed by Billie Rinaldi.
svn merge --ignore-ancestry -c 1577693 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577695 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 20:37:10 +00:00
Vinod Kumar Vavilapalli 3ac3e5f897 YARN-1658. Modified web-app framework to let standby RMs redirect web-service calls to the active RM. Contributed by Cindy Li.
svn merge --ignore-ancestry -c 1577408 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577409 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 02:40:28 +00:00
Christopher Douglas 9695bc7af6 YARN-1771. Reduce the number of NameNode operations during localization of
public resources using a cache. Contributed by Sangjin Lee



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577392 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 00:31:08 +00:00
Zhijie Shen 7834c7f193 YARN-1389. Made ApplicationClientProtocol and ApplicationHistoryProtocol expose analogous getApplication(s)/Attempt(s)/Container(s) APIs. Contributed by Mayank Bansal.
svn merge --ignore-ancestry -c 1577052 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577053 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 07:45:30 +00:00
Jason Darrell Lowe 16669efeea svn merge -c 1576960 FIXES: YARN-1789. ApplicationSummary does not escape newlines in the app name. Contributed by Tsuyoshi OZAWA
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576962 13f79535-47bb-0310-9956-ffa450edef68
2014-03-12 22:27:01 +00:00
Vinod Kumar Vavilapalli eac9b53bd8 YARN-1816. Fixed ResourceManager to get RMApp correctly handle ATTEMPT_FINISHED event at ACCEPTED state that can happen after RM restarts. Contributed by Jian He.
svn merge --ignore-ancestry -c 1576911 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576913 13f79535-47bb-0310-9956-ffa450edef68
2014-03-12 21:19:22 +00:00
Vinod Kumar Vavilapalli 15744c614b YARN-1812. Fixed ResourceManager to synchrously renew tokens after recovery and thus recover app itself synchronously and avoid races with resyncing NodeManagers. Contributed by Jian He.
svn merge --ignore-ancestry -c 1576843 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576844 13f79535-47bb-0310-9956-ffa450edef68
2014-03-12 18:26:53 +00:00
Arun Murthy 8c4578e894 Merge -c 1576751 from trunk to branch-2 to fix YARN-1444. Fix CapacityScheduler to deal with cases where applications specify host/rack requests without off-switch request. Contributed by Wangda Tan.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576752 13f79535-47bb-0310-9956-ffa450edef68
2014-03-12 14:37:27 +00:00
Vinod Kumar Vavilapalli d5120ccc6b YARN-1800. Fixed NodeManager to gracefully handle RejectedExecutionException in the public-localizer thread-pool. Contributed by Varun Vasudev.
svn merge --ignore-ancestry -c 1576545 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576546 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 23:34:20 +00:00