Commit Graph

324 Commits

Author SHA1 Message Date
Junping Du b81b6b6366 Merge r1607833 from Trunk: YARN-2251. Avoid negative elapsed time in JHS/MRAM web UI and services (Contributed by Zhijie Shen)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1607834 13f79535-47bb-0310-9956-ffa450edef68
2014-07-04 11:20:43 +00:00
Karthik Kambatla 9133e3142c Preparing for 2.6.0 development. Updated versions.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1607541 13f79535-47bb-0310-9956-ffa450edef68
2014-07-03 06:11:57 +00:00
Vinod Kumar Vavilapalli 8f1f458729 YARN-1713. Added get-new-app and submit-app functionality to RM web services. Contributed by Varun Vasudev.
svn merge --ignore-ancestry -c 1607216 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1607217 13f79535-47bb-0310-9956-ffa450edef68
2014-07-02 00:23:49 +00:00
Ravi Prakash b2e3e76e2f YARN 2163. WebUI: Order of AppId in apps table should be consistent with ApplicationId.compareTo()
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1605965 13f79535-47bb-0310-9956-ffa450edef68
2014-06-27 06:45:57 +00:00
Vinod Kumar Vavilapalli a2e2c8ad97 YARN-2152. Added missing information into ContainerTokenIdentifier so that NodeManagers can report the same to RM when RM restarts. Contributed Jian He.
svn merge --ignore-ancestry -c 1605205 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1605206 13f79535-47bb-0310-9956-ffa450edef68
2014-06-24 21:44:00 +00:00
Vinod Kumar Vavilapalli 319e422733 YARN-1368. Added core functionality of recovering container state into schedulers after ResourceManager Restart so as to preserve running work in the cluster. Contributed by Jian He.
svn merge --ignore-ancestry -c 1601303 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1601304 13f79535-47bb-0310-9956-ffa450edef68
2014-06-09 03:10:06 +00:00
Junping Du 2b98e9cfd9 Merge r1599115 from trunk: YARN-2103. Inconsistency between viaProto flag and initial value of SerializedExceptionProto.Builder (Contributed by Binglin Chang)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1599121 13f79535-47bb-0310-9956-ffa450edef68
2014-06-02 08:27:08 +00:00
Chris Nauroth 2a5f932bdb YARN-1868. Merging change r1598686 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1598690 13f79535-47bb-0310-9956-ffa450edef68
2014-05-30 18:18:18 +00:00
Karthik Kambatla 49f6a1d7a6 YARN-2010. Document yarn.resourcemanager.zk-auth and its scope. (Robert Kanter via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1598637 13f79535-47bb-0310-9956-ffa450edef68
2014-05-30 15:32:53 +00:00
Karthik Kambatla 4c4ebe2353 YARN-2054. Better defaults for YARN ZK configs for retries and retry-inteval when HA is enabled. (kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1598632 13f79535-47bb-0310-9956-ffa450edef68
2014-05-30 15:25:27 +00:00
Vinod Kumar Vavilapalli 0071e48d21 YARN-2112. Fixed yarn-common's pom.xml to include jackson dependencies so that both Timeline Server and client can access them. Contributed by Zhijie Shen.
svn merge --ignore-ancestry -c 1598373 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1598374 13f79535-47bb-0310-9956-ffa450edef68
2014-05-29 19:07:16 +00:00
Zhijie Shen 57c01b1bda YARN-2107. Refactored timeline classes into o.a.h.y.s.timeline package. Contributed by Vinod Kumar Vavilapalli.
svn merge --ignore-ancestry -c 1598094 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1598095 13f79535-47bb-0310-9956-ffa450edef68
2014-05-28 18:11:45 +00:00
Vinod Kumar Vavilapalli 65a5e2cc46 YARN-1936. Added security support for the Timeline Client. Contributed by Zhijie Shen.
svn merge --ignore-ancestry -c 1597153 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1597154 13f79535-47bb-0310-9956-ffa450edef68
2014-05-23 18:49:21 +00:00
Vinod Kumar Vavilapalli 18457f8ed0 YARN-2049. Added delegation-token support for the Timeline Server. Contributed by Zhijie Shen.
svn merge --ignore-ancestry -c 1597130 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1597131 13f79535-47bb-0310-9956-ffa450edef68
2014-05-23 17:10:22 +00:00
Vinod Kumar Vavilapalli e3fae25785 YARN-1938. Added kerberos login for the Timeline Server. Contributed by Zhijie Shen.
svn merge --ignore-ancestry -c 1596710 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1596711 13f79535-47bb-0310-9956-ffa450edef68
2014-05-21 23:20:40 +00:00
Jason Darrell Lowe dfd4a7398b svn merge -c 1596310 FIXES: YARN-2050. Fix LogCLIHelpers to create the correct FileContext. Contributed by Ming Ma
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1596311 13f79535-47bb-0310-9956-ffa450edef68
2014-05-20 16:21:30 +00:00
Junping Du 644743599c Merge r1595413 from trunk: Wrong field is referenced in GetApplicationsRequestPBImpl#mergeLocalToBuilder (Contributed by Hong Zhiguo)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1595416 13f79535-47bb-0310-9956-ffa450edef68
2014-05-17 01:50:15 +00:00
Jason Darrell Lowe 890cd33bf7 svn merge -c 1594326 FIXES: MAPREDUCE-5814. fat jar with *-default.xml may fail when mapreduce.job.classloader=true. Contributed by Gera Shegalov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1594328 13f79535-47bb-0310-9956-ffa450edef68
2014-05-13 18:59:56 +00:00
Jian He 634485191e Merge r1594085 from trunk. YARN-2016. Fix a bug in GetApplicationsRequestPBImpl to add the missed fields to proto. Contributed by Junping Du
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1594086 13f79535-47bb-0310-9956-ffa450edef68
2014-05-12 20:48:23 +00:00
Vinod Kumar Vavilapalli a098b692f4 YARN-1962. Changed Timeline Service client configuration to be off by default given the non-readiness of the feature yet. Contributed by Mohammad Kamrul Islam.
svn merge --ignore-ancestry -c 1593750 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1593751 13f79535-47bb-0310-9956-ffa450edef68
2014-05-11 00:15:48 +00:00
Zhijie Shen cf42886750 YARN-1701. Improved default paths of the timeline store and the generic history store. Contributed by Tsuyoshi Ozawa.
svn merge --ignore-ancestry -c 1593481 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1593484 13f79535-47bb-0310-9956-ffa450edef68
2014-05-09 04:26:36 +00:00
Sanford Ryza 44d9ee78e9 YARN-483. Improve documentation on log aggregation in yarn-default.xml (Akira Ajisaka via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1590151 13f79535-47bb-0310-9956-ffa450edef68
2014-04-25 21:14:25 +00:00
Jason Darrell Lowe 1b0ed842d0 svn merge -c 1588572 FIXES: YARN-1932. Javascript injection on the job status page. Contributed by Mit Desai
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1588573 13f79535-47bb-0310-9956-ffa450edef68
2014-04-18 21:46:14 +00:00
Junping Du d0db2c2b78 Merge r1588324 from trunk: YARN-1870. FileInputStream is not closed in ProcfsBasedProcessTree#constructProcessSMAPInfo. (Fengdong Yu via junping_du)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1588328 13f79535-47bb-0310-9956-ffa450edef68
2014-04-17 18:09:32 +00:00
Thomas Graves bbabbe5ccb Merge 1588281 to branch-2. YARN-1931. Private API change in YARN-1824 in 2.4 broke compatibility with previous releases (Sandy Ryza via tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1588283 13f79535-47bb-0310-9956-ffa450edef68
2014-04-17 15:25:27 +00:00
Vinod Kumar Vavilapalli 51a39fdbb9 YARN-1914. Fixed resource-download on NodeManagers to skip permission verification of public cache files in Windows+local file-system environment. Contribued by Varun Vasudev.
svn merge --ignore-ancestry -c 1586434 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1586435 13f79535-47bb-0310-9956-ffa450edef68
2014-04-10 19:28:56 +00:00
Steve Loughran 770aef5053 HADOOP-10104. Update jackson to 1.9.13 (Akira Ajisaka via stevel)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1585933 13f79535-47bb-0310-9956-ffa450edef68
2014-04-09 11:11:31 +00:00
Karthik Kambatla a2cdf208dd YARN-1757. NM Recovery. Auxiliary service support. (Jason Lowe via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1585784 13f79535-47bb-0310-9956-ffa450edef68
2014-04-08 17:17:59 +00:00
Chris Nauroth 48ad73577b YARN-1905. Merging change r1585376 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1585378 13f79535-47bb-0310-9956-ffa450edef68
2014-04-07 03:08:54 +00:00
Vinod Kumar Vavilapalli 4c91a402f5 YARN-1452. Added documentation about the configuration and usage of generic application history and the timeline data service. Contributed by Zhijie Shen.
svn merge --ignore-ancestry -c 1581656 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1581657 13f79535-47bb-0310-9956-ffa450edef68
2014-03-26 02:54:00 +00:00
Vinod Kumar Vavilapalli 45d6277ec2 YARN-1850. Introduced the ability to optionally disable sending out timeline-events in the TimelineClient. Contributed by Zhijie Shen.
svn merge --ignore-ancestry -c 1581189 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1581190 13f79535-47bb-0310-9956-ffa450edef68
2014-03-25 02:23:08 +00:00
Jonathan Turner Eagles c2d8ab6085 YARN-1670. aggregated log writer can write more log data then it says is the log length (Mit Desai via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580963 13f79535-47bb-0310-9956-ffa450edef68
2014-03-24 18:17:02 +00:00
Vinod Kumar Vavilapalli 08a194fb55 YARN-1775. Enhanced ProcfsBasedProcessTree to optionally add the ability to use smaps for obtaining used memory information. Contributed by Rajesh Balamohan.
svn merge --ignore-ancestry -c 1580087 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580088 13f79535-47bb-0310-9956-ffa450edef68
2014-03-22 00:02:26 +00:00
Vinod Kumar Vavilapalli d1f2d89a43 YARN-1670. Fixed a bug in log-aggregation that can cause the writer to write more log-data than the log-length that it records. Contributed by Mit Desai.
svn merge --ignore-ancestry -c 1580005 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580006 13f79535-47bb-0310-9956-ffa450edef68
2014-03-21 18:36:25 +00:00
Vinod Kumar Vavilapalli e1bda693dd YARN-1811. Fixed AMFilters in YARN to correctly accept requests from either web-app proxy or the RMs when HA is enabled. Contributed by Robert Kanter.
svn merge --ignore-ancestry -c 1579877 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1579878 13f79535-47bb-0310-9956-ffa450edef68
2014-03-21 04:21:14 +00:00
Vinod Kumar Vavilapalli 597c7d054f YARN-1591. Fixed AsyncDispatcher to handle interrupts on shutdown in a sane manner and thus fix failure of TestResourceTrackerService. Contributed by Tsuyoshi Ozawa.
svn merge --ignore-ancestry -c 1578628 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578629 13f79535-47bb-0310-9956-ffa450edef68
2014-03-17 22:18:09 +00:00
Vinod Kumar Vavilapalli 6aae750762 YARN-1685. Fixed few bugs related to handling of containers' log-URLs on ResourceManager and history-service. Contributed by Zhijie Shen.
svn merge --ignore-ancestry -c 1578602 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578603 13f79535-47bb-0310-9956-ffa450edef68
2014-03-17 21:36:51 +00:00
Jonathan Turner Eagles 8e38068076 YARN-1136. Replace junit.framework.Assert with org.junit.Assert (Chen He via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578546 13f79535-47bb-0310-9956-ffa450edef68
2014-03-17 20:20:22 +00:00
Jonathan Turner Eagles 92db0d3558 YARN-1845. Elapsed time for failed tasks that never started is wrong (Rushabh S Shah via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578460 13f79535-47bb-0310-9956-ffa450edef68
2014-03-17 16:56:43 +00:00
Vinod Kumar Vavilapalli d470c7b71a YARN-1824. Improved NodeManager and clients to be able to handle cross platform application submissions. Contributed by Jian He.
MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to handle cross platform application submissions. Contributed by Jian He.
svn merge --ignore-ancestry -c 1578135 ../../trunk/ with a couple of minor edits for working in branch-2.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578139 13f79535-47bb-0310-9956-ffa450edef68
2014-03-16 19:13:16 +00:00
Zhijie Shen f1947fcb28 YARN-1717. Enabled periodically discarding old data in LeveldbTimelineStore. Contributed by Billie Rinaldi.
svn merge --ignore-ancestry -c 1577693 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577695 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 20:37:10 +00:00
Vinod Kumar Vavilapalli 3ac3e5f897 YARN-1658. Modified web-app framework to let standby RMs redirect web-service calls to the active RM. Contributed by Cindy Li.
svn merge --ignore-ancestry -c 1577408 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577409 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 02:40:28 +00:00
Christopher Douglas 9695bc7af6 YARN-1771. Reduce the number of NameNode operations during localization of
public resources using a cache. Contributed by Sangjin Lee



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577392 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 00:31:08 +00:00
Zhijie Shen 7834c7f193 YARN-1389. Made ApplicationClientProtocol and ApplicationHistoryProtocol expose analogous getApplication(s)/Attempt(s)/Container(s) APIs. Contributed by Mayank Bansal.
svn merge --ignore-ancestry -c 1577052 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577053 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 07:45:30 +00:00
Vinod Kumar Vavilapalli 2fbec50fed YARN-1781. Modified NodeManagers to allow admins to specify max disk utilization for local disks so as to be able to offline full disks. Contributed by Varun Vasudev.
svn merge --ignore-ancestry -c 1575463 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1575464 13f79535-47bb-0310-9956-ffa450edef68
2014-03-08 00:52:32 +00:00
Karthik Kambatla 09671ddb46 YARN-1525. Web UI should redirect to active RM when HA is enabled. (Cindy Li via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1575167 13f79535-47bb-0310-9956-ffa450edef68
2014-03-07 04:35:01 +00:00
Haohui Mai 94ac5ba669 Merge r1574716 from trunk.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1574717 13f79535-47bb-0310-9956-ffa450edef68
2014-03-05 23:12:22 +00:00
Vinod Kumar Vavilapalli 7c17cff55c YARN-986. Changed client side to be able to figure out the right RM Delegation token for the right ResourceManager when HA is enabled. Contributed by Karthik Kambatla.
svn merge --ignore-ancestry -c 1574190 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1574191 13f79535-47bb-0310-9956-ffa450edef68
2014-03-04 20:39:33 +00:00
Vinod Kumar Vavilapalli dd0942a90d YARN-1758. Fixed ResourceManager to not mandate the presence of site specific configuration files and thus fix failures in downstream tests. Contributed by Xuan Gong.
svn merge --ignore-ancestry -c 1573695 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1573697 13f79535-47bb-0310-9956-ffa450edef68
2014-03-03 19:42:24 +00:00
Vinod Kumar Vavilapalli 04474afc81 MAPREDUCE-5770. Fixed MapReduce ApplicationMaster to correctly redirect to the YARN's web-app proxy with the correct scheme prefix. Contributed by Jian He.
svn merge --ignore-ancestry -c 1572711 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1572712 13f79535-47bb-0310-9956-ffa450edef68
2014-02-27 19:24:49 +00:00