Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,363 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

472 Commits

Author SHA1 Message Date
Zhijie Shen 89427419a3 YARN-2673. Made timeline client put APIs retry if ConnectException happens. Contributed by Li Lu. 2014-10-20 12:22:17 -07:00
Vinod Kumar Vavilapalli 8256766498 YARN-2504. Enhanced RM Admin CLI to support management of node-labels. Contribyted by Wangda Tan. 2014-10-18 12:07:40 -07:00
Vinod Kumar Vavilapalli 39063cd36f YARN-2676. Enhanced Timeline auth-filter to support proxy users. Contributed by Zhijie Shen. 2014-10-17 22:02:50 -07:00
Vinod Kumar Vavilapalli e9c66e8fd2 YARN-2705. Fixed bugs in ResourceManager node-label manager that were causing test-failures: added a dummy in-memory labels-manager. Contributed by Wangda Tan. 2014-10-17 18:26:12 -07:00
Vinod Kumar Vavilapalli abae63caf9 YARN-2699. Fixed a bug in CommonNodeLabelsManager that caused tests to fail when using ephemeral ports on NodeIDs. Contributed by Wangda Tan. 2014-10-17 08:58:08 -07:00
Vinod Kumar Vavilapalli b3056c266a YARN-2685. Fixed a bug in CommonNodeLabelsManager that caused wrong resource tracking per label when a host runs multiple node-managers. Contributed by Wangda Tan. 2014-10-15 18:47:26 -07:00
Jian He 0af1a2b5bc YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 2014-10-15 15:22:07 -07:00
Karthik Kambatla cc93e7e683 YARN-2566. DefaultContainerExecutor should pick a working directory randomly. (Zhihai Xu via kasha) 2014-10-13 16:32:01 -07:00
Zhijie Shen 4aed2d8e91 YARN-2651. Spun off LogRollingInterval from LogAggregationContext. Contributed by Xuan Gong. 2014-10-13 10:54:09 -07:00
Vinod Kumar Vavilapalli db7f165319 YARN-2494. Added NodeLabels Manager internal API and implementation. Contributed by Wangda Tan. 2014-10-10 11:44:21 -07:00
Zhijie Shen cb81bac002 YARN-2583. Modified AggregatedLogDeletionService to be able to delete rolling aggregated logs. Contributed by Xuan Gong. 2014-10-10 00:11:30 -07:00
Zhijie Shen e532ed8faa YARN-2617. Fixed ApplicationSubmissionContext to still set resource for backward compatibility. Contributed by Wangda Tan. 2014-10-09 17:57:50 -07:00
Karthik Kambatla 4f426fe223 YARN-2180. [YARN-1492] In-memory backing store for cache manager. (Chris Trezzo via kasha) 2014-10-09 17:16:06 -07:00
Vinod Kumar Vavilapalli 596702a025 YARN-2544. Added admin-API objects for using node-labels. Contributed by Wangda Tan. 2014-10-09 15:46:24 -07:00
Zhijie Shen 1d4612f5ad YARN-2629. Made the distributed shell use the domain-based timeline ACLs. Contributed by Zhijie Shen. 2014-10-09 12:59:47 -07:00
Vinod Kumar Vavilapalli 180afa2f86 YARN-2493. Added user-APIs for using node-labels. Contributed by Wangda Tan. 2014-10-09 11:26:26 -07:00
Steve Loughran 6a326711aa YARN-913 service registry: YARN-2652 add hadoop-yarn-registry package under hadoop-yarn 2014-10-08 13:02:25 -07:00
Jian He ea26cc0b4a YARN-2615. Changed ClientToAMTokenIdentifier/RM(Timeline)DelegationTokenIdentifier to use protobuf as payload. Contributed by Junping Du 2014-10-06 10:47:43 -07:00
Chris Douglas c8212bacb1 YARN-1051. Add a system for creating reservations of cluster capacity. 
Contributed by Subru Krishnan and Carlo Curino.
 2014-10-03 17:09:02 -07:00
subru 5e10a13bb4 YARN-2576. Making test patch pass in branch. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit 90ac0be86b)
 2014-10-03 15:43:13 -07:00
subru c9266df404 YARN-1708. Public YARN APIs for creating/updating/deleting reservations. 
(cherry picked from commit 3f2e3b275b)
 2014-10-03 15:41:39 -07:00
Vinod Kumar Vavilapalli 34cdcaad71 YARN-2468. Enhanced NodeManager to support log handling APIs (YARN-2569) for use by long running services. Contributed by Xuan Gong. 2014-10-03 12:15:40 -07:00
Jian He eb6ce5e97c YARN-2562. Changed ContainerId#toString() to be more readable. Contributed by Tsuyoshi OZAWA 2014-10-03 11:04:02 -07:00
Zhijie Shen 1c93025a1b YARN-2527. Fixed the potential NPE in ApplicationACLsManager and added test cases for it. Contributed by Benoy Antony. 2014-10-02 14:56:13 -07:00
Vinod Kumar Vavilapalli 9e40de6af7 YARN-2446. Augmented Timeline service APIs to start taking in domains as a parameter while posting entities and events. Contributed by Zhijie Shen. 2014-10-01 17:52:03 -07:00
junping_du 0708827a93 YARN-2613. Support retry in NMClient for rolling-upgrades. (Contributed by Jian He) 2014-10-01 16:50:30 -07:00
Karthik Kambatla 17d1202c35 YARN-2179. [YARN-1492] Initial cache manager structure and context. (Chris Trezzo via kasha) 2014-09-30 19:01:47 -07:00
Jason Lowe feaf139b4f YARN-2387. Resource Manager crashes with NPE due to lack of synchronization. Contributed by Mit Desai 2014-09-30 22:37:28 +00:00
Karthik Kambatla f7743dd07d YARN-2610. Hamlet should close table tags. (Ray Chiang via kasha) 2014-09-30 11:46:07 -07:00
Jian He 5391919b09 YARN-668. Changed NMTokenIdentifier/AMRMTokenIdentifier/ContainerTokenIdentifier to use protobuf object as the payload. Contributed by Junping Du. 2014-09-26 17:48:41 -07:00
Zhijie Shen c86674a3a4 YARN-2581. Passed LogAggregationContext to NM via ContainerTokenIdentifier. Contributed by Xuan Gong. 2014-09-24 17:50:26 -07:00
Vinod Kumar Vavilapalli d78b452a4f YARN-2102. Added the concept of a Timeline Domain to handle read/write ACLs on Timeline service event data. Contributed by Zhijie Shen. 2014-09-24 11:27:53 -07:00
Zhijie Shen 5338ac416a YARN-2569. Added the log handling APIs for the long running services. Contributed by Xuan Gong. 2014-09-23 10:36:57 -07:00
Zhijie Shen 376233cdd4 MAPREDUCE-5279. Made MR headroom calculation honor cpu dimension when YARN scheduler resource type is memory plus cpu. Contributed by Peng Zhang and Varun Vasudev. 2014-09-22 09:28:47 -07:00
Allen Wittenauer aa1052c34b YARN-2460. Remove obsolete entries from yarn-default.xml (Ray Chiang via aw) 2014-09-19 14:07:38 -07:00
Vinod Kumar Vavilapalli 485c96e3cb YARN-2001. Added a time threshold for RM to wait before starting container allocations after restart/failover. Contributed by Jian He. 2014-09-18 11:03:12 -07:00
Vinod Kumar Vavilapalli a3d9934f91 YARN-1779. Fixed AMRMClient to handle AMRMTokens correctly across ResourceManager work-preserving-restart or failover. Contributed by Jian He. 2014-09-18 10:16:18 -07:00
Jian He f4886111aa YARN-2558. Updated ContainerTokenIdentifier#read/write to use ContainerId#getContainerId. Contributed by Tsuyoshi OZAWA. 2014-09-17 15:13:59 -07:00
Vinod Kumar Vavilapalli 9f6891d9ef YARN-2531. Added a configuration for admins to be able to override app-configs and enforce/not-enforce strict control of per-container cpu usage. Contributed by Varun Vasudev. 2014-09-16 10:14:46 -07:00
Vinod Kumar Vavilapalli 14e2639fd0 YARN-611. Added an API to let apps specify an interval beyond which AM failures should be ignored towards counting max-attempts. Contributed by Xuan Gong. 2014-09-13 18:04:05 -07:00
Jian He 3122daa802 YARN-2229. Changed the integer field of ContainerId to be long type. Contributed by Tsuyoshi OZAWA 2014-09-12 10:33:33 -07:00
junping_du 6b8b1608e6 YARN-2033. Merging generic-history into the Timeline Store (Contributed by Zhijie Shen) 2014-09-12 10:04:51 +08:00
Vinod Kumar Vavilapalli 4be95175cd YARN-2440. Enabled Nodemanagers to limit the aggregate cpu usage across all containers to a preconfigured limit. Contributed by Varun Vasudev. 2014-09-10 19:22:52 -07:00
Jian He 83be3ad444 YARN-415. Capture aggregate memory allocation at the app-level for chargeback. Contributed by Eric Payne & Andrey Klochkov 2014-09-10 18:20:54 -07:00
Vinod Kumar Vavilapalli b67d5ba784 YARN-2448. Changed ApplicationMasterProtocol to expose RM-recognized resource types to the AMs. Contributed by Varun Vasudev. 2014-09-10 10:15:47 -07:00
Jian He 0974f434c4 YARN-2515. Updated ConverterUtils#toContainerId to parse epoch. Contributed by Tsuyoshi OZAWA 2014-09-07 18:29:22 -07:00
cnauroth cbea1b10ef YARN-2519. Credential Provider related unit tests failed on Windows. Contributed by Xiaoyu Yao. 2014-09-06 20:05:07 -07:00
Jian He 5c14bc426b YARN-1506. Changed RMNode/SchedulerNode to update resource with event notification. Contributed by Junping Du 2014-08-29 23:05:51 -07:00
Jian He e2d0ff364a YARN-2182. Updated ContainerId#toString() to append RM Epoch number. Contributed by Tsuyoshi OZAWA 2014-08-27 10:02:45 -07:00
Jason Darrell Lowe da4ba50269 HADOOP-10893. isolated classloader on the client side. Contributed by Sangjin Lee 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619604 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 21:38:16 +00:00
Allen Wittenauer 7e75226e68 YARN-2424. LCE should support non-cgroups, non-secure mode (Chris Douglas via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619421 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 14:57:11 +00:00
Jason Darrell Lowe 55a2b550b5 YARN-2034. Description for yarn.nodemanager.localizer.cache.target-size-mb is incorrect. Contributed by Chen He 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619176 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-20 17:38:51 +00:00
Jian He e4539e88e3 YARN-2174. Enable HTTPs for the writer REST API of TimelineServer. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619160 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-20 17:05:07 +00:00
Allen Wittenauer 0350ea3c72 YARN-1918. Typo in description and error message for yarn.resourcemanager.cluster-id (Anandha L Ranganathan via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618070 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 21:53:12 +00:00
Jian He 0c9b8f2d7f YARN-2373. Changed WebAppUtils to use Configuration#getPassword for accessing SSL passwords. Contributed by Larry McCay 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617555 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 18:29:42 +00:00
Xuan Gong eeb4acd955 YARN-2212: ApplicationMaster needs to find a way to update the AMRMToken periodically. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616892 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 21:38:24 +00:00
Jian He 8437df8ba9 YARN-2008. Fixed CapacityScheduler to calculate headroom based on max available capacity instead of configured max capacity. Contributed by Craig Welch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616580 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 20:00:04 +00:00
Junping Du b47f65214c YARN-2298. Move TimelineClient to yarn-common project (Contributed by Zhijie Shen) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616100 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-06 05:48:32 +00:00
Junping Du 1141edc424 YARN-2051. Fix bug in PBimpls and add more unit tests with reflection. (Contributed by Binglin Chang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1615025 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 04:16:08 +00:00
Xuan Gong e52f67e389 YARN-1994. Expose YARN/MR endpoints on multiple interfaces. Contributed by Craig Welch, Milan Potocnik,and Arpit Agarwal 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614981 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-31 20:06:02 +00:00
Zhijie Shen d6532d3a77 YARN-2247. Made RM web services authenticate users via kerberos and delegation token. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613821 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-27 17:55:06 +00:00
Zhijie Shen 70dededdc9 YARN-2270. Made TestFSDownload#testDownloadPublicWithStatCache be skipped when there’s no ancestor permissions. Contributed by Akira Ajisaka. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612460 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 04:58:00 +00:00
Xuan Gong f1b831ccfb YARN-2208. AMRMTokenManager need to have a way to roll over AMRMToken. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611820 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-18 21:46:29 +00:00
Jian He 40e1bb9d31 YARN-2264. Fixed a race condition in DrainDispatcher which may cause random test failures. Contributed by Li Lu 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611126 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-16 18:25:05 +00:00
Vinod Kumar Vavilapalli a6cd712ca7 YARN-2228. Augmented TimelineServer to load pseudo authentication filter when authentication = simple. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1610575 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-15 04:33:48 +00:00
Xuan Gong c0439b732c YARN-2269. Remove external links from YARN UI. Contributed by Craig Welch 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609590 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-10 23:31:55 +00:00
Jian He 50be8561a7 YARN-2088. Fixed a bug in GetApplicationsRequestPBImpl#mergeLocalToBuilder. Contributed by Binglin Chang 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609584 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-10 22:36:13 +00:00
Junping Du 4074843fea YARN-2251. Avoid negative elapsed time in JHS/MRAM web UI and services (Contributed by Zhijie Shen) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607833 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-04 11:18:05 +00:00
Vinod Kumar Vavilapalli 075ff276ca YARN-1713. Added get-new-app and submit-app functionality to RM web services. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607216 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-02 00:23:07 +00:00
Ravi Prakash f084170a0a YARN 2163. WebUI: Order of AppId in apps table should be consistent with ApplicationId.compareTo() 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605964 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-27 06:43:46 +00:00
Vinod Kumar Vavilapalli e285b98f0f YARN-2152. Added missing information into ContainerTokenIdentifier so that NodeManagers can report the same to RM when RM restarts. Contributed Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605205 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 21:43:22 +00:00
Vinod Kumar Vavilapalli 424fd9494f YARN-1368. Added core functionality of recovering container state into schedulers after ResourceManager Restart so as to preserve running work in the cluster. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601303 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-09 03:09:21 +00:00
Junping Du 790cbbf3e2 YARN-2103. Inconsistency between viaProto flag and initial value of SerializedExceptionProto.Builder (Contributed by Binglin Chang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1599115 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-02 08:15:30 +00:00
Chris Nauroth d599adcc6a YARN-1868. YARN status web ui does not show correctly in IE 11. Contributed by Chuan Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598686 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-30 18:07:23 +00:00
Karthik Kambatla 75112e97b4 YARN-2010. Document yarn.resourcemanager.zk-auth and its scope. (Robert Kanter via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598636 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-30 15:32:23 +00:00
Karthik Kambatla 49a3a0cd0c YARN-2054. Better defaults for YARN ZK configs for retries and retry-inteval when HA is enabled. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598630 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-30 15:24:49 +00:00
Vinod Kumar Vavilapalli cd24e955cf YARN-2112. Fixed yarn-common's pom.xml to include jackson dependencies so that both Timeline Server and client can access them. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598373 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-29 19:06:36 +00:00
Zhijie Shen 001078e067 YARN-2107. Refactored timeline classes into o.a.h.y.s.timeline package. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598094 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-28 18:09:04 +00:00
Vinod Kumar Vavilapalli af8b09af75 YARN-1936. Added security support for the Timeline Client. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597153 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-23 18:47:57 +00:00
Vinod Kumar Vavilapalli b867b69556 YARN-2049. Added delegation-token support for the Timeline Server. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1597130 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-23 17:09:47 +00:00
Vinod Kumar Vavilapalli fdb5870d49 YARN-1938. Added kerberos login for the Timeline Server. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596710 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-21 23:20:08 +00:00
Jason Darrell Lowe 64ec743e6c YARN-2050. Fix LogCLIHelpers to create the correct FileContext. Contributed by Ming Ma 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596310 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-20 16:19:55 +00:00
Junping Du 461ded8bd4 YARN-2066. Wrong field is referenced in GetApplicationsRequestPBImpl#mergeLocalToBuilder (Contributed by Hong Zhiguo) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595413 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-17 01:43:15 +00:00
Jason Darrell Lowe b0e80d1a47 MAPREDUCE-5814. fat jar with *-default.xml may fail when mapreduce.job.classloader=true. Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594326 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-13 18:55:05 +00:00
Jian He 4f6493bcf9 YARN-2016. Fix a bug in GetApplicationsRequestPBImpl to add the missed fields to proto. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594085 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-12 20:45:21 +00:00
Vinod Kumar Vavilapalli 58c81d56e9 YARN-1962. Changed Timeline Service client configuration to be off by default given the non-readiness of the feature yet. Contributed by Mohammad Kamrul Islam. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593750 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-11 00:15:12 +00:00
Zhijie Shen 0c437e90b8 YARN-1701. Improved default paths of the timeline store and the generic history store. Contributed by Tsuyoshi Ozawa. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593481 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-09 04:23:27 +00:00
Haohui Mai d777cb8996 HADOOP-10563. Remove the dependency of jsp in trunk. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592069 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-02 22:26:02 +00:00
Sanford Ryza faffedeb24 YARN-483. Improve documentation on log aggregation in yarn-default.xml (Akira Ajisaka via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1590150 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-25 21:14:03 +00:00
Jason Darrell Lowe d667df4ed0 YARN-1932. Javascript injection on the job status page. Contributed by Mit Desai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588572 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-18 21:41:37 +00:00
Junping Du 36ed8c74ea YARN-1870. FileInputStream is not closed in ProcfsBasedProcessTree#constructProcessSMAPInfo. (Fengdong Yu via junping_du) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588324 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-17 18:05:43 +00:00
Thomas Graves ad9edb02b9 YARN-1931. Private API change in YARN-1824 in 2.4 broke compatibility with previous releases (Sandy Ryza via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588281 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-17 15:23:22 +00:00
Vinod Kumar Vavilapalli cde7b17566 YARN-1914. Fixed resource-download on NodeManagers to skip permission verification of public cache files in Windows+local file-system environment. Contribued by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586434 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-10 19:28:31 +00:00
Steve Loughran b8f250a99d HADOOP-10104. Update jackson to 1.9.13 (Akira Ajisaka via stevel) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585932 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-09 11:11:20 +00:00
Karthik Kambatla 245012a9d9 YARN-1757. NM Recovery. Auxiliary service support. (Jason Lowe via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585783 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-08 17:15:58 +00:00
Chris Nauroth 4eacde67f8 YARN-1905. TestProcfsBasedProcessTree must only run on Linux. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585376 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-07 03:04:09 +00:00
Vinod Kumar Vavilapalli dce1d20383 YARN-1452. Added documentation about the configuration and usage of generic application history and the timeline data service. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581656 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-26 02:52:58 +00:00
Vinod Kumar Vavilapalli 09f383254c YARN-1850. Introduced the ability to optionally disable sending out timeline-events in the TimelineClient. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581189 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-25 02:22:39 +00:00
Jonathan Turner Eagles 2d89dcb121 YARN-1670. aggregated log writer can write more log data then it says is the log length (Mit Desai via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580957 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-24 18:11:48 +00:00
Vinod Kumar Vavilapalli 06579878df YARN-1775. Enhanced ProcfsBasedProcessTree to optionally add the ability to use smaps for obtaining used memory information. Contributed by Rajesh Balamohan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580087 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-22 00:01:59 +00:00
Vinod Kumar Vavilapalli 2826a35469 YARN-1670. Fixed a bug in log-aggregation that can cause the writer to write more log-data than the log-length that it records. Contributed by Mit Desai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580005 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-21 18:35:54 +00:00
Vinod Kumar Vavilapalli 4224e61340 YARN-1811. Fixed AMFilters in YARN to correctly accept requests from either web-app proxy or the RMs when HA is enabled. Contributed by Robert Kanter. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579877 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-21 04:20:46 +00:00
Vinod Kumar Vavilapalli 81a456e638 YARN-1591. Fixed AsyncDispatcher to handle interrupts on shutdown in a sane manner and thus fix failure of TestResourceTrackerService. Contributed by Tsuyoshi Ozawa. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578628 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 22:17:43 +00:00
Vinod Kumar Vavilapalli 8314674947 YARN-1685. Fixed few bugs related to handling of containers' log-URLs on ResourceManager and history-service. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578602 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 21:36:21 +00:00
Jonathan Turner Eagles 7944eab64a YARN-1136. Replace junit.framework.Assert with org.junit.Assert (Chen He via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578539 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 20:03:35 +00:00
Jonathan Turner Eagles 1190f19ddd YARN-1845. Elapsed time for failed tasks that never started is wrong (Rushabh S Shah via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578459 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 16:53:17 +00:00
Jonathan Turner Eagles f8e439105c YARN-1845. Elapsed time for failed tasks that never started is wrong (Rushabh S Shah via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578457 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 16:50:56 +00:00
Vinod Kumar Vavilapalli 96e0ca2d27 YARN-1824. Improved NodeManager and clients to be able to handle cross platform application submissions. Contributed by Jian He. 
MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to handle cross platform application submissions. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578135 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-16 18:32:05 +00:00
Zhijie Shen b3ea4aebff YARN-1717. Enabled periodically discarding old data in LeveldbTimelineStore. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577693 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 20:35:00 +00:00
Vinod Kumar Vavilapalli d9cdcb9474 YARN-1658. Modified web-app framework to let standby RMs redirect web-service calls to the active RM. Contributed by Cindy Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577408 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 02:39:59 +00:00
Christopher Douglas 53790d3300 YARN-1771. Reduce the number of NameNode operations during localization of 
public resources using a cache. Contributed by Sangjin Lee



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577391 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 00:30:35 +00:00
Zhijie Shen 6af044bb75 YARN-1389. Made ApplicationClientProtocol and ApplicationHistoryProtocol expose analogous getApplication(s)/Attempt(s)/Container(s) APIs. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577052 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-13 07:42:29 +00:00
Vinod Kumar Vavilapalli 0b1304d098 YARN-1781. Modified NodeManagers to allow admins to specify max disk utilization for local disks so as to be able to offline full disks. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575463 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-08 00:52:06 +00:00
Karthik Kambatla 9b15c5b11a YARN-1525. Web UI should redirect to active RM when HA is enabled. (Cindy Li via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575166 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-07 04:34:16 +00:00
Haohui Mai ad61eec072 HADOOP-10386. Log proxy hostname in various exceptions being thrown in a HA setup. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574716 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-05 23:09:47 +00:00
Vinod Kumar Vavilapalli 88245b6a41 YARN-986. Changed client side to be able to figure out the right RM Delegation token for the right ResourceManager when HA is enabled. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574190 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 20:39:06 +00:00
Vinod Kumar Vavilapalli a60434368c YARN-1758. Fixed ResourceManager to not mandate the presence of site specific configuration files and thus fix failures in downstream tests. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573695 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 19:41:52 +00:00
Vinod Kumar Vavilapalli 94b29b3348 MAPREDUCE-5770. Fixed MapReduce ApplicationMaster to correctly redirect to the YARN's web-app proxy with the correct scheme prefix. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572711 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-27 19:23:55 +00:00
Vinod Kumar Vavilapalli 0fbc7fe816 YARN-1588. Enhanced RM and the scheduling protocol to also send NMTokens of transferred containers from previous app-attempts to new AMs after YARN-1490. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572230 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 20:20:07 +00:00
Vinod Kumar Vavilapalli e167e585e9 YARN-1749. Updated application-history related configs to reflect the latest reality and to be consistently named. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570948 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-23 02:04:58 +00:00
Vinod Kumar Vavilapalli 84425fb435 YARN-1687. Renamed user-facing records for the timeline-service to be simply named after 'timeline' instead of 'apptimeline'. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570922 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-22 20:55:06 +00:00
Vinod Kumar Vavilapalli 9da9f7d4d8 YARN-1666. Modified RM HA handling of include/exclude node-lists to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569856 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-19 17:52:57 +00:00
Vinod Kumar Vavilapalli 990cffdcfa YARN-1553. Modified YARN and MR to stop using HttpConfig.isSecure() and 
instead rely on the http policy framework. And also fix some bugs related
to https handling in YARN web-apps. Contributed by Haohui Mai.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1568501 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 20:01:02 +00:00
Zhijie Shen 3587b6774c YARN-1706. Created an utility method to dump timeline records to JSON strings. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1566982 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-11 04:39:37 +00:00
Vinod Kumar Vavilapalli 1fa6ab249b YARN-1459. Changed ResourceManager to depend its service initialization on the configuration-provider mechanism during startup too. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1566791 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-10 22:50:15 +00:00
Zhijie Shen 23b2e43f5d YARN-1635. Implemented a Leveldb based ApplicationTimelineStore. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565868 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-08 02:15:46 +00:00
Vinod Kumar Vavilapalli 8b2336fcef YARN-1660. Simplified the RM HA configuration to accept and be able to simply depend just on 
configuration properties of the form yarn.resourcemanager.hostname.RMID and use the default
ports for all service addresses. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565523 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 03:18:39 +00:00
Vinod Kumar Vavilapalli 30294a2196 YARN-1665. Simplify the configuration of RM HA by having better default values. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565517 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-07 02:39:32 +00:00
Vinod Kumar Vavilapalli 484faadffe YARN-1636. Augmented Application-history server's web-services to also expose new APIs for retrieving and storing timeline information. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564829 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 17:02:26 +00:00
Zhijie Shen ebe0c17a95 YARN-1461. Added tags for YARN applications and changed RM to handle them. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564633 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 04:55:00 +00:00
Vinod Kumar Vavilapalli 39ecc46ff6 YARN-1285. Changed the default value of yarn.acl.enable in yarn-default.xml to be consistent with what exists (false) in the code and documentation. Contributed by Kenji Kikushima. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564190 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-04 04:21:40 +00:00
Vinod Kumar Vavilapalli eac066b13f YARN-1659. Defined the ApplicationTimelineStore store as an abstraction for implementing different storage impls for storing timeline information. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564037 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 20:07:01 +00:00
Vinod Kumar Vavilapalli 3e7d56678c YARN-1639. Modified RM HA configuration handling to have a way of not requiring separate configuration files for each RM. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564032 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:53:46 +00:00
Vinod Kumar Vavilapalli ca72e11158 YARN-1611. Introduced the concept of a configuration provider which can be used by ResourceManager to read configuration locally or from remote systems so as to help RM failover. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564002 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-03 19:07:55 +00:00
Karthik Kambatla 88d8ea9509 HADOOP-10085. CompositeService should allow adding services while being inited. (Steve Loughran via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563694 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-02 19:51:39 +00:00
Vinod Kumar Vavilapalli 6e8c1bf200 YARN-1633. Defined user-facing entity, entity-info and event objects related to Application Timeline feature. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563356 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-01 00:09:53 +00:00
Jason Darrell Lowe 4de2bd3fbf YARN-1600. RM does not startup when security is enabled without spnego configured. Contributed by Haohui Mai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562482 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-29 15:30:04 +00:00
Karthik Kambatla 8006467e58 YARN-1642. RMDTRenewer#getRMClient should use ClientRMProxy (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561990 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 08:26:33 +00:00
Suresh Srinivas 33a47743a5 HADOOP-10255. Rename HttpServer to HttpServer2 to retain older HttpServer in branch-2 for compatibility. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561959 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 07:32:52 +00:00
Vinod Kumar Vavilapalli df000e69e8 YARN-1625. Fixed RAT warnings after YARN-321 merge. Contributed by Shinichi Yamashita. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561458 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-26 07:20:51 +00:00
Vinod Kumar Vavilapalli cbee889711 YARN-321. Merging YARN-321 branch to trunk. 
svn merge ../branches/YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561452 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-26 04:51:10 +00:00
Karthik Kambatla 482993ac53 YARN-1573. ZK store should use a private password for root-node-acls. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560594 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-23 06:22:05 +00:00
Sanford Ryza 163348701c YARN-1623. Include queue name in RegisterApplicationMasterResponse (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560545 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-22 22:25:03 +00:00
Jason Darrell Lowe d1963ad4bd MAPREDUCE-5693. Restore MRv1 behavior for log flush. Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560148 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-21 19:23:34 +00:00
Sanford Ryza 67cbde3008 YARN-1496. Protocol additions to allow moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1559050 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-17 08:56:22 +00:00
Jason Darrell Lowe a6ea460a91 MAPREDUCE-5672. Provide optional RollingFileAppender for container log4j (syslog). Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558948 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-16 22:56:09 +00:00
Karthik Kambatla 3928de29fc YARN-1606. Fix the default value of yarn.resourcemanager.zk-timeout-ms in yarn-default.xml (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558882 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-16 18:54:49 +00:00
Vinod Kumar Vavilapalli 7c8b654ba5 YARN-888. Cleaned up POM files so that non-leaf modules don't include any dependencies and thus compact the dependency list for leaf modules. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557801 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-13 18:34:02 +00:00
Vinod Kumar Vavilapalli 25bc68d15e YARN-1041. Added the ApplicationMasterProtocol API for applications to use the ability in ResourceManager to optionally not kill containers when the ApplicationMaster exits. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557318 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-11 07:07:17 +00:00
Karthik Kambatla 6608b75829 YARN-1574. RMDispatcher should be reset on transition to standby. (Xuan Gong via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557248 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-10 20:15:00 +00:00