Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,752 Commits 326 Branches 371 Tags
Commit Graph

389 Commits

Author SHA1 Message Date
Zhijie Shen
e51ae64761 YARN-2651. Spun off LogRollingInterval from LogAggregationContext. Contributed by Xuan Gong. 
(cherry picked from commit 4aed2d8e91c7dccc78fbaffc409d3076c3316289)
 2014-10-13 10:55:09 -07:00
Vinod Kumar Vavilapalli
8ef83a9480 YARN-2494. Added NodeLabels Manager internal API and implementation. Contributed by Wangda Tan. 
(cherry picked from commit db7f1653198b950e89567c06898d64f6b930a0ee)
 2014-10-10 11:45:15 -07:00
Zhijie Shen
ad47a27dbf YARN-2617. Fixed ApplicationSubmissionContext to still set resource for backward compatibility. Contributed by Wangda Tan. 
(cherry picked from commit e532ed8faa8db4b008a5b8d3f82b48a1b314fa6c)
 2014-10-09 18:00:01 -07:00
Karthik Kambatla
7e20187f91 YARN-2180. [YARN-1492] In-memory backing store for cache manager. (Chris Trezzo via kasha) 
(cherry picked from commit 4f426fe2232ed90d8fdf8619fbdeae28d788b5c8)
 2014-10-09 17:16:45 -07:00
Vinod Kumar Vavilapalli
d3125f3de5 YARN-2544. Added admin-API objects for using node-labels. Contributed by Wangda Tan. 
(cherry picked from commit 596702a02501e9cb09aabced168027189eaf02ba)
 2014-10-09 15:47:20 -07:00
Vinod Kumar Vavilapalli
b42cd54d7a YARN-2493. Added user-APIs for using node-labels. Contributed by Wangda Tan. 
(cherry picked from commit 180afa2f86f9854c536c3d4ff7476880e41ac48d)
 2014-10-09 11:27:17 -07:00
Chris Douglas
4ba102bdc3 YARN-1051. Add a system for creating reservations of cluster capacity. 
Contributed by Subru Krishnan and Carlo Curino.
(cherry picked from commit c8212bacb1b2a7e6ee83cc56f72297465ce99390)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-06 10:30:53 -07:00
subru
ae0f16ccc8 YARN-2576. Making test patch pass in branch. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit 90ac0be86b898aefec5471db4027554c8e1b310c)
(cherry picked from commit 5e10a13bb4759984494c6a870c7f08fb6693c9c0)
 2014-10-06 10:21:06 -07:00
subru
cbfbdf60d6 YARN-2080. Integrating reservation system with ResourceManager and client-RM protocol. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit 8baeaead8532898163f1006276b731a237b1a559)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
(cherry picked from commit 6261f7cc69a0eb3eebc9898c7599c7c20f432b4e)
 2014-10-06 10:21:06 -07:00
subru
3a167056d1 YARN-1708. Public YARN APIs for creating/updating/deleting reservations. 
(cherry picked from commit 3f2e3b275bcf29264a112c5d2f4eae289fe7f52c)
(cherry picked from commit c9266df40434fdde05bd3a4c75f7a6bc29db760f)
 2014-10-06 10:21:05 -07:00
Jian He
1c58ab9206 YARN-2562. Changed ContainerId#toString() to be more readable. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit eb6ce5e97c6a379caf9de0b34cc7770c4447bb1a)
 2014-10-03 11:04:46 -07:00
Vinod Kumar Vavilapalli
34dd2d1d1c YARN-2446. Augmented Timeline service APIs to start taking in domains as a parameter while posting entities and events. Contributed by Zhijie Shen. 
(cherry picked from commit 9e40de6af7959ac7bb5f4e4d2833ca14ea457614)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-01 17:54:12 -07:00
Vinod Kumar Vavilapalli
3326fba382 YARN-1972. Added a secure container-executor for Windows. Contributed by Remus Rusanu. 
commit ba7f31c2ee8d23ecb183f88920ef06053c0b9769 is the corresponding trunk commit, this is a slightly different patch for branch-2.
 2014-10-01 17:07:21 -07:00
junping_du
625456746c YARN-2613. Support retry in NMClient for rolling-upgrades. (Contributed by Jian He) 2014-10-01 17:08:55 -07:00
Karthik Kambatla
b616c69835 YARN-2179. [YARN-1492] Initial cache manager structure and context. (Chris Trezzo via kasha) 
(cherry picked from commit 17d1202c35a1992eab66ea05dfd2baf219a17aec)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/bin/yarn
 2014-09-30 19:05:10 -07:00
Vinod Kumar Vavilapalli
5d251d99d6 YARN-2102. Added the concept of a Timeline Domain to handle read/write ACLs on Timeline service event data. Contributed by Zhijie Shen. 
(cherry picked from commit d78b452a4f413c6931a494c33df0666ce9b44973)
 2014-09-24 11:29:01 -07:00
Zhijie Shen
19cfe2f800 YARN-2569. Added the log handling APIs for the long running services. Contributed by Xuan Gong. 
(cherry picked from commit 5338ac416ab8ab3e7e0a7bfb4a53151fc457f673)
 2014-09-23 10:38:27 -07:00
Vinod Kumar Vavilapalli
d24ef142b7 YARN-2001. Added a time threshold for RM to wait before starting container allocations after restart/failover. Contributed by Jian He. 
(cherry picked from commit 485c96e3cb9b0b05d6e490b4773506da83ebc61d)
 2014-09-18 11:04:27 -07:00
Vinod Kumar Vavilapalli
9d34dc87e1 YARN-2531. Added a configuration for admins to be able to override app-configs and enforce/not-enforce strict control of per-container cpu usage. Contributed by Varun Vasudev. 
(cherry picked from commit 9f6891d9ef7064d121305ca783eb62586c8aa018)
 2014-09-16 10:15:37 -07:00
Jian He
1c3f837835 YARN-2529. Generic history service RPC interface doesn't work when service authorization is enabled. Contributed by Zhijie Shen 
(cherry picked from commit 88e329fdfb27c11f1964c0345188bcf328c95fd6)
 2014-09-15 13:15:07 -07:00
Vinod Kumar Vavilapalli
5cdb24d4b1 YARN-611. Added an API to let apps specify an interval beyond which AM failures should be ignored towards counting max-attempts. Contributed by Xuan Gong. 
(cherry picked from commit 14e2639fd0d53f7e0b58f2f4744af44983d4e867)
 2014-09-13 18:05:02 -07:00
Jian He
f21a5236e5 YARN-2229. Changed the integer field of ContainerId to be long type. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 3122daa80261b466e309e88d88d1e2c030525e3f)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/preemption/CheckpointAMPreemptionPolicy.java
 2014-09-12 10:45:42 -07:00
junping_du
efa9ab3638 YARN-2033. Merging generic-history into the Timeline Store (Contributed by Zhijie Shen) 
(cherry picked from commit 6b8b1608e64e300e4e1d23c60476febaca29ca38)
 2014-09-12 10:08:06 +08:00
Vinod Kumar Vavilapalli
a2a61eec6d YARN-2440. Enabled Nodemanagers to limit the aggregate cpu usage across all containers to a preconfigured limit. Contributed by Varun Vasudev. 
(cherry picked from commit 4be95175cdb58ff12a27ab443d609d3b46da7bfa)
 2014-09-10 19:24:14 -07:00
Jian He
ceae7be6b2 YARN-415. Capture aggregate memory allocation at the app-level for chargeback. Contributed by Eric Payne & Andrey Klochkov 
(cherry picked from commit 83be3ad44484bf8a24cb90de4b9c26ab59d226a8)
 2014-09-10 18:22:07 -07:00
Vinod Kumar Vavilapalli
b4b59ef749 YARN-2448. Changed ApplicationMasterProtocol to expose RM-recognized resource types to the AMs. Contributed by Varun Vasudev. 
(cherry picked from commit b67d5ba7842cc10695d987f217027848a5a8c3d8)
 2014-09-10 10:17:30 -07:00
Jian He
c17810a84c YARN-2515. Updated ConverterUtils#toContainerId to parse epoch. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 0974f434c47ffbf4b77a8478937fd99106c8ddbd)
 2014-09-07 18:31:07 -07:00
Jonathan Eagles
7e76039278 YARN-2509. Enable Cross Origin Filter for timeline server only and not all Yarn servers (Mit Desai via jeagles) 
(cherry picked from commit 91d45f0f017281c8ab7ac858aa5d618accb9ea44)
 2014-09-04 11:54:43 -05:00
Jian He
38853c97c2 YARN-1506. Changed RMNode/SchedulerNode to update resource with event notification. Contributed by Junping Du 
(cherry picked from commit 5c14bc426b4be381383018ebc2236be83eef15cd)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-08-29 23:39:58 -07:00
Jian
518ba11843 YARN-2406. Move RM recovery related proto to yarn_server_resourcemanager_recovery.proto. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 7b3e27ab7393214e35a575bc9093100e94dd8c89)
 2014-08-28 21:54:25 -07:00
Jian He
5c63482aaa YARN-2182. Updated ContainerId#toString() to append RM Epoch number. Contributed by Tsuyoshi OZAWA 
(cherry-picked from commit e2d0ff364a84a4de10e7b11fe83cd3dab155a571)
 2014-08-27 10:11:24 -07:00
Allen Wittenauer
5d965f2f3c YARN-2424. LCE should support non-cgroups, non-secure mode (Chris Douglas via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1619424 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 14:57:53 +00:00
Jason Darrell Lowe
8991c7d1b2 svn merge -c 1619176 FIXES: YARN-2034. Description for yarn.nodemanager.localizer.cache.target-size-mb is incorrect. Contributed by Chen He 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1619178 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-20 17:40:55 +00:00
Allen Wittenauer
b07e476da7 YARN-1918. Typo in description and error message for yarn.resourcemanager.cluster-id (Anandha L Ranganathan via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1618072 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 21:54:02 +00:00
Xuan Gong
4365c4530b YARN-2212 ApplicationMaster needs to find a way to update the AMRMToken periodically. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1616891 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 21:37:35 +00:00
Junping Du
75349f2d83 Merge r1615025 from trunk: YARN-2051. Fix bug in PBimpls and add more unit tests with reflection. (Contributed by Binglin Chang) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1615027 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-01 04:25:49 +00:00
Xuan Gong
ecc257e384 YARN-1994. Expose YARN/MR endpoints on multiple interfaces. Contributed by Craig Welch, Milan Potocnik and Arpit Agarwal 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1615010 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-31 21:29:01 +00:00
Zhijie Shen
f52092be46 YARN-2347. Consolidated RMStateVersion and NMDBSchemaVersion into Version in yarn-server-common. Contributed by Junping Du. 
svn merge --ignore-ancestry -c 1614838 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1614839 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-31 09:31:22 +00:00
Zhijie Shen
b41896f0a2 YARN-2247. Made RM web services authenticate users via kerberos and delegation token. Contributed by Varun Vasudev. 
svn merge --ignore-ancestry -c 1613821 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1613822 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-27 17:57:44 +00:00
Karthik Kambatla
9133e3142c Preparing for 2.6.0 development. Updated versions. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1607541 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-03 06:11:57 +00:00
Vinod Kumar Vavilapalli
8f1f458729 YARN-1713. Added get-new-app and submit-app functionality to RM web services. Contributed by Varun Vasudev. 
svn merge --ignore-ancestry -c 1607216 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1607217 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-02 00:23:49 +00:00
Jian He
a385b77df7 Merge r1606557 from trunk. YARN-2052. Embedded an epoch number in container id to ensure the uniqueness of container id after RM restarts. Contributed by Tsuyoshi OZAWA 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1606558 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-29 18:26:08 +00:00
Jian He
63940d6e13 Merge r1605263 from trunk. YARN-1365. Changed ApplicationMasterService to allow an app to re-register after RM restart. Contributed by Anubhav Dhoot 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1605264 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-25 04:45:50 +00:00
Vinod Kumar Vavilapalli
a2e2c8ad97 YARN-2152. Added missing information into ContainerTokenIdentifier so that NodeManagers can report the same to RM when RM restarts. Contributed Jian He. 
svn merge --ignore-ancestry -c 1605205 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1605206 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 21:44:00 +00:00
Vinod Kumar Vavilapalli
c8dfed19ba YARN-2074. Changed ResourceManager to not count AM preemptions towards app failures. Contributed by Jian He. 
svn merge --ignore-ancestry -c 1605106 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1605107 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 15:16:36 +00:00
Devarajulu K
7741cc62b4 YARN-2195. Clean a piece of code in ResourceRequest. Contributed by Wei Yan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1605085 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 13:58:14 +00:00
Bikas Saha
dc5ee5ff7c Merge 1601762 from trunk to branch-2 for YARN-2091. Add more values to ContainerExitStatus and pass it from NM to RM and then to app masters (Tsuyoshi OZAWA via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1601763 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-10 20:13:23 +00:00
Karthik Kambatla
d4d1ccc992 YARN-1424. RMAppAttemptImpl should return the DummyApplicationResourceUsageReport for all invalid accesses. (Ray Chiang via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1601745 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-10 19:06:32 +00:00
Vinod Kumar Vavilapalli
319e422733 YARN-1368. Added core functionality of recovering container state into schedulers after ResourceManager Restart so as to preserve running work in the cluster. Contributed by Jian He. 
svn merge --ignore-ancestry -c 1601303 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1601304 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-09 03:10:06 +00:00
Karthik Kambatla
33439defa1 YARN-2119. DEFAULT_PROXY_ADDRESS should use DEFAULT_PROXY_PORT. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1600487 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-04 20:18:49 +00:00