Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,373 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1357 Commits

Author SHA1 Message Date
subru fec639cda1 YARN-1712. Plan follower that synchronizes the current state of reservation subsystem with the scheduler. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit 169085319b)
(cherry picked from commit 3418c56bcf)
 2014-10-06 10:21:06 -07:00
carlo curino 6bfdaf06c4 YARN-1711. Policy to enforce instantaneous and over-time quotas on user reservation. Contributed by Carlo Curino and Subru Krishnan. 
(cherry picked from commit c4918cb4cb)
(cherry picked from commit b6df0dddcd)
 2014-10-06 10:21:05 -07:00
carlo curino 6a3c167175 YARN-1710. Logic to find allocations within a Plan that satisfy user ReservationRequest(s). Contributed by Carlo Curino and Subru Krishnan. 
(cherry picked from commit aef7928899)
(cherry picked from commit f66ffcf832)
 2014-10-06 10:21:05 -07:00
subru 63250ef9d6 YARN-1709. In-memory data structures used to track resources over time to enable reservations. 
(cherry picked from commit 0d8b2cd88b)
(cherry picked from commit cf4b34282a)
 2014-10-06 10:21:05 -07:00
subru 3a167056d1 YARN-1708. Public YARN APIs for creating/updating/deleting reservations. 
(cherry picked from commit 3f2e3b275b)
(cherry picked from commit c9266df404)
 2014-10-06 10:21:05 -07:00
carlo curino b81f571e60 YARN-2475. Logic for responding to capacity drops for the ReservationSystem. Contributed by Carlo Curino and Subru Krishnan. 
(cherry picked from commit f83a07f266)
(cherry picked from commit 1c6950354f)
 2014-10-06 10:21:05 -07:00
carlo curino cf5ef00b96 YARN-1707. Introduce APIs to add/remove/resize queues in the CapacityScheduler. Contributed by Carlo Curino and Subru Krishnan 
(cherry picked from commit aac47fda7f)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
(cherry picked from commit eb3e40b833)
 2014-10-06 10:21:04 -07:00
Vinod Kumar Vavilapalli 7ed61e150c YARN-2468. Enhanced NodeManager to support log handling APIs (YARN-2569) for use by long running services. Contributed by Xuan Gong. 
(cherry picked from commit 34cdcaad71)
 2014-10-03 12:17:03 -07:00
Karthik Kambatla b6ce0a1f69 YARN-2635. TestRM, TestRMRestart, TestClientToAMTokens should run with both CS and FS. (Wei Yan and kasha via kasha) 
(cherry picked from commit 80d11eb68e)
 2014-10-03 11:52:05 -07:00
Jian He 1c58ab9206 YARN-2562. Changed ContainerId#toString() to be more readable. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit eb6ce5e97c)
 2014-10-03 11:04:46 -07:00
Jian He 6f4c77409b YARN-2628. Capacity scheduler with DominantResourceCalculator carries out reservation even though slots are free. Contributed by Varun Vasudev 
(cherry picked from commit 054f285526)
 2014-10-02 15:16:07 -07:00
Zhijie Shen ecaae67ab3 YARN-2527. Fixed the potential NPE in ApplicationACLsManager and added test cases for it. Contributed by Benoy Antony. 
(cherry picked from commit 1c93025a1b)
 2014-10-02 14:57:27 -07:00
Jason Lowe 531c1fd00a YARN-2624. Resource Localization fails on a cluster due to existing cache directories. Contributed by Anubhav Dhoot 
(cherry picked from commit 29f520052e)
 2014-10-02 17:40:44 +00:00
Karthik Kambatla b6f4d1ef42 YARN-2254. TestRMWebServicesAppsModification should run against both CS and FS. (Zhihai Xu via kasha) 
(cherry picked from commit 5e0b49da9c)
 2014-10-02 10:15:41 -07:00
Jian He 61c7ceaf82 YARN-2617. Fixed NM to not send duplicate container status whose app is not running. Contributed by Jun Gong 
(cherry picked from commit 3ef1cf187f)
 2014-10-02 10:04:42 -07:00
junping_du 6483342a61 YARN-1979. TestDirectoryCollection fails when the umask is unusual. (Contributed by Vinod Kumar Vavilapalli and Tsuyoshi OZAWA) 
(cherry picked from commit c7cee9b455)
 2014-10-02 08:04:25 -07:00
Vinod Kumar Vavilapalli 34dd2d1d1c YARN-2446. Augmented Timeline service APIs to start taking in domains as a parameter while posting entities and events. Contributed by Zhijie Shen. 
(cherry picked from commit 9e40de6af7)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-01 17:54:12 -07:00
Vinod Kumar Vavilapalli 3326fba382 YARN-1972. Added a secure container-executor for Windows. Contributed by Remus Rusanu. 
commit ba7f31c2ee is the corresponding trunk commit, this is a slightly different patch for branch-2.
 2014-10-01 17:07:21 -07:00
junping_du 625456746c YARN-2613. Support retry in NMClient for rolling-upgrades. (Contributed by Jian He) 2014-10-01 17:08:55 -07:00
Zhijie Shen 4b50e23271 YARN-2630. Prevented previous AM container status from being acquired by the current restarted AM. Contributed by Jian He. 
(cherry picked from commit 52bbe0f11b)
 2014-10-01 15:39:36 -07:00
Vinod Kumar Vavilapalli 64bed0460e YARN-1063. Augmented Hadoop common winutils to have the ability to create containers as domain users. Contributed by Remus Rusanu. 
Committed as a YARN patch even though all the code changes are in common.

(cherry picked from commit 5ca97f1e60)
 2014-10-01 09:55:57 -07:00
Karthik Kambatla 3664659592 YARN-2179. Fix version in sharedcachemanager-pom. 2014-10-01 07:34:24 -07:00
Karthik Kambatla b616c69835 YARN-2179. [YARN-1492] Initial cache manager structure and context. (Chris Trezzo via kasha) 
(cherry picked from commit 17d1202c35)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/bin/yarn
 2014-09-30 19:05:10 -07:00
Jian He 823f02725f YARN-2602. Fixed possible NPE in ApplicationHistoryManagerOnTimelineStore. Contributed by Zhijie Shen 
(cherry picked from commit bbff96be48)
 2014-09-30 16:44:44 -07:00
Karthik Kambatla 5a43b795b7 YARN-2594. Potential deadlock in RM when querying ApplicationResourceUsageReport. (Wangda Tan via kasha) 
(cherry picked from commit 14d60dadc2)
 2014-09-30 16:17:38 -07:00
Jason Lowe a0305ba3fc YARN-2387. Resource Manager crashes with NPE due to lack of synchronization. Contributed by Mit Desai 
(cherry picked from commit feaf139b4f)
 2014-09-30 22:38:27 +00:00
Zhijie Shen d977205b9b YARN-2627. Added the info logs of attemptFailuresValidityInterval and number of previous failed attempts. Contributed by Xuan Gong. 
(cherry picked from commit 9582a50176)
 2014-09-30 12:59:34 -07:00
Karthik Kambatla 21f920ab86 YARN-2610. Hamlet should close table tags. (Ray Chiang via kasha) 
(cherry picked from commit f7743dd07d)
 2014-09-30 11:47:01 -07:00
Arun C. Murthy 6a60fa279a Preparing to branch hadoop-2.6 2014-09-30 10:37:03 -07:00
Jonathan Eagles 9bc676aa6e YARN-2606. Application History Server tries to access hdfs before doing secure login (Mit Desai via jeagles) 2014-09-29 16:40:46 -05:00
Jason Lowe 4f6a367166 YARN-1769. CapacityScheduler: Improve reservations. Contributed by Thomas Graves 
(cherry picked from commit 9c22065109)
 2014-09-29 14:13:21 +00:00
Jian He cb08ed1484 YARN-668. Changed NMTokenIdentifier/AMRMTokenIdentifier/ContainerTokenIdentifier to use protobuf object as the payload. Contributed by Junping Du. 
(cherry picked from commit 5391919b09)
 2014-09-26 17:53:35 -07:00
Allen Wittenauer 6f5d555ad0 YARN-2372. There are Chinese Characters in the FairScheduler's document (Fengdong Yu via aw) 2014-09-26 12:47:26 -07:00
Allen Wittenauer f694b44036 YARN-2577. Clarify ACL delimiter and how to configure ACL groups only (Mikos Christine via aw) 2014-09-26 11:06:04 -07:00
Karthik Kambatla b923c291b4 YARN-2608. FairScheduler: Potential deadlocks in loading alloc files and clock access. (Wei Yan via kasha) 
(cherry picked from commit c9811af09a3d3f9f2f1b86fc9d6f2763d3225e44)
 2014-09-25 17:43:08 -07:00
Jason Lowe e4d46e5ff6 YARN-2523. ResourceManager UI showing negative value for "Decommissioned Nodes" field. Contributed by Rohith 
(cherry picked from commit 8269bfa613)
 2014-09-25 22:39:57 +00:00
Zhijie Shen 83926b3c57 YARN-2546. Made REST API for application creation/submission use numeric and boolean types instead of the string of them. Contributed by Varun Vasudev. 
(cherry picked from commit 72b0881ca6)
 2014-09-24 17:58:48 -07:00
Zhijie Shen 3a2e400377 YARN-2581. Passed LogAggregationContext to NM via ContainerTokenIdentifier. Contributed by Xuan Gong. 
(cherry picked from commit c86674a3a4)
 2014-09-24 17:51:54 -07:00
Karthik Kambatla 30165d4cdd YARN-2596. TestWorkPreservingRMRestart fails with FairScheduler. (kasha) 
(cherry picked from commit 39c87344e1)
 2014-09-24 15:24:17 -07:00
Vinod Kumar Vavilapalli 5d251d99d6 YARN-2102. Added the concept of a Timeline Domain to handle read/write ACLs on Timeline service event data. Contributed by Zhijie Shen. 
(cherry picked from commit d78b452a4f)
 2014-09-24 11:29:01 -07:00
Allen Wittenauer e5a531a8a7 YARN-2161. Fix build on macosx: YARN parts (Binglin Chang via aw) 2014-09-24 08:48:42 -07:00
Zhijie Shen 19cfe2f800 YARN-2569. Added the log handling APIs for the long running services. Contributed by Xuan Gong. 
(cherry picked from commit 5338ac416a)
 2014-09-23 10:38:27 -07:00
Karthik Kambatla 024bcfdc0d YARN-2252. Intermittent failure of TestFairScheduler.testContinuousScheduling. (Ratandeep Ratti and kasha via kasha) 
(cherry picked from commit f5578207d2)
 2014-09-23 00:03:47 -07:00
Karthik Kambatla 8ba8521de5 YARN-1959. Fix headroom calculation in FairScheduler. (Anubhav Dhoot via kasha) 
(cherry picked from commit 568d3dc2bb)
 2014-09-22 23:51:45 -07:00
junping_du 34ee084c3d YARN-2584. TestContainerManagerSecurity fails on trunk. (Contributed by Jian He) 
(cherry picked from commit a9a55db065)
 2014-09-22 22:47:59 -07:00
Karthik Kambatla 3984435661 YARN-2539. FairScheduler: Set the default value for maxAMShare to 0.5. (Wei Yan via kasha) 
(cherry picked from commit 43efdd30b5)
 2014-09-22 16:10:20 -07:00
Karthik Kambatla 24a814771e YARN-2540. FairScheduler: Queue filters not working on scheduler page in RM UI. (Ashwin Shankar via kasha) 
(cherry picked from commit 23e17ce0ff)
 2014-09-22 15:42:36 -07:00
Zhijie Shen b8d645539d MAPREDUCE-5279. Made MR headroom calculation honor cpu dimension when YARN scheduler resource type is memory plus cpu. Contributed by Peng Zhang and Varun Vasudev. 2014-09-22 10:41:55 -07:00
Jian He 3ce97a9efd YARN-1372. Ensure all completed containers are reported to the AMs across RM restart. Contributed by Anubhav Dhoot 
(cherry picked from commit 0a641496c7)
 2014-09-22 10:32:44 -07:00
Karthik Kambatla c1a3819a4d YARN-2453. TestProportionalCapacityPreemptionPolicy fails with FairScheduler. (Zhihai Xu via kasha) 
(cherry picked from commit 9721e2c1fe)
 2014-09-21 23:14:27 -07:00