Weiwei Yang
|
b451889e8e
|
YARN-8011. TestOpportunisticContainerAllocatorAMService#testContainerPromoteAndDemoteBeforeContainerStart fails intermittently. Contributed by Tao Yang.
|
2018-03-08 18:13:36 +08:00 |
Wangda Tan
|
e718ac597f
|
Revert "YARN-7891. LogAggregationIndexedFileController should support read from HAR file. (Xuan Gong via wangda)"
This reverts commit 4d53ef7eef .
|
2018-03-07 15:46:47 -08:00 |
Wangda Tan
|
4d53ef7eef
|
YARN-7891. LogAggregationIndexedFileController should support read from HAR file. (Xuan Gong via wangda)
Change-Id: Ie16e34039d57df50128c73b37516ad0bc7c9590e
|
2018-03-07 11:30:06 -08:00 |
Arun Suresh
|
1054b48c27
|
YARN-7972. Support inter-app placement constraints for allocation tags by application ID. (Weiwei Yang via asuresh)
|
2018-03-05 11:24:17 -08:00 |
Yufei Gu
|
6ca4e3017b
|
YARN-5028. RMStateStore should trim down app state for completed applications(addendum). Contributed by Rohith Sharma K S.
|
2018-03-01 08:38:53 -08:00 |
Arpit Agarwal
|
2fa7963c3d
|
HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi.
|
2018-02-24 14:41:55 -08:00 |
Carlo Curino
|
514794e1a5
|
YARN-7934. [GQ] Refactor preemption calculators to allow overriding for Federation Global Algos. (Contributed by curino)
|
2018-02-22 18:12:12 -08:00 |
Yufei Gu
|
92cbbfe79e
|
YARN-5028. RMStateStore should trim down app state for completed applications. Contributed by Gergo Repas.
|
2018-02-21 11:42:51 -08:00 |
Sunil G
|
bdd2a184d7
|
YARN-7947. Capacity Scheduler intra-queue preemption can NPE for non-schedulable apps. Contributed by Eric Payne.
|
2018-02-21 14:35:57 +05:30 |
Jason Lowe
|
9497215084
|
YARN-7813. Capacity Scheduler Intra-queue Preemption should be configurable for each queue. Contributed by Eric Payne
|
2018-02-19 14:06:28 -06:00 |
fang zhenyi
|
4d4dde5112
|
HADOOP-15223. Replace Collections.EMPTY* with empty* when available
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
|
2018-02-18 22:19:39 +09:00 |
Arun Suresh
|
a1e56a6286
|
YARN-7918. Fix TestAMRMClientPlacementConstraints. (Gergely Novák via asuresh)
|
2018-02-17 03:24:55 -08:00 |
Sunil G
|
aae629913c
|
YARN-7292. Retrospect Resource Profile Behavior for overriding capability. Contributed by Wangda Tan.
|
2018-02-16 12:37:40 +05:30 |
Konstantinos Karanasos
|
0b489e564c
|
YARN-7920. Simplify configuration for PlacementConstraints. Contributed by Wangda Tan.
|
2018-02-15 14:23:38 -08:00 |
Eric Payne
|
bddfe42e2c
|
Revert "YARN-7813: Capacity Scheduler Intra-queue Preemption should be configurable for each queue"
This reverts commit c5e6e3de1c .
|
2018-02-14 14:29:20 -06:00 |
Wangda Tan
|
042ef2fa7b
|
YARN-7789. Should fail RM if 3rd resource type is configured but RM uses DefaultResourceCalculator. (Zian Chen via wangda)
Change-Id: I1f6a451fe16758def6f47c046a9b8a67ad7255af
|
2018-02-14 23:11:10 +08:00 |
Eric Payne
|
c5e6e3de1c
|
YARN-7813: Capacity Scheduler Intra-queue Preemption should be configurable for each queue
|
2018-02-13 10:11:02 -06:00 |
Wangda Tan
|
d02e42cee4
|
YARN-7739. DefaultAMSProcessor should properly check customized resource types against minimum/maximum allocation. (wangda)
Change-Id: I10cc9341237d9a2fc0f8c855efb98a36b91389e2
|
2018-02-12 10:29:37 +08:00 |
Wangda Tan
|
60f9e60b3b
|
Preparing for 3.2.0 development
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
|
2018-02-11 11:17:38 +08:00 |
Yufei Gu
|
1bc03ddf97
|
YARN-7655. Avoid AM preemption caused by RRs for specific nodes or racks. Contributed by Steven Rand.
|
2018-02-08 12:32:43 -08:00 |
Robert Kanter
|
814d701d46
|
YARN-7841. Cleanup AllocationFileLoaderService's reloadAllocations method (snemeth via rkanter)
|
2018-02-06 14:36:49 -08:00 |
Weiwei Yang
|
bff858e910
|
YARN-7860. Fix UT failure TestRMWebServiceAppsNodelabel#testAppsRunning. Contributed by Sunil G.
|
2018-02-06 19:33:04 +08:00 |
Konstantinos Karanasos
|
50723889cc
|
YARN-7778. Merging of placement constraints defined at different levels. Contributed by Weiwei Yang.
|
2018-02-02 14:46:20 -08:00 |
Arun Suresh
|
6e5ba9366f
|
YARN-7839. Modify PlacementAlgorithm to Check node capacity before placing request on node. (Panagiotis Garefalakis via asuresh)
|
2018-02-02 10:28:22 -08:00 |
Arun Suresh
|
d481344783
|
YARN-7822. Constraint satisfaction checker support for composite OR and AND constraints. (Weiwei Yang via asuresh)
|
2018-01-31 01:51:08 -08:00 |
Arun Suresh
|
add993e26a
|
YARN-6597. Add RMContainer recovery test to verify tag population in the AllocationTagsManager. (Panagiotis Garefalakis via asuresh)
|
2018-01-31 01:30:17 -08:00 |
Arun Suresh
|
f8c5f5b237
|
YARN-7784. Fix Cluster metrics when placement processor is enabled. (asuresh)
|
2018-01-31 01:30:17 -08:00 |
Arun Suresh
|
c23980c4f2
|
YARN-7795. Fix jenkins issues of YARN-6592 branch. (Sunil G via asuresh)
|
2018-01-31 01:30:17 -08:00 |
Arun Suresh
|
644afe5fd8
|
YARN-7807. Assume intra-app anti-affinity as default for scheduling request inside AppPlacementAllocator. (Wangda Tan via asuresh)
|
2018-01-31 01:30:17 -08:00 |
Arun Suresh
|
a4c539fcdb
|
YARN-7783. Add validation step to ensure constraints are not violated due to order in which a request is processed. (asuresh)
|
2018-01-31 01:30:17 -08:00 |
Sunil G
|
9b81cb0537
|
YARN-7779. Display allocation tags in RM web UI and expose same through REST API. Contributed by Weiwei Yang.
|
2018-01-31 01:30:17 -08:00 |
Konstantinos Karanasos
|
adbe87abf8
|
YARN-7788. Factor out management of temp tags from AllocationTagsManager. (Arun Suresh via kkaranasos)
|
2018-01-31 01:30:17 -08:00 |
Arun Suresh
|
8bf7c44436
|
YARN-7763. Allow Constraints specified in the SchedulingRequest to override application level constraints. (Weiwei Yang via asuresh)
|
2018-01-31 01:30:17 -08:00 |
Arun Suresh
|
28fe7f3318
|
YARN-7774. Miscellaneous fixes to the PlacementProcessor. (asuresh)
|
2018-01-31 01:30:17 -08:00 |
Arun Suresh
|
38af237969
|
YARN-6599. Support anti-affinity constraint via AppPlacementAllocator. (Wangda Tan via asuresh)
|
2018-01-31 01:30:17 -08:00 |
Arun Suresh
|
8779a35742
|
YARN-7709. Remove SELF from TargetExpression type. (Konstantinos Karanasos via asuresh)
|
2018-01-31 01:30:17 -08:00 |
Wangda Tan
|
29d9e4d581
|
YARN-6619. AMRMClient Changes to use the PlacementConstraint and SchcedulingRequest objects. (Arun Suresh via wangda)
|
2018-01-31 01:30:17 -08:00 |
Arun Suresh
|
a5c1fc881e
|
YARN-7696. Add container tags to ContainerTokenIdentifier, api.Container and NMContainerStatus to handle all recovery cases. (asuresh)
|
2018-01-31 01:30:17 -08:00 |
Arun Suresh
|
4eda58c136
|
YARN-7681. Double-check placement constraints in scheduling phase before actual allocation is made. (Weiwei Yang via asuresh)
|
2018-01-31 01:30:17 -08:00 |
Arun Suresh
|
bdba01f73b
|
YARN-7682. Expose canSatisfyConstraints utility function to validate a placement against a constraint. (Panagiotis Garefalakis via asuresh)
|
2018-01-31 01:30:17 -08:00 |
Arun Suresh
|
a52d11fb8c
|
YARN-7613. Implement Basic algorithm for constraint based placement. (Panagiotis Garefalakis via asuresh)
|
2018-01-31 01:30:17 -08:00 |
Arun Suresh
|
f9af15d659
|
YARN-7612. Add Processor Framework for Rich Placement Constraints. (asuresh)
|
2018-01-31 01:30:17 -08:00 |
Arun Suresh
|
1efb2b6f25
|
YARN-6596. Introduce Placement Constraint Manager module. (Konstantinos Karanasos via asuresh)
|
2018-01-31 01:30:17 -08:00 |
Arun Suresh
|
37f1a7b64f
|
YARN-7653. Node group support for AllocationTagsManager. (Panagiotis Garefalakis via asuresh)
|
2018-01-31 01:30:17 -08:00 |
Arun Suresh
|
06eb63e64b
|
YARN-7669. API and interface modifications for placement constraint processor. (asuresh)
|
2018-01-31 01:30:17 -08:00 |
Arun Suresh
|
88d8d3f40b
|
YARN-7670. Modifications to the ResourceScheduler API to support SchedulingRequests. (asuresh)
|
2018-01-31 01:30:17 -08:00 |
Arun Suresh
|
801c0988b5
|
YARN-7522. Introduce AllocationTagsManager to associate allocation tags to nodes. (Wangda Tan via asuresh)
|
2018-01-31 01:30:17 -08:00 |
Sunil G
|
e9c72d04be
|
YARN-7790. Improve Capacity Scheduler Async Scheduling to better handle node failures. Contributed by Wangda Tan.
|
2018-01-29 20:44:38 +05:30 |
Jinjiang Ling
|
3400d0c535
|
YARN-7698. A misleading variable's name in ApplicationAttemptEventDispatcher
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
|
2018-01-30 00:00:57 +09:00 |
Wangda Tan
|
75618c0dec
|
YARN-7760. [UI2] Clicking 'Master Node' or link next to 'AM Node Web UI' under application's appAttempt page goes to OLD RM UI. (Vasudevan Skm / Sunil G via wangda)
Change-Id: Ia7e2b955d3551ca43c6852fe253fd01983b56565
|
2018-01-26 15:44:55 +08:00 |
Wangda Tan
|
e0cfb0a31a
|
YARN-7817. Add Resource reference to RM's NodeInfo object so REST API can get non memory/vcore resource usages. (Sunil G via wangda)
Change-Id: Ia7ceeabd82046645ddeaf487c763288f36cfbdee
|
2018-01-26 15:43:27 +08:00 |
Jason Lowe
|
ff8378eb1b
|
YARN-7102. NM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang
|
2018-01-25 17:47:19 -06:00 |
Sunil G
|
82cc6f6968
|
YARN-7728. Expose container preemptions related information in Capacity Scheduler queue metrics. Contributed by Eric Payne.
|
2018-01-25 14:32:54 +05:30 |
Billie Rinaldi
|
e307edcb47
|
YARN-7540 and YARN-7605. Convert yarn app cli to call yarn api services and implement doAs for Api Service REST API. Contributed by Eric Yang and Jian He
|
2018-01-23 17:54:39 -08:00 |
Yufei Gu
|
22ee6f77e5
|
YARN-7755. Clean up deprecation messages for allocation increments in FS config. Contributed by Wilfred Spiegelenburg.
|
2018-01-22 10:30:01 -08:00 |
Miklos Szegedi
|
bc93ac229e
|
YARN-7139. FairScheduler: finished applications are always restored to default queue. Contributed by Wilfred Spiegelenburg.
|
2018-01-18 17:43:47 -08:00 |
Sunil G
|
06cceba1cb
|
YARN-7738. CapacityScheduler: Support refresh maximum allocation for multiple resource types. Contributed by Wangda Tan.
|
2018-01-18 19:05:26 +05:30 |
Yufei Gu
|
370f1c6283
|
YARN-6486. FairScheduler: Deprecate continuous scheduling. (Contributed by Wilfred Spiegelenburg)
|
2018-01-16 14:46:55 -08:00 |
Rohith Sharma K S
|
d09058b2fd
|
YARN-6736. Consider writing to both ats v1 & v2 from RM for smoother upgrades. Contributed by Aaron Gresch.
|
2018-01-16 07:58:29 +05:30 |
Sunil G
|
128d773a23
|
YARN-7727. Incorrect log levels in few logs with QueuePriorityContainerCandidateSelector. Contributed by Prabhu Joseph.
|
2018-01-12 15:41:19 +05:30 |
Sunil G
|
afd8caba27
|
YARN-7722. Rename variables in MockNM, MockRM for better clarity. Contributed by Lovekesh bansal
|
2018-01-10 20:51:34 +05:30 |
Miklos Szegedi
|
55066cc53d
|
YARN-7689. TestRMContainerAllocator fails after YARN-6124. Contributed by Wilfred Spiegelenburg.
|
2018-01-09 16:59:39 -08:00 |
Haibo Chen
|
8ee7080e5d
|
YARN-7665. Allow FS scheduler state dump to be turned on/off separately from FS debug log. (Wilfred Spiegelenburg via Haibo Chen)
|
2018-01-09 16:45:35 -08:00 |
Jason Lowe
|
90c86d6e09
|
YARN-6948. Invalid event: ATTEMPT_ADDED at FINAL_SAVING. Contributed by lujie
|
2018-01-09 10:51:15 -06:00 |
Jason Lowe
|
ebff4decfe
|
YARN-7663. RMAppImpl:Invalid event: START at KILLED. Contributed by lujie
|
2018-01-09 09:52:59 -06:00 |
bibinchundatt
|
b26e30ab16
|
YARN-7508. NPE in FiCaSchedulerApp when debug log enabled in async-scheduling mode. Contributed by Tao Yang.
|
2018-01-09 17:57:01 +05:30 |
Ray Chiang
|
59ab5da0a0
|
YARN-4227. Ignore expired containers from removed nodes in FairScheduler. (Wilfred Spiegelenburg via rchiang)
|
2018-01-08 16:19:02 -08:00 |
Rohith Sharma K S
|
c2d6fa3656
|
YARN-7699. queueUsagePercentage is coming as INF for getApp REST api call. Contributed by Sunil G.
|
2018-01-08 10:39:31 +05:30 |
Wangda Tan
|
a81144daa0
|
YARN-7666. Introduce scheduler specific environment variable support in ApplicationSubmissionContext for better scheduling placement configurations. (Sunil G via wangda)
Change-Id: I0fd826490f5160d47d42af2a9ac0bd8ec4e959dc
|
2018-01-05 15:12:04 -08:00 |
Robert Kanter
|
2aa4f0a559
|
YARN-7645. TestContainerResourceUsage#testUsageAfterAMRestartWithMultipleContainers is flakey with FairScheduler (rkanter)
|
2018-01-05 13:55:09 -08:00 |
Robert Kanter
|
f8e7dd9b10
|
YARN-7557. It should be possible to specify resource types in the fair scheduler increment value (grepas via rkanter)
|
2018-01-05 11:15:06 -08:00 |
Sunil G
|
0c75d0634b
|
YARN-7619. Max AM Resource value in Capacity Scheduler UI has to be refreshed for every user. Contributed by Eric Payne.
|
2018-01-05 14:42:17 +05:30 |
Robert Kanter
|
7a55044803
|
YARN-7622. Allow fair-scheduler configuration on HDFS (gphillips via rkanter)
|
2018-01-03 15:31:50 -08:00 |
Rohith Sharma K S
|
c9bf813c9a
|
YARN-7692. Skip validating priority acls while recovering applications. Contributed by Sunil G.
|
2018-01-03 18:20:04 +05:30 |
Robert Kanter
|
382215c72b
|
YARN-7577. Unit Fail: TestAMRestart#testPreemptedAMRestartOnRMRestart (miklos.szegedi@cloudera.com via rkanter)
|
2017-12-20 13:39:00 -08:00 |
Eric Yang
|
94a2ac6b71
|
YARN-7466. addendum patch for failing unit test. (Contributed by Chandni Singh)
|
2017-12-19 18:42:27 -05:00 |
Akira Ajisaka
|
001008958d
|
YARN-7664. Several javadoc errors. Contributed by Sean Mackrory.
|
2017-12-18 22:24:51 +09:00 |
Rohith Sharma K S
|
09d996fdd4
|
YARN-7190. Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath. Contributed by Varun Saxena.
|
2017-12-15 21:50:28 +05:30 |
Sunil G
|
890d3d0645
|
YARN-7638. Unit tests related to preemption for auto created leaf queues feature.Contributed by Suma Shivaprasad.
|
2017-12-15 13:00:57 +05:30 |
Sunil G
|
cb87e4dc92
|
YARN-7643. Handle recovery of applications in case of auto-created leaf queue mapping. Contributed by Suma Shivaprasad.
|
2017-12-13 22:49:58 +05:30 |
Sunil G
|
8bb83a8f62
|
Queue ACL validations should validate parent queue ACLs before auto-creating leaf queues. Contributed by Suma Shivaprasad.
|
2017-12-12 15:20:59 +05:30 |
Sunil G
|
5c87fb2f62
|
YARN-7635. TestRMWebServicesSchedulerActivities fails in trunk. Contributed by Sunil G.
|
2017-12-12 15:08:18 +05:30 |
Sunil G
|
312ceebde8
|
YARN-7632. Effective min and max resource need to be set for auto created leaf queues upon creation and capacity management. Contributed by Suma Shivaprasad.
|
2017-12-11 19:20:02 +05:30 |
Wangda Tan
|
adca1a72e4
|
YARN-7591. NPE in async-scheduling mode of CapacityScheduler. (Tao Yang via wangda)
Change-Id: I46689e530550ee0a6ac7a29786aab2cc1bdf314f
|
2017-12-08 15:17:02 -08:00 |
Wangda Tan
|
a8316df8c0
|
YARN-7520. Queue Ordering policy changes for ordering auto created leaf queues within Managed parent Queues. (Suma Shivaprasad via wangda)
Change-Id: I482f086945bd448d512cb5b3879d7371e37ee134
|
2017-12-08 15:11:28 -08:00 |
Wangda Tan
|
f548bfffbd
|
YARN-7420. YARN UI changes to depict auto created queues. (Suma Shivaprasad via wangda)
Change-Id: I8039d3772a191ddede132cd1f8b08a8ca2e275b7
|
2017-12-08 15:10:47 -08:00 |
Wangda Tan
|
b38643c9a8
|
YARN-7473. Implement Framework and policy for capacity management of auto created queues. (Suma Shivaprasad via wangda)
Change-Id: Icca7805fe12f6f7fb335effff4b121b6f7f6337b
|
2017-12-08 15:10:16 -08:00 |
Wangda Tan
|
74665e3a7d
|
YARN-7274. Ability to disable elasticity at leaf queue level. (Zian Chen via wangda)
Change-Id: Ic8d43e297f0f5de788b562f7eff8106c5c35e8d2
|
2017-12-08 15:07:56 -08:00 |
Sunil G
|
4db4a4a165
|
YARN-7575. NPE in scheduler UI when max-capacity is not configured. Contributed by Sunil G.
|
2017-12-07 18:56:54 -08:00 |
Sunil G
|
daa1cdd062
|
YARN-7564. Cleanup to fix checkstyle issues of YARN-5881 branch. Contributed by Sunil G.
|
2017-12-07 18:56:54 -08:00 |
Wangda Tan
|
1012b901c8
|
YARN-7544. Use queue-path.capacity/maximum-capacity to specify absolute min/max resources. (Sunil G via wangda)
Change-Id: I685341be213eee500f51e02f01c91def89391c17
|
2017-12-07 18:56:54 -08:00 |
Wangda Tan
|
b7b8cd5324
|
YARN-7538. Fix performance regression introduced by Capacity Scheduler absolute min/max resource refactoring. (Sunil G via wangda)
Change-Id: Ic9bd7e599c56970fe01cb0e1bba6df7d1f77eb29
|
2017-12-07 18:56:54 -08:00 |
Wangda Tan
|
7462c38277
|
YARN-7483. CapacityScheduler test cases cleanup post YARN-5881. (Sunil G via wangda)
Change-Id: I9741a6baf5cb7352d05636efb6c0b24790e7589a
|
2017-12-07 18:56:54 -08:00 |
Rohith Sharma K S
|
e65ca92fb6
|
YARN-7482. Max applications calculation per queue has to be retrospected with absolute resource support. Contributed by Sunil G.
|
2017-12-07 18:56:54 -08:00 |
Wangda Tan
|
034b312d9f
|
YARN-7411. Inter-Queue preemption's computeFixpointAllocation need to handle absolute resources while computing normalizedGuarantee. (Sunil G via wangda)
Change-Id: I41b1d7558c20fc4eb2050d40134175a2ef6330cb
|
2017-12-07 18:56:54 -08:00 |
Wangda Tan
|
aa3f62740f
|
YARN-7332. Compute effectiveCapacity per each resource vector. (Sunil G via wangda)
|
2017-12-07 18:56:54 -08:00 |
Wangda Tan
|
d52627a7cb
|
YARN-7254. UI and metrics changes related to absolute resource configuration. (Sunil G via wangda)
|
2017-12-07 18:56:54 -08:00 |
Wangda Tan
|
5e798b1a0d
|
YARN-6471. Support to add min/max resource configuration for a queue. (Sunil G via wangda)
Change-Id: I9213f5297a6841fab5c573e85ee4c4e5f4a0b7ff
|
2017-12-07 18:56:54 -08:00 |
Sunil G
|
a957f1c60e
|
YARN-7438. Additional changes to make SchedulingPlacementSet agnostic to ResourceRequest / placement algorithm. Contributed by Wangda Tan
|
2017-12-05 22:50:07 +05:30 |
Sunil G
|
f9f317b702
|
YARN-7586. Application Placement should be done before ACL checks in ResourceManager. Contributed by Suma Shivaprasad.
|
2017-12-05 18:28:31 +05:30 |
Robert Kanter
|
d8863fc16f
|
YARN-5594. Handle old RMDelegationToken format when recovering RM (rkanter)
|
2017-12-04 13:14:55 -08:00 |
Arun Suresh
|
37ca416950
|
YARN-7587. Skip dispatching opportunistic containers to nodes whose queue is already full. (Weiwei Yang via asuresh)
|
2017-12-03 22:22:01 -08:00 |
Robert Kanter
|
c83fe44917
|
YARN-4813. TestRMWebServicesDelegationTokenAuthentication.testDoAs fails intermittently (grepas via rkanter)
|
2017-12-01 12:18:13 -08:00 |
Wangda Tan
|
a63d19d365
|
YARN-6124. Make SchedulingEditPolicy can be enabled / disabled / updated with RMAdmin -refreshQueues. (Zian Chen via wangda)
Change-Id: Id93656f3af7dcd78cafa94e33663c78d410d43c2
|
2017-11-30 15:57:22 -08:00 |
Daniel Templeton
|
8498d287cd
|
YARN-7541. Node updates don't update the maximum cluster capability for resources other than CPU and memory
|
2017-11-29 11:11:36 -08:00 |
Jason Lowe
|
a2c7a73e33
|
YARN-6647. RM can crash during transitionToStandby due to InterruptedException. Contributed by Bibin A Chundatt
|
2017-11-28 11:15:44 -06:00 |
Jian He
|
fedabcad42
|
YARN-6168. Restarted RM may not inform AM about all existing containers. Contributed by Chandni Singh
|
2017-11-27 10:19:58 -08:00 |
Yufei Gu
|
2bde3aedf1
|
YARN-7290. Method canContainerBePreempted can return true when it shouldn't. (Contributed by Steven Rand)
|
2017-11-24 23:32:46 -08:00 |
Wangda Tan
|
834e91ee91
|
YARN-7509. AsyncScheduleThread and ResourceCommitterService are still running after RM is transitioned to standby. (Tao Yang via wangda)
Change-Id: I7477fe355419fd4a0a6e2bdda7319abad4c4c748
|
2017-11-23 19:59:03 -08:00 |
Arun Suresh
|
b46ca7e73b
|
YARN-6483. Add nodes transitioning to DECOMMISSIONING state to the list of updated nodes returned to the AM. (Juan Rodriguez Hortala via asuresh)
|
2017-11-22 19:18:30 -08:00 |
Yufei Gu
|
4cc9479dae
|
YARN-7524. Remove unused FairSchedulerEventLog. (Contributed by Wilfred Spiegelenburg)
|
2017-11-22 14:18:36 -08:00 |
yufei
|
03c311eae3
|
YARN-7513. Remove the scheduler lock in FSAppAttempt.getWeight() (Contributed by Wilfred Spiegelenburg)
|
2017-11-21 10:33:34 -08:00 |
Wangda Tan
|
0d781dd03b
|
YARN-7527. Over-allocate node resource in async-scheduling mode of CapacityScheduler. (Tao Yang via wangda)
Change-Id: I51ae6c2ab7a3d1febdd7d8d0519b63a13295ac7d
|
2017-11-20 11:48:15 -08:00 |
bibinchundatt
|
b5b81a4f08
|
YARN-7489. ConcurrentModificationException in RMAppImpl#getRMAppMetrics. Contributed by Tao Yang.
|
2017-11-18 19:25:29 +05:30 |
Eric Yang
|
0940e4f692
|
YARN-7218. Decouple YARN Services REST API namespace from RM. (Contributed by Eric Yang)
|
2017-11-17 12:28:12 -05:00 |
Wangda Tan
|
0987a7b8cb
|
YARN-7419. CapacityScheduler: Allow auto leaf queue creation after queue mapping. (Suma Shivaprasad via wangda)
Change-Id: Ia1704bb8cb5070e5b180b5a85787d7b9ca57ebc6
|
2017-11-16 11:25:52 -08:00 |
Sunil G
|
61ace174cd
|
YARN-7469. Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit. Contributed by Eric Payne.
|
2017-11-16 22:34:23 +05:30 |
Daniel Templeton
|
b246c54749
|
YARN-7414. FairScheduler#getAppWeight() should be moved into FSAppAttempt#getWeight()
(Contributed by Soumabrata Chakraborty via Daniel Templeton)
|
2017-11-15 10:03:29 -08:00 |
Wangda Tan
|
dd07038ffa
|
YARN-6909. Use LightWeightedResource when number of resource types more than two. (Sunil G via wangda)
Change-Id: I90e021c5dea7abd9ec6bd73b2287c8adebe14595
|
2017-11-09 14:51:15 -08:00 |
Konstantinos Karanasos
|
ac4d2b1081
|
YARN-7437. Rename PlacementSet and SchedulingPlacementSet. (Wangda Tan via kkaranasos)
|
2017-11-09 13:01:24 -08:00 |
Robert Kanter
|
a2c150a736
|
YARN-7386. Duplicate Strings in various places in Yarn memory (misha@cloudera.com via rkanter)
|
2017-11-09 12:12:52 -08:00 |
Haibo Chen
|
a1382a18df
|
YARN-7388. TestAMRestart should be scheduler agnostic.
|
2017-11-09 10:49:50 -08:00 |
bibinchundatt
|
0a72c2f56c
|
YARN-7454. RMAppAttemptMetrics#getAggregateResourceUsage can NPE due to double lookup. Contributed by Jason Lowe.
|
2017-11-09 21:01:19 +05:30 |
Arun Suresh
|
a9c70b0e84
|
YARN-7453. Fix issue where RM fails to switch to active after first successful start. (Rohith Sharma K S via asuresh)
|
2017-11-08 08:00:53 -08:00 |
Daniel Templeton
|
8db9d61ac2
|
YARN-7401. Reduce lock contention in ClusterNodeTracker#getClusterCapacity()
|
2017-11-07 14:53:48 -08:00 |
Wangda Tan
|
13fa2d4e3e
|
YARN-7394. Merge code paths for Reservation/Plan queues and Auto Created queues. (Suma Shivaprasad via wangda)
|
2017-11-06 21:38:24 -08:00 |
Haibo Chen
|
8f214dc4f8
|
YARN-7360. TestRM.testNMTokenSentForNormalContainer() should be scheduler agnostic.
|
2017-11-06 15:45:37 -08:00 |
Jian He
|
a55d0738f1
|
YARN-7371. Added allocateRequestId in NMContainerStatus for recovery. Contributed by Chandni Singh
|
2017-11-06 13:30:20 -08:00 |
Jian He
|
c723021579
|
YARN-6626. Embed REST API service into RM. Contributed by Eric Yang
|
2017-11-06 13:30:17 -08:00 |
Jian He
|
673c0db43c
|
Revert "YARN-6626. Embed REST API service into RM. Contributed by Eric Yang"
This reverts commit 63d1084e9781e0fee876916190b69f6242dd00e4.
|
2017-11-06 13:30:17 -08:00 |
Jian He
|
9e677fa05c
|
YARN-6626. Embed REST API service into RM. Contributed by Eric Yang
|
2017-11-06 13:30:17 -08:00 |
Jian He
|
bd96c4c235
|
Rebase onto latest trunk. minor conflicts
|
2017-11-06 13:30:13 -08:00 |
Billie Rinaldi
|
1888318c89
|
YARN-6903. Yarn-native-service framework core rewrite. Contributed by Jian He
|
2017-11-06 13:30:11 -08:00 |
Eric Payne
|
e6ec02001f
|
YARN-7370: Preemption properties should be refreshable. Contrubted by Gergely Novák.
|
2017-11-02 12:37:33 -05:00 |
Rohith Sharma K S
|
940ffe3f9c
|
addendum patch for YARN-7289.
|
2017-11-02 13:55:19 +05:30 |
Daniel Templeton
|
9711b78998
|
YARN-7374. Improve performance of DRF comparisons for resource types in fair scheduler
|
2017-10-29 18:54:33 -07:00 |
Yufei Gu
|
d4811c8cfa
|
YARN-6747. TestFSAppStarvation.testPreemptionEnable fails intermittently. (Contributed by Miklos Szegedi)
|
2017-10-29 16:44:16 -07:00 |
Daniel Templeton
|
e62bbbca7a
|
YARN-7397. Reduce lock contention in FairScheduler#getAppWeight()
|
2017-10-28 09:13:13 -07:00 |
Rohith Sharma K S
|
5c799ecf09
|
YARN-7289. Application lifetime does not work with FairScheduler. Contributed by Miklos Szegedi.
|
2017-10-27 22:46:38 +05:30 |
Sunil G
|
792388e1c0
|
YARN-7375. Possible NPE in RMWebapp when HA is enabled and the active RM fails. Contributed by Chandni Singh.
|
2017-10-27 20:53:57 +05:30 |
Wangda Tan
|
36e158ae98
|
YARN-7307. Allow client/AM update supported resource types via YARN APIs. (Sunil G via wangda)
Change-Id: I14c5ea7252b7c17e86ab38f692b5f9d43196dbe0
|
2017-10-26 20:15:19 -07:00 |
Robert Kanter
|
b1de78619f
|
YARN-7262. Add a hierarchy into the ZKRMStateStore for delegation token znodes to prevent jute buffer overflow (rkanter)
|
2017-10-26 17:47:32 -07:00 |
Subru Krishnan
|
25932da6d1
|
YARN-5516. Add REST API for supporting recurring reservations. (Sean Po via Subru).
|
2017-10-26 12:10:14 -07:00 |
Robert Kanter
|
2da654e34a
|
YARN-7358. TestZKConfigurationStore and TestLeveldbConfigurationStore should explicitly set capacity scheduler (haibochen via rkanter)
|
2017-10-26 10:25:10 -07:00 |
Subru Krishnan
|
3fae675383
|
YARN-4827. Document configuration of ReservationSystem for FairScheduler. (Yufei Gu via Subru).
|
2017-10-25 15:07:50 -07:00 |
Haibo Chen
|
d7f3737f3b
|
YARN-7389. Make TestResourceManager Scheduler agnostic. (Robert Kanter via Haibo Chen)
|
2017-10-24 22:17:56 -07:00 |
Robert Kanter
|
03af442e76
|
YARN-7385. TestFairScheduler#testUpdateDemand and TestFSLeafQueue#testUpdateDemand are failing with NPE (yufeigu via rkanter)
|
2017-10-24 13:36:50 -07:00 |
Robert Kanter
|
025c656572
|
YARN-7382. NoSuchElementException in FairScheduler after failover causes RM crash (rkanter)
|
2017-10-24 10:21:44 -07:00 |
Daniel Templeton
|
9e77dc2bd1
|
YARN-7357. Several methods in TestZKRMStateStore.TestZKRMStateStoreTester.TestZKRMStateStoreInternal should have @Override annotations
(Contributed by Sen Zhao via Daniel Templeton)
|
2017-10-23 13:51:19 -07:00 |
Eric Payne
|
921338cd86
|
YARN-4163: Audit getQueueInfo and getApplications calls
|
2017-10-23 11:43:41 -05:00 |
Yufei Gu
|
1f4cdf1068
|
YARN-4090. Make Collections.sort() more efficient by caching resource usage. (Contributed by Yufei Gu, Shilong Zhang and Xianyin Xin)
|
2017-10-20 01:32:20 -07:00 |
Yufei Gu
|
7b4b018780
|
YARN-7359. TestAppManager.testQueueSubmitWithNoPermission() should be scheduler agnostic. (Contributed by Haibo Chen)
|
2017-10-19 16:51:47 -07:00 |
Yufei Gu
|
cbd2b73ef8
|
YARN-7294. TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently with Fair Scheduler. (Contributed by Miklos Szegedi)
|
2017-10-19 16:39:25 -07:00 |
Subru Krishnan
|
75323394fb
|
YARN-7311. Fix TestRMWebServicesReservation parametrization for fair scheduler. (Yufei Gu via Subru).
|
2017-10-17 12:38:06 -07:00 |
Robert Kanter
|
8a61525928
|
YARN-7308. TestApplicationACLs fails with FairScheduler (rkanter)
|
2017-10-16 15:34:32 -07:00 |
Rohith Sharma K S
|
f4fb6695a3
|
YARN-7180. Remove class ResourceType. Contributed by Sunil G.
|
2017-10-13 15:17:05 +05:30 |
Robert Kanter
|
ec8bf9e48a
|
YARN-7309. TestClientRMService#testUpdateApplicationPriorityRequest and TestClientRMService#testUpdatePriorityAndKillAppWithZeroClusterResource test functionality not supported by FairScheduler (rkanter)
|
2017-10-10 10:18:42 -07:00 |
Wangda Tan
|
5cfbad68c0
|
YARN-7237. Cleanup usages of ResourceProfiles. (Wangda Tan)
Change-Id: I959c8531bd48231956068cb7d2312f3235549f9c
(cherry picked from commit 8536db5a7549fa44d9f9e444b41bd5978b5c6ae1)
|
2017-10-09 20:36:57 -07:00 |
Jonathan Hung
|
1d36b53ab6
|
YARN-7252. Removing queue then failing over results in exception
|
2017-10-09 11:11:23 -07:00 |
Jonathan Hung
|
09c5dfe937
|
YARN-7251. Misc changes to YARN-5734
|
2017-10-09 11:11:23 -07:00 |
Jonathan Hung
|
d6622daaa3
|
YARN-7046. Add closing logic to configuration store
|
2017-10-09 11:11:23 -07:00 |
Wangda Tan
|
ff39c0de20
|
YARN-6840. Implement zookeeper based store for scheduler configuration updates. (Jonathan Hung via wangda)
Change-Id: I9debea674fe8c7e4109d4ca136965a1ea4c48bcc
|
2017-10-09 11:11:23 -07:00 |
Xuan
|
4d8abd84f4
|
YARN-7024: Fix issues on recovery in LevelDB store. Contributed by Jonathan Hung
|
2017-10-09 11:11:23 -07:00 |
Xuan
|
4c8b208adb
|
YARN-6322: Disable queue refresh when configuration mutation is enabled. Contributed by Jonathan Hung
|
2017-10-09 11:11:23 -07:00 |
Xuan
|
74ba6ffa0b
|
YARN-5947: Create LeveldbConfigurationStore class using Leveldb as backing store. Contributed by Jonathan Hung
|
2017-10-09 11:11:23 -07:00 |
Xuan
|
04f9e80bb2
|
YARN-5953 addendum: Move QueueConfigInfo and SchedConfUpdateInfo to package org.apache.hadoop.yarn.webapp.dao
|
2017-10-09 11:11:23 -07:00 |
Xuan
|
916bdbd6be
|
YARN-5953:Create CLI for changing YARN configurations. (Jonathan Hung via xgong)
|
2017-10-09 11:11:23 -07:00 |
Xuan
|
e566fd8b58
|
YARN-6575. Support global configuration mutation in MutableConfProvider. (Jonathan Hung via Xuan Gong)
|
2017-10-09 11:11:23 -07:00 |
Wangda Tan
|
a4e6253046
|
YARN-5949. Add pluggable configuration ACL policy interface and implementation. (Jonathan Hung via wangda)
Change-Id: Ib98e82ff753bede21fcab2e6ca9ec1e7a5a2008f
|
2017-10-09 11:11:23 -07:00 |
Wangda Tan
|
d8a5136b1f
|
YARN-5952. Create REST API for changing YARN scheduler configurations. (Jonathan Hung via wangda)
|
2017-10-09 11:11:23 -07:00 |
Jonathan Hung
|
ef686635c1
|
YARN-5948. Implement MutableConfigurationManager for handling storage into configuration store
|
2017-10-09 11:11:23 -07:00 |
Xuan
|
e3579a8c3b
|
YARN-5946: Create YarnConfigurationStore interface and
InMemoryConfigurationStore class. Contributed by Jonathan Hung
|
2017-10-09 11:11:23 -07:00 |
Jonathan Hung
|
ef59cbe08a
|
YARN-5951. Changes to allow CapacityScheduler to use configuration store
|
2017-10-09 11:11:22 -07:00 |
Daniel Templeton
|
6d6ca4c923
|
YARN-2612 addendum: fixed javadoc error.
|
2017-10-09 14:12:13 +02:00 |
Sunil G
|
8137254a1f
|
YARN-7245. Max AM Resource column in Active Users Info section of Capacity Scheduler UI page should be updated per-user. Contributed by Eric Payne.
|
2017-10-09 14:39:41 +05:30 |
Rohith Sharma K S
|
6fe6a72352
|
Revert "YARN-7180. Remove class ResourceType. Contributed by Sunil G."
This reverts commit 6bd81ab054 .
|
2017-10-09 13:21:13 +05:30 |
Rohith Sharma K S
|
6bd81ab054
|
YARN-7180. Remove class ResourceType. Contributed by Sunil G.
|
2017-10-09 10:20:40 +05:30 |
Yufei Gu
|
72d22b753a
|
YARN-7207. Cache the RM proxy server address. (Yufei Gu)
|
2017-10-06 09:49:11 -07:00 |
Yufei Gu
|
49ae538164
|
YARN-2162. Add ability in Fair Scheduler to optionally configure maxResources in terms of percentage(addendum). (Yufei Gu)
|
2017-10-06 00:08:25 -07:00 |
Yufei Gu
|
99292adcef
|
YARN-2162. Add ability in Fair Scheduler to optionally configure maxResources in terms of percentage. (Yufei Gu)
|
2017-10-05 23:51:58 -07:00 |
Arun Suresh
|
b733348dde
|
YARN-7258. Add Node and Rack Hints to Opportunistic Scheduler. (Kartheek Muthyala via asuresh).
|
2017-10-05 09:58:04 -07:00 |
Subru Krishnan
|
e6e614e380
|
YARN-5329. Placement Agent enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru).
|
2017-10-04 19:28:27 -07:00 |
Akira Ajisaka
|
2df1b2ac05
|
YARN-7044. TestContainerAllocation#testAMContainerAllocationWhenDNSUnavailable fails.
|
2017-10-05 06:07:32 +09:00 |
Akira Ajisaka
|
c9b525bd94
|
YARN-6943. Update Yarn to YARN in documentation. Contributed by Chetna Chaudhari.
|
2017-10-04 07:01:54 +09:00 |
Subru Krishnan
|
d4d2fd1acd
|
YARN-2037. Add work preserving restart support for Unmanaged AMs. (Botong Huang via Subru).
|
2017-10-02 18:14:44 -07:00 |
Eric Payne
|
7f6118f918
|
YARN-7084. TestSchedulingMonitor#testRMStarts fails sporadically. Contributed by Jason Lowe.
|
2017-09-29 10:33:57 -05:00 |
Jason Lowe
|
85d81ae58e
|
YARN-7248. NM returns new SCHEDULED container status to older clients. Contributed by Arun Suresh
|
2017-09-28 14:10:15 -05:00 |
Sunil G
|
8facf1f976
|
YARN-6871. Add additional deSelects params in RMWebServices#getAppReport. Contributed by Tanuj Nayak.
|
2017-09-27 14:37:32 +05:30 |
Rohith Sharma K S
|
06e5a7b5cf
|
YARN-65. Reduce RM app memory footprint once app has completed. Contributed by Manikandan R.
|
2017-09-26 14:34:09 +05:30 |
Daniel Templeton
|
164a063e74
|
YARN-6878. TestCapacityScheduler.testDefaultNodeLabelExpressionQueueConfig() has the args to assertEqual() in the wrong order
(Contributed by Sen Zhao via Daniel Templeton)
|
2017-09-22 23:53:32 +02:00 |
Yufei Gu
|
a92ef030a2
|
YARN-7045. Remove FSLeafQueue#addAppSchedulable. (Contributed by Sen Zhao via Yufei Gu)
|
2017-09-21 12:31:07 -07:00 |
Yufei Gu
|
a9019e1fb7
|
YARN-6499. Remove the doc about Schedulable#redistributeShare(). (Contributed by Chetna Chaudhari via Yufei Gu)
|
2017-09-19 18:28:31 -07:00 |
Arun Suresh
|
56ef5279c1
|
YARN-7203. Add container ExecutionType into ContainerReport. (Botong Huang via asuresh)
|
2017-09-18 15:49:31 -07:00 |
Wangda Tan
|
e81596d06d
|
YARN-7172. ResourceCalculator.fitsIn() should not take a cluster resource parameter. (Sen Zhao via wangda)
Change-Id: Icc3670c9381ce7591ca69ec12da5aa52d3612d34
|
2017-09-17 21:20:43 -07:00 |
Wangda Tan
|
38c14ef8d8
|
YARN-7149. Cross-queue preemption sometimes starves an underserved queue. (Eric Payne via wangda)
Change-Id: Ib269991dbebce160378e8372ee6d24849c4a5ed6
(cherry picked from commit 3dfa937a1fadfc62947755872515f549b3b15e6a)
|
2017-09-15 21:29:39 -07:00 |
Wangda Tan
|
958e8c0e25
|
YARN-6977. Node information is not provided for non am containers in RM logs. (Suma Shivaprasad via wangda)
Change-Id: I0c44d09a560446dee2ba68c2b9ae69fce0ec1d3e
(cherry picked from commit 8a42e922fad613f3cf1cc6cb0f3fa72546a9cc56)
|
2017-09-15 21:29:31 -07:00 |
Yufei Gu
|
09b476e6da
|
YARN-6612. Update fair scheduler policies to be aware of resource types. (Contributed by Daniel Templeton via Yufei Gu)
|
2017-09-14 11:23:37 -07:00 |
Robert Kanter
|
bb34ae9554
|
YARN-7146. Many RM unit tests failing with FairScheduler (rkanter)
|
2017-09-13 15:44:54 -07:00 |
Rohith Sharma K S
|
5324388cf2
|
YARN-7157. Add admin configuration to filter per-user's apps in secure cluster. Contributed by Sunil G.
|
2017-09-13 23:36:47 +05:30 |
Yufei Gu
|
82c5dd1d50
|
YARN-7057. FSAppAttempt#getResourceUsage doesn't need to consider resources queued for preemption. (Contributed by Karthik via Yufei Gu)
|
2017-09-12 16:13:39 -07:00 |