04cd3115ba
YARN-10658. CapacityScheduler QueueInfo add queue path field to avoid ambiguous QueueName. Contributed by Qi Zhu.
2021-03-08 12:29:44 +01:00
c6b30a59dd
YARN-10672. All testcases in TestReservations are flaky. Contributed By Szilard Nemeth.
2021-03-05 23:13:35 +01:00
9d088639bf
YARN-10664. Allow parameter expansion in NM_ADMIN_USER_ENV. Contributed by Jim
...
Brennan.
2021-03-05 19:46:40 +00:00
e82e7c597a
YARN-10640. Adjust the queue Configured capacity to Configured weight number for weight mode in UI. Contributed by Qi Zhu.
2021-03-05 15:56:56 +01:00
e7cad3811f
YARN-10642. Race condition: AsyncDispatcher can get stuck by the changes introduced in YARN-8995. Contributed by zhengchenyu.
2021-03-05 13:50:45 +01:00
e19c00925f
YARN-10639. Queueinfo related capacity, should adjusted to weight mode. Contributed by Qi Zhu.
2021-03-05 13:18:06 +01:00
6699198b54
YARN-10532. Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used. Contributed by Qi Zhu.
2021-03-04 17:18:35 +01:00
a85aeee876
YARN-10623. Capacity scheduler should support refresh queue automatically by a thread policy. Contributed by Qi Zhu.
2021-03-04 12:23:11 +01:00
d615e2d3bd
YARN-10649. Fix RMNodeImpl.updateExistContainers leak ( #2719 ). Contributed by Max Xie
2021-03-04 14:52:58 +05:30
7e8040e6ad
YARN-10655. Limit queue creation depth relative to its first static parent. Contributed by Andras Gyori.
2021-03-03 17:44:30 +01:00
8af56de1fa
HADOOP-17560. Fix some spelling errors ( #2730 )
...
Co-authored-by: jiaguodong5 <jiaguodong5@jd.com>
2021-03-03 11:41:05 +09:00
3101c39b49
YARN-10656. Parsing error in CapacityScheduler.md ( #2725 )
...
Reviewed-by: Peter Bacsko <pbacsko@apache.org>
2021-02-27 12:46:36 +09:00
947b50489d
YARN-10627. Extend logging to give more information about weight mode. Contributed by Benjamin Teke.
2021-02-26 21:57:54 +01:00
e04bcb3a06
MAPREDUCE-7320. organize test directories for ClusterMapReduceTestCase ( #2722 ). Contributed by Ahmed Hussein
2021-02-26 13:42:33 -06:00
fcdb1e7fec
YARN-10653. Fixed the findbugs issues introduced by YARN-10647. Contributed by
...
Qi Zhu
2021-02-26 19:03:59 +00:00
ac37ca0b67
YARN-10651. CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource(). Contributed by Haibo Chen
2021-02-25 16:01:18 -08:00
c373da9f88
[YARN-10613] Config to allow Intra- and Inter-queue preemption to enable/disable conservativeDRF. Contributed by Eric Payne
2021-02-25 16:46:19 +00:00
a89c624002
YARN-10609. Update the document for YARN-10531(Be able to disable user limit factor for CapacityScheduler Leaf Queue). Contributed by Qi Zhu.
2021-02-24 20:41:35 +01:00
47420ae3ed
YARN-10647. Fix TestRMNodeLabelsManager failed after YARN-10501. Contributed by
...
Qi Zhu.
2021-02-22 18:48:14 +00:00
b722c52f4e
YARN-10513. CS Flexible Auto Queue Creation RM UIv2 modifications. Contributed by Andras Gyori.
2021-02-22 12:55:45 +01:00
4891e68c2b
YARN-10501. Can't remove all node labels after add node label without
...
nodemanager port. Contributed by caozhiqiang.
2021-02-19 22:29:10 +00:00
d28b6f90c8
YARN-10636. CS Auto Queue creation should reject submissions with empty path parts. Contributed by Gergely Pollak.
2021-02-19 13:57:30 +01:00
4383726d19
YARN-10635. CSMapping rule can return paths with empty parts. Contributed by Gergely Pollak.
2021-02-19 12:01:31 +01:00
9a298d180d
Revert "HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin ( #2454 )"
...
This reverts commit 4cf3531583
2021-02-19 11:09:10 +09:00
4cf3531583
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin ( #2454 )
...
Use spotbugs instead of findbugs. Removed findbugs from the hadoop build images,
and added spotbugs in the images instead.
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Dinesh Chitlangia <dineshc@apache.org>
2021-02-17 10:38:20 +09:00
9882a22365
YARN-10625. FairScheduler: add global flag to disable AM-preemption. Contributed by Peter Bacsko
2021-02-16 18:06:57 +01:00
3fcc7a55b6
[YARN-10626] Log resource allocation in NM log at container start time. Contributed by Eric Badger
2021-02-16 16:45:35 +00:00
bad6038a48
YARN-10622. Fix preemption policy to exclude childless ParentQueues. Contributed by Andras Gyori
2021-02-15 14:48:42 +01:00
852aac34f2
Support max queues limit configuration in new auto created queue, consistent with old auto created. Contributed by Qi Zhu
2021-02-15 14:20:50 +01:00
f9a073c6c1
YARN-10500. TestDelegationTokenRenewer fails intermittently. ( #2619 )
2021-02-11 13:55:04 -06:00
98ca6afd17
YARN-10618. RM UI2 Application page shows the AM preempted containers instead of the nonAM ones. Contributed by Benjamin Teke
2021-02-11 09:14:37 +01:00
cacc870389
YARN-10593. Fix incorrect string comparison in GpuDiscoverer. Contributed by Peter Bacsko
2021-02-10 13:09:32 +01:00
a8bd516e39
YARN-10620. fs2cs: parentQueue for certain placement rules are not set during conversion. Contributed by Peter Bacsko
2021-02-10 11:15:53 +01:00
fc5f241535
YARN-10619. CS Mapping Rule %specified rule catches default submissions. Contributed by Gergely Pollak
2021-02-09 18:16:42 +01:00
0b05dd8802
YARN-9650. Set thread names for CapacityScheduler AsyncScheduleThread ( #2665 )
...
Amogh Rajesh Desai via Zhankun Tang
2021-02-09 11:05:14 +08:00
c22c77af43
[YARN-10607] User environment is unable to prepend PATH when mapreduce.admin.user.env also sets PATH. Contributed by Eric Badger.
2021-02-05 17:33:01 +00:00
5aa9866ec2
YARN-10615. Fix Auto Queue Creation hierarchy construction to use queue path instead of short queue name. Contributed by Andras Gyori
2021-02-05 17:43:01 +01:00
c19326c051
YARN-10610. Add queuePath to RESTful API for CapacityScheduler consistent with FairScheduler queuePath. Contributed by Qi Zhu
2021-02-05 17:34:45 +01:00
79a46599f7
YARN-10428. Zombie applications in the YARN queue using FAIR + sizebasedweight. Contributed by Guang Yang, Andras Gyori
2021-02-05 17:09:08 +01:00
52217fce3d
HADOOP-17432. [JDK 16] KerberosUtil#getOidInstance is broken by JEP 396 ( #2546 )
...
Reviewed-by: Steve Loughran <stevel@apache.org>
2021-02-05 16:14:10 +09:00
5e719bf588
YARN-10611. Fix that shaded should be used for google guava imports in YARN-10352. Contributed by Qi Zhu
2021-02-04 18:30:39 +00:00
c17414111b
YARN-10612. Fix findbugs issue introduced in YARN-10585. Contributed by Gergely Pollak
2021-02-04 08:12:13 +01:00
6fc26ad539
YARN-10352 Skip schedule on not heartbeated nodes in Multi Node Placement. Contributed by Prabhu Joseph and Qi Zhu
2021-02-03 08:50:45 +05:30
9bf2ac07bb
YARN-10605. Add queue-mappings-override.enable property in FS2CS conversions. Contributed by Andras Gyori
2021-02-02 09:13:46 +01:00
7d88953768
YARN-10604. Support auto queue creation without mapping rules. Contributed by Andras Gyori
2021-02-02 09:07:25 +01:00
21a3fc3d2d
HADOOP-17482: Remove Commons Logger from FileSystem Class ( #2633 )
2021-02-01 09:40:01 -08:00
fa15594ae6
YARN-10600. Convert root queue in fs2cs weight mode conversion. Contributed by Benjamin Teke.
2021-01-28 20:58:51 +01:00
9628aa87bf
YARN-10598. CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to extend the creation type with additional information. Contributed by Benjamin Teke
2021-01-27 18:14:06 +01:00
7c4ef42837
YARN-10599. fs2cs should generate new 'auto-queue-creation-v2.enabled' properties for all parents. Contributed by Peter Bacsko
2021-01-27 15:49:26 +01:00
e2a7008d50
YARN-10585. Create a class which can convert from legacy mapping rule format to the new JSON format. Contributed by Gergely Pollak
2021-01-26 18:31:39 +01:00
4f008153ef
YARN-10587. Fix AutoCreateLeafQueueCreation cap related caculation when in absolute mode. (Qi Zhu via wangda)
...
Change-Id: I8889c393a9f08b633c72a2e873c6f45ca12cc82b
2021-01-26 08:24:29 -08:00
b7384a8d02
YARN-10531. Be able to disable user limit factor for CapacityScheduler Leaf Queue. (Qi Zhu via wangda)
...
Change-Id: I670e5525619b320745254609c48e7e1afb084835
2021-01-26 08:24:28 -08:00
f1766e5bb4
YARN-10596. Allow static definition of childless ParentQueues with auto-queue-creation-v2 enabled. Contributed by Andras Gyori
2021-01-26 16:22:35 +01:00
8620984b8d
YARN-10515. Fix flaky test TestCapacitySchedulerAutoQueueCreation.testDynamicAutoQueueCreationWithTags. Contributed by Peter Bacsko
2021-01-21 20:58:57 +01:00
1ccba3734d
YARN-10490. yarn top command not quitting completely with ctrl+c. Contributed by Agshin Kazimli
2021-01-21 20:30:19 +01:00
17fdac8de5
YARN-10581. CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to include queue creation type for queues. Contributed by Szilard Nemeth.
2021-01-21 18:06:53 +01:00
06fef5ee43
YARN-10579. CS Flexible AQC: Modify RM /scheduler endpoint to include weight values for queues. Contributed by Szilard Nemeth
2021-01-21 09:23:11 +01:00
8bc2dfbf36
YARN-10519. Refactor QueueMetricsForCustomResources class to move to yarn-common package. Contributed by Minni Mittal
2021-01-21 11:23:30 +05:30
cfe6e1f7da
YARN-10578. Fix Auto Queue Creation parent handling. Contributed by Andras Gyori
2021-01-20 15:22:44 +01:00
4b5bc05a78
YARN-10512. CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to include mode of operation for CS. Contributed by Szilard Nemeth.
2021-01-20 12:28:59 +05:30
a326f22606
YARN-10573. Enhance placement rule conversion in fs2cs in weight mode and enable it by default. Contributed by Peter Bacsko
2021-01-19 09:42:40 +01:00
6abdb148e4
YARN-10535. Make queue placement in CapacityScheduler compliant with auto-queue-placement. Contributed by Gergely Pollak
2021-01-18 20:19:36 +01:00
acea862d59
YARN-10574. Fix the FindBugs warning introduced in YARN-10506. Contributed by Gergely Pollak
2021-01-18 16:04:45 +01:00
3d46141583
YARN-10506. Update queue creation logic to use weight mode and allow the flexible static/dynamic creation. (Contributed by Qi Zhu, Andras Gyori)
...
Change-Id: I118862fd5e11ee6888275e2bcf667fedfa56c5d7
2021-01-15 14:20:08 -08:00
768e2f42ba
YARN-10562. Follow up changes for YARN-9833. Contributed by Jim Brennan.
2021-01-13 23:41:52 +00:00
d8fadec944
YARN-4589: Diagnostics for localization timeouts is lacking. Contributed by Chang Li (lichangleo) and Jim Brennan (Jim_Brennan)
2021-01-13 17:01:28 +00:00
836c630430
YARN-10525. Add weight mode conversion to fs2cs. Contributed by Peter Bacsko
2021-01-13 14:24:12 +01:00
61f77b7674
YARN-8529. Add timeout to RouterWebServiceUtil#invokeRMWebService. Contributed by Minni Mittal
2021-01-13 16:00:11 +05:30
85b1c017ee
YARN-10563. Fix dependency exclusion problem in poms. Contributed by Peter Bacsko
2021-01-12 15:29:20 +01:00
9c6e093783
YARN-10570. Remove experimental warning message from fs2cs. Contributed by Peter Bacsko
2021-01-12 15:21:27 +01:00
b0eec09097
YARN-10504. Implement weight mode in Capacity Scheduler. (Contributed by Wangda Tan, Benjamin Teke, zhuqi, Andras Gyori)
...
Change-Id: Ic49c730b0ab502ba86527fb662d25c4c8b1c2588
2021-01-11 17:46:09 -08:00
ec22850dbe
Make upstream aware of 3.2.2 release.
2021-01-09 18:06:08 +08:00
890f2da624
YARN-10553. Refactor TestDistributedShell ( #2581 )
2021-01-09 08:39:21 +09:00
4176759909
YARN-10507. Add the capability to fs2cs to write the converted placement rules inside capacity-scheduler.xml. Contributed by Peter Bacsko
2021-01-08 13:40:33 +01:00
580a6a75a3
YARN-10528. maxAMShare should only be accepted for leaf queues, not parent queues. Contributed by Siddharth Ahuja
2021-01-08 12:38:43 +01:00
1b1791075a
YARN-10538: Add RECOMMISSIONING nodes to the list of updated nodes returned to the AM ( #2564 )
...
Contributed by Srinivas S T
2021-01-08 10:52:52 +05:30
ae4945fb2c
YARN-10560. Upgrade node.js to 10.23.1 and yarn to 1.22.5 in Web UI v2. ( #2592 )
2021-01-06 12:12:46 +09:00
5abeecd33f
YARN-10558. Fix failure of TestDistributedShell#testDSShellWithOpportunisticContainers. ( #2586 )
2021-01-05 13:38:34 +09:00
fa4cf91b57
YARN-10541. capture the performance metrics of ZKRMStateStore ( #2568 )
2020-12-30 22:20:12 -08:00
513f1995ad
YARN-10334. Close clients in TestDistributedShell ( #2571 )
2020-12-28 11:09:10 -08:00
4c5d88e230
YARN-10540. Node page is broken in YARN UI1 and UI2 including RMWebService api
...
for nodes. Contributed by Jim Brennan.
2020-12-21 23:10:31 +00:00
bb528e3239
YARN-10463: For Federation, we should support getApplicationAttemptReport. ( #2563 )
...
Qi Zhu via Zhankun Tang
2020-12-21 10:04:16 +08:00
7a88f45366
YARN-10536. Client in distributedShell swallows interrupt exceptions ( #2554 )
2020-12-17 15:13:28 -08:00
df7f1e5199
YARN-10499. TestRouterWebServiceREST fails ( #2490 ). Contributed by Akira Ajisaka
2020-12-16 17:26:15 +01:00
df35c7f519
YARN-10526. RMAppManager CS Placement ignores parent path. Contributed by Gergely Pollak
2020-12-15 17:30:14 +01:00
3234e5eaf3
YARN-10031. Create a general purpose log request with additional query parameters. Contributed by Andras Gyori
2020-12-12 09:42:22 +01:00
0a45bd034e
YARN-10520. Deprecated the residual nested class for the LCEResourceHandler ( #2527 ). Contributed by Wanqiang Ji
2020-12-09 12:55:24 +01:00
d67ccd03e3
YARN-10380: Import logic of multi-node allocation in CapacityScheduler ( #2494 )
...
Contributed by Qi Zhu.
2020-12-09 19:48:39 +08:00
7d3c8ef606
YARN-10495. make the rpath of container-executor configurable. Contributed by
...
angerszhu.
2020-12-08 00:53:24 +00:00
9170eb566b
YARN-10511. Update yarn.nodemanager.env-whitelist value in docs ( #2512 )
...
Reviewed-by: Adam Antal <adamantal@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
2020-12-04 00:14:04 +09:00
9969745343
YARN-9883. Reshape SchedulerHealth class. Contributed by D M Murali Krishna Reddy
2020-12-03 09:55:06 +01:00
fa773a8326
YARN-10278: CapacityScheduler test framework ProportionalCapacityPreemptionPolicyMockFramework. Contributed by Szilard Nemeth (snemeth)
2020-12-01 22:06:47 +00:00
4d2ae5b398
YARN-10498. Fix typo in CapacityScheduler Markdown document ( #2484 )
...
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
2020-11-30 11:17:10 +09:00
2ce2198287
HADOOP-17394. [JDK 11] Fix error in mvn package -Pdocs ( #2488 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
2020-11-26 11:32:10 +09:00
65002c9486
Revert "[JDK 11] Fix error in mvn package -Pdocs ( #2488 )" because JIRA ID is missing.
...
This reverts commit ce5b3d7975
2020-11-26 11:30:53 +09:00
ce5b3d7975
[JDK 11] Fix error in mvn package -Pdocs ( #2488 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
2020-11-26 11:27:30 +09:00
08b2e285db
YARN-10488. Several typos in package: org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair ( #2485 )
...
Co-authored-by: akumar <akumar@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
2020-11-25 13:00:09 +09:00
569b20e31c
YARN-10468. Fix TestNodeStatusUpdater timeouts and broken conditions ( #2461 )
2020-11-24 11:09:30 -08:00
c4ba0ab7df
YARN-10470. When building new web ui with root user, the bower install should support it. Contributed by zhuqi.
2020-11-24 15:22:07 +09:00
0d3155a687
YARN-10457. Add a configuration switch to change between legacy and JSON placement rule format. Contributed by Gergely Pollak
2020-11-18 17:02:47 +01:00
5ff70a59c4
YARN-10486. FS-CS converter: handle case when weight=0 and allow more lenient capacity checks in Capacity Scheduler. Contributed by Peter Bacsko
2020-11-18 16:51:42 +01:00
0b2510ee1f
YARN-10485. TimelineConnector swallows InterruptedException ( #2450 ). Contributed by Ahmed Hussein
2020-11-16 14:55:40 -06:00
567600fd80
YARN-10425. Replace the legacy placement engine in CS with the new one. Contributed by Gergely Pollak.
2020-11-11 17:10:15 +01:00
cd0490e8c6
Revert "YARN-10425. Replace the legacy placement engine in CS with the new one. Contributed by Gergely Pollak."
...
This reverts commit b0ab222a6c
2020-11-11 17:06:12 +01:00
b0ab222a6c
YARN-10425. Replace the legacy placement engine in CS with the new one. Contributed by Gergely Pollak.
2020-11-11 17:03:16 +01:00
61f8c5767e
YARN-10480. replace href tags with ng-href ( #2426 )
2020-11-10 10:54:28 -08:00
4331c88352
HADOOP-17344. Harmonize guava version and shade guava in yarn-csi. ( #2434 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-11-10 15:19:40 +09:00
55339c2bdd
YARN-10479. RMProxy should retry on SocketTimeout Exceptions. Contributed by Jim Brennan (Jim_Brennan)
2020-11-05 21:50:46 +00:00
0461a07c01
YARN-10475: Scale RM-NM heartbeat interval based on node utilization. Contributed by Jim Brennan (Jim_Brennan).
2020-11-02 16:55:06 +00:00
deea5d8f2b
YARN-10469. YARN-UI2 The accuracy of the percentage values in the same chart on the YARN 'Cluster OverView' page are inconsistent ( #2401 )
2020-11-02 17:52:58 +08:00
4890b855ec
YARN-10420. Update CS MappingRule documentation with the new format and features. Contributed by Peter Bacsko
2020-10-31 15:17:01 +01:00
7169ec4509
YARN-10442. RM should make sure node label file highly available. ( #2390 )
...
* YARN-10442. RM should make sure node label file highly available.
* YARN-10442. Corrected method name.
Added license header for file TestNodeLabelFileReplication.java
Added Test code.
* YARN-10442. Changed property name and log.
* YARN-10442. Changed default replication for FS store file.
2020-10-30 23:05:16 +05:30
c47c9fd65d
YARN-10458. Hive On Tez queries fails upon submission to dynamically created pools. (Peter Bacsko via wangda)
...
Change-Id: I518dc925187ce55e9d35a37ba20878c0f4e37e5c
2020-10-30 09:49:48 -07:00
8ee6bc2518
YARN-10471. Prevent logs for any container from becoming larger than a configurable size. Contributed by Eric Payne
2020-10-29 19:49:18 +00:00
bab5bf9743
YARN-10467. ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers. Contributed by Haibo Chen
2020-10-28 10:33:18 -07:00
7435604a91
YARN-10453. Add partition resource info to get-node-labels and label-mappings api responses. Contributed Akhil PB.
2020-10-22 12:52:58 +05:30
31477a224f
YARN-10460. Upgrading to JUnit 4.13 causes tests in TestNodeStatusUpdater to fail. Contributed by Peter Bacsko
2020-10-19 13:17:51 +02:00
1e3a6efcef
HADOOP-17288. Use shaded guava from thirdparty. ( #2342 ). Contributed by Ayush Saxena.
2020-10-17 12:01:18 +05:30
8b8c672780
YARN-10450. Add cpu and memory utilization per node and cluster-wide metrics.
...
Contributed by Jim Brennan.
2020-10-15 17:57:43 +00:00
42358169f6
YARN-10454: Add applicationName policy. Contributed by Peter Bacsko
2020-10-13 10:11:10 +02:00
deb35a32ba
YARN-10455. TestNMProxy.testNMProxyRPCRetry is not consistent. Contributed by Ahmed Hussein
2020-10-08 18:46:47 +00:00
b361f29dda
YARN-10451. RM (v1) UI NodesPage can NPE when yarn.io/gpu resource type is defined. Contributed by Eric Payne
2020-10-06 18:18:08 +00:00
a1f7e760df
YARN-10393. MR job live lock caused by completed state container leak in heartbeat between node manager and RM. Contributed by zhenzhao wang and Jim Brennan
2020-10-05 10:09:14 +02:00
d0d10f7e8f
HADOOP-17276. Extend CallerContext to make it include many items ( #2327 )
2020-10-04 02:02:28 +09:00
bb8446e80c
YARN-10447. TestLeafQueue: ActivitiesManager thread might interfere with ongoing stubbing. Contributed by Peter Bacsko
2020-10-01 11:42:04 +02:00
a7a1f1541a
YARN-10413. Change fs2cs to generate mapping rules in the new format. Contributed by Peter Bacsko
2020-09-29 20:18:26 +02:00
bbbfa7d415
HADOOP-17286. Updated jQuery in yarn-common. ( #2340 )
2020-09-27 03:18:01 +09:00
e3cd627069
YARN-10438. Handle null containerId in ClientRMService#getContainerReport() ( #2313 )
...
Co-authored-by: Shubham Gupta <gshubham@microsoft.com>
2020-09-25 12:20:39 +05:30
486ddb73f9
HADOOP-17283. Hadoop - Upgrade to jQuery 3.5.1 ( #2330 )
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2020-09-24 21:15:55 +09:00
7ec16ac7f6
YARN-10443. Document options of logs CLI. Contributed by Ankit Kumar.
...
Fixes #2325
2020-09-23 17:42:56 +02:00
18f9d77c46
YARN-6754. Fair scheduler docs should explain meaning of weight=0 for a queue. ( #2300 )
2020-09-23 14:50:28 +09:00
474fa80bfb
HADOOP-17277. Correct spelling errors for separator ( #2322 )
...
Contributed by Hui Fei.
2020-09-22 17:10:33 +01:00
eacbe07b56
YARN-9333. TestFairSchedulerPreemption.testRelaxLocalityPreemptionWithNoLessAMInRemainingNodes fails intermittently. Contributed by Peter Bacsko
2020-09-17 19:57:19 +02:00
90894ea641
YARN-10430. Log improvements in NodeStatusUpdaterImpl. Contributed by Bilwa S T.
2020-09-14 21:19:14 +00:00
9afec2ed17
YARN-10390: LeafQueue: retain user limits cache across assignContainers() calls. Contributed by Samir Khan (samkhan).
2020-09-11 13:29:26 +00:00
773ac799c6
YARN-10387. Implement logic which returns MappingRule objects based on mapping rules. Contributed by Peter Bacsko
2020-09-09 15:12:58 +02:00
43572fc7f8
YARN-10397. SchedulerRequest should be forwarded to scheduler if custom scheduler supports placement constraints. Contributed by Bilwa S T.
2020-09-09 17:08:13 +05:30
aba4a506d6
YARN-10424. Adapt existing AppName and UserGroupMapping unittests to ensure backwards compatibility. Contributed by Benjamin Teke.
2020-09-09 09:41:11 +02:00
c4fb4044b2
YARN-10415. Create a group matcher which checks ALL groups of the user. Contributed by Gergely Pollak.
2020-09-08 10:57:00 +02:00
34fe74da0e
YARN-10332. RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state. Contributed by yehuanhuan
2020-09-07 11:44:09 +02:00
360bbcd3bc
YARN-10411. Create an allowCreate flag for MappingRuleAction. Contributed by Gergely Pollak.
2020-09-07 11:39:03 +02:00
dd6c66556e
YARN-9136. getNMResourceInfo NodeManager REST API method is not documented
2020-09-07 10:29:20 +02:00
1841a5bb03
YARN-10376. Create a class that covers the functionality of UserGroupMappingPlacementRule and AppNameMappingPlacementRule using the new mapping rules. Contributed by Gergely Pollak.
2020-09-05 10:00:33 +02:00
696e4fe50e
Typos in YarnScheduler#allocate method's doc comment. Contributed by Siddharth Ahuja
2020-09-04 09:59:54 +02:00
5c15815773
YARN-10375. CS Mapping rule config parser should return MappingRule objects. Contributed by Gergely Pollak.
2020-09-03 18:54:22 +02:00
288dab7563
YARN-10372. Create MappingRule class to represent each CS mapping rule. Contributed by Gergely Pollak
2020-09-03 15:02:48 +02:00
5e12dc51f0
YARN-10419. Javadoc error in hadoop-yarn-server-common module. ( #2271 )
2020-09-03 16:17:52 +09:00
c7dab2b23e
YARN-10374. Create Actions for CS mapping rules. Contributed by Gergely Pollak.
2020-09-01 17:38:49 +02:00
73a0d149e2
YARN-10386. Addendum patch: Fixing ASF Warnings. Contributed by Peter Bacsko.
2020-09-01 14:39:24 +02:00
4ffe26f9b8
[UI1] Provide a way to hide Tools section in Web UIv1. Contributed by Andras Gyori
2020-08-31 16:00:39 +02:00
56a5c360a1
YARN-10373. Create Matchers for CS mapping rules. Contributed by Gergely Pollak
2020-08-29 21:34:55 +02:00
f4f872b778
YARN-10371. Create variable context class for CS queue mapping rules. Contributed by Gergely Pollak
2020-08-29 17:31:48 +02:00
a888d580d8
YARN-10409. Improve MockQueueHierarchyBuilder to detect queue ambiguity. Contributed by Gergely Pollak
2020-08-29 15:24:50 +02:00
44542863f4
YARN-10408. Extract MockQueueHierarchyBuilder to a separate class. Contributed by Gergely Pollak
2020-08-28 17:35:45 +02:00
6e3d7050a9
YARN-10386. Create new JSON schema for Placement Rules. Contributed by Peter Bacsko and Benjamin Teke
2020-08-28 17:09:31 +02:00
75db5526b5
YARN-1806. Add ThreadDump Option in YARN UI2 to fetch for running containers
...
Contributed by Siddharth Ahuja. Reviewed by Akhil PB.
2020-08-26 13:08:17 +05:30
82a7505646
YARN-10304. Create an endpoint for remote application log directory path query. Contributed by Andras Gyori
2020-08-25 13:29:12 +02:00
c049296235
YARN-10106. Yarn logs CLI filtering by application attempt. Contributed by Hudáky Márton Gyula
2020-08-25 09:53:39 +02:00
5c927eb550
YARN-10360. Support Multi Node Placement in SingleConstraintAppPlacementAllocator
...
Reviewed by Sunil G.
2020-08-24 15:00:02 +05:30
9b9f7ea16a
YARN-10399 Refactor NodeQueueLoadMonitor class to make it extendable ( #2228 )
...
Refactor NodeQueueLoadMonitor class to make it extendable
2020-08-19 10:14:50 -07:00
82ec28f442
YARN-10396. Max applications calculation per queue disregards queue level settings in absolute mode. Contributed by Benjamin Teke.
2020-08-19 11:54:48 +05:30
4a7deae478
YARN-10391. --module-gpu functionality is broken in container-executor. Contributed by Eric Badger
2020-08-17 16:41:15 +00:00
86bbd38c8d
YARN-10336. RM page should throw exception when command injected in RM REST API to get applications. Contributed by Bilwa S T.
2020-08-13 23:44:42 +05:30
6c2ce3d56b
YARN-10389. Option to override RMWebServices with custom WebService class
...
Contributed by Tanu Ajmera. Reviewed by Bilwa ST and Sunil G.
2020-08-11 15:46:47 +05:30
5e0f879779
YARN-10364. Fix logic of queue capacity is absolute resource or percentage.
...
Contributed by Bilwa ST. Reviewed by Sunil G.
2020-08-08 22:33:58 +05:30
64753addba
YARN-10251. Show extended resources on legacy RM UI. Contributed by Eric Payne
2020-08-07 17:43:01 -07:00
81da221c75
YARN-10388. RMNode updatedCapability flag not set while RecommissionNodeTransition. Contributed by Pranjal Protim Borah
2020-08-07 08:36:52 +05:30
c7e71a6c0b
YARN-10361. Make custom DAO classes configurable into RMWebApp#JAXBContextResolver.
...
Contributed by Bilwa ST.
2020-08-06 13:19:52 +05:30
5edd8b925e
YARN-4575. ApplicationResourceUsageReport should return ALL reserved resource.
...
Contributed by Bibin Chundatt and Eric Payne.
2020-08-05 18:47:49 +00:00
0277856738
YARN-10377. Fix filter index to show apps while clicking on queue in RM UI
...
Contributed by Tarun Parimi.
2020-08-04 23:10:45 +05:30
aa5afa72c5
YARN-10381. Add application attempt state in AppAttempts RM REST API
...
Contributed by Siddharth Ahuja. Reviewed by Bilwa ST.
2020-08-04 23:07:28 +05:30
ab2b3df2de
YARN-10383. YarnCommands.md is inconsistent with the source code ( #2177 )
2020-08-04 12:02:16 +09:00
eac558380f
YARN-10229. [Federation] Client should be able to submit application to RM directly using normal client conf. Contributed by Bilwa S T.
2020-08-03 12:54:36 +05:30
5323e83edf
YARN-10359. Log container report only if list is not empty. Contributed by Bilwa S T.
2020-08-01 13:03:46 +05:30
e0c9653166
YARN-1529: Add Localization overhead metrics to NM. Contributed by Jim_Brennan.
2020-07-30 15:30:22 +00:00
5d8600e80a
YARN-10369. Make NMTokenSecretManagerInRM sending NMToken for nodeId DEBUG. Contributed by Jim Brennan.
2020-07-29 23:26:08 +05:30
3eaf62726f
YARN-10343. Legacy RM UI should include labeled metrics for allocated, total, and reserved resources. Contributed by Eric Payne
2020-07-28 13:43:19 -07:00
5dadf963d3
YARN-10208. Add capacityScheduler metric for NODE_UPDATE interval. Contributed by Pranjal Protim Borah.
2020-07-28 11:55:47 +05:30
60a254621a
YARN-10366. Fix Yarn rmadmin help message shows two labels for one node for --replaceLabelsOnNode.
...
Contributed by Tanu Ajmera.
2020-07-27 12:23:21 +05:30
d02be17a26
YARN-10362. Javadoc for TimelineReaderAuthenticationFilterInitializer is broken. Contributed by Xieming Li.
2020-07-27 01:55:04 +09:00
ac5f21dbef
YARN-4771. Some containers can be skipped during log aggregation after NM
...
restart. Contributed by Jason Lowe and Jim Brennan.
2020-07-24 22:35:16 +00:00
247eb0979b
YARN-10319. Record Last N Scheduler Activities from ActivitiesManager
...
Reviewed by Tao Yang and Adam Antal.
2020-07-24 11:43:31 +05:30
bfcd775381
YARN-10315. Avoid sending RMNodeResourceupdate event if resource is same. Contributed by Sushil Ks.
2020-07-23 16:41:35 +05:30
1b29c9bfee
HADOOP-17138. Fix spotbugs warnings surfaced after upgrade to 4.0.6. ( #2155 )
2020-07-22 13:40:20 +09:00
736bed6d6d
[YARN-10353] Log vcores used and cumulative cpu in containers monitor.
...
Contributed by Jim Brennan
2020-07-20 18:08:27 +00:00
6bcb24d269
HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. Contributed by Ahmed Hussein.
2020-07-18 14:33:43 +05:30
cc71d50b21
YARN-10339. Fix TimelineClient in NodeManager failing when Simple Http Auth used in Secure Cluster
...
Contributed by Tarun Parimi.
2020-07-16 23:36:49 +05:30
1f71c4ae71
HADOOP-17099. Replace Guava Predicate with Java8+ Predicate
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2020-07-15 11:39:06 -05:00
98fcffe93f
HADOOP-17101. Replace Guava Function with Java8+ Function
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2020-07-15 09:53:18 -05:00
8980c23fc2
YARN-10350. TestUserGroupMappingPlacementRule fails
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2020-07-15 14:31:34 +09:00
48f90115b5
YARN-10348. Allow RM to always cancel tokens after app completes. Contributed by
...
Jim Brennan
2020-07-13 23:10:39 +00:00
0427100b75
YARN-10297. TestContinuousScheduling#testFairSchedulerContinuousSchedulingInitTime fails intermittently. Contributed by Jim Brennan (Jim_Brennan)
2020-07-13 18:57:50 +00:00
f91a8ad88b
HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet. ( #2085 )
2020-07-09 11:33:37 -07:00
5dd270e208
YARN-10333. YarnClient obtain Delegation Token for Log Aggregation Path. Contributed by Prabhu Joseph.
2020-07-09 12:50:25 +05:30
dfe60392c9
YARN-10341. Yarn Service Container Completed event doesn't get processed. Contributed by Bilwa S T.
2020-07-09 12:34:52 +05:30
10d218934c
YARN-10344. Sync netty versions in hadoop-yarn-csi. ( #2126 )
2020-07-09 10:29:47 +05:30
3a4d05b850
YARN-8047. RMWebApp make external class pluggable.
...
Contributed by Bilwa S T.
2020-07-08 12:43:20 +05:30
2bbd00dff4
YARN-10337. Fix failing testcase TestRMHATimelineCollectors.
...
Contributed by Bilwa S T.
2020-07-07 17:10:59 +05:30
639acb6d89
HADOOP-17111. Replace Guava Optional with Java8+ Optional. Contributed by Ahmed Hussein.
2020-07-06 16:08:36 +09:00
04abd0eb17
YARN-10330. Add missing test scenarios to TestUserGroupMappingPlacementRule and TestAppNameMappingPlacementRule. Contributed by Peter Bacsko
2020-07-01 14:10:55 +02:00
9b5557a9e8
YARN-10325. Document max-parallel-apps for Capacity Scheduler. Contributed by Peter Bacsko
2020-07-01 13:41:30 +02:00
e8dc862d38
YARN-9809. Added node manager health status to resource manager registration call.
...
Contributed by Eric Badger via eyang
2020-06-30 11:39:16 -07:00
cd188ea9f0
YARN-10331. Upgrade node.js to 10.21.0. ( #2106 )
2020-06-30 16:52:57 +09:00
74fc13cf91
YARN-9903: Support reservations continue looking for Node Labels. Contributed by Jim Brennan (Jim_Brennan).
2020-06-29 18:39:53 +00:00
0be26811f3
YARN-10328. Fixed ZK Curator NodeExists exception in YARN service AM logs
...
Contributed by Bilwa S T via eyang
2020-06-29 09:21:24 -07:00
2c03524fa4
YARN-6526. Refactoring SQLFederationStateStore by avoiding to recreate a connection at every call. COntributed by Bilwa S T.
2020-06-26 20:43:27 +05:30
e0c1d8a969
YARN-10327. Remove duplication of checking for invalid application ID in TestLogsCLI. Contributed by Marton Hudaky
2020-06-26 13:21:55 +02:00
6459cc73e6
YARN-10318. ApplicationHistory Web UI incorrect column indexing. Contributed by Andras Gyori
2020-06-26 13:15:14 +02:00
33d3df7ee2
YARN-10277. CapacityScheduler test TestUserGroupMappingPlacementRule should build proper hierarchy. Contributed by Szilard Nemeth
2020-06-26 11:14:24 +02:00
6a8fd73b27
YARN-10279. Avoid unnecessary QueueMappingEntity creations. Contributed by Marton Hudaky
2020-06-25 17:27:58 +02:00
03f855e3e7
YARN-10316. FS-CS converter: convert maxAppsDefault, maxRunningApps settings. Contributed by Peter Bacsko
2020-06-23 12:13:04 +02:00
ce1008fe61
YARN-10321. Break down TestUserGroupMappingPlacementRule#testMapping into test scenarios
...
Contributed by Szilard Nemeth. Reviewed by Gergely Pollak.
2020-06-22 12:49:14 +05:30
b2facc84a1
YARN-9460. QueueACLsManager and ReservationsACLManager should not use instanceof checks. Contributed by Bilwa S T.
2020-06-20 20:09:31 +05:30
469841446f
YARN-9930. Support max running app logic for CapacityScheduler. Contributed by Peter Bacsko
2020-06-19 14:50:24 +02:00
89689c52c3
YARN-10308. Update javadoc and variable names for YARN service.
...
Contributed by Bilwa S T via eyang
2020-06-17 09:04:26 -07:00
5b1a56f9f1
YARN-10281. Redundant QueuePath usage in UserGroupMappingPlacementRule and AppNameMappingPlacementRule. Contributed by Gergely Pollak
2020-06-17 14:34:40 +02:00
e154084770
HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes ( #2026 )
2020-06-12 23:16:33 +05:30
7c4de59fc1
YARN-10293. Reserved Containers not allocated from available space of other nodes in CandidateNodeSet for MultiNodePlacement. Contributed by Prabhu Joseph.
2020-06-12 22:10:25 +08:00
fed6fecd3a
YARN-10312. Add support for yarn logs -logFile to retain backward compatibility.
...
Contributed by Jim Brennan.
2020-06-11 21:02:41 +00:00
56247db302
YARN-10300: appMasterHost not set in RM ApplicationSummary when AM fails before first heartbeat. Contributed by Eric Badger (ebadger).
2020-06-09 18:43:16 +00:00
8abff5151a
YARN-10292. FS-CS converter: add an option to enable asynchronous scheduling in CapacityScheduler. Contributed by Benjamin Teke
2020-06-05 15:00:39 +02:00
8b146c17b3
YARN-10274. Merge QueueMapping and QueueMappingEntity. Contributed by Gergely Pollak
2020-06-05 11:38:32 +02:00
a4835db95a
YARN-10296. Make ContainerPBImpl#getId/setId synchronized. Contributed by Benjamin Teke
2020-06-05 10:11:40 +02:00
b5efdea4fd
YARN-10254. CapacityScheduler incorrect User Group Mapping after leaf queue change. Contributed by Gergely Pollak
2020-06-02 18:29:15 +02:00
aa6d13455b
YARN-10284. Add lazy initialization of LogAggregationFileControllerFactory in LogServlet. Contributed by Adam Antal
2020-06-02 15:53:48 +02:00
e0a0741ac8
YARN-10286. PendingContainers bugs in the scheduler outputs. Contributed by Andras Gyori
2020-06-02 15:42:39 +02:00
593af878c0
YARN-10108. FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak
2020-05-27 13:05:52 +02:00
fb030f3239
Revert "FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak"
...
This reverts commit ac4540dd8e
2020-05-27 13:04:48 +02:00
c30c23cb66
YARN-6492. Generate queue metrics for each partition. Contributed by Manikandan R
2020-05-26 16:29:42 -07:00
ac4540dd8e
FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak
2020-05-21 13:11:44 +02:00
726b8e324b
YARN-10228. Relax restriction of file path character in yarn.service.am.java.opts.
...
Contributed by Bilwa S T via eyang
2020-05-20 09:23:56 -07:00
cef0756929
YARN-9606. Set sslfactory for AuthenticatedURL() while creating LogsCLI#webServiceClient.
...
Contributed by Bilwa S T. Reviewed by Akira Ajisaka.
2020-05-20 11:36:52 +05:30
b65815d691
Revert "YARN-9606. Set sslfactory for AuthenticatedURL() while creating LogsCLI#webServiceClient."
...
This reverts commit 7836bc4c35
2020-05-18 16:29:07 +09:00
7836bc4c35
YARN-9606. Set sslfactory for AuthenticatedURL() while creating LogsCLI#webServiceClient.
...
Contributed by Bilwa S T. Reviewed by Brahma Reddy Battula.
2020-05-14 19:40:42 +05:30
6ce295b787
YARN-10259. Fix reservation logic in Multi Node Placement.
...
Reviewed by Wangda Tan.
2020-05-14 16:52:11 +05:30
1958cb7c2b
YARN-10265. Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue. Contributed by liusheng
2020-05-14 15:09:34 +05:30
0918433b4d
YARN-9898. Dependency netty-all-4.1.27.Final doesn't support ARM platform. Contributed by liusheng.
2020-05-14 00:36:20 +05:30
108ecf992f
YARN-8942. PriorityBasedRouterPolicy throws exception if all sub-cluster weights have negative value. Contributed by Bilwa S T.
2020-05-13 10:04:12 -07:00
450e5aa9dd
YARN-10154. Addendum Patch which fixes below bugs
...
1. RM fails to start when LeafQueueTemplate max capacity is not specified.
2. Job stuck in ACCEPTED state with DominantResourceCalculator as Queue
Capacity is set to NaN during RM startup with clusterResource is zero.
Reviewed by Sunil G and Manikandan R.
2020-05-13 14:35:37 +05:30
fff1d2c122
YARN-10260. Allow transitioning queue from DRAINING to RUNNING state. Contributed by Bilwa S T
2020-05-12 10:48:54 -07:00
a3f945fb84
HADOOP-17035. fixed typos (timeout, interruped) ( #2007 )
...
Co-authored-by: Sungpeo Kook <elixir.kook@kakaocorp.com>
2020-05-12 10:50:04 -05:00
35010120fb
YARN-9017. PlacementRule order is not maintained in CS. Contributed by Bilwa S T.
2020-05-06 13:22:54 -07:00
92e3ebb401
YARN-8959. TestContainerResizing fails randomly (Ahmed Hussein via jeagles)
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2020-05-06 12:27:17 -05:00
cb6399c109
YARN-10257. FS-CS converter: skip increment properties for mem/vcores and fix DRF check. Contributed by Peter Bacsko
2020-05-05 16:00:43 +02:00
0debe55d6c
YARN-10160. Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph
2020-05-05 13:11:28 +02:00
f5081a9a5d
YARN-10256. Refactor TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic (Ahmed Hussein via jeagles)
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2020-05-04 10:13:51 -05:00
257b04ce4e
YARN-6553. Replace MockResourceManagerFacade with MockRM for AMRMProxy/Router tests. Contributed by Bilwa S T.
2020-05-02 12:43:20 +05:30
d125d39108
YARN-6973. Adding RM Cluster Id in ApplicationReport. Contributed by Bilwa S T.
2020-04-29 11:23:58 -07:00
410c605aec
YARN-10247. Application priority queue ACLs are not respected. Contributed by Sunil G
2020-04-29 15:53:30 +02:00
ab36429559
YARN-10215. Endpoint for obtaining direct URL for the logs. Contributed by Andras Gyori
2020-04-28 18:53:28 +02:00
6ef01646ba
YARN-9848. Revert YARN-4946. Contributed by Steven Rand.
2020-04-28 05:35:36 +09:00
9224568b0e
YARN-10156. Destroy Jersey Client in TimelineConnector.
...
Contributed by Tanu Ajmera.
2020-04-27 18:49:15 +05:30
f91e21ac10
YARN-10194. YARN RMWebServices /scheduler-conf/validate leaks ZK Connections. Contributed by Prabhu Joseph
2020-04-27 08:26:11 +02:00
d4874585f4
YARN-10237. Add isAbsoluteResource config for queue in scheduler response. Contributed by Prabhu Joseph
2020-04-24 17:04:47 +02:00
9827ff2961
YARN-10223. Remove jersey-test-framework-core dependency from yarn-server-common. ( #1939 )
2020-04-24 09:32:25 +09:00
60fa15366e
YARN-10240. Prevent Fatal CancelledException in TimelineV2Client when stopping
...
Contributed by Tarun Parimi.
2020-04-21 12:01:33 +05:30
92b28e38b7
YARN-9996. Code cleanup in QueueAdminConfigurationMutationACLPolicy. Contributed by Siddharth Ahuja
2020-04-18 10:13:37 +02:00
76900b4f5b
YARN-10189. Code cleanup in LeveldbRMStateStore. Contributed by Benjamin Teke
2020-04-18 09:57:27 +02:00
3601054822
YARN-9954. Configurable max application tags and max tag length. Contributed by Bilwa S T
2020-04-17 10:34:44 -07:00
2fe122e322
YARN-10154. CS Dynamic Queues cannot be configured with absolute resources. Contributed by Manikandan R.
2020-04-17 18:51:19 +05:30
4db598e0e6
YARN-9999. TestFSSchedulerConfigurationStore: Extend from ConfigurationStoreBaseTest, general code cleanup. Contributed by Benjamin Teke
2020-04-15 13:18:44 +02:00
94f7470c71
YARN-10233. Fix YARN UI2 Daemon Logs
...
Contributed by Akhil PB.
2020-04-15 13:39:37 +05:30
55fcbcb5c2
YARN-10234. FS-CS converter: don't enable auto-create queue property for root. Contributed by Peter Bacsko
2020-04-15 07:24:04 +02:00
c791b0e90e
YARN-10219. Fix YARN Native Service Placement Constraints with Node Attributes.
...
Contributed by Eric Yang.
2020-04-14 12:34:53 +05:30
23481ad378
YARN-10212. Create separate configuration for max global AM attempts. Contributed by Bilwa S T
2020-04-09 11:06:11 -07:00
c04a484aed
YARN-9995. Code cleanup in TestSchedConfCLI. Contributed by Bilwa S T.
2020-04-09 15:46:25 +02:00
6fb29d0f17
YARN-10226. NPE in Capacity Scheduler while using %primary_group queue mapping. Contributed by Peter Bacsko.
2020-04-09 18:51:42 +05:30
8e4517f251
YARN-5277. When localizers fail due to resource timestamps being out, provide more diagnostics. Contributed by Siddharth Ahuja
2020-04-09 12:31:01 +02:00
e53d472bb0
HADOOP-14836. Upgrade maven-clean-plugin to 3.1.0 ( #1933 )
2020-04-09 01:47:31 +09:00
2214005c0f
YARN-10063. Add container-executor arguments --http/--https to usage. Contributed by Siddharth Ahuja
2020-04-08 11:30:03 +10:00
bffb43b00e
YARN-10207. CLOSE_WAIT socket connection leaks during rendering of (corrupted) aggregated logs on the JobHistoryServer Web UI. Contributed by Siddharth Ahuja
2020-04-07 17:03:17 +02:00
45362a9f4c
YARN-10001. Add explanation of unimplemented methods in InMemoryConfigurationStore. Contributed by Siddharth Ahuja
2020-04-07 15:51:55 +02:00
7a39da3537
YARN-10120. Amendment fix for Java Doc.
...
Contributed by Bilwa S T.
2020-04-03 13:07:41 +05:30
25361b077b
YARN-10120. Https Support in Router WebServiceClient.
...
Contributed by Bilwa S T.
2020-04-02 18:11:43 +05:30
c162648aff
YARN-10202. Fix documentation about NodeAttributes. Contributed by Sen Zhao.
2020-04-01 16:05:47 +09:00
8914cf9167
Preparing for 3.4.0 development
2020-03-29 23:24:25 +05:30
50f7f6dfd1
YARN-10210. Add a RMFailoverProxyProvider that does DNS resolution on failover.
2020-03-26 08:14:30 -07:00
3d5ade1839
YARN-10043. FairOrderingPolicy Improvements. Contributed by Manikandan R
2020-03-26 13:23:43 +01:00
9a297ff31d
YARN-10197. FS-CS converter: fix emitted ordering policy string and max-am-resource percent value. Contributed by Peter Bacsko
2020-03-26 12:35:10 +01:00
348685dcb9
YARN-9998. Code cleanup in LeveldbConfigurationStore. Contributed by Benjamin Teke
2020-03-26 12:18:15 +01:00
6ce189c621
YARN-10200. Add number of containers to RMAppManager summary
2020-03-25 10:41:59 -07:00
2d294bd575
HADOOP-16938. Make non-HA proxy providers pluggable
2020-03-25 08:06:58 -07:00
cdb2107066
YARN-9879. Allow multiple leaf queues with the same name in CapacityScheduler. Contributed by Gergely Pollak.
2020-03-25 16:50:19 +05:30
d312991ead
YARN-10198. Remove Validation of Queue Exist under Managed Parent Queue
...
Contributed by Peter Bacsko. Reviewed by Manikandan R.
2020-03-23 18:01:14 +05:30
f2d3ac2a3f
YARN-10034. Remove Allocation Tags from released container from Decommission node
...
Contributed by Kyungwan Nam. Reviewed by Adam Antal.
2020-03-19 12:55:14 +05:30
1975479285
YARN-2710. RM HA tests failed intermittently on trunk. Contributed by Ahmed
...
Hussein.
2020-03-16 22:24:02 +00:00
b9d825f178
YARN-9567. Add diagnostics for outstanding resource requests on app attempts page. Contributed by Tao Yang.
2020-03-13 14:39:36 -07:00
d273f6a2b4
YARN-9538. Document scheduler/app activities and REST APIs. Contributed by Tao Yang.
2020-03-13 14:36:52 -07:00
c83644deac
YARN-10110. Adding RouterPolicyProvider for RM Federation.
...
Contributed by Bilwa S T.
2020-03-13 16:56:44 +05:30
5ead9c15ca
YARN-9997. Code cleanup in ZKConfigurationStore. Contributed by Andras Gyori
2020-03-12 12:29:03 +01:00
38d87883b6
YARN-10193. FS-CS converter: fix incorrect capacity conversion. Contributed by Peter Bacsko
2020-03-12 12:18:37 +01:00
0fd8bf5f6b
YARN-10191. FS-CS converter: call System.exit function call for every code path in main method. Contributed by Peter Bacsko.
2020-03-12 15:03:12 +05:30
cf9cf83a43
YARN-9354. Resources should be created with ResourceTypesTestHelper instead of TestUtils. Contributed by Andras Gyori
2020-03-10 16:44:48 +01:00
61f4cf3055
YARN-10002. Code cleanup and improvements in ConfigurationStoreBaseTest. Contributed by Benjamin Teke
2020-03-10 16:35:04 +01:00
9314ef947f
YARN-10168. FS-CS Converter: tool doesn't handle min/max resource conversion correctly. Contributed by Peter Bacsko
2020-03-10 16:07:46 +01:00
ede05b19d1
YARN-942. TestContainerSchedulerQueuing.testKillOnlyRequiredOpportunisticContainers fails sporadically Contributed by Ahmed Hussein (ahussein)
2020-03-10 14:17:45 +00:00
44afe1154d
YARN-9419. Log a warning if GPU isolation is enabled but LinuxContainerExecutor is disabled. Contribued by Andras Gyori
2020-03-09 16:08:24 +01:00
3859fa76d0
YARN-6924. Metrics for Federation AMRMProxy. Contributed by Young Chen
2020-03-07 09:34:42 +05:30
ea0444851d
YARN.10003. YarnConfigurationStore#checkVersion throws exception that belongs to RMStateStore. Contributed by Benjamin Teke
2020-03-06 17:49:45 +01:00
004e955348
YARN-10167. FS-CS Converter: Need to validate c-s.xml after converting. Contributed by Peter Bacsko
2020-03-05 09:56:42 +01:00
2649f8b327
YARN-10173. Make pid file generation timeout configurable in case of reacquired
...
container. Contributed by Adam Antal.
2020-03-04 23:31:57 +00:00
c0d084247c
YARN-10175. FS-CS converter: only convert placement rules if a cmd line switch is defined. Contributed by Peter Bacsko
2020-03-03 09:46:32 +01:00
e9eecedf69
YARN-10148. addendum: Fix method call parameter order of setAdminAndSubmitACL in TestCapacitySchedulerQueueACLs. Contributed by Kinga Marton
2020-03-02 16:25:14 +01:00
a43510e21d
YARN-10161. TestRouterWebServicesREST is corrupting STDOUT. Contributed by Jim Brennan.
2020-02-27 13:18:30 -08:00
b420ddeada
YARN-10155. TestDelegationTokenRenewer.testTokenThreadTimeout fails in trunk. Contributed by Manikandan R.
2020-02-27 12:53:20 -08:00
10461e0193
YARN-10148. Add Unit test for queue ACL for both FS and CS. Contributed by Kinga Marton
2020-02-27 20:10:32 +01:00
7dfa37e8f0
YARN-10152. Fix findbugs warnings in hadoop-yarn-applications-mawo-core module ( #1852 )
2020-02-27 10:36:49 +09:00
3a9ccf7f6d
YARN-10141.Interceptor in FederationInterceptorREST doesnt update on RM switchover. Contributed by D M Murali Krishna Reddy.
2020-02-26 23:24:00 +05:30
b5df1da3a1
YARN-9593. Support Comma in the value of Scheduler Configuration Mutation API.
2020-02-26 15:04:29 +05:30
d608e94f92
YARN-10156. Fix typo 'complaint' which means quite different in Federation.md ( #1856 )
2020-02-26 17:29:26 +09:00
d68616b045
YARN-10130. FS-CS converter: Do not allow output dir to be the same as input dir. Contributed by Adam Antal
2020-02-25 21:30:04 +01:00
dda00d3ff5
YARN-10074. Update netty to 4.1.42Final in yarn-csi. Contributed by Wei-Chiu Chuang. ( #1807 )
2020-02-25 13:47:52 +09:00
72fa9c7f1b
YARN-10135. FS-CS converter tool: issue warning on dynamic auto-create mapping rules. Contributed by Peter Bacsko
2020-02-24 21:54:07 +01:00
34cf63c7d0
YARN-10157. FS-CS converter: initPropertyActions() is not called without rules file. Contributed by Peter Bacsko
2020-02-24 21:45:49 +01:00
181e6d033e
YARN-10119. Option to reset AM failure count for YARN Service
...
Contributed by kyungwan nam.
2020-02-20 20:34:06 +05:30
4af2556b48
YARN-10143. YARN-10101 broke Yarn logs CLI. Contributed by Adam Antal
2020-02-20 15:27:15 +01:00
3f1aad05f0
YARN-10147. FPGA plugin can't find the localized aocx file. Contributed by Peter Bacsko
2020-02-19 15:54:25 +01:00
6526f95bd2
YARN-10139. ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB. Contributed by Prabhu Joseph.
2020-02-19 11:17:22 +05:30
d6d7f8d8c5
YARN-8374. Upgrade objenesis to 2.6 ( #1798 )
2020-02-19 09:50:37 +09:00
20add89718
YARN-10136. [Router] : Application metrics are hardcode as N/A in UI. Contributed by Bilwa S T.
2020-02-14 16:50:29 +05:30
da99ac7e93
YARN-10137. UIv2 build is broken in trunk. Contributed by Adam Antal
2020-02-13 16:31:35 +01:00
f1b1b332f5
YARN-10029. Add option to UIv2 to get container logs from the new JHS API. Contributed by Adam Antal
2020-02-13 12:08:54 +01:00
fe7d67a8a2
YARN-9521. Handle FileSystem close in ApiServiceClient
...
Contributed by kyungwan nam. Reviewed by Eric Yang.
2020-02-13 09:39:13 +05:30
e637797211
YARN-10127. Remove setting App Ordering Policy to ParentQueue in FSQueueConverter
...
Contributed by Peter Bacsko.
2020-02-11 22:01:58 +05:30
28f730b317
YARN-10109. Allow stop and convert from leaf to parent queue in a single Mutation API call. Contributed by Prabhu Joseph
2020-02-09 21:14:53 +05:30
3f0a7cd17a
YARN-9624. Use switch case for ProtoUtils#convertFromProtoFormat containerState. Contributed by Bilwa S T
2020-02-09 19:14:18 +05:30
7dac7e1d13
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency ( #1635 ). Contributed by Vinayakumar B.
2020-02-07 14:51:24 +05:30
71b2c2ffe9
YARN-10101. Support listing of aggregated logs for containers belonging to an application attempt. Contributed by Adam Antal
2020-02-06 12:25:06 +01:00
314e2f9d2e
YARN-10116. Expose diagnostics in RMAppManager summary
2020-02-04 17:44:05 -08:00
bf8686f43f
YARN-8982. [Router] Add locality policy. Contributed by Young Chen.
2020-01-30 16:59:36 -08:00
a7d72c523a
YARN-10099. FS-CS converter: handle allow-undeclared-pools and user-as-default-queue properly and fix misc issues. Contributed by Peter Bacsko
2020-01-30 16:03:38 +01:00
a5ef08b619
YARN-9743. [JDK11] TestTimelineWebServices.testContextFactory fails. ( #1824 ) Contributed by Akira Ajisaka and Kinga Marton.
2020-01-30 14:10:31 +09:00
825db8fe2a
YARN-10107. Fix GpuResourcePlugin#getNMResourceInfo to honor Auto Discovery Enabled
...
Contributed by Szilard Nemeth.
2020-01-29 13:30:00 +05:30
e578e52aae
YARN-10084. Allow inheritance of max app lifetime / default app lifetime. Contributed by Eric Payne.
2020-01-29 03:54:43 +00:00
5abd0148eb
YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R.
2020-01-28 10:06:37 -08:00
1ab9c692fa
YARN-10022. RM Rest API to validate the CapacityScheduler Configuration change
...
Contributed by Kinga Marton.
2020-01-28 23:16:04 +05:30
94f060242e
YARN-10104. FS-CS converter: dry run should work without output defined. Contributed by Peter Bacsko
2020-01-28 15:36:04 +01:00
ca29768035
YARN-10085. FS-CS converter: remove mixed ordering policy check. Contributed by Peter Bacsko
2020-01-28 15:22:48 +01:00
9520b2ad79
YARN-10083. Provide utility to ask whether an application is in final status. Contributed by Adam Antal
2020-01-22 16:25:07 +01:00
b4870bce3a
Revert "YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R."
...
This reverts commit 0696828a09
2020-01-21 17:45:17 -08:00
0696828a09
YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R.
2020-01-21 13:41:01 -08:00
1defe3a65a
HADOOP-16753. Refactor HAAdmin. Contributed by Xieming Li.
2020-01-21 10:58:32 +09:00
581072a8f0
YARN-7913. Improve error handling when application recovery fails with exception. Contributed by Wilfred Spiegelenburg
2020-01-20 13:10:32 +01:00
6d52bbbfcf
YARN-9525. IFile format is not working against s3a remote folder. Contributed by Adam Antal
2020-01-20 12:36:55 +01:00
8b3ee2f7e9
YARN-9462. TestResourceTrackerService.testNodeRemovalGracefully fails sporadically. Contributed by Prabhu Joseph
2020-01-20 09:54:22 +01:00
57aad0f43a
YARN-10081. Exception message from ClientRMProxy#getRMAddress is misleading. Contributed by Ravuri Sushma Sree
2020-01-20 09:41:06 +01:00
14d0f9a775
YARN-8148. Update decimal values for queue capacities shown on queue status CLI. Contributed by Prabhu Joseph
2020-01-20 09:28:23 +01:00
a0ff42d761
YARN-10070. Fix NPE if no queue mapping defined for proxy user when using application tag based placement.
...
Contributed by Kinga Marton. Reviewed by Adam Antal.
2020-01-16 12:55:27 +05:30
14c2c3d69d
YARN-9512. [JDK11] TestAuxServices#testCustomizedAuxServiceClassPath fails because of ClassCastException. ( #1806 )
2020-01-16 12:21:50 +09:00
2aa065d98f
YARN-10082. FS-CS converter: disable terminal placement rule checking. Contributed by Peter Bacsko
2020-01-15 12:42:13 +01:00
7c5cecc3b3
YARN-9970. Refactor TestUserGroupMappingPlacementRule#verifyQueueMapping. Contributed by Manikandan R
2020-01-15 09:55:41 +01:00
1c51f36be7
YARN-9788. Queue Management API does not support parallel updates. Contributed by Prabhu Joseph
2020-01-14 12:26:03 +01:00
13cea0412c
YARN-10028. Integrate the new abstract log servlet to the JobHistory server. Contributed by Adam Antal
2020-01-14 11:00:08 +01:00
621c5eac38
YARN-9912. Capacity scheduler: support u:user2:%secondary_group queue mapping. Contributed by Manikandan R
2020-01-13 17:23:00 +01:00
b7ef8a333f
YARN-9868. Validate %primary_group queue in CS queue manager. Contributed by Manikandan R
2020-01-13 17:15:09 +01:00
2576c31644
YARN-9989. Typo in CapacityScheduler documentation: Runtime Configuration. Contributed by Kevin Su
2020-01-13 16:50:07 +01:00
d842dfffa5
YARN-9866. u:user2:%primary_group is not working as expected. Contributed by Manikandan R
2020-01-12 14:04:15 +01:00
24e6a9e43a
YARN-10067. Add dry-run feature to FS-CS converter tool. Contributed by Peter Bacsko
2020-01-12 13:48:39 +01:00
2301b25899
YARN-10019. Improved container-executor exec() calls.
...
Contributed by Peter Bacsko
2020-01-10 19:04:04 -05:00
20a90c0b5b
YARN-10071. Sync Mockito version with other modules
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2020-01-10 17:41:04 +09:00
93233a7d6e
YARN-9018. Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path. Contributed by Kuhu Shukla (kshukla)
2020-01-09 17:18:44 +00:00
b1e07d27cc
YARN-7387: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer fails intermittently. Contributed by Jim Brennan (Jim_Brennan)
2020-01-08 19:26:01 +00:00
6899be5a17
YARN-10072: TestCSAllocateCustomResource failures. Contributed by Jim Brennan (Jim_Brennan)
2020-01-08 17:29:56 +00:00
571795cd18
YARN-10068. Fix TimelineV2Client leaking File Descriptors.
...
Contributed by Anand Srinivasan. Reviewed by Adam Antal.
2020-01-08 12:01:30 +05:30
d81d45ff2f
YARN-9956. Improved connection error message for YARN ApiServerClient.
...
Contributed by Prabhu Joseph
2020-01-06 13:24:16 -05:00
dd2607e3ec
YARN-10026. Pull out common code pieces from ATS v1.5 and v2. Contributed by Adam Antal
2020-01-06 17:16:11 +01:00
768ee22e9e
YARN-10035. Add ability to filter the Cluster Applications API request by name. Contributed by Adam Antal
2020-01-06 16:26:33 +01:00
eca7e14c2f
YARN-10053. Use Shared Group Mapping Service in Placement Rules.
...
Contributed by Wilfred Spiegelenburg.
2020-01-02 14:13:57 +05:30
21ada4d1b0
Revert "YARN-10053. Use Shared Group Mapping Service in Placement Rules."
...
This reverts commit 217b56ffdd
2020-01-02 14:12:43 +05:30
217b56ffdd
YARN-10053. Use Shared Group Mapping Service in Placement Rules.
...
Contributed by Wilfred Spiegelenburg.
2020-01-02 14:07:49 +05:30
0fed874adf
YARN-10041. Should not use AbstractPath to create unix domain socket ( #1771 )
2019-12-27 16:50:15 +05:30
40887c9b12
YARN-10057. Upgrade the dependencies managed by yarnpkg. ( #1780 )
2019-12-25 14:10:44 +09:00
34ff7dbaf5
YARN-10055. bower install fails. ( #1778 )
2019-12-23 19:45:47 +09:00
412035b47a
YARN-10009. In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined. Contributed by Eric Payne.
2019-12-20 18:07:00 +00:00
12722ab0c7
YARN-10042. Upgrade grpc-xxx depdencies to 1.26.0. Contributed by Sheng Liu.
2019-12-20 11:10:27 +08:00
ef59ffd362
YARN-10038. [UI] Finish Time is not correctly parsed in the RM Apps page. Contributed by Inigo Goiri.
2019-12-19 11:37:17 -08:00
7868da894a
YARN-10037. Upgrade build tools for YARN Web UI v2.
2019-12-19 19:27:31 +09:00
7b93575b92
YARN-9894. CapacitySchedulerPerf test for measuring hundreds of apps in a large number of queues. Contributed by Eric Payne
2019-12-18 13:18:11 -08:00
fddc3d55c3
YARN-10039. Allow disabling app submission from REST endpoints
2019-12-18 10:48:05 -08:00
92c8962f98
YARN-10020. Fix build instruction of hadoop-yarn-ui. ( #1750 )
2019-12-18 13:34:12 +09:00
f47dcf2d4c
YARN-10033. TestProportionalCapacityPreemptionPolicy not initializing vcores for effective max resources. Contributed by Eric Payne.
2019-12-17 17:11:23 +00:00
631dbbc6f2
YARN-9923. Introduce HealthReporter interface to support multiple health checker files. Contributed by Adam Antal
2019-12-15 17:28:04 +01:00
d4bde134e3
YARN-7721. Fix TestContinuousScheduling fails sporadically with NPE.
...
Contributed by Wilfred Spiegelenburg.
2019-12-10 15:00:23 +05:30
dc66de7448
YARN-9985. Unsupported transitionToObserver option displaying for rmadmin command. Contributed by Ayush Saxena.
2019-12-09 18:37:34 +09:00
e9c5bb8078
Bump bower from 1.7.7 to 1.8.8 ( #1683 )
...
Bumps [bower](https://github.com/bower/bower ) from 1.7.7 to 1.8.8.
- [Release notes](https://github.com/bower/bower/releases )
- [Changelog](https://github.com/bower/bower/blob/master/CHANGELOG.md )
- [Commits](https://github.com/bower/bower/compare/v1.7.7...v1.8.8 )
Signed-off-by: dependabot[bot] <support@github.com>
2019-12-09 15:46:08 +09:00
8dffd8dc89
YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger
2019-12-09 01:25:10 +00:00
ab2cc5ac92
Revert "YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger"
...
This reverts commit 76bb297db9
2019-12-09 01:25:10 +00:00
6d970d8100
YARN-10012. Guaranteed and max capacity queue metrics for custom resources. Contributed by Manikandan R
2019-12-08 16:35:02 -08:00
cd4779f317
Revert "YARN-10012. Guaranteed and max capacity queue metrics for custom resources"
...
This reverts commit 92bce918dc
2019-12-08 16:34:46 -08:00
92bce918dc
YARN-10012. Guaranteed and max capacity queue metrics for custom resources
2019-12-08 15:52:17 -08:00
76bb297db9
YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger
2019-12-06 18:33:58 +00:00
e575df5ca6
YARN-10006. IOException used in place of YARNException in CapaitySheduler
...
Contributed by Adam Antal. Reviewed by Szilard Nemeth.
2019-12-06 12:51:28 +05:30
4f758dd4c6
YARN-9607. Auto-configuring rollover-size of IFile format for non-appendable filesystems. Contributed by Adam Antal
2019-12-05 21:49:00 +01:00
cf68857631
YARN-9938. Validate Parent Queue for QueueMapping contains dynamic group as parent queue. Contributed by Manikandan R
2019-12-05 21:04:42 +01:00
0ab9c0398d
YARN-9969. Improve yarn.scheduler.capacity.queue-mappings documentation. Contributed by Manikandan R
2019-12-05 20:53:43 +01:00
5cc6f945da
YARN-9789. Disable Option for Write Ahead Logs of LogMutation. Contributed by Prabhu Joseph
2019-12-05 20:43:26 +01:00
4627dd6708
YARN-9780. SchedulerConf Mutation API does not Allow Stop and Remove Queue in a single call. Contributed by Prabhu Joseph
2019-12-05 20:38:37 +01:00
c71befaf8f
YARN-9781. SchedConfCli to get current stored scheduler configuration. Contributed by Prabhu Joseph
2019-12-05 20:31:24 +01:00
520fe2c99b
YARN-5106. Provide a builder interface for FairScheduler allocations for use in tests. Contributed by Adam Antal
2019-12-05 17:37:40 +01:00
682e6fdeda
YARN-9052. Replace all MockRM submit method definitions with a builder. Contributed by Szilard Nemeth.
2019-12-05 13:26:23 +05:30
Peter Bacsko
Eric Badger
Neil
JJiaguodong
Akira Ajisaka
Ahmed Hussein
Jonathan Hung
Jim Brennan
Szilard Nemeth
Masatake Iwasaki
Amogh Rajesh Desai
bibinchundatt
belugabehr
Wangda Tan
Sunil G
Eric Payne
He Xiaoqiao
srinivasst
zhuqi
adamantal
zhuqi
Andrea Scarpino
kevinzhao1661
Ankit Kumar
Gabriel Medeiros Coelho
neko
Surendra Singh Lilhore
Ayush Saxena
Adam Antal
Fei Hui
Aryan Gupta
Shubham Gupta
tkuramoto33
Eric E Payne
Brahma Reddy Battula
Prabhu Joseph
Zhengbo Li
hemanthboyina
Ahmed Hussein
Bilwa S T
Xiaoyu Yao
Eric Yang
Surendra Singh Lilhore
Vinayakumar B
Tao Yang
Prabhu Joseph
Inigo Goiri
Elixir Kook
Prabhu Joseph
Wilfred Spiegelenburg
RogPodge
Weiwei Yang
tanu.ajmera
Wei-Chiu Chuang
Giovanni Matteo Fumarola
Liu sheng
Zhankun Tang
dependabot[bot]