e31d06032b
YARN-10802. Change Capacity Scheduler minimum-user-limit-percent to accept decimal values. Contributed by Benjamin Teke
2021-06-14 22:33:04 +02:00
f0bdc422aa
YARN-10816. Avoid doing delegation token ops when yarn.timeline-service.http-authentication.type=simple. Contributed by Tarun Parimi
2021-06-12 15:18:41 +02:00
7003997e36
YARN-10789. RM HA startup can fail due to race conditions in ZKConfigurationStore. Contributed by Tarun Parimi
2021-06-12 14:49:52 +02:00
4ef27a596f
HADOOP-17753. Keep restrict-imports-enforcer-rule for Guava Lists in top level hadoop-main pom ( #3087 )
2021-06-11 12:15:52 +09:00
9a17f97133
YARN-10803. [JDK 11] TestRMFailoverProxyProvider and TestNoHaRMFailoverProxyProvider fails by ClassCastException. ( #3068 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
2021-06-10 14:29:47 +09:00
81d7069316
YARN-10805. Replace Guava Lists usage by Hadoop's own Lists in hadoop-yarn-project ( #3075 )
2021-06-09 15:15:47 +09:00
ca27744770
YARN-10809. Missing dependency causing NoClassDefFoundError in TestHBaseTimelineStorageUtils ( #3081 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
2021-06-08 19:37:23 -07:00
9445abb500
YARN-10792. Set Completed AppAttempt LogsLink to Log Server URL. Contributed by Abhinaba Sarkar
2021-06-08 20:37:40 +05:30
ec16b1d3b9
YARN-10807. Parents node labels are incorrectly added to child queues in weight mode. Contributed by Benjamin Teke.
2021-06-08 21:03:43 +08:00
9c7b8cf54e
Fix container-executor
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-06-06 07:05:29 +09:00
200eec8f2e
YARN-10796. Capacity Scheduler: dynamic queue cannot scale out properly if its capacity is 0%. Contributed by Peter Bacsko
2021-06-03 13:27:36 +02:00
2707f69251
YARN-10787. Queue submit ACL check is wrong when CS queue is ambiguous. Contributed by Gergely Pollak
2021-06-01 16:01:39 +02:00
e9339aa376
YARN-10797. Logging parameter issues in scheduler package. Contributed by Szilard Nemeth
2021-06-01 15:57:22 +02:00
986d0a4f1d
HADOOP-17732. Keep restrict-imports-enforcer-rule for Guava Sets in hadoop-main pom ( #3049 )
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-05-26 17:14:31 +09:00
c4581827a9
YARN-10691. DominantResourceCalculator isInvalidDivisor should consider only countable resource types. Contributed by Bilwa S T.
2021-05-25 18:07:28 +00:00
b86a6eb871
YARN-10782. Extend /scheduler endpoint with template properties. Contributed by Andras Gyori
2021-05-25 18:27:53 +02:00
2541efa496
YARN-10783. Allow definition of auto queue template properties in root. Contributed by Andras Gyori
2021-05-25 13:55:59 +02:00
1cbcde04f3
YARN-10770. container-executor permission is wrong in SecureContainer.md. Contributed by Siddharth Ahuja.
2021-05-25 18:28:09 +09:00
996d31f2dc
HADOOP-17721. Replace Guava Sets usage by Hadoop's own Sets in hadoop-yarn-project ( #3033 )
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-05-25 18:10:20 +09:00
59172ada90
YARN-10771. Add cluster metric for size of SchedulerEventQueue and RMEventQueue. Contributed by chaosju.
2021-05-24 23:12:07 +08:00
6bb0892f6e
YARN-10753. Document the removal of FS default queue creation. Contributed by Benjamin Teke
2021-05-22 15:00:08 +02:00
1e44bdb84c
YARN-7769. FS QueueManager should not create default queue at init. Contributed by Benjamin Teke
2021-05-22 14:55:01 +02:00
d146ab3af7
YARN-10766. [UI2] Bump moment-timezone to 0.5.33. Contributed by Andras Gyori
2021-05-21 18:07:32 +02:00
43e77caa71
YARN-10779. Add option to disable lowercase conversion in GetApplicationsRequestPBImpl and ApplicationSubmissionContextPBImpl. Contributed by Peter Bacsko
2021-05-21 18:04:24 +02:00
c8073811b2
YARN-9279. Remove the old hamlet package. ( #2986 )
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-05-21 12:40:50 +09:00
f8b0063201
Bump node-sass ( #3004 )
...
Bumps [node-sass](https://github.com/sass/node-sass ) from 4.13.0 to 4.14.1.
- [Release notes](https://github.com/sass/node-sass/releases )
- [Changelog](https://github.com/sass/node-sass/blob/master/CHANGELOG.md )
- [Commits](https://github.com/sass/node-sass/compare/v4.13.0...v4.14.1 )
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Reviewed-by: Gergely Pollak <shuzirra@cloudera.com>
2021-05-18 20:14:18 -07:00
eb72628e15
YARN-10258. Add metrics for 'ApplicationsRunning' in NodeManager. Contributed by ANANDA G B.
2021-05-17 21:07:29 +02:00
8891e5c028
YARN-10763. Add the number of containers assigned per second metrics to ClusterMetrics. Contributed by chaosju.
2021-05-17 13:30:12 +02:00
d92a25b790
YARN-10555. Missing access check before getAppAttempts ( #2608 )
...
Co-authored-by: lujie <lujie@foxmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-05-17 13:44:38 +09:00
344cacce23
YARN-10545. Improve the readability of diagnostics log in yarn-ui2 web page. #2540
...
Contributed by kyoty.
2021-05-14 23:57:54 +08:00
e7f0e8073b
YARN-10761: Add more event type to RM Dispatcher event metrics. Contributed by Qi Zhu.
2021-05-14 13:36:07 +08:00
d2b0675d61
YARN-10737: Fix typos in CapacityScheduler#schedule. ( #2911 )
...
Contributed by Qi Zhu.
2021-05-14 13:12:28 +08:00
881ab4ed39
Bump underscore ( #2997 )
...
Bumps [underscore](https://github.com/jashkenas/underscore ) from 1.9.1 to 1.13.1.
- [Release notes](https://github.com/jashkenas/underscore/releases )
- [Commits](https://github.com/jashkenas/underscore/compare/1.9.1...1.13.1 )
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Reviewed-by: Gergely Pollak <shuzirra@cloudera.com>
2021-05-13 05:54:07 -07:00
2f67a2646e
Bump acorn ( #3003 )
...
Bumps [acorn](https://github.com/acornjs/acorn ) from 5.7.3 to 5.7.4.
- [Release notes](https://github.com/acornjs/acorn/releases )
- [Commits](https://github.com/acornjs/acorn/compare/5.7.3...5.7.4 )
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Reviewed-by: Gergely Pollak <shuzirra@cloudera.com>
2021-05-13 05:51:43 -07:00
626be24c3e
YARN-10571. Refactor dynamic queue handling logic. Contributed by Andras Gyori.
2021-05-12 14:54:47 +02:00
91430889a5
HADOOP-17665 Ignore missing keystore configuration in reloading mechanism
2021-05-10 13:31:48 -07:00
7cb7b8adc1
YARN-10756. Remove additional junit 4.11 dependency from javadoc. ( #2960 )
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Bilwa S T <stbilwa@gmail.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-05-06 23:37:45 +09:00
f1e1809029
HADOOP-17653. Do not use guava's Files.createTempDir(). ( #2945 )
...
Reviewed-by: Steve Loughran <stevel@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-05-02 11:10:32 +09:00
803ac4b1a0
YARN-10707. Support custom resources in ResourceUtilization, and update Node GPU Utilization to use. Contributed by Qi Zhu
2021-04-29 17:42:13 +00:00
5b09ba75ec
YARN-10747. Bump YARN CSI protobuf version to 3.7.1 ( #2946 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Change-Id: Ib2a4bf775a5804641a91c6f9a486772c9d87c17e
2021-04-28 12:44:13 +09:00
a967ab06f2
YARN-10739. GenericEventHandler.printEventQueueDetails causes RM recovery to take too much time. Contributed by Qi Zhu.
2021-04-27 14:01:53 +02:00
9166bfeb74
YARN-10637. fs2cs: add queue autorefresh policy during conversion. Contributed by Qi Zhu.
2021-04-24 10:10:10 +02:00
538ce9c354
HADOOP-17650. Bump solr to unblock build failure with Maven 3.8.1 ( #2939 )
...
Reviewed-by: Siyao Meng <siyao@apache.org>
2021-04-25 23:48:39 -07:00
6857a05d6a
YARN-10479. Can't remove all node labels after add node label without
...
nodemanager port, broken by YARN-10647. Contributed by D M Murali Krishna Reddy
2021-04-23 22:02:04 +00:00
1cbe35946b
YARN-10743. Add a policy for not aggregating for containers which are killed because exceeding container log size limit. Contributed by Qi Zhu.
2021-04-23 21:35:09 +00:00
ebf6b14c67
YARN-10123. Error message around yarn app -stop/start can be improved. Contributed by Siddharth Ahuja
...
This is required to highlight that an implementation at framework level is needed for the stop/start functionality to work.
2021-04-23 22:49:26 +02:00
14a84c47b0
YARN-10705. Misleading DEBUG log for container assignment needs to be removed when the container is actually reserved, not assigned in FairScheduler. Contributed by Siddharth Ahuja.
2021-04-23 17:38:47 +02:00
3b6b21e459
YARN-10750. TestMetricsInvariantChecker.testManyRuns is broken since HADOOP-17524. Contributed by Gergely Pollak
2021-04-23 16:17:29 +02:00
919daec36b
YARN-10746. RmWebApp add default-node-label-expression to the queue info. Contributed by Gergely Pollak
2021-04-23 16:12:12 +02:00
f76a2a7606
YARN-10654. Dots '.' in CSMappingRule path variables should be replaced. Contributed by Peter Bacsko
2021-04-23 16:07:58 +02:00
6cb90005a7
YARN-10723. Change CS nodes page in UI to support custom resource. Contributed by Qi Zhu
2021-04-20 17:34:49 +00:00
9a261b0a16
HADOOP-17642. Remove appender EventCounter to avoid instantiation ( #2922 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-04-17 20:47:44 +09:00
ba3bc53f4e
YARN-10736. Fix GetApplicationsRequest JavaDoc. Contributed by Miklos Gergely. ( #2906 )
2021-04-14 17:16:30 +02:00
6a9a5ae5fd
Bump y18n ( #2843 )
...
Bumps [y18n](https://github.com/yargs/y18n ) from 3.2.1 to 3.2.2.
- [Release notes](https://github.com/yargs/y18n/releases )
- [Changelog](https://github.com/yargs/y18n/blob/master/CHANGELOG.md )
- [Commits](https://github.com/yargs/y18n/commits )
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-04-11 01:30:49 +09:00
213d3deb26
YARN-10503. Support queue capacity in terms of absolute resources with custom
...
resourceType. Contributed by Qi Zhu.
2021-04-09 00:34:15 +00:00
ca9aa91d10
YARN-10564. Support Auto Queue Creation template configurations. Contributed by Andras Gyori.
2021-04-08 12:42:48 +02:00
9cd69c20c4
YARN-10714. Remove dangling dynamic queues on reinitialization. Contributed by Andras Gyori
2021-04-07 11:52:21 +02:00
26b8f678b2
YARN-10702. Add cluster metric for amount of CPU used by RM Event Processor.
...
Contributed by Jim Brennan.
2021-04-06 01:16:14 +00:00
158758c5bf
YARN-10726. Log the size of DelegationTokenRenewer event queue in case of too many pending events. Contributed by Qi Zhu.
2021-04-01 16:09:52 +02:00
9f1655baf2
YARN-9618. NodesListManager event improvement. Contributed by Qi Zhu.
2021-04-01 11:39:40 +02:00
a0deda1a77
YARN-10720. YARN WebAppProxyServlet should support connection timeout to prevent proxy server from hanging. Contributed by Qi Zhu.
2021-04-01 09:21:15 +02:00
2c482fbacf
HADOOP-16524. Automatic keystore reloading for HttpServer2
...
Reapply of issue reverted first because it caused yarn failures and
then again because the commit message was incorrectly formatted
(and yet again because of commit message format).
Signed-off-by: stack <stack@apache.org>
2021-03-31 10:46:35 -07:00
22961a615d
Revert "HADOOP-16524. Automatic keystore reloading for HttpServer2"
...
This reverts commit a2975d2153
2021-03-31 10:43:09 -07:00
a2975d2153
HADOOP-16524. Automatic keystore reloading for HttpServer2
...
Reapply of issue reverted first because it caused yarn failures and
then again because the commit message was incorrectly formatted.
2021-03-31 10:40:20 -07:00
5183aaeda2
Revert "Hadoop 16524 - resubmission following some unit test fixes ( #2693 )"
...
Revert to fix the summary message.
This reverts commit 9509bebf7f
2021-03-31 10:39:55 -07:00
9509bebf7f
Hadoop 16524 - resubmission following some unit test fixes ( #2693 )
...
Signed-off-by: stack <stack@apache.org>
2021-03-31 10:07:42 -07:00
6fd0c661b6
YARN-10597. CSMappingPlacementRule should not create new instance of Groups. Contributed by Gergely Pollak
2021-03-31 16:14:21 +02:00
ff6ec20d84
YARN-10718. Fix CapacityScheduler#initScheduler log error. Contributed by Qi Zhu.
2021-03-31 10:55:14 +02:00
8ad77a26c9
YARN-10716. Fix typo in ContainerRuntime. Contributed by xishuhai.
2021-03-30 17:46:51 +09:00
413a4c3c05
YARN-10466.Fix NullPointerException in yarn-services Component.java. Contributed by D M Murali Krishna Reddy
2021-03-30 13:52:07 +05:30
6577bf1891
YARN-10439. addendum fix for shaded guva.
2021-03-30 13:48:40 +05:30
b61f52ec56
YARN-10441. Add support for hadoop.http.rmwebapp.scheduler.page.class. Contributed by D M Murali Krishna Reddy
2021-03-30 09:51:47 +05:30
d0dcfc405c
YARN-10439. Yarn Service AM listens on all IP's on the machine. Contributed by D M Murali Krishna Reddy
2021-03-30 09:46:12 +05:30
2d62dced4b
YARN-10437. Destroy yarn service if any YarnException occurs during submitApp.Contributed by D M Murali Krishna Reddy
2021-03-30 09:39:00 +05:30
19e418c10d
YARN-10713. ClusterMetrics should support custom resource capacity related metrics. Contributed by Qi Zhu.
2021-03-25 22:33:58 +00:00
ceb75e1e2a
YARN-10674. fs2cs should generate auto-created queue deletion properties. Contributed by Qi Zhu.
2021-03-24 08:15:06 +01:00
174f3a96b1
YARN-10697. Resources are displayed in bytes in UI for schedulers other than capacity. Contributed by Bilwa S T.
2021-03-23 18:21:45 +00:00
d8ec8ab965
HADOOP-17599. Remove NULL checks before instanceof ( #2804 )
2021-03-23 08:46:11 -07:00
cd44e917d0
YARN-10476. Queue metrics for Unmanaged applications ( #2674 ). Contributed by Cyrus Jackson
2021-03-19 15:49:05 +05:30
3d4ea4b336
YARN-10701. The yarn.resource-types should support multi types without trimmed. Contributed by Qi Zhu.
2021-03-18 19:36:50 +00:00
181b9ddc51
YARN-10703. Fix potential null pointer error of gpuNodeResourceUpdateHandler in NodeResourceMonitorImpl. Contributed by Qi Zhu.
2021-03-18 17:45:09 +00:00
ce6bfd5718
YARN-10641. Refactor the max app related update, and fix maxApllications update error when add new queues. Contributed by Qi Zhu.
2021-03-18 13:40:16 +01:00
38495af325
YARN-10692. Add Node GPU Utilization and apply to NodeMetrics. Contributed by Qi Zhu.
2021-03-18 12:46:11 +01:00
a5745711dd
YARN-10659. Improve CS MappingRule %secondary_group evaluation. Contributed by Gergely Pollak
2021-03-18 12:43:01 +01:00
d7eeca4d0c
YARN-10685. Fix typos in AbstractCSQueue. Contributed by Qi Zhu.
2021-03-18 11:49:16 +01:00
49f89f1d3d
YARN-10688. ClusterMetrics should support GPU capacity related metrics.. Contributed by Qi Zhu.
2021-03-17 18:11:37 +00:00
3e58d5611d
YARN-10497. Fix an issue in CapacityScheduler which fails to delete queues. Contributed by Wangda Tan and Qi Zhu.
2021-03-17 13:38:20 +01:00
f276f1af80
YARN-10652. Capacity Scheduler fails to handle user weights for a user that has a "." (dot) in it
...
This only fixes the user name resolution for weights in the queues. It
does not add generic support for user names with dots in all use cases
in the capacity scheduler.
Contributed by: Siddharth Ahuja
2021-03-17 10:55:05 +11:00
73394fabc7
YARN-10686. Fix TestCapacitySchedulerAutoQueueCreation#testAutoQueueCreationFailsForEmptyPathWithAQCAndWeightMode. Contributed by Qi Zhu.
2021-03-16 16:09:12 +01:00
b80588b688
YARN-10682. The scheduler monitor policies conf should trim values separated by comma. Contributed by Qi Zhu.
2021-03-16 15:23:27 +01:00
e9c98548e9
YARN-10689. Fix the finding bugs in extractFloatValueFromWeightConfig. ( #2760 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-03-16 13:50:29 +09:00
aa4c17b9d7
YARN-10588. Percentage of queue and cluster is zero in WebUI . Contributed by Bilwa S T
2021-03-15 19:09:40 +00:00
5f067cf0f3
[YARN-10687] Add option to disable/enable free disk space checking and percentage checking for full and not-full disks. Contributed by Qi Zhu.
2021-03-12 17:17:31 +00:00
5db4c0bf70
YARN-10412. Move CS placement rule related changes to a separate package. Contributed by Gergely Pollak
2021-03-12 14:10:16 +01:00
23b343aed1
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin ( #2753 )
...
Removed findbugs from the hadoop build images and added spotbugs instead.
Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0.
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
2021-03-11 10:56:07 +09:00
b2a565629d
YARN-10671.Fix Typo in TestSchedulingRequestContainerAllocation. Contributed by D M Murali Krishna Reddy.
2021-03-09 20:26:07 +05:30
d5e035dbe1
YARN-9615. Add dispatcher metrics to RM. Contributed by Jonathan Hung and Qi Zhu.
2021-03-09 14:33:14 +01:00
3851994cd6
Revert "YARN-9615. Add dispatcher metrics to RM. Contributed by Qi Zhu."
...
This reverts commit 369f75b7a7
2021-03-09 14:32:02 +01:00
369f75b7a7
YARN-9615. Add dispatcher metrics to RM. Contributed by Qi Zhu.
2021-03-09 14:28:23 +01:00
20416bc62d
YARN-10675. Consolidate YARN-10672 and YARN-10447. Contributed by Szilard Nemeth.
2021-03-09 11:34:37 +01:00
Szilard Nemeth
Viraj Jasani
Akira Ajisaka
Prabhu Josephraj
zhuqi-lucas
Hideyuki Furue
Gergely Pollak
Jim Brennan
dependabot[bot]
Peter Bacsko
lujiefsi
kyoty
zhuqi
Borislav Iordanov
Wei-Chiu Chuang
Eric Badger
Siyao Meng
Miklos Gergely
stack
Brahma Reddy Battula
Jack Jiang
Cyrus Jackson
Wilfred Spiegelenburg
Eric Payne