75bc6cfced
YARN-11176. Refactor TestAggregatedLogDeletionService. Contributed by Szilard Nemeth.
2022-06-14 16:14:44 +02:00
c9a174a260
YARN-11175. Refactor LogAggregationFileControllerFactory
2022-06-13 13:58:13 +02:00
985161f6ec
YARN-11173. remove redeclaration of os-maven-plugin.version from yarn-csi ( #4417 )
...
This is a followup to HADOOP-18275 and its upgrade of os-maven-plugin.version
When that change is merged in, this MUST follow it.
Contributed by Steve Loughran
2022-06-09 13:10:46 +01:00
98ca6fa10a
YARN-11159. Support failApplicationAttempt, updateApplicationPriority, updateApplicationTimeouts API's for Federation ( #4396 )
2022-06-08 17:34:43 -07:00
1cc2416f68
YARN-10487. Support getQueueUserAcls, listReservations, getApplicationAttempts, getContainerReport, getContainers, getResourceTypeInfo API's for Federation ( #4341 )
2022-06-02 12:54:29 -07:00
7c66266b4e
YARN-11142.Remove unused Imports in Hadoop YARN project ( #4381 ). Contributed by Ashutosh Gupta.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-05-31 23:17:11 +05:30
3228142f53
Make upstream aware of 2.10.2 release.
...
(cherry picked from commit 3dcb6367ed
2022-06-01 00:53:01 +09:00
e3e9369c1d
YARN-11128. Fix comments in TestProportionalCapacityPreemptionPolicy* ( #4271 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: Hemanth Boyina <hemanthboyina@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-05-26 15:51:55 +09:00
678768035d
YARN-11137. Improve log message in FederationClientInterceptor ( #4336 )
2022-05-24 17:21:04 -07:00
f390edaec4
YARN-11162. Set the zk acl for nodes created by ZKConfigurationStore. ( #4350 )
2022-05-24 14:07:19 +09:00
8dd3ef1f08
YARN-10465. Support getNodeToLabels, getLabelsToNodes, getClusterNodeLabels API's for Federation ( #4317 )
2022-05-20 13:19:42 -07:00
0733e968ab
HADOOP-18224. Upgrade maven compiler plugin to 3.10.1 ( #4267 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-05-21 03:21:13 +09:00
0e6a6d1880
YARN-11152. QueueMetrics is leaking memory when creating a new queue during reinitialisation
2022-05-18 18:22:42 +02:00
54cd0174c0
YARN-11147. ResourceUsage and QueueCapacities classes provide node label iterators that are not thread safe
2022-05-18 18:16:34 +02:00
6f261ed4a2
HADOOP-18198. Release 3.3.3: release notes and jdiff files.
...
* Add the changelog and release notes
* add all jdiff XML files
* update the project pom with the new stable version
Change-Id: Iaea846c3e451bbd446b45de146845a48953d580d
2022-05-17 19:00:54 +01:00
79cc0caf9f
YARN-11133. YarnClient gets the wrong EffectiveMinCapacity value ( #4280 )
...
* Update QueueConfigurationsPBImpl.java
* Update TestPBImplRecords.java
* Update TestPBImplRecords.java
* Update TestPBImplRecords.java
* Update TestPBImplRecords.java
2022-05-16 22:10:09 +05:30
931abbd927
YARN-11092. Upgrade jquery ui to 1.13.1 ( #4260 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-05-16 23:11:17 +09:00
6896c35a8d
YARN-11122. Support getClusterNodes API in FederationClientInterceptor ( #4274 )
2022-05-15 09:16:06 -07:00
6985f9aabe
YARN-10080. Support show app id on localizer thread pool ( #4283 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-05-14 01:41:06 +09:00
d2c9eb653a
YARN-11073. Avoid unnecessary preemption for tiny queues under certain corner cases ( #4110 )
...
Co-authored-by: Jian Chen <jian.chen@airbnb.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-05-14 01:11:42 +09:00
eced5bec68
YARN-11123. ResourceManager webapps test failures due to org.apache.hadoop.metrics2.MetricsException and subsequent java.net.BindException: Address already in use. Contributed by Szilard Nemeth
2022-05-13 13:16:12 +02:00
5e2f4339fa
YARN-11126. ZKConfigurationStore Java deserialisation vulnerability. Contributed by Tamas Domok
2022-05-12 13:42:06 +02:00
9af3eabdca
YARN-11114. RMWebServices returns only apps matching exactly the submitted queue name. Contributed by Szilard Nemeth
2022-05-11 18:01:31 +02:00
eeedc0c7e4
YARN-10850. TimelineService v2 lists containers for all attempts when filtering for one. Contributed by Benjamin Teke
2022-05-11 14:39:42 +02:00
b62d6ce6fd
YARN-11141. Capacity Scheduler does not support ambiguous queue names when moving application across queues. Contributed by Andras Gyori
2022-05-11 14:29:05 +02:00
09a69f996b
YARN-11130. removed unused import ( #4276 )
2022-05-11 12:14:52 +05:30
ae47846a5b
HADOOP-16515. Update the link to compatibility guide ( #4226 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-05-08 07:38:32 +09:00
d4a91bd0c0
YARN-11116. Migrate Times util from SimpleDateFormat to thread-safe DateTimeFormatter class ( #4242 )
...
Co-authored-by: Jonathan Eagles <jeagles@verizonmedia.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-05-03 03:14:26 +09:00
a74acc755e
YARN-10187. Removing hadoop-yarn-project/hadoop-yarn/README as it is no longer maintained. ( #4222 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-05-03 03:08:30 +09:00
63187083cc
HADOOP-15983. Use jersey-json that is built to use jackson2 ( #3988 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-04-28 14:18:19 +09:00
006767eb94
YARN-10303. Fixing rest api example, status code error and other typos in ResourceManagerRest.md ( #4221 )
2022-04-26 09:51:11 +08:00
6999acf520
HADOOP-16202. Enhanced openFile(): mapreduce and YARN changes. (#2584/2)
...
These changes ensure that sequential files are opened with the
right read policy, and split start/end is passed in.
As well as offering opportunities for filesystem clients to
choose fetch/cache/seek policies, the settings ensure that
processing text files on an s3 bucket where the default policy
is "random" will still be processed efficiently.
This commit depends on the associated hadoop-common patch,
which must be committed first.
Contributed by Steve Loughran.
Change-Id: Ic6713fd752441cf42ebe8739d05c2293a5db9f94
2022-04-24 17:33:05 +01:00
b35b744670
YARN-11111. Recovery failure when node-label configure-type transit from delegated-centralized to centralized ( #4200 )
2022-04-21 22:37:28 -04:00
d5cba5c3d1
YARN-11107. Addendum. When NodeLabel is enabled for a YARN cluster, AM blacklist program does not work properly. ( #4175 )
2022-04-15 17:50:49 +09:00
3b46aae977
YARN-11107. When NodeLabel is enabled for a YARN cluster, AM blacklist program does not work properly. Contributed by zhangxiping1
2022-04-13 15:57:18 +02:00
e044a46f97
YARN-11088. Introduce the config to control the AM allocated to non-exclusive nodes. Contributed by Junfan Zhang
2022-03-31 12:39:56 +02:00
6e00a799e7
YARN-11106. Fix the test failure due to missing conf of yarn.resourcemanager.node-labels.am.default-node-label-expression. Contributed by Junfan Zhang
2022-03-30 19:43:23 +02:00
61e809b245
HADOOP-13386. Upgrade Avro to 1.9.2 ( #3990 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-03-26 20:31:16 +09:00
da09d68056
YARN-11069. Dynamic Queue ACL handling in Legacy and Flexible Auto Created Queues. Contributed by Tamas Domok
2022-03-25 21:53:18 +01:00
921267ca31
YARN-11084. Introduce new config to specify AM default node-label when not specified. Contributed by Junfan Zhang.
2022-03-23 08:57:00 +01:00
2beb7296fb
YARN-11087. Introduce the config to control the refresh interval in RMDelegatedNodeLabelsUpdater. Contributed by Junfan Zhang.
2022-03-22 13:14:43 +01:00
c3124a3f6e
YARN-10565. Refactor CS queue initialization to simplify weight mode calculation. Contributed by Benjamin Teke.
2022-03-21 14:55:49 +01:00
e2701e227f
YARN-11086. Add space in debug log of ParentQueue. Contributed by Junfan Zhang.
2022-03-21 14:37:10 +01:00
4537b34e1c
YARN-11089. Fix typo in RM audit log. Contributed by Junfan Zhang.
2022-03-21 14:18:27 +01:00
ed65aa2324
YARN-11067. Resource overcommitment due to incorrect resource normalisation logical order. Contributed by Andras Gyori
2022-03-10 22:22:58 +01:00
481da19494
YARN-10049. FIFOOrderingPolicy Improvements. Contributed by Benjamin Teke
2022-03-10 22:15:35 +01:00
383b73417d
YARN-11036. Do not inherit from TestRMWebServicesCapacitySched. Contributed by Tamas Domok
2022-03-10 13:11:19 +01:00
9539ff108a
YARN-11052. Improve code quality in TestRMWebServicesNodeLabels. Contributed by Szilard Nemeth
2022-03-10 08:40:43 +01:00
db8ae4b654
YARN-10918. Simplify method: CapacitySchedulerQueueManager#parseQueue. Contributed by Andras Gyori
2022-03-09 19:36:09 +01:00
2ece95064b
YARN-10945. Add javadoc to all methods of AbstractCSQueue. Contributed by Andras Gyori
2022-03-09 19:29:53 +01:00
56d807175d
YARN-11081. TestYarnConfigurationFields consistently keeps failing ( #4027 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-03-08 21:32:51 +09:00
66b72406bd
HADOOP-18131. Upgrade maven enforcer plugin and relevant dependencies ( #4000 )
...
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2022-03-08 17:27:04 +09:00
851a4dc2d9
YARN-11049. MutableConfScheduler is referred as plain String instead of class name. Contributed by Szilard Nemeth
2022-03-07 09:01:18 +01:00
379baa5eb6
YARN-10947. Simplify AbstractCSQueue#initializeQueueState. Contributed by Andras Gyori
2022-03-03 16:44:12 +01:00
1c27c69f44
Revert "YARN-10947. Simplify AbstractCSQueue#initializeQueueState. Contributed by Andras Gyori"
...
This reverts commit de526e166chttps://issues.apache.org/jira/browse/YARN-10850 has been committed.
2022-03-03 16:38:56 +01:00
22fe79cee3
YARN-11076. Upgrade jQuery version in Yarn UI2. ( #4046 )
...
Change-Id: I3cb1677741df5a1978e83029443d4a2d5d7e3d7f
2022-03-03 23:54:03 +09:00
356d337d1e
YARN-11042. Fix testQueueSubmitWithACLsEnabledWithQueueMapping in TestAppManager. Contributed by Tamas Domok
2022-03-02 16:19:17 +01:00
9e475aede6
YARN-10983. Follow-up changes for YARN-10904. Contributed by Benjamin Teke
2022-03-02 11:16:24 +01:00
365375412f
YARN-10590. Consider legacy auto queue creation absolute resource template to avoid rounding errors. Contributed by Andras Gyori
2022-02-22 12:26:35 +01:00
0463498adc
YARN-10944. AbstractCSQueue: Eliminate code duplication in overloaded versions of setMaxCapacity. Contributed by Andras Gyori
2022-02-22 11:55:34 +01:00
de526e166c
YARN-10947. Simplify AbstractCSQueue#initializeQueueState. Contributed by Andras Gyori
2022-02-17 21:05:01 +01:00
7dd288ce8c
YARN-11070. Minimum resource ratio is overridden by subsequent labels. Contributed by Andras Gyori
2022-02-17 19:58:14 +01:00
ad0a1dc897
YARN-11075. Explicitly declare serialVersionUID in LogMutation class. Contributed by Benjamin Teke
2022-02-17 19:48:42 +01:00
4c05d257ba
YARN-10995. Move PendingApplicationComparator from GuaranteedOrZeroCapacityOverTimePolicy. Contributed by Benjamin Teke
2022-02-17 19:43:37 +01:00
51001fc7bf
YARN-11022. Fix the documentation for max-parallel-apps in CS. Contributed by Tamas Domok
2022-02-17 19:38:18 +01:00
48e02f7998
YARN-10580. Fix some issues in TestRMWebServicesCapacitySchedDynamicConfig. Contributed by Tamas Domok
2022-02-17 19:17:31 +01:00
bb1135c77c
YARN-10894. Follow up YARN-10237: fix the new test case in TestRMWebServicesCapacitySched. Contributed by Tamas Domok
2022-02-17 19:12:51 +01:00
d58c10c3a5
YARN-11071. AutoCreatedQueueTemplate incorrect wildcard level. Contributed by Tamas Domok
2022-02-17 19:03:44 +01:00
7fd90cdcbe
YARN-10788. TestCsiClient fails ( #3989 )
...
Create unix domain socket in java.io.tmpdir instead of
test.build.dir to avoid 'File name too long' error.
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2022-02-15 01:13:13 +09:00
ec2fd01333
YARN-10459. containerLaunchedOnNode method not need to hold scheduler… ( #3195 )
2022-02-01 09:37:40 -08:00
87abc437c7
YARN-10822. Containers going from New to Scheduled transition for kil… ( #3632 )
2022-02-01 09:35:59 -08:00
e8f767f2f4
YARN-11026. Make default AppPlacementAllocator configurable in AppSch… ( #3741 )
...
* YARN-11026. Make default AppPlacementAllocator configurable in AppSchedulingInfo
Co-authored-by: Minni Mittal <mimittal@microsoft.com>
2022-02-01 12:18:54 +05:30
9cb535caf2
YARN-10561. Upgrade node.js to 12.22.1 and yarn to 1.22.5 in YARN application catalog webapp ( #2591 )
...
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
2022-01-28 15:51:21 +09:00
02f6bad1ff
Revert "YARN-11068. Exclude transitive log4j2 dependency coming from solr 8. ( #3936 )"
...
This reverts commit 1c01944f35
2022-01-28 00:36:25 +09:00
1c01944f35
YARN-11068. Exclude transitive log4j2 dependency coming from solr 8. ( #3936 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-01-28 00:04:01 +09:00
4b26635a34
YARN-11034. Add enhanced headroom in AllocateResponse ( #3766 )
2022-01-25 09:25:18 -08:00
3ed3c74a6c
YARN-11015. Decouple queue capacity with ability to run OPPORTUNISTIC container ( #3779 )
2022-01-24 08:03:36 -08:00
dae33cf935
YARN-11065. Bump follow-redirects from 1.13.3 to 1.14.7 in hadoop-yarn-ui ( #3890 )
...
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects ) from 1.13.3 to 1.14.7.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases )
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.13.3...v1.14.7 )
---
updated-dependencies:
- dependency-name: follow-redirects
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-01-20 21:44:10 +09:00
a94e9fcbde
YARN-11055. Add missing newline in cgroups-operations.c ( #3851 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-01-17 16:20:48 +09:00
e2d620192a
HADOOP-18044. Hadoop - Upgrade to jQuery 3.6.0 ( #3791 )
...
Co-authored-by: luoyuan <luoyuan@shopee.com>
2022-01-12 11:40:32 +08:00
21fe0e63fd
YARN-10632. Make auto queue creation maximum allowed depth configurable ( #3859 )
2022-01-11 14:38:59 +08:00
3bf014d871
YARN-8859. Add audit logs for router service ( #3607 )
...
Co-authored-by: Minni Mittal <mimittal@microsoft.com>
2022-01-03 10:38:59 +05:30
4db27acf06
YARN-11053. AuxService should not use class name as default system classes ( #3816 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-12-24 11:08:04 +09:00
00e2405fbd
YARN-8234. Improve RM system metrics publisher's performance by pushing events to timeline server in batch ( #3793 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-12-23 17:14:51 +09:00
e2d6fd075d
YARN-10178: Global Scheduler async thread crash caused by 'Comparison method violates its general contract. Contributed by Andras Gyori (gandras) and Qi Zhu (zhuqi).
2021-12-21 19:05:39 +00:00
6185056f4d
YARN-11047. ResourceManager and NodeManager unable to connect to Hbase when ATSv2 is enabled ( #3802 )
2021-12-20 18:02:57 +08:00
04b6b9a87b
HADOOP-16908. Prune Jackson 1 from the codebase and restrict it's usage for future ( #3789 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-12-20 16:01:34 +09:00
07141426e0
YARN-11050 ( #3805 )
2021-12-18 11:52:03 +08:00
8d214cb785
YARN-10951. CapacityScheduler: Move all fields and initializer code that belongs to async scheduling to a new class ( #3800 ). Contributed by Szilard Nemeth
2021-12-17 00:18:14 +01:00
aec9cdb467
YARN-10963. Split TestCapacityScheduler by test categories. Contributed by Tamas Domok
2021-12-16 23:39:18 +01:00
a9a5830f31
YARN-11048. Add tests that shows how to delete config values with Mutation API ( #3799 ). Contributed by Szilard Nemeth
2021-12-16 15:53:08 +01:00
6ea1685abd
YARN-11044. Fix TestApplicationLimits.testLimitsComputation() ineffective asserts. Contributed by Benjamin Teke
2021-12-16 00:01:09 +01:00
f43ac31b44
YARN-11045. ATSv2 storage monitor fails to read from hbase cluster ( #3796 )
2021-12-15 16:47:51 +08:00
843f66f4dc
Clean up checkstyle warnings from YARN-11024/10907/10929. Contributed by Benjamin Teke
2021-12-14 22:00:46 +01:00
364d38f00d
YARN-10929. Do not use a separate config in legacy CS AQC. Contributed by Benjamin Teke
2021-12-14 14:57:08 +01:00
a5bcf4c792
YARN-10907. Minimize usages of AbstractCSQueue#csContext. Contributed by Benjamin Teke
2021-12-13 21:57:46 +01:00
898055e204
YARN-11024. Create an AbstractLeafQueue to store the common LeafQueue + AutoCreatedLeafQueue functionality. Contributed by Benjamin Teke
2021-12-13 16:12:55 +01:00
4e326f0c97
HADOOP-18039. Upgrade hbase2 version and fix TestTimelineWriterHBaseDown ( #3768 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-12-13 14:03:32 +09:00
f544ef4a51
YARN-11033. isAbsoluteResource is not correct for dynamically created queues. Contributed by Tamas Domok
2021-12-10 15:09:53 +01:00
a0d8cde133
YARN-10982. Replace all occurences of queuePath with the new QueuePath class. Contributed by Tibor Kovacs
2021-12-09 17:51:44 +01:00
9uapaw
Steve Loughran
slfan1989
Ashutosh Gupta
Masatake Iwasaki
Owen O'Malley
Viraj Jasani
Zilong Zhu
Jian Chen
Szilard Nemeth
Szilard Nemeth
jteagles
PJ Fanning
Junfan Zhang
章锡平
Tamas Domok
Akira Ajisaka
minni31
Akira Ajisaka
Wei-Chiu Chuang
Andrew Chung
dependabot[bot]
Gera Shegalov
luoyuan3471
Cheng Pan
Eric Payne
Szilard Nemeth