9628aa87bf
YARN-10598. CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to extend the creation type with additional information. Contributed by Benjamin Teke
2021-01-27 18:14:06 +01:00
28cc912a5c
HADOOP-17493. Revert name of DELEGATION_TOKENS_ISSUED constant/statistic ( #2649 )
...
Follow-on to HADOOP-16830/HADOOP-17271.
Contributed by Steve Loughran.
2021-01-27 16:39:29 +00:00
7c4ef42837
YARN-10599. fs2cs should generate new 'auto-queue-creation-v2.enabled' properties for all parents. Contributed by Peter Bacsko
2021-01-27 15:49:26 +01:00
80c7404b51
HADOOP-17414. Magic committer files don't have the count of bytes written collected by spark ( #2530 )
...
This needs SPARK-33739 in the matching spark branch in order to work
Contributed by Steve Loughran.
2021-01-26 19:30:51 +00:00
e2a7008d50
YARN-10585. Create a class which can convert from legacy mapping rule format to the new JSON format. Contributed by Gergely Pollak
2021-01-26 18:31:39 +01:00
4f008153ef
YARN-10587. Fix AutoCreateLeafQueueCreation cap related caculation when in absolute mode. (Qi Zhu via wangda)
...
Change-Id: I8889c393a9f08b633c72a2e873c6f45ca12cc82b
2021-01-26 08:24:29 -08:00
b7384a8d02
YARN-10531. Be able to disable user limit factor for CapacityScheduler Leaf Queue. (Qi Zhu via wangda)
...
Change-Id: I670e5525619b320745254609c48e7e1afb084835
2021-01-26 08:24:28 -08:00
f1766e5bb4
YARN-10596. Allow static definition of childless ParentQueues with auto-queue-creation-v2 enabled. Contributed by Andras Gyori
2021-01-26 16:22:35 +01:00
a9ff726e42
HADOOP-17496. Install a supported version of pip ( #2653 )
...
pip version 21 ended support for python 2.7. Thus we need to
install a version of pip that supports python 2.7.
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-01-26 23:06:38 +09:00
e40f99f6d5
HDFS-15767. RBF: Router federation rename of directory. Contributed by Jinglun.
2021-01-26 14:25:27 +05:30
b441ca8699
HADOOP-17465. Upgrade to Focal ( #2613 )
2021-01-25 21:15:43 -08:00
06a5d3437f
HADOOP-17480. Document that AWS S3 is consistent and that S3Guard is not needed ( #2636 )
...
Contributed by Steve Loughran.
2021-01-25 13:21:34 +00:00
2fa73a2387
HADOOP-17452. Upgrade Guice to 4.2.3 ( #2582 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-01-25 18:00:48 +09:00
d82009599a
HADOOP-17478. Improve the description of hadoop.http.authentication.signature.secret.file ( #2628 )
...
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
2021-01-24 22:56:47 +09:00
9cf408d6ba
HDFS-15731. Reduce threadCount for unit tests to reduce the memory usage ( #2556 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2021-01-24 22:50:05 +09:00
91d4ba57c5
HDFS-15632. AbstractContractDeleteTest should set recursive peremeter to true for recursive test cases. Contributed by Anton Kutuzov.
2021-01-22 17:55:37 -08:00
d09e3c929f
HADOOP-17224. Install Intel ISA-L library in Dockerfile. ( #2537 )
2021-01-22 10:27:06 +09:00
8620984b8d
YARN-10515. Fix flaky test TestCapacitySchedulerAutoQueueCreation.testDynamicAutoQueueCreationWithTags. Contributed by Peter Bacsko
2021-01-21 20:58:57 +01:00
1ccba3734d
YARN-10490. yarn top command not quitting completely with ctrl+c. Contributed by Agshin Kazimli
2021-01-21 20:30:19 +01:00
e2f8503ebd
HADOOP-17484. Typo in hadop-aws index.md ( #2634 )
...
Contributed by Maksim Bober.
2021-01-21 17:30:58 +00:00
17fdac8de5
YARN-10581. CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to include queue creation type for queues. Contributed by Szilard Nemeth.
2021-01-21 18:06:53 +01:00
06fef5ee43
YARN-10579. CS Flexible AQC: Modify RM /scheduler endpoint to include weight values for queues. Contributed by Szilard Nemeth
2021-01-21 09:23:11 +01:00
8bc2dfbf36
YARN-10519. Refactor QueueMetricsForCustomResources class to move to yarn-common package. Contributed by Minni Mittal
2021-01-21 11:23:30 +05:30
45b3a84709
HDFS-15783. Speed up BlockPlacementPolicyRackFaultTolerant#verifyBlockPlacement ( #2626 )
2021-01-21 10:07:46 +09:00
cfe6e1f7da
YARN-10578. Fix Auto Queue Creation parent handling. Contributed by Andras Gyori
2021-01-20 15:22:44 +01:00
4b5bc05a78
YARN-10512. CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to include mode of operation for CS. Contributed by Szilard Nemeth.
2021-01-20 12:28:59 +05:30
68bc721841
HADOOP-17433. Skipping network I/O in S3A getFileStatus(/) breaks ITestAssumeRole. ( #2600 )
...
Contributed by Steve Loughran.
2021-01-19 17:19:27 +00:00
a326f22606
YARN-10573. Enhance placement rule conversion in fs2cs in weight mode and enable it by default. Contributed by Peter Bacsko
2021-01-19 09:42:40 +01:00
6abdb148e4
YARN-10535. Make queue placement in CapacityScheduler compliant with auto-queue-placement. Contributed by Gergely Pollak
2021-01-18 20:19:36 +01:00
9b4f09a6bb
HDFS-15758. Fix typos in MutableMetric. Contributed by Haibin Huang.
2021-01-18 20:35:37 +05:30
acea862d59
YARN-10574. Fix the FindBugs warning introduced in YARN-10506. Contributed by Gergely Pollak
2021-01-18 16:04:45 +01:00
97f843de3a
HADOOP-16947. Stale record should be remove when MutableRollingAverages generating aggregate data. Contributed by Haibin Huang.
2021-01-16 14:06:56 +08:00
7743d40ac5
HDFS-15549. Use Hardlink to move replica between DISK and ARCHIVE storage if on same filesystem mount ( #2583 ). Contributed by Leon Gao.
2021-01-15 16:28:11 -08:00
3d46141583
YARN-10506. Update queue creation logic to use weight mode and allow the flexible static/dynamic creation. (Contributed by Qi Zhu, Andras Gyori)
...
Change-Id: I118862fd5e11ee6888275e2bcf667fedfa56c5d7
2021-01-15 14:20:08 -08:00
6cd540e964
YARN-7200. SLS generates a realtimetrack.json file but that file is missing the closing ']'. Contributed by Agshin Kazimli
2021-01-15 22:32:30 +01:00
630f8ddd2c
HDFS-15672. TestBalancerWithMultipleNameNodes#testBalancingBlockpoolsWithBlockPoolPolicy fails on trunk. ( #2475 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-01-14 16:19:35 +09:00
768e2f42ba
YARN-10562. Follow up changes for YARN-9833. Contributed by Jim Brennan.
2021-01-13 23:41:52 +00:00
d8fadec944
YARN-4589: Diagnostics for localization timeouts is lacking. Contributed by Chang Li (lichangleo) and Jim Brennan (Jim_Brennan)
2021-01-13 17:01:28 +00:00
836c630430
YARN-10525. Add weight mode conversion to fs2cs. Contributed by Peter Bacsko
2021-01-13 14:24:12 +01:00
61f77b7674
YARN-8529. Add timeout to RouterWebServiceUtil#invokeRMWebService. Contributed by Minni Mittal
2021-01-13 16:00:11 +05:30
8ec824f2ba
HDFS-15762. TestMultipleNNPortQOP#testMultipleNNPortOverwriteDownStream fails intermittently ( #2598 )
...
Co-authored-by: Toshihiko Uchida <toshihiko.uchida@linecorp.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-01-13 11:23:07 +09:00
724edb0354
HADOOP-17451. IOStatistics test failures in S3A code. ( #2594 )
...
Caused by HADOOP-16830 and HADOOP-17271.
Fixes tests which fail intermittently based on configs and
in the case of the HugeFile tests, bulk runs with existing
FS instances meant statistic probes sometimes ended up probing those
of a previous FS.
Contributed by Steve Loughran.
Change-Id: I65ba3f44444e59d298df25ac5c8dc5a8781dfb7d
2021-01-12 17:30:32 +00:00
05c9c2ed02
Revert "HADOOP-17451. IOStatistics test failures in S3A code. ( #2594 )"
...
This reverts commit d3014e01f3
2021-01-12 17:29:59 +00:00
d3014e01f3
HADOOP-17451. IOStatistics test failures in S3A code. ( #2594 )
...
Caused by HADOOP-16380 and HADOOP-17271.
Fixes tests which fail intermittently based on configs and
in the case of the HugeFile tests, bulk runs with existing
FS instances meant statistic probes sometimes ended up probing those
of a previous FS.
Contributed by Steve Loughran.
2021-01-12 17:25:14 +00:00
0a6ddfa145
HADOOP-17272. ABFS Streams to support IOStatistics API ( #2604 )
...
Contributed by Mehakmeet Singh.
2021-01-12 15:48:09 +00:00
0d7ac54510
HDFS-14558. RBF: Isolation/Fairness documentation. Contributed by Fengnan Li.
2021-01-12 23:38:55 +08:00
85b1c017ee
YARN-10563. Fix dependency exclusion problem in poms. Contributed by Peter Bacsko
2021-01-12 15:29:20 +01:00
9c6e093783
YARN-10570. Remove experimental warning message from fs2cs. Contributed by Peter Bacsko
2021-01-12 15:21:27 +01:00
612330661b
HADOOP-17459. ADLS Gen1: Fixes for rename contract tests #2607
...
Contributed by Bilaharith
2021-01-12 14:00:48 +00:00
ca7dd5fad3
MAPREDUCE-7310. Clear the fileMap in JHEventHandlerForSigtermTest ( #2499 )
...
Co-authored-by: Zhengxi Li <zli89@illinois.edu>
Reviewed-by: Wanqiang Ji <jiwq@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-01-12 15:45:13 +09:00
Szilard Nemeth
Steve Loughran
Wangda Tan
Gautham B A
Ayush Saxena
Yuming Wang
Akira Ajisaka
Anton Kutuzov
Takanobu Asanuma
Maksim Bober
Peter Bacsko
bibinchundatt
Sunil G
He Xiaoqiao
LeonGao
Masatake Iwasaki
Eric Badger
Eric Payne
touchida
Mehakmeet Singh
Yiqun Lin
bilaharith
lzx404243