5bb11cecea
HADOOP-15327. Upgrade MR ShuffleHandler to use Netty4 #3259 . Contributed by Szilard Nemeth.
2022-11-11 09:05:01 +01:00
552ee44eba
HDFS-16834: Removes request stateID consistency constraint between clients in different connection pools. ( #5121 )
...
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Zander Xu <zanderxu@apache.org>
2022-11-11 15:26:31 +08:00
b398a7b003
YARN-11367. [Federation] Fix DefaultRequestInterceptorREST Client NPE. ( #5100 )
2022-11-09 10:25:10 -08:00
f68f1a4578
HADOOP-18433. Fix main thread name for . ( #4838 )
2022-11-09 19:18:31 +08:00
7002e214b8
HADOOP-18502. MutableStat should return 0 when there is no change ( #5058 )
2022-11-09 10:21:43 +08:00
7f9ca101e2
HADOOP-18517. ABFS: Add fs.azure.enable.readahead option to disable readahead ( #5103 )
...
* HADOOP-18517. ABFS: Add fs.azure.enable.readahead option to disable readahead
Adds new config option to turn off readahead
* also allows it to be passed in through openFile(),
* extends ITestAbfsReadWriteAndSeek to use the option, including one
replicated test...that shows that turning it off is slower.
Important: this does not address the critical data corruption issue
HADOOP-18521. ABFS ReadBufferManager buffer sharing across concurrent HTTP requests
What is does do is provide a way to completely bypass the ReadBufferManager.
To mitigate the problem, either fs.azure.enable.readahead needs to be set to false,
or set "fs.azure.readaheadqueue.depth" to 0 -this still goes near the (broken)
ReadBufferManager code, but does't trigger the bug.
For safe reading of files through the ABFS connector, readahead MUST be disabled
or the followup fix to HADOOP-18521 applied
Contributed by Steve Loughran
2022-11-08 11:43:04 +00:00
845cf8bc28
YARN-11368. [Federation] Improve Yarn Router's Federation Page style. ( #5105 )
2022-11-07 10:13:23 -08:00
44b8bb7224
HDFS-16821: Fixes regression in HDFS-13522 that enables observer reads by default ( #5078 )
2022-11-07 08:43:04 -08:00
660530205e
HDFS-16833. NameNode should log internal EC blocks instead of the EC block group when it receives block reports. ( #5106 )
...
Reviewed-by: Tao Li <tomscut@apache.org>
2022-11-07 10:49:48 +09:00
5d6ab15860
YARN-11354. [Federation] Add Yarn Router's NodeLabel Web Page. ( #5073 )
2022-11-04 14:39:57 -07:00
2ba982a061
MAPREDUCE-7386. Maven parallel builds (skipping tests) fail ( #4415 )
...
Contributed by Steve Vaughan Jr
2022-11-04 11:50:43 +00:00
e9319e696c
HDFS-16811. Support DecommissionBackoffMonitor Parameters reconfigurablereconfigurable ( #5068 )
...
Signed-off-by: Tao Li <tomscut@apache.org>
2022-11-04 14:18:59 +08:00
b90dfdff3f
YARN-11366. Improve equals, hashCode(), toString() methods of the Federation Base Object. ( #5096 )
2022-11-03 21:33:53 -07:00
e62ba16a02
HADOOP-18484. Upgrade hsqldb to v2.7.1 to mitigate CVE-2022-41853 ( #4991 )
2022-11-02 08:41:27 +01:00
83acb55981
YARN-11364. Docker Container to accept docker Image name with sha256 digest ( #5092 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: slfan1989 <55643692+slfan1989@users.noreply.github.com>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2022-11-01 14:44:35 -07:00
69225ae5b9
YARN-11363. Remove unused TimelineVersionWatcher and TimelineVersion from hadoop-yarn-server-tests ( #5091 )
...
Reviewed-by: slfan1989 <55643692+slfan1989@users.noreply.github.com>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2022-11-01 14:02:06 -07:00
388f2f182f
document fix for MAPREDUCE-7425 ( #5090 )
...
Reviewed-by: Ashutosh Gupta <ashutosh.gupta@st.niituniversity.in>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2022-11-01 13:34:59 -07:00
7ba304d1c6
HADOOP-18512: upgrade woodstox-core to 5.4.0 for security fix ( #5087 ). Contributed by PJ Fanning.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-11-02 00:11:41 +05:30
0b577992ef
HADOOP-18482. ITestS3APrefetchingInputStream to skip if CSV test file unavailable ( #4983 )
...
Contributed by Danny Jones
2022-10-31 21:19:34 +00:00
3b10cb5a3b
HADOOP-18507. VectorIO FileRange type to support a "reference" field ( #5076 )
...
Contributed by Steve Loughran
2022-10-31 21:12:13 +00:00
b1f418f802
YARN-11365. Fix NM class not found on Windows ( #5093 )
2022-10-31 09:19:07 -07:00
af7dd660e0
HADOOP-18233. Possible race condition with TemporaryAWSCredentialsProvider ( #5024 )
...
This fixes a race condition with the TemporaryAWSCredentialProvider
one which has existed for a long time but which only surfaced
(usually in Spark) when the bucket existence probe was disabled
by setting fs.s3a.bucket.probe to 0 -a performance speedup
which was made the default in HADOOP-17454.
Contributed by Jimmy Wong.
2022-10-31 12:43:30 +00:00
cbe02c2e77
YARN-11264. Upgrade JUnit from 4 to 5 in hadoop-yarn-server-tests ( #4776 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-30 04:33:57 +09:00
e04c9e810b
MAPREDUCE-7426. Fix typo in StartEndTimeBase ( #4894 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-30 04:23:30 +09:00
c096803387
YARN-11339. Upgrade Junit 4 to 5 in hadoop-yarn-services-api ( #4995 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-30 03:07:16 +09:00
d6a65a4180
HADOOP-18472. Upgrade to snakeyaml 1.33 ( #4958 )
...
Reviewed-by: Dinesh Chitlangia <dineshc@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-30 02:30:41 +09:00
2aae7ffe08
YARN-11337. Upgrade Junit 4 to 5 in hadoop-yarn-applications-mawo ( #4993 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-30 02:25:53 +09:00
070a2d4880
YARN-11332. [Federation] Improve FederationClientInterceptor#ThreadPool thread pool configuration. ( #4982 )
2022-10-28 15:39:04 -07:00
b1cd88c598
YARN-11229. [Federation] Add checkUserAccessToQueue REST APIs for Router. ( #4929 )
2022-10-28 15:37:35 -07:00
bfb84cd7f6
YARN-11360: Add number of decommissioning/shutdown nodes to YARN cluster metrics. ( #5060 )
2022-10-28 11:07:01 -07:00
88f7f5bc01
HDFS-16802.Print options when accessing ClientProtocol#rename2(). ( #5013 )
...
Reviewed-by: Tao Li <tomscut@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Zander Xu <zanderxu@apache.org>
2022-10-28 16:31:20 +08:00
8396caa484
HDFS-16716. Improve appendToFile command: support appending on file with new block ( #4697 )
...
Reviewed-by: xuzq <15040255127@163.com>
Signed-off-by: Tao Li <tomscut@apache.org>
2022-10-27 19:03:15 +08:00
d26c35b228
HDFS-16817. Remove useless DataNode lock related configuration ( #5072 )
...
Reviewed-by: litao <tomleescut@gmail.com>
Signed-off-by: ZanderXu <zanderxu@apache.org>
2022-10-27 17:28:19 +08:00
545a556883
HDFS-16822. HostRestrictingAuthorizationFilter should pass through requests if they don't access WebHDFS API. ( #5079 )
...
Reviewed-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: Tao Li <tomscut@apache.org>
2022-10-27 14:39:01 +09:00
ba77530ff4
YARN-11357. Fix FederationClientInterceptor#submitApplication Can't Update SubClusterId ( #5055 )
2022-10-26 16:42:22 -07:00
562b693374
YARN-11356. Upgrade DataTables to 1.11.5 to fix CVEs. Contributed by Bence Kosztolnik.
2022-10-26 22:29:01 +02:00
fba46aa5bb
HADOOP-18499. S3A to support HTTPS web proxies ( #5051 )
...
The option "fs.s3a.proxy.ssl.enabled" controls
whether the s3a connects to a proxy over HTTP (default) or HTTPS.
Set to "true" to use HTTPS.
Contributed by Mehakmeet Singh
2022-10-26 11:45:20 +01:00
37bff63c0f
Refactor CallerContext's constructor to eliminate duplicate code ( #5070 )
...
Reviewed-by: Tao Li <tomscut@apache.org>
Reviewed-by: Zander Xu <zanderxu@apache.org>
2022-10-26 06:40:31 +08:00
f140506d67
HADOOP-18504. Fixed an unhandled NullPointerException in class KeyProvider ( #5064 )
...
Contributed by FuzzingTeam
2022-10-25 18:07:49 +01:00
0a26d84df1
HADOOP-9946. NumAllSinks metrics shows lower value than NumActiveSinks ( #5002 )
...
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-25 17:22:25 +08:00
e6edbf1b4b
YARN-11338. Upgrade Junit 4 to 5 in hadoop-yarn-applications-unmanaged-am-launcher ( #4994 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-25 17:21:12 +09:00
21b7790866
YARN-11336. Upgrade Junit 4 to 5 in hadoop-yarn-applications-catalog-webapp ( #4992 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-25 16:52:31 +09:00
454157a384
YARN-11345. [Federation] Refactoring Yarn Router's Application Web Page. ( #5030 )
2022-10-24 09:35:16 -07:00
833750f72a
HADOOP-18506. Update build instructions for Windows using VS2019 ( #5066 )
2022-10-24 09:28:29 -07:00
aac87ffe76
MAPREDUCE-7411: use secure XML parsers in mapreduce modules ( #4980 )
...
Lockdown of parsers in hadoop-mapreduce.
Follow-on to HADOOP-18469. Add secure XML parser factories to XMLUtils
Contributed by P J Fanning
2022-10-21 14:02:11 +01:00
d93e6f0cbb
YARN-11295. [Federation] Router Support DelegationToken in MemoryStore mode. ( #5032 )
2022-10-20 13:14:38 -07:00
c4aa41aa80
HADOOP-18500. Upgrade maven-shade-plugin to 3.3.0 ( #5045 )
...
Contributed by Willi Raschkowski
2022-10-20 18:47:33 +01:00
7f69e09290
HADOOP-18471. Fixed ArrayIndexOutOfBoundsException in DefaultStringifier ( #4957 )
...
Contributed by FuzzingTeam
2022-10-20 18:12:17 +01:00
a996d889ec
HADOOP-17767. ABFS: Update test scripts ( #3124 )
...
Contributed by Sneha Vijayarajan
2022-10-20 18:07:04 +01:00
9adf0ca089
YARN-11342. [Federation] Refactor getNewApplication, submitApplication Use FederationActionRetry. ( #5005 )
2022-10-20 09:22:24 -07:00
Szilard Nemeth
Simbarashe Dzinamarira
slfan1989
zhengchenyu
ted12138
Steve Loughran
Takanobu Asanuma
Steve Vaughan
huhaiyang
Ashutosh Gupta
wangteng13
PJ Fanning
Daniel Carl Jones
Gautham B A
sabertiger
Samrat
Chris Nauroth
jianghuazhu
M1eyu2018
Bence Kosztolnik
Mehakmeet Singh
Wang Yu
FuzzingTeam
Willi Raschkowski
Sneha Vijayarajan