3c37a01654
YARN-8262. get_executable in container-executor should provide meaningful error codes. Contributed by Susheel Gupta
2022-11-22 13:37:55 +01:00
8f971b0e54
HDFS-16547. [SBN read] Namenode in safe mode should not be transfer to observer state ( #4201 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
Reviewed-by: Zengqiang Xu <xuzq_zander@163.com>
2022-11-21 10:14:07 -08:00
dc2fba45fe
HDFS-16832. [SBN READ] Follow-on to HDFS-16732. Fix NPE when check the block location of empty directory ( #5099 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
Reviewed-by: Zengqiang Xu <xuzq_zander@163.com>
2022-11-21 08:26:16 -08:00
069bd973d8
HADOOP-18532. Update command usage in FileSystemShell.md ( #5141 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-11-21 15:55:46 +09:00
696d042054
HADOOP-8728. Display (fs -text) shouldn't hard-depend on Writable serialized sequence files. ( #5010 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-11-21 14:54:50 +09:00
2e993fdf4e
YARN-6946. Upgrade JUnit from 4 to 5 in hadoop-yarn-common ( #4717 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-11-21 14:40:02 +09:00
dcde414570
MAPREDUCE-7422. Upgrade Junit 4 to 5 in hadoop-mapreduce-examples ( #5029 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-11-21 14:36:35 +09:00
c71a68ca80
HDFS-16844: Adds resilancy when StateStore gets exceptions. ( #5138 )
...
Allows the StateStore to stay up when there are errors reading the data.
2022-11-18 17:24:08 +00:00
1ea5db52dd
HADOOP-18324. Interrupting RPC Client calls can lead to thread exhaustion. ( #4527 )
...
* Exactly 1 sending thread per an RPC connection.
* If the calling thread is interrupted before the socket write, it will be skipped instead of sending it anyways.
* If the calling thread is interrupted during the socket write, the write will finish.
* RPC requests will be written to the socket in the order received.
* Sending thread is only started by the receiving thread.
* The sending thread periodically checks the shouldCloseConnection flag.
2022-11-18 16:24:45 +00:00
7d39abd799
HADOOP-18429. fix infinite loop in MutableGaugeFloat#incr(float) ( #4823 )
2022-11-17 17:50:39 +08:00
eccd2d0492
YARN-11359. [Federation] Routing admin invocations transparently to multiple RMs. ( #5057 )
2022-11-16 18:00:38 -08:00
142df247ed
YARN-11369. Commons.compress throws an IllegalArgumentException with large uids after 1.21. Contributed by Benjamin Teke
2022-11-16 13:07:05 +01:00
cd929457c9
HDFS-16836: StandbyCheckpointer shouldn't trigger rollback fs image after RU is finalized ( #5135 )
...
Co-authored-by: Lei Yang <leyang@linkedin.com>
2022-11-15 23:06:37 +00:00
69e50c7b44
HADOOP-18528. Disable abfs prefetching by default ( #5134 )
...
Disables block prefetching on ABFS InputStreams, by setting
fs.azure.enable.readahead to false in core-default.xml and
the matching java constant.
This prevents
HADOOP-18521. ABFS ReadBufferManager buffer sharing across concurrent HTTP requests.
Once a fix for that is committed, this change can be reverted.
Contributed by Mehakmeet Singh.
2022-11-15 14:28:41 +00:00
a48e8c9beb
MAPREDUCE-5608. Replace and deprecate mapred.tasktracker.indexcache.mb ( #5014 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-11-14 11:07:40 +09:00
04b31d7ecf
MAPREDUCE-7390. Remove WhiteBox in mapreduce module. ( #4462 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-11-14 10:45:20 +09:00
d3c1c453f0
HDFS-16785. Avoid to hold write lock to improve performance when add volume. ( #4945 ). Contributed by ZanderXu.
...
Signed-off-by: Tao Li <tomscut@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-11-13 22:01:35 +08:00
22c9f28f4d
YARN-10005. Code improvements in MutableCSConfigurationProvider. Contributed by Peter Szucs
2022-11-12 18:35:49 +01:00
d340c4a7a1
HADOOP-18496. Upgrade okhttp3 and dependencies due to kotlin CVEs ( #5035 )
...
Updates okhttp3 and okio so their transitive dependency on Kotlin
stdlib is free from recent CVEs.
okhttp3:okhttp => 4.10.0
okio:okio => 3.2.0
kotlin stdlib => 1.6.20
kotlin CVEs fixed:
CVE-2022-24329
CVE-2020-29582
Contributed by PJ Fanning.
2022-11-12 14:14:19 +00:00
5bb11cecea
HADOOP-15327. Upgrade MR ShuffleHandler to use Netty4 #3259 . Contributed by Szilard Nemeth.
2022-11-11 09:05:01 +01:00
552ee44eba
HDFS-16834: Removes request stateID consistency constraint between clients in different connection pools. ( #5121 )
...
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Zander Xu <zanderxu@apache.org>
2022-11-11 15:26:31 +08:00
b398a7b003
YARN-11367. [Federation] Fix DefaultRequestInterceptorREST Client NPE. ( #5100 )
2022-11-09 10:25:10 -08:00
f68f1a4578
HADOOP-18433. Fix main thread name for . ( #4838 )
2022-11-09 19:18:31 +08:00
7002e214b8
HADOOP-18502. MutableStat should return 0 when there is no change ( #5058 )
2022-11-09 10:21:43 +08:00
7f9ca101e2
HADOOP-18517. ABFS: Add fs.azure.enable.readahead option to disable readahead ( #5103 )
...
* HADOOP-18517. ABFS: Add fs.azure.enable.readahead option to disable readahead
Adds new config option to turn off readahead
* also allows it to be passed in through openFile(),
* extends ITestAbfsReadWriteAndSeek to use the option, including one
replicated test...that shows that turning it off is slower.
Important: this does not address the critical data corruption issue
HADOOP-18521. ABFS ReadBufferManager buffer sharing across concurrent HTTP requests
What is does do is provide a way to completely bypass the ReadBufferManager.
To mitigate the problem, either fs.azure.enable.readahead needs to be set to false,
or set "fs.azure.readaheadqueue.depth" to 0 -this still goes near the (broken)
ReadBufferManager code, but does't trigger the bug.
For safe reading of files through the ABFS connector, readahead MUST be disabled
or the followup fix to HADOOP-18521 applied
Contributed by Steve Loughran
2022-11-08 11:43:04 +00:00
845cf8bc28
YARN-11368. [Federation] Improve Yarn Router's Federation Page style. ( #5105 )
2022-11-07 10:13:23 -08:00
44b8bb7224
HDFS-16821: Fixes regression in HDFS-13522 that enables observer reads by default ( #5078 )
2022-11-07 08:43:04 -08:00
660530205e
HDFS-16833. NameNode should log internal EC blocks instead of the EC block group when it receives block reports. ( #5106 )
...
Reviewed-by: Tao Li <tomscut@apache.org>
2022-11-07 10:49:48 +09:00
5d6ab15860
YARN-11354. [Federation] Add Yarn Router's NodeLabel Web Page. ( #5073 )
2022-11-04 14:39:57 -07:00
2ba982a061
MAPREDUCE-7386. Maven parallel builds (skipping tests) fail ( #4415 )
...
Contributed by Steve Vaughan Jr
2022-11-04 11:50:43 +00:00
e9319e696c
HDFS-16811. Support DecommissionBackoffMonitor Parameters reconfigurablereconfigurable ( #5068 )
...
Signed-off-by: Tao Li <tomscut@apache.org>
2022-11-04 14:18:59 +08:00
b90dfdff3f
YARN-11366. Improve equals, hashCode(), toString() methods of the Federation Base Object. ( #5096 )
2022-11-03 21:33:53 -07:00
e62ba16a02
HADOOP-18484. Upgrade hsqldb to v2.7.1 to mitigate CVE-2022-41853 ( #4991 )
2022-11-02 08:41:27 +01:00
83acb55981
YARN-11364. Docker Container to accept docker Image name with sha256 digest ( #5092 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: slfan1989 <55643692+slfan1989@users.noreply.github.com>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2022-11-01 14:44:35 -07:00
69225ae5b9
YARN-11363. Remove unused TimelineVersionWatcher and TimelineVersion from hadoop-yarn-server-tests ( #5091 )
...
Reviewed-by: slfan1989 <55643692+slfan1989@users.noreply.github.com>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2022-11-01 14:02:06 -07:00
388f2f182f
document fix for MAPREDUCE-7425 ( #5090 )
...
Reviewed-by: Ashutosh Gupta <ashutosh.gupta@st.niituniversity.in>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2022-11-01 13:34:59 -07:00
7ba304d1c6
HADOOP-18512: upgrade woodstox-core to 5.4.0 for security fix ( #5087 ). Contributed by PJ Fanning.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-11-02 00:11:41 +05:30
0b577992ef
HADOOP-18482. ITestS3APrefetchingInputStream to skip if CSV test file unavailable ( #4983 )
...
Contributed by Danny Jones
2022-10-31 21:19:34 +00:00
3b10cb5a3b
HADOOP-18507. VectorIO FileRange type to support a "reference" field ( #5076 )
...
Contributed by Steve Loughran
2022-10-31 21:12:13 +00:00
b1f418f802
YARN-11365. Fix NM class not found on Windows ( #5093 )
2022-10-31 09:19:07 -07:00
af7dd660e0
HADOOP-18233. Possible race condition with TemporaryAWSCredentialsProvider ( #5024 )
...
This fixes a race condition with the TemporaryAWSCredentialProvider
one which has existed for a long time but which only surfaced
(usually in Spark) when the bucket existence probe was disabled
by setting fs.s3a.bucket.probe to 0 -a performance speedup
which was made the default in HADOOP-17454.
Contributed by Jimmy Wong.
2022-10-31 12:43:30 +00:00
cbe02c2e77
YARN-11264. Upgrade JUnit from 4 to 5 in hadoop-yarn-server-tests ( #4776 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-30 04:33:57 +09:00
e04c9e810b
MAPREDUCE-7426. Fix typo in StartEndTimeBase ( #4894 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-30 04:23:30 +09:00
c096803387
YARN-11339. Upgrade Junit 4 to 5 in hadoop-yarn-services-api ( #4995 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-30 03:07:16 +09:00
d6a65a4180
HADOOP-18472. Upgrade to snakeyaml 1.33 ( #4958 )
...
Reviewed-by: Dinesh Chitlangia <dineshc@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-30 02:30:41 +09:00
2aae7ffe08
YARN-11337. Upgrade Junit 4 to 5 in hadoop-yarn-applications-mawo ( #4993 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-30 02:25:53 +09:00
070a2d4880
YARN-11332. [Federation] Improve FederationClientInterceptor#ThreadPool thread pool configuration. ( #4982 )
2022-10-28 15:39:04 -07:00
b1cd88c598
YARN-11229. [Federation] Add checkUserAccessToQueue REST APIs for Router. ( #4929 )
2022-10-28 15:37:35 -07:00
bfb84cd7f6
YARN-11360: Add number of decommissioning/shutdown nodes to YARN cluster metrics. ( #5060 )
2022-10-28 11:07:01 -07:00
88f7f5bc01
HDFS-16802.Print options when accessing ClientProtocol#rename2(). ( #5013 )
...
Reviewed-by: Tao Li <tomscut@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Zander Xu <zanderxu@apache.org>
2022-10-28 16:31:20 +08:00
Szilard Nemeth
litao
zhengchenyu
GuoPhilipse
Ashutosh Gupta
Owen O'Malley
Hu Xinqiu
slfan1989
Lei Yang
Mehakmeet Singh
ZanderXu
PJ Fanning
Szilard Nemeth
Simbarashe Dzinamarira
ted12138
Steve Loughran
Takanobu Asanuma
Steve Vaughan
huhaiyang
wangteng13
Daniel Carl Jones
Gautham B A
sabertiger
Samrat
Chris Nauroth
jianghuazhu