Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,447 Commits 326 Branches 371 Tags
Commit Graph

15447 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jim Brennan
99c34a3576 MAPREDUCE-7325. Intermediate data encryption is broken in LocalJobRunner. Contributed by Ahmed Hussein 
(cherry picked from commit ede490d13182f0fd3831afe6db97e01917add5b4)
 2021-03-22 20:31:56 +00:00
Jim Brennan
b3532a7c22 MAPREDUCE-7322. revisiting TestMRIntermediateDataEncryption. Contributed by Ahmed Hussein. 2021-03-22 20:30:19 +00:00
kwangsun
e6b206b33d
HADOOP-17952. Fix the wrong CIDR range example in Proxy User documentation. (#2780) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c8d327a4f1a7f15d6be35051414199d1d3fdc5ef)
 2021-03-22 11:47:03 +09:00
touchida
7183e269ac
HDFS-13975. TestBalancer#testMaxIterationTime fails sporadically (#2726) 
Co-authored-by: toshihiko.uchida <toshihiko.uchida@linecorp.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 25390babc74ab8a6bbc53305b7a49df2437406f2)
 2021-03-17 16:34:52 +09:00
Viraj Jasani
3d0a0ed92b HDFS-15895 : Remove redundant String#format in DFSAdmin#printOpenFiles (#2772) 
(cherry picked from commit 9c43b60348bea7f89f7c0fc86f2813fdf2d1929f)
 2021-03-17 15:16:18 +09:00
Wangda Tan
c6033eeeef YARN-1151. Ability to configure auxiliary services from HDFS-based JAR files. (Xuan Gong via wangda) 
Change-Id: Ied37ff11e507fc86847753ba79486652c8fadfe9
(cherry picked from commit c962371430b8957500397fe71e944c953f46ed7d)
 2021-03-16 11:03:56 -07:00
Eric Payne
d53ca0b887 YARN-10588. Percentage of queue and cluster is zero in WebUI . Contributed by Bilwa S T 
(cherry picked from commit aa4c17b9d7af122163789a731ced05f740562e45)
 2021-03-15 20:12:58 +00:00
Ayush Saxena
f0bde198f0 HADOOP-17586. Upgrade org.codehaus.woodstox:stax2-api to 4.2.1. (#2769). Contributed by Ayush Saxena. 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 Conflicts:
	LICENSE-binary
 2021-03-13 15:25:04 +05:30
Viraj Jasani
3bb40d2a56
HADOOP-17571 : Bump up woodstox-core to 5.3.0 due to security concerns (#2757) 
Contributed by Viraj Jasani.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-03-11 12:48:14 -08:00
Akira Ajisaka
8bd58c7391
HADOOP-17572. [branch-2.10] Docker image build fails due to the removal of openjdk-7-jdk package (#2755) 
Reviewed-by: Steve Loughran <stevel@apache.org>
 2021-03-11 21:56:19 +09:00
Akira Ajisaka
82b3cafc1c
HADOOP-17557. skip-dir option is not processed by Yetus. Contributed by Ahmed Hussein. 
(cherry picked from commit 63aa142b770eda31e83ca6c6e5e96cb802b8b6a9)
 2021-03-09 11:11:48 +09:00
tomscut
68cccba893 HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time. (#2668) Contributed by tomscut. 
(cherry picked from commit 9cb51bf106802c78b1400fba9f1d1c7e772dd5e7)
 2021-03-06 14:22:51 -08:00
Akira Ajisaka
9c3c753525
HADOOP-17546. Update Description of hadoop-http-auth-signature-secret in HttpAuthentication.md. Contributed by Ravuri Sushma sree. 
(cherry picked from commit 9fd2198daa58610c04bdc1ac7570476accdc25b7)
 2021-03-04 14:58:01 +09:00
Eric Badger
0b3b68cfde [MAPREDUCE-7234] ClientHSSecurityInfo class is in wrong META-INF file. 
Contributed by Eric Badger.

(cherry picked from commit a17dc4e8218b77e7fb5b5e253cb2d20014073b49)
 2021-03-02 21:50:41 +00:00
Arpit Agarwal
9f713bb825 HDFS-13677. Dynamic refresh Disk configuration results in overwriting VolumeMap. Contributed by xuzq and Stephen O'Donnell. 
(cherry picked from commit 102c8fca10f3c626ab8bc47f818c8391a5c35289)
(cherry picked from commit 4a1d51dea2149e8f458467467798e81b126b7cc5)
 2021-03-02 20:31:33 +00:00
Konstantin V Shvachko
cdd3982db4 HDFS-15849. ExpiredHeartbeats metric should be of Type.COUNTER. Contributed by Qi Zhu. 
(cherry picked from commit 9501c698f4789188f744f3a1fba7f1a9bb9b8aa4)
 2021-03-01 17:30:40 -08:00
Jim Brennan
f31c122ec9 MAPREDUCE-7320. organize test directories for ClusterMapReduceTestCase (#2722). Contributed by Ahmed Hussein 
(cherry picked from commit 4d323cd47c0787c1bb7592ed56acd4aced3e11e7)
 2021-02-26 21:51:47 +00:00
Jonathan Hung
1d76a8e73f YARN-10651. CapacityScheduler crashed with NPE in AbstractYarnScheduler.updateNodeResource(). Contributed by Haibo Chen 
(cherry picked from commit f348ab3f2f468751af329a1ffce4917cb000fcbf)
(cherry picked from commit be6e99963ded94adf6f447ff53f2ba66b99120ca)
(cherry picked from commit 6863a5bb8ace591de3374102920bba916dbebfda)
(cherry picked from commit eb6c08e423dd06bf37ff44665ffb98c97e26ad08)
 2021-02-25 15:47:36 -08:00
Jim Brennan
4ed7b80b19 [YARN-10613] Config to allow Intra- and Inter-queue preemption to enable/disable conservativeDRF. Contributed by Eric Payne 2021-02-25 20:30:42 +00:00
S O'Donnell
628ccf81ef HDFS-15422. Reported IBR is partially replaced with stored info when queuing. Contributed by Stephen O'Donnell and Kihwal Lee 2021-02-24 14:47:36 +00:00
Renukaprasad C
7c83923192 HADOOP-17510. Hadoop prints sensitive Cookie information. (#2673) 
(cherry picked from commit 924fa87010ac193a8242d59b14f9fa9d17957fe5)
(cherry picked from commit 9951ea9fa8427c134861f047acc164b5cee4eef0)
 2021-02-24 17:34:45 +08:00
Xieming Li
ee40e98d10
HADOOP-16052. Remove Subversion and Forrest from Dockerfile 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit bdbca0e53b4e0e728c71bc51f99e035d9081c7b1)
 2021-02-22 23:37:19 +09:00
Akira Ajisaka
f7ed17e636
MAPREDUCE-7323. Remove job_history_summary.py. (#2712) 
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 8bca105655d91d949f8aa8daae656f34ed950e78)
 2021-02-22 23:17:12 +09:00
Kengo Seki
5f969ced67
HADOOP-16764. Rewrite Python example codes using Python3 (#1762) 
(cherry picked from commit fd7de2b82a6b9d600b43909a0770a6b0afea899a)
 2021-02-20 19:16:30 +09:00
Jim Brennan
7363e6eee1 [YARN-10626] Log resource allocation in NM log at container start time. Contributed by Eric Badger 
(cherry picked from commit e6f5dbbe7f556e3525293814ad2239e4f04d88ad)
 2021-02-16 17:30:08 +00:00
Jim Brennan
d0562d6cd0 YARN-10500. TestDelegationTokenRenewer fails intermittently. (#2619) Contributed by Masatake Iwasaki 2021-02-11 22:45:08 +00:00
Akira Ajisaka
a5a6c973af
HADOOP-17516. Upgrade ant to 1.10.9. (#2683) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 933fbeffee1e9e2b9d78b5be31466d4d640768c5)
 2021-02-11 08:45:42 +09:00
Eric Badger
585d285e1d MAPREDUCE-7319. Log list of mappers at trace level in ShuffleHandler audit log. 
Contributed by Jim Brennan.

(cherry picked from commit 462561654bb37dc1d453bfe15c3e75c274bbfb02)
 2021-02-09 19:29:22 +00:00
yzhangal
bac234b0a8
HADOOP-17338 Intermittent S3AInputStream failures: Premature end of Content-Length delimited message body etc (#2692)) 
Backported from commit ID 3d2193cd648

Contributed by Yongjun Zhang
 2021-02-09 12:04:14 +00:00
Eric Badger
7b4034cd88 YARN-6977. Node information is not provided for non am containers in RM logs. (Suma Shivaprasad via wangda) 
Change-Id: I0c44d09a560446dee2ba68c2b9ae69fce0ec1d3e
(cherry picked from commit 8a42e922fad613f3cf1cc6cb0f3fa72546a9cc56)
(cherry picked from commit 958e8c0e257216c82f68fee726e5280a919da94a)
 2021-02-08 20:04:56 +00:00
yzhangal
ceea43115b
HADOOP-16767 Handle non-IO exceptions in reopen() (#2685) 
Contributed by Sergei Poganshev.

Catches Exception instead of IOException in closeStream() 
and so handle exceptions such as SdkClientException by 
aborting the wrapped stream. This will increase resilience
to failures, as any which occuring during stream closure
will be caught. Furthermore, because the
underlying HTTP connection is aborted, rather than closed,
it will not be recycled to cause problems on subsequent
operations.
 2021-02-08 16:03:43 +00:00
He Xiaoqiao
ef14e16d36
HDFS-15792. ClasscastException while loading FSImage. Contributed by Renukaprasad C. 2021-02-07 15:50:10 +08:00
Akira Ajisaka
961bd01b17
YARN-10036. Install yarnpkg and upgrade nodejs in Dockerfile (#1772) 
(cherry picked from commit bc825b9628f2fca9d250f91728d4aebb155f1331)
 2021-02-04 14:47:02 +09:00
Symious
5dd6db1856
HDFS-15801. Backport HDFS-14582 to branch-2.10 (Failed to start DN with ArithmeticException when NULL checksum used) (#2659) 
Co-authored-by: Symious <yiyang0203@gmail.com>
 2021-02-01 15:43:44 -08:00
Akira Ajisaka
dded428e9a
HADOOP-17354. Move Jenkinsfile outside of the root directory. (#2647) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 84b154ebc054843234e9a3c4c27748f87582807f)
 2021-02-01 18:54:06 +09:00
He Xiaoqiao
1ded537146
HADOOP-17501. Fix logging typo in ShutdownHookManager. Contributed by Fengnan Li. 2021-01-31 20:01:43 +08:00
He Xiaoqiao
b30e9198bb
HADOOP-16947. Stale record should be remove when MutableRollingAverages generating aggregate data. Contributed by Felix N, Haibin Huang. 2021-01-30 12:11:44 +08:00
Jim Brennan
c1a3e8158d [HDFS-10498] Intermittent test failure TestSnapshotFileLength.testSnapshotFileLength. Contributed by Jim Brennan. 
(cherry picked from commit 6184ab07f670c181d1fda99e76729e920fff287e)
 2021-01-28 17:46:06 +00:00
Akira Ajisaka
130d5af258
HADOOP-17478. Improve the description of hadoop.http.authentication.signature.secret.file (#2628) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit d82009599a2e9f48050e0c41440b36c759ec068f)
 2021-01-24 22:59:54 +09:00
Takanobu Asanuma
3dcb06a1ba
HADOOP-16841. The description of hadoop.http.authentication.signature.secret.file contains outdated information. Contributed by Xieming Li. 
(cherry picked from commit 5cbc4c54611f062690c3bfcf78d09d37c78ffd12)
 2021-01-24 22:59:46 +09:00
Anton Kutuzov
0ed997abab HDFS-15632. AbstractContractDeleteTest should set recursive peremeter to true for recursive test cases. Contributed by Anton Kutuzov. 
(cherry picked from commit 91d4ba57c5b85379303ac8fb2a1a03ba10b07d4e)
 2021-01-22 18:19:51 -08:00
Wei-Chiu Chuang
e48b91c83c HDFS-13723. Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh. Contributed by Siyao Meng. 
(cherry picked from commit 162228e8db937d4bdb9cf61d15ed555f1c96368f)
 2021-01-21 17:48:14 +00:00
Yiqun Lin
1b046fb26f HADOOP-13965. Groups should be consistent in using default group mapping class. Contributed by Yiqun Lin. 
(cherry picked from commit 383aa9c9f55983cdb1c225b2ed22f6f48616cb9c)
 2021-01-21 17:33:54 +00:00
Eric Badger
ea3c3d9022 YARN-10562. Follow up changes for YARN-9833. Contributed by Jim Brennan. 
(cherry picked from commit 768e2f42ba3f82802e2ebf39d44fe6c30037583c)
 2021-01-15 19:53:02 +00:00
Eric Badger
b80ff6ea57 YARN-9833. Race condition when DirectoryCollection.checkDirs() runs during container launch. Contributed by Peter Bacsko. 
(cherry picked from commit c474e24c0b73c0f52a7d5af2495355f4a0799344)
 2021-01-15 19:18:58 +00:00
Akira Ajisaka
db5b804238
HDFS-15717. Improve fsck logging. Contributed by Kihwal Lee and Ahmed Hussein (#2620) 
(cherry picked from commit be35fa186cc0c5c1f90028a9383851d47791bf04)
 2021-01-15 10:28:11 +09:00
Ahmed Hussein
4ddd41bd47
HADOOP-17438. Increase docker memory limit in Jenkins (#2560) 
Reviewed-by: Eric Badger <ebadger@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ae894ecfcc46e40d9e2f535ac6acec50f8136fae)
 2021-01-12 15:37:21 +09:00
He Xiaoqiao
1efadde067
HDFS-15751. Add documentation for msync() API to filesystem.md. Contributed by Konstantin V Shvachko. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Chao Sun <sunchao@apache.org>
 2021-01-03 16:38:47 +08:00
He Xiaoqiao
45774daee8
HADOOP-17445. Update the year to 2021. Contributed by He Xiaoqiao. 2020-12-24 20:51:01 +08:00
David Mollitor
16fd00d62d HADOOP-16208. Do Not Log InterruptedException in Client. 
Contributed by David Mollitor.

(cherry picked from commit c90736350ba158c7872a39426e7a29c5e5e0bb48)
 2020-12-22 10:19:44 -08:00