Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,483 Commits 326 Branches 371 Tags
Commit Graph

3845 Commits

Author SHA1 Message Date
Takanobu Asanuma
cc842231df HADOOP-15293. TestLogLevel fails on Java 9 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 99ab511cbac570bea9d31a55898b95590a8e3159)

 Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/log/TestLogLevel.java
 2021-12-18 01:24:52 +09:00
Dhananjay Badaya
ebf569793b
HADOOP-13500. Synchronizing iteration of Configuration properties object (#3776) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-12-18 01:07:38 +09:00
Akira Ajisaka
7d88a54d40 HADOOP-18040. Use maven.test.failure.ignore instead of ignoreTestFailure (#3774) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 9b9e2ef87ffe7b33dc01699892acf62965fb3550)

 Conflicts:
	hadoop-tools/hadoop-federation-balance/pom.xml

(cherry picked from commit 35c5c6bb83beb4f68e1ff18907589b062a0467e8)

 Conflicts:
	hadoop-common-project/hadoop-registry/pom.xml

(cherry picked from commit 94ca965e21067f1d957d21f19ae8e6abca9da971)
 2021-12-10 01:47:31 +09:00
Akira Ajisaka
eda934025b HADOOP-18035. Ignore unit test failures to run all the unit tests from root (#3765) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 3854b59bd1b06e9740cf33efc7844b66eec30ba4)
 2021-12-10 01:27:38 +09:00
Yufei Gu
68ac742d94 HADOOP-15331. Fix a race condition causing parsing error of java.io.BufferedInputStream in class org.apache.hadoop.conf.Configuration. Contributed by Miklos Szegedi. 
(cherry picked from commit 268c29a5f541449659f4b4ea1975c6f04c7b6a70)
 2021-11-29 14:13:05 +09:00
Abhishek Das
f9463e5111 HADOOP-17999. No-op implementation of setWriteChecksum and setVerifyChecksum in ViewFileSystem. Contributed by Abhishek Das. (#3639) 
(cherry picked from commit 54a1d78e16533e286455de62a545ee75cbc1eff5)
 2021-11-16 22:28:10 -08:00
Szilard Nemeth
5db1b2d7fa HADOOP-17857. Check real user ACLs in addition to proxied user ACLs. Contributed by Eric Payne 
(cherry picked from commit 5428d36b56fab319ab68258139d6133ded9bbafc)
 2021-10-21 17:40:02 +00:00
Xing Lin
2565ec5804 HADOOP-16532. Fix TestViewFsTrash to use the correct homeDir. Contributed by Xing Lin. (#3514) 
(cherry picked from commit 97c0f968792e1a45a1569a3184af7b114fc8c022)
 2021-10-13 14:59:38 -07:00
hchaverr
2187c6d366 HADOOP-17819. Add extensions to ProtobufRpcEngine RequestHeaderProto. Contributed by Hector Sandoval Chaverri. (#3242) 
(cherry picked from commit 3c8a48e681babe4038e49205624e7be183b3466e)
 2021-07-28 16:04:46 -07:00
Abhishek Das
99b952fcb4 HADOOP-17028. ViewFS should initialize mounted target filesystems lazily. Contributed by Abhishek Das (#2260, #3218) 
(cherry picked from commit 1dd03cc4b573270dc960117c3b6c74bb78215caa)
 2021-07-20 18:26:24 -07:00
Akira Ajisaka
992d2c5f4e
HADOOP-12665. Document hadoop.security.token.service.use_ip. (#3187) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
Reviewed-by: Chris Nauroth <cnauroth@apache.org>
(cherry picked from commit c81f82e21d137b6d45c20c481e7bbc6160d72f94)
 2021-07-12 10:19:06 +09:00
Artem Smotrakov
9241470dfd HADOOP-17793. Better token validation (#3189) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba325a8ada573291266c4d6447862072fdf88af5)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/Token.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockTokenSecretManager.java
 2021-07-10 15:06:37 +09:00
S O'Donnell
caa59d7a58 HDFS-15150. Introduce read write lock to Datanode. Contributed by Stephen O'Donnell and Ahmed Hussein 2021-06-22 16:24:19 +01:00
Konstantin V Shvachko
149325484f HDFS-16040. RpcQueueTime metric counts requeued calls as unique events. Contributed by Simbarashe Dzinamarira. 
(cherry picked from commit 8ce30f51f999c0a80db53a2a96b5be5505d4d151)
 2021-05-27 20:19:44 -07:00
hchaverri
7f6185b17c
HDFS-15912. Allow ProtobufRpcEngine to be extensible (#2999) 2021-05-17 00:37:07 -07:00
Eric Badger
58b33d6570 MAPREDUCE-7302. Upgrading to JUnit 4.13 causes testcase TestFetcher.testCorruptedIFile() to fail. Contributed by Peter Bacsko. Reviewed by Akira Ajisaka. 
Conflicts only in imports

(cherry picked from commit da93cd962c8d19874d09360726628cdde8595a7d)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java

(cherry picked from commit 7ad844ee9ed7eaab4fbb4d38576be10d497460cf)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestFetcher.java
 2021-04-23 20:16:11 +00:00
Eric Badger
eb176bb2af YARN-10460. Upgrading to JUnit 4.13 causes tests in TestNodeStatusUpdater to fail. Contributed by Peter Bacsko 2021-04-20 00:31:08 +00:00
Akira Ajisaka
f207d9e620
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin. Contributed by Ahmed Hussein. 
Removed findbugs from the hadoop build images and added spotbugs instead.
Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0.
 2021-03-29 09:44:28 +09:00
Jim Brennan
b3532a7c22 MAPREDUCE-7322. revisiting TestMRIntermediateDataEncryption. Contributed by Ahmed Hussein. 2021-03-22 20:30:19 +00:00
kwangsun
e6b206b33d
HADOOP-17952. Fix the wrong CIDR range example in Proxy User documentation. (#2780) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c8d327a4f1a7f15d6be35051414199d1d3fdc5ef)
 2021-03-22 11:47:03 +09:00
Akira Ajisaka
9c3c753525
HADOOP-17546. Update Description of hadoop-http-auth-signature-secret in HttpAuthentication.md. Contributed by Ravuri Sushma sree. 
(cherry picked from commit 9fd2198daa58610c04bdc1ac7570476accdc25b7)
 2021-03-04 14:58:01 +09:00
Jim Brennan
f31c122ec9 MAPREDUCE-7320. organize test directories for ClusterMapReduceTestCase (#2722). Contributed by Ahmed Hussein 
(cherry picked from commit 4d323cd47c0787c1bb7592ed56acd4aced3e11e7)
 2021-02-26 21:51:47 +00:00
Renukaprasad C
7c83923192 HADOOP-17510. Hadoop prints sensitive Cookie information. (#2673) 
(cherry picked from commit 924fa87010ac193a8242d59b14f9fa9d17957fe5)
(cherry picked from commit 9951ea9fa8427c134861f047acc164b5cee4eef0)
 2021-02-24 17:34:45 +08:00
Kengo Seki
5f969ced67
HADOOP-16764. Rewrite Python example codes using Python3 (#1762) 
(cherry picked from commit fd7de2b82a6b9d600b43909a0770a6b0afea899a)
 2021-02-20 19:16:30 +09:00
He Xiaoqiao
1ded537146
HADOOP-17501. Fix logging typo in ShutdownHookManager. Contributed by Fengnan Li. 2021-01-31 20:01:43 +08:00
He Xiaoqiao
b30e9198bb
HADOOP-16947. Stale record should be remove when MutableRollingAverages generating aggregate data. Contributed by Felix N, Haibin Huang. 2021-01-30 12:11:44 +08:00
Akira Ajisaka
130d5af258
HADOOP-17478. Improve the description of hadoop.http.authentication.signature.secret.file (#2628) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit d82009599a2e9f48050e0c41440b36c759ec068f)
 2021-01-24 22:59:54 +09:00
Takanobu Asanuma
3dcb06a1ba
HADOOP-16841. The description of hadoop.http.authentication.signature.secret.file contains outdated information. Contributed by Xieming Li. 
(cherry picked from commit 5cbc4c54611f062690c3bfcf78d09d37c78ffd12)
 2021-01-24 22:59:46 +09:00
Anton Kutuzov
0ed997abab HDFS-15632. AbstractContractDeleteTest should set recursive peremeter to true for recursive test cases. Contributed by Anton Kutuzov. 
(cherry picked from commit 91d4ba57c5b85379303ac8fb2a1a03ba10b07d4e)
 2021-01-22 18:19:51 -08:00
Wei-Chiu Chuang
e48b91c83c HDFS-13723. Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh. Contributed by Siyao Meng. 
(cherry picked from commit 162228e8db937d4bdb9cf61d15ed555f1c96368f)
 2021-01-21 17:48:14 +00:00
Yiqun Lin
1b046fb26f HADOOP-13965. Groups should be consistent in using default group mapping class. Contributed by Yiqun Lin. 
(cherry picked from commit 383aa9c9f55983cdb1c225b2ed22f6f48616cb9c)
 2021-01-21 17:33:54 +00:00
He Xiaoqiao
1efadde067
HDFS-15751. Add documentation for msync() API to filesystem.md. Contributed by Konstantin V Shvachko. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Chao Sun <sunchao@apache.org>
 2021-01-03 16:38:47 +08:00
David Mollitor
16fd00d62d HADOOP-16208. Do Not Log InterruptedException in Client. 
Contributed by David Mollitor.

(cherry picked from commit c90736350ba158c7872a39426e7a29c5e5e0bb48)
 2020-12-22 10:19:44 -08:00
He Xiaoqiao
2c45e93682 HADOOP-17068. Client fails forever when namenode ipaddr changed. Contributed by Sean Chow. 
(cherry picked from commit fa14e4bc001e28d9912e8d985d09bab75aedb87c)
(cherry picked from commit 3a860e876e799ef44fca34e3cb19a89dbd4f6744)
(cherry picked from commit 8d7600ae6b7fadba1b4695666cf9287be021d481)
(cherry picked from commit 1fe15113cf464015212e6ad7ecbca22dc1c50309)
 2020-12-15 14:08:20 -08:00
Ahmed Hussein
c46643549f HADOOP-17358. Improve excessive reloading of Configurations (#2436) 
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
(cherry picked from commit 71071e5c0fcaf73a3989000dbc60fa214a317da1)
(cherry picked from commit 23fe3bdab3410b8ed370fa6af9f2ed63817006ab)
(cherry picked from commit ac3e10af63c4a70e45b26acd50b01a93fa8fd605)
(cherry picked from commit 487eaa6dea14bba100583f30f196c0d51215bd7c)
 2020-11-12 10:55:35 -08:00
Steve Loughran
894614ee4e HADOOP-17340. TestLdapGroupsMapping failing -string mismatch in exception validation. (#2427). Contributed by Steve Loughran. 2020-11-07 17:06:17 +05:30
hchaverr
9160b0e8c8 HDFS-15623. Respect configured values of rpc.engine (#2403) Contributed by Hector Chaverri. 
(cherry picked from commit 6eacaffeea21e7e9286497ee17f89fd939d2eead)
 2020-11-06 15:07:34 -08:00
Eric Badger
7a830d2633 HADOOP-17342. Creating a token identifier should not do kerberos name 
resolution. Contributed by Jim Brennan.

(cherry picked from commit af389d989770c4218c24e640f572bc7492f7d4b3)
 2020-11-05 22:17:32 +00:00
Wei-Chiu Chuang
fc4c98dc49 Revert "HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. (#2291)" 
This reverts commit 52f011905182dae2ddfbc68bb06341e2565424b9.
 2020-11-04 16:19:13 -08:00
Chen Liang
ac82334041 HADOOP-17336. Backport HADOOP-16005 NativeAzureFileSystem does not support setXAttr and HADOOP-16785. Improve wasb and abfs resilience on double close() calls. followup to abfs close() fix to branch-2.10. Contributed by Sally Zuo. 2020-11-03 12:00:57 -08:00
Akira Ajisaka
52f0119051 HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. (#2291) 
Reviewed-by: Steve Loughran <stevel@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7f5caca04cb8e5dab8410ef64daa0fe799e389fd)
(cherry picked from commit dd1634ec3b676d6c3da18af7da46734584ddd9f6)
(cherry picked from commit 1c7043b1ff89ba9c995cbf546a99bbbcccbe56b1)
(cherry picked from commit 166bee93af6fb68ee2b3fab4c355413cae9fc1d6)
 2020-11-03 11:24:08 -08:00
Konstantin V Shvachko
104dd85ad8 HDFS-15567. [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. Contributed by Konstantin V Shvachko. 
(cherry picked from commit b3786d6c3cc13b0b92b9f42da1731c4ce35c9ded)
 2020-10-12 18:02:05 -07:00
Masatake Iwasaki
0fd2ceb1b5 Publishing the bits for release 2.10.1 2020-09-22 11:48:18 +09:00
Owen O'Malley
ca65409836 Revert "HADOOP-13119. Add ability to secure log servlet using proxy users. Contribute by Yuanbo Liu." 
This reverts commit a847903b6e64c6edb11d852b91f2c816b1253eb3.

Change-Id: I3122a2142f5bdf8507dece930e447556a43cd9ae
(cherry picked from commit 8fad3ec76070ccfcd3ed80feaba4355077bc6f5c)
(cherry picked from commit fa6a8b78d481d3b4d355e1bf078f30dd5e09850d)

 Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServerWithSpengo.java
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestAuthenticationWithProxyUserFilter.java
 2020-09-14 21:48:17 +09:00
Masatake Iwasaki
f4e0c14fe9 Preparing for 2.10.2 development 2020-09-13 14:33:36 +09:00
Steve Loughran
a75f2915f9 HADOOP-17199. S3A Directory Marker HADOOP-13230 backport 
This backports the listing-side changes of HADOOP-13230.

With this patch in, this branch of Hadoop is compatible with S3A clients
which do not delete directory markers when files are created underneath.

It does not allow this version to disable marker deletion; if the
fs.s3a.marker.retention option is changed to request this, a message
is printed at INFO and the policy remains at "delete"

The s3guard bucket-info command has been extended to support
probing for marker retention, as has the hasPathCapability method on
S3AFileSystem.

Read the documentation!

Change-Id: Ief23c39256b194abcc445231b5f7a018185ea9f6
 2020-09-03 18:29:30 +01:00
sguggilam
e645204733
HADOOP-17159. Make UGI support forceful relogin from keytab ignoring the last login time (#2245) 
Contributed by Sandeep Guggilam.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Steve Loughran <stevel@apache.org>
 2020-08-27 15:21:20 -07:00
Akira Ajisaka
b87e080433
HADOOP-17204. Fix typo in Hadoop KMS document. Contributed by Xieming Li. 
(cherry picked from commit 141c62584bd253f0831d39557f54627f26cef7f7)
 2020-08-12 16:10:11 +09:00
sguggilam
4eb1f81876
HADOOP-17164 UGI loginUserFromKeytab doesn't set the last login time (#2194) 
Contributed by Sandeep Guggilam.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Steve Loughran <stevel@apache.org>
 2020-08-05 09:04:01 -07:00
Chen Liang
3044a13080 HDFS-15404. ShellCommandFencer should expose info about source. Contributed by Chen Liang. 
(cherry picked from commit 3833c616e087518196bcb77ac2479c66a0b188d8)
(cherry picked from commit 370efc6d95fde8065dd5ba742446d0e9e570a5e9)
(cherry picked from commit 1e33696bc8fa74fbb00248c5de485e951a692814)
 2020-07-20 15:26:44 -07:00