Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,720 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4795 Commits

Author SHA1 Message Date
Yongjun Zhang 425fe4e217 Update CHANGES, RELEASENOTES, and jdiff for 3.0.3 release. 
(cherry picked from commit 5a82f10e32)
 2018-06-11 14:31:46 -07:00
Sammi Chen e3c96354a7 HADOOP-15499. Performance severe drops when running RawErasureCoderBenchmark with NativeRSRawErasureCoder. Contributed by Sammi Chen. 
(cherry picked from commit 18201b882a)
(cherry picked from commit b874110275)
 2018-06-11 14:03:39 +08:00
Inigo Goiri 8526ba9a69 HADOOP-15516. Add test cases to cover FileUtil#readLink. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 12be8bad7d)
 2018-06-07 13:35:32 -07:00
Xiao Chen f1c91e9fda HADOOP-15217. FsUrlConnection does not handle paths with spaces. Contributed by Joseph Fourny and Zsolt Venczel. 
(cherry picked from commit ba4011d64f)
 2018-06-05 21:19:11 -07:00
Daniel Templeton 09fd1348e8 HDFS-13636. Cross-Site Scripting vulnerability in HttpServer2 
(Contributed by Haibo Yan via Daniel Templeton)

Change-Id: I28edde8125dd20d8d270f0e609d1c04d8173c8b7
(cherry picked from commit cba3194998)
 2018-06-01 14:44:22 -07:00
Kihwal Lee 11a425d11a Additional check when unpacking archives. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit e3236a9680)
 2018-05-29 15:02:33 -05:00
Inigo Goiri 1dd9670ddd HADOOP-15497. TestTrash should use proper test path to avoid failing on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 3c75f8e493)
 2018-05-29 09:11:38 -07:00
Karthik Palanisamy b0ce76d71f
HADOOP-15449. Increase default timeout of ZK session to avoid frequent NameNode failover 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 61df174e8b)
 2018-05-28 19:41:58 +09:00
Xiao Chen 1c0ee35662 HADOOP-15473. Configure serialFilter in KeyProvider to avoid UnrecoverableKeyException caused by JDK-8189997. Contributed by Gabor Bota. 
(cherry picked from commit 02322de3f9)
 2018-05-25 09:41:04 -07:00
Miklos Szegedi 1483c90379 YARN-8310. Handle old NMTokenIdentifier, AMRMTokenIdentifier, and ContainerTokenIdentifier formats. Contributed by Robert Kanter. 
(cherry picked from commit 3e5f7ea986)
 2018-05-24 15:14:15 -07:00
Arpit Agarwal 76f38f3f8b HADOOP-15486. Make NetworkTopology#netLock fair. Contributed by Nanda kumar. 2018-05-23 10:30:37 -07:00
Sammi Chen 7d71b3a1cc HDFS-13540. DFSStripedInputStream should only allocate new buffers when reading. Contributed by Xiao Chen. 
(cherry picked from commit 34e8b9f9a8)
 2018-05-23 19:12:04 +08:00
Akira Ajisaka fc4c20fc34
Additional check when unpacking archives. Contributed by Jason Lowe and Akira Ajisaka. 
(cherry picked from commit 745f203e57)
 2018-05-23 17:18:19 +09:00
Kihwal Lee 6b8a5af6ef HADOOP-15450. Avoid fsync storm triggered by DiskChecker and handle disk full situation. Contributed by Arpit Agarwal. 
(cherry picked from commit bcc8e76bad)
 2018-05-22 11:22:22 -05:00
Rushabh Shah 1c594639f1 Skip the proxy user check if the ugi has not been initialized. Contributed by Daryn Sharp 
(cherry picked from commit 73e9120ad7)
 2018-05-21 12:39:59 -05:00
Inigo Goiri 472495ed1e HDFS-13586. Fsync fails on directories on Windows. Contributed by Lukas Majercack. 
(cherry picked from commit 8783613696)
 2018-05-17 19:27:23 -07:00
Inigo Goiri df54cdbaea HDFS-13560. Insufficient system resources exist to complete the requested service for some tests on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 53b807a6a8)
 2018-05-17 17:03:56 -07:00
Steve Loughran f729faaabb HADOOP-15250. Split-DNS MultiHomed Server Network Cluster Network IPC Client Bind Addr Wrong 
Contributed by Ajay Kumar

(cherry picked from commit 8f42dafcf8)
 2018-05-17 10:14:27 -07:00
Sammi Chen e5281d9f8b Add 2.9.1 release notes and changes documents 2018-05-14 15:26:00 +08:00
Rushabh Shah 4d9a58756c HADOOP-15441. Log kms url and token service at debug level. Contributed by Gabor Bota 
(cherry picked from commit a5449d3631)
 2018-05-12 12:21:01 -05:00
Inigo Goiri f4f2912820 HADOOP-15454. TestRollingFileSystemSinkWithLocal fails on Windows. Contributed by Xiao Liang. 
(cherry picked from commit 1da8d4190d)
 2018-05-10 09:41:42 -07:00
Xiao Chen 752092a860 Revert "HADOOP-14445. Delegation tokens are not shared between KMS instances. Contributed by Xiao Chen and Rushabh S Shah." 
This reverts commit 583fa6ed48.

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSTokenRenewer.java
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java

(cherry picked from commit a3a1552c33)

 Conflicts:
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
 2018-05-07 13:47:52 -07:00
Rushabh Shah 864640c1e3 HADOOP-14841 Kms client should disconnect if unable to get output stream from connection. Contributed by Rushabh S Shah 
(cherry picked from commit 4cdbdce752)
 2018-05-04 15:49:39 -05:00
Daniel Templeton 57e4115436 HDFS-13481. TestRollingFileSystemSinkWithHdfs#testFlushThread: test failed intermittently 
(Contributed by Gabor Bota via Daniel Templeton)

Change-Id: I9921981dfa69669fe7912dd2a31ae8b638283204
(cherry picked from commit 87c23ef643)
 2018-05-02 17:17:46 -07:00
Jason Lowe cacd267b0d
HADOOP-15406. hadoop-nfs dependencies for mockito and junit are not test scope 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e07156e8b0)
 2018-05-02 17:32:08 +09:00
Wangda Tan da9d5a3fee HADOOP-15411. AuthenticationFilter should use Configuration.getPropsWithPrefix instead of iterator. (Suma Shivaprasad via wangda) 
Change-Id: Ia19458a43aceb4ffdb1e8eccc519655235d92ba9
(cherry picked from commit 3559d8b1dacf5cf207424de37cb6ba8865d26ffe)
 2018-04-25 22:16:07 -07:00
Robert Kanter ea7ad50499 HADOOP-15390. Yarn RM logs flooded by DelegationTokenRenewer trying to renew KMS tokens (xiaochen via rkanter) 
(cherry picked from commit 7ab08a9c37)
 2018-04-23 16:25:41 -07:00
Mukul Kumar Singh 94076c4028 HADOOP-15366. Add a helper shutdown routine in HadoopExecutor to ensure clean shutdown. Contributed by Shashikant Banerjee. 2018-04-18 16:30:59 +05:30
Virajith Jalaparti 70830d9f9c HDFS-13311. RBF: TestRouterAdminCLI#testCreateInvalidEntry fails on Windows. Contributed by Inigo Goiri 2018-04-16 16:06:25 -07:00
Xiao Chen 805e33b62c HDFS-13056. Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts. Contributed by Dennis Huo. 2018-04-16 11:58:10 -07:00
Yufei Gu 6891da1e75 HADOOP-15331. Fix a race condition causing parsing error of java.io.BufferedInputStream in class org.apache.hadoop.conf.Configuration. Contributed by Miklos Szegedi. 2018-04-13 10:13:34 -07:00
fang zhenyi 922a0db13c HADOOP-15330. Remove jdk1.7 profile from hadoop-annotations module 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-13 10:12:27 -07:00
Yufei Gu 9014f98b60 HADOOP-15062. TestCryptoStreamsWithOpensslAesCtrCryptoCodec fails on Debian 9. Contributed by Miklos Szegedi. 2018-04-13 10:12:01 -07:00
Chris Douglas ffc39ec3ab HADOOP-14667. Flexible Visual Studio support. Contributed by Allen Wittenauer 2018-04-13 10:11:35 -07:00
Bharat Viswanadham b652e915fb HADOOP-15379. Make IrqHandler.bind() public. Contributed by Ajay Kumar 
(cherry picked from commit ec1e8c1a8c)
 2018-04-12 21:55:40 -07:00
Xiao Chen 6d6f65f224 HADOOP-14445. Delegation tokens are not shared between KMS instances. Contributed by Xiao Chen and Rushabh S Shah. 
(cherry picked from commit 583fa6ed48)
 2018-04-10 15:45:35 -07:00
Xiao Chen 96af1af28f HADOOP-15313. TestKMS should close providers. 
(cherry picked from commit c22d62b338)
 2018-04-10 15:45:34 -07:00
Jason Lowe ffceb907fe HADOOP-15357. Configuration.getPropsWithPrefix no longer does variable substitution. Contributed by Jim Brennan 
(cherry picked from commit e81397545a)
 2018-04-10 16:49:27 -05:00
Ewan Higgs c9da1e97f8
HADOOP-15376. Remove double semi colons on imports that make Clover fall over. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit cef8eb7981)
 2018-04-10 23:59:34 +09:00
Wangda Tan 25bf1d3ac6 Updated branch-3.1 to track 3.1.1 
Change-Id: Idd55c79921b8acacc8a8a3258450e476f3abb706
 2018-04-09 10:58:50 -07:00
Wangda Tan a336561e2b Added CHANGES/RELEASES/Jdiff for 3.1.0 release 
Change-Id: Ied5067a996151c04d15cad46c46ac98b60c37b39
(cherry picked from commit 2d96570452a72569befdf9cfe9b90c9fa2e0e261)
 2018-04-05 15:52:02 -07:00
Wei Yan 89efe30f65 HDFS-13353. RBF: TestRouterWebHDFSContractCreate failed. Contributed by Takanobu Asanuma. 
(cherry picked from commit 3121e8c293)
 2018-04-05 13:13:16 -07:00
Xiao Chen 8d755371f2 HADOOP-15355. TestCommonConfigurationFields is broken by HADOOP-15312. Contributed by LiXin Ge. 
(cherry picked from commit f614b44fe583033947cc78d8f04cfe6fb64fc892)
 2018-04-02 23:10:17 -07:00
Xiao Chen dcb2176c5d HADOOP-15317. Improve NetworkTopology chooseRandom's loop. 
(cherry picked from commit 57374c4737)
 2018-04-02 22:48:54 -07:00
Konstantin V Shvachko 4ce45a8ec7 HADOOP-15253. Should update maxQueueSize when refresh call queue. Contributed by Tao Jie. 
(cherry picked from commit acfd764fcc)
 2018-03-30 18:57:18 -07:00
Konstantin V Shvachko 99b5b9dce1 HADOOP-12862. LDAP Group Mapping over SSL can not specify trust store. Contributed by Wei-Chiu Chuang and Konstantin Shvachko. 
(cherry picked from commit 2216bde322)
 2018-03-29 17:20:28 -07:00
Wei-Chiu Chuang 1176a128d6 HADOOP-15312. Undocumented KeyProvider configuration keys. Contributed by LiXin Ge. 
(cherry picked from commit 3fe41c65d8)
 2018-03-27 13:36:00 -07:00
Jitendra Pandey 91b98ad03a HADOOP-14067. VersionInfo should load version-info.properties from its own classloader. Contributed by Thejas M Nair. 2018-03-27 10:24:04 -07:00
Lei Xu 98d7a5aaef Update releasenotes and changelogs for 3.0.1 release 2018-03-23 11:53:16 -07:00
Inigo Goiri 9b96e7522b HDFS-13268. TestWebHdfsFileContextMainOperations fails on Windows. Contributed by Xiao Liang. 
(cherry picked from commit fe224ff972)
 2018-03-20 11:20:37 -07:00
Inigo Goiri 7eb0bdbd32 HDFS-13296. GenericTestUtils generates paths with drive letter in Windows and fail webhdfs related test cases. Contributed by Xiao Liang. 
(cherry picked from commit e65ff1c8be)
 2018-03-19 17:20:36 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 0bd99621c5 Preparing for 3.1.0 development 2018-03-19 15:16:54 -07:00
Xiao Chen 21c6661461 HADOOP-15234. Throw meaningful message on null when initializing KMSWebApp. Contributed by fang zhenyi. 2018-03-15 20:17:16 -07:00
Steve Loughran 1976e0066e HADOOP-15209. DistCp to eliminate needless deletion of files under already-deleted directories. 
Contributed by Steve Loughran.
 2018-03-15 18:05:14 +00:00
Aaron Fabbri e840b4adf4
HADOOP-15278 log s3a at info. Contributed by Steve Loughran. 2018-03-14 15:47:56 -07:00
Wangda Tan 252c2b4d52 Revert "HADOOP-13707. If kerberos is enabled while HTTP SPNEGO is not configured, some links cannot be accessed. Contributed by Yuanbo Liu" 
Change-Id: I946a466a43d56c73bb0135384e73cb8513595461
(cherry picked from commit 80ee5248b2dda1cb8d122d4f362f2f8cf02b9467)
 2018-03-14 13:36:53 -07:00
Takanobu Asanuma fea16a440d
HADOOP-15294. TestUGILoginFromKeytab fails on Java9 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-14 13:38:07 +09:00
Inigo Goiri 427fd027a3 HADOOP-15308. TestConfiguration fails on Windows because of paths. Contributed by Xiao Liang. 2018-03-13 20:01:07 -07:00
Chris Douglas 9d6994da19 HADOOP-15311. HttpServer2 needs a way to configure the acceptor/selector count. Contributed by Erik Krogen 2018-03-13 13:55:18 -07:00
Chris Douglas 45d1b0fdcc HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer 2018-03-12 20:05:39 -07:00
Steve Loughran dd05871b8b HADOOP-15297. Make S3A etag => checksum feature optional. 
Contributed by Steve Loughran.
 2018-03-12 14:01:42 +00:00
Owen O'Malley fa6a8b78d4 Revert "HADOOP-13119. Add ability to secure log servlet using proxy users. Contribute by Yuanbo Liu." 
This reverts commit a847903b6e.

Change-Id: I3122a2142f5bdf8507dece930e447556a43cd9ae
(cherry picked from commit 8fad3ec76070ccfcd3ed80feaba4355077bc6f5c)
 2018-03-09 22:46:41 -08:00
Owen O'Malley 3a8dade9b1 Revert "HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu." 
This reverts commit 172b23af33.

(cherry picked from commit d0d2d4c51e9534e08893ae14cf3fff7b2ee70b1d)
 2018-03-09 22:46:30 -08:00
Takanobu Asanuma 99ab511cba
HADOOP-15293. TestLogLevel fails on Java 9 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-09 10:20:35 -08:00
Steve Loughran 3f7bd46797 HADOOP-15277. Remove .FluentPropertyBeanIntrospector from CLI operation log output. 
Contributed by Steve Loughran.
 2018-03-09 10:44:07 +00:00
Xiao Chen a906a22645 HADOOP-15280. TestKMS.testWebHDFSProxyUserKerb and TestKMS.testWebHDFSProxyUserSimple fail in trunk. Contributed by Bharat Viswanadham. 2018-03-08 10:17:02 -08:00
Takanobu Asanuma e6f99e205b
HADOOP-15291. TestMiniKdc fails on Java 9 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-06 06:01:33 -08:00
Steve Loughran 12ecb55ffe HADOOP-15289. FileStatus.readFields() assertion incorrect. 
Contributed by Steve Loughran.
 2018-03-06 10:05:12 +00:00
Takanobu Asanuma 4971276631
HADOOP-15271. Remove unicode multibyte characters from JavaDoc 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-05 15:07:58 -08:00
Ajay Kumar 3dc30bc24e
HADOOP-15286. Remove unused imports from TestKMSWithZK.java 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-04 07:12:52 -08:00
Ray Chiang 923e1773e1 HADOOP-14652. Update metrics-core version to 3.2.4. (rchiang) 2018-03-01 15:36:10 -08:00
Nanda kumar 55d04a6db1
HADOOP-15275. Incorrect javadoc for return type of RetryPolicy#shouldRetry 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-01 19:03:47 +09:00
Xiaoyu Yao f20e10b2dd HDFS-13081. Datanode#checkSecureConfig should allow SASL and privileged HTTP. Contributed by Ajay Kumar. 2018-02-28 09:44:39 -08:00
Arpit Agarwal ac42dfc888 MAPREDUCE-7061. SingleCluster setup document needs to be updated. Contributed by Bharat Viswanadham. 2018-02-27 10:27:18 -08:00
Arpit Agarwal 137f0d324a HADOOP-14959. DelegationTokenAuthenticator.authenticate() to wrap network exceptions. Contributed by Ajay Kumar. 2018-02-27 10:15:43 -08:00
Arpit Agarwal 28f644bf25 HADOOP-15178. Generalize NetUtils#wrapException to handle other subclasses with String Constructor. Contributed by Ajay Kumar. 2018-02-27 07:38:29 -08:00
Arpit Agarwal 78a10029ec HADOOP-15265. Exclude json-smart explicitly in hadoop-auth avoid being pulled in transitively. Contributed by Nishant Bangarwa. 2018-02-26 13:56:53 -08:00
Arpit Agarwal 2fa7963c3d HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi. 2018-02-24 14:41:55 -08:00
Arpit Agarwal 033f9c68ea HADOOP-15255. Upper/Lower case conversion support for group names in LdapGroupsMapping. Contributed by Nanda kumar. 2018-02-23 15:37:17 -08:00
Kihwal Lee 59cf758877 HADOOP-9747. Reduce unnecessary UGI synchronization. Contributed by Daryn Sharp. 2018-02-23 13:10:56 -06:00
Anu Engineer 3688e491d5 HADOOP-15007. Stabilize and document Configuration <tag> element. Contributed by Ajay Kumar. 2018-02-23 10:26:22 -08:00
Sean Mackrory 2bc3351eaf HADOOP-6852. apparent bug in concatenated-bzip2 support (decoding). Contributed by Zsolt Venczel. 2018-02-21 12:57:14 -07:00
Arpit Agarwal b0d3c877e3 HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar. 2018-02-20 18:18:58 -08:00
Robert Kanter 324e5a7cf2 HADOOP-15235. Authentication Tokens should use HMAC instead of MAC (rkanter) 2018-02-20 17:24:37 -08:00
Steve Loughran 1d37cf675c HADOOP-15070. add test to verify FileSystem and paths differentiate on user info. 
Contributed by Steve Loughran.
 2018-02-19 20:43:40 +00:00
fang zhenyi 4d4dde5112
HADOOP-15223. Replace Collections.EMPTY* with empty* when available 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-02-18 22:19:39 +09:00
Wangda Tan 31db977cb3 HADOOP-14875. Create end user documentation from the compatibility guidelines. (Daniel Templeton via wangda) 
Change-Id: I5d66d0f034874623af22aecae0a0b858b65396ed
 2018-02-17 12:28:55 +08:00
Jason Lowe 0898ff42e9 HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka 2018-02-16 14:49:00 -06:00
Steve Loughran 9a013b255f HADOOP-15176. Enhance IAM Assumed Role support in S3A client. 
Contributed by Steve Loughran

(cherry picked from commit 96c047fbb9)
 2018-02-15 15:57:10 +00:00
Anu Engineer 8f66affd62 HADOOP-15204. Add Configuration API for parsing storage sizes. Contributed by Anu Engineer. 2018-02-14 13:11:37 -08:00
Xiao Chen 1f20f432d2 Revert "HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar." 
This reverts commit 332269de06.
 2018-02-14 10:25:05 -08:00
Steve Loughran f20dc0d577 HADOOP-10571. Use Log.*(Object, Throwable) overload to log exceptions. 
Contributed by Andras Bokor.
 2018-02-14 16:20:14 +00:00
Arpit Agarwal 332269de06 HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar. 2018-02-13 10:14:16 -08:00
Chris Douglas 0c5d7d71a8 HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine 2018-02-12 21:07:15 -08:00
Chris Douglas 9cc6d1dfb3 Revert "HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine" 
This reverts commit 5b88cb3398.
 2018-02-12 21:06:10 -08:00
Chris Douglas 5b88cb3398 HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine 2018-02-12 21:00:47 -08:00
Wangda Tan 60f9e60b3b Preparing for 3.2.0 development 
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
 2018-02-11 11:17:38 +08:00
Steve Loughran 996796f104 HADOOP-15214. Make Hadoop compatible with Guava 21.0. 
Contributed by Igor Dvorzhak
 2018-02-08 10:55:54 -08:00
Anu Engineer 4304fcd5bd HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen. 2018-02-06 13:43:45 -08:00
Kihwal Lee 2dd960de98 HADOOP-15212. Add independent secret manager method for logging expired tokens. Contributed by Daryn Sharp. 2018-02-06 15:33:18 -06:00