Commit Graph

202 Commits

Author SHA1 Message Date
Ahmed Hussein 07050339e0
HADOOP-17367. Add InetAddress api to ProxyUsers.authorize (#2449). Contributed by Daryn Sharp and Ahmed Hussein 2020-11-19 14:37:14 -06:00
Liang-Chi Hsieh 34aa6137bd
HADOOP-17292. Using lz4-java in Lz4Codec (#2350)
Contributed by Liang-Chi Hsieh.
2020-11-18 12:03:25 -08:00
Akira Ajisaka 6a9ceedfb3
HADOOP-17175. [JDK 11] Fix javadoc errors in hadoop-common module. (#2397) 2020-10-23 03:15:45 +09:00
Ayush Saxena 1e3a6efcef
HADOOP-17288. Use shaded guava from thirdparty. (#2342). Contributed by Ayush Saxena. 2020-10-17 12:01:18 +05:30
Liang-Chi Hsieh c9ea344f98
HADOOP-17125. Use snappy-java in SnappyCodec (#2297)
This switches the SnappyCodec to use the java-snappy codec, rather than the native one. 

To use the codec, snappy-java.jar (from org.xerial.snappy) needs to be on the classpath.

This comesin as an avro dependency,  so it is already on the hadoop-common classpath,
as well as in hadoop-common/lib.
The version used is now managed in the hadoop-project POM; initially 1.1.7.7

Contributed by DB Tsai and Liang-Chi Hsieh
2020-10-06 17:07:54 +01:00
Vinayakumar B 82b86e3754
HDFS-15098. Add SM4 encryption method for HDFS. Contributed by liusheng 2020-09-27 19:27:13 +05:30
Akira Ajisaka c40cbc57fa
HADOOP-17091. [JDK11] Fix Javadoc errors (#2098) 2020-08-03 10:46:51 +09:00
Vinayakumar B e154084770
HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes (#2026) 2020-06-12 23:16:33 +05:30
Akira Ajisaka 52b21de1d8
HADOOP-17049. javax.activation-api and jakarta.activation-api define overlapping classes (#2027)
* Removed javax.activation-api from dependency
2020-05-22 11:19:11 +09:00
Mike 18d7dfbf35
HDFS-1820. FTPFileSystem attempts to close the outputstream even when it is not initialised. (#1952)
Contributed by Mikhail Pryakhin.
2020-04-27 14:43:51 +01:00
Brahma Reddy Battula 8914cf9167 Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
Vinayakumar B 7dac7e1d13
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency (#1635). Contributed by Vinayakumar B. 2020-02-07 14:51:24 +05:30
Sahil Takiar f206b736f0
HADOOP-16346. Stabilize S3A OpenSSL support.
Introduces `openssl` as an option for `fs.s3a.ssl.channel.mode`.
The new option is documented and marked as experimental.

For details on how to use this, consult the peformance document
in the s3a documentation.

This patch is the successor to HADOOP-16050 "S3A SSL connections
should use OpenSSL" -which was reverted because of
incompatibilities between the wildfly OpenSSL client and the AWS
HTTPS servers (HADOOP-16347). With the Wildfly release moved up
to 1.0.7.Final (HADOOP-16405) everything should now work.

Related issues:

* HADOOP-15669. ABFS: Improve HTTPS Performance
* HADOOP-16050: S3A SSL connections should use OpenSSL
* HADOOP-16371: Option to disable GCM for SSL connections when running on Java 8
* HADOOP-16405: Upgrade Wildfly Openssl version to 1.0.7.Final

Contributed by Sahil Takiar

Change-Id: I80a4bc5051519f186b7383b2c1cea140be42444e
2020-01-21 16:37:51 +00:00
Akira Ajisaka bc366d4ea7
HADOOP-16773. Fix duplicate assertj-core dependency in hadoop-common module. Contributed by Xieming Li. 2020-01-07 20:49:24 +09:00
Vinayakumar B e2a5448d2b
HADOOP-16774. TestDiskChecker and TestReadWriteDiskValidator fails when run with -Pparallel-tests (#1776). Contributed by Vinayakumar B. 2019-12-21 00:08:15 +05:30
Vinayakumar B 07c81e9bfc
HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes (#1494). Contributed by Vinayakumar B. 2019-09-23 12:37:47 +05:30
Sahil Takiar 55ce454ce4
HADOOP-16371: Option to disable GCM for SSL connections when running on Java 8.
Contributed by Sahil Takiar.

This moves the SSLSocketFactoryEx class from hadoop-azure into hadoop-common
as the DelegatingSSLSocketFactory and binds the S3A connector to it so that
it can avoid using those HTTPS algorithms which are underperformant on Java 8.

Change-Id: Ie9e6ac24deac1aa05e136e08899620efa7d22abd
2019-09-17 11:32:03 +01:00
Takanobu Asanuma 54f9f75a44 HADOOP-16369. Fix zstandard shortname misspelled as zts. Contributed by Jonathan Eagles. 2019-06-14 10:17:25 +09:00
Sammi Chen d1aad44490 HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He. 2019-06-05 21:33:00 +08:00
Steve Loughran 309501c6fa
Revert "HADOOP-16050: s3a SSL connections should use OpenSSL"
This reverts commit b067f8acaa.

Change-Id: I584b050a56c0e6f70b11fa3f7db00d5ac46e7dd8
2019-06-05 13:54:55 +01:00
Akira Ajisaka afd844059c HADOOP-16331. Fix ASF License check in pom.xml
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:25:13 +09:00
Akira Ajisaka 9f933e6446
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:24:59 +09:00
Eric Yang ea0b1d8fba HADOOP-16287. Implement ProxyUserAuthenticationFilter for web protocol impersonation.
Contributed by Prabhu Joseph
2019-05-23 11:36:32 -04:00
Akira Ajisaka a771e2a638
HADOOP-12948. Remove the defunct startKdc profile from hadoop-common. Contributed by Wei-Chiu Chuang. 2019-05-23 13:59:42 +09:00
Sahil Takiar b067f8acaa HADOOP-16050: s3a SSL connections should use OpenSSL
(cherry picked from commit aebf229c17)
2019-05-16 08:57:54 -06:00
Todd Lipcon 65deb1ac42 HADOOP-16179. hadoop-common pom should not depend on kerb-simplekdc
The hadoop-common pom currently has a dependency on kerb-simplekdc. In
fact, the only classes used from Kerby are in kerb-core and kerb-util
(which is a transitive dependency frmo kerb-core). Depending on
kerb-simplekdc pulls a bunch of other unnecessary classes into the
hadoop-common classpath.

This changes the hadoop-common pom to depend only on kerb-core.

hadoop-minikdc already had the appropriate dependency on kerb-simplekdc
so it continues to pull in what it needs.

Signed-off-by: Todd Lipcon <todd@apache.org>
2019-04-10 08:49:46 -07:00
Akira Ajisaka 1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki. 2019-01-29 18:29:56 -08:00
Akira Ajisaka 6fa3feb577
HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter. 2018-10-10 10:16:57 +09:00
Takanobu Asanuma 9bbeb52486 HADOOP-15775. [JDK9] Add missing javax.activation-api dependency. Contributed by Akira Ajisaka. 2018-10-09 13:06:14 +09:00
Akira Ajisaka 31fa9bb799
HADOOP-15767. [JDK10] Building native package on JDK10 fails due to missing javah. Contributed by Takanobu Asanuma. 2018-10-03 11:29:18 +09:00
Sunil G 58fa96b697 Changed version in trunk to 3.3.0-SNAPSHOT. 2018-10-02 22:41:41 +05:30
Ewan Higgs 429a07e08c HADOOP-15764. [JDK10] Migrate from sun.net.dns.ResolverConfiguration to the replacement. Contributed by Akira Ajisaka. 2018-09-20 15:13:55 +02:00
Allen Wittenauer 4b8c2b1c1c
HADOOP-14671. Upgrade Apache Yetus to 0.8.0
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-09-11 08:55:34 -07:00
Akira Ajisaka 88625f5cd9
HADOOP-15531. Use commons-text instead of commons-lang in some classes to fix deprecation warnings. Contributed by Takanobu Asanuma. 2018-07-13 11:42:12 -04:00
Akira Ajisaka 2b2399d623
HADOOP-15495. Upgrade commons-lang version to 3.7 in hadoop-common-project and hadoop-tools. Contributed by Takanobu Asanuma. 2018-06-28 14:37:22 +09:00
Chris Douglas 3fc3fa9711 HADOOP-14667. Flexible Visual Studio support. Contributed by Allen Wittenauer 2018-03-19 16:05:55 -07:00
Chris Douglas 45d1b0fdcc HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer 2018-03-12 20:05:39 -07:00
Wangda Tan 60f9e60b3b Preparing for 3.2.0 development
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
2018-02-11 11:17:38 +08:00
Andrew Wang 0d419c984f Preparing for 3.1.0 development 2017-09-01 11:53:48 -07:00
Steve Loughran 621b43e254
HADOOP-13345 HS3Guard: Improved Consistency for S3A.
Contributed by: Chris Nauroth, Aaron Fabbri, Mingliang Liu, Lei (Eddy) Xu,
Sean Mackrory, Steve Loughran and others.
2017-09-01 14:13:41 +01:00
Allen Wittenauer 190410085b
HADOOP-14364. refresh changelog/release notes with newer Apache Yetus build
Signed-off-by: Andrew Wang <wang@apache.org>
2017-08-31 19:06:49 -07:00
wenxinhe 691bf5ec5d
HADOOP-14706. Adding a helper method to determine whether a log is Log4j implement.
This closes #257

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2017-08-04 14:35:57 +09:00
Chris Douglas 12e44e7bda HDFS-6984. Serialize FileStatus via protobuf. 2017-08-02 12:12:48 -07:00
Steve Loughran 09653ea098
HADOOP-14634. Remove jline from main Hadoop pom.xml.
Contributed by Ray Chiang.
2017-07-10 11:53:13 +01:00
Andrew Wang af2773f609 Updating version for 3.0.0-beta1 development 2017-06-29 17:57:40 -07:00
Jonathan Eagles a81916ea89 HADOOP-14501. Switch from aalto-xml to woodstox to handle odd XML features (jeagles) 2017-06-12 17:07:53 -05:00
Andrew Wang 16ad896d5c Update maven version for 3.0.0-alpha4 development 2017-05-26 14:09:44 -07:00
Haohui Mai ff5ec3b841 HADOOP-14383. Implement FileSystem that reads from HTTP / HTTPS endpoints. 2017-05-08 15:28:45 -07:00
Chris Douglas a5e57df3c5 HADOOP-14225. Remove xmlenc dependency 2017-04-10 11:48:40 -07:00
Jonathan Eagles 523f467d93 HADOOP-14216. Improve Configuration XML Parsing Performance (jeagles) 2017-03-29 10:12:02 -05:00