4b5b54c73f
HADOOP-17089: WASB: Update azure-storage-java SDK
...
Contributed by Thomas Marquardt
DETAILS: WASB depends on the Azure Storage Java SDK. There is a concurrency
bug in the Azure Storage Java SDK that can cause the results of a list blobs
operation to appear empty. This causes the Filesystem listStatus and similar
APIs to return empty results. This has been seen in Spark work loads when jobs
use more than one executor core.
See Azure/azure-storage-java#546 for details on the bug in the Azure Storage SDK.
TESTS: A new test was added to validate the fix. All tests are passing:
wasb:
mvn -T 1C -Dparallel-tests=wasb -Dscale -DtestsThreadCount=8 clean verify
Tests run: 248, Failures: 0, Errors: 0, Skipped: 11
Tests run: 651, Failures: 0, Errors: 0, Skipped: 65
abfs:
mvn -T 1C -Dparallel-tests=abfs -Dscale -DtestsThreadCount=8 clean verify
Tests run: 64, Failures: 0, Errors: 0, Skipped: 0
Tests run: 437, Failures: 0, Errors: 0, Skipped: 33
Tests run: 206, Failures: 0, Errors: 0, Skipped: 24
2020-06-25 02:32:42 +00:00
9cbd76cc77
HDFS-15346. FedBalance tool implementation. Contributed by Jinglun.
2020-06-18 13:33:25 +08:00
76fa0222f0
HDFS-15330. Document the ViewFSOverloadScheme details in ViewFS guide. Contributed by Uma Maheswara Rao G.
2020-06-05 10:58:21 -07:00
52b21de1d8
HADOOP-17049. javax.activation-api and jakarta.activation-api define overlapping classes ( #2027 )
...
* Removed javax.activation-api from dependency
2020-05-22 11:19:11 +09:00
1958cb7c2b
YARN-10265. Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue. Contributed by liusheng
2020-05-14 15:09:34 +05:30
0918433b4d
YARN-9898. Dependency netty-all-4.1.27.Final doesn't support ARM platform. Contributed by liusheng.
2020-05-14 00:36:20 +05:30
bd342bef64
HADOOP-17033. Update commons-codec from 1.11 to 1.14. ( #2000 )
2020-05-11 08:41:14 -07:00
82ff7bc9ab
HADOOP-16959. Resolve hadoop-cos dependency conflict. Contributed by Yang Yu.
2020-04-20 18:14:11 +08:00
ae474e1ae5
HADOOP-16982. Update Netty to 4.1.48.Final. Contributed by Lisheng Sun.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2020-04-15 06:41:32 +09:00
e53d472bb0
HADOOP-14836. Upgrade maven-clean-plugin to 3.1.0 ( #1933 )
2020-04-09 01:47:31 +09:00
3eeb2466e9
upate the hadoop.version property in the root pom.xml and hadoop.assemblies.version in hadoop-project/pom.xml (see HADOOP-15369)
2020-03-29 23:44:20 +05:30
8914cf9167
Preparing for 3.4.0 development
2020-03-29 23:24:25 +05:30
f02d5abacd
HADOOP-16927. Update hadoop-thirdparty dependency version to 1.0.0 ( #1900 ). Contributed by Vinayakumar B.
2020-03-20 15:13:07 +05:30
cd91ef649e
YARN-10195. Dependency divergence building Timeline Service on HBase 2.2.0 and above. ( #1895 )
2020-03-13 11:20:28 -07:00
c6b8a30386
HADOOP-16871. Upgrade Netty version to 4.1.45.Final to handle CVE-2019-20444,CVE-2019-16869
2020-03-09 19:21:58 +05:30
999096d82e
HADOOP-14206. TestSFTPFileSystem#testFileExists failure: Invalid encoding for signature. Contributed by Jim Brennan.
2020-03-07 22:56:11 -08:00
69faaa1d58
HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from the endless CVE patches. ( #1876 )
2020-03-07 12:18:01 +09:00
e36b272608
HADOOP-16891. Upgrade jackson-databind to 2.9.10.3 ( #1865 )
2020-02-28 17:02:41 -08:00
7f35676f90
HADOOP-16869. Upgrade findbugs-maven-plugin to 3.0.5 to fix mvn findbugs:findbugs failure ( #1855 )
2020-02-21 12:22:16 +09:00
d6d7f8d8c5
YARN-8374. Upgrade objenesis to 2.6 ( #1798 )
2020-02-19 09:50:37 +09:00
7dac7e1d13
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency ( #1635 ). Contributed by Vinayakumar B.
2020-02-07 14:51:24 +05:30
d40d7cc4f9
HADOOP-16670. Stripping Submarine code from Hadoop codebase. Contributed by Zhankun Tang.
...
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-01-21 20:07:31 -08:00
f206b736f0
HADOOP-16346. Stabilize S3A OpenSSL support.
...
Introduces `openssl` as an option for `fs.s3a.ssl.channel.mode`.
The new option is documented and marked as experimental.
For details on how to use this, consult the peformance document
in the s3a documentation.
This patch is the successor to HADOOP-16050 "S3A SSL connections
should use OpenSSL" -which was reverted because of
incompatibilities between the wildfly OpenSSL client and the AWS
HTTPS servers (HADOOP-16347). With the Wildfly release moved up
to 1.0.7.Final (HADOOP-16405) everything should now work.
Related issues:
* HADOOP-15669. ABFS: Improve HTTPS Performance
* HADOOP-16050: S3A SSL connections should use OpenSSL
* HADOOP-16371: Option to disable GCM for SSL connections when running on Java 8
* HADOOP-16405: Upgrade Wildfly Openssl version to 1.0.7.Final
Contributed by Sahil Takiar
Change-Id: I80a4bc5051519f186b7383b2c1cea140be42444e
2020-01-21 16:37:51 +00:00
7f3510048d
HADOOP-16803. Upgrade jackson-databind to 2.9.10.2. ( #1805 )
2020-01-15 19:50:25 +09:00
20a90c0b5b
YARN-10071. Sync Mockito version with other modules
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2020-01-10 17:41:04 +09:00
a40dc9ee31
HADOOP-15993. Upgrade Kafka to 2.4.0 in hadoop-kafka module. ( #1796 )
2020-01-09 16:24:58 +09:00
f1f3f23c3c
HADOOP-16772. Extract version numbers to head of pom.xml (addendum) ( #1773 )
...
Follow up task of HADOOP-16729, extract even more version numbers.
Change-Id: I2aba142657f3978b24be2560ed6161f1132a8f9e
2020-01-08 12:25:01 +01:00
1b04bcc0d9
HADOOP-16784. Update the year to 2020. Contributed by Ayush Saxena.
2020-01-02 21:42:36 +05:30
578bd101a6
HADOOP-16765. Fix curator dependencies for gradle projects using hadoop-minicluster. ( #1763 )
...
Gradle behaves differently than Maven when processing exclude rules through transitive
dependencies. This causes gradle projects (e.g. in Kudu) that are using Hadoop MiniCluster
to fail because different Guava version come to the classpath through Curator. In this
fix we make sure that ZooKeeper, SLF4J and Gradle gets always excluded from Curator Client
even if they would come through other Curator dependencies transitively.
2019-12-16 14:43:59 -08:00
c1d393a156
Bump nimbus-jose-jwt from 4.41.1 to 7.9 ( #1682 )
...
Bumps [nimbus-jose-jwt](https://bitbucket.org/connect2id/nimbus-jose-jwt ) from 4.41.1 to 7.9.
- [Changelog](https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/CHANGELOG.txt )
- [Commits](https://bitbucket.org/connect2id/nimbus-jose-jwt/branches/compare/7.9..4.41.1 )
Signed-off-by: dependabot[bot] <support@github.com>
2019-12-09 10:14:44 +09:00
7c869b4df2
HADOOP-16729. Extract version numbers to head of pom.xml ( #1730 ). Contributed by Tamas Penzes.
2019-12-04 10:34:29 +01:00
0e22e9ab83
YARN-9562. Add Java changes for the new RuncContainerRuntime. Contributed by Eric Badger
2019-11-18 21:56:31 +00:00
a98c4dfc47
HADOOP-16702. Move documentation of hadoop-cos to under src directory.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-11-12 17:47:17 +09:00
b988487cae
HADOOP-16701. Fix broken links in site index.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-11-12 16:43:52 +09:00
012756a0d9
HADOOP-16675. Upgrade jackson-databind to 2.9.10.1. Contributed by Lisheng Sun.
2019-10-29 13:36:30 -07:00
ac6b6a6a85
HADOOP-16614. Add aarch64 support for dependent leveldbjni.
...
Contributed by liusheng
close #1546
2019-10-24 11:45:57 -04:00
6d92aa7c30
HADOOP-16579. Upgrade to Curator 4.2.0 and ZooKeeper 3.5.5 ( #1656 ). Contributed by Norbert Kalmár, Mate Szalay-Beko
...
* HADOOP-16579 - Upgrade to Apache Curator 4.2.0 and ZooKeeper 3.5.5
- Add a static initializer for the unit tests using ZooKeeper to enable
the four-letter-words diagnostic telnet commands. (this is an interface
that become disabled by default, so to keep the ZooKeeper 3.4.x behavior
we enabled it for the tests)
- Also fix ZKFailoverController to look for relevant fail-over ActiveAttempt
records. The new ZooKeeper seems to respond quicker during the fail-over
tests than the ZooKeeper, so we made sure to catch all the relevant records
by adding a new parameter to ZKFailoverontroller.waitForActiveAttempt().
Co-authored-by: Norbert Kalmár <nkalmar@cloudera.com>
2019-10-18 13:26:20 -07:00
3d41f33018
HADOOP-16152. Upgrade Eclipse Jetty version to 9.4.x. Contributed by Yuming Wang, Siyao Meng.
...
Co-authored-By: Siyao Meng <smeng@cloudera.com>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-10-18 08:25:02 -07:00
85af77c757
HADOOP-16643. Update netty4 to the latest 4.1.42. Contributed by Lisheng Sun.
2019-10-15 08:55:14 -07:00
55c5436f39
Revert "HADOOP-16579. Upgrade to Apache Curator 4.2.0 excluding ZK ( #1531 ). Contributed by Norbert Kalmár."
...
This reverts commit 10bdc592d6
2019-10-05 11:16:11 -07:00
10bdc592d6
HADOOP-16579. Upgrade to Apache Curator 4.2.0 excluding ZK ( #1531 ). Contributed by Norbert Kalmár.
2019-10-04 10:57:07 -07:00
d947ded053
HADOOP-16619. Upgrade jackson and jackson-databind to 2.9.10 ( #1554 ). Contributed by Siyao Meng.
2019-10-01 12:46:40 -07:00
8a9ede5cff
HADOOP-15616. Incorporate Tencent Cloud COS File System Implementation. Contributed by Yang Yu.
2019-09-27 16:53:13 +08:00
07c81e9bfc
HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes ( #1494 ). Contributed by Vinayakumar B.
2019-09-23 12:37:47 +05:30
efed4450bf
HADOOP-16589. [pb-upgrade] Update docker image to make 3.7.1 protoc as default ( #1482 ). Contributed by Vinayakumar B.
2019-09-21 10:07:26 +05:30
1654497f98
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1 ( #1432 )
...
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1. Contributed by Vinayakumar B.
2019-09-20 16:08:30 +05:30
1843c4688a
HADOOP-16555. Update commons-compress to 1.19. ( #1425 ) Contributed by YiSheng Lien.
2019-09-14 02:11:04 +08:00
38c1a10024
HADOOP-16542. Update commons-beanutils version to 1.9.4. Contributed by kevin su.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-09-10 19:58:34 +08:00
d85d68f6ff
HADOOP-16533. Upgrade jackson-databind to 2.9.9.3. ( #1354 ). Contributed by Akira Ajisaka.
...
Reviewed-by: Siyao Meng <smeng@cloudera.com>
2019-08-27 15:54:35 -07:00
3cc0ace203
HADOOP-16491. Upgrade jetty version to 9.3.27. Contributed by Hrishikesh Gadre.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-07 13:50:14 -07:00
0b0ba70b35
HADOOP-15908. hadoop-build-tools jar is downloaded from remote repository instead of using from local. Contributed by Oleksandr Shevchenko.
2019-08-07 09:29:18 +09:00
b77761b0e3
HADOOP-16275. Upgrade Mockito to the latest version. Contributed by kevin su.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-06 12:18:24 -07:00
9680a8b237
HADOOP-16487. Update jackson-databind to 2.9.9.2. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-04 20:03:07 -07:00
12a526c080
HADOOP-16460: ABFS: fix for Sever Name Indication (SNI)
...
Contributed by Sneha Vijayarajan <snvijaya@microsoft.com>
2019-07-30 17:23:23 +00:00
98653ecccb
HADOOP-16447 Upgrade JUnit5 from 5.3.1 to 5.5.1 to support global timeout
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-07-26 17:05:34 +09:00
9b8b3acb0a
HADOOP-16451. Update jackson-databind to 2.9.9.1. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-07-24 17:25:25 -07:00
5043840b1d
HDFS-14410. Make Dynamometer documentation properly compile onto the Hadoop site. Contributed by Erik Krogen.
2019-07-11 23:47:27 +08:00
062eb605ac
YARN-9477. Implement VE discovery using libudev. Contributed by Peter Bacsko.
2019-06-26 23:53:14 +08:00
48d7f00da8
HDFS-14590. [SBN Read] Add the document link to the top page. Contributed by Takanobu Asanuma.
2019-06-25 19:11:09 +09:00
eb6be4643f
YARN-9630. [UI2] Add a link in docs's top page
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-06-18 14:54:21 +09:00
cf84881dea
HADOOP-16365. Upgrade jackson-databind to 2.9.9. Contributed by Shweta Yakkali.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-12 10:37:58 -07:00
ec26c431f9
HADOOP-16117. Update AWS SDK to 1.11.563.
...
Contributed by Steve Loughran.
Change-Id: I7c46ed2a6378e1370f567acf4cdcfeb93e43fa13
2019-06-06 10:08:18 +01:00
afd844059c
HADOOP-16331. Fix ASF License check in pom.xml
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:25:13 +09:00
9f933e6446
HADOOP-16323. https everywhere in Maven settings.
2019-05-27 15:24:59 +09:00
f257497b0f
HADOOP-16299. [JDK 11] Build fails without specifying -Djavac.version=11
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-09 14:49:46 +09:00
66c2a4ef89
HADOOP-16270. [JDK 11] Remove unintentional override of the version of Maven Dependency Plugin. Contributed by Xieming Li.
2019-05-08 10:50:07 +09:00
a54c1e3ace
YARN-8551. Project setup for MaWo application.
...
Contributed by Yesha Vora
2019-04-22 16:56:26 -04:00
a5ceed26f3
HADOOP-16253. Update AssertJ to 3.12.2.
2019-04-16 12:53:52 +09:00
65deb1ac42
HADOOP-16179. hadoop-common pom should not depend on kerb-simplekdc
...
The hadoop-common pom currently has a dependency on kerb-simplekdc. In
fact, the only classes used from Kerby are in kerb-core and kerb-util
(which is a transitive dependency frmo kerb-core). Depending on
kerb-simplekdc pulls a bunch of other unnecessary classes into the
hadoop-common classpath.
This changes the hadoop-common pom to depend only on kerb-core.
hadoop-minikdc already had the appropriate dependency on kerb-simplekdc
so it continues to pull in what it needs.
Signed-off-by: Todd Lipcon <todd@apache.org>
2019-04-10 08:49:46 -07:00
d7979079ea
HADOOP-16210. Update guava to 27.0-jre in hadoop-project trunk. Contributed by Gabor Bota.
2019-04-03 12:59:39 -06:00
7ee32a2eb1
YARN-7129. Application Catalog for YARN applications. Contributed by Eric Yang
2019-03-29 13:51:21 -07:00
ab574ffd72
Revert "YARN-7129. Application Catalog for YARN applications. Contributed by Eric Yang"
...
This reverts commit 2576401928
2019-03-12 11:50:16 -07:00
2576401928
YARN-7129. Application Catalog for YARN applications. Contributed by Eric Yang
2019-03-05 06:14:14 -08:00
5b43e42d0c
HADOOP-16149 hadoop-mapreduce-client-app build not converging due to transient dependencies
...
Change-Id: If95b12b223770b057041f99f0f8fd8ba370c377f
2019-02-27 12:44:46 -08:00
7b11b404a3
HADOOP-16097. Provide proper documentation for FairCallQueue. Contributed by Erik Krogen.
2019-02-13 11:16:04 +08:00
529791cb18
HADOOP-15938. [JDK 11] Remove animal-sniffer-maven-plugin to fix build. Contributed by Dinesh Chitlangia.
2019-02-05 09:42:44 +09:00
758e9ce781
HADOOP-16076. SPNEGO+SSL Client Connections with HttpClient Broken.
...
Contributed by Larry McCay
2019-02-04 17:38:40 +00:00
0f9aa5b4d5
HADOOP-16089. AliyunOSS: update oss-sdk version to 3.4.1. Contributed by wujinhu.
2019-02-04 14:46:35 +09:00
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki.
2019-01-29 18:29:56 -08:00
01405df87f
HADOOP-15941. Addendum patch. Contributed by Takanobu Asanuma.
2019-01-16 10:28:08 +09:00
6d0bffe17e
HADOOP-14556. S3A to support Delegation Tokens.
...
Contributed by Steve Loughran and Daryn Sharp.
2019-01-14 17:59:27 +00:00
3bb745df18
HADOOP-15994. Upgrade Jackson2 to 2.9.8. Contributed by lqjacklee.
2019-01-12 15:23:49 +09:00
709ddb10e5
HADOOP-15941. [JDK 11] Compilation failure: package com.sun.jndi.ldap is not visible.
2019-01-09 13:55:25 +09:00
7f78397036
Revert "HADOOP-14556. S3A to support Delegation Tokens."
...
This reverts commit d7152332b3
2019-01-08 14:51:30 +09:00
d7152332b3
HADOOP-14556. S3A to support Delegation Tokens.
...
Contributed by Steve Loughran.
2019-01-07 13:18:03 +00:00
32d5caa0a8
HADOOP-15937. [JDK 11] Update maven-shade-plugin.version to 3.2.1. Contributed by Dinesh Chitlangia.
2019-01-07 19:17:23 +09:00
1f425271a7
Revert "HADOOP-15759. AliyunOSS: Update oss-sdk version to 3.0.0. Contributed by Jinhu Wu."
...
This reverts commit e4fca6aae42b635125fb
2019-01-05 17:35:29 +09:00
cb26f15428
HADOOP-16025. Update the year to 2019.
...
Contributed by Ayush Saxena.
2019-01-02 22:24:01 +00:00
eee29ed706
HADOOP-16009. Replace the url of the repository in Apache Hadoop source code.
2018-12-31 06:28:24 +09:00
9287ab3642
HADOOP-15974. Upgrade Curator version to 2.13.0 to fix ZK tests. Contributed by Akira Ajisaka
2018-12-04 15:44:03 -06:00
ff31313d83
HADOOP-15970. Upgrade plexus-utils from 2.0.5 to 3.1.0.
2018-12-04 22:45:29 +09:00
f63e4e4f2e
YARN-8936. Bump up Atsv2 hbase versions. Contributed by Vrushali C.
2018-11-21 15:02:22 +05:30
27ffec7ba7
YARN-8778. Add command line interface to invoke interactive docker shell. Contributed by Eric Yang
2018-11-19 17:59:12 -08:00
a736b5da15
HADOOP-15916. Upgrade Maven Surefire plugin to 3.0.0-M1.
2018-11-10 00:24:56 +09:00
78e43bccf6
HADOOP-15902. [JDK 11] Specify the HTML version of Javadoc to 4.01. Contributed by Takanobu Asanuma.
2018-11-06 20:44:05 +09:00
d174b91635
HADOOP-15895. [JDK9+] Add missing javax.annotation-api dependency to hadoop-yarn-csi.
...
Contributed by Takanobu Asanuma.
2018-11-01 12:56:20 +00:00
e28c00c290
HADOOP-15815. Upgrade Eclipse Jetty version to 9.3.24. Cotributed by Boris Vulikh.
2018-10-26 13:46:15 +05:30
a4b9b7c130
HDDS-702:Used fixed/external version from hadoop jars in hdds/ozone projects. Contributed by Elek Marton.
2018-10-25 16:26:48 -07:00
34b2521f5e
HADOOP-15882. Upgrade maven-shade-plugin from 2.4.3 to 3.2.0. Contributed by Takanobu Asanuma.
2018-10-25 13:29:59 -07:00
ace06a93ba
HDDS-714. Bump protobuf version to 3.5.1. Contributed by Mukul Kumar Singh.
2018-10-25 09:34:06 +05:30
Thomas Marquardt
Yiqun Lin
Uma Maheswara Rao G
Akira Ajisaka
Surendra Singh Lilhore
Ayush Saxena
Wei-Chiu Chuang
Sammi Chen
sunlisheng
Brahma Reddy Battula
Vinayakumar B
John Zhuge
Siyao Meng
Zhankun Tang
Sahil Takiar
Masatake Iwasaki
Adam Antal
Tamás Pénzes
Mate Szalay-Beko
dependabot[bot]
Tamás Pénzes
Eric Badger
Eric Yang
Yuming Wang
Norbert Kalmar
Yi Sheng
HUAN-PING SU
Hrishikesh Gadre
Siyao Meng
Sneha Vijayarajan
Wanqiang Ji
Shweta Yakkali
Steve Loughran
Takanobu Asanuma
Todd Lipcon
Gabor Bota
Billie Rinaldi
Steve Loughran
Larry McCay
Sunil G
Ayush Saxena
Jason Lowe
Rohith Sharma K S
Bharat Viswanadham
Mukul Kumar Singh