Commit Graph

749 Commits

Author SHA1 Message Date
PJ Fanning f856611121 HADOOP-18587: upgrade to jettison 1.5.3 due to cve (#5270)
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
(cherry picked from commit b9eb760ed2)
2023-01-06 23:41:18 +00:00
Ayush Saxena f63f20259b
HADOOP-18586. Update the year to 2023. (#5265). Contributed by Ayush Saxena.
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
2023-01-01 22:45:23 +05:30
Steve Loughran cda1d45a61
HADOOP-18470. Update index md with section on ABFS prefetching 2022-12-19 13:03:57 +00:00
Steve Loughran 223046cb64
HADOOP-18561. Update commons-net to 3.9.0 (#5214)
Addresses CVE-2021-37533, which *only* relates to FTP.

Applications not using the ftp:// filesystem, which, as
anyone who has used it will know is very minimal and
so rarely used, is not a critical part of the project.

Furthermore, the FTP-related issue is at worst information leakage
if someone connects to a malicious server.

This is a due diligence PR rather than an emergency fix.

Contributed by Steve Loughran
2022-12-19 11:57:47 +00:00
Steve Loughran 36889005f7
HADOOP-18470. index.md update for 3.3.5 release 2022-12-05 16:22:40 +00:00
Melissa You 853ffb545a
HADOOP-18515. Backport HADOOP-17612 to branch-3.3(Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0) (#5097)
* HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 (#3241)

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Co-authored-by: Viraj Jasani <vjasani@apache.org>
Co-authored-by: Melissa You <myou@myou-mn1.linkedin.biz>
2022-11-05 09:28:24 -07:00
Ashutosh Gupta 7b84f6458b
HADOOP-18484. Upgrade hsqldb to v2.7.1 to mitigate CVE-2022-41853 (#5101) 2022-11-04 11:00:17 +01:00
PJ Fanning d88a6ee962
HADOOP-18512: upgrade woodstox-core to 5.4.0 for security fix (#5087). Contributed by PJ Fanning.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-11-02 00:14:01 +05:30
PJ Fanning 41e3c7edaf
HADOOP-18472. Upgrade to snakeyaml 1.33 (#4958)
Reviewed-by: Dinesh Chitlangia <dineshc@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit d6a65a4180)

 Conflicts:
	LICENSE-binary
	hadoop-project/pom.xml
2022-10-30 02:32:44 +09:00
PJ Fanning ea851c5e4a
HADOOP-15983. Use jersey-json that is built to use jackson2 ((#3988)
Moves from com.sun.jersey 1.19 to the artifact
com.github.pjfanning:jersey-json:1.20

This allows jackson 1 to be removed from the classpath.

Contains

* HADOOP-16908. Prune Jackson 1 from the codebase and restrict
   its usage for future
* HADOOP-18219. Fix shaded client test failure

These are needed for the HADOOP-15983 changes to build.

Contributed by PJ Fanning.
2022-10-20 17:37:56 +01:00
Hexiaoqiao 84c7fd909b
HADOOP-18497. Upgrade commons-text version to 1.10.0 to fix CVE-2022-42889. (#5037).
Contributed by PJ Fanning.
2022-10-18 15:05:08 +01:00
slfan1989 2e3f91bdf5
HADOOP-18360. Update commons-csv from 1.0 to 1.9.0. (#4928). Contributed by fanshilun.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-10-17 10:23:13 +05:30
PJ Fanning 96d4b9e6a7
HADOOP-18493: upgrade jackson-databind to 2.12.7.1 (#5011). Contributed by PJ Fanning.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-10-17 10:04:21 +05:30
Steve Loughran cd856b7195
HADOOP-17563. Upgrade BouncyCastle to 1.68 (#3980) (#5015)
Addresses CVE-2020-15522 and CVE-2020-26939.

This can break builds with older maven shade plugins or
other code using asm.jar which is not aware of recent java bytecodes
and/or multi-release JARs. fix: use a later version of asm.jar

Contributed by PJ Fanning
2022-10-15 15:09:05 +01:00
Steve Loughran 80525615e5
HADOOP-18480. Upgrade aws sdk to 1.12.316 (#4972)
Contributed by Steve Loughran
2022-10-10 10:29:41 +01:00
Steve Loughran e360e7620c
HADOOP-18468: Upgrade jettison to 1.5.1 to fix CVE-2022-40149 (#4937)
Contributed by PJ Fanning
2022-10-10 10:05:39 +01:00
Steve Loughran c70b8709cc
HADOOP-18442. Remove openstack support (#4855)
The swift:// connector for openstack support has been removed.
The hadoop-openstack jar remains, only now it is empty of code. 
This is to ensure that projects which declare the JAR a dependency
will still have successful builds.

Contributed by Steve Loughran
2022-10-07 12:03:08 +01:00
Ashutosh Gupta 51605f9dcc
HADOOP-18443. Upgrade snakeyaml to 1.32 (#4873)
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-25 23:50:46 +09:00
PJ Fanning d66dea300e
HADOOP-18341: upgrade commons-configuration2 to 2.8.0 and commons-text to 1.9 (#4916) 2022-09-22 10:44:27 +09:00
Ayush Saxena 9890a4aea4
Revert "HADOOP-18417. Upgrade to M7 of surefire plugin (#4795)"
This reverts commit 1ff121041c.
2022-08-25 03:53:34 +05:30
Steve Vaughan 98dd2b534f
HADOOP-18417. Upgrade to M7 of surefire plugin (#4795)
This addresses an issue where the plugin's default classpath
for executing tests fails to include
org.junit.platform.launcher.core.LauncherFactory.

Contributed by: Steve Vaughan Jr
2022-08-24 11:07:34 +01:00
Wei-Chiu Chuang c4d94f5623
HADOOP-18333. Upgrade jetty version to 9.4.48.v20220622 (#4600)
* HADOOP-18001. Upgrade jetty version to 9.4.44 (#3700). Contributed by Yuan Luo.

Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit b85c66a035)

* HADOOP-18333.Upgrade jetty version to 9.4.48.v20220622 (#4553)

Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
(cherry picked from commit e664f81ce7)

 Conflicts:
	LICENSE-binary

Change-Id: I5a758df2551539c2780e170c3738c5b21eb0c79d

Co-authored-by: better3471 <46600375+better3471@users.noreply.github.com>
Co-authored-by: Ashutosh Gupta <ashutosh.gupta@st.niituniversity.in>
2022-08-24 08:16:49 +08:00
Steve Loughran 7aebacef77 HADOOP-18344. Upgrade AWS SDK to 1.12.262 (#4637)
Fixes CVE-2018-7489 in shaded jackson.

+Add more commands in testing.md
 to the CLI tests needed when qualifying
 a release

Contributed by Steve Loughran
2022-07-28 11:39:40 +01:00
Wei-Chiu Chuang 0c12873487
HADOOP-18079. Upgrade Netty to 4.1.77. (#3977) (#4592)
Upgrade netty to address

CVE-2019-20444,
CVE-2019-20445
CVE-2022-24823

Contributed by Wei-Chiu Chuang

(cherry picked from commit a55ace7bc0)
2022-07-27 03:10:20 +08:00
PJ Fanning 36cb8a6a2b
HADOOP-18354. Upgrade reload4j to 1.22.2 due to XXE vulnerability (#4607). Contributed by PJ Fanning.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-07-24 16:01:47 +05:30
PJ Fanning 6733ba56b8
HADOOP-18332. Remove rs-api dependency by downgrading jackson to 2.12.7. (#4552)
This downgrades jackson from the version switched to in 
HADOOP-18033 (2.13.0), to Jackson 2.12.7.
This removes the dependency on javax.ws.rs-api,
so avoiding runtime problems with applications using
jersey-core v1 and/or jsr311-api.

The 2.12.7 release still contains the fix for CVE-2020-36518.

Contributed by PJ Fanning
2022-07-16 18:18:52 +01:00
Mukund Thakur 5c348c41ab HADOOP-11867. Add a high-performance vectored read API. (#3904)
part of HADOOP-18103.
Add support for multiple ranged vectored read api in PositionedReadable.
The default iterates through the ranges to read each synchronously,
but the intent is that FSDataInputStream subclasses can make more
efficient readers especially in object stores implementation.

Also added implementation in S3A where smaller ranges are merged and
sliced byte buffers are returned to the readers. All the merged ranged are
fetched from S3 asynchronously.

Contributed By: Owen O'Malley and Mukund Thakur

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
	pom.xml
2022-06-23 17:09:16 -05:00
Igor Dvorzhak d41e0a9cc3 HADOOP-18300. Upgrade Gson dependency to version 2.9.0 (#4454)
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
(cherry picked from commit 77d1b194c7)
2022-06-22 23:42:59 +00:00
Steve Loughran 9ca4ac0af0
HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482)
Updating the hadoop version of branch-3.3 to 3.3.9-SNAPSHOT
pending agreement on what number its future release should take.

Using 3.3.9-SNAPSHOT puts space in for other incremental releases,
while avoiding creating JIRA release ordering and autocompletion
confusion the way adding a 3.3.10 or higher version would do.

Contributed by Steve Loughran
2022-06-22 13:09:50 +01:00
Steve Loughran 03c2941d4b
HADOOP-18275. Update os-maven-plugin to 1.7.0 (#4397)
Contributed by Steve Loughran

Change-Id: Ic4d442a37299dc8098b0bca3cc51beca6f058283
2022-06-06 13:20:00 +01:00
Ashutosh Gupta 57fe613299
HDFS-16453. Upgrade okhttp from 2.7.5 to 4.9.3 (#4229)
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit fb910bd906)

 Conflicts:
	hadoop-project/pom.xml
2022-05-21 03:17:15 +09:00
Ashutosh Gupta b0012ee578 HADOOP-18237. Upgrade Apache Xerces Java to 2.12.2 (#4318)
Upgrade Apache Xerces Java to 2.12.2 due to handle vulnerability CVE-2022-23437

Contributed by Ashutosh Gupta
2022-05-17 20:37:30 +01:00
Steve Loughran caecec45f5
HADOOP-17650. Bump solr to unblock build failure with Maven 3.8.1 (#2939)
Reviewed-by: Siyao Meng <siyao@apache.org>

Contributed by Viraj Jasani
2022-04-20 16:36:51 +01:00
Dongjoon Hyun af3558d61a
HADOOP-17341. Upgrade commons-codec to 1.15 (#2428)
Change-Id: Iab26db901570b507ab25ddbf316a9579a9e92620
Reviewed-by: Chao Sun <sunchao@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
2022-04-20 12:29:00 +01:00
Steve Loughran 44e662272f
HADOOP-18198. Preparing for 3.3.4 development
Change-Id: I2bf19beb541739af22fced38c2545f09c4e1bd53
2022-04-12 14:09:08 +01:00
Akira Ajisaka 603367c54f
HADOOP-18178. Upgrade jackson to 2.13.2 and jackson-databind to 2.13.2.2 (#4147)
(cherry picked from commit 4b786c797a)

 Conflicts:
	LICENSE-binary

Co-authored-by: PJ Fanning <pjfanning@users.noreply.github.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-04-11 14:58:28 +09:00
Masatake Iwasaki 160b6d106d
HADOOP-18088. Replace log4j 1.x with reload4j. (#4052)
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
2022-04-07 08:33:13 +09:00
Steve Loughran 3238bdab89
HADOOP-18163. hadoop-azure support for the Manifest Committer of MAPREDUCE-7341
Follow-on patch to MAPREDUCE-7341, adding ABFS support and tests

* resilient rename
* tests for job commit through the manifest committer.

contains
- HADOOP-17976. ABFS etag extraction inconsistent between LIST and HEAD calls
- HADOOP-16204. ABFS tests to include terasort

Contributed by Steve Loughran.

Change-Id: I0a7d4043bdf19bcb00c033fc389730109b93b77f
2022-03-17 11:47:15 +00:00
PJ Fanning a302a19b48 HADOOP-18126. update junit 5 version due to build issues (#3993)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 5f6a294fab)
2022-02-17 14:07:57 +09:00
Akira Ajisaka 8032b680fb YARN-10561. Upgrade node.js to 12.22.1 and yarn to 1.22.5 in YARN application catalog webapp (#2591)
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 9cb535caf2)
2022-01-28 15:52:33 +09:00
Steve Loughran 8ccc586af6
HADOOP-17409. Remove s3guard from S3A module (#3534)
Completely removes S3Guard support from the S3A codebase.

If the connector is configured to use any metastore other than
the null and local stores (i.e. DynamoDB is selected) the s3a client
will raise an exception and refuse to initialize.

This is to ensure that there is no mix of S3Guard enabled and disabled
deployments with the same configuration but different hadoop releases
-it must be turned off completely.

The "hadoop s3guard" command has been retained -but the supported
subcommands have been reduced to those which are not purely S3Guard
related: "bucket-info" and "uploads".

This is major change in terms of the number of files
changed; before cherry picking subsequent s3a patches into
older releases, this patch will probably need backporting
first.

Goodbye S3Guard, your work is done. Time to die.

Contributed by Steve Loughran.
2022-01-18 18:04:48 +00:00
Steve Loughran 47ba977ca9
HADOOP-18068. upgrade AWS SDK to 1.12.132 (#3864)
With this update, the versions of key shaded dependencies are

  jackson    2.12.3
  httpclient 4.5.13

This backport patch does not include the TestArn changes needed
for the test to work with this version of the SDK; it is only
to be applied to branches without HADOOP-17198. "Support S3 Access Points".
If that patch is backported later, that test suite MUST be
updated to the latest version.

Contributed by Steve Loughran

Change-Id: I8d2b71781ee8472b16469531f9cd0de32dd3356f
2022-01-18 12:20:12 +00:00
Ayush Saxena 53249a40db
HADOOP-18061. Update the year to 2022. (#3845). Contributed by Ayush Saxena.
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
2022-01-04 07:59:45 +05:30
Igor Dvorzhak 5d72fdfcb2 HADOOP-13464. Upgrade Gson dependency to version 2.8.9 (#2524)
Change-Id: Ifd3fb9ec6ebfc8874bb799bc198219511fe55a2f

Update pom.xml

Update pom.xml

(cherry picked from commit 795054882a)
2021-12-30 21:37:14 +00:00
Viraj Jasani b0c1158829
HADOOP-18033. Upgrade fasterxml Jackson to 2.13.0 (#3764)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-12-13 13:52:44 +09:00
Akira Ajisaka 35c5c6bb83 HADOOP-18040. Use maven.test.failure.ignore instead of ignoreTestFailure (#3774)
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 9b9e2ef87f)

 Conflicts:
	hadoop-tools/hadoop-federation-balance/pom.xml
2021-12-10 01:38:26 +09:00
Chao Sun e079fa6577 Preparing for 3.3.3 development 2021-11-16 16:02:34 -08:00
Renukaprasad C 3bb4a09295
HADOOP-17946. Upgrade commons-lang to 3.12.0 (#3575)
(cherry picked from commit b923fa7a1c)
2021-11-16 22:59:25 +08:00
Masatake Iwasaki 9e2936f8d1
HADOOP-17424. Replace HTrace with No-Op tracer (#3520)
(cherry picked from commit 1a205cc3ad)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/tracing/TestTracing.java

Co-authored-by: Siyao Meng <50227127+smengcl@users.noreply.github.com>
2021-10-12 00:07:09 +09:00
Takanobu Asanuma 038fa67db4 HADOOP-17955. Bump netty to the latest 4.1.68. (#3528)
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit cf4d2d826c)
2021-10-08 10:37:14 +09:00