Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
24,467 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

24467 Commits

Author SHA1 Message Date
Wei-Chiu Chuang a3b9c37a39
Revert "HADOOP-16878. FileUtil.copy() to throw IOException if the source and destination are the same (#2383)" 
This reverts commit 54c40cbf49.
 2021-06-01 16:37:28 +08:00
Wei-Chiu Chuang 1067920757
Revert "MAPREDUCE-7303. Fix TestJobResourceUploader failures after HADOOP-16878. Contributed by Peter Bacsko." 
This reverts commit c40f0f1eb3.
 2021-06-01 16:37:10 +08:00
Wei-Chiu Chuang b2d1449986
HADOOP-17739. Use hadoop-thirdparty 1.1.1. (#3064) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9a0a808338)
(cherry picked from commit 5cebd02bbe)
 2021-06-01 11:36:41 +08:00
Takanobu Asanuma 30fb36fc25
Revert "HADOOP-17563. Update Bouncy Castle to 1.68. (#2740)" (#3055) 
This reverts commit 0774116756.

Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 53ff2dfed3)
(cherry picked from commit d8689f1a08)
 2021-06-01 10:19:24 +08:00
Akira Ajisaka 4ea7d39466
MAPREDUCE-7348. TestFrameworkUploader#testNativeIO fails. (#3053) 
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit 8a489ce78e)
(cherry picked from commit e14d00469a)
 2021-05-26 17:48:52 +08:00
Wei-Chiu Chuang 3c960ac548
HADOOP-17666. Update LICENSE for 3.3.1 (#3011) 
* Inspected the jar files in the produced tarball and updated LICENSE-binary accordingly.

* add LICENSE from hadoop-thirdparty jars.
* remove any dependencies no longer in the tarball.
* Updated the license of thirdparty javascripts and C/C++ files.

Added LICENSE-asio.txt, copied from hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/third_party/asio-1.10.2/COPYING
Added LICENSE-gmock.txt, copied from hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/third_party/gmock-1.7.0/LICENSE
Added LICENSE-rapidxml.txt, copied from hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/third_party/rapidxml-1.13/rapidxml/license.txt
Added LICENSE-uriparser2.txt, copied from hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/third_party/uriparser2/uriparser2/uriparser/COPYING
Added LICENSE-tr2.txt, copied from the header of hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/third_party/uriparser2/uriparser2/tr2/optional.hpp
Added LICENSE-cJSON.txt, moved from the bottom of LICENSE.txt

* Generated license report for yarn-managed packages.
* Add LICENSE and NOTICES file of jaxb-api.
* Exclude LICENSE-binary-{yarn-applications-catalog-webapp|yarn-ui} from rat report.
These two files are autogenerated.

(cherry picked from commit 526dbe4716)
 2021-05-26 17:25:57 +08:00
Wei-Chiu Chuang 1aaec6944e
HADOOP-17723. [build] fix the Dockerfile for ARM (#3037) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 1576f81dfe)
(cherry picked from commit 745a2f5ec3)
 2021-05-24 18:20:56 +08:00
Vinayakumar B d77bbdb47d
HDFS-15790. Make ProtobufRpcEngineProtos and ProtobufRpcEngineProtos2 Co-Exist (#2767) 
(cherry picked from commit 2bbeae3240)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngine.java
 2021-05-24 18:11:36 +08:00
Wei-Chiu Chuang baf385e77e
Revert "HADOOP-17669. Backport HADOOP-17079, HADOOP-17505 to branch-3.3 (#2959)" 
This reverts commit 4ffe5eb1dd.
 2021-05-24 17:37:31 +08:00
Yiqun Lin 27096212b9
HDFS-15660. StorageTypeProto is not compatiable between 3.x and 2.6. Contributed by Ryan Wu. 
(cherry picked from commit da1ea2530f)
(cherry picked from commit ca52d9946b)
 2021-05-24 10:49:04 +08:00
Gautham B A 66bca98db0
HADOOP-17509. Parallelize building of dependencies (#2669) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 6c891c0dd4)
(cherry picked from commit 23ada04dac)
 2021-05-22 01:32:47 +08:00
Wei-Chiu Chuang bcd9fa64a9
HADOOP-17718. Explicitly set locale in the Dockerfile. (#3034) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ad923ad564)

Conflicts:
	dev-support/docker/Dockerfile_centos_7
	dev-support/docker/Dockerfile_centos_8
(cherry picked from commit 5e09d38b25)
 2021-05-22 00:33:30 +08:00
Akira Ajisaka 72ad7d6538
HADOOP-17663. Remove useless property hadoop.assemblies.version in pom file. (#3025) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3f5a66c158)

 Conflicts:
	hadoop-project/pom.xml

(cherry picked from commit 86682a83aa)
 2021-05-20 10:50:47 +09:00
Wei-Chiu Chuang f4018a1f41
Preparing for 3.3.1 release. 2021-05-19 21:56:22 +08:00
zhuqi-lucas 7d2eeaecc8 YARN-10701. The yarn.resource-types should support multi types without trimmed. Contributed by Qi Zhu. 2021-05-19 21:24:26 +08:00
Peter Bacsko a45d6437f5 YARN-10120. In Federation Router Nodes/Applications/About pages throws 500 exception when https is enabled. Contributed by Bilwa S T. 2021-05-19 15:07:47 +02:00
Takanobu Asanuma 932a16837a
HADOOP-17224. Install Intel ISA-L library in Dockerfile. (#2537) 
(cherry picked from commit d09e3c929f)
 2021-05-19 20:27:22 +08:00
Peter Bacsko 864b710787 YARN-10258. Add metrics for 'ApplicationsRunning' in NodeManager. Contributed by ANANDA G B. 2021-05-19 10:31:57 +02:00
Wei-Chiu Chuang 611ed04aff
HADOOP-17426. Upgrade to hadoop-thirdparty-1.1.0. (#3024) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>

Change-Id: Ibf27246b3eb1b074b938f44f4906db9184466303
(cherry picked from commit 2960d83c25)
 2021-05-19 14:57:45 +09:00
Wei-Chiu Chuang 63392376c9
HDFS-16027. Replace abstract methods with default methods in JournalNodeMXBean. (#3018) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-05-18 19:49:00 -07:00
Peter Bacsko 27749ce85d YARN-9615. addendum commit, removing accedentially committed patch file 2021-05-18 22:17:20 +02:00
Wei-Chiu Chuang 0df4743284
HADOOP-17703. checkcompatibility.py errors out when specifying annotations. (#3017) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 110cda3de6)
 2021-05-18 11:36:13 +08:00
Wei-Chiu Chuang 4ffe5eb1dd
HADOOP-17669. Backport HADOOP-17079, HADOOP-17505 to branch-3.3 (#2959) 
* HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet.

Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
Change-Id: I0f31409923ece24a82dfba4c4610d8a38c52d9fb

* HADOOP-17505. public interface GroupMappingServiceProvider needs default impl for getGroupsSet() (#2661). Contributed by Vinayakumar B.

(cherry picked from commit c4c0683dff)

Co-authored-by: Xiaoyu Yao <xyao@apache.org>
Co-authored-by: Vinayakumar B <vinayakumarb@apache.org>
 2021-05-17 18:57:46 -07:00
lujiefsi 137e20cc9b
YARN-10555. Missing access check before getAppAttempts (#2608) 
Co-authored-by: lujie <lujie@foxmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit d92a25b790)
 2021-05-17 13:53:27 +09:00
litao 2f56e2b54d
HDFS-15873. Add namenode address in logs for block report (#2743) 
(cherry picked from commit 6a922f9896)
 2021-05-17 12:29:11 +08:00
Xiaoyu Yao 3f9c9ccf46
HADOOP-17284. Support BCFKS keystores for Hadoop Credential Provider.… (#3010) 
* HADOOP-17284. Support BCFKS keystores for Hadoop Credential Provider. (#2334)

(cherry picked from commit 4c5ad57818)
 2021-05-13 16:57:58 -07:00
Wei-Chiu Chuang 1b8eb2654a
HADOOP-16822. Provide source artifacts for hadoop-client-api. Contributed by Karel Kolman. 
(cherry picked from commit 2c4ab72a60)
 2021-05-13 20:30:40 +08:00
Aryan Gupta 7142dfec04
HADOOP-17302. Upgrade to jQuery 3.5.1 in hadoop-sls. (#2379) 
Co-authored-by: Aryan Gupta
(cherry picked from commit d60d5fe43d)
 2021-05-13 12:16:24 +08:00
Aryan Gupta 5f147112cf
HADOOP-17286. Updated jQuery in yarn-common. (#2340) 
(cherry picked from commit bbbfa7d415)
 2021-05-13 12:16:20 +08:00
Aryan Gupta 28079e9c30
HADOOP-17283. Hadoop - Upgrade to jQuery 3.5.1 (#2330) 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 486ddb73f9)
 2021-05-13 12:16:17 +08:00
Ayush Saxena 41b39ff292
HDFS-15245. Improve JournalNode web UI. Contributed by Jianfei Jiang. 
(cherry picked from commit 960c9ebaea)

Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JournalNode.java
 2021-05-13 12:14:09 +08:00
bilaharith 1c6c3920b6
HADOOP-17444. ADLS Gen1: Update adls SDK to 2.3.9 (#2842) 
Contributed by bilaharith
 2021-05-12 18:15:27 +01:00
dependabot[bot] b2897fdd66
HADOOP-17683. Update commons-io to 2.8.0 (#2974) 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 29105ffb63)
 2021-05-12 10:58:39 +09:00
Peter Bacsko 051a5068dd YARN-9615. Add dispatcher metrics to RM. Contributed by Jonathan Hung and Qi Zhu. 2021-05-11 19:23:45 +02:00
Viraj Jasani 3176ab028b
HDFS-16007. Deserialization of ReplicaState should avoid throwing ArrayIndexOutOfBoundsException (#2982) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit b944084b32)
 2021-05-11 12:39:42 +09:00
Borislav Iordanov f5ef78d46f HADOOP-17665 Ignore missing keystore configuration in reloading mechanism 2021-05-10 15:41:56 -05:00
Ayush Saxena 43db94c7a3 HDFS-16003. ProcessReport print invalidatedBlocks should judge debug level at first. Contributed by lei w. 2021-05-10 17:11:09 +05:30
sumangala-patki b20bc668d5
HADOOP-17548. ABFS: Toggle Store Mkdirs request overwrite parameter (#2729) (#2781) 
Contributed by Sumangala Patki.

(cherry picked from commit fe633d4739)
 2021-05-10 11:50:01 +05:30
Ayush Saxena 2a95ba29bb HADOOP-17644. Add back the exceptions removed by HADOOP-17432 for compatibility. Contributed by Quan Li. 2021-05-09 04:00:30 +05:30
Akira Ajisaka 1326cad66a
HDFS-16001. TestOfflineEditsViewer.testStored() fails reading negative value of FSEditsLogOpCodes (branch-3.3) (#2989). Contributed by Akira Ajisaka. 2021-05-08 10:48:56 +08:00
Szilard Nemeth 3d715c2e4c
YARN-9333. TestFairSchedulerPreemption.testRelaxLocalityPreemptionWithNoLessAMInRemainingNodes fails intermittently. Contributed by Peter Bacsko 
(cherry picked from commit eacbe07b56)
 2021-05-07 14:32:17 +09:00
Szilard Nemeth 3303aa5947
YARN-10515. Fix flaky test TestCapacitySchedulerAutoQueueCreation.testDynamicAutoQueueCreationWithTags. Contributed by Peter Bacsko 
(cherry picked from commit 8620984b8d)
 2021-05-07 14:26:58 +09:00
Mike 0f12f3e125
HADOOP-17036. TestFTPFileSystem failing as ftp server dir already exists. 
Contributed by Mikhail Pryakhin.

(cherry picked from commit 017d24e970)
 2021-05-07 14:20:29 +09:00
Takanobu Asanuma df50e210dd
HADOOP-17375. Fix the error of TestDynamometerInfra. (#2471) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 207210263a)
 2021-05-07 14:09:03 +09:00
hchaverr cedebf1c27 HADOOP-17680. Allow ProtobufRpcEngine to be extensible (#2905) Contributed by Hector Chaverri. 
(cherry picked from commit f40e3eb059)
 2021-05-06 17:50:32 -07:00
Akira Ajisaka 217655269a YARN-10756. Remove additional junit 4.11 dependency from javadoc. (#2960) 
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Bilwa S T <stbilwa@gmail.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 7cb7b8adc1)
 2021-05-06 23:47:24 +09:00
Viraj Jasani be34c1222a HADOOP-11616. Remove workaround for Curator's ChildReaper requiring Guava 15+ (#2973) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit b93e448f9a)
 2021-05-06 04:53:06 +09:00
Takanobu Asanuma 65bf544118 HADOOP-16954. Add -S option in "Count" command to show only Snapshot Counts. Contributed by hemanthboyina. 
(cherry picked from commit b89d875f7b)
 2021-05-04 17:44:34 +01:00
Steve Loughran e944cc0338
HADOOP-16742. NullPointerException in S3A MultiObjectDeleteSupport 
Contributed by Tor Arvid Lund.

Change-Id: Iadfe9b2f355cf373031075bfbe681705a2c65bdc
 2021-05-04 16:15:13 +01:00
Istvan Fajth 00ba886a22 HADOOP-17675. LdapGroupsMapping$LdapSslSocketFactory ClassNotFoundException (#2965) 
(cherry picked from commit 0d78d73973)
 2021-05-04 11:36:15 +01:00