Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
24,161 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

24161 Commits

Author SHA1 Message Date
Uma Maheswara Rao G ae8261c671 HDFS-15464: ViewFsOverloadScheme should work when -fs option pointing to remote cluster without mount links (#2132). Contributed by Uma Maheswara Rao G. 
(cherry picked from commit 3e70006639)
 2020-07-31 01:31:15 -07:00
Uma Maheswara Rao G 10f8010519 HDFS-15449. Optionally ignore port number in mount-table name when picking from initialized uri. Contributed by Uma Maheswara Rao G. 
(cherry picked from commit dc0626b5f2)
 2020-07-31 01:30:54 -07:00
Uma Maheswara Rao G aea1a8e2bd HDFS-15450. Fix NN trash emptier to work if ViewFSOveroadScheme enabled. Contributed by Uma Maheswara Rao G. 
(cherry picked from commit 55a2ae80dc)
 2020-07-31 01:30:30 -07:00
Uma Maheswara Rao G 35fe6fd54f HDFS-15430. create should work when parent dir is internalDir and fallback configured. Contributed by Uma Maheswara Rao G. 
(cherry picked from commit 1f2a80b5e5)
 2020-07-31 01:29:46 -07:00
Yuan e1ac16a318
HDFS-14950. fix missing libhdfspp lib in dist-package (#1947) 
libhdfspp.{a,so} are missed in dist-package.
This patch fixed this by copying these libs to the right directory

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
(cherry picked from commit e756fe3590)
 2020-07-31 16:51:05 +09:00
Eric E Payne 0cb1b178ec YARN-1529: Add Localization overhead metrics to NM. Contributed by Jim_Brennan. 
(cherry picked from commit e0c9653166)
 2020-07-30 16:39:56 +00:00
ywheel 2efa28cb79
MAPREDUCE-7051. Fix typo in MultipleOutputFormat (#338) 
(cherry picked from commit cf4eb75608)
 2020-07-30 13:28:35 +09:00
Jonathan Hung d3c7f7cdbd YARN-10343. Legacy RM UI should include labeled metrics for allocated, total, and reserved resources. Contributed by Eric Payne 
(cherry picked from commit 3eaf62726f)
 2020-07-28 13:43:44 -07:00
Akira Ajisaka 9263b75b3c
YARN-10367. Failed to get nodejs 10.21.0 when building docker image (#2171) 
(cherry picked from commit e277d338da)
 2020-07-27 00:53:24 +09:00
Sneha Vijayarajan 18ca80331c
Hadoop 17132. ABFS: Fix Rename and Delete Idempotency check trigger 
- Contributed by Sneha Vijayarajan
 2020-07-25 13:13:18 +00:00
ishaniahuja f24e2ec487
HADOOP-17058. ABFS: Support for AppendBlob in Hadoop ABFS Driver 
- Contributed by Ishani Ahuja
 2020-07-25 13:12:32 +00:00
Mehakmeet Singh 7c9b459786
HADOOP-16961. ABFS: Adding metrics to AbfsInputStream (#2076) 
Contributed by Mehakmeet Singh.
 2020-07-25 13:12:09 +00:00
Mehakmeet Singh bbd3278d09
HADOOP-17065. Add Network Counters to ABFS (#2056) 
Contributed by Mehakmeet Singh.
 2020-07-25 13:11:34 +00:00
Karthik Amarnath 8b7e77443d
HDFS-15168: ABFS enhancement to translate AAD to Linux identities. (#1978) 2020-07-25 13:10:39 +00:00
Sneha Vijayarajan 903935da0f
HADOOP-17053. ABFS: Fix Account-specific OAuth config setting parsing 
Contributed by Sneha Vijayarajan
 2020-07-25 13:10:30 +00:00
Sneha Vijayarajan 869a68b81e
HADOOP-16852: Report read-ahead error back 
Contributed by Sneha Vijayarajan
 2020-07-25 13:10:19 +00:00
Sneha Vijayarajan 27b20f9689
HADOOP-17054. ABFS: Fix test AbfsClient authentication instance 
Contributed by Sneha Vijayarajan
 2020-07-25 13:09:26 +00:00
Sneha Vijayarajan eed06b46eb
Hadoop-17015. ABFS: Handling Rename and Delete idempotency 
Contributed by Sneha Vijayarajan.
 2020-07-25 13:08:01 +00:00
Eric Badger 7ec692aa83 YARN-4771. Some containers can be skipped during log aggregation after NM 
restart. Contributed by Jason Lowe and Jim Brennan.

(cherry picked from commit ac5f21dbef)
 2020-07-24 22:46:57 +00:00
bilaharith 1ae72d2438
HADOOP-17092. ABFS: Making AzureADAuthenticator.getToken() throw HttpException 
- Contributed by Bilahari T H

Change-Id: Id9576d9509faaf057bf419ccb1879ac0cef7a07b
 2020-07-22 18:26:36 +01:00
Akira Ajisaka d35f7eaeb9
HADOOP-17147. Dead link in hadoop-kms/index.md.vm. Contributed by Xieming Li. 
(cherry picked from commit d5b4766158)
 2020-07-23 00:40:31 +09:00
Ayush Saxena e3b8d4eb05 HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. Contributed by Ahmed Hussein. 2020-07-22 18:21:14 +05:30
Chen Liang c8c40be761 HDFS-15404. ShellCommandFencer should expose info about source. Contributed by Chen Liang. 
(cherry picked from commit 3833c616e0)
 2020-07-20 15:22:34 -07:00
Ayush Saxena 8a58a12626 HADOOP-17119. Jetty upgrade to 9.4.x causes MR app fail with IOException. Contributed by Bilwa S T. 2020-07-20 22:26:11 +05:30
Steve Loughran 5aa9396a58
HADOOP-17107. hadoop-azure parallel tests not working on recent JDKs (#2118) 
Contributed by Steve Loughran.

Change-Id: I972264aed36f384b7ae23e214326ef7870261cf5
 2020-07-20 10:54:22 +01:00
bilaharith e01852181a
HADOOP-16682. ABFS: Removing unnecessary toString() invocations 
- Contributed by Bilahari T H

Change-Id: Id55495b44d81533d1d3654de2553c709f505f7eb
 2020-07-20 10:53:59 +01:00
Mukund Thakur 187b161c20
HADOOP-17130. Configuration.getValByRegex() shouldn't be updating the results while fetching. (#2142) 
Contributed by Mukund Thakur

Change-Id: Ic0c9c44a1b14b4bfa962cede3f782a98fdfd7513
 2020-07-16 18:15:40 +01:00
Mehakmeet Singh 0d88ed2794
HADOOP-17129. Validating storage keys in ABFS correctly (#2141) 
Contributed by Mehakmeet Singh

Change-Id: I8016ee2f9ffbc86ea867f4a3d960b134e507d099
 2020-07-16 18:11:52 +01:00
Ahmed Hussein 9e7266df6c HADOOP-17099. Replace Guava Predicate with Java8+ Predicate 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 1f71c4ae71)
 2020-07-15 11:40:13 -05:00
Erik Krogen 67e01ed2ca HADOOP-17127. Use RpcMetrics.TIMEUNIT to initialize rpc queueTime and processingTime. Contributed by Jim Brennan. 
(cherry picked from 317fe4584a)
 2020-07-15 08:26:38 -07:00
Ahmed Hussein 5969922305 HADOOP-17101. Replace Guava Function with Java8+ Function 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 98fcffe93f)
 2020-07-15 09:57:36 -05:00
Shashikant Banerjee 292e22578a HDFS-15313. Ensure inodes in active filesytem are not deleted during snapshot delete. Contributed by Shashikant Banerjee. 
(cherry picked from commit 82343790ee)
 2020-07-15 13:13:27 +01:00
Mukund Thakur 8b601ad7e6 HADOOP-17022. Tune S3AFileSystem.listFiles() API. 
Contributed by Mukund Thakur.

Change-Id: I17f5cfdcd25670ce3ddb62c13378c7e2dc06ba52
 2020-07-14 15:28:27 +01:00
Anoop Sam John cac2fc1f58 HADOOP-16998. WASB : NativeAzureFsOutputStream#close() throwing IllegalArgumentException (#2073) 
Contributed by Anoop Sam John.
 2020-07-14 14:08:46 +01:00
Eric Badger 41bcef9486 YARN-10348. Allow RM to always cancel tokens after app completes. Contributed by 
Jim Brennan

(cherry picked from commit 48f90115b5)
 2020-07-13 23:12:18 +00:00
Eric E Payne 7044a007b3 YARN-10297. TestContinuousScheduling#testFairSchedulerContinuousSchedulingInitTime fails intermittently. Contributed by Jim Brennan (Jim_Brennan) 
(cherry picked from commit 0427100b75)
 2020-07-13 19:02:40 +00:00
jimmy-zuber-amzn 79fc58def3
HADOOP-17105. S3AFS - Do not attempt to resolve symlinks in globStatus (#2113) 
Contributed by Jimmy Zuber.

Change-Id: I2f247c2d2ab4f38214073e55f5cfbaa15aeaeb11
 2020-07-13 19:09:50 +01:00
Steve Loughran a51d72f0c6 HDFS-13934. Multipart uploaders to be created through FileSystem/FileContext. 
Contributed by Steve Loughran.

Change-Id: Iebd34140c1a0aa71f44a3f4d0fee85f6bdf123a3
 2020-07-13 13:32:04 +01:00
He Xiaoqiao e7e7a6d503 HDFS-14498 LeaseManager can loop forever on the file for which create has failed. Contributed by Stephen O'Donnell. 2020-07-13 14:31:50 +08:00
Siyao Meng 358934059f HDFS-15462. Add fs.viewfs.overload.scheme.target.ofs.impl to core-default.xml (#2131) 
(cherry picked from commit 0e694b20b9)
 2020-07-09 16:30:58 -07:00
Uma Maheswara Rao G f85ce2570e HDFS-15394. Add all available fs.viewfs.overload.scheme.target.<scheme>.impl classes in core-default.xml bydefault. Contributed by Uma Maheswara Rao G. 
(cherry picked from commit 3ca15292c5)
 2020-07-09 16:26:04 -07:00
Brahma Reddy Battula 7b175739a9 YARN-10341. Yarn Service Container Completed event doesn't get processed. Contributed by Bilwa S T. 
(cherry picked from commit dfe60392c9)
 2020-07-09 12:36:21 +05:30
Akira Ajisaka 0aa2d7d506
YARN-10344. Sync netty versions in hadoop-yarn-csi. (#2126) 
(cherry picked from commit 10d218934c)
 2020-07-09 15:07:44 +09:00
Masatake Iwasaki 7522b44067 HADOOP-17120. Fix failure of docker image creation due to pip2 install error. (#2130) 
(cherry picked from commit 53cfe60904)
 2020-07-09 03:19:18 +00:00
Sebastian Nagel f9619b0b97
HADOOP-17117 Fix typos in hadoop-aws documentation (#2127) 
(cherry picked from commit 5b1ed2113b)
 2020-07-09 00:04:46 +09:00
Shanyu Zhao 10c9df1d0a HDFS-15451. Do not discard non-initial block report for provided storage. (#2119). Contributed by Shanyu Zhao. 
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
 2020-07-07 11:44:42 +08:00
Madhusoodan Pataki 0789ae5b78 HADOOP-17081. MetricsSystem doesn't start the sink adapters on restart (#2089) 
Contributed by Madhusoodan P
 2020-07-06 16:26:48 +01:00
Akira Ajisaka 20df70a895
HADOOP-17111. Replace Guava Optional with Java8+ Optional. Contributed by Ahmed Hussein. 
(cherry picked from commit 639acb6d89)
 2020-07-06 16:09:37 +09:00
Ayush Saxena b8d9218189 HDFS-15446. CreateSnapshotOp fails during edit log loading for /.reserved/raw/path with error java.io.FileNotFoundException: Directory does not exist: /.reserved/raw/path. Contributed by Stephen O'Donnell. 2020-07-04 12:25:29 +05:30
bilaharith 19fb204011
HADOOP-17086. ABFS: Making the ListStatus response ignore unknown properties. (#2101) 
Contributed by Bilahari T H.

Change-Id: I82e4683fba8481aef2abab7a6a99e5752f6fffa9
 2020-07-03 19:02:21 +01:00