Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,040 Commits 326 Branches 371 Tags
Commit Graph

15040 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Steve Loughran
73bbada59a HADOOP-15809. ABFS: better exception handling when making getAccessToken call. 
Contributed by Da Zhou
 2019-04-24 21:46:25 +00:00
Steve Loughran
1b08e8d537 HADOOP-15795. Make HTTPS the default protocol for ABFS. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Steve Loughran
e3033a5ba8 HADOOP-15792. typo in AzureBlobFileSystem.getIsNamespaceEnabeld. 
Contributed by Abhishek Modi.
 2019-04-24 21:46:25 +00:00
Steve Loughran
d5e4c40760 HADOOP-15801. ABFS: Fixing skipUserGroupMetadata in AzureBlobFileSystemStore. 
Contributed by Da Zhou
 2019-04-24 21:46:25 +00:00
Steve Loughran
0c5074a228 HADOOP-15793. ABFS: Skip unsupported test cases when non namespace enabled in ITestAzureBlobFileSystemAuthorization 
Contributed by Yuan Gao.
 2019-04-24 21:46:25 +00:00
Steve Loughran
ee0fa0d15c HADOOP-15739. ABFS: remove unused maven dependencies and add used undeclared dependencies. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Steve Loughran
bde00d56df HADOOP-15723. ABFS: Ranger Support. 
Contributed by Yuan Gao.
 2019-04-24 21:46:25 +00:00
Steve Loughran
4f3cc1d417 HADOOP-15778. ABFS: Fix client side throttling for read. 
Contributed by Sneha Varma.
 2019-04-24 21:46:25 +00:00
Steve Loughran
6c7c8b83f7 HADOOP-15704. Mark ABFS extension package and interfaces as LimitedPrivate/Unstable. 
Contributed by Steve Loughran.
 2019-04-24 21:46:25 +00:00
Sean Mackrory
920b77c8d7 HADOOP-15773. Fixing checkstyle and other issues raised by Yetus. 2019-04-24 21:46:25 +00:00
Steve Loughran
10e149d882 HADOOP-15769. ABFS: distcp tests are always skipped. 
Contributed by Steve Loughran
 2019-04-24 21:46:25 +00:00
Steve Loughran
1efa1b73a9 HADOOP-15719. Fail-fast when using OAuth over http. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Steve Loughran
8acab6c6aa HADOOP-15714. Tune abfs/wasb parallel and sequential test execution. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Steve Loughran
26ce4e9301 HADOOP-15715. ITestAzureBlobFileSystemE2E timing out with non-scale timeout of 10 min. 
Contributed by Da Zhou
 2019-04-24 21:46:25 +00:00
Steve Loughran
94d0de2eab HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch. 
Contributed by Steve Loughran.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
3599cd3dc9 HADOOP-15757. ABFS: remove dependency on common-codec Base64. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
263094f962 HADOOP-15753. ABFS: support path "abfs://mycluster/file/path" 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
e52f4a1ad2 HADOOP-15694. ABFS: Allow OAuth credentials to not be tied to accounts. 
Contributed by Sean Mackrory.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
36b2bab888 HADOOP-15740. ABFS: Check variable names during initialization of AbfsClientThrottlingIntercept. 
Contributed by Sneha Varma.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
a192d85a5f HADOOP-15728. ABFS: Add backward compatibility to handle Unsupported Operation 
for storage account with no namespace feature.

Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
8bb713e31a Fixing findbugs and license issues related to: 
HADOOP-15703. ABFS - Implement client-side throttling.
Contributed by Sneha Varma and Thomas Marquardt.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
305e750e82 HADOOP-15703. ABFS - Implement client-side throttling. 
Contributed by Sneha Varma and Thomas Marquardt.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
12c621eda4 HADOOP-15664. ABFS: Reduce test run time via parallelization and grouping. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
3bd089ddca HADOOP-15663. ABFS: Simplify configuration. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
92f3f9cf09 HADOOP-15692. ABFS: extensible support for custom oauth. 
Contributed by Junhua Gu and Rajeev Bansal.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
b59741f66b HADOOP-15682. ABFS: Add support for StreamCapabilities. Fix javadoc and checkstyle. 
Contributed by Thomas Marquardt.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
08d3d50275 HADOOP 15688. ABFS: InputStream wrapped in FSDataInputStream twice. 
Contributed by Sean Mackrory.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
e727c87500 HADOOP-15661. ABFS: Add support for ACL. 
Contributed by Junhua Gu and Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
9d47230c8f HADOOP-15660. ABFS: Add support for OAuth 
Contributed by Da Zhou, Rajeev Bansal, and Junhua Gu.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
92e37a0b73 HADOOP-15669. ABFS: Improve HTTPS Performance. 
Contributed by Vishwajeet Dusane.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
e320b64367 Fixing issue due to commit 2b2399d6 after rebase onto trunk. 2019-04-24 21:46:25 +00:00
Thomas Marquardt
5da4bb8892 HADOOP-15659. Code changes for bug fix and new tests. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt
b537d8c442 HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests. 
Contributed by Steve Loughran and Da Zhou.
 2019-04-24 21:46:25 +00:00
Steve Loughran
37426c2da8 HADOOP-15560. ABFS: removed dependency injection and unnecessary dependencies. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Steve Loughran
1b64b0f924 HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit. 
Contributed by Shane Mainali, Thomas Marquardt, Zichen Sun, Georgi Chalakov, Esfandiar Manii, Amit Singh, Dana Kaban, Da Zhou, Junhua Gu, Saher Ahwal, Saurabh Pant, James Baker, Shaoyu Zhang, Lawrence Chen, Kevin Chen and Steve Loughran
 2019-04-24 21:46:25 +00:00
Wanqiang Ji
313608e459
HADOOP-16232. Fix errors in the checkstyle configration xmls. Contributed by Wanqiang Ji. 
(cherry picked from commit 8b6deebb1dda49e5e35180ed5c5fb5b5221c1516)
(cherry picked from commit 875435dc7d6a7cb7925a25ca3347a7ebaabec595)
(cherry picked from commit 6f0896e08742a792049137ed19700135f8594a42)
 2019-04-24 11:11:51 +01:00
Puleya7
8579ab13cb HDFS-14432. dfs.datanode.shared.file.descriptor.paths duplicated in hdfs-default.xml 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 13907d847986aa14a51cc2a3cc83151f22c43781)
 2019-04-17 21:58:43 +09:00
Masatake Iwasaki
50e30bd30a HADOOP-14544. DistCp documentation for command line options is misaligned. Contributed by Masatake Iwasaki. 
(cherry picked from commit bbdbc7a9a158f36955c2253acb0edb14219ccb04)

Conflicts:
	hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm

(cherry picked from commit 7985d9b1ced4371f4cdc48ea74fbf120eab50309)

Conflicts:
	hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
 2019-04-12 12:35:32 +09:00
Wei-Chiu Chuang
8af767b069 HDFS-10477. Stop decommission a rack of DataNodes caused NameNode fail over to standby. Contributed by yunjiong zhao and Wei-Chiu Chuang. 2019-04-10 14:36:07 -07:00
Dinesh Chitlangia
66ac003128 HDFS-14414. Clean up findbugs warning in branch-2. Contributed by Dinesh Chitlangia. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-04-08 10:47:13 -07:00
Chen Liang
dce76e0355 HDFS-14415. Backport HDFS-13799 to branch-2. Contributed by Chao Sun. 2019-04-05 11:27:37 -07:00
Akira Ajisaka
2abc7927d1
HDFS-14407. Fix misuse of SLF4j logging API in DatasetVolumeChecker#checkAllVolumes. Contributed by Wanqiang Ji. 
(cherry picked from commit 67020f09502a4f07342dee457e47bb52b03441ae)
 2019-04-05 12:32:07 +09:00
Chen Liang
a5bca767f7 HDFS-14397. Backport HADOOP-15684 to branch-2. Contributed by Chao Sun. 2019-04-03 17:32:56 -07:00
Chen Liang
d6083e9920 HDFS-14399. Backport HDFS-10536 to branch-2. Contributed by Chao Sun. 2019-04-01 12:47:02 -07:00
Chen Liang
812e6d5c81 HDFS-14392. Backport HDFS-9787 to branch-2. Contributed by Chao Sun. 2019-03-28 19:07:58 -07:00
Chen Liang
99e41293ee HDFS-14391. Backport HDFS-9659 to branch-2. Contributed by Chao Sun. 2019-03-28 10:39:45 -07:00
Chen Liang
7935485eec HDFS-14205. Backport HDFS-6440 to branch-2. Contributed by Chao Sun. 2019-03-26 12:12:14 -07:00
Weiwei Yang
5f3abefab3 HADOOP-16191. AliyunOSS: improvements for copyFile/copyDirectory and logging. Contributed by wujinhu. 
(cherry picked from commit 568d3ab8b65d1348dec9c971feffe200e6cba2ef)
 2019-03-19 10:15:50 +08:00
Erik Krogen
c3973c9ae6 HADOOP-16192. Fix CallQueue backoff bugs: perform backoff when add() is used and update backoff when refreshed. 
(cherry-picked from 8c95cb9d6bef369fef6a8364f0c0764eba90e44a)
(cherry-picked from 0de8b55a095ada2b98c0a41899651bd8e524f42e)
(cherry-picked from d4fbbc83ad8c4d818deccf62b4c54cead1d17a8f)
(cherry-picked from e172fc62ce3fa10a36582bc6667edec931fea9fa)
 2019-03-18 09:00:40 -07:00
Inigo Goiri
525f4e52f3 HDFS-14366. Improve HDFS append performance. Contributed by Chao Sun. 
(cherry picked from commit ff06ef0631cb8a0f67bbc39b5b5a1b0a81ca3b3c)
 2019-03-15 14:02:20 -07:00