Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,863 Commits 326 Branches 371 Tags
Commit Graph

118 Commits

Author SHA1 Message Date
Mingliang Liu
b0f8de184a HADOOP-14091. AbstractFileSystem implementaion for 'wasbs' scheme. Contributed Varada Hemeswari. 
(cherry picked from commit 82607fce39151fc6ba5bced738088e2bc176dc77)
 2017-02-23 14:11:08 -08:00
Mingliang Liu
ec646b1a9d HADOOP-13942. Build failure due to errors of javadoc build in hadoop-azure. Contributed by Kai Sasaki 
(cherry picked from commit c6a5b689db573046819f0e9193cb042e6c7298f4)
 2016-12-29 16:24:38 -08:00
Mingliang Liu
edac82737a HADOOP-13863. Azure: Add a new SAS key mode for WASB. Contributed by Dushyanth 
(cherry picked from commit e92a77099b91620cee84513cc879089907468075)
 2016-12-22 20:33:51 -08:00
Mingliang Liu
867389247f HADOOP-13831. Correct check for error code to detect Azure Storage Throttling and provide retries. Contributed by Gaurav Kanade 
(cherry picked from commit f92913c35bfda0d565606f9fb9a301ddd4105fd8)
 2016-12-15 12:38:31 -08:00
Mingliang Liu
89eaa94c59 HADOOP-13675. Bug in return value for delete() calls in WASB. Contributed by Dushyanth 
(cherry picked from commit 15dd1f3381069c5fdc6690e3ab1907a133ba14bf)
 2016-12-05 12:06:35 -08:00
Mingliang Liu
4e0fcff8ab HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu 
(cherry picked from commit 5af572b6443715b7a741296c1bd520a1840f9a7c)
 2016-11-15 11:36:08 -08:00
Chris Nauroth
804a40720b HADOOP-13513. Java 1.7 support for org.apache.hadoop.fs.azure testcases. Contributed by Tibor Kiss. 
(cherry picked from commit ae4db2544346370404826d5b55b2678f5f92fe1f)
 2016-08-18 10:45:03 -07:00
Chris Nauroth
287688d6ca HADOOP-13403. AzureNativeFileSystem rename/delete performance improvements. Contributed by Subramanyam Pattipaka. 
(cherry picked from commit 2ed58c40e5dcbf5c5303c00e85096085b1055f85)
 2016-08-08 12:28:46 -07:00
Chris Nauroth
8ec3d702a3 HADOOP-13459. hadoop-azure runs several test cases repeatedly, causing unnecessarily long running time. Contributed by Chris Nauroth. 
(cherry picked from commit 4ef1bc0bdb6b2c907431492e4c8395035fa96c84)
 2016-08-02 22:32:37 -07:00
Chris Nauroth
59eec8248a HADOOP-13354. Update WASB driver to use the latest version (4.2.0) of SDK for Microsoft Azure Storage Clients. Contributed by Sivaguru Sankaridurg. 
(cherry picked from commit b43de80031d1272e8a08ea5bd31027efe45e9d70)

Conflicts:
	hadoop-project/pom.xml
 2016-07-27 15:52:07 -07:00
Steve Loughran
e29d7fb585 HADOOP-13145 In DistCp, prevent unnecessary getFileStatus call when not preserving metadata. Contributed by Chris Nauroth. 2016-05-20 12:21:47 +01:00
Chris Nauroth
26a23eff82 HADOOP-12994. Specify PositionedReadable, add contract tests, fix problems. Contributed by Steve Loughran. 
(cherry picked from commit 843ee8d59d8bacbca0d87ccf0790772e39d16138)
 2016-04-08 13:37:46 -07:00
Chris Nauroth
2e60387401 HADOOP-12717. NPE when trying to rename a directory in Windows Azure Storage FileSystem. Contributed by Robert Yokota and Gaurav Kanade. 
(cherry picked from commit c50aad0f854b74ede9668e35db314b0a93be81b2)
 2016-03-04 22:57:56 -08:00
Chris Nauroth
bce33b5179 HADOOP-12853. Change WASB documentation regarding page blob support. Contributed by Madhumita Chakraborty. 
(cherry picked from commit f98dff329b1f94c9f53022baf0209fc1a7aaf7c2)
 2016-03-01 13:18:57 -08:00
Chris Nauroth
2e761dabd8 HADOOP-12846. Credential Provider Recursive Dependencies. Contributed by Larry McCay. 
(cherry picked from commit 7634d404b750eafa135a37fa275325d0398255fb)
 2016-02-28 11:23:02 -08:00
cnauroth
9ea19a861f HADOOP-12535. Run FileSystem contract tests with hadoop-azure. Contributed by Madhumita Chakraborty. 
(cherry picked from commit 3369a4f6916f12e9d6b97072badd1b176be443bd)
 2016-02-23 21:37:56 -08:00
cnauroth
a5511debdb HADOOP-12555. WASB to read credentials from a credential provider. Contributed by Larry McCay. 
(cherry picked from commit 27b77751c1163ab4a1ce081a426e5190d1b8aff4)
 2016-02-22 17:30:32 -08:00
cnauroth
3f14eed580 HADOOP-11613. Remove commons-httpclient dependency from hadoop-azure. Contributed by Masatake Iwasaki. 
(cherry picked from commit d4f5fc23b208635e8f9a14c375d4101141aefa4a)
 2016-02-22 13:39:45 -08:00
cnauroth
ac5a793590 HADOOP-12780. During WASB atomic rename handle crash when one directory has been renamed but not file under it. Contributed by Madhumita Chakraborty. 
(cherry picked from commit 91a96eaa534dbb27e81b6c24bbb8138200a80a83)
 2016-02-12 15:50:19 -08:00
cnauroth
62d6166211 HADOOP-12635. Adding Append API support for WASB. Contributed by Dushyanth. 
(cherry picked from commit 8bc93db2e7c64830b6a662f28c8917a9eef4e7c9)
 2016-01-18 09:09:05 -08:00
cnauroth
813841e10b HADOOP-12551. Introduce FileNotFoundException for WASB FileSystem API. Contributed by Dushyanth. 
(cherry picked from commit 0e76f1fceaaaeb66bdb4818f43b9a55fc092bf79)
 2016-01-09 22:18:22 -08:00
cnauroth
debd13387d HADOOP-12678. Handle empty rename pending metadata file during atomic rename in redo path. Contributed by Madhumita Chakraborty. 
(cherry picked from commit f0fa6d869b9abb5a900ea1c9eb4eb19ec9831dc4)
 2016-01-08 20:19:11 -08:00
cnauroth
4753676a34 HADOOP-12634. Change Lazy Rename Pending Operation Completion of WASB to address case of potential data loss due to partial copy. Contributed by Gaurav Kanade. 
(cherry picked from commit 978bbdfeb2d12efd6e750da6a14849e072fb814b)
 2016-01-06 11:16:18 -08:00
Xiaoyu Yao
a65e159c59 HADOOP-12608. Fix exception message in WASB when connecting with anonymous credential. Contributed by Dushyanth. 
(cherry picked from commit 007a8decc61750720033e06b6d08861ac3788e41)
 2015-12-30 11:12:02 -08:00
Vinod Kumar Vavilapalli
b20e77eba3 Preparing for 2.9.0 development: mvn versions:set -DnewVersion=2.9.0 2015-11-25 17:47:22 -08:00
cnauroth
81f7e8af22 HADOOP-12540. TestAzureFileSystemInstrumentation#testClientErrorMetrics fails intermittently due to assumption that a lease error will be thrown. Contributed by Gaurav Kanade. 
(cherry picked from commit 0fb1867fd62b5df664ad66386d6067db8fbf2317)
 2015-11-04 10:29:01 -08:00
cnauroth
c1d7b26e94 HADOOP-12508. delete fails with exception when lease is held on blob. Contributed by Gaurav Kanade. 
(cherry picked from commit 9e7dcab185abf2fdabb28f2799b9952b5664a4b0)
 2015-11-02 10:21:46 -08:00
cnauroth
c260d7ec78 HADOOP-12533. Introduce FileNotFoundException in WASB for read and seek API. Contributed by Dushyanth. 
(cherry picked from commit 28790692624177d89fb1e4f59e2f83a659fc3089)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2015-11-02 09:39:27 -08:00
cnauroth
c7f87dc2d0 HADOOP-12519. hadoop-azure tests should avoid creating a metrics configuration fiale in the module root directory. Contributed by Chris Nauroth. 
(cherry picked from commit 2d10cb8e00fd84be26b51f9fba819e3e5b6cdcbf)
 2015-10-28 09:44:21 -07:00
cnauroth
0377795e06 HADOOP-11685. StorageException complaining " no lease ID" during HBase distributed log splitting. Contributed by Duo Xu. 
(cherry picked from commit 1f7ecb0c84042783f9fcf3f77d7d889dc58c9ead)
 2015-10-27 22:56:33 -07:00
cnauroth
a23f79b92c HADOOP-12520. Use XInclude in hadoop-azure test configuration to isolate Azure Storage account keys for service integration tests. Contributed by Chris Nauroth. 
(cherry picked from commit 73822de7c38e189f7654444ff48d15cbe0df7404)
 2015-10-27 22:49:04 -07:00
cnauroth
b023539c19 HADOOP-12484. Single File Rename Throws Incorrectly In Potential Race Condition Scenarios. Contributed by Gaurav Kanade. 
(cherry picked from commit cb282d5b89fdece4719cc4ad37a6e27f13371534)
 2015-10-22 14:30:09 -07:00
cnauroth
10398daadf HADOOP-12334. Change Mode Of Copy Operation of HBase WAL Archiving to bypass Azure Storage Throttling after retries. Contributed by Gaurav Kanade. 
(cherry picked from commit 47641fcbc9c41f4a338d8899501e4a310d2e81ad)
 2015-10-22 12:21:44 -07:00
cnauroth
535762fadb HADOOP-12350. WASB Logging: Improve WASB Logging around deletes, reads and writes. Contributed by Dushyanth. 
(cherry picked from commit 5f6edb30c2bb648d5564c951edc25645e17e6636)
 2015-10-05 20:12:33 -07:00
Kihwal Lee
95b8f1f3a6 HADOOP-12446. Undeprecate createNonRecursive(). Contributed by Ted Yu. 
(cherry picked from commit 3abbdc929bde05f8819f5410cef1eaeb8940203f)
 2015-09-28 17:33:20 -05:00
cnauroth
b83d475252 HADOOP-12304. Applications using FileContext fail with the default file system configured to be wasb/s3/etc. Contributed by Chris Nauroth. 
(cherry picked from commit 3da0bedaec38714a62c3b7244e84bc4f981ddf45)
 2015-08-05 14:56:54 -07:00
cnauroth
bb080b32d4 HADOOP-12239. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB. Contributed by Duo Xu. 
(cherry picked from commit efa97243ecb84b3b468e732897cd685e3869f480)
 2015-07-22 11:17:00 -07:00
cnauroth
399df10c58 HADOOP-12089. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB. Contributed by Duo Xu. 
(cherry picked from commit 460e98f7b3ec84f3c5afcb2aad4f4e7031d16e3a)
 2015-06-29 13:49:12 -07:00
cnauroth
f5b0cce7fa HADOOP-12073. Azure FileSystem PageBlobInputStream does not return -1 on EOF. Contributed by Ivan Mitic. 
(cherry picked from commit c45784bc9031353b938f4756473937cca759b3dc)
 2015-06-08 22:42:24 -07:00
cnauroth
39f451e721 HADOOP-11959. WASB should configure client side socket timeout in storage client blob request options. Contributed by Ivan Mitic. 
(cherry picked from commit 94e7d49a6dab7e7f4e873dcca67e7fcc98e7e1f8)

Conflicts:
	hadoop-project/pom.xml
 2015-05-28 12:32:55 -07:00
cnauroth
91855c2340 HADOOP-11960. Enable Azure-Storage Client Side logging. Contributed by Dushyanth. 
(cherry picked from commit cb8e69a80cecb95abdfc93a787bea0bedef275ed)
 2015-05-14 22:22:46 -07:00
Vinayakumar B
14d88a85e4 HADOOP-11922. Misspelling of threshold in log4j.properties for tests in hadoop-tools (Contributed by Gabor Liptak) 
(cherry picked from commit f33efbc82709e75d9b4c4ed9835629668dbade7d)
 2015-05-08 10:23:27 +05:30
Jonathan Eagles
9ed6b77da1 HADOOP-11889. Make checkstyle runnable from root project (Gera Shegalov via jeagles) 
(cherry picked from commit 2d7363b27360e36fdd62546c0f9d0b1d78133f29)

Conflicts:
	pom.xml
 2015-05-01 19:04:41 -07:00
Tsuyoshi Ozawa
77297017d8 HADOOP-11720. [JDK8] Fix javadoc errors caused by incorrect or illegal tags in hadoop-tools. Contributed by Akira AJISAKA. 
(cherry picked from commit ef9946cd52d54200c658987c1dbc3e6fce133f77)
 2015-03-17 16:09:38 +09:00
cnauroth
dcc4850ff3 HADOOP-11693. Azure Storage FileSystem rename operations are throttled too aggressively to complete HBase WAL archiving. Contributed by Duo Xu. 
(cherry picked from commit 7a346bcb4fa5b56191ed00a39e72e51c9bdf1b56)
 2015-03-11 14:37:22 -07:00
Tsuyoshi Ozawa
b46f9e72db HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 2015-03-09 19:53:55 +09:00
Vinod Kumar Vavilapalli
d9416317a9 Preparing for 2.8.0 development. Changing SNAPSHOT version to be 2.8 on branch-2. 2015-03-08 20:32:51 -07:00
cnauroth
02cadbd24b HADOOP-11642. Upgrade azure sdk version from 0.6.0 to 2.0.0. Contributed by Shashank Khandelwal and Ivan Mitic. 
(cherry picked from commit 608ebd52bafecf980e9726d397c786a9c2422eba)

Conflicts:
	hadoop-project/pom.xml
 2015-03-06 15:29:22 -08:00
cnauroth
1bc4c6808d HADOOP-11629. WASB filesystem should not start BandwidthGaugeUpdater if fs.azure.skip.metrics set to true. Contributed by Shanyu Zhao. 
(cherry picked from commit 5731c0e0d08c3048fafdf62a14ca7611be4df5d7)
 2015-02-25 09:09:15 -08:00
Tsuyoshi Ozawa
f92d4fa20f HADOOP-11546. Checkstyle failing: Unable to instantiate DoubleCheckedLockingCheck. (ozawa) 
(cherry picked from commit ba58da29f49da15f7eb6300f996e0ae7ba9d0d1e)
 2015-02-05 00:51:40 +09:00