Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,858 Commits 326 Branches 371 Tags
Commit Graph

56 Commits

Author SHA1 Message Date
Chris Nauroth
1c4e039ba7 HADOOP-13513. Java 1.7 support for org.apache.hadoop.fs.azure testcases. Contributed by Tibor Kiss. 
(cherry picked from commit ae4db2544346370404826d5b55b2678f5f92fe1f)
(cherry picked from commit 804a40720bd5cf04415831b7bb33e98d11b2046e)
 2016-08-18 10:45:20 -07:00
Chris Nauroth
5c0db4e522 HADOOP-13459. hadoop-azure runs several test cases repeatedly, causing unnecessarily long running time. Contributed by Chris Nauroth. 
(cherry picked from commit 4ef1bc0bdb6b2c907431492e4c8395035fa96c84)
(cherry picked from commit 8ec3d702a38fdf0cd8b0b85cbe82c475ab2eaa8f)
 2016-08-02 22:32:44 -07:00
Steve Loughran
5930e813c5 HADOOP-13145 In DistCp, prevent unnecessary getFileStatus call when not preserving metadata. Contributed by Chris Nauroth. 2016-05-21 19:10:16 +01:00
Steve Loughran
ce17c9fd51 Revert "HADOOP-13145 In DistCp, prevent unnecessary getFileStatus call when not preserving metadata. Contributed by Chris Nauroth." 
This reverts commit edb912c57123ce4cba109a41a7dc21a45da7d27c.
 2016-05-20 14:00:26 +01:00
Steve Loughran
edb912c571 HADOOP-13145 In DistCp, prevent unnecessary getFileStatus call when not preserving metadata. Contributed by Chris Nauroth. 2016-05-20 12:21:35 +01:00
Chris Nauroth
1b5b85e560 HADOOP-12994. Specify PositionedReadable, add contract tests, fix problems. Contributed by Steve Loughran. 
(cherry picked from commit 843ee8d59d8bacbca0d87ccf0790772e39d16138)
(cherry picked from commit 26a23eff828a3f4fcbd4362761c329657603277d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
 2016-04-08 13:44:27 -07:00
Chris Nauroth
cb063d5699 HADOOP-12717. NPE when trying to rename a directory in Windows Azure Storage FileSystem. Contributed by Robert Yokota and Gaurav Kanade. 
(cherry picked from commit c50aad0f854b74ede9668e35db314b0a93be81b2)
(cherry picked from commit 2e60387401e1dff3fa36e32b267f2855c2e4ba49)
 2016-03-04 22:58:04 -08:00
Chris Nauroth
f563c53535 HADOOP-12853. Change WASB documentation regarding page blob support. Contributed by Madhumita Chakraborty. 
(cherry picked from commit f98dff329b1f94c9f53022baf0209fc1a7aaf7c2)
(cherry picked from commit bce33b51790b90f0655e36df3a0394f5691f8d6e)
 2016-03-01 13:19:03 -08:00
Chris Nauroth
60611c9c75 HADOOP-12846. Credential Provider Recursive Dependencies. Contributed by Larry McCay. 
(cherry picked from commit 7634d404b750eafa135a37fa275325d0398255fb)
(cherry picked from commit 2e761dabd8b0458bb324a32b28c4a8daf31a69b5)
 2016-02-28 11:23:09 -08:00
cnauroth
8ad907f06a HADOOP-12535. Run FileSystem contract tests with hadoop-azure. Contributed by Madhumita Chakraborty. 
(cherry picked from commit 3369a4f6916f12e9d6b97072badd1b176be443bd)
(cherry picked from commit 9ea19a861f10ca6281eaed2ee5032af21eaa53b3)
 2016-02-23 21:38:01 -08:00
cnauroth
606547dbd2 HADOOP-12555. WASB to read credentials from a credential provider. Contributed by Larry McCay. 
(cherry picked from commit 27b77751c1163ab4a1ce081a426e5190d1b8aff4)
(cherry picked from commit a5511debdb63770b58e406926343a1a189225d78)
 2016-02-22 17:30:53 -08:00
cnauroth
746d669847 HADOOP-11613. Remove commons-httpclient dependency from hadoop-azure. Contributed by Masatake Iwasaki. 
(cherry picked from commit d4f5fc23b208635e8f9a14c375d4101141aefa4a)
(cherry picked from commit 3f14eed58055d30b7c2dcb9240355920ff48af7e)
 2016-02-22 13:39:57 -08:00
cnauroth
b846401307 HADOOP-12780. During WASB atomic rename handle crash when one directory has been renamed but not file under it. Contributed by Madhumita Chakraborty. 
(cherry picked from commit 91a96eaa534dbb27e81b6c24bbb8138200a80a83)
(cherry picked from commit ac5a7935901255e5fc56b88574e72f875942b906)
 2016-02-12 15:50:28 -08:00
cnauroth
ffc0d98886 HADOOP-12635. Adding Append API support for WASB. Contributed by Dushyanth. 
(cherry picked from commit 8bc93db2e7c64830b6a662f28c8917a9eef4e7c9)
(cherry picked from commit 62d616621134cc5e68f4d5fd32f49ea4d731417c)
 2016-01-18 09:13:31 -08:00
cnauroth
e26f702866 HADOOP-12551. Introduce FileNotFoundException for WASB FileSystem API. Contributed by Dushyanth. 
(cherry picked from commit 0e76f1fceaaaeb66bdb4818f43b9a55fc092bf79)
(cherry picked from commit 813841e10bf9ac30481852bb5aeacc98921bb41c)
 2016-01-09 22:18:30 -08:00
cnauroth
f0f8e3ee7c HADOOP-12678. Handle empty rename pending metadata file during atomic rename in redo path. Contributed by Madhumita Chakraborty. 
(cherry picked from commit f0fa6d869b9abb5a900ea1c9eb4eb19ec9831dc4)
(cherry picked from commit debd13387daad868de466586370846c32040b7ad)
 2016-01-08 20:19:18 -08:00
cnauroth
506a517f9f HADOOP-12634. Change Lazy Rename Pending Operation Completion of WASB to address case of potential data loss due to partial copy. Contributed by Gaurav Kanade. 
(cherry picked from commit 978bbdfeb2d12efd6e750da6a14849e072fb814b)
(cherry picked from commit 4753676a34c4a3de29f9d20281de09533b73ce53)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2016-01-06 11:17:01 -08:00
Xiaoyu Yao
c16e126d16 HADOOP-12608. Fix exception message in WASB when connecting with anonymous credential. Contributed by Dushyanth. 
(cherry picked from commit 007a8decc61750720033e06b6d08861ac3788e41)
 2015-12-30 11:16:06 -08:00
cnauroth
81f7e8af22 HADOOP-12540. TestAzureFileSystemInstrumentation#testClientErrorMetrics fails intermittently due to assumption that a lease error will be thrown. Contributed by Gaurav Kanade. 
(cherry picked from commit 0fb1867fd62b5df664ad66386d6067db8fbf2317)
 2015-11-04 10:29:01 -08:00
cnauroth
c1d7b26e94 HADOOP-12508. delete fails with exception when lease is held on blob. Contributed by Gaurav Kanade. 
(cherry picked from commit 9e7dcab185abf2fdabb28f2799b9952b5664a4b0)
 2015-11-02 10:21:46 -08:00
cnauroth
c260d7ec78 HADOOP-12533. Introduce FileNotFoundException in WASB for read and seek API. Contributed by Dushyanth. 
(cherry picked from commit 28790692624177d89fb1e4f59e2f83a659fc3089)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2015-11-02 09:39:27 -08:00
cnauroth
c7f87dc2d0 HADOOP-12519. hadoop-azure tests should avoid creating a metrics configuration fiale in the module root directory. Contributed by Chris Nauroth. 
(cherry picked from commit 2d10cb8e00fd84be26b51f9fba819e3e5b6cdcbf)
 2015-10-28 09:44:21 -07:00
cnauroth
0377795e06 HADOOP-11685. StorageException complaining " no lease ID" during HBase distributed log splitting. Contributed by Duo Xu. 
(cherry picked from commit 1f7ecb0c84042783f9fcf3f77d7d889dc58c9ead)
 2015-10-27 22:56:33 -07:00
cnauroth
a23f79b92c HADOOP-12520. Use XInclude in hadoop-azure test configuration to isolate Azure Storage account keys for service integration tests. Contributed by Chris Nauroth. 
(cherry picked from commit 73822de7c38e189f7654444ff48d15cbe0df7404)
 2015-10-27 22:49:04 -07:00
cnauroth
b023539c19 HADOOP-12484. Single File Rename Throws Incorrectly In Potential Race Condition Scenarios. Contributed by Gaurav Kanade. 
(cherry picked from commit cb282d5b89fdece4719cc4ad37a6e27f13371534)
 2015-10-22 14:30:09 -07:00
cnauroth
10398daadf HADOOP-12334. Change Mode Of Copy Operation of HBase WAL Archiving to bypass Azure Storage Throttling after retries. Contributed by Gaurav Kanade. 
(cherry picked from commit 47641fcbc9c41f4a338d8899501e4a310d2e81ad)
 2015-10-22 12:21:44 -07:00
cnauroth
535762fadb HADOOP-12350. WASB Logging: Improve WASB Logging around deletes, reads and writes. Contributed by Dushyanth. 
(cherry picked from commit 5f6edb30c2bb648d5564c951edc25645e17e6636)
 2015-10-05 20:12:33 -07:00
Kihwal Lee
95b8f1f3a6 HADOOP-12446. Undeprecate createNonRecursive(). Contributed by Ted Yu. 
(cherry picked from commit 3abbdc929bde05f8819f5410cef1eaeb8940203f)
 2015-09-28 17:33:20 -05:00
cnauroth
b83d475252 HADOOP-12304. Applications using FileContext fail with the default file system configured to be wasb/s3/etc. Contributed by Chris Nauroth. 
(cherry picked from commit 3da0bedaec38714a62c3b7244e84bc4f981ddf45)
 2015-08-05 14:56:54 -07:00
cnauroth
bb080b32d4 HADOOP-12239. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB. Contributed by Duo Xu. 
(cherry picked from commit efa97243ecb84b3b468e732897cd685e3869f480)
 2015-07-22 11:17:00 -07:00
cnauroth
399df10c58 HADOOP-12089. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB. Contributed by Duo Xu. 
(cherry picked from commit 460e98f7b3ec84f3c5afcb2aad4f4e7031d16e3a)
 2015-06-29 13:49:12 -07:00
cnauroth
f5b0cce7fa HADOOP-12073. Azure FileSystem PageBlobInputStream does not return -1 on EOF. Contributed by Ivan Mitic. 
(cherry picked from commit c45784bc9031353b938f4756473937cca759b3dc)
 2015-06-08 22:42:24 -07:00
cnauroth
39f451e721 HADOOP-11959. WASB should configure client side socket timeout in storage client blob request options. Contributed by Ivan Mitic. 
(cherry picked from commit 94e7d49a6dab7e7f4e873dcca67e7fcc98e7e1f8)

Conflicts:
	hadoop-project/pom.xml
 2015-05-28 12:32:55 -07:00
cnauroth
91855c2340 HADOOP-11960. Enable Azure-Storage Client Side logging. Contributed by Dushyanth. 
(cherry picked from commit cb8e69a80cecb95abdfc93a787bea0bedef275ed)
 2015-05-14 22:22:46 -07:00
Vinayakumar B
14d88a85e4 HADOOP-11922. Misspelling of threshold in log4j.properties for tests in hadoop-tools (Contributed by Gabor Liptak) 
(cherry picked from commit f33efbc82709e75d9b4c4ed9835629668dbade7d)
 2015-05-08 10:23:27 +05:30
Jonathan Eagles
9ed6b77da1 HADOOP-11889. Make checkstyle runnable from root project (Gera Shegalov via jeagles) 
(cherry picked from commit 2d7363b27360e36fdd62546c0f9d0b1d78133f29)

Conflicts:
	pom.xml
 2015-05-01 19:04:41 -07:00
Tsuyoshi Ozawa
77297017d8 HADOOP-11720. [JDK8] Fix javadoc errors caused by incorrect or illegal tags in hadoop-tools. Contributed by Akira AJISAKA. 
(cherry picked from commit ef9946cd52d54200c658987c1dbc3e6fce133f77)
 2015-03-17 16:09:38 +09:00
cnauroth
dcc4850ff3 HADOOP-11693. Azure Storage FileSystem rename operations are throttled too aggressively to complete HBase WAL archiving. Contributed by Duo Xu. 
(cherry picked from commit 7a346bcb4fa5b56191ed00a39e72e51c9bdf1b56)
 2015-03-11 14:37:22 -07:00
Tsuyoshi Ozawa
b46f9e72db HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 2015-03-09 19:53:55 +09:00
cnauroth
02cadbd24b HADOOP-11642. Upgrade azure sdk version from 0.6.0 to 2.0.0. Contributed by Shashank Khandelwal and Ivan Mitic. 
(cherry picked from commit 608ebd52bafecf980e9726d397c786a9c2422eba)

Conflicts:
	hadoop-project/pom.xml
 2015-03-06 15:29:22 -08:00
cnauroth
1bc4c6808d HADOOP-11629. WASB filesystem should not start BandwidthGaugeUpdater if fs.azure.skip.metrics set to true. Contributed by Shanyu Zhao. 
(cherry picked from commit 5731c0e0d08c3048fafdf62a14ca7611be4df5d7)
 2015-02-25 09:09:15 -08:00
Tsuyoshi Ozawa
f92d4fa20f HADOOP-11546. Checkstyle failing: Unable to instantiate DoubleCheckedLockingCheck. (ozawa) 
(cherry picked from commit ba58da29f49da15f7eb6300f996e0ae7ba9d0d1e)
 2015-02-05 00:51:40 +09:00
cnauroth
4607ca2854 HADOOP-11523. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB. Contributed by Duo Xu. 
(cherry picked from commit f2c91098c400da6db0f5e8e49e9bf0e6444af531)
 2015-01-29 17:02:48 -08:00
cnauroth
fa2378d866 HADOOP-11441. Hadoop-azure: Change few methods scope to public. Contributed by Shashank Khandelwal. 
(cherry picked from commit ad55083f75cb920fde58302d7bc8857086c1978c)
 2015-01-29 13:07:59 -08:00
arp
0a5b28605f HADOOP-11395. Add site documentation for Azure Storage FileSystem integration. (Contributed by Chris Nauroth) 2014-12-19 18:54:35 -08:00
Haohui Mai
c28449d4b5 HADOOP-11381. Fix findbugs warnings in hadoop-distcp, hadoop-aws, hadoop-azure, and hadoop-openstack. Contributed by Li Lu. 
(cherry picked from commit 2e98ad34ce64a9e5184c53447004de20a637f927)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2014-12-17 14:57:14 -08:00
cnauroth
5c32c4b2e4 HADOOP-10809. SyncableDataOutputStream must implement sync method on branch-2. 2014-12-17 14:57:13 -08:00
cnauroth
535d98bbb3 HADOOP-11248. Add hadoop configuration to disable Azure Filesystem metrics collection. Contributed by Shanyu Zhao. 
(cherry picked from commit 734eeb4f357ad3210355a0d3fdbc80706a770d61)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2014-12-17 14:57:13 -08:00
cnauroth
1646cc9f29 HADOOP-11188. hadoop-azure: automatically expand page blobs when they become full. Contributed by Eric Hanson. 
(cherry picked from commit f4b7e99f4ebac5b0295b7f7f42eb5705af41f079)
 2014-12-17 14:57:13 -08:00
cnauroth
5a737026cc HADOOP-10809. hadoop-azure: page blob support. Contributed by Dexter Bradshaw, Mostafa Elhemali, Eric Hanson, and Mike Liddell. 
(cherry picked from commit 2217e2f8ff418b88eac6ad36cafe3a9795a11f40)
 2014-12-17 14:57:13 -08:00