96142efa2d
HADOOP-13621. s3:// should have been fully cut off from trunk. Contributed by Mingliang Liu.
2016-09-17 22:07:46 -07:00
53a12fa721
HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran
2016-09-14 15:45:04 +01:00
58ed4fa544
HADOOP-13341. Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTS
...
This commit includes the following changes:
HADOOP-13356. Add a function to handle command_subcommand_OPTS
HADOOP-13355. Handle HADOOP_CLIENT_OPTS in a function
HADOOP-13554. Add an equivalent of hadoop_subcmd_opts for secure opts
HADOOP-13562. Change hadoop_subcommand_opts to use only uppercase
HADOOP-13358. Modify HDFS to use hadoop_subcommand_opts
HADOOP-13357. Modify common to use hadoop_subcommand_opts
HADOOP-13359. Modify YARN to use hadoop_subcommand_opts
HADOOP-13361. Modify hadoop_verify_user to be consistent with hadoop_subcommand_opts (ie more granularity)
HADOOP-13564. modify mapred to use hadoop_subcommand_opts
HADOOP-13563. hadoop_subcommand_opts should print name not actual content during debug
HADOOP-13360. Documentation for HADOOP_subcommand_OPTS
This closes apache/hadoop#126
2016-09-12 11:10:00 -07:00
9faccd1046
HADOOP-13587. distcp.map.bandwidth.mb is overwritten even when -bandwidth flag isn't set. Contributed by Zoran Dimitrijevic
2016-09-12 08:26:08 -07:00
4b6d795f28
HADOOP-13540 improve section on troubleshooting s3a auth problems. Contributed by Steve Loughran
2016-09-09 18:55:32 +01:00
7fdfcd8a6c
HADOOP-13541 explicitly declare the Joda time version S3A depends on. Contributed by Stevel Loughran
2016-09-07 12:25:23 +01:00
d152557cf7
HADOOP-13447. Refactor S3AFileSystem to support introduction of separate metadata repository and tests. Contributed by Chris Nauroth.
2016-09-06 09:36:21 -07:00
d6d9cff21b
YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh)
2016-08-30 15:52:29 -07:00
8b57be108f
HADOOP-12608. Fix exception message in WASB when connecting with anonymous credential. Contributed by Dushyanth.
...
(cherry picked from commit 007a8decc61750720033e06b6d08861ac3788e41)
(cherry picked from commit a65e159c59
2016-08-29 15:30:49 -07:00
c37346d0e3
HADOOP-12726. Unsupported FS operations should throw UnsupportedOperationException. Contributed by Daniel Templeton.
2016-08-23 14:12:52 -07:00
6f9c346e57
HADOOP-13446. Support running isolated unit tests separate from AWS integration tests. Contributed by Chris Nauroth.
2016-08-23 07:18:49 -07:00
763f0497bb
HADOOP-13252. Tune S3A provider plugin mechanism. Contributed by Steve Loughran.
2016-08-19 10:48:10 -07:00
ae4db25443
HADOOP-13513. Java 1.7 support for org.apache.hadoop.fs.azure testcases. Contributed by Tibor Kiss.
2016-08-18 10:44:36 -07:00
0da69c324d
YARN-4676. Automatic and Asynchronous Decommissioning Nodes Status Tracking. Contributed by Diniel Zhi.
...
(cherry picked from commit d464483bf7
2016-08-18 07:27:23 -07:00
040c185d62
HADOOP-13405 doc for fs.s3a.acl.default indicates incorrect values. Contributed by Shen Yinjie
2016-08-18 14:36:55 +01:00
822d661b8f
HADOOP-13208. S3A listFiles(recursive=true) to do a bulk listObjects instead of walking the pseudo-tree of directories. Contributed by Steve Loughran.
2016-08-17 14:54:54 -07:00
3808876c73
HADOOP-13324. s3a tests don't authenticate with S3 frankfurt (or other V4 auth only endpoints). Contributed by Steve Loughran.
2016-08-16 17:05:52 -07:00
2ed58c40e5
HADOOP-13403. AzureNativeFileSystem rename/delete performance improvements. Contributed by Subramanyam Pattipaka.
2016-08-08 12:28:40 -07:00
4ef1bc0bdb
HADOOP-13459. hadoop-azure runs several test cases repeatedly, causing unnecessarily long running time. Contributed by Chris Nauroth.
2016-08-02 22:32:21 -07:00
b43de80031
HADOOP-13354. Update WASB driver to use the latest version (4.2.0) of SDK for Microsoft Azure Storage Clients. Contributed by Sivaguru Sankaridurg.
2016-07-27 15:50:38 -07:00
54fe17a607
HADOOP-9427. Use JUnit assumptions to skip platform-specific tests. Contributed by Gergely Novák.
2016-07-27 19:41:09 +09:00
86ae218893
HADOOP-13188 S3A file-create should throw error rather than overwrite directories. Contributed by Steve Loughran
2016-07-25 15:50:23 +01:00
be9e46b42d
HADOOP-13406 S3AFileSystem: Consider reusing filestatus in delete() and mkdirs(). Contributed by Rajesh Balamohan
2016-07-25 14:45:03 +01:00
7052ca8804
HADOOP-13389 TestS3ATemporaryCredentials.testSTS error when using IAM credentials. Contributed by Steven K. Wong
2016-07-23 18:00:08 +01:00
c60a68ce1d
HADOOP-13392. [Azure Data Lake] OAuth2 configuration should be default set to true when AdlFileSystem is configured. Contributed by Vishwajeet Dusane and Chris Douglas.
2016-07-22 09:16:38 -07:00
dca6335dbc
HADOOP-13393. Omit unsupported fs.defaultFS setting in ADLS documentation
2016-07-21 23:21:47 -07:00
12aa184479
HADOOP-13382. Remove unneeded commons-httpclient dependencies from POM files in Hadoop and sub-projects, while adding it in to the only place it is still needed, hadoop-openstack. Contributed by Matt Foley.
2016-07-21 14:43:57 -07:00
37362c2f92
HADOOP-13212 Provide an option to set the socket buffers in S3AFileSystem (Rajesh Balamohan)
2016-07-20 13:42:51 +01:00
da456ffd62
Preparing for 3.0.0-alpha2 development
2016-07-15 19:04:17 -07:00
c2f9cd584c
HADOOP-13368. DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should be O(1) operation. Contributed by Mingliang Liu.
2016-07-15 14:28:53 -07:00
819224dcf9
YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan
2016-07-11 22:36:20 -07:00
f292624bd8
HDFS-10300. TestDistCpSystem should share MiniDFSCluster. Contributed by John Zhuge.
2016-07-11 18:06:28 -07:00
6e597600f7
HADOOP-13283. Support reset operation for new global storage statistics and per FS storage stats. Contributed by Mingliang Liu.
2016-07-07 14:58:19 -07:00
aa42c7a6dd
HADOOP-13305. Define common statistics names across schemes. Contributed by Mingliang Liu.
2016-07-01 15:34:03 -07:00
96fa0f848b
HADOOP-12709 Cut s3:// from trunk. Contributed by Mingliang Liu.
2016-06-29 16:04:50 +01:00
8d202f1258
HADOOP-13235. Use Date and Time API in KafkaSink.
2016-06-29 07:40:12 -07:00
8113855b3a
HDFS-10396. Using -diff option with DistCp may get "Comparison method violates its general contract" exception. Contributed by Yongjun Zhang.
2016-06-28 23:15:13 -07:00
422c73a865
HADOOP-13034. Log message about input options in distcp lacks some items (Takashi Ohnishi via aw)
2016-06-28 07:21:04 -07:00
4ee3543625
HADOOP-13203 S3A: Support fadvise "random" mode for high performance readPositioned() reads. Contributed by Rajesh Balamohan and stevel.
2016-06-22 15:45:25 +01:00
b2c596cdda
HADOOP-13287. TestS3ACredentials#testInstantiateFromURL fails if AWS secret key contains +. Contributed by Chris Nauroth.
2016-06-21 11:28:52 -07:00
b7c4cf7129
HADOOP-13291. Probing stats in DFSOpsCountStatistics/S3AStorageStatistics should be correctly implemented. Contributed by Mingliang Liu.
2016-06-20 16:25:30 -07:00
51d16e7b38
HADOOP-13242. Authenticate to Azure Data Lake using client ID and keys. Contributed by Atul Sikaria.
2016-06-16 23:35:20 -07:00
127d2c7281
HADOOP-13241. document s3a better. Contributed by Steve Loughran.
2016-06-16 11:18:02 -07:00
4aefe119a0
HADOOP-3733. "s3x:" URLs break when Secret Key contains a slash, even if encoded. Contributed by Steve Loughran.
2016-06-16 11:13:35 -07:00
c9e71382a5
HADOOP-12875. [Azure Data Lake] Support for contract test and unit test cases. Contributed by Vishwajeet Dusane.
2016-06-16 09:46:05 -07:00
c77a1095dc
YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda)
2016-06-14 15:06:38 -07:00
9581fb715c
HADOOP-12666. Support Microsoft Azure Data Lake - as a file system in Hadoop. Contributed by Vishwajeet Dusane.
2016-06-09 14:33:31 -07:00
58c3486850
HADOOP-13175. Remove hadoop-ant from hadoop-tools. Contributed by Chris Douglas.
2016-06-09 13:49:52 -07:00
31ffaf76f2
HADOOP-12537 S3A to support Amazon STS temporary credentials. Contributed by Sean Mackrory.
2016-06-09 21:00:47 +01:00
656c460c0e
HADOOP-13237: s3a initialization against public bucket fails if caller lacks any credentials. Contributed by Chris Nauroth
2016-06-09 17:28:49 +01:00
Mingliang Liu
Steve Loughran
Allen Wittenauer
Ravi Prakash
Chris Nauroth
Arun Suresh
Xiaoyu Yao
Chris Douglas
Junping Du
Akira Ajisaka
mattf-horton
Andrew Wang
Jitendra Pandey
Jian He
Yongjun Zhang
Wangda Tan