524776625d
HADOOP-15715. ITestAzureBlobFileSystemE2E timing out with non-scale timeout of 10 min.
...
Contributed by Da Zhou
2018-09-18 11:48:46 +01:00
1cf38a38da
HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch.
...
Contributed by Steve Loughran.
2018-09-18 10:56:56 +01:00
26d0c63a1e
HADOOP-15754. s3guard: testDynamoTableTagging should clear existing config.
...
Contributed by Gabor Bota.
2018-09-17 22:40:08 +01:00
b4c23043d3
HADOOP-15757. ABFS: remove dependency on common-codec Base64.
...
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
26211019c8
HADOOP-15753. ABFS: support path "abfs://mycluster/file/path"
...
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
e5593cbd83
HADOOP-15694. ABFS: Allow OAuth credentials to not be tied to accounts.
...
Contributed by Sean Mackrory.
2018-09-17 19:54:01 +00:00
13c70e9ba3
HADOOP-15740. ABFS: Check variable names during initialization of AbfsClientThrottlingIntercept.
...
Contributed by Sneha Varma.
2018-09-17 19:54:01 +00:00
6801b30733
HADOOP-15728. ABFS: Add backward compatibility to handle Unsupported Operation
...
for storage account with no namespace feature.
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
347a52a866
Fixing findbugs and license issues related to:
...
HADOOP-15703. ABFS - Implement client-side throttling.
Contributed by Sneha Varma and Thomas Marquardt.
2018-09-17 19:54:01 +00:00
97f06b3fc7
HADOOP-15703. ABFS - Implement client-side throttling.
...
Contributed by Sneha Varma and Thomas Marquardt.
2018-09-17 19:54:01 +00:00
4410eacba7
HADOOP-15664. ABFS: Reduce test run time via parallelization and grouping.
...
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
81dc4a995c
HADOOP-15663. ABFS: Simplify configuration.
...
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
df57c6c3b1
HADOOP-15692. ABFS: extensible support for custom oauth.
...
Contributed by Junhua Gu and Rajeev Bansal.
2018-09-17 19:54:01 +00:00
dd2b22fa31
HADOOP-15682. ABFS: Add support for StreamCapabilities. Fix javadoc and checkstyle.
...
Contributed by Thomas Marquardt.
2018-09-17 19:54:01 +00:00
6b6f8cc2be
HADOOP 15688. ABFS: InputStream wrapped in FSDataInputStream twice.
...
Contributed by Sean Mackrory.
2018-09-17 19:54:01 +00:00
9c1e4e8139
HADOOP-15661. ABFS: Add support for ACL.
...
Contributed by Junhua Gu and Da Zhou.
2018-09-17 19:54:01 +00:00
9149b9703e
HADOOP-15660. ABFS: Add support for OAuth
...
Contributed by Da Zhou, Rajeev Bansal, and Junhua Gu.
2018-09-17 19:54:01 +00:00
d6a4f39bd5
HADOOP-15669. ABFS: Improve HTTPS Performance.
...
Contributed by Vishwajeet Dusane.
2018-09-17 19:54:01 +00:00
cc5cc60c41
Fixing issue due to commit 2b2399d6 after rebase onto trunk.
2018-09-17 19:54:01 +00:00
b54b0c1b67
HADOOP-15659. Code changes for bug fix and new tests.
...
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
ce03a93f78
HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests.
...
Contributed by Steve Loughran and Da Zhou.
2018-09-17 19:54:01 +00:00
a271fd0eca
HADOOP-15560. ABFS: removed dependency injection and unnecessary dependencies.
...
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
f044deedbb
HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit.
...
Contributed by Shane Mainali, Thomas Marquardt, Zichen Sun, Georgi Chalakov, Esfandiar Manii, Amit Singh, Dana Kaban, Da Zhou, Junhua Gu, Saher Ahwal, Saurabh Pant, James Baker, Shaoyu Zhang, Lawrence Chen, Kevin Chen and Steve Loughran
2018-09-17 19:54:01 +00:00
d7c0a08a1c
HADOOP-15426 Make S3guard client resilient to DDB throttle events and network failures (Contributed by Steve Loughran)
2018-09-12 21:04:49 -07:00
d32a8d5d58
HADOOP-14734 add option to tag DDB table(s) created. (Contributed by Gabor Bota and Abe Fine)
2018-09-12 16:36:01 -07:00
1f6c4545cf
HADOOP-15750. Remove obsolete S3A test ITestS3ACredentialsInURL. Contributed by Steve Loughran
2018-09-12 10:58:39 -07:00
47b72c87eb
HADOOP-15635. s3guard set-capacity command to fail fast if bucket is unguarded.
...
Contributed by Gabor Bota.
2018-09-12 09:12:38 -06:00
c44088ac19
YARN-8739. Fix jenkins issues for Node Attributes branch. Contributed by Sunil Govindan.
2018-09-12 16:01:01 +05:30
eb08543c7a
YARN-8103. Add CLI interface to query node attributes. Contributed by Bibin A Chundatt.
2018-09-12 16:01:01 +05:30
89b3ebd11e
YARN-8092. Expose Node Attributes info via RM nodes REST API. Contributed by Weiwei Yang.
2018-09-12 16:01:00 +05:30
87f63b6479
HADOOP-14833. Remove s3a user:secret authentication. Contributed by Steve Loughran
2018-09-11 17:18:42 -07:00
1eef0acfe5
HADOOP-15677. WASB: Add support for StreamCapabilities.
...
Contributed by Thomas Marquardt.
2018-09-10 14:24:41 +01:00
36c7c78260
HADOOP-15709 Move S3Guard LocalMetadataStore constants to org.apache.hadoop.fs.s3a.Constants (Contributed by Gabor Bota)
2018-09-07 10:25:20 -07:00
5a0babf765
HADOOP-15107. Stabilize/tune S3A committers; review correctness & docs.
...
Contributed by Steve Loughran.
2018-08-30 14:49:53 +01:00
e8d138ca7c
HADOOP-15680. ITestNativeAzureFileSystemConcurrencyLive times out.
...
Contributed by Andras Bokor.
2018-08-30 14:36:00 +01:00
2e6c1109dc
HADOOP-15667. FileSystemMultipartUploader should verify that UploadHandle has non-0 length.
...
Contributed by Ewan Higgs
2018-08-30 14:33:16 +01:00
8563fd67be
YARN-8632. Threads in SLS quit without logging exception. Contributed by Xianghao Lu.
2018-08-24 11:23:17 -07:00
96c4575d73
HDFS-13805. Journal Nodes should allow to format non-empty directories with -force option. Contributed by Surendra Singh Lilhore.
2018-08-24 08:14:57 +05:30
d7232857d8
HADOOP-14154 Persist isAuthoritative bit in DynamoDBMetaStore (Contributed by Gabor Bota)
2018-08-17 10:15:39 -07:00
0e832e7a74
HADOOP-15642. Update aws-sdk version to 1.11.375.
...
Contributed by Steve Loughran.
2018-08-16 09:58:46 -07:00
3e3963b035
HADOOP-15552. Move logging APIs over to slf4j in hadoop-tools - Part2. Contributed by Ian Pickering.
2018-08-16 00:31:59 +09:00
a13929ddcb
HADOOP-15645. ITestS3GuardToolLocal.testDiffCommand fails if bucket has per-bucket binding to DDB. Contributed by Steve Loughran.
2018-08-13 12:57:45 +02:00
da9a39eed1
HADOOP-15583. Stabilize S3A Assumed Role support.
...
Contributed by Steve Loughran.
2018-08-08 22:57:24 -07:00
2ec97abb2e
HADOOP-15576. S3A Multipart Uploader to work with S3Guard and encryption Originally contributed by Ewan Higgs with refinements by Steve Loughran.
2018-08-08 13:50:23 +02:00
7862f1523f
HADOOP-15400. Improve S3Guard documentation on Authoritative Mode implementation. (Contributed by Gabor Bota)
2018-08-07 20:13:09 -06:00
48673bc2a8
HADOOP-15626. FileContextMainOperationsBaseTest.testBuilderCreateAppendExistingFile fails on filesystems without append.
...
Contributed by Steve Loughran.
2018-08-03 16:06:00 -07:00
9fea5c9ee7
YARN-8175. Add support for Node Labels in SLS. Contributed by Abhishek Modi.
2018-07-31 09:36:34 -07:00
0857f116b7
HADOOP-15607. AliyunOSS: fix duplicated partNumber issue in AliyunOSSBlockOutputStream. Contributed by Jinhu Wu.
2018-07-30 10:53:44 +08:00
59adeb8d7f
HADOOP-15636. Follow-up from HADOOP-14918; restoring test under new name. Contributed by Gabor Bota.
2018-07-27 18:23:29 -06:00
3d3158cea4
YARN-8577. Fix the broken anchor in SLS site-doc. Contributed by Weiwei Yang.
2018-07-25 18:59:05 +05:30
45d9568aaa
HADOOP-15547/ WASB: improve listStatus performance.
...
Contributed by Thomas Marquardt.
(cherry picked from commit 749fff577e
2018-07-19 12:31:19 -07:00
752dcce5f4
YARN-8511. When AM releases a container, RM removes allocation tags before it is released by NM. (Weiwei Yang via wangda)
...
Change-Id: I6f9f409f2ef685b405cbff547dea9623bf3322d9
2018-07-16 10:54:41 -07:00
a08812a1b1
HADOOP-15349. S3Guard DDB retryBackoff to be more informative on limits exceeded. Contributed by Gabor Bota.
2018-07-12 17:24:01 +02:00
d503f65b66
HADOOP-15541. [s3a] Shouldn't try to drain stream before aborting
...
connection in case of timeout.
2018-07-10 17:52:57 +02:00
ca8b80bf59
HADOOP-15384. distcp numListstatusThreads option doesn't get to -delete scan.
...
Contributed by Steve Loughran.
2018-07-10 10:43:59 +01:00
93ac01cb59
HADOOP-15215 s3guard set-capacity command to fail on read/write of 0 (Gabor Bota)
2018-07-03 13:50:11 -07:00
1804a31515
HDFS-13536. [PROVIDED Storage] HA for InMemoryAliasMap. Contributed by Virajith Jalaparti.
2018-07-02 10:48:20 -07:00
73746c5da7
HDFS-13707. [PROVIDED Storage] Fix failing integration tests in ITestProvidedImplementation. Contributed by Virajith Jalaparti.
2018-06-29 09:56:13 -07:00
2b2399d623
HADOOP-15495. Upgrade commons-lang version to 3.7 in hadoop-common-project and hadoop-tools. Contributed by Takanobu Asanuma.
2018-06-28 14:37:22 +09:00
7a3c6e9c3c
HADOOP-15550. Avoid static initialization of ObjectMappers
2018-06-25 15:36:45 -07:00
c687a6617d
HADOOP-15423. Merge fileCache and dirCache into ine single cache in LocalMetadataStore. Contributed by Gabor Bota.
2018-06-25 14:59:41 -06:00
e16e5b307d
HDDS-177. Create a releasable ozonefs artifact
...
Contributed by Marton, Elek.
2018-06-24 01:05:04 -07:00
ca14fec02f
HDDS-184. Upgrade common-langs version to 3.7 in hadoop-tools/hadoop-ozone.
...
Contributed by Takanobu Asanuma.
2018-06-23 15:49:44 -07:00
55fad6a3de
HADOOP-15416. Clear error message in S3Guard diff if source not found. Contributed by Gabor Bota.
2018-06-22 11:36:56 -06:00
b089a06793
HADOOP-14918. Remove the Local Dynamo DB test option. Contributed by Gabor Bota.
2018-06-20 16:45:08 -06:00
f34744603e
HADOOP-15533. Make WASB listStatus messages consistent. Contributed by Esfandiar Manii
2018-06-17 23:12:18 -07:00
980031bb04
HADOOP-13186. Multipart Uploader API. Contributed by Ewan Higgs
2018-06-17 11:54:26 -07:00
f4c7c91123
HADOOP-15483. Upgrade jquery to version 3.3.1. Contributed by Lokesh Jain, Mukul Kumar Singh and Sunil Govindan.
2018-06-13 00:36:02 -07:00
c35481594f
YARN-8422. TestAMSimulator failing with NPE. Contributed by Giovanni Matteo Fumarola.
2018-06-12 10:59:50 -07:00
d901be6795
HADOOP-15506. Upgrade Azure Storage Sdk version to 7.0.0 and update corresponding code blocks.
...
Contributed by Esfandiar Manii.
2018-06-06 18:28:14 +01:00
3b34148c4f
HDDS-88. Create separate message structure to represent ports in DatanodeDetails.
...
Contributed by Nanda Kumar.
2018-05-30 08:52:07 -07:00
5f6769f796
HADOOP-15480 AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo (Gabor Bota)
2018-05-29 19:20:22 -07:00
30284d020d
HADOOP-14946 S3Guard testPruneCommandCLI can fail. Contributed by Gabor Bota.
2018-05-29 13:38:15 -07:00
8fdc993a99
HADOOP-15498. TestHadoopArchiveLogs (#testGenerateScript, #testPrepareWorkingDir) fails on Windows. Contributed by Anbang Hu.
2018-05-28 16:45:42 -07:00
d14e26b31f
HADOOP-15477. Make unjar in RunJar overrideable
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-05-28 17:29:59 +09:00
ba842847c9
HADOOP-15478. WASB: hflush() and hsync() regression.
...
Contributed by Thomas Marquardt.
2018-05-21 11:12:34 +01:00
cc3600aabd
HADOOP-15469. S3A directory committer commit job fails if _temporary directory created under dest.
...
Contributed by Steve Loughran.
(cherry picked from commit 170f1040d4
2018-05-17 19:08:55 +01:00
b670837469
HADOOP-15442. ITestS3AMetrics.testMetricsRegister can't know metrics source's name.
...
Contributed by Sean Mackrory.
2018-05-15 16:13:56 +01:00
ba051b0686
HADOOP-15354. hadoop-aliyun & hadoop-azure modules to mark hadoop-common as provided
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-05-10 18:38:08 +09:00
46e0f27862
HDDS-18. Ozone Shell should use RestClient and RpcClient. Contributed by Lokesh Jain.
2018-05-09 23:00:13 -07:00
1cfe7506f7
HADOOP-15356. Make HTTP timeout configurable in ADLS connector. Contributed by Atul Sikaria and Sean Mackrory.
2018-05-09 15:15:51 -06:00
343b51dcf4
HDDS-28. Ozone:Duplicate declaration in hadoop-tools/hadoop-ozone/pom.xml. Contributed by Tsz Wo Nicholas Sze.
2018-05-09 20:20:21 +05:30
208b97e969
HDDS-20. Ozone: Add support for rename key within a bucket for rpc client. Contributed by Lokesh Jain.
2018-05-09 19:06:07 +05:30
8981674bbc
HADOOP-15420 s3guard ITestS3GuardToolLocal failures in diff tests (Gabor Bota)
2018-05-08 18:58:42 -07:00
69aac696d9
HADOOP-13649 s3guard: implement time-based (TTL) expiry for LocalMetadataStore (Gabor Bota)
2018-05-08 15:29:54 -07:00
5b11b9fd41
HADOOP-15446. WASB: PageBlobInputStream.skip breaks HBASE replication.
...
Contributed by Thomas Marquardt
2018-05-07 11:54:08 +01:00
96c843f64b
HADOOP-15444 ITestS3GuardToolDynamo should only run with -Ddynamo (Aaron Fabbri)
2018-05-04 11:34:45 -07:00
85381c7b60
HADOOP-15434. Upgrade to ADLS SDK that exposes current timeout.
2018-05-02 21:30:31 -06:00
919865a34b
HADOOP-15239 S3ABlockOutputStream.flush() be no-op when stream closed. Contributed by Gabor Bota.
2018-04-30 16:02:57 -07:00
ba2db39c65
YARN-8152. Add chart in SLS to illustrate the throughput of the scheduler. Contributed by Tao Yang.
2018-04-27 15:36:21 +08:00
7d8bcf534a
HADOOP-13756 LocalMetadataStore#put(DirListingMetadata) should also put file metadata into fileHash. (Gabor Bota)
2018-04-26 20:41:29 -07:00
2adda92de1
HDFS-7240 git commit fixup
...
Signed-off-by: Owen O'Malley <omalley@apache.org>
2018-04-26 13:52:17 -07:00
ae8ac7f082
HDFS-13424. Ozone: Refactor MiniOzoneClassicCluster. Contributed by Nanda Kumar.
2018-04-26 05:36:04 -07:00
025058f251
HDFS-13446. Ozone: Fix OzoneFileSystem contract test failures. Contributed by Mukul Kumar Singh.
2018-04-26 05:36:04 -07:00
8b832f3c35
HDFS-13405. Ozone: Rename HDSL to HDDS.
...
Contributed by Ajay Kumar, Elek Marton, Mukul Kumar Singh, Shashikant Banerjee and Anu Engineer.
2018-04-26 05:36:04 -07:00
7ae9543a57
HDFS-13391. Ozone: Make dependency of internal sub-module scope as provided in maven. Contributed by Nanda kumar.
2018-04-26 05:36:04 -07:00
04e1287771
HDFS-13298. Ozone: Make ozone/hdsl/cblock modules turned off by default. Contributed by Elek, Marton.
2018-04-26 05:36:04 -07:00
4e61bc431e
HDFS-13258. Ozone: restructure Hdsl/Ozone code to separated maven subprojects.
...
Contributed by Elek Marton, Mukul Kumar Singh, Xiaoyu Yao, Ajay Kumar, Anu Engineer, Lokesh Jain, Nanda Kumar.
2018-04-26 05:36:04 -07:00
5e7164c614
HDFS-13137. Ozone: Ozonefs read fails because ChunkGroupInputStream#read does not iterate through all the blocks in the key.
...
Contributed by Mukul Kumar Singh.
2018-04-26 05:36:04 -07:00
61651dcf5c
HDFS-13108. Ozone: OzoneFileSystem: Simplified url schema for Ozone File System. Contributed by Elek, Marton.
2018-04-26 05:36:04 -07:00
Steve Loughran
Thomas Marquardt
Aaron Fabbri
Mingliang Liu
Sean Mackrory
bibinchundatt
Naganarasimha
Sunil G
Gabor Bota
Yufei Gu
Surendra Singh Lilhore
Akira Ajisaka
Ewan Higgs
Inigo Goiri
Sammi Chen
Wangda Tan
Todd Lipcon
Anu Engineer
Chris Douglas
Jitendra Pandey
Johan Gustavsson
Steve Loughran
Mukul Kumar Singh
Weiwei Yang
Owen O'Malley
Nanda kumar