Commit Graph

4548 Commits

Author SHA1 Message Date
Steve Loughran 70c95d15e3
HADOOP-13421. Switch to v2 of the S3 List Objects API in S3A.
Contributed by Aaron Fabbri

(cherry picked from commit 5bbca80428)
2017-09-08 12:08:36 +01:00
Xiao Chen 3a3762d9e7 HDFS-12400. Provide a way for NN to drain the local key cache before re-encryption.
(cherry picked from commit 420baed31fdd811ac94c74dc53502ee79f37ba91)
2017-09-07 20:51:09 -07:00
Jason Lowe b2c564bca1 HADOOP-14827. Allow StopWatch to accept a Timer parameter for tests. Contributed by Erik Krogen
(cherry picked from commit dd814946f6)
2017-09-06 16:05:31 -05:00
Chris Douglas 270bca6aec HADOOP-12077. Provide a multi-URI replication Inode for ViewFs. Contributed by Gera Shegalov
(cherry picked from commit 1f3bc63e67)
2017-09-05 23:52:08 -07:00
Wei-Chiu Chuang c16ecf0fe8 HADOOP-14688. Intern strings in KeyVersion and EncryptedKeyVersion. Contributed by Xiao Chen.
(cherry picked from commit ad32759fd9)
2017-09-05 15:13:20 -07:00
Jitendra Pandey 063b6d0c93 HADOOP-14674. Correct javadoc for getRandomizedTempPath. Contributed by Mukul Kumar Singh. 2017-09-01 10:17:12 -07:00
Allen Wittenauer a3fee475f7
HADOOP-14414. Calling maven-site-plugin directly for docs profile is unnecessary (Andras Bokor via aw) 2017-09-01 08:38:30 -07:00
Steve Loughran 621b43e254
HADOOP-13345 HS3Guard: Improved Consistency for S3A.
Contributed by: Chris Nauroth, Aaron Fabbri, Mingliang Liu, Lei (Eddy) Xu,
Sean Mackrory, Steve Loughran and others.
2017-09-01 14:13:41 +01:00
Allen Wittenauer 27359b7139
HADOOP-14781. Clarify that HADOOP_CONF_DIR shouldn't actually be set in hadoop-env.sh
Signed-off-by: Andrew Wang <wang@apache.org>
2017-08-31 21:10:52 -07:00
Allen Wittenauer 0adc3a0533
YARN-6721. container-executor should have stack checking
Signed-off-by: Chris Douglas <cdouglas@apache.org>
2017-08-31 19:39:31 -07:00
Allen Wittenauer 190410085b
HADOOP-14364. refresh changelog/release notes with newer Apache Yetus build
Signed-off-by: Andrew Wang <wang@apache.org>
2017-08-31 19:06:49 -07:00
Yiqun Lin ac12e153a8 HDFS-12317. HDFS metrics render error in the page of Github. Contributed by Yiqun Lin. 2017-08-31 17:23:17 +08:00
Allen Wittenauer 71bbb86d69
HADOOP-14670. Increase minimum cmake version for all platforms
Signed-off-by: Chris Douglas <cdouglas@apache.org>
2017-08-30 21:47:24 -07:00
Junping Du 41480233a9 HADOOP-14814. Fix incompatible API change on FsServerDefaults to HADOOP-14104. Contributed by Junping Du. 2017-08-30 13:30:14 -07:00
Rohith Sharma K S 32188d3295 YARN-7038. [Atsv2 Security] CollectorNodemanagerProtocol RPC interface doesn't work when service authorization is enabled. Contributed by Varun Saxena. 2017-08-30 11:29:54 +05:30
Steve Loughran 021974f4cb
HADOOP-14802. Add support for using container saskeys for all accesses.
Contributed by Sivaguru Sankaridurg
2017-08-29 19:02:43 +01:00
Arpit Agarwal 8b7cbe3840 HADOOP-14729. Upgrade JUnit 3 test cases to JUnit 4. Contributed by Ajay Kumar. 2017-08-25 09:55:46 -07:00
Steve Loughran 8196a07c32
HDFS-12344. LocatedFileStatus regression: no longer accepting null FSPermission.
Contributed by Ewan Higgs
2017-08-24 16:17:30 +01:00
John Zhuge 7e6463d2fb HADOOP-14251. Credential provider should handle property key deprecation. Contributed by John Zhuge. 2017-08-23 11:37:18 -07:00
Subru Krishnan 4249172e14 YARN-7053. Move curator transaction support to ZKCuratorManager. (Jonathan Hung via Subru). 2017-08-22 19:20:57 -07:00
Jason Lowe c379310212 HADOOP-14687. AuthenticatedURL will reuse bad/expired session cookies. Contributed by Daryn Sharp 2017-08-22 16:50:01 -05:00
Xiao Chen 4ec5acc704 HADOOP-14705. Add batched interface reencryptEncryptedKeys to KMS. 2017-08-22 07:47:39 -07:00
Arpit Agarwal 736ceab2f5 HDFS-12325. SFTPFileSystem operations should restore cwd. Contributed by Chen Liang. 2017-08-21 11:48:51 -07:00
Arpit Agarwal 2d105a2068 Revert "HADOOP-14732. ProtobufRpcEngine should use Time.monotonicNow to measure durations. Contributed by Hanisha Koneru."
This reverts commit 8bef4eca28.
2017-08-18 10:15:52 -07:00
Lei Xu 99e558b13b HADOOP-14398. Modify documents for the FileSystem Builder API. (Lei (Eddy) Xu) 2017-08-17 18:06:23 -07:00
Andrew Wang dd7916d3cd HDFS-12250. Reduce usage of FsPermissionExtension in unit tests. Contributed by Chris Douglas. 2017-08-17 09:35:36 -07:00
Alexandr Krasheninnikov 1f04cb45f7 HADOOP-14560. Make HttpServer2 backlog size configurable. Contributed by Alexander Krasheninnikov.
This closes #242.
2017-08-17 01:05:19 -07:00
Andrew Wang 96b3a6b972 HDFS-11082. Provide replicated EC policy to replicate files. Contributed by SammiChen. 2017-08-16 22:17:06 -07:00
Akira Ajisaka 08aaa4b36f
HDFS-12269. Better to return a Map rather than HashMap in getErasureCodingCodecs. Contributed by Huafeng Wang. 2017-08-17 13:20:27 +09:00
Subru Krishnan 75dd866bfb HADOOP-14773. Extend ZKCuratorManager API for more reusability. (Íñigo Goiri via Subru). 2017-08-15 16:53:59 -07:00
Chris Douglas 645a8f2a4d HADOOP-14726. Mark FileStatus::isDir as final 2017-08-14 21:57:20 -07:00
Arpit Agarwal 0446511398 HADOOP-14673. Remove leftover hadoop_xml_escape from functions. Contributed by Ajay Kumar. 2017-08-14 16:22:10 -07:00
Arpit Agarwal 8bef4eca28 HADOOP-14732. ProtobufRpcEngine should use Time.monotonicNow to measure durations. Contributed by Hanisha Koneru. 2017-08-14 15:53:35 -07:00
John Zhuge 7769e96149 HADOOP-14627. Support MSI and DeviceCode token provider in ADLS. Contributed by Atul Sikaria. 2017-08-13 00:22:34 -07:00
Subru Krishnan bbbf0e2a41 HADOOP-14741. Refactor curator based ZooKeeper communication into common library. (Íñigo Goiri via Subru). 2017-08-11 13:58:45 -07:00
Xiao Chen 07fff43f4a HADOOP-14760. Add missing override to LoadBalancingKMSClientProvider. 2017-08-11 11:41:41 -07:00
John Zhuge 582648befa HADOOP-14260. Configuration.dumpConfiguration should redact sensitive information. Contributed by John Zhuge. 2017-08-11 10:16:08 -07:00
Allen Wittenauer 4222c97108
HADOOP-10392. Use FileSystem#makeQualified(Path) instead of Path#makeQualified(FileSystem) (ajisakaa via aw) 2017-08-11 09:25:56 -07:00
Steve Loughran d964062f66
HADOOP-14754. TestCommonConfigurationFields failed: core-default.xml has 2 wasb properties missing in classes.
Contributed by John Zhuge.
2017-08-11 10:18:17 +01:00
Wei-Chiu Chuang a8b75466b2 HADOOP-14743. CompositeGroupsMapping should not swallow exceptions. Contributed by Wei-Chiu Chuang. 2017-08-10 09:35:27 -07:00
Steve Loughran 54356b1e83
HADOOP-14183. Remove service loader config file for wasb fs.
Contributed by Esfandiar Manii.
2017-08-10 16:46:33 +01:00
Haohui Mai 1db4788b7d HADOOP-14598. Blacklist Http/HttpsFileSystem in FsUrlStreamHandlerFactory. Contributed by Steve Loughran. 2017-08-08 16:33:18 -07:00
Jian He acf9bd8b1d YARN-6890. Not display killApp button on UI if UI is unsecured but cluster is secured. Contributed by Junping Du 2017-08-08 11:09:38 -07:00
Arpit Agarwal bc206806da HDFS-12264. DataNode uses a deprecated method IoUtils#cleanup. Contributed by Ajay Yadav. 2017-08-07 15:05:10 -07:00
Xiao Chen a3a9c976c3 HADOOP-14727. Socket not closed properly when reading Configurations with BlockReaderRemote. Contributed by Jonathan Eagles. 2017-08-07 10:25:52 -07:00
Akira Ajisaka a4eb7016cb
HDFS-12198. Document missing namenode metrics that were added recently. Contributed by Yiqun Lin. 2017-08-07 18:47:33 +09:00
Lei Xu fe3341786a HDFS-12251. Add document for StreamCapabilities. (Lei (Eddy) Xu) 2017-08-04 11:21:58 -07:00
Arpit Agarwal a6fdeb8a87 HADOOP-13963. /bin/bash is hard coded in some of the scripts. Contributed by Ajay Yadav. 2017-08-04 10:40:52 -07:00
wenxinhe 691bf5ec5d
HADOOP-14706. Adding a helper method to determine whether a log is Log4j implement.
This closes #257

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2017-08-04 14:35:57 +09:00
Andrew Wang f4c6b00a9f HDFS-12131. Add some of the FSNamesystem JMX values as metrics. Contributed by Erik Krogen. 2017-08-03 15:44:51 -07:00
Chris Douglas 12e44e7bda HDFS-6984. Serialize FileStatus via protobuf. 2017-08-02 12:12:48 -07:00
Sean Mackrory 1a1bf6b7d0 HADOOP-13595. Rework hadoop_usage to be broken up by clients/daemons/etc. Contributed by Allen Wittenauer. 2017-08-02 12:25:05 -06:00
Steve Loughran 5e4434f628
HADOOP-14709. Fix checkstyle warnings in ContractTestUtils.
Contributed by Thomas Marquardt.

(cherry picked from commit cb46792610)
2017-08-02 12:48:06 +01:00
Masatake Iwasaki 3a0b84d9c9 HADOOP-14702. Fix formatting issue and regression caused by conversion from APT to Markdown. Contributed by Doris Gu. 2017-08-02 06:06:25 +00:00
Junping Du cf6794ebe9 HADOOP-14683. FileStatus.compareTo binary compatible issue. Contributed by Akira Ajisaka. 2017-08-01 17:13:02 -07:00
Lei Xu 02cd71ba9d HADOOP-14495. Add set options interface to FSDataOutputStreamBuilder. (Lei (Eddy) Xu) 2017-08-01 16:00:53 -07:00
Andrew Wang a11c230236 HADOOP-14701. Configuration can log misleading warnings about an attempt to override final parameter. Contributed by Andrew Sherman. 2017-08-01 15:13:29 -07:00
Steve Loughran 778d4edd9a
HADOOP-14660 wasb: improve throughput by 34% when account limit exceeded.
Contributed by Thomas Marquardt.
2017-08-01 21:36:28 +01:00
Arpit Agarwal 72eba25e09 HDFS-14676. Wrong default value for fs.df.interval. Contributed by xiangguang zheng. 2017-08-01 13:31:44 -07:00
Lei Xu 9586b0e24f HADOOP-14397. Pull up the builder pattern to FileSystem and add AbstractContractCreateTest for it. (Lei (Eddy) Xu) 2017-07-31 20:12:40 -07:00
Andras Bokor abbf4129a2
HADOOP-14343. Wrong pid file name in error message when starting secure daemon
Signed-off-by: Allen Wittenauer <aw@apache.org>
2017-07-31 20:03:43 -07:00
Akira Ajisaka f14be0d241
HADOOP-14690. RetryInvocationHandler should override toString(). Contributed by Yeliang Cang. 2017-07-31 14:08:30 +09:00
Ravi Prakash 746189ad8c HADOOP-14229. hadoop.security.auth_to_local example is incorrect in the documentation. Contributed by Andras Bokor. 2017-07-28 11:43:36 -07:00
Yongjun Zhang c6330f22a5 HDFS-12190. Enable 'hdfs dfs -stat' to display access time. Contributed by Yongjun Zhang. 2017-07-27 16:48:24 -07:00
Kihwal Lee 27a1a5fde9 HADOOP-14578. Bind IPC connections to kerberos UPN host for proxy users. Contributed by Daryn Sharp. 2017-07-26 13:12:39 -05:00
Brahma Reddy Battula 6d983cca52 HADOOP-14455. ViewFileSystem#rename should support be supported within same nameservice with different mountpoints. Contributed by Brahma Reddy Battula. 2017-07-25 23:51:53 +08:00
Akira Ajisaka cca51e916b
HADOOP-14681. Remove MockitoMaker class. Contributed by Andras Bokor. 2017-07-25 15:24:56 +09:00
Jitendra Pandey f2921e51f0 HADOOP-14518. Customize User-Agent header sent in HTTP/HTTPS requests by WASB. Contributed by Georgi Chalakov. 2017-07-24 23:01:01 -07:00
Ravi Prakash 94ca52ae9e HADOOP-14597. Native compilation broken with OpenSSL-1.1.0. Contributed by Ravi Prakash. 2017-07-24 16:01:45 -07:00
Jason Lowe 24853bf32a YARN-6768. Improve performance of yarn api record toString and fromString. Contributed by Jonathan Eagles 2017-07-24 10:19:20 -05:00
Wei-Chiu Chuang 465c213439 HDFS-12094. Log torrent when none isa-l EC is used. Contributed by LiXin Ge. 2017-07-21 12:25:03 -07:00
Kihwal Lee b3269f7cc1 HADOOP-14659. UGI getShortUserName does not need to search the Subject. Contributed by Daryn Sharp. 2017-07-20 16:12:21 -05:00
Daniel Templeton c21c260392 HADOOP-14666. Tests use assertTrue(....equals(...)) instead of assertEquals() 2017-07-19 13:58:55 -07:00
Jason Lowe df180259b0 HADOOP-14669. GenericTestUtils.waitFor should use monotonic time. Contributed by Daniel Templeton 2017-07-19 09:41:22 -05:00
Jitendra Pandey 2843c688bc HADOOP-14642. wasb: add support for caching Authorization and SASKeys. Contributed by Sivaguru Sankaridurg. 2017-07-19 00:13:06 -07:00
Jason Lowe 5aa2bf231f HADOOP-14637. GenericTestUtils.waitFor needs to check condition again after max wait time. Contributed by Daniel Templeton 2017-07-18 16:23:41 -05:00
Akira Ajisaka ccaf036662
HADOOP-14539. Move commons logging APIs over to slf4j in hadoop-common. Contributed by Wenxin He. 2017-07-18 13:32:37 +09:00
Anu Engineer 02b141ac60 HDFS-11786. Add support to make copyFromLocal multi threaded. Contributed by Mukul Kumar Singh. 2017-07-16 10:59:34 -07:00
Xiao Chen 0a6d5c0cf1 HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 2017-07-14 22:22:53 -07:00
Masatake Iwasaki b61ab8573e HADOOP-14646. FileContextMainOperationsBaseTest#testListStatusFilterWithSomeMatches never runs. Contributed by Andras Bokor. 2017-07-13 21:41:43 +09:00
Akira Ajisaka 9144fd9e9b
HADOOP-14629. Improve exception checking in FileContext related JUnit tests. Contributed by Andras Bokor. 2017-07-12 11:35:50 +09:00
Steve Loughran d670c3a4da
HADOOP-14535 wasb: implement high-performance random access and seek of block blobs.
Contributed by Thomas Marquardt
2017-07-11 21:34:27 +01:00
Yiqun Lin 3a7f02b815 HDFS-12085. Reconfigure namenode heartbeat interval fails if the interval was set with time unit. Contributed by Weiwei Yang. 2017-07-11 20:40:11 +08:00
wenxinhe fce7951014
HADOOP-14638. Replace commons-logging APIs with slf4j in StreamPumper.
This closes #247

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2017-07-11 13:30:13 +09:00
Jitendra Pandey f1efa14fc6 HADOOP-10829. Iteration on CredentialProviderFactory.serviceLoader is thread-unsafe. Contributed by Benoy Antony and Rakesh R. 2017-07-10 17:48:27 -07:00
Steve Loughran 09653ea098
HADOOP-14634. Remove jline from main Hadoop pom.xml.
Contributed by Ray Chiang.
2017-07-10 11:53:13 +01:00
Andrew Wang f10864a820 Add release notes, changes, jdiff for 3.0.0-alpha4 2017-07-07 11:01:59 -07:00
Akira Ajisaka 7cd095272c
HADOOP-14587. Use GenericTestUtils.setLogLevel when available in hadoop-common. Contributed by Wenxin He. 2017-07-08 02:54:24 +09:00
Akira Ajisaka 8fc5dcc2a1
Revert "HADOOP-14587. Use GenericTestUtils.setLogLevel when available in hadoop-common. Contributed by Wenxin He."
This reverts commit 82cb2a6497.
2017-07-08 02:53:18 +09:00
Wei-Chiu Chuang 8153fe2bd3 HADOOP-14563. LoadBalancingKMSClientProvider#warmUpEncryptedKeys swallows IOException. Contributed by Rushabh S Shah. 2017-07-07 06:13:10 -07:00
Akira Ajisaka 82cb2a6497
HADOOP-14587. Use GenericTestUtils.setLogLevel when available in hadoop-common. Contributed by Wenxin He. 2017-07-07 15:00:47 +09:00
John Zhuge 946dd25675 HADOOP-14608. KMS JMX servlet path not backwards compatible. Contributed by John Zhuge. 2017-07-05 11:16:56 -07:00
Vinayakumar B a180ba4081 HADOOP-13414. Hide Jetty Server version header in HTTP responses. Contributed by Surendra Singth Lilhore. 2017-07-05 16:05:18 +05:30
Steve Loughran b17e655b70 HADOOP-14617. Add ReflectionUtils.logThreadInfo that accept slf4j logger API.
Contributed by Wenxin He.
2017-07-04 11:41:07 +01:00
Wenxin He f2aba1da30
HADOOP-14571. Deprecate public APIs relate to log4j1
This closes #244

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2017-07-04 18:55:20 +09:00
Steve Loughran 1aaa7f1eac
HADOOP-14615. Add ServiceOperations.stopQuietly that accept slf4j logger API.
Contributed by Wenxin He.
2017-07-04 10:48:02 +01:00
Andrew Wang af2773f609 Updating version for 3.0.0-beta1 development 2017-06-29 17:57:40 -07:00
Daniel Templeton 5a75f73893 HADOOP-14611. NetworkTopology.DEFAULT_HOST_LEVEL is unused
(Contributed by Chen Liang via Daniel Templeton)
2017-06-29 12:28:43 -07:00
Kai Zheng ea1da39b19 HADOOP-14479. Erasurecode testcase failures with native enabled. Contributed by Sammi Chen 2017-06-29 11:26:36 +08:00
Lei Xu d6df0fdbbd Add -E option in 'ls' to list erasure coding policy of each file and directory if applicable. Contributed by luhuichun via lei. 2017-06-28 13:47:23 -07:00