Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,529 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3653 Commits

Author SHA1 Message Date
Akira Ajisaka 6581f2dea3
HADOOP-7308. Remove unused TaskLogAppender configurations from log4j.properties. Contributed by Todd Lipcon and J.Andreina. 
(cherry picked from commit 7e58b2478c)
 2017-09-20 21:08:54 +09:00
Robert Kanter 4e36452d02 YARN-7162. Remove XML excludes file format (rkanter) 2017-09-15 09:53:55 -07:00
John Zhuge eaef8446f6 HADOOP-14864. FSDataInputStream#unbuffer UOE should include stream class name. Contributed by Bharat Viswanadham. 
(cherry picked from commit 7ee02d1065)
 2017-09-14 11:29:34 -07:00
Xiao Chen 0a8ed025e4 Revert "HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah." 
This reverts commit 5337c2511d.
 2017-09-12 19:53:14 -07:00
Jason Lowe c314a66353 HADOOP-14843. Improve FsPermission symbolic parsing unit test coverage. Contributed by Bharat Viswanadham 
(cherry picked from commit 86f4d1c66c)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/permission/TestFsPermission.java
 2017-09-12 17:00:22 -05:00
Steve Loughran a46204437b
HADOOP-14851 LambdaTestUtils.eventually() doesn't spin on Assertion failures. 
Contributed by Steve Loughran

(cherry picked from commit 180e814b08)
 2017-09-11 21:33:44 +01:00
Xuan 02ae3f8202 YARN-7072. Add a new log aggregation file format controller (xgong) 2017-09-10 23:18:31 -07:00
John Zhuge 1421196d20 HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria. 
(cherry picked from commit a4661850c1)
 2017-09-08 14:15:51 -07:00
Jason Lowe f39a444120 HADOOP-14827. Allow StopWatch to accept a Timer parameter for tests. Contributed by Erik Krogen 
(cherry picked from commit dd814946f6)
 2017-09-06 16:05:58 -05:00
Wei-Chiu Chuang aada99a376 HADOOP-14688. Intern strings in KeyVersion and EncryptedKeyVersion. Contributed by Xiao Chen. 
(cherry picked from commit ad32759fd9)
(cherry picked from commit c16ecf0fe8)
 2017-09-05 15:14:13 -07:00
Yiqun Lin 41d8e4e9b3 HDFS-12317. HDFS metrics render error in the page of Github. Contributed by Yiqun Lin. 2017-09-01 10:13:01 +08:00
Steve Loughran 25e9c29608
HADOOP-14802. Add support for using container saskeys for all accesses. 
Contributed by Sivaguru Sankaridurg

(cherry picked from commit 021974f4cb)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
 2017-08-30 22:40:50 +01:00
Junping Du 7dd5bb9492 HADOOP-14814. Fix incompatible API change on FsServerDefaults to HADOOP-14104. Contributed by Junping Du. 
(cherry picked from commit 41480233a9)
 2017-08-30 13:30:48 -07:00
Arpit Agarwal ae69c99068 HDFS-11577. Combine the old and the new chooseRandom for better performance. Contributed by Chen Liang. 2017-08-24 11:17:36 -07:00
John Zhuge 6e8c696651 HADOOP-14251. Credential provider should handle property key deprecation. Contributed by John Zhuge. 
(cherry picked from commit 7e6463d2fb)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
 2017-08-23 11:59:47 -07:00
Subru Krishnan a45ffdcddc YARN-7053. Move curator transaction support to ZKCuratorManager. (Jonathan Hung via Subru). 
(cherry picked from commit 4249172e14)
 2017-08-22 19:21:35 -07:00
Jason Lowe 45c1f8ac8e HADOOP-14687. AuthenticatedURL will reuse bad/expired session cookies. Contributed by Daryn Sharp 
(cherry picked from commit c379310212)
 2017-08-22 16:53:51 -05:00
Arpit Agarwal b3ea11dfdb HDFS-11482. Add storage type demand to into DFSNetworkTopology#chooseRandom. Contributed by Chen Liang. 2017-08-21 14:07:59 -07:00
Arpit Agarwal d5379cc564 HDFS-12325. SFTPFileSystem operations should restore cwd. Contributed by Chen Liang. 2017-08-20 23:41:16 -07:00
Arpit Agarwal d8d2a221f8 HDFS-11450. HDFS specific network topology classes with storage type info included. Contributed by Chen Liang. 2017-08-18 11:43:17 -07:00
Arpit Agarwal 850593133b Revert "HADOOP-14732. ProtobufRpcEngine should use Time.monotonicNow to measure durations. Contributed by Hanisha Koneru." 
This reverts commit bdc425057e.
 2017-08-18 10:16:13 -07:00
Mingliang Liu 03d15fa58e HDFS-11430. Separate class InnerNode from class NetworkTopology and make it extendable. Contributed by Tsz Wo Nicholas Sze 2017-08-16 16:06:13 -07:00
Steve Loughran d5680c08ed
HADOOP-14660. wasb: improve throughput by 34% when account limit exceeded. 
Contributed by Thomas Marquardt.
 2017-08-16 13:49:47 +01:00
Subru Krishnan ad46a90872 HADOOP-14773. Extend ZKCuratorManager API for more reusability. (Íñigo Goiri via Subru). 
(cherry picked from commit 75dd866bfb)
 2017-08-15 16:54:48 -07:00
Arpit Agarwal bdc425057e HADOOP-14732. ProtobufRpcEngine should use Time.monotonicNow to measure durations. Contributed by Hanisha Koneru. 2017-08-14 15:53:43 -07:00
Subru Krishnan a70efb6138 HADOOP-14741. Refactor curator based ZooKeeper communication into common library. (Íñigo Goiri via Subru). 2017-08-14 11:03:50 -07:00
John Zhuge 0efc590e6d HADOOP-14627. Support MSI and DeviceCode token provider in ADLS. Contributed by Atul Sikaria. 
(cherry picked from commit 7769e96149)
 2017-08-13 00:27:33 -07:00
John Zhuge e208a4e7ee HADOOP-14260. Configuration.dumpConfiguration should redact sensitive information. Contributed by John Zhuge. 
(cherry picked from commit 582648befa)
 2017-08-11 10:54:41 -07:00
John Zhuge 76c4aee2e3 HADOOP-13628. Support to retrieve specific property from configuration via REST API. Contributed by Weiwei Yang 
(cherry picked from commit 00160f71b6)
 2017-08-11 10:54:30 -07:00
John Zhuge 9ba3268fa3 HADOOP-13588. ConfServlet should respect Accept request header. Contributed by Weiwei Yang 
(cherry picked from commit 59d59667a8)
 2017-08-11 10:54:25 -07:00
Steve Loughran 9c6e8ece98
HADOOP-14754. TestCommonConfigurationFields failed: core-default.xml has 2 wasb properties missing in classes. 
Contributed by John Zhuge.

(cherry picked from commit d964062f66)
 2017-08-11 10:58:26 +01:00
Wei-Chiu Chuang 304cc1a856 HADOOP-14743. CompositeGroupsMapping should not swallow exceptions. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit a8b75466b2)
 2017-08-10 09:36:21 -07:00
Steve Loughran 96befde480
HADOOP-14183. Remove service loader config file for wasb fs. 
Contributed by Esfandiar Manii.

(cherry picked from commit 54356b1e83)
 2017-08-10 16:49:51 +01:00
Haohui Mai 3e2753daa1 HADOOP-14598. Blacklist Http/HttpsFileSystem in FsUrlStreamHandlerFactory. Contributed by Steve Loughran. 2017-08-08 16:33:54 -07:00
Jian He ad1567c487 YARN-6890. Not display killApp button on UI if UI is unsecured but cluster is secured. Contributed by Junping Du 2017-08-08 11:10:29 -07:00
Yiqun Lin 817c0f4564 HDFS-12198. Document missing namenode metrics that were added recently. Contributed by Yiqun Lin. 2017-08-08 19:30:03 +08:00
Arpit Agarwal 2906c494c5 HDFS-12264. DataNode uses a deprecated method IoUtils#cleanup. Contributed by Ajay Yadav. 2017-08-07 15:33:36 -07:00
Xiao Chen 050c50e3c5 HADOOP-14727. Socket not closed properly when reading Configurations with BlockReaderRemote. Contributed by Jonathan Eagles. 
(cherry picked from commit 9a755e6da7158783789f55796644578330653efb)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
 2017-08-07 10:26:10 -07:00
Lei Xu d87a92bf55 HDFS-12251. Add document for StreamCapabilities. (Lei (Eddy) Xu) 
(cherry picked from commit fe3341786a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md
 2017-08-04 11:25:21 -07:00
wenxinhe 11e1e0c275
HADOOP-14706. Adding a helper method to determine whether a log is Log4j implement. 
This closes #258

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2017-08-04 14:42:35 +09:00
Andrew Wang 2da20df1dd HDFS-12131. Add some of the FSNamesystem JMX values as metrics. Contributed by Erik Krogen. 2017-08-03 15:45:21 -07:00
Steve Loughran cb46792610
HADOOP-14709. Fix checkstyle warnings in ContractTestUtils. 
Contributed by Thomas Marquardt.
 2017-08-02 12:46:30 +01:00
Masatake Iwasaki d9a07b3904 HADOOP-14702. Fix formatting issue and regression caused by conversion from APT to Markdown. Contributed by Doris Gu. 2017-08-02 06:21:37 +00:00
Andrew Wang 5467d1a4d1 HADOOP-14701. Configuration can log misleading warnings about an attempt to override final parameter. Contributed by Andrew Sherman. 
(cherry picked from commit a11c230236)

 Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
 2017-08-01 15:18:18 -07:00
Arpit Agarwal d19504866c HDFS-14676. Wrong default value for fs.df.interval. Contributed by xiangguang zheng. 2017-08-01 13:31:55 -07:00
Lei Xu f09d20cffb HADOOP-14397. Pull up the builder pattern to FileSystem and add AbstractContractCreateTest for it. (Lei (Eddy) Xu) 
(cherry picked from commit 667ee003bf47e44beb3fdff8d06a7264a13dd22c)
 2017-07-31 20:07:13 -07:00
Akira Ajisaka 9c8a296169
HADOOP-14690. RetryInvocationHandler should override toString(). Contributed by Yeliang Cang. 
(cherry picked from commit f14be0d241)
 2017-07-31 14:10:18 +09:00
Brahma Reddy Battula 8bfb9971ca HADOOP-14455. ViewFileSystem#rename should support be supported within same nameservice with different mountpoints. Contributed by Brahma Reddy Battula. 2017-07-30 14:01:41 +08:00
Yongjun Zhang c3e8c7d56e HDFS-12190. Enable 'hdfs dfs -stat' to display access time. Contributed by Yongjun Zhang. 
(cherry picked from commit c6330f22a5)
 2017-07-27 17:05:30 -07:00
Kihwal Lee 4edb849611 HADOOP-14578. Bind IPC connections to kerberos UPN host for proxy users. Contributed by Daryn Sharp. 
(cherry picked from commit 27a1a5fde9)
 2017-07-26 13:16:34 -05:00
Jitendra Pandey ab95086ed1 HADOOP-14518. Customize User-Agent header sent in HTTP/HTTPS requests by WASB. Contributed by Georgi Chalakov. 2017-07-25 08:31:35 -07:00
Akira Ajisaka 22f237c5d9
HADOOP-14681. Remove MockitoMaker class. Contributed by Andras Bokor. 
(cherry picked from commit cca51e916b)
 2017-07-25 15:25:36 +09:00
Jason Lowe db7c5636b6 YARN-6768. Improve performance of yarn api record toString and fromString. Contributed by Jonathan Eagles 
(cherry picked from commit 24853bf32a)
 2017-07-24 10:22:31 -05:00
Allen Wittenauer daa7685344
HADOOP-13178. TestShellBasedIdMapping.testStaticMapUpdate doesn't work on OS X (Kai Sasaki via aw) 
(cherry picked from commit d8c1fd1944)
 2017-07-24 11:57:10 +09:00
wenxinhe 7e583a3824
HADOOP-14539. Move commons logging APIs over to slf4j in hadoop-common. 
This closes #251

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2017-07-24 11:14:08 +09:00
Kihwal Lee a735916b2c HADOOP-14659. UGI getShortUserName does not need to search the Subject. Contributed by Daryn Sharp. 
(cherry picked from commit b3269f7cc1)
 2017-07-20 16:15:06 -05:00
Daniel Templeton 1d629c95af HADOOP-14666. Tests use assertTrue(....equals(...)) instead of assertEquals() 
(cherry picked from commit c21c260392)
 2017-07-19 14:01:10 -07:00
Jitendra Pandey 23b920cd7a HADOOP-14642. wasb: add support for caching Authorization and SASKeys. Contributed by Sivaguru Sankaridurg. 2017-07-19 11:37:36 -07:00
Jason Lowe 057631c031 HADOOP-14669. GenericTestUtils.waitFor should use monotonic time. Contributed by Daniel Templeton 
(cherry picked from commit df180259b0)
 2017-07-19 09:42:38 -05:00
Jason Lowe 9b961db569 HADOOP-14637. GenericTestUtils.waitFor needs to check condition again after max wait time. Contributed by Daniel Templeton 
(cherry picked from commit 5aa2bf231f)
 2017-07-18 16:27:34 -05:00
Xiao Chen 5337c2511d HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 
(cherry picked from commit aa395c1ec9a170f2da732c6e6114c60412161d40)
 2017-07-14 22:23:07 -07:00
Masatake Iwasaki 956e83c100 HADOOP-14646. FileContextMainOperationsBaseTest#testListStatusFilterWithSomeMatches never runs. Contributed by Andras Bokor. 
(cherry picked from commit b61ab8573e)
 2017-07-13 21:43:10 +09:00
Akira Ajisaka 2653cf4466
HADOOP-14629. Improve exception checking in FileContext related JUnit tests. Contributed by Andras Bokor. 
(cherry picked from commit 9144fd9e9b)
 2017-07-12 11:40:48 +09:00
Steve Loughran 41e83b2ca2
HADOOP-14535 wasb: implement high-performance random access and seek of block blobs. 
Contributed by Thomas Marquardt

(cherry picked from commit d670c3a4da)
 2017-07-11 21:45:24 +01:00
wenxinhe e494a169ed
HADOOP-14638. Replace commons-logging APIs with slf4j in StreamPumper. 
This closes #247

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit fce7951014)
 2017-07-11 13:36:56 +09:00
Jitendra Pandey b82485d6fe HADOOP-10829. Iteration on CredentialProviderFactory.serviceLoader is thread-unsafe. Contributed by Benoy Antony and Rakesh R. 2017-07-10 17:55:02 -07:00
Akira Ajisaka 1c40df5bcd
HADOOP-14587. Use GenericTestUtils.setLogLevel when available in hadoop-common. Contributed by Wenxin He. 2017-07-08 03:01:18 +09:00
Wei-Chiu Chuang 730b21e9f9 HADOOP-14563. LoadBalancingKMSClientProvider#warmUpEncryptedKeys swallows IOException. Contributed by Rushabh S Shah. 
(cherry picked from commit 8153fe2bd3)
 2017-07-07 06:14:07 -07:00
Vinayakumar B 1e8301b215 HADOOP-13414. Hide Jetty Server version header in HTTP responses. Contributed by Surendra Singth Lilhore. 2017-07-05 16:06:19 +05:30
Steve Loughran 3404824854 HADOOP-14617. Add ReflectionUtils.logThreadInfo that accept slf4j logger API. 
Contributed by Wenxin He.

(cherry picked from commit 128d8bc3e9d792fde9cacd9e1d786db8ef682b0b)
 2017-07-04 11:40:46 +01:00
Wenxin He ca8102962f
HADOOP-14571. Deprecate public APIs relate to log4j1 
This closes #244

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit f2aba1da30)
 2017-07-04 18:56:41 +09:00
Steve Loughran 4ceb3bd4b3
HADOOP-14615. Add ServiceOperations.stopQuietly that accept slf4j logger API. 
Contributed by Wenxin He.

(cherry picked from commit 1aaa7f1eac)
 2017-07-04 10:49:20 +01:00
Daniel Templeton 367f525e81 HADOOP-14611. NetworkTopology.DEFAULT_HOST_LEVEL is unused 
(Contributed by Chen Liang via Daniel Templeton)

(cherry picked from commit 5a75f73893)
 2017-06-29 16:56:55 -07:00
John Zhuge 1272ee153b HADOOP-14024. KMS JMX endpoint throws ClassNotFoundException. Contributed by John Zhuge. 2017-06-29 15:54:30 -07:00
Xiao Chen 7e66e133dd HADOOP-14515. Addendum. Specifically configure zookeeper-related log levels in KMS log4j. 
(cherry picked from commit aff7a3f8eddd55501366a43d2f44f531c6e8ef48)
 2017-06-27 23:49:16 -07:00
Akira Ajisaka b59b96deee
HDFS-11992. Replace commons-logging APIs with slf4j in FsDatasetImpl. Contributed by hu xiaodong. 2017-06-27 03:28:09 +09:00
Akira Ajisaka 239b382153
HADOOP-14549. Use GenericTestUtils.setLogLevel when available in hadoop-tools. Contributed by wenxin he. 
(cherry picked from commit a594f96fb1)
 2017-06-26 16:41:44 +09:00
Arpit Agarwal 4b420e0678 HADOOP-14543. ZKFC should use getAversion() while setting the zkacl. Contributed by Brahma Reddy Battula. 2017-06-23 16:50:56 -07:00
Kihwal Lee 0719345909 Addendum for HADOOP-14146. Use commons Base64 instead of jdk8 Base64. 2017-06-23 13:26:29 -05:00
Kihwal Lee 4c6184b41c Addendum for HADOOP-14146 jdk7 support. 2017-06-23 09:57:51 -05:00
Steve Loughran 5858bc70a2
HADOOP-14568. GenericTestUtils#waitFor missing parameter verification. 
Contributed by Yiqun Lin

(cherry picked from commit e5db9af8a0)
 2017-06-23 14:41:08 +01:00
Arpit Agarwal f9f6ef83eb HDFS-12010. TestCopyPreserveFlag fails consistently because of mismatch in access time. Contributed by Mukul Kumar Singh. 2017-06-22 15:43:00 -07:00
Akira Ajisaka 897d5a4d24
HADOOP-14542. Add IOUtils.cleanupWithLogger that accepts slf4j logger API. Contributed by Chen Liang. 
(cherry picked from commit b64951905e)
 2017-06-22 17:43:33 +09:00
Daryn Sharp d4115d71b5 HADOOP-14146. KerberosAuthenticationHandler should authenticate with SPN in AP-REQ. Contributed by Daryn Sharp 2017-06-21 11:05:12 -05:00
Yongjun Zhang ba487ead42 HADOOP-14429. FTPFileSystem#getFsAction always returns FsAction.NONE. (Hongyuan Li via Yongjun Zhang) 
(cherry picked from commit 5157f6c46e)
 2017-06-20 17:07:01 -07:00
Brahma Reddy Battula 933fad2e17 HADOOP-14533. Size of args cannot be less than zero in TraceAdmin#run as its linkedlist. Contributed by Weisen Han. 
(cherry picked from commit 2e9daa2e27)
 2017-06-20 20:45:33 +08:00
Xiao Chen b9bd72c676 HADOOP-14515. Specifically configure zookeeper-related log levels in KMS log4j. 
(cherry picked from commit 4cdae5d88ef7851ae284a115868dc4bbe24ae1bd)
 2017-06-19 23:12:28 -07:00
Akira Ajisaka 6d82db40d5
HADOOP-14540. Replace MRv1 specific terms in HostsFileReader. Contributed by hu xiaodong. 
(cherry picked from commit 300804595e)
 2017-06-19 19:40:19 +09:00
Akira Ajisaka 2d0b6ab65d
HADOOP-14538. Fix TestFilterFileSystem and TestHarFileSystem failures after DistributedFileSystem.append API. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 7ade5124b8)
 2017-06-19 19:10:13 +09:00
Masatake Iwasaki 023dcdc73d HADOOP-14424. Add CRC32C performance test. Contributed by LiXin Ge. 
(cherry picked from commit 75043d3429)
 2017-06-19 09:18:02 +09:00
Lei Xu 33afa1fdca HADOOP-14395. Provide Builder pattern for DistributedFileSystem.append. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 6460df21a0)

    Conflicts:
    hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
 2017-06-16 17:32:29 -07:00
Arpit Agarwal 9529513f18 HADOOP-14503. Make RollingAverages a mutable metric. Contributed by Hanisha Koneru. 2017-06-16 16:41:08 -07:00
Xiao Chen 2afe9722af HADOOP-14524. Make CryptoCodec Closeable so it can be cleaned up proactively. 2017-06-16 09:49:41 -07:00
Steve Loughran 4e6348f346
HADOOP-14486 TestSFTPFileSystem#testGetAccessTime test failure using openJDK 1.8.0. 
Contributed by Hongyuan Li.

(cherry picked from commit 942ecbbc98)
 2017-06-16 09:46:34 +01:00
Akira Ajisaka 12cb3456d7
HADOOP-14289. Move log4j APIs over to slf4j in hadoop-common. 
(cherry picked from commit e855cc4ed4)
 2017-06-16 13:32:16 +09:00
Lei Xu abac844c90 HADOOP-14394. Provide Builder pattern for DistributedFileSystem.create. (lei) 
(cherry picked from commit 5fbec46525d6d49837d934556b59ba77bd2301a8)

    Conflicts:
    hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/NameNodeConnector.java
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicies.java
 2017-06-15 10:59:11 -07:00
Xiao Chen 7d81b0beab HADOOP-14523. OpensslAesCtrCryptoCodec.finalize() holds excessive amounts of memory. Contributed by Misha Dmitriev. 
(cherry picked from commit ef8edab930)
 2017-06-15 10:41:34 -07:00
Steve Loughran 31446f8e10
HADOOP-14506. Add create() contract test that verifies ancestor dir creation. 
Contributed by Sean Mackrory.

(cherry picked from commit d780a67864)
 2017-06-15 14:51:50 +01:00
Jonathan Eagles f901132a8a HADOOP-14501. Switch from aalto-xml to woodstox to handle odd XML features (jeagles) 2017-06-12 17:19:46 -05:00
Daniel Templeton 6d7e847cfc HADOOP-14310. RolloverSignerSecretProvider.LOG should be @VisibleForTesting 
(Contributed by Arun Shanmugam Kumar via Daniel Templeton)

(cherry picked from commit 86368cc766)
 2017-06-12 09:56:35 -07:00
Mingliang Liu ae2457e1dd HADOOP-14465. LdapGroupsMapping - support user and group search base. Contributed by Shwetha G S and Mingliang Liu 
(cherry picked from commit a2121cb0d9)
 2017-06-09 16:26:10 -07:00
Xiao Chen aaf533e8a0 HADOOP-13854. KMS should log error details in KMSExceptionsProvider. 
(cherry picked from commit c4b5c32669)
 2017-06-08 22:38:31 -07:00
Xiao Chen ffaf24e308 HADOOP-13174. Add more debug logs for delegation tokens and authentication. 
(cherry picked from commit 4a56bde6ba1f72588a25cd96acc76089706cb786)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticationHandler.java
 2017-06-08 21:34:09 -07:00
John Zhuge 87f4428de0 HADOOP-14511. WritableRpcEngine.Invocation#toString NPE on null parameters. Contributed by John Zhuge. 2017-06-08 21:26:02 -07:00
John Zhuge 2448d84572 HDFS-11861. ipc.Client.Connection#sendRpcRequest should log request name. Contributed by John Zhuge. 
(cherry picked from commit 5672ae7b37)
 2017-06-07 15:54:08 -07:00
Kihwal Lee f7e597b56d HADOOP-14035. Reduce fair call queue backoff's impact on clients. Contributed by Daryn Sharp. 2017-06-06 12:19:44 -05:00
Brahma Reddy Battula e889c826d7 HADOOP-14431. ModifyTime of FileStatus returned by SFTPFileSystem's getFileStatus method is wrong. Contributed by Hongyuan Li. 
(cherry picked from commit 66c6fd8314)
 2017-06-06 12:32:37 +08:00
Mingliang Liu 867026d94e HADOOP-14428. s3a: mkdir appears to be broken. Contributed by Mingliang Liu 
(cherry picked from commit 6aeda55bb8)
 2017-06-05 11:31:13 -07:00
Brahma Reddy Battula ca76378aad HADOOP-14440. Add metrics for connections dropped. Contributed by Eric Badger. 
(cherry picked from commit abdd609e51)
 2017-06-06 00:21:59 +08:00
Brahma Reddy Battula ac2af66e77 HDFS-11905. Fix license header inconsistency in hdfs. Contributed by Yeliang Cang. 2017-06-02 22:22:43 +08:00
Andrew Wang d0a4e7e95d HDFS-11383. Intern strings in BlockLocation and ExtendedBlock. Contributed by Misha Dmitriev. 
(cherry picked from commit 7101477e47)
 2017-06-01 15:20:22 -07:00
Kihwal Lee 5b0baeab5e HDFS-5042. Completed files lost after power failure. Contributed by Vinayakumar B. 2017-05-31 11:02:34 -05:00
Arpit Agarwal f200399496 HDFS-11891. DU#refresh should print the path of the directory when an exception is caught. Contributed by Chen Liang. 2017-05-26 16:02:53 -07:00
Steve Loughran 1efba62fd7
HADOOP-14430 the accessTime of FileStatus returned by SFTPFileSystem's 
getFileStatus method is always 0.
Contributed by Hongyuan Li.
 2017-05-25 15:17:43 +01:00
Steve Loughran 07c14b35ec
HADOOP-14399. Configuration does not correctly XInclude absolute file URIs. 
Contributed by Jonathan Eagles

(cherry picked from commit 1ba9704eec)
 2017-05-25 15:03:01 +01:00
Amar Kamat 3a8f3bcccb HADOOP-7851. Configuration.getClasses() never returns the default value. (amarrk) 
(cherry picked from commit 82d57ee7fe)
 2017-05-24 17:01:49 -07:00
Brahma Reddy Battula 6f55be197c HADOOP-14166. Reset the DecayRpcScheduler AvgResponseTime metric to zero when queue is not used. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit dcf4559ebd)
 2017-05-25 01:27:52 +08:00
Brahma Reddy Battula e3a65b24bf HADOOP-14449. The ASF Header in ComparableVersion.java and SSLHostnameVerifier.java is not correct. Contributed by ZhangBing Lin. 
(cherry picked from commit b7a0c0e701)
 2017-05-25 01:04:06 +08:00
Brahma Reddy Battula d61418f34a HDFS-11864. Document Metrics to track usage of memory for writes. Contributed by Yiqun Lin. 
(cherry picked from commit 52661e0912)
 2017-05-23 23:53:27 +08:00
Sunil G ec346d6377 YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang. 2017-05-23 07:57:36 +05:30
Brahma Reddy Battula 2755255744 HDFS-11863. Document missing metrics for blocks count in pending IBR. Contributed By Yiqun Lin. 
(cherry picked from commit 2a8fcf0c9a)
 2017-05-23 02:23:54 +08:00
John Zhuge dad7d0e2ff HADOOP-13911. Remove TRUSTSTORE_PASSWORD related scripts from KMS. Contributed by John Zhuge. 
(cherry picked from commit 30f85d7a88)

Conflicts:
	hadoop-common-project/hadoop-kms/src/main/sbin/kms.sh
	hadoop-common-project/hadoop-kms/src/main/tomcat/ssl-server.xml
 2017-05-18 21:33:59 -07:00
John Zhuge 145d716a2b HADOOP-14242. Make KMS Tomcat SSL property sslEnabledProtocols and clientAuth configurable. Contributed by John Zhuge. 2017-05-18 18:03:54 -07:00
Chris Douglas 4bb0565443 HADOOP-14434. Use MoveFileEx to allow renaming a file when the destination exists. Contributed by Lukas Majercak 
(cherry picked from commit ef9e536a71)
 2017-05-17 17:24:36 -07:00
Mingliang Liu 94755e2d6d HADOOP-14427. Avoid reloading of Configuration in ViewFileSystem creation. Contributed by Vinayakumar B 
(cherry picked from commit eb7791b474)
 2017-05-17 13:21:34 -07:00
Rohith Sharma K S 379e5bd26f HADOOP-14412. HostsFileReader#getHostDetails is very expensive on large clusters. Contributed by Jason Lowe. 2017-05-17 08:27:50 +05:30
John Zhuge 2be559b842 HADOOP-14417. Update default SSL cipher list for KMS. Contributed by John Zhuge. 2017-05-15 16:24:45 -07:00
Andrew Wang 18b89e902b HDFS-11644. Support for querying outputstream capabilities. Contributed by Manoj Govindassamy. 2017-05-13 21:41:11 -07:00
Jason Lowe 955e8316d5 HADOOP-14376. Memory leak when reading a compressed file using the native library. Contributed by Eli Acherkan 
(cherry picked from commit 7bc2172248)
 2017-05-12 16:55:49 -05:00
Daniel Templeton 7213c935c6 HADOOP-14413. Add Javadoc comment for jitter parameter on CachingGetSpaceUsed 
(Contributed by Erik Krogen via Daniel Templeton)

(cherry picked from commit 09f28da2d2)
 2017-05-11 12:06:39 -07:00
Brahma Reddy Battula 3766e54893 HADOOP-14410. Correct spelling of 'beginning' and variants. Contributed By Dongtao Zhang 
(cherry picked from commit 4aae2d40a3)
 2017-05-11 23:38:18 +05:30
Mingliang Liu 214f3abe23 HDFS-11800. Document output of 'hdfs count -u' should contain PATHNAME. Contributed by Xiaobing Zhou 
(cherry picked from commit 3873904a94)
 2017-05-10 14:34:11 -07:00
Akira Ajisaka d9c3c78b78
HADOOP-14373. License error in org.apache.hadoop.metrics2.util.Servers. Contributed by hu xiaodong. 
(cherry picked from commit a5fa954684)
 2017-05-10 05:30:37 -05:00
Masatake Iwasaki 05dda3063b HADOOP-14405. Fix performance regression due to incorrect use of DataChecksum. Contributed by LiXin Ge. 
(cherry picked from commit 06ffeb8d4d)
 2017-05-10 14:19:40 +09:00
Jason Lowe f9ba4ef7d5 HADOOP-14377. Increase Common test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 
(cherry picked from commit f7faac8e90)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestGlobPattern.java
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java
 2017-05-09 16:16:49 -05:00
Lei Xu 4a95cff95d HADOOP-14384. Reduce the visibility of FileSystem.newFSDataOutputStreamBuilder before the API becomes stable. (lei) 
(cherry picked from commit 149b903be3)
 2017-05-09 10:28:21 -07:00
Haohui Mai b8870d8159 HADOOP-14383. Implement FileSystem that reads from HTTP / HTTPS endpoints. 2017-05-08 15:28:30 -07:00
Arpit Agarwal 89baea96d5 HDFS-11722. Change Datanode file IO profiling sampling to percentage. Contributed by Hanisha Koneru. 2017-05-03 16:30:01 -07:00
Arpit Agarwal 4b7d1ff326 Revert "HDFS-11488. JN log segment syncing should support HA upgrade. Contributed by Hanisha Koneru." 
This reverts commit dfc6ef42c4.
 2017-05-03 16:29:44 -07:00
Arpit Agarwal dfc6ef42c4 HDFS-11488. JN log segment syncing should support HA upgrade. Contributed by Hanisha Koneru. 2017-05-03 15:53:43 -07:00
Mingliang Liu 616413ecb2 HDFS-11739. Fix regression in tests caused by YARN-679. Contributed by Steve Loughran 
(cherry picked from commit 83dded556d)
 2017-05-03 11:23:15 -07:00
Kihwal Lee f843c8ccf5 HADOOP-14372. TestSymlinkLocalFS timeouts are too low. Contributed by Eric Badger. 
(cherry picked from commit d4631e466b)
 2017-05-03 09:49:17 -05:00
Jason Lowe 9e9c3a511c HADOOP-14306. TestLocalFileSystem tests have very low timeouts. Contributed by Eric Badger 
(cherry picked from commit cedaf4cab9)
 2017-05-02 16:52:52 -05:00
Andrew Wang 484bab3944 HADOOP-14369. NetworkTopology calls expensive toString() when logging. Contributed by Inigo Goiri. 
(cherry picked from commit dcc292d7de)
 2017-05-02 10:53:57 -07:00
Akira Ajisaka 036624900b
HADOOP-14371. License error in TestLoadBalancingKMSClientProvider.java. Contributed by hu xiaodong. 
(cherry picked from commit 20cde55274)
 2017-05-02 23:39:00 +09:00
Mingliang Liu 9b8940366e Revert "HADOOP-14363. Inconsistent default block location in FileSystem javadoc. Contributed by Chen Liang" 
This reverts commit 18a5925256.
 2017-04-29 23:51:30 -07:00
Mingliang Liu 18a5925256 HADOOP-14363. Inconsistent default block location in FileSystem javadoc. Contributed by Chen Liang 
(cherry picked from commit c735362df7)
 2017-04-29 23:50:04 -07:00
Eric Payne 2bf29ca2f9 HADOOP-14320. TestIPC.testIpcWithReaderQueuing fails intermittently. Contributed by Eric Badger. 
(cherry picked from commit 6fe11f6155)
 2017-04-28 14:49:02 -05:00
Junping Du 5d81b6b29a YARN-679. Add an entry point that can start any Yarn service. Contributed by Steve Loughran. 
(cherry picked from commit 373bb4931f)
 2017-04-28 11:10:00 -07:00
Steve Loughran b500714e98
HADOOP-14346. CryptoOutputStream throws IOException if stream is closed. 
Contributed by Pierre Lacave

(cherry picked from commit 40b9c30d64)
 2017-04-27 12:18:30 +01:00