Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,927 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3574 Commits

Author SHA1 Message Date
Kihwal Lee 4c6184b41c Addendum for HADOOP-14146 jdk7 support. 2017-06-23 09:57:51 -05:00
Steve Loughran 5858bc70a2
HADOOP-14568. GenericTestUtils#waitFor missing parameter verification. 
Contributed by Yiqun Lin

(cherry picked from commit e5db9af8a0)
 2017-06-23 14:41:08 +01:00
Arpit Agarwal f9f6ef83eb HDFS-12010. TestCopyPreserveFlag fails consistently because of mismatch in access time. Contributed by Mukul Kumar Singh. 2017-06-22 15:43:00 -07:00
Akira Ajisaka 897d5a4d24
HADOOP-14542. Add IOUtils.cleanupWithLogger that accepts slf4j logger API. Contributed by Chen Liang. 
(cherry picked from commit b64951905e)
 2017-06-22 17:43:33 +09:00
Daryn Sharp d4115d71b5 HADOOP-14146. KerberosAuthenticationHandler should authenticate with SPN in AP-REQ. Contributed by Daryn Sharp 2017-06-21 11:05:12 -05:00
Yongjun Zhang ba487ead42 HADOOP-14429. FTPFileSystem#getFsAction always returns FsAction.NONE. (Hongyuan Li via Yongjun Zhang) 
(cherry picked from commit 5157f6c46e)
 2017-06-20 17:07:01 -07:00
Brahma Reddy Battula 933fad2e17 HADOOP-14533. Size of args cannot be less than zero in TraceAdmin#run as its linkedlist. Contributed by Weisen Han. 
(cherry picked from commit 2e9daa2e27)
 2017-06-20 20:45:33 +08:00
Xiao Chen b9bd72c676 HADOOP-14515. Specifically configure zookeeper-related log levels in KMS log4j. 
(cherry picked from commit 4cdae5d88ef7851ae284a115868dc4bbe24ae1bd)
 2017-06-19 23:12:28 -07:00
Akira Ajisaka 6d82db40d5
HADOOP-14540. Replace MRv1 specific terms in HostsFileReader. Contributed by hu xiaodong. 
(cherry picked from commit 300804595e)
 2017-06-19 19:40:19 +09:00
Akira Ajisaka 2d0b6ab65d
HADOOP-14538. Fix TestFilterFileSystem and TestHarFileSystem failures after DistributedFileSystem.append API. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 7ade5124b8)
 2017-06-19 19:10:13 +09:00
Masatake Iwasaki 023dcdc73d HADOOP-14424. Add CRC32C performance test. Contributed by LiXin Ge. 
(cherry picked from commit 75043d3429)
 2017-06-19 09:18:02 +09:00
Lei Xu 33afa1fdca HADOOP-14395. Provide Builder pattern for DistributedFileSystem.append. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 6460df21a0)

    Conflicts:
    hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
 2017-06-16 17:32:29 -07:00
Arpit Agarwal 9529513f18 HADOOP-14503. Make RollingAverages a mutable metric. Contributed by Hanisha Koneru. 2017-06-16 16:41:08 -07:00
Xiao Chen 2afe9722af HADOOP-14524. Make CryptoCodec Closeable so it can be cleaned up proactively. 2017-06-16 09:49:41 -07:00
Steve Loughran 4e6348f346
HADOOP-14486 TestSFTPFileSystem#testGetAccessTime test failure using openJDK 1.8.0. 
Contributed by Hongyuan Li.

(cherry picked from commit 942ecbbc98)
 2017-06-16 09:46:34 +01:00
Akira Ajisaka 12cb3456d7
HADOOP-14289. Move log4j APIs over to slf4j in hadoop-common. 
(cherry picked from commit e855cc4ed4)
 2017-06-16 13:32:16 +09:00
Lei Xu abac844c90 HADOOP-14394. Provide Builder pattern for DistributedFileSystem.create. (lei) 
(cherry picked from commit 5fbec46525d6d49837d934556b59ba77bd2301a8)

    Conflicts:
    hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/NameNodeConnector.java
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicies.java
 2017-06-15 10:59:11 -07:00
Xiao Chen 7d81b0beab HADOOP-14523. OpensslAesCtrCryptoCodec.finalize() holds excessive amounts of memory. Contributed by Misha Dmitriev. 
(cherry picked from commit ef8edab930)
 2017-06-15 10:41:34 -07:00
Steve Loughran 31446f8e10
HADOOP-14506. Add create() contract test that verifies ancestor dir creation. 
Contributed by Sean Mackrory.

(cherry picked from commit d780a67864)
 2017-06-15 14:51:50 +01:00
Jonathan Eagles f901132a8a HADOOP-14501. Switch from aalto-xml to woodstox to handle odd XML features (jeagles) 2017-06-12 17:19:46 -05:00
Daniel Templeton 6d7e847cfc HADOOP-14310. RolloverSignerSecretProvider.LOG should be @VisibleForTesting 
(Contributed by Arun Shanmugam Kumar via Daniel Templeton)

(cherry picked from commit 86368cc766)
 2017-06-12 09:56:35 -07:00
Mingliang Liu ae2457e1dd HADOOP-14465. LdapGroupsMapping - support user and group search base. Contributed by Shwetha G S and Mingliang Liu 
(cherry picked from commit a2121cb0d9)
 2017-06-09 16:26:10 -07:00
Xiao Chen aaf533e8a0 HADOOP-13854. KMS should log error details in KMSExceptionsProvider. 
(cherry picked from commit c4b5c32669)
 2017-06-08 22:38:31 -07:00
Xiao Chen ffaf24e308 HADOOP-13174. Add more debug logs for delegation tokens and authentication. 
(cherry picked from commit 4a56bde6ba1f72588a25cd96acc76089706cb786)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticationHandler.java
 2017-06-08 21:34:09 -07:00
John Zhuge 87f4428de0 HADOOP-14511. WritableRpcEngine.Invocation#toString NPE on null parameters. Contributed by John Zhuge. 2017-06-08 21:26:02 -07:00
John Zhuge 2448d84572 HDFS-11861. ipc.Client.Connection#sendRpcRequest should log request name. Contributed by John Zhuge. 
(cherry picked from commit 5672ae7b37)
 2017-06-07 15:54:08 -07:00
Kihwal Lee f7e597b56d HADOOP-14035. Reduce fair call queue backoff's impact on clients. Contributed by Daryn Sharp. 2017-06-06 12:19:44 -05:00
Brahma Reddy Battula e889c826d7 HADOOP-14431. ModifyTime of FileStatus returned by SFTPFileSystem's getFileStatus method is wrong. Contributed by Hongyuan Li. 
(cherry picked from commit 66c6fd8314)
 2017-06-06 12:32:37 +08:00
Mingliang Liu 867026d94e HADOOP-14428. s3a: mkdir appears to be broken. Contributed by Mingliang Liu 
(cherry picked from commit 6aeda55bb8)
 2017-06-05 11:31:13 -07:00
Brahma Reddy Battula ca76378aad HADOOP-14440. Add metrics for connections dropped. Contributed by Eric Badger. 
(cherry picked from commit abdd609e51)
 2017-06-06 00:21:59 +08:00
Brahma Reddy Battula ac2af66e77 HDFS-11905. Fix license header inconsistency in hdfs. Contributed by Yeliang Cang. 2017-06-02 22:22:43 +08:00
Andrew Wang d0a4e7e95d HDFS-11383. Intern strings in BlockLocation and ExtendedBlock. Contributed by Misha Dmitriev. 
(cherry picked from commit 7101477e47)
 2017-06-01 15:20:22 -07:00
Kihwal Lee 5b0baeab5e HDFS-5042. Completed files lost after power failure. Contributed by Vinayakumar B. 2017-05-31 11:02:34 -05:00
Arpit Agarwal f200399496 HDFS-11891. DU#refresh should print the path of the directory when an exception is caught. Contributed by Chen Liang. 2017-05-26 16:02:53 -07:00
Steve Loughran 1efba62fd7
HADOOP-14430 the accessTime of FileStatus returned by SFTPFileSystem's 
getFileStatus method is always 0.
Contributed by Hongyuan Li.
 2017-05-25 15:17:43 +01:00
Steve Loughran 07c14b35ec
HADOOP-14399. Configuration does not correctly XInclude absolute file URIs. 
Contributed by Jonathan Eagles

(cherry picked from commit 1ba9704eec)
 2017-05-25 15:03:01 +01:00
Amar Kamat 3a8f3bcccb HADOOP-7851. Configuration.getClasses() never returns the default value. (amarrk) 
(cherry picked from commit 82d57ee7fe)
 2017-05-24 17:01:49 -07:00
Brahma Reddy Battula 6f55be197c HADOOP-14166. Reset the DecayRpcScheduler AvgResponseTime metric to zero when queue is not used. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit dcf4559ebd)
 2017-05-25 01:27:52 +08:00
Brahma Reddy Battula e3a65b24bf HADOOP-14449. The ASF Header in ComparableVersion.java and SSLHostnameVerifier.java is not correct. Contributed by ZhangBing Lin. 
(cherry picked from commit b7a0c0e701)
 2017-05-25 01:04:06 +08:00
Brahma Reddy Battula d61418f34a HDFS-11864. Document Metrics to track usage of memory for writes. Contributed by Yiqun Lin. 
(cherry picked from commit 52661e0912)
 2017-05-23 23:53:27 +08:00
Sunil G ec346d6377 YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang. 2017-05-23 07:57:36 +05:30
Brahma Reddy Battula 2755255744 HDFS-11863. Document missing metrics for blocks count in pending IBR. Contributed By Yiqun Lin. 
(cherry picked from commit 2a8fcf0c9a)
 2017-05-23 02:23:54 +08:00
John Zhuge dad7d0e2ff HADOOP-13911. Remove TRUSTSTORE_PASSWORD related scripts from KMS. Contributed by John Zhuge. 
(cherry picked from commit 30f85d7a88)

Conflicts:
	hadoop-common-project/hadoop-kms/src/main/sbin/kms.sh
	hadoop-common-project/hadoop-kms/src/main/tomcat/ssl-server.xml
 2017-05-18 21:33:59 -07:00
John Zhuge 145d716a2b HADOOP-14242. Make KMS Tomcat SSL property sslEnabledProtocols and clientAuth configurable. Contributed by John Zhuge. 2017-05-18 18:03:54 -07:00
Chris Douglas 4bb0565443 HADOOP-14434. Use MoveFileEx to allow renaming a file when the destination exists. Contributed by Lukas Majercak 
(cherry picked from commit ef9e536a71)
 2017-05-17 17:24:36 -07:00
Mingliang Liu 94755e2d6d HADOOP-14427. Avoid reloading of Configuration in ViewFileSystem creation. Contributed by Vinayakumar B 
(cherry picked from commit eb7791b474)
 2017-05-17 13:21:34 -07:00
Rohith Sharma K S 379e5bd26f HADOOP-14412. HostsFileReader#getHostDetails is very expensive on large clusters. Contributed by Jason Lowe. 2017-05-17 08:27:50 +05:30
John Zhuge 2be559b842 HADOOP-14417. Update default SSL cipher list for KMS. Contributed by John Zhuge. 2017-05-15 16:24:45 -07:00
Andrew Wang 18b89e902b HDFS-11644. Support for querying outputstream capabilities. Contributed by Manoj Govindassamy. 2017-05-13 21:41:11 -07:00
Jason Lowe 955e8316d5 HADOOP-14376. Memory leak when reading a compressed file using the native library. Contributed by Eli Acherkan 
(cherry picked from commit 7bc2172248)
 2017-05-12 16:55:49 -05:00
Daniel Templeton 7213c935c6 HADOOP-14413. Add Javadoc comment for jitter parameter on CachingGetSpaceUsed 
(Contributed by Erik Krogen via Daniel Templeton)

(cherry picked from commit 09f28da2d2)
 2017-05-11 12:06:39 -07:00
Brahma Reddy Battula 3766e54893 HADOOP-14410. Correct spelling of 'beginning' and variants. Contributed By Dongtao Zhang 
(cherry picked from commit 4aae2d40a3)
 2017-05-11 23:38:18 +05:30
Mingliang Liu 214f3abe23 HDFS-11800. Document output of 'hdfs count -u' should contain PATHNAME. Contributed by Xiaobing Zhou 
(cherry picked from commit 3873904a94)
 2017-05-10 14:34:11 -07:00
Akira Ajisaka d9c3c78b78
HADOOP-14373. License error in org.apache.hadoop.metrics2.util.Servers. Contributed by hu xiaodong. 
(cherry picked from commit a5fa954684)
 2017-05-10 05:30:37 -05:00
Masatake Iwasaki 05dda3063b HADOOP-14405. Fix performance regression due to incorrect use of DataChecksum. Contributed by LiXin Ge. 
(cherry picked from commit 06ffeb8d4d)
 2017-05-10 14:19:40 +09:00
Jason Lowe f9ba4ef7d5 HADOOP-14377. Increase Common test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 
(cherry picked from commit f7faac8e90)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestGlobPattern.java
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java
 2017-05-09 16:16:49 -05:00
Lei Xu 4a95cff95d HADOOP-14384. Reduce the visibility of FileSystem.newFSDataOutputStreamBuilder before the API becomes stable. (lei) 
(cherry picked from commit 149b903be3)
 2017-05-09 10:28:21 -07:00
Haohui Mai b8870d8159 HADOOP-14383. Implement FileSystem that reads from HTTP / HTTPS endpoints. 2017-05-08 15:28:30 -07:00
Arpit Agarwal 89baea96d5 HDFS-11722. Change Datanode file IO profiling sampling to percentage. Contributed by Hanisha Koneru. 2017-05-03 16:30:01 -07:00
Arpit Agarwal 4b7d1ff326 Revert "HDFS-11488. JN log segment syncing should support HA upgrade. Contributed by Hanisha Koneru." 
This reverts commit dfc6ef42c4.
 2017-05-03 16:29:44 -07:00
Arpit Agarwal dfc6ef42c4 HDFS-11488. JN log segment syncing should support HA upgrade. Contributed by Hanisha Koneru. 2017-05-03 15:53:43 -07:00
Mingliang Liu 616413ecb2 HDFS-11739. Fix regression in tests caused by YARN-679. Contributed by Steve Loughran 
(cherry picked from commit 83dded556d)
 2017-05-03 11:23:15 -07:00
Kihwal Lee f843c8ccf5 HADOOP-14372. TestSymlinkLocalFS timeouts are too low. Contributed by Eric Badger. 
(cherry picked from commit d4631e466b)
 2017-05-03 09:49:17 -05:00
Jason Lowe 9e9c3a511c HADOOP-14306. TestLocalFileSystem tests have very low timeouts. Contributed by Eric Badger 
(cherry picked from commit cedaf4cab9)
 2017-05-02 16:52:52 -05:00
Andrew Wang 484bab3944 HADOOP-14369. NetworkTopology calls expensive toString() when logging. Contributed by Inigo Goiri. 
(cherry picked from commit dcc292d7de)
 2017-05-02 10:53:57 -07:00
Akira Ajisaka 036624900b
HADOOP-14371. License error in TestLoadBalancingKMSClientProvider.java. Contributed by hu xiaodong. 
(cherry picked from commit 20cde55274)
 2017-05-02 23:39:00 +09:00
Mingliang Liu 9b8940366e Revert "HADOOP-14363. Inconsistent default block location in FileSystem javadoc. Contributed by Chen Liang" 
This reverts commit 18a5925256.
 2017-04-29 23:51:30 -07:00
Mingliang Liu 18a5925256 HADOOP-14363. Inconsistent default block location in FileSystem javadoc. Contributed by Chen Liang 
(cherry picked from commit c735362df7)
 2017-04-29 23:50:04 -07:00
Eric Payne 2bf29ca2f9 HADOOP-14320. TestIPC.testIpcWithReaderQueuing fails intermittently. Contributed by Eric Badger. 
(cherry picked from commit 6fe11f6155)
 2017-04-28 14:49:02 -05:00
Junping Du 5d81b6b29a YARN-679. Add an entry point that can start any Yarn service. Contributed by Steve Loughran. 
(cherry picked from commit 373bb4931f)
 2017-04-28 11:10:00 -07:00
Steve Loughran b500714e98
HADOOP-14346. CryptoOutputStream throws IOException if stream is closed. 
Contributed by Pierre Lacave

(cherry picked from commit 40b9c30d64)
 2017-04-27 12:18:30 +01:00
John Zhuge 1e60cfed7e HADOOP-14341. Support multi-line value for ssl.server.exclude.cipher.list. Contributed by John Zhuge. 
(cherry picked from commit 9ccb849eb6)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/SSLFactory.java
 2017-04-25 22:36:21 -07:00
Xiaoyu Yao fc0e6de929 HADOOP-14207. "dfsadmin -refreshCallQueue" fails with DecayRpcScheduler. Contributed by Surendra Singh Lihore. 
(cherry picked from commit 918dee10f56861d15bb6644edcefb4246a3e00a4)
 2017-04-25 18:49:44 -07:00
Mingliang Liu 6fd072f924 HADOOP-12856. FileUtil.checkDest() and RawLocalFileSystem.mkdirs() to throw stricter IOEs; RawLocalFS contract tests to verify. Contributed by Steve Loughran 
(cherry picked from commit d11d280b8b)
 2017-04-25 18:34:26 -07:00
Mingliang Liu 3b7bb7b94b HADOOP-14324. Refine S3 server-side-encryption key as encryption secret; improve error reporting and diagnostics. Contributed by Steve Loughran 2017-04-20 17:07:57 -07:00
Daniel Templeton d77137a03f YARN-6453. fairscheduler-statedump.log gets generated regardless of service 
(Contributed by Yufei Gu via Daniel Templeton)

(cherry picked from commit e74ca7216aecc998d277e5ca1f4d0abae43c7e31)
 2017-04-20 14:14:38 -07:00
John Zhuge 1bbba00a20 HADOOP-14141. Store KMS SSL keystore password in catalina.properties. Contributed by John Zhuge. 
Change-Id: I8326b16aab5c3908529e1a0cc7bfe121d18f42f3
 2017-04-20 12:36:40 -07:00
John Zhuge 6865746ea4 HADOOP-14241. Add ADLS sensitive config keys to default list. Contributed by John Zhuge. 2017-04-19 22:22:36 -07:00
John Zhuge ef74a5661b Revert "HADOOP-14241. Add ADLS sensitive config keys to default list. Contributed by John Zhuge." 
This reverts commit 3a39dd5391.
 2017-04-19 13:33:38 -07:00
John Zhuge 3a39dd5391 HADOOP-14241. Add ADLS sensitive config keys to default list. Contributed by John Zhuge. 
(cherry picked from commit 0344bea3fd)
 2017-04-19 12:04:03 -07:00
Daniel Templeton 5225ae50b7 HADOOP-13997. Typo in metrics docs (Contributed by Ana Krasteva via Daniel Templeton) 
(cherry picked from commit d4483a2fe4)
 2017-04-19 09:42:30 -07:00
Kengo Seki 6cfceee865
HADOOP-14315. Python example in the rack awareness document doesn't work due to bad indentation 
This closes #214

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 654372db85)
 2017-04-18 17:15:35 +09:00
Akira Ajisaka a17bd70892
HADOOP-14318. Remove non-existent setfattr command option from FileSystemShell.md. Contributed by Doris Gu. 
(cherry picked from commit 84a8848aae)
 2017-04-18 16:45:14 +09:00
Mingliang Liu 508389ee54 HADOOP-14255. S3A to delete unnecessary fake directory objects in mkdirs(). Contributed by Mingliang Liu 
(cherry picked from commit 507845074970b90330db86bc1ff0124def550bb5)
 2017-04-12 14:30:54 -07:00
Daniel Templeton f20aa38a1d HADOOP-14246. Authentication Tokens should use SecureRandom instead of Random and 256 bit secrets 
(Conttributed by Robert Konter via Daniel Templeton)

(cherry picked from commit 4dd6206547)
 2017-04-12 11:19:53 -07:00
Chris Nauroth d5e2bd4096 HDFS-11163. Mover should move the file blocks to default storage once policy is unset. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 00ed21a6fedb45a7c8992b8d45adaa83f14af34c)
 2017-04-11 21:18:47 -07:00
Steve Loughran 5efb3ade1f HADOOP-14066 VersionInfo should be marked as public API 
(cherry picked from commit 2ed65540a3)
 2017-04-11 15:06:40 -07:00
Jason Lowe 0cb22a0f15 YARN-6195. Export UsedCapacity and AbsoluteUsedCapacity to JMX. Contributed by Benson Qiu 
(cherry picked from commit 0e065f2ede)
 2017-04-11 08:45:16 -05:00
Andrew Wang dab1deb9a0 HADOOP-14293. Initialize FakeTimer with a less trivial value. 
(cherry picked from commit be144117a8)
 2017-04-10 11:37:31 -07:00
John Zhuge ff2870310e HADOOP-14174. Set default ADLS access token provider type to ClientCredential. Contributed by John Zhuge. 
Signed-off-by: John Zhuge <jzhuge@apache.org>
(cherry picked from commit 56e81f2a20)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestCommonConfigurationFields.java
 2017-04-07 15:01:29 -07:00
Zhe Zhang c85026038a HADOOP-14276. Add a nanosecond API to Time/Timer/FakeTimer. Contributed by Erik Krogen. 
(cherry picked from commit 95b7f1d29a)
 2017-04-07 10:39:40 -07:00
Andrew Wang ed8f738be5 HADOOP-14104. Client should always ask namenode for kms provider path. Contributed by Rushabh S Shah. 2017-04-06 14:38:22 -07:00
Xiaoyu Yao 9016614eb1 HDFS-11302. Improve Logging for SSLHostnameVerifier. Contributed by Chen Liang. 
(cherry picked from commit 32bb36b750)
 2017-04-05 18:30:33 -07:00
Jonathan Eagles 60e65a3e55 HADOOP-14216. Addendum to Improve Configuration XML Parsing Performance (jeagles) 2017-03-30 14:15:37 -05:00
Mingliang Liu 3d5e9e61d5 HADOOP-14247. FileContextMainOperationsBaseTest should clean up test root path. Contributed by Mingliang Liu 
(cherry picked from commit 13c766b62c)
 2017-03-29 11:19:28 -07:00
Jonathan Eagles 719ae5f2e8 HADOOP-14216. Improve Configuration XML Parsing Performance (jeagles) 2017-03-29 10:14:49 -05:00
John Zhuge ef43a62482 HADOOP-14038. Rename ADLS credential properties. Contributed by John Zhuge. 
(cherry picked from commit 253e3e78ab)
 2017-03-27 23:07:08 -07:00
Akira Ajisaka 643298b282
HADOOP-14218. Replace assertThat with assertTrue in MetricsAsserts. 
(cherry picked from commit 448ec81fd7)
 2017-03-28 14:33:36 +09:00
Andrew Wang 10deafee82 HDFS-11170. Add builder-based create API to FileSystem. Contributed by SammiChen and Wei Zhou. 2017-03-27 14:35:03 -07:00
Jonathan Eagles e41aac0cd4 Configuration#get return value optimization (jeagles) 2017-03-27 12:52:59 -05:00
Jonathan Eagles 675bc97f18 Delay construction of PreCondition.check failure message in Configuration#set (jeagles) 
(cherry picked from commit 858d597be0)
 2017-03-27 11:02:41 -05:00
Junping Du 3ad4d37363 Add COMMON/MAPREDUCE/YARN jdiff of 2.8.0 release. 
(cherry picked from commit 6df029db36)
 2017-03-25 14:33:20 -07:00
Junping Du e796105a6f Add 2.8.0 RELEASENOTEs.md and CHANGES.md. 
(cherry picked from commit d8e6d6df4c)
 2017-03-25 14:32:47 -07:00
Daniel Templeton bbd08bb7a4 YARN-6360. Prevent FS state dump logger from cramming other log files 
(Contributed by Yufei Gu via Daniel Templeton)

(cherry picked from commit d49d1862a6)
 2017-03-24 11:31:58 -07:00
Andrew Wang 96fe940e59 HADOOP-14211. FilterFs and ChRootedFs are too aggressive about enforcing 'authorityNeeded'. Contributed by Erik Krogen. 
(cherry picked from commit 0e556a5ba6)
 2017-03-24 11:12:31 -07:00
John Zhuge d946dfcbe1 HADOOP-14195. CredentialProviderFactory$getProviders is not thread-safe. Contributed by Vihang Karajgaonkar. 
(cherry picked from commit 128015584d)
 2017-03-23 13:57:41 -07:00
Mingliang Liu 1e64c582f5 HADOOP-14214. DomainSocketWatcher::add()/delete() should not self interrupt while looping await(). Contributed by Mingliang Liu 
(cherry picked from commit d35e79abc2)
 2017-03-23 13:34:53 -07:00
Ravi Prakash 87339d7691 HADOOP-14213. Move Configuration runtime check for hadoop-site.xml to initialization. Contributed by Jonathan Eagles 
(cherry picked from commit 595f62e362)
 2017-03-23 09:30:22 -07:00
Zhe Zhang f98f4bb6ca HADOOP-9631. ViewFs should use underlying FileSystem's server side defaults. Contributed by Lohit Vijayarenu and Erik Krogen. 
(cherry picked from commit 59d69257a8)
 2017-03-22 22:32:07 -07:00
John Zhuge 84e2c11f96 HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge. 2017-03-21 08:23:30 -07:00
John Zhuge 2c5de0c60e HDFS-11132. Allow AccessControlException in contract tests when getFileStatus on subdirectory of existing files. Contributed by Vishwajeet Dusane 
(cherry picked from commit 19f373a46b)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java
 2017-03-20 21:16:50 -07:00
Erik Krogen e44569b736 HADOOP-14169. Implement listStatusIterator, listLocatedStatus for ViewFs. Contributed by Erik Krogen. 
(cherry picked from commit e1a99802fc)
 2017-03-17 17:40:14 -07:00
Andrew Wang 7cc88eca43 HDFS-11516. Admin command line should print message to stderr in failure case. Contributed by Kai Sasaki. 
(cherry picked from commit 92ea6d74ec)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/cli/util/ErasureCodingCliCmdExecutor.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testErasureCodingConf.xml
 2017-03-17 11:34:43 -07:00
Jitendra Pandey d564d1f44d HDFS-11533. reuseAddress option should be used for child channels in Portmap and SimpleTcpServer. Contributed by Mukul Kumar Singh. 2017-03-16 12:37:22 -07:00
Mingliang Liu ed0d426a88 HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu 2017-03-14 15:00:59 -07:00
Ray Chiang fa59f4e490 YARN-6042. Dump scheduler and queue state information into FairScheduler DEBUG log. (Yufei Gu via rchiang) 2017-03-14 14:45:13 -07:00
Jing Zhao e03d8ff488 HDFS-11395. RequestHedgingProxyProvider#RequestHedgingInvocationHandler hides the Exception thrown from NameNode. Contributed by Nandakumar. 
(cherry picked from commit 55796a0946)
 2017-03-13 14:26:44 -07:00
Jian He 8c44a9db21 HADOOP-14062. ApplicationMasterProtocolPBClientImpl.allocate fails with EOFException when RPC privacy is enabled. Contributed by Steven Rand 2017-03-10 09:26:19 -08:00
Mingliang Liu 9ea6b24c49 HADOOP-13946. Document how HDFS updates timestamps in the FS spec; compare with object stores. Contributed by Steve Loughran 
(cherry picked from commit fd26783aaf)
 2017-03-10 00:21:58 -08:00
Lei Xu 01624f0c1b HADOOP-13956. Read ADLS credentials from Credential Provider. (John Zhuge via lei) 
(cherry picked from commit e015b56319)
 2017-03-09 15:16:28 -08:00
Arpit Agarwal 5cdcda34a2 HDFS-11477. Simplify file IO profiling configuration. Contributed by Hanisha Koneru. 2017-03-09 14:34:10 -08:00
Jian He ea8d53de6a Revert "HADOOP-14062. ApplicationMasterProtocolPBClientImpl.allocate fails with EOFException when RPC privacy is enabled. Contributed by Steven Rand" 
This reverts commit 36eda5af75.
 2017-03-08 13:20:45 -08:00
John Zhuge f1fd20be82 Revert "HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran" 
This reverts commit 53a12fa721.

(cherry picked from commit 98142d2f72)

Change-Id: Ia5fa901fa97391cd46d41f59cd05f7a19c7c7532
 2017-03-08 11:46:27 -08:00
Jian He 36eda5af75 HADOOP-14062. ApplicationMasterProtocolPBClientImpl.allocate fails with EOFException when RPC privacy is enabled. Contributed by Steven Rand 2017-03-08 10:49:07 -08:00
Arpit Agarwal 8be80ccb72 HDFS-11508. Fix bind failure in SimpleTCPServer & Portmap where bind fails because socket is in TIME_WAIT state. Contributed by Mukul Kumar Singh. 2017-03-07 11:41:12 -08:00
Mingliang Liu ff050866bf HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth 
(cherry picked from commit 686823529b)
 2017-03-06 17:35:06 -08:00
Mingliang Liu 8323e651a1 Revert "HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth" 
This reverts commit b26870c58e.
 2017-03-06 17:18:36 -08:00
Ray Chiang d943480147 YARN-5665. Enhance documentation for yarn.resourcemanager.scheduler.class property. (Yufei Gu via rchiang) 
(cherry picked from commit d9dc444dc7)
 2017-03-06 14:05:09 -08:00
Andrew Wang cf5a1390aa HDFS-11441. Add escaping to error message in KMS web UI. Contributed by Aaron T. Myers. 
(cherry picked from commit ec839b94c0)
 2017-03-06 10:47:28 -08:00
Akira Ajisaka ed55bcee1f
HADOOP-14108. CLI MiniCluster: add an option to specify NameNode HTTP port. Contributed by Takanobu Asanuma. 
(cherry picked from commit 209ecd1a5c)
 2017-03-07 01:36:10 +09:00
Mingliang Liu b26870c58e HADOOP-13930. Azure: Add Authorization support to WASB. Contributed by Sivaguru Sankaridurg and Dushyanth 2017-03-03 17:09:05 -08:00
Jason Lowe 9b949f49e3 YARN-6274. Documentation refers to incorrect nodemanager health checker interval property. Contributed by WeiWei Yang 
(cherry picked from commit 05237636d3)
 2017-03-03 08:01:40 -06:00
Harsh J d4c13df515 HADOOP-6801. io.sort.mb and io.sort.factor were renamed and moved to mapreduce but are still in CommonConfigurationKeysPublic.java and used in SequenceFile.java. 
This closes #146

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit eb5a17954a)
 2017-03-03 18:39:47 +09:00
Andrew Wang d737a26c5a Revert "HDFS-11477. Simplify file IO profiling configuration. Contributed by Hanisha Koneru." 
This reverts commit 5e760f8dc4.
 2017-03-02 16:21:32 -08:00
Arpit Agarwal 5e760f8dc4 HDFS-11477. Simplify file IO profiling configuration. Contributed by Hanisha Koneru. 2017-03-02 15:38:32 -08:00
John Zhuge 133857ba04 HADOOP-14131. kms.sh creates bogus dir for tomcat logs. Contributed by John Zhuge. 
Change-Id: I444e10318eb64dcd4c0416b985d12049c72e8943
 2017-03-01 16:31:34 -08:00
Arpit Agarwal ff08f54f41 HDFS-11479. Socket re-use address option should be used in SimpleUdpServer. Contributed by Mukul Kumar Singh. 2017-03-01 12:59:09 -08:00
Brahma Reddy Battula a545ee4d7f HDFS-11432. Federation : Support fully qualified path for Quota/Snapshot/cacheadmin/cryptoadmin commands. Contributed by Brahma Reddy Battula. 
(cherry picked from commit dcd03df9f9)
 2017-03-01 10:47:47 +05:30
Lei Xu 76bf47cb91 Add missing file hadoop-common-project/hadoop-kms/src/main/tomcat/catalina-default.properties for HADOOP-14083. (lei) 2017-02-27 20:49:35 -08:00
Lei Xu d440ad46b5 HADOOP-14083. KMS should support old SSL clients. (John Zhuge via Lei Xu) 2017-02-27 18:12:00 -08:00
Yiqun Lin 6cd39803ba HADOOP-14119. Remove unused imports from GzipCodec.java. Contributed by Yiqun Lin. 
(cherry picked from commit 9db2e0c8d4)
 2017-02-27 18:41:47 +08:00
Xuan 5c509f5f0c HADOOP-14116:FailoverOnNetworkExceptionRetry does not wait when failover 
on certain exception. Contributed by Jian He

(cherry picked from commit 289bc50e66)
 2017-02-24 11:43:22 -08:00
Harsh J c097d0343c HADOOP-13817. Add a finite shell command timeout to ShellBasedUnixGroupsMapping. (harsh) 
(cherry picked from commit e8694deb6a)
 2017-02-24 21:34:31 +05:30
Akira Ajisaka b5a8c49cf3 HADOOP-14097. Remove Java6 specific code from GzipCodec.java. Contributed by Elek, Marton. 
(cherry picked from commit 50decd3613)
 2017-02-25 00:29:40 +09:00
Wei-Chiu Chuang 12ad077aa6 HADOOP-13233. help of stat is confusing. Contributed by Attila Bukor. 
(cherry picked from commit cc45da79fd)
 2017-02-13 10:17:41 -08:00
Lei Xu 6d62d0ea87 HADOOP-13204. Add support for SSE-KMS and SSE-C in s3a filesystem. (Steve Moist via lei) 2017-02-11 15:02:19 -08:00
Arpit Agarwal 4c5655ef1a HDFS-11403. Addendum fix for branch-2. 2017-02-11 01:25:43 -08:00
Arpit Agarwal 32a0c2b5e7 HDFS-11403. Zookeper ACLs on NN HA enabled clusters should be handled consistently. Contributed by Hanisha Koneru. 2017-02-11 01:18:07 -08:00
Kihwal Lee 5de6f1b791 HADOOP-14033. Reduce fair call queue lock contention. Contributed by Daryn Sharp. 
(cherry picked from commit 0c01cf5798)
 2017-02-09 16:20:06 -06:00
Kihwal Lee d008b55153 HADOOP-14034. Allow ipc layer exceptions to selectively close connections. Contributed by Daryn Sharp. 2017-02-09 10:55:40 -06:00