Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,235 Commits 326 Branches 371 Tags
Commit Graph

3491 Commits

Author SHA1 Message Date
Brahma Reddy Battula
e889c826d7 HADOOP-14431. ModifyTime of FileStatus returned by SFTPFileSystem's getFileStatus method is wrong. Contributed by Hongyuan Li. 
(cherry picked from commit 66c6fd831497944f4f49c5ce42c69a302b7d7bc0)
 2017-06-06 12:32:37 +08:00
Mingliang Liu
867026d94e HADOOP-14428. s3a: mkdir appears to be broken. Contributed by Mingliang Liu 
(cherry picked from commit 6aeda55bb8f741d9dafd41f6dfbf1a88acdd4003)
 2017-06-05 11:31:13 -07:00
Brahma Reddy Battula
ca76378aad HADOOP-14440. Add metrics for connections dropped. Contributed by Eric Badger. 
(cherry picked from commit abdd609e51a80388493417126c3bc9b1badc0ac1)
 2017-06-06 00:21:59 +08:00
Brahma Reddy Battula
ac2af66e77 HDFS-11905. Fix license header inconsistency in hdfs. Contributed by Yeliang Cang. 2017-06-02 22:22:43 +08:00
Andrew Wang
d0a4e7e95d HDFS-11383. Intern strings in BlockLocation and ExtendedBlock. Contributed by Misha Dmitriev. 
(cherry picked from commit 7101477e4726a70ab0eab57c2d4480a04446a8dc)
 2017-06-01 15:20:22 -07:00
Kihwal Lee
5b0baeab5e HDFS-5042. Completed files lost after power failure. Contributed by Vinayakumar B. 2017-05-31 11:02:34 -05:00
Arpit Agarwal
f200399496 HDFS-11891. DU#refresh should print the path of the directory when an exception is caught. Contributed by Chen Liang. 2017-05-26 16:02:53 -07:00
Steve Loughran
1efba62fd7
HADOOP-14430 the accessTime of FileStatus returned by SFTPFileSystem's 
getFileStatus method is always 0.
Contributed by Hongyuan Li.
 2017-05-25 15:17:43 +01:00
Steve Loughran
07c14b35ec
HADOOP-14399. Configuration does not correctly XInclude absolute file URIs. 
Contributed by Jonathan Eagles

(cherry picked from commit 1ba9704eec22c75f8aec653ee15eb6767b5a7f4b)
 2017-05-25 15:03:01 +01:00
Amar Kamat
3a8f3bcccb HADOOP-7851. Configuration.getClasses() never returns the default value. (amarrk) 
(cherry picked from commit 82d57ee7fee82919b42066af38063a27d0998806)
 2017-05-24 17:01:49 -07:00
Brahma Reddy Battula
6f55be197c HADOOP-14166. Reset the DecayRpcScheduler AvgResponseTime metric to zero when queue is not used. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit dcf4559ebdb5bb11d03b264a9875bff316b89eef)
 2017-05-25 01:27:52 +08:00
Brahma Reddy Battula
e3a65b24bf HADOOP-14449. The ASF Header in ComparableVersion.java and SSLHostnameVerifier.java is not correct. Contributed by ZhangBing Lin. 
(cherry picked from commit b7a0c0e7019528fac492e1cb32d374ed014f8673)
 2017-05-25 01:04:06 +08:00
Brahma Reddy Battula
d61418f34a HDFS-11864. Document Metrics to track usage of memory for writes. Contributed by Yiqun Lin. 
(cherry picked from commit 52661e0912a79d1e851afc2b46c941ce952ca63f)
 2017-05-23 23:53:27 +08:00
Sunil G
ec346d6377 YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang. 2017-05-23 07:57:36 +05:30
Brahma Reddy Battula
2755255744 HDFS-11863. Document missing metrics for blocks count in pending IBR. Contributed By Yiqun Lin. 
(cherry picked from commit 2a8fcf0c9a9b5293238a8ab76c19d74a6a3bae72)
 2017-05-23 02:23:54 +08:00
Chris Douglas
4bb0565443 HADOOP-14434. Use MoveFileEx to allow renaming a file when the destination exists. Contributed by Lukas Majercak 
(cherry picked from commit ef9e536a7137d209985d25a4769712c2db2c52bf)
 2017-05-17 17:24:36 -07:00
Mingliang Liu
94755e2d6d HADOOP-14427. Avoid reloading of Configuration in ViewFileSystem creation. Contributed by Vinayakumar B 
(cherry picked from commit eb7791b4743c593ac7d0f0d08962132723ffe3c6)
 2017-05-17 13:21:34 -07:00
Rohith Sharma K S
379e5bd26f HADOOP-14412. HostsFileReader#getHostDetails is very expensive on large clusters. Contributed by Jason Lowe. 2017-05-17 08:27:50 +05:30
Andrew Wang
18b89e902b HDFS-11644. Support for querying outputstream capabilities. Contributed by Manoj Govindassamy. 2017-05-13 21:41:11 -07:00
Jason Lowe
955e8316d5 HADOOP-14376. Memory leak when reading a compressed file using the native library. Contributed by Eli Acherkan 
(cherry picked from commit 7bc217224891b7f7f0a2e35e37e46b36d8c5309d)
 2017-05-12 16:55:49 -05:00
Daniel Templeton
7213c935c6 HADOOP-14413. Add Javadoc comment for jitter parameter on CachingGetSpaceUsed 
(Contributed by Erik Krogen via Daniel Templeton)

(cherry picked from commit 09f28da2d2aae1487bae85ff110794f8346f92a4)
 2017-05-11 12:06:39 -07:00
Brahma Reddy Battula
3766e54893 HADOOP-14410. Correct spelling of 'beginning' and variants. Contributed By Dongtao Zhang 
(cherry picked from commit 4aae2d40a3e2e732e09b4b8a82623cacc0dc8861)
 2017-05-11 23:38:18 +05:30
Mingliang Liu
214f3abe23 HDFS-11800. Document output of 'hdfs count -u' should contain PATHNAME. Contributed by Xiaobing Zhou 
(cherry picked from commit 3873904a943fbbd5dcfd154e923b90c0610ed05b)
 2017-05-10 14:34:11 -07:00
Akira Ajisaka
d9c3c78b78
HADOOP-14373. License error in org.apache.hadoop.metrics2.util.Servers. Contributed by hu xiaodong. 
(cherry picked from commit a5fa954684f517bdf0a50f3fa7a0314f24cc7049)
 2017-05-10 05:30:37 -05:00
Masatake Iwasaki
05dda3063b HADOOP-14405. Fix performance regression due to incorrect use of DataChecksum. Contributed by LiXin Ge. 
(cherry picked from commit 06ffeb8d4d705d14e8755364099286ac4ed3bb83)
 2017-05-10 14:19:40 +09:00
Jason Lowe
f9ba4ef7d5 HADOOP-14377. Increase Common test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 
(cherry picked from commit f7faac8e90efaed4fbe9975c97e1b861c32cc3a3)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestGlobPattern.java
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java
 2017-05-09 16:16:49 -05:00
Lei Xu
4a95cff95d HADOOP-14384. Reduce the visibility of FileSystem.newFSDataOutputStreamBuilder before the API becomes stable. (lei) 
(cherry picked from commit 149b903be30e2b973691c30ee02f0d5b53ca0abc)
 2017-05-09 10:28:21 -07:00
Haohui Mai
b8870d8159 HADOOP-14383. Implement FileSystem that reads from HTTP / HTTPS endpoints. 2017-05-08 15:28:30 -07:00
Arpit Agarwal
89baea96d5 HDFS-11722. Change Datanode file IO profiling sampling to percentage. Contributed by Hanisha Koneru. 2017-05-03 16:30:01 -07:00
Arpit Agarwal
4b7d1ff326 Revert "HDFS-11488. JN log segment syncing should support HA upgrade. Contributed by Hanisha Koneru." 
This reverts commit dfc6ef42c4d96133c4d6245c959ee8540bd12f91.
 2017-05-03 16:29:44 -07:00
Arpit Agarwal
dfc6ef42c4 HDFS-11488. JN log segment syncing should support HA upgrade. Contributed by Hanisha Koneru. 2017-05-03 15:53:43 -07:00
Mingliang Liu
616413ecb2 HDFS-11739. Fix regression in tests caused by YARN-679. Contributed by Steve Loughran 
(cherry picked from commit 83dded556dc179fcff078451fb80533065e116f0)
 2017-05-03 11:23:15 -07:00
Kihwal Lee
f843c8ccf5 HADOOP-14372. TestSymlinkLocalFS timeouts are too low. Contributed by Eric Badger. 
(cherry picked from commit d4631e466bc85ce605061673a31f451353da9713)
 2017-05-03 09:49:17 -05:00
Jason Lowe
9e9c3a511c HADOOP-14306. TestLocalFileSystem tests have very low timeouts. Contributed by Eric Badger 
(cherry picked from commit cedaf4cab9d5013acd559dcb92d4976823ccdf85)
 2017-05-02 16:52:52 -05:00
Andrew Wang
484bab3944 HADOOP-14369. NetworkTopology calls expensive toString() when logging. Contributed by Inigo Goiri. 
(cherry picked from commit dcc292d7ded200a4976f6d348952ecba10f01db2)
 2017-05-02 10:53:57 -07:00
Akira Ajisaka
036624900b
HADOOP-14371. License error in TestLoadBalancingKMSClientProvider.java. Contributed by hu xiaodong. 
(cherry picked from commit 20cde552748cc6be1c5ca590755d4e5a67baf187)
 2017-05-02 23:39:00 +09:00
Mingliang Liu
9b8940366e Revert "HADOOP-14363. Inconsistent default block location in FileSystem javadoc. Contributed by Chen Liang" 
This reverts commit 18a5925256a4d8ff70fba483c55c55dccad4e8e0.
 2017-04-29 23:51:30 -07:00
Mingliang Liu
18a5925256 HADOOP-14363. Inconsistent default block location in FileSystem javadoc. Contributed by Chen Liang 
(cherry picked from commit c735362df719227adf9a648b2d425288cd4e4c2e)
 2017-04-29 23:50:04 -07:00
Eric Payne
2bf29ca2f9 HADOOP-14320. TestIPC.testIpcWithReaderQueuing fails intermittently. Contributed by Eric Badger. 
(cherry picked from commit 6fe11f61556407f8efb9b376623cd61609a40d5f)
 2017-04-28 14:49:02 -05:00
Junping Du
5d81b6b29a YARN-679. Add an entry point that can start any Yarn service. Contributed by Steve Loughran. 
(cherry picked from commit 373bb4931fb392e3ca6bfd78992887e5a405e186)
 2017-04-28 11:10:00 -07:00
Steve Loughran
b500714e98
HADOOP-14346. CryptoOutputStream throws IOException if stream is closed. 
Contributed by Pierre Lacave

(cherry picked from commit 40b9c30d64ee1b07ade423533e627a7d4b099261)
 2017-04-27 12:18:30 +01:00
John Zhuge
1e60cfed7e HADOOP-14341. Support multi-line value for ssl.server.exclude.cipher.list. Contributed by John Zhuge. 
(cherry picked from commit 9ccb849eb69f05bccc3435306ec4bde104e411bf)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/SSLFactory.java
 2017-04-25 22:36:21 -07:00
Xiaoyu Yao
fc0e6de929 HADOOP-14207. "dfsadmin -refreshCallQueue" fails with DecayRpcScheduler. Contributed by Surendra Singh Lihore. 
(cherry picked from commit 918dee10f56861d15bb6644edcefb4246a3e00a4)
 2017-04-25 18:49:44 -07:00
Mingliang Liu
6fd072f924 HADOOP-12856. FileUtil.checkDest() and RawLocalFileSystem.mkdirs() to throw stricter IOEs; RawLocalFS contract tests to verify. Contributed by Steve Loughran 
(cherry picked from commit d11d280b8b527d572ca0b29ef57434e9ff533ffe)
 2017-04-25 18:34:26 -07:00
Mingliang Liu
3b7bb7b94b HADOOP-14324. Refine S3 server-side-encryption key as encryption secret; improve error reporting and diagnostics. Contributed by Steve Loughran 2017-04-20 17:07:57 -07:00
Daniel Templeton
d77137a03f YARN-6453. fairscheduler-statedump.log gets generated regardless of service 
(Contributed by Yufei Gu via Daniel Templeton)

(cherry picked from commit e74ca7216aecc998d277e5ca1f4d0abae43c7e31)
 2017-04-20 14:14:38 -07:00
John Zhuge
6865746ea4 HADOOP-14241. Add ADLS sensitive config keys to default list. Contributed by John Zhuge. 2017-04-19 22:22:36 -07:00
John Zhuge
ef74a5661b Revert "HADOOP-14241. Add ADLS sensitive config keys to default list. Contributed by John Zhuge." 
This reverts commit 3a39dd5391527623a70296410f4754d8d235310a.
 2017-04-19 13:33:38 -07:00
John Zhuge
3a39dd5391 HADOOP-14241. Add ADLS sensitive config keys to default list. Contributed by John Zhuge. 
(cherry picked from commit 0344bea3fd4031622edd828a610c9fdc23c53d26)
 2017-04-19 12:04:03 -07:00
Daniel Templeton
5225ae50b7 HADOOP-13997. Typo in metrics docs (Contributed by Ana Krasteva via Daniel Templeton) 
(cherry picked from commit d4483a2fe47ccc589e71c6fe5501878e6142c26f)
 2017-04-19 09:42:30 -07:00