Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,920 Commits 326 Branches 371 Tags
Commit Graph

3093 Commits

Author SHA1 Message Date
Xiaoyu Yao
691a32c6f6 HADOOP-13890. Maintain HTTP/host as SPNEGO SPN support and fix KerberosName parsing. Contributed by Xiaoyu Yao. 
(cherry picked from commit f5e0bd30fde654ed48fe73e5c0523030365385a4)
(cherry picked from commit 85083567be22eba76dbf1f8c18149a0493f07526)
 2016-12-14 14:01:32 -08:00
Akira Ajisaka
901bc5eef5 HDFS-11204. Document the missing options of hdfs zkfc command. Contributed by Yiqun Lin. 
(cherry picked from commit 72bff192cd37ff97442e0f8dd477fbc2e58fc12d)
(cherry picked from commit 190b2dca6204e659ed8561c682c4a03f30258298)
 2016-12-14 18:54:39 +09:00
Wei-Chiu Chuang
983d487e98 HADOOP-13824. FsShell can suppress the real error if no error message is present. Contributed by John Zhuge. 
(cherry picked from commit b606e025f10daed18b90b45ac00cd0c82e818581)

Conflicts:
   hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShell.java

(cherry picked from commit 6c93c1870c08be16876ee379cba6d59775ab6a14)
 2016-12-09 15:40:14 -08:00
Brahma Reddy Battula
3957161a97 HADOOP-13883. Add description of -fs option in generic command usage. Contributed By Yiqun Lin. 
(cherry picked from commit 80b8023276126353229ab8d6427c1761b2e4bb38)
 2016-12-09 18:44:27 +05:30
Andrew Wang
092d0602c2 HADOOP-13861. Spelling errors in logging and exceptions for code. Contributed by Grant Sohn. 
(cherry picked from commit 7b988e88992528a0cac2ca8893652c5d4a90c6b9)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/util/GF256.java
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/KDiag.java

(cherry picked from commit 6bd600eef2f4a2ab1515947bd4e909c9954d6acf)
 2016-12-05 23:21:17 -08:00
Xiao Chen
49f9e7cf71 HADOOP-13847. KMSWebApp should close KeyProviderCryptoExtension. Contributed by John Zhuge. 
(cherry picked from commit 291df5c7fb713d5442ee29eb3f272127afb05a3c)
 2016-12-05 09:35:54 -08:00
Robert Kanter
92ee5e2077 HADOOP-13838. KMSTokenRenewer should close providers (xiaochen via rkanter) 
(cherry picked from commit 47ca9e26fba4a639e43bee5bfc001ffc4b42330d)
(cherry picked from commit 285edf691db42a3e17138d960f8560c62d8d0dde)
 2016-11-28 18:23:23 -08:00
Brahma Reddy Battula
713d800e7b HADOOP-13820. Replace ugi.getUsername() with ugi.getShortUserName() in viewFS. Contributed By Brahma Reddy Battula. 
(cherry picked from commit ee3d437a33c2106ea416078e722ff6184106a0bc)
(cherry picked from commit 1c0fa25dc1c4bc787fe4d2809f162bea475a8b76)
 2016-11-25 20:20:37 +05:30
Steve Loughran
24715cefe8 HADOOP-10776 Open up already widely-used APIs for delegation-token fetching & renewal to ecosystem projects. Contributed by Vinod Kumar Vavilapalli 2016-11-24 17:51:49 +00:00
Brahma Reddy Battula
1e25fd6b3b HADOOP-11603. Metric Snapshot log can be changed #MetricsSystemImpl.java since all the services will be initialized. Contributed By Brahma Reddy Battula. 
(cherry picked from commit 13501053dd95f41ac14091f5bbd79bcb2a3896a3)
 2016-11-24 22:25:36 +05:30
Mingliang Liu
834e2ddbab HADOOP-13166. Add getFileStatus("/") test to AbstractContractGetFileStatusTest. Contributed by Steve Loughran 
(cherry picked from commit 2bf9a15e8a1ac92923b45130c3369720d4a76fe1)
 2016-11-22 14:35:32 -08:00
Mingliang Liu
ed9dcdae57 HADOOP-13655. document object store use with fs shell and distcp. Contributed by Steve Loughran 2016-11-22 13:24:42 -08:00
Brahma Reddy Battula
9be7b353dd HADOOP-13646. Remove outdated overview.html. Contributed By Brahma Reddy Battula. 
(cherry picked from commit afcf8d38e750f935c06629e641a1321b79c4cace)
 2016-11-22 20:36:32 +05:30
Mingliang Liu
416f6f3605 HADOOP-11601. Enhance FS spec & tests to mandate FileStatus.getBlocksize() >0 for non-empty files. Contributed by Steve Loughran 
(cherry picked from commit ae8849fe378e11b9db642ef7784c8e6a08199b96)
 2016-11-17 14:27:34 -08:00
Kihwal Lee
0fb1306f93 HADOOP-13742. Expose NumOpenConnectionsPerUser as a metric. Brahma Reddy Battula. 
(cherry picked from commit bd3735554fa5c3bc064c57ec78f4308430b14b48)
 2016-11-17 12:20:01 -06:00
Mingliang Liu
c60ed3a3e4 HADOOP-13822. Use GlobalStorageStatistics.INSTANCE.reset() at FileSystem#clearStatistics(). Contribute by Brahma Reddy Battula 
(cherry picked from commit aab9737a058c0bdeeb17c173e530b7e571315a56)
 2016-11-16 20:11:10 -08:00
Mingliang Liu
0feb9bb4a9 HADOOP-13810. Add a test to verify that Configuration handles &-encoded characters. Contributed by Steve Loughran 
This closes #158

(cherry picked from commit 04014c4c739bb4e3bc3fdf9299abc0f47521e8fd)
 2016-11-14 19:55:45 -08:00
Zhe Zhang
c6e7b631bd HDFS-10872. Add MutableRate metrics for FSNamesystemLock operations. Contributed by Erik Krogen. 
(cherry picked from commit 818915aabb22653e212f4646b1b3cd0673330328)
 2016-11-14 16:39:26 -08:00
Yongjun Zhang
51f5bab695 HADOOP-13720. Add more info to the msgs printed in AbstractDelegationTokenSecretManager. Contributed by Yongjun Zhang. 
(cherry picked from commit fd2f22adec5c2f21f792045dbfde9385c21403ec)
 2016-11-10 23:41:48 -08:00
Xiao Chen
b1b609c003 HADOOP-13590. Retry until TGT expires even if the UGI renewal thread encountered exception. 
(cherry picked from commit 367c3d41217728c2e61252c5a5235e5bc1f9822f)
(cherry picked from commit ede909144da6fe8bde2f6561fa44a458674f96e2)
 2016-11-09 09:10:06 -08:00
Zhe Zhang
d93dc01e3d HADOOP-13782. Make MutableRates metrics thread-local write, aggregate-on-read. Contributed by Erik Krogen. 
(cherry picked from commit 77c13c385774c51766fe505397fa916754ac08d4)
(cherry picked from commit 758638d08d3337159dd3e9fa651093d0430b399a)
 2016-11-08 16:10:53 -08:00
Zhe Zhang
384b7b71a3 HADOOP-13804. MutableStat mean loses accuracy if add(long, long) is used. Contributed by Erik Krogen. 
(cherry picked from commit 3dbad5d823b8bf61b643dd1057165044138b99e0)
(cherry picked from commit b245e9ce2f20bb84690bffe902a60d5e96130cdb)
 2016-11-07 16:15:20 -08:00
Steve Loughran
5231c527aa HADOOP-13797 Remove hardcoded absolute path for ls. Contributed by Christine Koppelt 2016-11-07 12:36:38 +00:00
Robert Kanter
f9a7b83a53 HADOOP-7930. Kerberos relogin interval in UserGroupInformation should be configurable (xiaochen via rkanter) 
(cherry picked from commit 0b36dcda7cbac51c725224a8e0abef4eaa004fd8)
 2016-11-04 09:24:34 -07:00
Xiao Chen
a22ca0aa58 HADOOP-12453. Support decoding KMS Delegation Token with its own Identifier. Contributed by Xiaoyu Yao. 
(cherry picked from commit 7154a20bcb1559c23aeb3b78b920bed03d834cb5)
 2016-11-03 13:10:41 -07:00
Andrew Wang
e393dd788e HDFS-11080. Update HttpFS to use ConfigRedactor. Contributed by Sean Mackrory. 2016-11-02 19:11:25 -07:00
Ravi Prakash
8504e20b2e HADOOP-13773. Wrong HADOOP_CLIENT_OPTS in hadoop-env on branch-2. Contributed by Fei Hui 
(cherry picked from commit 045501a538800a48b77e688e46c55dce54473041)
 2016-11-01 10:49:45 -07:00
Steve Loughran
a83f10c90d HADOOP-13680. fs.s3a.readahead.range to use getLongBytes. Contributed by Abhishek Modi. 2016-10-31 20:53:22 +00:00
Wei-Chiu Chuang
9b6d27793d HDFS-11061. Update dfs -count -t command line help and documentation. Contributed by Yiqun Lin. 
(cherry picked from commit 112f04eb1b771e410ffd232a46e0ee2bc7b99d06)
(cherry picked from commit 9ba283c27d73962a894b33a89f17d3ec697bb388)
 2016-10-28 15:37:53 -07:00
Jason Lowe
558e53b10b HADOOP-13770. Shell.checkIsBashSupported swallowed an interrupted exception. Contributed by Wei-Chiu Chuang 
(cherry picked from commit c017171da00a6cd71a2901c84a0298ce14a49e23)
 2016-10-28 15:06:57 +00:00
Brahma Reddy Battula
26cf988c3a HADOOP-13201. Print the directory paths when ViewFs denies the rename operation on internal dirs. Contributed by Tianyin Xiu 
(cherry picked from commit 0c837db8a874079dd5db83a7eef9c4d2b9d0e9ff)
 2016-10-27 19:51:05 +05:30
Masatake Iwasaki
6d9b6fac86 HADOOP-13017. Implementations of InputStream.read(buffer, offset, bytes) to exit 0 if bytes==0. Contributed by Steve Loughran. 
(cherry picked from commit 0bdd263d82a4510f16df49238d57c9f78ac28ae7)
 2016-10-27 15:55:35 +09:00
Chris Nauroth
4df1780991 HADOOP-13502. Split fs.contract.is-blobstore flag into more descriptive flags for use by contract tests. Contributed by Chris Nauroth. 
(cherry picked from commit 1f8490a5bacd98d0d371447ada3b31f93ca40a4e)
(cherry picked from commit 082d69ee663d5c1a59f32c40b9fbde9996886de9)
 2016-10-26 08:57:46 -07:00
Chris Nauroth
c940c68c79 HADOOP-13614. Purge some superfluous/obsolete S3 FS tests that are slowing test runs down. Contributed by Steve Loughran. 
(cherry picked from commit 9cad3e235026dbe4658705ca85d263d0edf14521)
(cherry picked from commit 67e01f7218e592d7b18316d65f7b22ae8b9ad7a6)
 2016-10-26 08:27:39 -07:00
Chris Nauroth
3d5f41544a HADOOP-13309. Document S3A known limitations in file ownership and permission model. Contributed by Chris Nauroth. 
(cherry picked from commit 309a43925c078ff51cdb6bd1273e6f91f43311cb)
(cherry picked from commit 05d772e297fa3738b3fdddfa666bd6e23688f2b6)
 2016-10-25 09:03:19 -07:00
Chris Nauroth
78970e0db3 HADOOP-13727. S3A: Reduce high number of connections to EC2 Instance Metadata Service caused by InstanceProfileCredentialsProvider. Contributed by Chris Nauroth. 
(cherry picked from commit d8fa1cfa6722cbf7a4ec3d6b9c44b034da9aa351)
(cherry picked from commit 5b7cbb5a3c1877e51f63f6d6dfb201afa55dc4da)
 2016-10-24 21:23:00 -07:00
Xiaoyu Yao
7d826631ea HADOOP-13749. KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used. Contributed by Xiaoyu Yao. 
(cherry picked from commit d0a347984da175948b553a675dc357491df2fd0f)
 2016-10-23 10:59:10 -07:00
Xiaoyu Yao
751568e1d3 Revert "HDFS-10757. KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used. Contributed by Xiaoyu Yao." 
This reverts commit b82364759ca8600a43741e35c6781bec8249c547.
 2016-10-23 08:30:20 -07:00
Benoy Antony
653ceab7d3 HADOOP-12082 Support multiple authentication schemes via AuthenticationFilter. Contributed by Hrishikesh Gadre. 2016-10-21 19:43:22 -07:00
Xiaoyu Yao
b82364759c HDFS-10757. KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used. Contributed by Xiaoyu Yao. 
(cherry picked from commit be7237224819e2491aef91cd4f055c7efcf7b90d)
 2016-10-21 15:05:57 -07:00
Xiao Chen
7d19394bd5 HADOOP-13381. KMS clients should use KMS Delegation Tokens from current UGI. Contributed by Xiao Chen. 
(cherry picked from commit 8ebf2e95d2053cb94c6ff87ca018811fe8276f2b)

Conflicts:
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java

(cherry picked from commit 8fe4b2429a22cf932b701863170336a3b6986dd2)
 2016-10-21 14:59:56 -07:00
Steve Loughran
4b56954fea HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran. 2016-10-21 19:12:03 +01:00
Brahma Reddy Battula
5754772bd6 HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula 
(cherry picked from commit f872c6bc0390415f13e95b99749b0b1a690991b7)
 2016-10-21 16:22:42 +05:30
Steve Loughran
da7e2f08f1 Revert "HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran." 
This reverts commit 671d219c9cda32cfe7b7165dd1a9e174952f20e1.
 2016-10-20 22:49:11 +01:00
Anu Engineer
671d219c9c HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran. 2016-10-20 12:50:49 -07:00
Steve Loughran
2b575a90f3 Revert "HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula" 
This reverts commit b63dd7b63befc503a02289c1eabd0bd0da48e223.
 2016-10-20 19:43:16 +01:00
Brahma Reddy Battula
b63dd7b63b HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula 
(cherry picked from commit f872c6bc0390415f13e95b99749b0b1a690991b7)
 2016-10-20 21:49:21 +05:30
Steve Loughran
1ecbf323ba HADOOP-13560. S3ABlockOutputStream to support huge (many GB) file writes. Contributed by Steve Loughran 2016-10-18 19:34:07 +01:00
Vinayakumar B
36dad0abca HADOOP-12984. Add GenericTestUtils.getTestDir method and use it for emporary directory in tests (Contributed by Steve Loughran and Vinayakumar B 
This closes #89

(cherry picked from commit 8d29e2451f5ca60f864c7ece16722c0abdd1c657)
 2016-10-17 14:04:58 -07:00
Andrew Wang
15ff590c37 HADOOP-13724. Fix a few typos in site markdown documents. Contributed by Ding Fei. 
(cherry picked from commit 987ee51141a15d3f4d1df4dc792a192b92b87b5f)
(cherry picked from commit 4ed7cf3b362f94367c57d012608213f46f0e16e8)
 2016-10-17 13:32:52 -07:00