Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,419 Commits 326 Branches 371 Tags
Commit Graph

3174 Commits

Author SHA1 Message Date
Mingliang Liu
9af7d1e298 HADOOP-13655. document object store use with fs shell and distcp. Contributed by Steve Loughran 2016-11-22 13:19:32 -08:00
Brahma Reddy Battula
2207a84a7d HADOOP-13646. Remove outdated overview.html. Contributed By Brahma Reddy Battula. 
(cherry picked from commit afcf8d38e750f935c06629e641a1321b79c4cace)
 2016-11-22 20:32:30 +05:30
Mingliang Liu
92767f956f HADOOP-11601. Enhance FS spec & tests to mandate FileStatus.getBlocksize() >0 for non-empty files. Contributed by Steve Loughran 
(cherry picked from commit ae8849fe378e11b9db642ef7784c8e6a08199b96)

This closes #50
 2016-11-17 14:26:44 -08:00
Kihwal Lee
d3fa53a52b HADOOP-13742. Expose NumOpenConnectionsPerUser as a metric. Brahma Reddy Battula. 
(cherry picked from commit bd3735554fa5c3bc064c57ec78f4308430b14b48)
 2016-11-17 12:19:23 -06:00
Mingliang Liu
47526a25cc HADOOP-13822. Use GlobalStorageStatistics.INSTANCE.reset() at FileSystem#clearStatistics(). Contribute by Brahma Reddy Battula 
(cherry picked from commit aab9737a058c0bdeeb17c173e530b7e571315a56)
 2016-11-16 20:09:07 -08:00
Mingliang Liu
4e0fcff8ab HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu 
(cherry picked from commit 5af572b6443715b7a741296c1bd520a1840f9a7c)
 2016-11-15 11:36:08 -08:00
Mingliang Liu
0b428075a2 HADOOP-13810. Add a test to verify that Configuration handles &-encoded characters. Contributed by Steve Loughran 
This closes #158

(cherry picked from commit 04014c4c739bb4e3bc3fdf9299abc0f47521e8fd)
 2016-11-14 19:41:32 -08:00
Zhe Zhang
818915aabb HDFS-10872. Add MutableRate metrics for FSNamesystemLock operations. Contributed by Erik Krogen. 2016-11-14 16:33:15 -08:00
Yongjun Zhang
c684a00ed3 HADOOP-13720. Add more info to the msgs printed in AbstractDelegationTokenSecretManager. Contributed by Yongjun Zhang. 
(cherry picked from commit fd2f22adec5c2f21f792045dbfde9385c21403ec)
 2016-11-10 23:06:09 -08:00
Xiao Chen
ede909144d HADOOP-13590. Retry until TGT expires even if the UGI renewal thread encountered exception. 
(cherry picked from commit 367c3d41217728c2e61252c5a5235e5bc1f9822f)
 2016-11-09 09:09:13 -08:00
Subru Krishnan
6685a0a064 YARN-5833. Addendum patch to fix JDK7 compile issue. 2016-11-08 16:54:19 -08:00
Andrew Wang
4f25fd975e HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey. 
(cherry picked from commit e1c6ef2efa9d87fdfc7474ca63998a13a3929874)
 2016-11-08 16:33:58 -08:00
Zhe Zhang
758638d08d HADOOP-13782. Make MutableRates metrics thread-local write, aggregate-on-read. Contributed by Erik Krogen. 
(cherry picked from commit 77c13c385774c51766fe505397fa916754ac08d4)
 2016-11-08 16:08:03 -08:00
Mingliang Liu
38b1ed18ef HADOOP-13802. Make generic options help more consistent, and aligned. Contributed by Grant Sohn 
(cherry picked from commit 2a65eb121e23243fcb642d28b3f74241536485d8)
 2016-11-08 15:40:54 -08:00
Subru Krishnan
797c7bfab9 YARN-5833. Add validation to ensure default ports are unique in Configuration. (Konstantinos Karanasos via Subru). 
(cherry picked from commit 29e3b3417c16c83dc8e753f94d7ca9957dddbedd)
 2016-11-08 14:39:09 -08:00
Zhe Zhang
b245e9ce2f HADOOP-13804. MutableStat mean loses accuracy if add(long, long) is used. Contributed by Erik Krogen. 
(cherry picked from commit 3dbad5d823b8bf61b643dd1057165044138b99e0)
 2016-11-07 16:11:27 -08:00
Steve Loughran
29c6a0be0d HADOOP-13797 Remove hardcoded absolute path for ls. Contributed by Christine Koppelt 2016-11-07 12:36:25 +00:00
Xiao Chen
f30d338766 HDFS-10756. Expose getTrashRoot to HTTPFS and WebHDFS. Contributed by Yuanbo Liu. 
(cherry picked from commit d8bab3dcb693b2773ede9a6e4f71ae85ee056f79)
 2016-11-04 18:08:46 -07:00
Robert Kanter
0b36dcda7c HADOOP-7930. Kerberos relogin interval in UserGroupInformation should be configurable (xiaochen via rkanter) 2016-11-04 09:24:21 -07:00
Xiao Chen
1e29379244 HADOOP-12453. Support decoding KMS Delegation Token with its own Identifier. Contributed by Xiaoyu Yao. 
(cherry picked from commit 7154a20bcb1559c23aeb3b78b920bed03d834cb5)
 2016-11-03 13:10:37 -07:00
Andrew Wang
99d0b3eb2b HDFS-11080. Update HttpFS to use ConfigRedactor. Contributed by Sean Mackrory. 2016-11-02 19:11:22 -07:00
Arpit Agarwal
148bb3e737 HADOOP-13738. DiskChecker should perform some disk IO. 2016-11-01 18:09:12 -07:00
Ravi Prakash
045501a538 HADOOP-13773. Wrong HADOOP_CLIENT_OPTS in hadoop-env on branch-2. Contributed by Fei Hui 2016-11-01 10:49:04 -07:00
Steve Loughran
c4ccafdaff HADOOP-13680. fs.s3a.readahead.range to use getLongBytes. Contributed by Abhishek Modi. 2016-10-31 20:52:49 +00:00
Wei-Chiu Chuang
9ba283c27d HDFS-11061. Update dfs -count -t command line help and documentation. Contributed by Yiqun Lin. 
(cherry picked from commit 112f04eb1b771e410ffd232a46e0ee2bc7b99d06)
 2016-10-28 15:37:30 -07:00
Jason Lowe
0f224d405f HADOOP-13770. Shell.checkIsBashSupported swallowed an interrupted exception. Contributed by Wei-Chiu Chuang 
(cherry picked from commit c017171da00a6cd71a2901c84a0298ce14a49e23)
 2016-10-28 15:06:23 +00:00
Brahma Reddy Battula
5e760c89a1 HADOOP-13201. Print the directory paths when ViewFs denies the rename operation on internal dirs. Contributed by Tianyin Xiu 
(cherry picked from commit 0c837db8a874079dd5db83a7eef9c4d2b9d0e9ff)
 2016-10-27 19:48:21 +05:30
Masatake Iwasaki
7e918637de HADOOP-13017. Implementations of InputStream.read(buffer, offset, bytes) to exit 0 if bytes==0. Contributed by Steve Loughran. 
(cherry picked from commit 0bdd263d82a4510f16df49238d57c9f78ac28ae7)
 2016-10-27 15:54:03 +09:00
Chris Nauroth
082d69ee66 HADOOP-13502. Split fs.contract.is-blobstore flag into more descriptive flags for use by contract tests. Contributed by Chris Nauroth. 
(cherry picked from commit 1f8490a5bacd98d0d371447ada3b31f93ca40a4e)
 2016-10-26 08:57:29 -07:00
Chris Nauroth
67e01f7218 HADOOP-13614. Purge some superfluous/obsolete S3 FS tests that are slowing test runs down. Contributed by Steve Loughran. 
(cherry picked from commit 9cad3e235026dbe4658705ca85d263d0edf14521)
 2016-10-26 08:27:33 -07:00
Chris Nauroth
05d772e297 HADOOP-13309. Document S3A known limitations in file ownership and permission model. Contributed by Chris Nauroth. 
(cherry picked from commit 309a43925c078ff51cdb6bd1273e6f91f43311cb)
 2016-10-25 09:03:11 -07:00
Benoy Antony
b913b62aaf HADOOP-12082. Support multiple authentication schemes via AuthenticationFilter. Contributed by Hrishikesh Gadre. 2016-10-25 08:30:02 -07:00
Chris Nauroth
5b7cbb5a3c HADOOP-13727. S3A: Reduce high number of connections to EC2 Instance Metadata Service caused by InstanceProfileCredentialsProvider. Contributed by Chris Nauroth. 
(cherry picked from commit d8fa1cfa6722cbf7a4ec3d6b9c44b034da9aa351)
 2016-10-24 21:22:46 -07:00
Xiaoyu Yao
48c9064b33 HADOOP-13749. KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used. Contributed by Xiaoyu Yao. 
(cherry picked from commit d0a347984da175948b553a675dc357491df2fd0f)
 2016-10-23 10:59:01 -07:00
Xiaoyu Yao
68d91e0200 Revert "HDFS-10757. KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used. Contributed by Xiaoyu Yao." 
This reverts commit 0205ad5fa81c4001b4d5189dce88b4546a98b2eb.
 2016-10-23 08:26:01 -07:00
Xiaoyu Yao
0205ad5fa8 HDFS-10757. KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used. Contributed by Xiaoyu Yao. 
(cherry picked from commit be7237224819e2491aef91cd4f055c7efcf7b90d)
 2016-10-21 14:25:29 -07:00
Chris Douglas
25f4327f0b HADOOP-13702. Add instrumented ReadWriteLock. Contributed by Jingcheng Du 
(cherry picked from commit ae8bccd5090d8b42dae9a8e0c13a9766a7c42ecb)
 2016-10-21 13:01:08 -07:00
Steve Loughran
385c1daa46 HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran. 2016-10-21 19:11:31 +01:00
Brahma Reddy Battula
2ab149f0e4 HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula 
(cherry picked from commit f872c6bc0390415f13e95b99749b0b1a690991b7)
 2016-10-21 16:19:18 +05:30
Steve Loughran
f0af3dee25 Revert "HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran." 
This reverts commit 42f8a1d6eb5add3180094b126c4af2b3ddbc7cae.
 2016-10-20 22:48:26 +01:00
Anu Engineer
1f384b617d HADOOP-13737. Cleanup DiskChecker interface. Contributed by Arpit Agarwal. 2016-10-20 13:40:01 -07:00
Anu Engineer
42f8a1d6eb HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran. 2016-10-20 12:47:57 -07:00
Steve Loughran
ab36519b6f Revert "HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula" 
This reverts commit bf6379fdd2eef2fed30414559190afa8ccd963d5.
 2016-10-20 19:32:10 +01:00
Brahma Reddy Battula
bf6379fdd2 HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula 
(cherry picked from commit f872c6bc0390415f13e95b99749b0b1a690991b7)
 2016-10-20 21:48:23 +05:30
Steve Loughran
bc176961e6 HADOOP-13560. S3ABlockOutputStream to support huge (many GB) file writes. Contributed by Steve Loughran 2016-10-18 19:33:38 +01:00
Akira Ajisaka
b2618685fb HADOOP-13522. Add %A and %a formats for fs -stat command to print permissions. Contributed by Alex Garbarini. 
(cherry picked from commit bedfec0c10144087168bc79501ffd5ab4fa52606)
 2016-10-18 15:03:33 +09:00
Andrew Wang
4ed7cf3b36 HADOOP-13724. Fix a few typos in site markdown documents. Contributed by Ding Fei. 
(cherry picked from commit 987ee51141a15d3f4d1df4dc792a192b92b87b5f)
 2016-10-17 13:32:39 -07:00
Allen Wittenauer
fbdb23d2af HADOOP-11559. Add links to RackAwareness and InterfaceClassification to site index (Masatake Iwasaki via aw) 
(cherry picked from commit 7eeca90daabd74934d4c94af6f07fd598abdb4ed)

 Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-project/src/site/site.xml
 2016-10-17 13:32:28 -07:00
Eric Yang
7993fb5b81 HADOOP-13707. Skip authorization for anonymous user to access Hadoop 
web interface in non-secure environment.  (Yuanbo Liu via eyang)

    (cherry picked from commit dc308e98b9f8f7458a28a015515876ddea5666f2)
    With Addendum patch
 2016-10-17 07:57:45 -07:00
Eric Yang
a4356bb7a5 HADOOP-13707. Skip authorization for anonymous user to access Hadoop 
web interface in non-secure environment.  (Yuanbo Liu via eyang)

    (cherry picked from commit dc308e98b9f8f7458a28a015515876ddea5666f2)
    With Addendum patch
 2016-10-16 20:32:48 -07:00