Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,830 Commits 326 Branches 371 Tags
Commit Graph

11830 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Brahma Reddy Battula
f5b080e152 HDFS-11025. TestDiskspaceQuotaUpdate fails in trunk due to Bind exception. Contributed by Yiqun Lin 
(cherry picked from commit 73504b1bdc4b93c64741de5eb9d022817fdfa22f)
 2016-10-20 10:47:16 +05:30
Arpit Agarwal
e9592f14d5 HDFS-10752. Several log refactoring/improvement suggestion in HDFS. Contributed by Hanisha Koneru. 2016-10-19 17:22:23 -07:00
Rohith Sharma K S
34023ca418 YARN-4763. RMApps Page crashes with NPE. (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit 19ee1859071509bba9ecd0a8a7dc6a47e2979c88)
 2016-10-19 17:40:42 +00:00
Xiao Chen
d22a6a8b82 HDFS-11009. Add a tool to reconstruct block meta file from CLI. 
(cherry picked from commit bc4a32aea50e86819730312e89315c0244ce64bf)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetUtil.java

(cherry picked from commit ad7d3c4db8bfab007cc2ec1bad3c388fd7144369)
 2016-10-18 22:42:09 -07:00
Xiaoyu Yao
190a53b898 HDFS-10906. Add unit tests for Trash with HDFS encryption zones. Contributed by Hanisha Koneru. 
(cherry picked from commit 0c79d5af74944363f3d04f7e1702411db564bbfd)
 2016-10-18 14:09:03 -07:00
Steve Loughran
1ecbf323ba HADOOP-13560. S3ABlockOutputStream to support huge (many GB) file writes. Contributed by Steve Loughran 2016-10-18 19:34:07 +01:00
Kai Zheng
d26e11f30e HDFS-10920. TestStorageMover#testNoSpaceDisk is failing intermittently. Contributed by Rakesh R 2016-10-18 15:09:25 +06:00
Vinitha Reddy Gankidi
ae972dd080 HDFS-10712. TestDataNodeVolumeFailure should pass not null BlockReportContext. Contributed by Vinitha Gankidi. 
(cherry picked from commit 278eeb105611444bee0261ce7a0e6ea983a5c615)
 2016-10-17 19:02:29 -07:00
Xiaoyu Yao
f8b2d77207 HDFS-10922. Adding additional unit tests for Trash (II). Contributed by Weiwei Yang. 
(cherry picked from commit f1802d0be05ecc0b3248690b6f9efedbc7784112)
 2016-10-17 14:04:58 -07:00
Vinayakumar B
36dad0abca HADOOP-12984. Add GenericTestUtils.getTestDir method and use it for emporary directory in tests (Contributed by Steve Loughran and Vinayakumar B 
This closes #89

(cherry picked from commit 8d29e2451f5ca60f864c7ece16722c0abdd1c657)
 2016-10-17 14:04:58 -07:00
Andrew Wang
15ff590c37 HADOOP-13724. Fix a few typos in site markdown documents. Contributed by Ding Fei. 
(cherry picked from commit 987ee51141a15d3f4d1df4dc792a192b92b87b5f)
(cherry picked from commit 4ed7cf3b362f94367c57d012608213f46f0e16e8)
 2016-10-17 13:32:52 -07:00
Allen Wittenauer
9d473b8ddc HADOOP-11559. Add links to RackAwareness and InterfaceClassification to site index (Masatake Iwasaki via aw) 
(cherry picked from commit 7eeca90daabd74934d4c94af6f07fd598abdb4ed)

 Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-project/src/site/site.xml

(cherry picked from commit fbdb23d2afa993f96f073b9c4208282e8a280016)
 2016-10-17 13:32:51 -07:00
Eric Yang
b2667441e6 HADOOP-13707. Skip authorization for anonymous user to access Hadoop 
web interface in non-secure environment.  (Yuanbo Liu via eyang)
 2016-10-17 08:12:04 -07:00
Masatake Iwasaki
74782e7e42 HADOOP-13419. Fix javadoc warnings by JDK8 in hadoop-common package. Contributed by Kai Sasaki. 
(cherry picked from commit a13a607e2099153a6b63afe06be24c1bc453158f)
 2016-10-16 12:48:09 +09:00
Brahma Reddy Battula
1c473898aa HDFS-11003. Expose XmitsInProgress through DataNodeMXBean. Contributed By Brahma Reddy Battula 
(cherry picked from commit 5f4ae85bd8a20510948696467873498723b06477)
 2016-10-15 22:40:57 +05:30
Brahma Reddy Battula
34469cd98d Revert "HADOOP-13707. Skip authorization for anonymous user to access Hadoop" 
This reverts commit 3574e5692da6da9b0eb9fbee66ef197ffbf1d1cd.
 2016-10-15 22:18:24 +05:30
Akira Ajisaka
65e6a1181c HDFS-11012. Unnecessary INFO logging on DFSClients for InvalidToken. Contributed by Harsh J. 
(cherry picked from commit 5ad037df25ab3206509083276b7ef4ef001be48b)
(cherry picked from commit 35832901e87faa8cde687568df3145bea78b4121)
 2016-10-15 22:19:14 +09:00
Eric Yang
3574e5692d HADOOP-13707. Skip authorization for anonymous user to access Hadoop 
web interface in non-secure environment.  (Yuanbo Liu via eyang)
 2016-10-14 22:14:59 -07:00
Vinitha Reddy Gankidi
2304501bcf HDFS-10301. Remove FBR tracking state to fix false zombie storage detection for interleaving block reports. Contributed by Vinitha Gankidi. 
(cherry picked from commit 391ce535a739dc92cb90017d759217265a4fd969)
 2016-10-14 18:52:00 -07:00
Lei Xu
b9fdbd7107 HDFS-10960. TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove. (Manoj Govindassamy via lei) 
(cherry picked from commit 78cb79fa915ad0ca26d649ad88fdb44b310a99d8)
 2016-10-14 15:01:37 -07:00
Mingliang Liu
15b1d522ee HDFS-11008. Change unit test for testing parsing "-source" parameter in Balancer CLI. Contributed by Mingliang Liu 
(cherry picked from commit 76cc84e6d41c2b02218c2c98d60481cd565e067c)
 2016-10-14 14:44:34 -07:00
Kihwal Lee
8d1fbf786b Revert "HDFS-10960. TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove. (Manoj Govindassamy via lei)" 
This reverts commit f981dd1bca1006f34f55361ba0d72e5d0a621753.
 2016-10-14 16:37:51 -05:00
Lei Xu
f981dd1bca HDFS-10960. TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove. (Manoj Govindassamy via lei) 
(cherry picked from commit 8c520a27cbd9daba05367d3a83017a2eab5258eb)
 2016-10-14 13:44:18 -07:00
Andrew Wang
f13b84b4eb HDFS-10883. 's behavior is not consistent in DFS after enabling EZ. Contributed by Yuanbo Liu. 
(cherry picked from commit 0007360c3344b3485fa17de0fd2015a628de947c)
 2016-10-14 11:41:42 -07:00
Varun Saxena
28d75482ff YARN-5599. Publish AM launch command to ATS (Rohith Sharma K S via Varun Saxena) 2016-10-14 23:36:39 +05:30
Benoy Antony
ca20ffea46 HDFS-10735 Distcp using webhdfs on secure HA clusters fails with StandbyException 
(cherry picked from commit 701c27a7762294e1a5fb2b3ac81f5534aa37f667)
 2016-10-14 10:32:37 -07:00
Kihwal Lee
8c71fc1221 HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin. 
(cherry picked from commit 8a9f6635a33e9648e9396e9ec5571fa34aa0c773)
 2016-10-14 11:44:25 -05:00
Xiaoyu Yao
e41295182a HADOOP-13686. Adding additional unit test for Trash (I). Contributed by Weiwei Yang. 
(cherry picked from commit dbe663d5241feea0c88a3a9391ad48a029001d94)
(cherry picked from commit 1eb8c0e8804ced48aded091369d721e8061426ee)
 2016-10-14 08:45:14 -07:00
Xiao Chen
1ea7070042 HADOOP-13669. Addendum patch for KMS Server should log exceptions before throwing. 
(cherry picked from commit ae51b11f7872eaac558acf00fd23f6d7b1841cfe)
 2016-10-13 22:32:30 -07:00
Brahma Reddy Battula
0884c31aec HDFS-10986. DFSAdmin should log detailed error message if any. Contributed by MingLiang Liu 2016-10-14 10:28:13 +05:30
Jing Zhao
d3c4465147 HADOOP-13024. Distcp with -delete feature on raw data not implemented. Contributed by Mavin Martin. 
(cherry picked from commit 0a85d079838f532a13ca237300386d1b3bc1b178)
(cherry picked from commit ad69baf6a9139c0af81e9f72e41c1e3aeb119ebc)
 2016-10-13 13:28:05 -07:00
Andrew Wang
3892fd810b Revert "HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin." 
This reverts commit 7a5aaa789d9160ac55f022aae64a0b437ddccf06.
 2016-10-13 13:23:01 -07:00
Kihwal Lee
ded91992ad HDFS-10987. Make Decommission less expensive when lot of blocks present. Contributed by Brahma Reddy Battula. 
(cherry picked from commit c5a13037048eb1e3b5a500aeec0e2e953e7d509a)
 2016-10-13 14:57:44 -05:00
Andrew Wang
7a5aaa789d HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin. 
(cherry picked from commit fdce515091f0a61ffd6c9ae464a68447dedf1124)
(cherry picked from commit 2e153bc8abc3fe2f50660136af32711c50e7e7e7)
 2016-10-13 11:42:23 -07:00
Arpit Agarwal
3a9808e96c HADOOP-13710. Supress CachingGetSpaceUsed from logging interrupted exception stacktrace. Contributed by Hanisha Koneru. 2016-10-13 11:37:25 -07:00
Xiaoyu Yao
5abc788825 HADOOP-13565. KerberosAuthenticationHandler#authenticate should not rebuild SPN based on client request. Contributed by Xiaoyu Yao. 
(cherry picked from commit 9097e2efe4c92d83c8fab88dc11be84505a6cab5)
 2016-10-13 10:58:20 -07:00
Kihwal Lee
78777d4a96 HDFS-11000. webhdfs PUT does not work if requests are routed to call queue. Contributed by Kihwal Lee. 
(cherry picked from commit 9454dc5e8091354cd0a4b8c8aa5f4004529db5d5)
 2016-10-13 09:02:59 -05:00
Akira Ajisaka
c7780b066d HDFS-11002. Fix broken attr/getfattr/setfattr links in ExtendedAttributes.md. Contributed by Mingliang Liu. 
(cherry picked from commit 901eca004d0e7e413b109a93128892176c808d61)
(cherry picked from commit 43cf0b29732ebafb8b5b23b06d6015fe4c7777b4)
 2016-10-13 14:31:09 +09:00
Mingliang Liu
9b2a719031 HDFS-10903. Replace config key literal strings with config key names II: hadoop hdfs. Contributed by Chen Liang 
(cherry picked from commit b36aaa913c54829adbf5fa852af7f45be4db0f07)
 2016-10-12 17:41:07 -07:00
Andrew Wang
9bde45d2fe HADOOP-13700. Remove unthrown IOException from TrashPolicy#initialize and #getInstance signatures. 
(cherry picked from commit 12d739a34ba868b3f7f5adf7f37a60d4aca9061b)
(cherry picked from commit f131d61ff8bc87e74c67fec3ae2b50ee50dd0d2c)
 2016-10-12 15:20:10 -07:00
Kihwal Lee
136c6f6f7d HDFS-10789. Route webhdfs through the RPC call queue. Contributed by Daryn Sharp and Rushabh S Shah. 
(cherry picked from commit 5305a392c39d298ecf38ca2dfd2526adeee9cd38)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
 2016-10-12 15:38:34 -05:00
Mingliang Liu
0f481e2022 HDFS-10965. Add unit test for HDFS command 'dfsadmin -printTopology'. Contributed by Xiaobing Zhou 2016-10-11 17:18:48 -07:00
Xiao Chen
bc5a9931d7 HADOOP-13698. Document caveat for KeyShell when underlying KeyProvider does not delete a key. 
(cherry picked from commit b84c4891f9eca8d56593e48e9df88be42e24220d)
 2016-10-11 17:05:20 -07:00
Xiaoyu Yao
6f2de54eb1 HDFS-10984. Expose nntop output as metrics. Contributed by Siddharth Wagle. 
(cherry picked from commit 61f0490a73085bbaf6639d9234277e59dc1145db)
 2016-10-11 16:00:17 -07:00
Andrew Wang
67f0319db6 HDFS-10991. Export hdfsTruncateFile symbol in libhdfs. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit dacd3ec66b111be24131957c986f0c748cf9ea26)
 2016-10-11 15:07:28 -07:00
Wei-Chiu Chuang
773e2e6b6c HADOOP-13684. Snappy may complain Hadoop is built without snappy if libhadoop is not found. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 4b32b1420d98ea23460d05ae94f2698109b3d6f7)
(cherry picked from commit 56184de510e27990c36726e2a648e8ce4b06504c)
 2016-10-11 13:22:41 -07:00
Wei-Chiu Chuang
aeb75213d4 HDFS-10933. Refactor TestFsck. Contributed by Takanobu Asanuma. 
(cherry picked from commit a6b3dc69d7a0835fa9159a0536c24ff6525e6c77)

Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
 2016-10-11 13:18:32 -07:00
Kihwal Lee
7f514e0c05 HDFS-10980. Optimize check for existence of parent directory. Contributed by Daryn Sharp. 
(cherry picked from commit e57fa81d9559a93d77fd724f7792326c31a490be)
 2016-10-11 09:41:47 -05:00
Mingliang Liu
5a39a7560a HDFS-10972. Add unit test for HDFS command 'dfsadmin -getDatanodeInfo'. Contributed by Xiaobing Zhou 2016-10-10 17:46:31 -07:00
Chris Nauroth
67d8301e55 HADOOP-13447. Refactor S3AFileSystem to support introduction of separate metadata repository and tests. Contributed by Chris Nauroth. 
(cherry picked from commit d152557cf7f4d2288524c222fcbaf152bdc038b0)
(cherry picked from commit e28930a38b3868bc701aeef229831292320b4a3a)
 2016-10-10 14:01:19 -07:00