Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,331 Commits 326 Branches 371 Tags
Commit Graph

4298 Commits

Author SHA1 Message Date
Andrew Wang
99d0b3eb2b HDFS-11080. Update HttpFS to use ConfigRedactor. Contributed by Sean Mackrory. 2016-11-02 19:11:22 -07:00
Mingliang Liu
7a662bea89 HDFS-11095. BlockManagerSafeMode should respect extension period default config value (30s). Contributed by Mingliang Liu 
(cherry picked from commit 9b0c17f85232950ecd927427e5ad9d28dd9d277f)
 2016-11-02 15:02:08 -07:00
Brahma Reddy Battula
35e6081020 HDFS-11031. Add additional unit test for DataNode startup behavior when volumes fail. Contributed by Mingliang Liu. 2016-11-02 10:39:12 +05:30
Mingliang Liu
1dd59042ac HDFS-11065. Add space quota tests for heterogenous storages. Contributed by Xiaobing Zhou 
(cherry picked from commit fa1512a740b2ed2661743d6b5483ef3eb49e5634)
 2016-11-01 14:46:24 -07:00
Kihwal Lee
7c1a1834e4 HDFS-10997. Reduce number of path resolving methods. Contributed by Daryn Sharp. 2016-11-01 08:04:54 -05:00
Brahma Reddy Battula
1c5708f6de HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Rakesh R. 2016-11-01 10:49:43 +05:30
Mingliang Liu
6bb12ecc3f HDFS-11030. TestDataNodeVolumeFailure#testVolumeFailure is flaky (though passing). Contributed by Mingliang Liu 2016-10-31 16:03:02 -07:00
Arpit Agarwal
56092e351b HDFS-11074. Remove unused method FsDatasetSpi#getFinalizedBlocksOnPersistentStorage. Contributed by Hanisha Koneru. 2016-10-31 12:39:54 -07:00
Mingliang Liu
2848a27700 HDFS-11047. Remove deep copies of FinalizedReplica to alleviate heap consumption on DataNode. Contributed by Xiaobing Zhou 2016-10-28 11:11:31 -07:00
Mingliang Liu
d0da47ad9f HDFS-11048. Audit Log should escape control characters. Contributed by Eric Badger 
(cherry picked from commit 8a9388e5f6d622152798aaaa256064919e4fec37)
 2016-10-28 10:57:20 -07:00
Erik Krogen
ce8ace372c HDFS-9500. Fix software version counts for DataNodes during rolling upgrade. Contributed by Erik Krogen. 
(cherry picked from commit f3ac1f41b8fa82a0ac87a207d7afa2061d90a9bd)
 2016-10-27 16:00:39 -07:00
Wei-Chiu Chuang
2cf3138d72 HDFS-11055. Update default-log4j.properties for httpfs to imporve test logging. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 31ff42b51037632ec871f29efc0fa894e1b738d0)
 2016-10-27 13:38:02 -07:00
Kihwal Lee
5057abc390 HDFS-11069. Tighten the authorization of datanode RPC. Contributed by Kihwal Lee 
(cherry picked from commit ae48c496dce8d0eae4571fc64e6850d602bae688)
 2016-10-27 14:18:10 -05:00
Kihwal Lee
6c11a11918 Revert "HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Tianyin Xiu" 
This reverts commit f4300ea12af356c5d0fcc379b892ac918bafbc18.
 2016-10-27 14:10:07 -05:00
Brahma Reddy Battula
f4300ea12a HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Tianyin Xiu 
(cherry picked from commit ac35ee9393e0afce9fede1d2052e7bf4032312fd)
 2016-10-27 20:22:08 +05:30
Vinayakumar B
144f7a9e0e HDFS-8492. DN should notify NN when client requests a missing block (Contributed by Walter Su) 
(cherry picked from commit 1cf6ec4ad4e1f4ea71f912923b5e8627b61ef482)
 2016-10-27 16:46:09 +05:30
Akira Ajisaka
53ed19efec HDFS-11049. The description of dfs.block.replicator.classname is not clear. Contributed by Yiqun Lin. 
(cherry picked from commit 9c680d0b0508934b9b000ece84f25988272e901f)
 2016-10-27 16:37:32 +09:00
Masatake Iwasaki
7e918637de HADOOP-13017. Implementations of InputStream.read(buffer, offset, bytes) to exit 0 if bytes==0. Contributed by Steve Loughran. 
(cherry picked from commit 0bdd263d82a4510f16df49238d57c9f78ac28ae7)
 2016-10-27 15:54:03 +09:00
Vinayakumar B
72ce627533 HDFS-9929. Duplicate keys in NAMENODE_SPECIFIC_KEYS (Contributed by Akira Ajisaka) 
(cherry picked from commit f0642b02996ae90d85499ddef2e20bdf88d98c7b)
 2016-10-27 12:19:08 +05:30
Mingliang Liu
2aed61d887 HDFS-10921. TestDiskspaceQuotaUpdate doesn't wait for NN to get out of safe mode. Contributed by Eric Badger 
(cherry picked from commit 55e1fb8e3221941321e6f5e04b334246c5f23027)
 2016-10-26 14:12:08 -07:00
Kihwal Lee
0cd43dd2b7 HDFS-11053. Unnecessary superuser check in versionRequest(). Contributed by Kihwal Lee. 
(cherry picked from commit e90af4a89ba2741aeca20ead47699587b14f0de8)
 2016-10-26 09:17:29 -05:00
Kihwal Lee
366ebb0822 HDFS-11050. Change log level to 'warn' when ssl initialization fails and defaults to DEFAULT_TIMEOUT_CONN_CONFIGURATOR. Contributed by Kuhu Shukla. 
(cherry picked from commit ce6bbfb23c9aafaf1aaeaeceba88286d4270b316)
 2016-10-26 08:11:12 -05:00
Arpit Agarwal
a071e33a91 HDFS-11054. Suppress verbose log message in BlockPlacementPolicyDefault. Contributed by Chen Liang. 2016-10-25 19:51:32 -07:00
Xiaoyu Yao
590d19c0a8 HDFS-11011. Add unit tests for HDFS command 'dfsadmin -set/clrSpaceQuota'. Contributed by Xiaobing Zhou. 
(cherry picked from commit 85f6fec0370b3cd94d6c2f19920c0b6d33f127c6)
 2016-10-25 11:24:27 -07:00
Zhe Zhang
ff806cbfc7 HDFS-11015. Enforce timeout in balancer. Contributed by Kihwal Lee. 
(cherry picked from commit f6367c5f44a88cb5eb7edffb015b10b657504a61)
 2016-10-25 10:45:31 -07:00
Kihwal Lee
086577c67a HDFS-11042. Add missing cleanupSSLConfig() call for tests that use setupSSLConfig(). Contributed by Kuhu Shukla. 
(cherry picked from commit b18f35f097a3ee33f449e506676961f9b9456bbd)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
 2016-10-24 08:56:14 -05:00
Chris Douglas
25f4327f0b HADOOP-13702. Add instrumented ReadWriteLock. Contributed by Jingcheng Du 
(cherry picked from commit ae8bccd5090d8b42dae9a8e0c13a9766a7c42ecb)
 2016-10-21 13:01:08 -07:00
Brahma Reddy Battula
2ab149f0e4 HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula 
(cherry picked from commit f872c6bc0390415f13e95b99749b0b1a690991b7)
 2016-10-21 16:19:18 +05:30
Mingliang Liu
c37c00b061 HDFS-10998. Add unit tests for HDFS command 'dfsadmin -fetchImage' in HA. Contributed by Xiaobing Zhou 
(cherry picked from commit d7d87deece66333c188e9b7c10b4b56ddb529ce9)
 2016-10-20 19:54:35 -07:00
Aaron T. Myers
0f616cefeb HDFS-10423. Increase default value of httpfs maxHttpHeaderSize. Contributed by Nicolae Popa. 
(cherry picked from commit aa1b583cf99d1a7cfe554d1769fc4af252374663)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/libexec/httpfs-config.sh
 2016-10-20 13:34:52 -07:00
Wei-Chiu Chuang
6f4192d77d HDFS-11018. Incorrect check and message in FsDatasetImpl#invalidate. Contributed by Yiqun Lin. 
(cherry picked from commit 6d2da38d16cebe9b82f1048f87127eecee33664c)

Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2016-10-20 11:58:46 -07:00
Steve Loughran
ab36519b6f Revert "HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula" 
This reverts commit bf6379fdd2eef2fed30414559190afa8ccd963d5.
 2016-10-20 19:32:10 +01:00
Brahma Reddy Battula
bf6379fdd2 HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula 
(cherry picked from commit f872c6bc0390415f13e95b99749b0b1a690991b7)
 2016-10-20 21:48:23 +05:30
Kihwal Lee
cacf83b6c3 HDFS-10627. Volume Scanner marks a block as "suspect" even if the exception is network-related. Contributed by Rushabh S Shah. 
(cherry picked from commit 5c0bffddc0cb824a8a2751bcd0dc3e15ce081727)
 2016-10-20 11:02:02 -05:00
Brahma Reddy Battula
69a4b0c7d2 HDFS-10699. Log object instance get incorrectly in TestDFSAdmin. Contributed by Yiqun Lin 
(cherry picked from commit f5e401de3e7ad6e73b07fef1cb5915753b08fbdb)
 2016-10-20 21:08:39 +05:30
Brahma Reddy Battula
df5671db74 HDFS-9480. Expose nonDfsUsed via StorageTypeStats. Contributed by Brahma Reddy Battula 
(cherry picked from commit 4c73be135ca6ee2ba0b075a507097900db206b09)
 2016-10-20 20:47:37 +05:30
Brahma Reddy Battula
5a37668d1b HDFS-11025. TestDiskspaceQuotaUpdate fails in trunk due to Bind exception. Contributed by Yiqun Lin 
(cherry picked from commit 73504b1bdc4b93c64741de5eb9d022817fdfa22f)
 2016-10-20 10:42:34 +05:30
Arpit Agarwal
a3cbaf0c04 HDFS-10752. Several log refactoring/improvement suggestion in HDFS. Contributed by Hanisha Koneru. 2016-10-19 17:20:15 -07:00
Xiao Chen
ad7d3c4db8 HDFS-11009. Add a tool to reconstruct block meta file from CLI. 
(cherry picked from commit bc4a32aea50e86819730312e89315c0244ce64bf)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetUtil.java
 2016-10-18 22:40:58 -07:00
Xiaoyu Yao
74f1c49d76 HDFS-10906. Add unit tests for Trash with HDFS encryption zones. Contributed by Hanisha Koneru. 
(cherry picked from commit 0c79d5af74944363f3d04f7e1702411db564bbfd)
 2016-10-18 14:07:57 -07:00
Kai Zheng
3972bb3bba HDFS-10920. TestStorageMover#testNoSpaceDisk is failing intermittently. Contributed by Rakesh R 2016-10-18 15:08:15 +06:00
Akira Ajisaka
b2618685fb HADOOP-13522. Add %A and %a formats for fs -stat command to print permissions. Contributed by Alex Garbarini. 
(cherry picked from commit bedfec0c10144087168bc79501ffd5ab4fa52606)
 2016-10-18 15:03:33 +09:00
Vinitha Reddy Gankidi
278eeb1056 HDFS-10712. TestDataNodeVolumeFailure should pass not null BlockReportContext. Contributed by Vinitha Gankidi. 2016-10-17 18:55:26 -07:00
Ming Ma
d55a7f8935 HDFS-9390. Block management for maintenance states. 2016-10-17 17:46:29 -07:00
Xiaoyu Yao
61aceb87df HDFS-10922. Adding additional unit tests for Trash (II). Contributed by Weiwei Yang. 
(cherry picked from commit f1802d0be05ecc0b3248690b6f9efedbc7784112)
 2016-10-17 14:21:51 -07:00
Brahma Reddy Battula
64e532393b HDFS-11003. Expose XmitsInProgress through DataNodeMXBean. Contributed By Brahma Reddy Battula 
(cherry picked from commit 5f4ae85bd8a20510948696467873498723b06477)
 2016-10-15 22:29:55 +05:30
Akira Ajisaka
35832901e8 HDFS-11012. Unnecessary INFO logging on DFSClients for InvalidToken. Contributed by Harsh J. 
(cherry picked from commit 5ad037df25ab3206509083276b7ef4ef001be48b)
 2016-10-15 22:17:55 +09:00
Vinitha Reddy Gankidi
863e1020c0 HDFS-10301. Remove FBR tracking state to fix false zombie storage detection for interleaving block reports. Contributed by Vinitha Gankidi. 
(cherry picked from commit 391ce535a739dc92cb90017d759217265a4fd969)
 2016-10-14 18:25:52 -07:00
Lei Xu
78cb79fa91 HDFS-10960. TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove. (Manoj Govindassamy via lei) 2016-10-14 15:00:27 -07:00
Mingliang Liu
bbcbf78f0f HDFS-11008. Change unit test for testing parsing "-source" parameter in Balancer CLI. Contributed by Mingliang Liu 
(cherry picked from commit 76cc84e6d41c2b02218c2c98d60481cd565e067c)
 2016-10-14 14:43:58 -07:00