Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,137 Commits 326 Branches 371 Tags
Commit Graph

4709 Commits

Author SHA1 Message Date
Lei Xu
f09d20cffb HADOOP-14397. Pull up the builder pattern to FileSystem and add AbstractContractCreateTest for it. (Lei (Eddy) Xu) 
(cherry picked from commit 667ee003bf47e44beb3fdff8d06a7264a13dd22c)
 2017-07-31 20:07:13 -07:00
Weiwei Yang
60ae10b14f HDFS-12154. Incorrect javadoc description in StorageLocationChecker#check. Contributed by Nandakumar. 2017-07-31 17:02:54 -07:00
Arpit Agarwal
8c63580981 HDFS-12082. BlockInvalidateLimit value is incorrectly set after namenode heartbeat interval reconfigured. Contributed by Weiwei Yang. 2017-07-31 11:34:00 -07:00
Brahma Reddy Battula
8bfb9971ca HADOOP-14455. ViewFileSystem#rename should support be supported within same nameservice with different mountpoints. Contributed by Brahma Reddy Battula. 2017-07-30 14:01:41 +08:00
Yongjun Zhang
c3e8c7d56e HDFS-12190. Enable 'hdfs dfs -stat' to display access time. Contributed by Yongjun Zhang. 
(cherry picked from commit c6330f22a5e5c2370bab885f9bea4bf8f5e9cf44)
 2017-07-27 17:05:30 -07:00
Brahma Reddy Battula
1c5095b3cb HDFS-11896. Non-dfsUsed will be doubled on dead node re-registration. Contributed by Brahma Reddy Battula. 
(cherry picked from commit c4a85c694fae3f814ab4e7f3c172da1df0e0e353)
 2017-07-27 12:30:57 -07:00
Kihwal Lee
3d8c7d701b HDFS-12171. Reduce IIP object allocations for inode lookup. Contributed by Daryn Sharp. 2017-07-25 11:07:43 -05:00
Kihwal Lee
9fe5dd098d HDFS-12143. Improve performance of getting and removing inode features. Contributed by Daryn Sharp. 
(cherry picked from commit 1a79dcfc457969d6a6c08ffffe4152fd7638e48a)
 2017-07-25 10:30:27 -05:00
Akira Ajisaka
c108372777
HDFS-12176. dfsadmin shows DFS Used%: NaN% if the cluster has zero block. Contributed by Weiwei Yang. 
(cherry picked from commit 770cc462281518545e3d1c0f8c21cf9ec9673200)
 2017-07-24 16:24:50 +09:00
Kihwal Lee
c12bf9a128 HDFS-11742. Improve balancer usability after HDFS-8818. Contributed by Kihwal Lee 
(cherry picked from commit 8e3a992eccff26a7344c3f0e719898fa97706b8c)
 2017-07-21 09:19:01 -05:00
Konstantin V Shvachko
af2fb6a909 HDFS-11472. Fix inconsistent replica size after a data pipeline failure. Contributed by Erik Krogen and Wei-Chiu Chuang. 
(cherry picked from commit 2a5a313539e211736fef12010918a60f9edad030)
 2017-07-20 15:57:34 -07:00
Yongjun Zhang
10a6b6b12c HDFS-12139. HTTPFS liststatus returns incorrect pathSuffix for path of file. Contributed by Yongjun Zhang. 
(cherry picked from commit 3556e36be30211f46ac38899ce11a4d4cac6d635)
 2017-07-19 11:44:35 -07:00
Anu Engineer
e0297ffbc8 HDFS-12158. Secondary Namenode's web interface lack configs for X-FRAME-OPTIONS protection. Contributed by Mukul Kumar Singh. 
(cherry picked from commit 413b23eb04eee24275257ab462133e0818f87449)
 2017-07-19 10:42:28 -07:00
Brahma Reddy Battula
979d37ae21 HDFS-12067. Correct dfsadmin commands usage message to reflects IPC port. Contributed by steven-wugang. 
(cherry picked from commit f8cd55fe33665faf2d1b14df231516fc891118fc)
 2017-07-19 23:22:40 +08:00
Tsz-Wo Nicholas Sze
02627f8c5f HDFS-12138. Remove redundant 'public' modifiers from BlockCollection. Contributed by Chen Liang 2017-07-17 13:59:52 -07:00
Brahma Reddy Battula
a33e89931b HDFS-12112. TestBlockManager#testBlockManagerMachinesArray sometimes fails with NPE. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit b778887af59d96f1fac30cae14be1cabbdb74c8b)
 2017-07-15 10:49:11 +08:00
Xiao Chen
5337c2511d HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 
(cherry picked from commit aa395c1ec9a170f2da732c6e6114c60412161d40)
 2017-07-14 22:23:07 -07:00
Tsz-Wo Nicholas Sze
bff39ca718 HDFS-12130. Optimizing permission check for getContentSummary. Contributed by Chen Liang 2017-07-14 14:40:11 -07:00
Tsz-Wo Nicholas Sze
d51395363a Revert "HDFS-12130. Optimizing permission check for getContentSummary." to fix commit message. 
This reverts commit 2ba41729abef22f9049fbd6260e8194ac1abf5cc.
 2017-07-14 14:38:27 -07:00
Kihwal Lee
0fd86de128 HDFS-12140. Remove BPOfferService lock contention to get block pool id. Contributed by Daryn Sharp. 
(cherry picked from commit e7d187a1b6a826edd5bd0f708184d48f3674d489)
 2017-07-14 16:12:28 -05:00
Kihwal Lee
a47820d72a HDFS-12137. DN dataset lock should be fair. Contributed by Daryn Sharp. 2017-07-14 15:45:26 -05:00
Tsz-Wo Nicholas Sze
2ba41729ab HDFS-12130. Optimizing permission check for getContentSummary. 2017-07-14 13:42:02 -07:00
Xiaoyu Yao
af8a227a68 HDFS-11502. Datanode UI should display hostname based on JMX bean instead of window.location.hostname. Contributed by Jeffrey E Rodriguez and Yuanbo Liu. 
(cherry picked from commit e15e2713e1e344b14d63726639d1c83451921515)
 2017-07-12 16:38:56 -07:00
Tsz-Wo Nicholas Sze
b4a108fa9f HDFS-6874. Add GETFILEBLOCKLOCATIONS operation to HttpFS. Contributed by Weiwei Yang 2017-07-12 14:18:22 -07:00
Akira Ajisaka
2653cf4466
HADOOP-14629. Improve exception checking in FileContext related JUnit tests. Contributed by Andras Bokor. 
(cherry picked from commit 9144fd9e9b5d84d71158451428341746a6567152)
 2017-07-12 11:40:48 +09:00
Mingliang Liu
b803641523 HDFS-12089. Fix ambiguous NN retry log message in WebHDFS. Contributed by Eric Badger 
(cherry picked from commit 6436768baf1b2ac05f6786edcd76fd3a66c03eaa)
 2017-07-05 11:12:15 -07:00
Akira Ajisaka
063f2132ef
HDFS-12078. Add time unit to the description of property dfs.namenode.stale.datanode.interval in hdfs-default.xml. Contributed by Weiwei Yang. 
(cherry picked from commit 186650d21d482e2f3bc4523ae989ebe76081b0e3)
 2017-07-04 14:52:39 +09:00
Akira Ajisaka
003b4a1675
HDFS-12079. Description of dfs.block.invalidate.limit is incorrect in hdfs-default.xml. Contributed by Weiwei Yang. 
(cherry picked from commit b0560e0624756e2b3ce7b6bc741eee3c18d2a873)
 2017-07-04 14:03:07 +09:00
Wei-Chiu Chuang
94bc5cdbb3 HDFS-12042. Lazy initialize AbstractINodeDiffList#diffs for snapshots to reduce memory consumption. Contributed by Misha Dmitriev. 
(cherry picked from commit bcba844d1144cc334e2babbc34c9d42eac1c203a)
 2017-06-30 11:42:51 -07:00
Wei-Chiu Chuang
6917c19167 HDFS-11881. NameNode consumes a lot of memory for snapshot diff report generation. Contributed by Manoj Govindassamy. 
(cherry picked from commit 16c8dbde574f49827fde5ee9add1861ee65d4645)
 2017-06-29 06:40:24 -07:00
Akira Ajisaka
a896c0f11e
HDFS-12040. TestFsDatasetImpl.testCleanShutdownOfVolume fails. Contributed by hu xiaodong. 
(cherry picked from commit 07defa4c09db6e8c552167019ca0d2444cfc8fe7)
 2017-06-27 20:49:56 +09:00
Ravi Prakash
ef491f4637 HDFS-11993. Add log info when connect to datanode socket address failed. Contributed by chencan 
(cherry picked from commit a9d3412b4ce40f5ab5a18756ede7e0606b653171)
 2017-06-26 13:25:32 -07:00
Akira Ajisaka
b59b96deee
HDFS-11992. Replace commons-logging APIs with slf4j in FsDatasetImpl. Contributed by hu xiaodong. 2017-06-27 03:28:09 +09:00
Kihwal Lee
8f533ca76b HDFS-11960. Successfully closed files can stay under-replicated. Contributed by Kihwal Lee. 2017-06-20 09:16:37 -05:00
Brahma Reddy Battula
933fad2e17 HADOOP-14533. Size of args cannot be less than zero in TraceAdmin#run as its linkedlist. Contributed by Weisen Han. 
(cherry picked from commit 2e9daa2e27536484846d9c5f020c6afaf27932ce)
 2017-06-20 20:45:33 +08:00
Akira Ajisaka
6b3c13d543
HDFS-11345. Document the configuration key for FSNamesystem lock fairness. Contributed by Erik Krogen. 
(cherry picked from commit 2c769167dbdb66c52d2ba7b7193a686444085570)
 2017-06-20 16:36:47 +09:00
Akira Ajisaka
71626fd0b9
HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin. 2017-06-20 13:55:42 +09:00
Masatake Iwasaki
b19ca498ae HDFS-11995. HDFS Architecture documentation incorrectly describes writing to a local temporary file. Contributed by Nandakumar. 
(cherry picked from commit d954a64730c00346476322743462cde857164177)
 2017-06-20 08:09:09 +09:00
Andrew Wang
1ec1aebd35 HDFS-10480. Add an admin command to list currently open files. Contributed by Manoj Govindassamy. 2017-06-19 09:52:07 -07:00
Lei Xu
33afa1fdca HADOOP-14395. Provide Builder pattern for DistributedFileSystem.append. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 6460df21a09a7fcc29eceb8dc3859d6298da6882)

    Conflicts:
    hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
 2017-06-16 17:32:29 -07:00
Arpit Agarwal
9529513f18 HADOOP-14503. Make RollingAverages a mutable metric. Contributed by Hanisha Koneru. 2017-06-16 16:41:08 -07:00
Xiao Chen
2afe9722af HADOOP-14524. Make CryptoCodec Closeable so it can be cleaned up proactively. 2017-06-16 09:49:41 -07:00
Lei Xu
abac844c90 HADOOP-14394. Provide Builder pattern for DistributedFileSystem.create. (lei) 
(cherry picked from commit 5fbec46525d6d49837d934556b59ba77bd2301a8)

    Conflicts:
    hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/NameNodeConnector.java
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicies.java
 2017-06-15 10:59:11 -07:00
Akira Ajisaka
0fd0612896
Revert "HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin." 
This reverts commit 88fa17c2383baafbfb6b43b69e08cf40493c774a.
 2017-06-16 00:21:54 +09:00
Akira Ajisaka
88fa17c238
HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin. 
(cherry picked from commit 315f07700dcfe32513b87f63070db854ecb3cf48)
 2017-06-15 13:47:22 +09:00
Tsz-Wo Nicholas Sze
670224ae59 HDFS-11947. When constructing a thread name, BPOfferService may print a bogus warning message. Contributed by Weiwei Yang 2017-06-13 09:47:14 +08:00
Arpit Agarwal
eda4bf5196 HDFS-11967. TestJMXGet fails occasionally. Contributed by Arpit Agarwal. 2017-06-12 16:04:08 -07:00
Arpit Agarwal
097b1f74e5 HDFS-11907. Add metric for time taken by NameNode resource check. Contributed by Chen Liang. 2017-06-12 15:22:36 -07:00
Mingliang Liu
facb6812d9 HDFS-11945. Internal lease recovery may not be retried for a long time. Contributed by Kihwal Lee 2017-06-08 14:46:31 -07:00
Brahma Reddy Battula
fca08f8362 HDFS-11711. DN should not delete the block On "Too many open files" Exception. Contributed by Brahma Reddy Battula. 2017-06-07 16:54:29 +08:00