Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,406 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4514 Commits

Author SHA1 Message Date
Manoj Govindassamy b0330c15da HDFS-12217. HDFS snapshots doesn't capture all open files when one of the open files is deleted. 
(cherry picked from commit 52d7bafcf4)
 2017-08-01 17:15:29 -07:00
Manoj Govindassamy 70c511598f HDFS-11402. HDFS Snapshots should capture point-in-time copies of OPEN files. 2017-08-01 17:11:27 -07:00
Akira Ajisaka 72bde1dad5
HADOOP-14245. Use Mockito.when instead of Mockito.stub. Contributed by Andras Bokor. 
(cherry picked from commit b38a1eea8e)
 2017-08-01 15:16:21 +09:00
Weiwei Yang 60ae10b14f HDFS-12154. Incorrect javadoc description in StorageLocationChecker#check. Contributed by Nandakumar. 2017-07-31 17:02:54 -07:00
Arpit Agarwal 8c63580981 HDFS-12082. BlockInvalidateLimit value is incorrectly set after namenode heartbeat interval reconfigured. Contributed by Weiwei Yang. 2017-07-31 11:34:00 -07:00
Brahma Reddy Battula 8bfb9971ca HADOOP-14455. ViewFileSystem#rename should support be supported within same nameservice with different mountpoints. Contributed by Brahma Reddy Battula. 2017-07-30 14:01:41 +08:00
Yongjun Zhang c3e8c7d56e HDFS-12190. Enable 'hdfs dfs -stat' to display access time. Contributed by Yongjun Zhang. 
(cherry picked from commit c6330f22a5)
 2017-07-27 17:05:30 -07:00
Brahma Reddy Battula 1c5095b3cb HDFS-11896. Non-dfsUsed will be doubled on dead node re-registration. Contributed by Brahma Reddy Battula. 
(cherry picked from commit c4a85c694f)
 2017-07-27 12:30:57 -07:00
Kihwal Lee 3d8c7d701b HDFS-12171. Reduce IIP object allocations for inode lookup. Contributed by Daryn Sharp. 2017-07-25 11:07:43 -05:00
Kihwal Lee 9fe5dd098d HDFS-12143. Improve performance of getting and removing inode features. Contributed by Daryn Sharp. 
(cherry picked from commit 1a79dcfc45)
 2017-07-25 10:30:27 -05:00
Akira Ajisaka c108372777
HDFS-12176. dfsadmin shows DFS Used%: NaN% if the cluster has zero block. Contributed by Weiwei Yang. 
(cherry picked from commit 770cc46228)
 2017-07-24 16:24:50 +09:00
Kihwal Lee c12bf9a128 HDFS-11742. Improve balancer usability after HDFS-8818. Contributed by Kihwal Lee 
(cherry picked from commit 8e3a992ecc)
 2017-07-21 09:19:01 -05:00
Konstantin V Shvachko af2fb6a909 HDFS-11472. Fix inconsistent replica size after a data pipeline failure. Contributed by Erik Krogen and Wei-Chiu Chuang. 
(cherry picked from commit 2a5a313539)
 2017-07-20 15:57:34 -07:00
Anu Engineer e0297ffbc8 HDFS-12158. Secondary Namenode's web interface lack configs for X-FRAME-OPTIONS protection. Contributed by Mukul Kumar Singh. 
(cherry picked from commit 413b23eb04)
 2017-07-19 10:42:28 -07:00
Brahma Reddy Battula 979d37ae21 HDFS-12067. Correct dfsadmin commands usage message to reflects IPC port. Contributed by steven-wugang. 
(cherry picked from commit f8cd55fe33)
 2017-07-19 23:22:40 +08:00
Tsz-Wo Nicholas Sze 02627f8c5f HDFS-12138. Remove redundant 'public' modifiers from BlockCollection. Contributed by Chen Liang 2017-07-17 13:59:52 -07:00
Brahma Reddy Battula a33e89931b HDFS-12112. TestBlockManager#testBlockManagerMachinesArray sometimes fails with NPE. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit b778887af5)
 2017-07-15 10:49:11 +08:00
Xiao Chen 5337c2511d HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 
(cherry picked from commit aa395c1ec9a170f2da732c6e6114c60412161d40)
 2017-07-14 22:23:07 -07:00
Tsz-Wo Nicholas Sze bff39ca718 HDFS-12130. Optimizing permission check for getContentSummary. Contributed by Chen Liang 2017-07-14 14:40:11 -07:00
Tsz-Wo Nicholas Sze d51395363a Revert "HDFS-12130. Optimizing permission check for getContentSummary." to fix commit message. 
This reverts commit 2ba41729ab.
 2017-07-14 14:38:27 -07:00
Kihwal Lee 0fd86de128 HDFS-12140. Remove BPOfferService lock contention to get block pool id. Contributed by Daryn Sharp. 
(cherry picked from commit e7d187a1b6)
 2017-07-14 16:12:28 -05:00
Kihwal Lee a47820d72a HDFS-12137. DN dataset lock should be fair. Contributed by Daryn Sharp. 2017-07-14 15:45:26 -05:00
Tsz-Wo Nicholas Sze 2ba41729ab HDFS-12130. Optimizing permission check for getContentSummary. 2017-07-14 13:42:02 -07:00
Xiaoyu Yao af8a227a68 HDFS-11502. Datanode UI should display hostname based on JMX bean instead of window.location.hostname. Contributed by Jeffrey E Rodriguez and Yuanbo Liu. 
(cherry picked from commit e15e2713e1)
 2017-07-12 16:38:56 -07:00
Akira Ajisaka 2653cf4466
HADOOP-14629. Improve exception checking in FileContext related JUnit tests. Contributed by Andras Bokor. 
(cherry picked from commit 9144fd9e9b)
 2017-07-12 11:40:48 +09:00
Akira Ajisaka 063f2132ef
HDFS-12078. Add time unit to the description of property dfs.namenode.stale.datanode.interval in hdfs-default.xml. Contributed by Weiwei Yang. 
(cherry picked from commit 186650d21d)
 2017-07-04 14:52:39 +09:00
Akira Ajisaka 003b4a1675
HDFS-12079. Description of dfs.block.invalidate.limit is incorrect in hdfs-default.xml. Contributed by Weiwei Yang. 
(cherry picked from commit b0560e0624)
 2017-07-04 14:03:07 +09:00
Wei-Chiu Chuang 94bc5cdbb3 HDFS-12042. Lazy initialize AbstractINodeDiffList#diffs for snapshots to reduce memory consumption. Contributed by Misha Dmitriev. 
(cherry picked from commit bcba844d11)
 2017-06-30 11:42:51 -07:00
Wei-Chiu Chuang 6917c19167 HDFS-11881. NameNode consumes a lot of memory for snapshot diff report generation. Contributed by Manoj Govindassamy. 
(cherry picked from commit 16c8dbde57)
 2017-06-29 06:40:24 -07:00
Akira Ajisaka a896c0f11e
HDFS-12040. TestFsDatasetImpl.testCleanShutdownOfVolume fails. Contributed by hu xiaodong. 
(cherry picked from commit 07defa4c09)
 2017-06-27 20:49:56 +09:00
Akira Ajisaka b59b96deee
HDFS-11992. Replace commons-logging APIs with slf4j in FsDatasetImpl. Contributed by hu xiaodong. 2017-06-27 03:28:09 +09:00
Kihwal Lee 8f533ca76b HDFS-11960. Successfully closed files can stay under-replicated. Contributed by Kihwal Lee. 2017-06-20 09:16:37 -05:00
Brahma Reddy Battula 933fad2e17 HADOOP-14533. Size of args cannot be less than zero in TraceAdmin#run as its linkedlist. Contributed by Weisen Han. 
(cherry picked from commit 2e9daa2e27)
 2017-06-20 20:45:33 +08:00
Akira Ajisaka 6b3c13d543
HDFS-11345. Document the configuration key for FSNamesystem lock fairness. Contributed by Erik Krogen. 
(cherry picked from commit 2c769167db)
 2017-06-20 16:36:47 +09:00
Akira Ajisaka 71626fd0b9
HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin. 2017-06-20 13:55:42 +09:00
Masatake Iwasaki b19ca498ae HDFS-11995. HDFS Architecture documentation incorrectly describes writing to a local temporary file. Contributed by Nandakumar. 
(cherry picked from commit d954a64730)
 2017-06-20 08:09:09 +09:00
Andrew Wang 1ec1aebd35 HDFS-10480. Add an admin command to list currently open files. Contributed by Manoj Govindassamy. 2017-06-19 09:52:07 -07:00
Lei Xu 33afa1fdca HADOOP-14395. Provide Builder pattern for DistributedFileSystem.append. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 6460df21a0)

    Conflicts:
    hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
 2017-06-16 17:32:29 -07:00
Arpit Agarwal 9529513f18 HADOOP-14503. Make RollingAverages a mutable metric. Contributed by Hanisha Koneru. 2017-06-16 16:41:08 -07:00
Lei Xu abac844c90 HADOOP-14394. Provide Builder pattern for DistributedFileSystem.create. (lei) 
(cherry picked from commit 5fbec46525d6d49837d934556b59ba77bd2301a8)

    Conflicts:
    hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/NameNodeConnector.java
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicies.java
 2017-06-15 10:59:11 -07:00
Akira Ajisaka 0fd0612896
Revert "HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin." 
This reverts commit 88fa17c238.
 2017-06-16 00:21:54 +09:00
Akira Ajisaka 88fa17c238
HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin. 
(cherry picked from commit 315f07700d)
 2017-06-15 13:47:22 +09:00
Tsz-Wo Nicholas Sze 670224ae59 HDFS-11947. When constructing a thread name, BPOfferService may print a bogus warning message. Contributed by Weiwei Yang 2017-06-13 09:47:14 +08:00
Arpit Agarwal eda4bf5196 HDFS-11967. TestJMXGet fails occasionally. Contributed by Arpit Agarwal. 2017-06-12 16:04:08 -07:00
Arpit Agarwal 097b1f74e5 HDFS-11907. Add metric for time taken by NameNode resource check. Contributed by Chen Liang. 2017-06-12 15:22:36 -07:00
Mingliang Liu facb6812d9 HDFS-11945. Internal lease recovery may not be retried for a long time. Contributed by Kihwal Lee 2017-06-08 14:46:31 -07:00
Brahma Reddy Battula fca08f8362 HDFS-11711. DN should not delete the block On "Too many open files" Exception. Contributed by Brahma Reddy Battula. 2017-06-07 16:54:29 +08:00
Yiqun Lin 72f0fdfb59 HDFS-11929. Document missing processor of hdfs oiv_legacy command. Contributed by LiXin Ge. 
(cherry picked from commit 5ec71639cf)
 2017-06-07 13:44:33 +08:00
Vinayakumar B 9a26120809 HDFS-11708. Positional read will fail if replicas moved to different DNs after stream is opened. Contributed by Vinayakumar B. 2017-06-07 10:57:47 +05:30
Andrew Wang 5c819585f2 HDFS-11840. Log HDFS Mover exception message of exit to its own log. Contributed by LiXin Ge. 
(cherry picked from commit c31cb879a3)
 2017-06-06 14:57:51 -07:00