Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,286 Commits 326 Branches 371 Tags
Commit Graph

4583 Commits

Author SHA1 Message Date
Akira Ajisaka
063f2132ef
HDFS-12078. Add time unit to the description of property dfs.namenode.stale.datanode.interval in hdfs-default.xml. Contributed by Weiwei Yang. 
(cherry picked from commit 186650d21d482e2f3bc4523ae989ebe76081b0e3)
 2017-07-04 14:52:39 +09:00
Akira Ajisaka
003b4a1675
HDFS-12079. Description of dfs.block.invalidate.limit is incorrect in hdfs-default.xml. Contributed by Weiwei Yang. 
(cherry picked from commit b0560e0624756e2b3ce7b6bc741eee3c18d2a873)
 2017-07-04 14:03:07 +09:00
Wei-Chiu Chuang
94bc5cdbb3 HDFS-12042. Lazy initialize AbstractINodeDiffList#diffs for snapshots to reduce memory consumption. Contributed by Misha Dmitriev. 
(cherry picked from commit bcba844d1144cc334e2babbc34c9d42eac1c203a)
 2017-06-30 11:42:51 -07:00
Wei-Chiu Chuang
6917c19167 HDFS-11881. NameNode consumes a lot of memory for snapshot diff report generation. Contributed by Manoj Govindassamy. 
(cherry picked from commit 16c8dbde574f49827fde5ee9add1861ee65d4645)
 2017-06-29 06:40:24 -07:00
Akira Ajisaka
a896c0f11e
HDFS-12040. TestFsDatasetImpl.testCleanShutdownOfVolume fails. Contributed by hu xiaodong. 
(cherry picked from commit 07defa4c09db6e8c552167019ca0d2444cfc8fe7)
 2017-06-27 20:49:56 +09:00
Ravi Prakash
ef491f4637 HDFS-11993. Add log info when connect to datanode socket address failed. Contributed by chencan 
(cherry picked from commit a9d3412b4ce40f5ab5a18756ede7e0606b653171)
 2017-06-26 13:25:32 -07:00
Akira Ajisaka
b59b96deee
HDFS-11992. Replace commons-logging APIs with slf4j in FsDatasetImpl. Contributed by hu xiaodong. 2017-06-27 03:28:09 +09:00
Kihwal Lee
8f533ca76b HDFS-11960. Successfully closed files can stay under-replicated. Contributed by Kihwal Lee. 2017-06-20 09:16:37 -05:00
Brahma Reddy Battula
933fad2e17 HADOOP-14533. Size of args cannot be less than zero in TraceAdmin#run as its linkedlist. Contributed by Weisen Han. 
(cherry picked from commit 2e9daa2e27536484846d9c5f020c6afaf27932ce)
 2017-06-20 20:45:33 +08:00
Akira Ajisaka
6b3c13d543
HDFS-11345. Document the configuration key for FSNamesystem lock fairness. Contributed by Erik Krogen. 
(cherry picked from commit 2c769167dbdb66c52d2ba7b7193a686444085570)
 2017-06-20 16:36:47 +09:00
Akira Ajisaka
71626fd0b9
HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin. 2017-06-20 13:55:42 +09:00
Masatake Iwasaki
b19ca498ae HDFS-11995. HDFS Architecture documentation incorrectly describes writing to a local temporary file. Contributed by Nandakumar. 
(cherry picked from commit d954a64730c00346476322743462cde857164177)
 2017-06-20 08:09:09 +09:00
Andrew Wang
1ec1aebd35 HDFS-10480. Add an admin command to list currently open files. Contributed by Manoj Govindassamy. 2017-06-19 09:52:07 -07:00
Lei Xu
33afa1fdca HADOOP-14395. Provide Builder pattern for DistributedFileSystem.append. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 6460df21a09a7fcc29eceb8dc3859d6298da6882)

    Conflicts:
    hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
 2017-06-16 17:32:29 -07:00
Arpit Agarwal
9529513f18 HADOOP-14503. Make RollingAverages a mutable metric. Contributed by Hanisha Koneru. 2017-06-16 16:41:08 -07:00
Xiao Chen
2afe9722af HADOOP-14524. Make CryptoCodec Closeable so it can be cleaned up proactively. 2017-06-16 09:49:41 -07:00
Lei Xu
abac844c90 HADOOP-14394. Provide Builder pattern for DistributedFileSystem.create. (lei) 
(cherry picked from commit 5fbec46525d6d49837d934556b59ba77bd2301a8)

    Conflicts:
    hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/NameNodeConnector.java
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicies.java
 2017-06-15 10:59:11 -07:00
Akira Ajisaka
0fd0612896
Revert "HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin." 
This reverts commit 88fa17c2383baafbfb6b43b69e08cf40493c774a.
 2017-06-16 00:21:54 +09:00
Akira Ajisaka
88fa17c238
HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin. 
(cherry picked from commit 315f07700dcfe32513b87f63070db854ecb3cf48)
 2017-06-15 13:47:22 +09:00
Tsz-Wo Nicholas Sze
670224ae59 HDFS-11947. When constructing a thread name, BPOfferService may print a bogus warning message. Contributed by Weiwei Yang 2017-06-13 09:47:14 +08:00
Arpit Agarwal
eda4bf5196 HDFS-11967. TestJMXGet fails occasionally. Contributed by Arpit Agarwal. 2017-06-12 16:04:08 -07:00
Arpit Agarwal
097b1f74e5 HDFS-11907. Add metric for time taken by NameNode resource check. Contributed by Chen Liang. 2017-06-12 15:22:36 -07:00
Mingliang Liu
facb6812d9 HDFS-11945. Internal lease recovery may not be retried for a long time. Contributed by Kihwal Lee 2017-06-08 14:46:31 -07:00
Brahma Reddy Battula
fca08f8362 HDFS-11711. DN should not delete the block On "Too many open files" Exception. Contributed by Brahma Reddy Battula. 2017-06-07 16:54:29 +08:00
Yiqun Lin
72f0fdfb59 HDFS-11929. Document missing processor of hdfs oiv_legacy command. Contributed by LiXin Ge. 
(cherry picked from commit 5ec71639cf41858c687781445b8360d50cad5b55)
 2017-06-07 13:44:33 +08:00
Vinayakumar B
9a26120809 HDFS-11708. Positional read will fail if replicas moved to different DNs after stream is opened. Contributed by Vinayakumar B. 2017-06-07 10:57:47 +05:30
Andrew Wang
5c819585f2 HDFS-11840. Log HDFS Mover exception message of exit to its own log. Contributed by LiXin Ge. 
(cherry picked from commit c31cb879a3595f76d01408e6d066a0bde20be043)
 2017-06-06 14:57:51 -07:00
Arpit Agarwal
226688a12c HDFS-11932. BPServiceActor thread name is not correctly set. Contributed by Chen Liang. 2017-06-06 13:51:25 -07:00
Yongjun Zhang
1e52314117 HDFS-11914. Add more diagnosis info for fsimage transfer failure. Contributed by Yongjun Zhang. 
(cherry picked from commit 6a28a2b914664fbbb5267f13e82d42b3bb49610c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java
 2017-06-06 12:31:11 -07:00
Kihwal Lee
e2ea601231 HDFS-10816. TestComputeInvalidateWork#testDatanodeReRegistration fails due to race between test and replication monitor. Contributed by Eric Badger. 2017-06-05 15:24:54 -05:00
Arpit Agarwal
baff06f019 HDFS-11928. Segment overflow in FileDistributionCalculator. Contributed by LiXin Ge. 2017-06-05 13:21:35 -07:00
Yiqun Lin
2e8557de12 HDFS-11899. ASF License warnings generated intermittently in trunk. Contributed by Yiqun Lin. 
(cherry picked from commit 367da9b7b8837d2ffa30a7d2716e463adf2d7007)
 2017-06-03 22:10:01 +08:00
Brahma Reddy Battula
8e119f1e90 HADOOP-14436. Remove the redundant colon in ViewFs.md. Contributed by maobaolong. 
(cherry picked from commit 056cc72885471d6952ff182670e4b4a38421603d)
 2017-06-02 22:42:23 +08:00
Brahma Reddy Battula
ac2af66e77 HDFS-11905. Fix license header inconsistency in hdfs. Contributed by Yeliang Cang. 2017-06-02 22:22:43 +08:00
Yiqun Lin
870b5e39d2 HDFS-11359. DFSAdmin report command supports displaying maintenance state datanodes. Contributed by Yiqun Lin. 2017-06-02 12:53:31 +08:00
Andrew Wang
d0a4e7e95d HDFS-11383. Intern strings in BlockLocation and ExtendedBlock. Contributed by Misha Dmitriev. 
(cherry picked from commit 7101477e4726a70ab0eab57c2d4480a04446a8dc)
 2017-06-01 15:20:22 -07:00
Xiao Chen
eb98773ac8 HDFS-11904. Reuse iip in unprotectedRemoveXAttrs calls. 
(cherry picked from commit 219f4c199e45f8ce7f41192493bf0dc8f1e5dc30)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java
 2017-06-01 14:16:58 -07:00
Xiao Chen
91b72a8aa5 HDFS-11741. Long running balancer may fail due to expired DataEncryptionKey. Contributed by Wei-Chiu Chuang and Xiao Chen. 
(cherry picked from commit 068e23b896c63b0f817e6b91d73c994be1551eb2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockTokenSecretManager.java
 2017-06-01 14:05:43 -07:00
Brahma Reddy Battula
5cbdefbcc0 HDFS-11893. Fix TestDFSShell.testMoveWithTargetPortEmpty failure. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0dcf843c008f2b9cece8c0a0cef78140398ac464)
 2017-06-01 22:39:22 +08:00
Kihwal Lee
212a566088 HDFS-11856. Ability to re-add Upgrading nodes to pipeline for future pipeline updates. Contributed by Vinayakumar B. 2017-05-31 12:45:33 -05:00
Kihwal Lee
5b0baeab5e HDFS-5042. Completed files lost after power failure. Contributed by Vinayakumar B. 2017-05-31 11:02:34 -05:00
Lei Xu
a3ad1a39c5 HDFS-11659. TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten fail due to no DataNode available for pipeline recovery. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 91d6fe151f2e3de21b0a9423ade921e771957d90)
 2017-05-30 11:11:00 -07:00
Yiqun Lin
d36e64b8b2 HDFS-11446. TestMaintenanceState#testWithNNAndDNRestart fails intermittently. Contributed by Yiqun Lin. 
(cherry picked from commit 31058b243e9575d90f038bb2fdf5a556710f6f7f)
 2017-05-28 11:27:09 +08:00
Inigo Goiri
e336d92b99 Fix NPE in LazyPersistFileScrubber. Contributed by Inigo Goiri. 
(cherry picked from commit 303c8dc9b6c853c0939ea9ba14388897cc258071)
 2017-05-26 13:40:16 -07:00
Xiao Chen
54971c4195 HDFS-11421. Make WebHDFS' ACLs RegEx configurable. Contributed by Harsh J. 2017-05-25 21:01:39 -07:00
Kihwal Lee
2cb63433ab HDFS-11817. A faulty node can cause a lease leak and NPE on accessing data. Contributed by Kihwal Lee. 2017-05-25 17:21:56 -05:00
Brahma Reddy Battula
7bad748091 HDFS-11445. FSCK shows overall health stauts as corrupt even one replica is corrupt. Contributed by Brahma Reddy Battula. 2017-05-25 22:38:12 +08:00
Yongjun Zhang
49dc547252 HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li. 2017-05-24 18:54:51 -07:00
Yongjun Zhang
97317b1aaa Revert "HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li." 
This reverts commit c411adac32cda42433b15979253248336487d977.
 2017-05-24 18:54:51 -07:00
Wei-Chiu Chuang
bd0138ea0a Revert "HDFS-10797. Disk usage summary of snapshots causes renamed blocks to get counted twice. Contributed by Sean Mackrory." 
This reverts commit 5c96ef365de1a7ffc6a13f2d64cda9a26d59e4aa.
 2017-05-24 18:08:30 -07:00