Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,475 Commits 326 Branches 371 Tags
Commit Graph

2792 Commits

Author SHA1 Message Date
Tsz-Wo Nicholas Sze
c52636df3f recommit "HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir." 
(cherry picked from commit acc172e3718e23ff6808ddcc01543212f1334a27)
 2015-02-27 14:20:26 -08:00
Akira Ajisaka
0f9289e848 HDFS-7685. Document dfs.namenode.heartbeat.recheck-interval in hdfs-default.xml. Contributed by Kai Sasaki. 
(cherry picked from commit 8719cdd4f68abb91bf9459bca2a5467dafb6b5ae)
 2015-02-27 12:18:46 -08:00
Tsz-Wo Nicholas Sze
657b027bb2 HDFS-7308. Change the packet chunk size computation in DFSOutputStream in order to enforce packet size <= 64kB. Contributed by Takuya Fukudome 2015-02-27 23:46:42 +08:00
Vinayakumar B
bc60404eaf HDFS-6753. Initialize checkDisk when DirectoryScanner not able to get files list for scanning (Contributed by J.Andreina) 
(cherry picked from commit 4f75b15628a76881efc39054612dc128e23d27be)
 2015-02-27 16:37:03 +05:30
Tsz-Wo Nicholas Sze
d223a4a594 Revert "HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir." 
This reverts commit acc172e3718e23ff6808ddcc01543212f1334a27.

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-02-27 18:27:19 +08:00
cnauroth
5dfb793b05 HDFS-7774. Unresolved symbols error while compiling HDFS on Windows 7/32 bit. Contributed by Kiran Kumar M R. 
(cherry picked from commit c6d5b37a1c4c3acc190f5f4e27109594efb7be8c)
 2015-02-26 13:37:58 -08:00
Colin Patrick Mccabe
8b3b9568b6 HDFS-7819. Log WARN message for the blocks which are not in Block ID based layout (Rakesh R via Colin P. McCabe) 
(cherry picked from commit f0c980abed3843923e0eb16b626fa27334195eda)
 2015-02-26 12:03:11 -08:00
Vinayakumar B
f0f3d1937d HDFS-7832. Show 'Last Modified' in Namenode's 'Browse Filesystem' (Contributed by Vinayakumar B) 
(cherry picked from commit 166eecf687765f369cd5bf210cc6d7bf493121ea)
 2015-02-26 14:37:03 +05:30
Tsz-Wo Nicholas Sze
22b13a0ddb HDFS-7537. Add "UNDER MIN REPL'D BLOCKS" count to fsck. Contributed by GAO Rui 2015-02-26 11:47:04 +08:00
Tsz-Wo Nicholas Sze
12fe3afcd3 HDFS-7843. A truncated file is corrupted after rollback from a rolling upgrade. 2015-02-26 10:15:49 +08:00
Benoy Antony
4e400030f6 HDFS-7467. Provide storage tier information for a directory via fsck. (Benoy Antony) 2015-02-25 16:46:41 -08:00
Colin Patrick Mccabe
d389a1ae98 HDFS-7495. Remove updatePosition argument from DFSInputStream#getBlockAt() (cmccabe) 
(cherry picked from commit caa42adf208bfb5625d1b3ef665fbf334ffcccd9)
 2015-02-25 13:39:02 -08:00
Colin Patrick Mccabe
5c0073e706 HDFS-7668. Backport "Convert site documentation from apt to markdown" to branch-2 (Masatake Iwasaki via Colin P. McCabe) 
(backported from commit 2f1e5dc6288972004b5bed335c4a8d038aaedcf4)

Conflicts:
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ExtendedAttributes.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/Federation.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSCommands.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSHighAvailabilityWithNFS.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSHighAvailabilityWithQJM.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsDesign.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsNfsGateway.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsQuotaAdminGuide.apt.vm
 2015-02-24 16:18:18 -08:00
Andrew Wang
efb7e287f4 HDFS-7763. fix zkfc hung issue due to not catching exception in a corner case. Contributed by Liang Xie. 
(cherry picked from commit 7105ebaa9f370db04962a1e19a67073dc080433b)
 2015-02-24 15:31:18 -08:00
Jing Zhao
8346427929 HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. 
(cherry picked from commit 73bcfa99af61e5202f030510db8954c17cba43cc)
 2015-02-24 10:34:42 -08:00
Tsuyoshi Ozawa
280dda5bcd HDFS-7008. xlator should be closed upon exit from DFSAdmin#genericRefresh(). (ozawa) 
(cherry picked from commit b53fd7163bc3a4eef4632afb55e5513c7c592fcf)
 2015-02-24 23:59:59 +09:00
Colin Patrick Mccabe
453b210d0a HDFS-7805. NameNode recovery prompt should be printed on console (Surendra Singh Lilhore via Colin P. McCabe) 
(cherry picked from commit faaddb6ecb44cdc9ef82a2ab392f64fc2561e938)
 2015-02-23 19:06:01 -08:00
Colin Patrick Mccabe
f4da6f7193 HDFS-7807. libhdfs htable.c: fix htable resizing, add unit test (cmccabe) 
(cherry picked from commit 585768667e443f56c2f97068276ec8768dc49cf8)
 2015-02-23 18:45:44 -08:00
cnauroth
657a6e389b HDFS-7009. Active NN and standby NN have different live nodes. Contributed by Ming Ma. 
(cherry picked from commit 769507bd7a501929d9a2fd56c72c3f50673488a4)
 2015-02-23 15:12:36 -08:00
Arpit Agarwal
83f779e49a HDFS-7806. Refactor: move StorageType from hadoop-hdfs to hadoop-common. (Contributed by Xiaoyu Yao) 2015-02-21 15:48:59 -08:00
yliu
eca1588db8 HDFS-7740. Test truncate with DataNodes restarting. (yliu) 2015-02-21 06:33:58 +08:00
cnauroth
9f06fb3aef HDFS-7813. TestDFSHAAdminMiniCluster#testFencer testcase is failing frequently. Contributed by Rakesh R. 
(cherry picked from commit 0d6af574e0056fc627461eb91ed0c365b026b470)
 2015-02-20 17:01:22 -08:00
cnauroth
e09ba94f28 HDFS-7773. Additional metrics in HDFS to be accessed via jmx. Contributed by Anu Engineer. 2015-02-20 12:38:34 -08:00
cnauroth
ad77dfc3e4 HDFS-7814. Fix usage string of storageType parameter for "dfsadmin -setSpaceQuota/clrSpaceQuota". Contributed by Xiaoyu Yao. 
(cherry picked from commit 8c6ae0d6199efa327d8f2761f2ad2163a60e5508)
 2015-02-20 12:22:00 -08:00
Kihwal Lee
b9157f92fc HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. 
(cherry picked from commit 7ae5255a1613ccfb43646f33eabacf1062c86e93)
 2015-02-20 09:09:56 -06:00
Harsh J
5c2c6b00dd HDFS-7752. Improve description for "dfs.namenode.num.extra.edits.retained" and "dfs.namenode.num.checkpoints.retained" properties on hdfs-default.xml. Contributed by Wellington Chevreuil. 
(cherry picked from commit b9a17909ba39898120a096cb6ae90104640690db)
 2015-02-20 19:21:34 +05:30
yliu
fee29e4a4f HDFS-7656. Expose truncate API for HDFS httpfs. (yliu) 2015-02-19 08:38:25 +08:00
Haohui Mai
3605928bd0 HDFS-7808. Remove obsolete -ns options in in DFSHAAdmin.java. Contributed by Arshad Mohammad. 2015-02-18 15:14:53 -08:00
cnauroth
b1b1e05e18 HDFS-7772. Document hdfs balancer -exclude/-include option in HDFSCommands.html. Contributed by Xiaoyu Yao. 2015-02-18 12:03:00 -08:00
Akira Ajisaka
6d6c684567 HDFS-7780. Update use of Iterator to Iterable in DataXceiverServer and SnapshotDiffInfo. Contributed by Ray Chiang. 
(cherry picked from commit 6dc8812a95bf369ec1f2e3d8a9473033172736cd)
 2015-02-17 14:49:03 -08:00
Haohui Mai
f92a4904bf HDFS-6662. WebHDFS cannot open a file if its path contains "%". Contributed by Gerson Carlos. 2015-02-17 13:05:00 -08:00
Kihwal Lee
8b37b4a78b HDFS-7795. Show warning if not all favored nodes were chosen by namenode. Contributed by Kihwal Lee. 
(cherry picked from commit db6606223ca2e17aa7e1b2e2be13c1a19d8e7465)
 2015-02-17 13:06:41 -06:00
Uma Maheswara Rao G
2cbac36fd3 HDFS-4266. BKJM: Separate write and ack quorum (Rakesh R via umamahesh) 
(cherry picked from commit f0412de1c1d42b3c2a92531f81d97a24df920523)
 2015-02-17 21:31:43 +05:30
Uma Maheswara Rao G
35fecb5306 HDFS-7797. Add audit log for setQuota operation (Rakesh R via umamahesh) 
(cherry picked from commit f24a56787a15e89a7c1e777b8043ab9ae8792505)
 2015-02-17 20:16:34 +05:30
Tsuyoshi Ozawa
00fb0710b6 HADOOP-11600. Fix up source codes to be compiled with Guava 17.0. (ozawa) 
(cherry picked from commit 2f0f756b26ea83e142a5b9379fa75862c2fc6ad5)
 2015-02-17 21:57:58 +09:00
Vinayakumar B
91a5d92916 HDFS-7703. Support favouredNodes for the append for new blocks ( Contributed by Vinayakumar B) 
(cherry picked from commit 89a544928083501625bc69f96b530040228f0a5f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-02-17 15:19:03 +05:30
yliu
54e33baaf6 HDFS-7798. Checkpointing failure caused by shared KerberosAuthenticator. (Chengbing Liu via yliu) 2015-02-17 07:50:14 +08:00
cnauroth
441dfa4867 HDFS-7604. Track and display failed DataNode storage locations in NameNode. Contributed by Chris Nauroth. 
(cherry picked from commit 9729b244de50322c2cc889c97c2ffb2b4675cf77)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HeartbeatManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
 2015-02-16 14:44:49 -08:00
Arpit Agarwal
c6aa6cbfe5 HDFS-7775. Use consistent naming for NN-internal quota related types and functions. (Contributed bu Xiaoyu Yao) 2015-02-13 21:01:54 -08:00
cnauroth
751b834ad1 HDFS-7778. Rename FsVolumeListTest to TestFsVolumeList and commit it to branch-2. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 2efb2347a969ecff75934cd10f2432eade1d77dc)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsVolumeList.java
 2015-02-13 16:30:53 -08:00
Akira Ajisaka
3ad1d8d533 HDFS-4265. BKJM doesn't take advantage of speculative reads. Contributed by Rakesh R. 
(cherry picked from commit 0d521e33262193e6cf709deaa69a54811a97ef6a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-02-13 15:25:13 -08:00
Colin Patrick Mccabe
7b9c978df2 HDFS-7686. Re-add rapid rescan of possibly corrupt block feature to the block scanner (cmccabe) 
(cherry picked from commit 8bb9a5000ed06856abbad268c43ce1d5ad5bdd43)
 2015-02-13 15:19:04 -08:00
Colin Patrick Mccabe
6edcb9a503 HDFS-7721. The HDFS BlockScanner may run fast during the first hour (cmccabe) 
(cherry picked from commit 115428176e1d919fe7d54d01b34dfda57d1b3950)
 2015-02-13 15:18:10 -08:00
Colin Patrick Mccabe
a54b4a7fc6 HDFS-7430 backport addendum. Add missing change. (cmccabe) 2015-02-13 15:11:14 -08:00
Colin Patrick Mccabe
a8c6a96f3b HDFS-7430. Refactor the BlockScanner to use O(1) memory and use multiple threads (cmccabe) 
(cherry picked from commit 6e62a1a6728b1f782f64065424f92b292c3f163a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsDatasetSpi.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeListTest.java
 2015-02-13 15:04:06 -08:00
Arpit Agarwal
f507791554 HDFS-7776. Adding additional unit tests for Quota By Storage Type. (Contributed by Xiaoyu Yao) 2015-02-13 12:02:54 -08:00
Allen Wittenauer
730597c20c HDFS-316. Balancer should run for a configurable # of iterations (Xiaoyu Yao via aw) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSCommands.apt.vm
 2015-02-12 18:02:23 -08:00
Colin Patrick Mccabe
a9a361c549 HDFS-7790. Do not create optional fields in DFSInputStream unless they are needed (cmccabe) 
(cherry picked from commit 871cb56152e6039ff56c6fabfcd45451029471c3)
 2015-02-12 17:49:11 -08:00
Akira Ajisaka
5696376e29 HDFS-7684. The host:port settings of the deamons should be trimmed before use. Contributed by Anu Engineer. 
(cherry picked from commit 9235e2fb4e94a86671e1494f293c973cf572281c)
 2015-02-12 17:40:48 -08:00
Arpit Agarwal
676fc2d1dc HDFS-7723. Quota By Storage Type namenode implemenation. (Contributed by Xiaoyu Yao) 2015-02-12 10:50:14 -08:00