Commit Graph

3827 Commits

Author SHA1 Message Date
Akira Ajisaka 2fe4225051 HDFS-3677. dfs.namenode.edits.dir.required is missing from hdfs-default.xml. Contributed by Mark Yang.
(cherry picked from commit 414f33126ac178a44146666cc7c00059cfc9cabf)
(cherry picked from commit 41621a994d)
2016-03-19 01:11:06 +09:00
Kihwal Lee 242c7f1fee HDFS-9874. Long living DataXceiver threads cause volume shutdown to block. Contributed by Rushabh Shah.
(cherry picked from commit 63c966a3fb)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
2016-03-18 10:33:13 -05:00
Colin Patrick Mccabe 73b5a44b0e HDFS-9949. Add a test case to ensure that the DataNode does not regenerate its UUID when a storage directory is cleared (Harsh J via cmccabe)
(cherry picked from commit dc951e606f)
(cherry picked from commit f69a6c3632)
2016-03-17 10:43:07 -07:00
Kihwal Lee fce39557d3 HDFS-9904. testCheckpointCancellationDuringUpload occasionally fails. Contributed by Lin Yiqun.
(cherry picked from commit d457401784)
2016-03-15 10:55:10 -05:00
Ravi Prakash 9e804d14c9 HDFS-9953. Download File from UI broken after pagination. Contributed by Brahma Reddy Battula
(cherry picked from commit 51b7d8935a)
2016-03-14 14:33:29 -07:00
Chris Nauroth e6c6f4ddac HDFS-9941. Do not log StandbyException on NN, other minor logging fixes. Contributed by Arpit Agarwal.
(cherry picked from commit 5644137ada)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockUnderConstructionFeature.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java

(cherry picked from commit dd462bc32f)
2016-03-14 09:55:34 -07:00
Colin Patrick Mccabe a1f110a145 HDFS-9933. ReverseXML should be capitalized in oiv usage message (cmccabe)
(cherry picked from commit 79961ecea8)
(cherry picked from commit 710811652c)
2016-03-10 13:45:24 -08:00
Colin Patrick Mccabe 56462678f9 HDFS-9934. ReverseXML oiv processor should bail out if the XML file's layoutVersion doesn't match oiv's (cmccabe)
(cherry picked from commit bd49354c6d)
(cherry picked from commit c82d1e1187)
2016-03-10 13:44:33 -08:00
Colin Patrick Mccabe 370a956196 HDFS-9927. Document the new OIV ReverseXML processor (Wei-Chiu Chuang via cmccabe)
(cherry picked from commit 500875dfcc)
(cherry picked from commit b8b580e6b8)
2016-03-10 13:39:43 -08:00
Colin Patrick Mccabe 56ab796094 HDFS-9835. OIV: add ReverseXML processor which reconstructs an fsimage from an XML file (cmccabe)
(cherry picked from commit 700b0e4019)
(cherry picked from commit 5998b20d63)
2016-03-09 16:38:15 -08:00
Allen Wittenauer 8b4a8ff1b2 Revert "HADOOP-12366. expose calculated paths (aw)"
This reverts commit 81a7ad45c9.
2016-03-09 09:59:33 -08:00
Chris Nauroth a3cd43990c HADOOP-12905. Clean up CHANGES.txt RAT exclusions from pom.xml files. Contributed by Chris Nauroth.
(cherry picked from commit 89b16d27e2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/pom.xml
	hadoop-mapreduce-project/pom.xml

(cherry picked from commit 38a031ce68)
2016-03-08 21:23:07 -08:00
Arpit Agarwal a63f1039f9 HDFS-9882. Add heartbeatsTotal in Datanode metrics. (Contributed by Hua Liu) 2016-03-07 21:10:42 -08:00
Akira Ajisaka d84d54e26c HDFS-9812. Streamer threads leak if failure happens when closing DFSOutputStream. Contributed by Lin Yiqun.
(cherry picked from commit 352d299cf8)
(cherry picked from commit fe0009a2bd)
2016-03-08 10:57:56 +09:00
Arpit Agarwal 02949b2151 HDFS-9906. Remove spammy log spew when a datanode is restarted. (Contributed by Brahma Reddy Battula) 2016-03-07 12:20:32 -08:00
Masatake Iwasaki 22f7730940 HDFS-9865. TestBlockReplacement fails intermittently in trunk (Lin Yiqun via iwasakims)
(cherry picked from commit d718fc1ee5)
2016-03-08 02:11:54 +09:00
Chris Nauroth d5134dadbc HDFS-9239. DataNode Lifeline Protocol: an alternative protocol for reporting DataNode liveness. Contributed by Chris Nauroth.
(cherry picked from commit 2759689d7d)
(cherry picked from commit 592f161e13)
2016-03-04 15:30:37 -08:00
Yongjun Zhang fb6661f3bd HDFS-9889. Update balancer/mover document about HDFS-6133 feature. Contributed by Yongjun Zhang.
(cherry picked from commit 8e08861a14)
2016-03-04 13:28:28 -08:00
Masatake Iwasaki 9d551a3adf HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims)
(cherry picked from commit cbd31328a6)

 Conflicts:
	hadoop-common-project/hadoop-common/src/site/markdown/CommandsManual.md
	hadoop-common-project/hadoop-common/src/site/markdown/UnixShellGuide.md

(cherry picked from commit 722182fdfb)
2016-03-04 14:13:30 +09:00
Andrew Wang c6e37f6782 HADOOP-11792. Remove all of the CHANGES.txt files. 2016-03-03 21:03:13 -08:00
Masatake Iwasaki 55f7ceb0db HDFS-9048. DistCp documentation is out-of-dated (Daisuke Kobayashi via iwasakims)
(cherry picked from commit 33a412e8a4)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2016-03-03 18:58:59 +09:00
Masatake Iwasaki 64213b5278 HADOOP-12793. Write a new group mapping service guide (Wei-Chiu Chuang via iwasakims)
(cherry picked from commit f70e317ccc)
2016-03-03 17:15:09 +09:00
Zhe Zhang 7dfe40a23b HDFS-9886. Configuration properties for hedged read is broken. Contributed by Akira AJISAKA. 2016-03-02 10:27:51 -08:00
Xiaoyu Yao 7b9a2bf278 HDFS-9887. WebHdfs socket timeouts should be configurable. Contributed by Austin Donnelly and Chris Douglas.
(cherry picked from commit 5abf051249)
2016-03-02 09:25:25 -08:00
Akira Ajisaka e3d5863677 HDFS-9851. NameNode throws NPE when setPermission is called on a path that does not exist. Contributed by Brahma Reddy Battula.
(cherry picked from commit 27e0681f28)
(cherry picked from commit 463ec6fa8c)
2016-03-02 14:44:04 +09:00
Akira Ajisaka b13bd04100 HDFS-9766. TestDataNodeMetrics#testDataNodeTimeSpend fails intermittently. Contributed by Xiao Chen.
(cherry picked from commit 81fbc40f084eab85750cc7d4cc967d43d70d8333)
(cherry picked from commit e840141726)
2016-03-02 14:14:52 +09:00
Andrew Wang 90f39b899c HDFS-9881. DistributedFileSystem#getTrashRoot returns incorrect path for encryption zones.
(cherry picked from commit 4abb2fa687)
(cherry picked from commit 81b7b9de27)
2016-03-01 16:35:46 -08:00
Kihwal Lee 2c8733a19d HDFS-9880. TestDatanodeRegistration fails occasionally. Contributed by Kihwal Lee.
(cherry picked from commit e76b13c415)
2016-03-01 17:02:08 -06:00
Kihwal Lee cc28118eed HDFS-8791. block ID-based DN storage layout can be very slow for datanode on ext4. Contributed by Chris Trezzo.
(cherry picked from commit b8dbffbef87c0ee0bee5e17649586fc9f0e1f55b)
2016-03-01 15:02:26 -06:00
Chris Nauroth 79ef70f861 HDFS-9870. Remove unused imports from DFSUtil. Contributed by Brahma Reddy Battula.
(cherry picked from commit 2137e8feeb)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 6659d13f12)
2016-03-01 11:16:29 -08:00
Akira Ajisaka 7ddff4b37a HDFS-9864. Correct reference for RENEWDELEGATIONTOKEN and CANCELDELEGATIONTOKEN in webhdfs doc. Contributed by Brahma Reddy Battula.
(cherry picked from commit 056f9013122dc3f0effc62d620261c312dd7f8ed)
(cherry picked from commit 6a9f8d95b7c4f9c66253d975ebbca1d186267d4e)
2016-02-29 18:42:48 +09:00
Tsz-Wo Nicholas Sze 5b95971f8a HDFS-9710. DN can be configured to send block receipt IBRs in batches. 2016-02-26 15:34:41 -08:00
Xiaoyu Yao fd9a0abba9 HDFS-9831. Document webhdfs retry configuration keys introduced by HDFS-5219/HDFS-5122. Contributed by Xiaobing Zhou.
(cherry picked from commit 1513c0179e)
2016-02-26 14:26:40 -08:00
Kihwal Lee e862fe7f02 HDFS-9855. Modify TestAuditLoggerWithCommands to workaround the absence of HDFS-8332. Contributed by Kuhu Shukla.
(cherry picked from commit 03afcdd39d)
2016-02-26 10:23:45 -06:00
Chris Nauroth 27edd164a0 HDFS-9843. Document distcp options required for copying between encrypted locations. Contributed by Xiaoyu Yao.
(cherry picked from commit dbbfc58c33)
(cherry picked from commit 4892f6a0eb)
2016-02-24 15:16:27 -08:00
cnauroth d5dfb278d6 HDFS-9854. Log cipher suite negotiation more verbosely. Contributed by Wei-Chiu Chuang.
(cherry picked from commit d1dd248b75)
(cherry picked from commit 59cd23e279)
2016-02-24 12:18:23 -08:00
Kihwal Lee 69acab5f23 HDFS-9395. Make HDFS audit logging consistant. Contributed by Kuhu Shukla.
(cherry picked from commit d27d7fc72e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java

(cherry picked from commit 83f7f62be3)
2016-02-24 12:12:41 -06:00
Zhe Zhang bab4fb55c9 HDFS-9844. Correct path creation in getTrashRoot to handle root dir.
Change-Id: I5afe6c7176063892e3e632698c849bd9f4b92906
2016-02-23 13:30:47 -08:00
Colin Patrick Mccabe 7812cfdd81 HDFS-9549. TestCacheDirectives#testExceedsCapacity is flaky (Xiao Chen via cmccabe)
(cherry picked from commit 211c78c090)
(cherry picked from commit 440379a0d4)
2016-02-23 12:06:16 -08:00
Tsuyoshi Ozawa 19a08eff6f HDFS-9084. Pagination, sorting and filtering of files/directories in the HDFS Web UI. (Contributed by Ravi Prakash)
(cherry picked from commit 48c61cd6aa)
2016-02-23 19:31:33 +09:00
Tsz-Wo Nicholas Sze 82cd1580d0 HDFS-8578. On upgrade, Datanode should process all storage/data dirs in parallel. Contributed by vinayakumarb and szetszwo 2016-02-22 15:16:59 -08:00
Zhe Zhang 6d4a4e785b HADOOP-12668. Support excluding weak Ciphers in HttpServer2 through ssl-server.conf. Contributed by Vijay Singh.
Change-Id: Ie46a5427d29188935427f67e55203c19fcd83335
2016-02-22 14:15:11 -08:00
Arpit Agarwal 08666fb87e HDFS-9842. dfs.datanode.balance.bandwidthPerSec should accept friendly size units. (Contributed by Lin Yiqun) 2016-02-22 11:34:26 -08:00
Vinayakumar B 426f213088 HDFS-9425. Expose number of blocks per volume as a metric (Contributed by Brahma Reddy Battula)
(cherry picked from commit 342c9572bf)
(cherry picked from commit 2e3c35a835)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
2016-02-22 09:31:52 +05:30
Vinayakumar B 8ca8d218a4 HDFS-7452. skip StandbyException log for getCorruptFiles() (Contributed by Bramha Reddy Battula)
(cherry picked from commit f313516731)
(cherry picked from commit 3e76768f21)
2016-02-22 09:22:45 +05:30
Arpit Agarwal 4823fbb490 HDFS-9839. Reduce verbosity of processReport logging. (Contributed by Arpit Agarwal)
This closes #78
2016-02-20 23:20:28 -08:00
Yongjun Zhang 5a5f9ca1f0 HDFS-9764. DistCp doesn't print value for several arguments including -numListstatusThreads. (Wei-Chiu Chuang via Yongjun Zhang)
(cherry picked from commit ba1c9d484a)
2016-02-19 10:29:46 -08:00
Tsz-Wo Nicholas Sze b30d71d450 Move HDFS-8923 to Release 2.8.0 in CHANGES.txt. 2016-02-18 18:44:23 -08:00
Tsz-Wo Nicholas Sze 11c6930a17 HDFS-8923. Add -source flag to balancer usage message. Contributed by Chris Trezzo 2016-02-18 18:39:45 -08:00
Tsz-Wo Nicholas Sze 207ed9601f HADOOP-10865. Add a Crc32 chunked verification benchmark for both directly and non-directly buffer cases. 2016-02-18 11:59:58 -08:00