Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,618 Commits 326 Branches 371 Tags
Commit Graph

2835 Commits

Author SHA1 Message Date
Plamen Jeliazkov
da318cbbde HDFS-7956. Improve logging for DatanodeRegistration. Contributed by Plamen Jeliazkov. 2015-03-23 23:09:52 -07:00
Andrew Wang
af0af28afc HDFS-7960. The full block report should prune zombie storages even if they're not empty. Contributed by Colin McCabe and Eddy Xu. 
(cherry picked from commit 50ee8f4e67a66aa77c5359182f61f3e951844db6)
 2015-03-23 22:01:37 -07:00
Tsz-Wo Nicholas Sze
87079cde7d HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula 2015-03-24 13:52:18 +09:00
cnauroth
8e1c33e703 HDFS-7917. Use file to replace data dirs in test to simulate a disk failure. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 2c238ae4e00371ef76582b007bb0e20ac8455d9c)
(cherry picked from commit 01c0bcb176e22ddefbc8086e382dd1ebd105f9c6)
 2015-03-23 16:30:48 -07:00
Akira Ajisaka
2742f12b58 HDFS-7881. TestHftpFileSystem#testSeek fails in branch-2. Contributed by Brahma Reddy Battula. 
(cherry picked from commit fad8c78173c4b7c55324033720f04a09943deac7)
 2015-03-24 06:25:21 +09:00
Brandon Li
6b9f2d9f39 HDFS-7942. NFS: support regexp grouping in nfs.exports.allowed.hosts. Contributed by Brandon Li 
(cherry picked from commit 36af4a913c97113bd0486c48e1cb864c5cba46fd)
(cherry picked from commit 503d8e4164ff3da29fcaf56436fe6fab6a450105)
 2015-03-23 10:13:11 -07:00
Kihwal Lee
08f2f7ed34 HDFS-6841. Use Time.monotonicNow() wherever applicable instead of Time.now(). Contributed by Vinayakumar B 
(cherry picked from commit 99a8dcd19528b265d4fda9ae09a17e4af52f2782)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfo.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
(cherry picked from commit 63b7870aec007081c82427587cb6e1d38e2c70f8)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2015-03-20 14:06:02 -05:00
Jing Zhao
03fb9b4ce0 HDFS-7957. Truncate should verify quota before making changes. Contributed by Jing Zhao. 
(cherry picked from commit d368d3647a858644b9fcd3be33d9fea2a6962f69)
(cherry picked from commit 6be52e42a9eb2069cedb12a8311bc36131a3804f)
 2015-03-20 11:56:39 -07:00
yliu
faeb8b9b53 HDFS-7962. Remove duplicated logs in BlockManager. (yliu) 2015-03-20 05:32:19 +08:00
yliu
0da3fcb4b0 HDFS-7930. commitBlockSynchronization() does not remove locations. (yliu) 2015-03-19 23:26:26 +08:00
Kihwal Lee
4e967d0bcd HDFS-7932. Speed up the shutdown of datanode during rolling upgrade. Contributed by Kihwal Lee. 
(cherry picked from commit 61a4c7fc9891def0e85edf7e41d74c6b92c85fdb)
 2015-03-19 12:29:16 -05:00
Vinayakumar B
76ce2e8ea0 HDFS.7849. Update documentation for enabling a new feature in rolling upgrade ( Contributed by J.Andreina ) 
(cherry picked from commit 5be5129d93481e9caae6a05b2361c463211a6d55)
 2015-03-19 19:10:23 +05:30
Kihwal Lee
b8f269af9d HDFS-7816. Unable to open webhdfs paths with "+". Contributed by Haohui Mai 
(cherry picked from commit e79be0ee123d05104eb34eb854afcf9fa78baef2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
(cherry picked from commit ceb39c1cc6b52178172c879d96cd743e0dc7a650)
 2015-03-19 08:04:19 -05:00
Colin Patrick Mccabe
9e61835678 HDFS-7929. inotify unable fetch pre-upgrade edit log segments once upgrade starts (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit 43b41f22411439c5e23629197fb2fde45dcf0f0f)
(cherry picked from commit 219eb22c1571f76df32967a930049d983cbf5024)
 2015-03-18 19:08:53 -07:00
Jing Zhao
0bf549728d HDFS-7587. Edit log corruption can happen if append fails with a quota violation. Contributed by Jing Zhao. 
(cherry picked from commit c7c71cdba50cb7d8282622cd496cc913c80cff54)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 5a5b2446485531f12d37f3d4ca791672b9921872)
 2015-03-18 19:00:06 -07:00
Jing Zhao
4b8de154aa HDFS-7943. Append cannot handle the last block with length greater than the preferred block size. Contributed by Jing Zhao. 
(cherry picked from commit bee5a6a64a1c037308fa4d52249be39c82791590)
(cherry picked from commit 6dcc79507d70ff70a1efee6475795430f519d66b)
 2015-03-18 18:47:21 -07:00
Colin Patrick Mccabe
94976cb369 HDFS-7054. Make DFSOutputStream tracing more fine-grained (cmccabe) 
(cherry picked from commit 8234fd0e1087e0e49aa1d6f286f292b7f70b368e)
(cherry picked from commit 79c07bbacae94bb0aae476dfbc17ae38564e2028)
 2015-03-18 18:15:34 -07:00
Haohui Mai
388696c089 HDFS-7945. The WebHdfs system on DN does not honor the length parameter. Contributed by Haohui Mai. 2015-03-18 16:20:07 -07:00
Arpit Agarwal
bb986988d4 HDFS-7914. TestJournalNode#testFailToStartWithBadConfig fails when the default dfs.journalnode.http-address port 8480 is in use. (Contributed by Xiaoyu Yao) 2015-03-18 14:05:37 -07:00
Haohui Mai
482aeee784 HDFS-7697. Mark the PB OIV tool as experimental. Contributed by Lei (Eddy) Xu. 2015-03-18 13:34:21 -07:00
Arpit Agarwal
5f7926dcf0 Fix CHANGES.txt for HDFS-7722. 2015-03-18 12:53:02 -07:00
Arpit Agarwal
2b589da779 HDFS-7951. Fix NPE for TestFsDatasetImpl#testAddVolumeFailureReleasesInUseLock on Linux. (Contributed by Xiaoyu Yao) 2015-03-18 12:46:17 -07:00
Arpit Agarwal
ae6d273d63 HDFS-7950. Fix TestFsDatasetImpl#testAddVolumes failure on Windows. (Contributed by Xiaoyu Yao) 2015-03-18 12:35:41 -07:00
Arpit Agarwal
8944bb3d2e HDFS-7948. TestDataNodeHotSwapVolumes#testAddVolumeFailures failed on Windows. (Contributed by Xiaoyu Yao) 2015-03-18 12:26:22 -07:00
Haohui Mai
5293ae0311 HDFS-7953. NN Web UI fails to navigate to paths that contain #. Contributed by kanaka kumar avvaru. 2015-03-18 11:08:49 -07:00
Arpit Agarwal
26c35438f3 HDFS-7946. TestDataNodeVolumeFailureReporting NPE on Windows. (Contributed by Xiaoyu Yao) 2015-03-17 21:30:13 -07:00
Colin Patrick Mccabe
d2dad74421 HDFS-7940. Add tracing to DFSClient#setQuotaByStorageType (Rakesh R via Colin P. McCabe) 
(cherry picked from commit d8846707c58c5c3ec542128df13a82ddc05fb347)
(cherry picked from commit 455d4aa8a12920fccad1bcde715f6fb6d9a63561)
 2015-03-17 11:02:05 -07:00
yliu
ef9d46dcb6 HDFS-7838. Expose truncate API for libhdfs. (yliu) 2015-03-17 07:25:58 +08:00
Colin Patrick Mccabe
861cc05092 HDFS-7722. DataNode#checkDiskError should also remove Storage when error is found. (Lei Xu via Colin P. McCabe) 
(cherry picked from commit b49c3a1813aa8c5b05fe6c02a653286c573137ca)
(cherry picked from commit 7455412a241486a86ad4232f2eaa666c6454329e)
 2015-03-16 18:42:21 -07:00
Colin Patrick Mccabe
647d9ae27a HDFS-7915. The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error (cmccabe) 
(cherry picked from commit bc9cb3e271b22069a15ca110cd60c860250aaab2)
(cherry picked from commit c71b54fa32ef9149692269a95c918506e338e181)
 2015-03-16 18:39:58 -07:00
Konstantin V Shvachko
ad4d704fbb HDFS-7886. Fix TestFileTruncate falures. Contributed by Plamen Jeliazkov and Konstantin Shvachko. 2015-03-16 12:43:46 -07:00
Konstantin V Shvachko
41530b4173 HDFS-2605. Remove redundant "Release 0.21.1" section from CHANGES.txt. Contributed by Allen Wittenauer. 2015-03-13 13:45:47 -07:00
Konstantin V Shvachko
f6bdcd938e HDFS-7903. Cannot recover block after truncate and delete snapshot. Contributed by Plamen Jeliazkov. 2015-03-13 12:45:47 -07:00
Kihwal Lee
116a7f1a16 HDFS-7435. PB encoding of block reports is very inefficient. Contributed by Daryn Sharp. 
(cherry picked from commit d324164a51a43d72c02567248bd9f0f12b244a40)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
(cherry picked from commit 464271a5ede6d05bc7a68ce3f86f84dc72ec1edd)
 2015-03-13 14:42:02 -05:00
Brandon Li
cdeb1079ea HDFS-7926. NameNode implementation of ClientProtocol.truncate(..) is not idempotent. Contributed by Tsz Wo Nicholas Sze 
(cherry picked from commit f446669afb5c3d31a00c65449f27088b39e11ae3)
(cherry picked from commit af80a98ace50934284dde417efc802bb094c8b4e)
 2015-03-13 10:47:25 -07:00
Tsz-Wo Nicholas Sze
a1fffc3fcc HDFS-6833. DirectoryScanner should not register a deleting block with memory of DataNode. Contributed by Shinichi Yamashita 2015-03-13 02:29:24 +08:00
Akira Ajisaka
718cd0f1f4 HDFS-7880. Remove the tests for legacy Web UI in branch-2. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 6c80a3d34a3abb9f5539741a0a62823cf1c23885)
 2015-03-12 10:55:47 +09:00
Colin Patrick Mccabe
8241228db7 HDFS-7830. DataNode does not release the volume lock when adding a volume fails. (Lei Xu via Colin P. McCabe) 
(cherry picked from commit 5c1036d598051cf6af595740f1ab82092b0b6554)
(cherry picked from commit eefca23e8c5e474de1e25bf2ec8a5b266bbe8cfe)
 2015-03-12 10:49:48 +09:00
cnauroth
0fe5f5b5b8 HDFS-7491. Add incremental blockreport latency to DN metrics. Contributed by Ming Ma. 
(cherry picked from commit fb34f45727e63ea55377fe90241328025307d818)
(cherry picked from commit 5e99b7cd2fdf49803703f00463754aa6c56a52a8)
 2015-03-11 14:12:06 -07:00
Jing Zhao
368ab2cd37 HDFS-7898. Change TestAppendSnapshotTruncate to fail-fast. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit e43882e84ae44301eabd0122b5e5492da5fe9f66)
(cherry picked from commit c7105fcff0ac65c5f85d7cc8ca7c24b984217c2c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-03-10 10:31:41 -07:00
Akira Ajisaka
c9a2e6f5db HDFS-6806. HDFS Rolling upgrade document should mention the versions available. Contributed by J.Andreina. 
(cherry picked from commit 82db3341bfb344f10c4c6cc8eea0d8c19e05956a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-03-10 11:27:30 +09:00
Tsuyoshi Ozawa
28f4e6b222 HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 
(cherry picked from commit b46f9e72dbed6fd1f8cae1e12973252462d6ee15)
 2015-03-09 19:54:45 +09:00
Chris Douglas
3bc4f3502a HDFS-7411. Change decommission logic to throttle by blocks rather 
than nodes in each interval. Contributed by Andrew Wang
 2015-03-08 18:32:20 -07:00
Brandon Li
c8af3b2768 HDFS-6488. Support HDFS superuser in NFSv3 gateway. Contributed by Brandon Li 
(cherry picked from commit 0f8ecb1d0ce6d3ee9a7caf5b15b299210c2b8875)
 2015-03-06 15:22:20 -08:00
Haohui Mai
ab251fd355 HDFS-7818. OffsetParam should return the default value instead of throwing NPE when the value is unspecified. Contributed by Eric Payne. 2015-03-06 14:26:47 -08:00
Jing Zhao
994dadb9ba HDFS-7885. Datanode should not trust the generation stamp provided by client. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit 24db0812be64e83a48ade01fc1eaaeaedad4dec0)
 2015-03-06 10:57:38 -08:00
Jing Zhao
c5d52c7b52 HDFS-7855. Separate class Packet from DFSOutputStream. Contributed by Li Bo. 
(cherry picked from commit 952640fa4cbdc23fe8781e5627c2e8eab565c535)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2015-03-05 11:11:08 -08:00
Tsuyoshi Ozawa
f805d48b19 HADOOP-11648. Set DomainSocketWatcher thread name explicitly. Contributed by Liang Xie. 
(cherry picked from commit 74a4754d1c790b8740a4221f276aa571bc5dbfd5)
 2015-03-05 16:06:37 +09:00
Tsz-Wo Nicholas Sze
f9a2007aff HDFS-7746. Add a test randomly mixing append, truncate and snapshot operations. 2015-03-05 10:23:14 +08:00
Dongming Liang
ada545d3b7 HDFS-1522. Combine two BLOCK_FILE_PREFIX constants into one. Contributed by Dongming Liang. 2015-03-04 17:52:25 -08:00